2021-04-03T02:57:12.473 INFO:root:teuthology version: 1.1.0-85d61eae 2021-04-03T02:57:12.476 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --description fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --name teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba --archive /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396 -- /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/orig.config.yaml 2021-04-03T02:57:12.519 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-04-03T02:57:12.574 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396 branch: octopus description: fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: ceph-qa@ceph.io first_in_suite: false job_id: '6016396' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: gibba name: teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 5feb67b5bda898b8c739180363606432d4f39831 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 5feb67b5bda898b8c739180363606432d4f39831 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: 5feb67b5bda898b8c739180363606432d4f39831 owner: scheduled_teuthology@teuthology priority: 1000 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 - - client.2 sha1: 5feb67b5bda898b8c739180363606432d4f39831 sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 5feb67b5bda898b8c739180363606432d4f39831 targets: gibba011.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6Y6vlWddm8prpu69iW8w+0Xj1ViCGO5eSrP31z2EPpVGK5c1Zw+qt/bed4Qa9Oaw+hJXm3zY5Uvx5h0Knf4YUrHcdr+XWK4FQ5Z/RxgFDYeZbPlOVZYXEfe9EPYYFKtwa+sPL2gKmmVu/qQRkcJ0+K/AO1ZCE09Nki29UVFkCay3Zrr0EkmUuwGIgNglWmwheQb3hbq8JQLtlBCSnev9smhbYAfXgjqmZgl4Wpme6MBYvzkrgE7nL+YdDv8HHRfoeuFrrs4DgDTGzfTYIgKvEYiPELEn2PYv15tvsJVjdVnasZ+905qlgObZP62sOve0Kap5eNM9mXN/fbs4ImpYf gibba015.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPASR2jxoPMxMglFCeXssE+skMFNq2HJCFV8IIvSQOF6JnO3vxPyffX6xI02mBjoRdalu0dm7iOwXMXoRHswN4a9hSchKtMiV4stmRtC5TN49uB1xuBId8PruyPJCKaAbpoOA5Xd4Gppvf/xx1L42umabhWuhf1sw+asCFMr9NUdcZCN3Qs5uU5M9h2Fj0FX5DdKxi4rBJBfnXQFIinLzGpZKmqECTKZrh1pPtz2iiDsrV9ZJdq/lgsgCyqEc8Obi1k+kX5XLjenibTqLD7LTMbrJtZ2VxyoNarBZWfNV+mVUztMcnKvNHEN8p/1IrlbNDtmOM3d6b6W229ymFf151 gibba026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC25XRVxi10grxW2V0+jAq9v8qB4FIZ6qm1DkQ3VcBl8MRJni2PZNP4foc/7sv5kTflBFdhTFkzAtIUcJ4F4BdaYaakV5+lKzoR3tq5s9JWlJg3YOEeHl1WOfkWcNmS4UD5W+ziVZwvBtUhWIbJmExPdWiHQthiMazPEZw07LGHisYvZUlhr1Ya69ON7LnL1BP8rAf0/B1voLKVLcBMZyj2B/f6EvQ/Fq4mtF08UsYBUeHKjnnHCfGuknoLviK8V0nOo2J5d8m8voaiJv+u4SBwz4VR8oqoSLIouyZDOo4jlkNIsaDR0oFZDDVIYTFX6o+xuGcX9wTtDSKSINOJa0ad gibba028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYcK4PoB7Wx9U2bM549upyWOVecrKGoclr+3W56PyhnCgosNR+vfeKOZBQq0pDpaLgyGlFRx/4rugnYMz/g4yEW4Usfy+SbN0DmkEOeSVJg9D0W8GjAu1rFtCERKK+UMP7Ti/2ZguzOOQB0Flu4aGMfnwRCFGyRROvWRC+lA4bIMbHdBlMGhmv/beD0uZXFN9IlDMuTvuh5dYNtZ/7oFMTFqZXxJSdUWZZpgZ2D4s7vbJQbTzJrZkF/6C11rrb3MmuZ6sgBdt3+BrtkWUM1wUP3YiZT8W7IgMgEHu2eMaKx7HKEXU+gQZL4VHb6Yyc5qvfxIs7LEfRFvwmznMkQU3/ gibba031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDESPwZOhAoh6gZ2+uiCjzA/B55OF/cnBcW9c99UmEjVQxLlTBe5HqWVMSb1u18m7oRiFg3NJEAH59rWYH+/518gP4aDw7SW73ZAzdOFdvYVJEsr2DDFerb6fsjGKuvdjJsaqN+poqHUWDZA1hCkEWwuQA/hDFt91IzF69wjGCYKUk6uIQDyenjO/6xN3ZUzrsIt6ltyDOZYR4xfr4TFnuIK7GXsx/qjNlDOPo5avsaTmJq9F75w0fh18Hu9AjzazaupJg019RFwdh2m6vtT1R7S9MGuBGgqi21fF1VOWtks2txIn6h8cJSvBMgVI1jbyfhB0O6dwrQqMussNk6clbJ tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: master teuthology_sha1: 85d61eae4759f46ce21e9a37cd816a7a1a66c9d5 timestamp: 2021-04-02_04:10:03 tube: gibba user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.gibba.13276 2021-04-03T02:57:12.575 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa; will attempt to use it 2021-04-03T02:57:12.576 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks 2021-04-03T02:57:12.577 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-04-03T02:57:12.578 INFO:teuthology.task.internal:Checking packages... 2021-04-03T02:57:12.643 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '5feb67b5bda898b8c739180363606432d4f39831' 2021-04-03T02:57:12.644 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-04-03T02:57:12.644 INFO:teuthology.packaging:ref: None 2021-04-03T02:57:12.645 INFO:teuthology.packaging:tag: None 2021-04-03T02:57:12.645 INFO:teuthology.packaging:branch: octopus 2021-04-03T02:57:12.646 INFO:teuthology.packaging:sha1: 5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T02:57:12.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-04-03T02:57:12.852 INFO:teuthology.task.internal:Found packages for ceph version 15.2.10-137-g5feb67b5-1bionic 2021-04-03T02:57:12.852 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-04-03T02:57:12.861 INFO:teuthology.task.internal:no buildpackages task found 2021-04-03T02:57:12.861 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-04-03T02:57:12.900 INFO:teuthology.task.internal:Saving configuration 2021-04-03T02:57:12.915 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-04-03T02:57:12.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-03T02:57:12.954 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-04-03 02:51:56.418435', 'description': '/home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBS/FB5WDEjxuLhQltFpsmkQZu4UPorfdp2YPtKzLnYc', 'os_version': '18.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba011.front.sepia.ceph.com'} 2021-04-03T02:57:12.970 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-04-03 02:51:56.418416', 'description': '/home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJPWyq8fuppohePYfvVpE3E9PdjHbp8DJSXImB0WPFHf', 'os_version': '18.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba015.front.sepia.ceph.com'} 2021-04-03T02:57:12.985 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-04-03 02:51:56.418463', 'description': '/home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB6J2/9EJI7MLdF8L8OzOjt4DBcT01Zdw2ez5YCnK1dG', 'os_version': '18.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba026.front.sepia.ceph.com'} 2021-04-03T02:57:13.001 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-04-03 02:51:56.418476', 'description': '/home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGO1m9jgrWkldJXkk6SN/Ne2KPm2KinkeVhj18Wpcywb', 'os_version': '18.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba028.front.sepia.ceph.com'} 2021-04-03T02:57:13.016 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-04-03 02:51:56.418449', 'description': '/home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINrlpBBc4kD9U+/uLQ23mOVEwgZtMCSulabq5t8mSssH', 'os_version': '18.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba031.front.sepia.ceph.com'} 2021-04-03T02:57:13.017 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-04-03T02:57:13.024 INFO:teuthology.task.internal:roles: ubuntu@gibba011.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2021-04-03T02:57:13.025 INFO:teuthology.task.internal:roles: ubuntu@gibba015.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-04-03T02:57:13.025 INFO:teuthology.task.internal:roles: ubuntu@gibba026.front.sepia.ceph.com - ['client.0'] 2021-04-03T02:57:13.025 INFO:teuthology.task.internal:roles: ubuntu@gibba028.front.sepia.ceph.com - ['client.1'] 2021-04-03T02:57:13.025 INFO:teuthology.task.internal:roles: ubuntu@gibba031.front.sepia.ceph.com - ['client.2'] 2021-04-03T02:57:13.026 INFO:teuthology.run_tasks:Running task console_log... 2021-04-03T02:57:13.194 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f505abd8a60>, signals=[15]) 2021-04-03T02:57:13.195 INFO:teuthology.run_tasks:Running task internal.connect... 2021-04-03T02:57:13.204 INFO:teuthology.task.internal:Opening connections... 2021-04-03T02:57:13.204 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T02:57:13.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba011.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T02:57:13.268 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T02:57:13.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba015.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T02:57:13.333 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba026.front.sepia.ceph.com 2021-04-03T02:57:13.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T02:57:13.395 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba028.front.sepia.ceph.com 2021-04-03T02:57:13.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T02:57:13.457 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba031.front.sepia.ceph.com 2021-04-03T02:57:13.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T02:57:13.562 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-04-03T02:57:13.571 DEBUG:teuthology.orchestra.run.gibba011:> uname -m 2021-04-03T02:57:13.574 INFO:teuthology.orchestra.run.gibba011.stdout:x86_64 2021-04-03T02:57:13.575 DEBUG:teuthology.orchestra.run.gibba011:> cat /etc/os-release 2021-04-03T02:57:13.618 INFO:teuthology.orchestra.run.gibba011.stdout:NAME="Ubuntu" 2021-04-03T02:57:13.619 INFO:teuthology.orchestra.run.gibba011.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-04-03T02:57:13.619 INFO:teuthology.orchestra.run.gibba011.stdout:ID=ubuntu 2021-04-03T02:57:13.619 INFO:teuthology.orchestra.run.gibba011.stdout:ID_LIKE=debian 2021-04-03T02:57:13.619 INFO:teuthology.orchestra.run.gibba011.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-04-03T02:57:13.620 INFO:teuthology.orchestra.run.gibba011.stdout:VERSION_ID="18.04" 2021-04-03T02:57:13.620 INFO:teuthology.orchestra.run.gibba011.stdout:HOME_URL="https://www.ubuntu.com/" 2021-04-03T02:57:13.620 INFO:teuthology.orchestra.run.gibba011.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-04-03T02:57:13.620 INFO:teuthology.orchestra.run.gibba011.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-04-03T02:57:13.621 INFO:teuthology.orchestra.run.gibba011.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-04-03T02:57:13.621 INFO:teuthology.orchestra.run.gibba011.stdout:VERSION_CODENAME=bionic 2021-04-03T02:57:13.621 INFO:teuthology.orchestra.run.gibba011.stdout:UBUNTU_CODENAME=bionic 2021-04-03T02:57:13.622 INFO:teuthology.lock.ops:Updating gibba011.front.sepia.ceph.com on lock server 2021-04-03T02:57:13.638 DEBUG:teuthology.orchestra.run.gibba015:> uname -m 2021-04-03T02:57:13.641 INFO:teuthology.orchestra.run.gibba015.stdout:x86_64 2021-04-03T02:57:13.642 DEBUG:teuthology.orchestra.run.gibba015:> cat /etc/os-release 2021-04-03T02:57:13.685 INFO:teuthology.orchestra.run.gibba015.stdout:NAME="Ubuntu" 2021-04-03T02:57:13.686 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-04-03T02:57:13.686 INFO:teuthology.orchestra.run.gibba015.stdout:ID=ubuntu 2021-04-03T02:57:13.686 INFO:teuthology.orchestra.run.gibba015.stdout:ID_LIKE=debian 2021-04-03T02:57:13.686 INFO:teuthology.orchestra.run.gibba015.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-04-03T02:57:13.686 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION_ID="18.04" 2021-04-03T02:57:13.687 INFO:teuthology.orchestra.run.gibba015.stdout:HOME_URL="https://www.ubuntu.com/" 2021-04-03T02:57:13.687 INFO:teuthology.orchestra.run.gibba015.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-04-03T02:57:13.687 INFO:teuthology.orchestra.run.gibba015.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-04-03T02:57:13.687 INFO:teuthology.orchestra.run.gibba015.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-04-03T02:57:13.687 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION_CODENAME=bionic 2021-04-03T02:57:13.688 INFO:teuthology.orchestra.run.gibba015.stdout:UBUNTU_CODENAME=bionic 2021-04-03T02:57:13.688 INFO:teuthology.lock.ops:Updating gibba015.front.sepia.ceph.com on lock server 2021-04-03T02:57:13.702 DEBUG:teuthology.orchestra.run.gibba026:> uname -m 2021-04-03T02:57:13.705 INFO:teuthology.orchestra.run.gibba026.stdout:x86_64 2021-04-03T02:57:13.705 DEBUG:teuthology.orchestra.run.gibba026:> cat /etc/os-release 2021-04-03T02:57:13.751 INFO:teuthology.orchestra.run.gibba026.stdout:NAME="Ubuntu" 2021-04-03T02:57:13.751 INFO:teuthology.orchestra.run.gibba026.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-04-03T02:57:13.751 INFO:teuthology.orchestra.run.gibba026.stdout:ID=ubuntu 2021-04-03T02:57:13.752 INFO:teuthology.orchestra.run.gibba026.stdout:ID_LIKE=debian 2021-04-03T02:57:13.752 INFO:teuthology.orchestra.run.gibba026.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-04-03T02:57:13.752 INFO:teuthology.orchestra.run.gibba026.stdout:VERSION_ID="18.04" 2021-04-03T02:57:13.752 INFO:teuthology.orchestra.run.gibba026.stdout:HOME_URL="https://www.ubuntu.com/" 2021-04-03T02:57:13.752 INFO:teuthology.orchestra.run.gibba026.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-04-03T02:57:13.753 INFO:teuthology.orchestra.run.gibba026.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-04-03T02:57:13.753 INFO:teuthology.orchestra.run.gibba026.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-04-03T02:57:13.753 INFO:teuthology.orchestra.run.gibba026.stdout:VERSION_CODENAME=bionic 2021-04-03T02:57:13.753 INFO:teuthology.orchestra.run.gibba026.stdout:UBUNTU_CODENAME=bionic 2021-04-03T02:57:13.754 INFO:teuthology.lock.ops:Updating gibba026.front.sepia.ceph.com on lock server 2021-04-03T02:57:13.766 DEBUG:teuthology.orchestra.run.gibba028:> uname -m 2021-04-03T02:57:13.775 INFO:teuthology.orchestra.run.gibba028.stdout:x86_64 2021-04-03T02:57:13.775 DEBUG:teuthology.orchestra.run.gibba028:> cat /etc/os-release 2021-04-03T02:57:13.822 INFO:teuthology.orchestra.run.gibba028.stdout:NAME="Ubuntu" 2021-04-03T02:57:13.822 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-04-03T02:57:13.823 INFO:teuthology.orchestra.run.gibba028.stdout:ID=ubuntu 2021-04-03T02:57:13.823 INFO:teuthology.orchestra.run.gibba028.stdout:ID_LIKE=debian 2021-04-03T02:57:13.823 INFO:teuthology.orchestra.run.gibba028.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-04-03T02:57:13.823 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION_ID="18.04" 2021-04-03T02:57:13.823 INFO:teuthology.orchestra.run.gibba028.stdout:HOME_URL="https://www.ubuntu.com/" 2021-04-03T02:57:13.824 INFO:teuthology.orchestra.run.gibba028.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-04-03T02:57:13.824 INFO:teuthology.orchestra.run.gibba028.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-04-03T02:57:13.824 INFO:teuthology.orchestra.run.gibba028.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-04-03T02:57:13.824 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION_CODENAME=bionic 2021-04-03T02:57:13.824 INFO:teuthology.orchestra.run.gibba028.stdout:UBUNTU_CODENAME=bionic 2021-04-03T02:57:13.825 INFO:teuthology.lock.ops:Updating gibba028.front.sepia.ceph.com on lock server 2021-04-03T02:57:13.837 DEBUG:teuthology.orchestra.run.gibba031:> uname -m 2021-04-03T02:57:13.845 INFO:teuthology.orchestra.run.gibba031.stdout:x86_64 2021-04-03T02:57:13.846 DEBUG:teuthology.orchestra.run.gibba031:> cat /etc/os-release 2021-04-03T02:57:13.894 INFO:teuthology.orchestra.run.gibba031.stdout:NAME="Ubuntu" 2021-04-03T02:57:13.894 INFO:teuthology.orchestra.run.gibba031.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-04-03T02:57:13.895 INFO:teuthology.orchestra.run.gibba031.stdout:ID=ubuntu 2021-04-03T02:57:13.895 INFO:teuthology.orchestra.run.gibba031.stdout:ID_LIKE=debian 2021-04-03T02:57:13.895 INFO:teuthology.orchestra.run.gibba031.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-04-03T02:57:13.895 INFO:teuthology.orchestra.run.gibba031.stdout:VERSION_ID="18.04" 2021-04-03T02:57:13.895 INFO:teuthology.orchestra.run.gibba031.stdout:HOME_URL="https://www.ubuntu.com/" 2021-04-03T02:57:13.896 INFO:teuthology.orchestra.run.gibba031.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-04-03T02:57:13.896 INFO:teuthology.orchestra.run.gibba031.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-04-03T02:57:13.896 INFO:teuthology.orchestra.run.gibba031.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-04-03T02:57:13.896 INFO:teuthology.orchestra.run.gibba031.stdout:VERSION_CODENAME=bionic 2021-04-03T02:57:13.896 INFO:teuthology.orchestra.run.gibba031.stdout:UBUNTU_CODENAME=bionic 2021-04-03T02:57:13.897 INFO:teuthology.lock.ops:Updating gibba031.front.sepia.ceph.com on lock server 2021-04-03T02:57:13.910 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-04-03T02:57:13.923 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-04-03T02:57:13.990 INFO:teuthology.task.internal:Checking for old test directory... 2021-04-03T02:57:13.991 DEBUG:teuthology.orchestra.run.gibba011:> test '!' -e /home/ubuntu/cephtest 2021-04-03T02:57:13.993 DEBUG:teuthology.orchestra.run.gibba015:> test '!' -e /home/ubuntu/cephtest 2021-04-03T02:57:13.995 DEBUG:teuthology.orchestra.run.gibba026:> test '!' -e /home/ubuntu/cephtest 2021-04-03T02:57:13.997 DEBUG:teuthology.orchestra.run.gibba028:> test '!' -e /home/ubuntu/cephtest 2021-04-03T02:57:14.001 DEBUG:teuthology.orchestra.run.gibba031:> test '!' -e /home/ubuntu/cephtest 2021-04-03T02:57:14.008 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-04-03T02:57:14.035 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-04-03T02:57:14.035 DEBUG:teuthology.orchestra.run.gibba011:> test -z $(ls -A /var/lib/ceph) 2021-04-03T02:57:14.042 DEBUG:teuthology.orchestra.run.gibba015:> test -z $(ls -A /var/lib/ceph) 2021-04-03T02:57:14.044 DEBUG:teuthology.orchestra.run.gibba026:> test -z $(ls -A /var/lib/ceph) 2021-04-03T02:57:14.046 DEBUG:teuthology.orchestra.run.gibba028:> test -z $(ls -A /var/lib/ceph) 2021-04-03T02:57:14.049 DEBUG:teuthology.orchestra.run.gibba031:> test -z $(ls -A /var/lib/ceph) 2021-04-03T02:57:14.067 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-04-03T02:57:14.157 INFO:teuthology.run_tasks:Running task kernel... 2021-04-03T02:57:14.172 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}} 2021-04-03T02:57:14.172 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-04-03T02:57:14.173 DEBUG:teuthology.orchestra.run.gibba011:> uname -r 2021-04-03T02:57:14.180 INFO:teuthology.orchestra.run.gibba011.stdout:4.15.0-122-generic 2021-04-03T02:57:14.180 INFO:teuthology.task.kernel:Running kernel on gibba011: 4.15.0-122-generic 2021-04-03T02:57:14.180 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-get clean 2021-04-03T02:57:14.251 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-get update 2021-04-03T02:57:14.575 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T02:57:14.576 INFO:teuthology.orchestra.run.gibba011.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T02:57:14.634 INFO:teuthology.orchestra.run.gibba011.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T02:57:14.890 INFO:teuthology.orchestra.run.gibba011.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T02:57:14.909 INFO:teuthology.orchestra.run.gibba011.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,639 kB] 2021-04-03T02:57:14.993 INFO:teuthology.orchestra.run.gibba011.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,978 kB] 2021-04-03T02:57:15.322 INFO:teuthology.orchestra.run.gibba011.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,253 kB] 2021-04-03T02:57:15.347 INFO:teuthology.orchestra.run.gibba011.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [950 kB] 2021-04-03T02:57:15.365 INFO:teuthology.orchestra.run.gibba011.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [310 kB] 2021-04-03T02:57:15.367 INFO:teuthology.orchestra.run.gibba011.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [402 kB] 2021-04-03T02:57:15.371 INFO:teuthology.orchestra.run.gibba011.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-04-03T02:57:15.371 INFO:teuthology.orchestra.run.gibba011.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [265 kB] 2021-04-03T02:57:15.375 INFO:teuthology.orchestra.run.gibba011.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [286 kB] 2021-04-03T02:57:15.376 INFO:teuthology.orchestra.run.gibba011.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [34.6 kB] 2021-04-03T02:57:15.377 INFO:teuthology.orchestra.run.gibba011.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,118 kB] 2021-04-03T02:57:15.397 INFO:teuthology.orchestra.run.gibba011.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-04-03T02:57:15.399 INFO:teuthology.orchestra.run.gibba011.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [38.2 kB] 2021-04-03T02:57:15.399 INFO:teuthology.orchestra.run.gibba011.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,724 kB] 2021-04-03T02:57:15.442 INFO:teuthology.orchestra.run.gibba011.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,562 kB] 2021-04-03T02:57:15.444 INFO:teuthology.orchestra.run.gibba011.stdout:Get:20 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [980 kB] 2021-04-03T02:57:15.468 INFO:teuthology.orchestra.run.gibba011.stdout:Get:21 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [251 kB] 2021-04-03T02:57:15.474 INFO:teuthology.orchestra.run.gibba011.stdout:Get:22 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-04-03T02:57:15.475 INFO:teuthology.orchestra.run.gibba011.stdout:Get:23 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-04-03T02:57:15.475 INFO:teuthology.orchestra.run.gibba011.stdout:Get:24 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-04-03T02:57:15.491 INFO:teuthology.orchestra.run.gibba011.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [365 kB] 2021-04-03T02:57:15.495 INFO:teuthology.orchestra.run.gibba011.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-04-03T02:57:15.496 INFO:teuthology.orchestra.run.gibba011.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-04-03T02:57:15.496 INFO:teuthology.orchestra.run.gibba011.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-04-03T02:57:16.675 INFO:teuthology.orchestra.run.gibba011.stdout:Fetched 13.5 MB in 2s (6,222 kB/s) 2021-04-03T02:57:17.751 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T02:57:17.777 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T02:57:17.822 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T02:57:17.993 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-04-03T02:57:17.993 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-04-03T02:57:18.047 INFO:teuthology.orchestra.run.gibba011.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T02:57:18.048 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T02:57:18.049 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T02:57:18.140 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T02:57:18.338 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-04-03T02:57:18.338 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-04-03T02:57:18.468 INFO:teuthology.orchestra.run.gibba011.stdout:The following additional packages will be installed: 2021-04-03T02:57:18.469 INFO:teuthology.orchestra.run.gibba011.stdout: linux-generic linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:57:18.469 INFO:teuthology.orchestra.run.gibba011.stdout: linux-headers-generic linux-image-4.15.0-140-generic 2021-04-03T02:57:18.469 INFO:teuthology.orchestra.run.gibba011.stdout: linux-modules-4.15.0-140-generic linux-modules-extra-4.15.0-140-generic 2021-04-03T02:57:18.470 INFO:teuthology.orchestra.run.gibba011.stdout:Suggested packages: 2021-04-03T02:57:18.470 INFO:teuthology.orchestra.run.gibba011.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-04-03T02:57:18.470 INFO:teuthology.orchestra.run.gibba011.stdout:Recommended packages: 2021-04-03T02:57:18.471 INFO:teuthology.orchestra.run.gibba011.stdout: thermald 2021-04-03T02:57:18.488 INFO:teuthology.orchestra.run.gibba011.stdout:The following NEW packages will be installed: 2021-04-03T02:57:18.488 INFO:teuthology.orchestra.run.gibba011.stdout: linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:57:18.488 INFO:teuthology.orchestra.run.gibba011.stdout: linux-image-4.15.0-140-generic linux-modules-4.15.0-140-generic 2021-04-03T02:57:18.488 INFO:teuthology.orchestra.run.gibba011.stdout: linux-modules-extra-4.15.0-140-generic 2021-04-03T02:57:18.489 INFO:teuthology.orchestra.run.gibba011.stdout:The following packages will be upgraded: 2021-04-03T02:57:18.489 INFO:teuthology.orchestra.run.gibba011.stdout: linux-generic linux-headers-generic linux-image-generic 2021-04-03T02:57:18.658 INFO:teuthology.orchestra.run.gibba011.stdout:3 upgraded, 5 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T02:57:18.658 INFO:teuthology.orchestra.run.gibba011.stdout:Need to get 66.0 MB of archives. 2021-04-03T02:57:18.658 INFO:teuthology.orchestra.run.gibba011.stdout:After this operation, 334 MB of additional disk space will be used. 2021-04-03T02:57:18.658 INFO:teuthology.orchestra.run.gibba011.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-140-generic amd64 4.15.0-140.144 [13.1 MB] 2021-04-03T02:57:19.659 INFO:teuthology.orchestra.run.gibba011.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-140-generic amd64 4.15.0-140.144 [8,021 kB] 2021-04-03T02:57:19.840 INFO:teuthology.orchestra.run.gibba011.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-140-generic amd64 4.15.0-140.144 [32.8 MB] 2021-04-03T02:57:20.740 INFO:teuthology.orchestra.run.gibba011.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.140.127 [1,868 B] 2021-04-03T02:57:20.740 INFO:teuthology.orchestra.run.gibba011.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.140.127 [2,580 B] 2021-04-03T02:57:20.740 INFO:teuthology.orchestra.run.gibba011.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140 all 4.15.0-140.144 [10.9 MB] 2021-04-03T02:57:21.056 INFO:teuthology.orchestra.run.gibba011.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140-generic amd64 4.15.0-140.144 [1,101 kB] 2021-04-03T02:57:21.065 INFO:teuthology.orchestra.run.gibba011.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.140.127 [2,484 B] 2021-04-03T02:57:21.371 INFO:teuthology.orchestra.run.gibba011.stdout:Fetched 66.0 MB in 3s (25.7 MB/s) 2021-04-03T02:57:21.504 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package linux-modules-4.15.0-140-generic. 2021-04-03T02:57:21.553 INFO:teuthology.orchestra.run.gibba011.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102071 files and directories currently installed.) 2021-04-03T02:57:21.555 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../0-linux-modules-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:21.600 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:22.858 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package linux-image-4.15.0-140-generic. 2021-04-03T02:57:22.861 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../1-linux-image-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:22.932 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:23.353 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-140-generic. 2021-04-03T02:57:23.356 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:23.395 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:25.811 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../3-linux-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:57:25.929 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:57:26.235 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:57:26.357 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-image-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:57:26.660 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package linux-headers-4.15.0-140. 2021-04-03T02:57:26.663 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../5-linux-headers-4.15.0-140_4.15.0-140.144_all.deb ... 2021-04-03T02:57:26.701 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:57:29.080 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package linux-headers-4.15.0-140-generic. 2021-04-03T02:57:29.088 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../6-linux-headers-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:29.122 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:30.473 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:57:30.596 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking linux-headers-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:57:31.399 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:57:31.626 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:32.060 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:32.377 INFO:teuthology.orchestra.run.gibba011.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:57:32.377 INFO:teuthology.orchestra.run.gibba011.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-140-generic 2021-04-03T02:57:32.503 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:32.846 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-image-generic (4.15.0.140.127) ... 2021-04-03T02:57:32.948 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:33.057 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-headers-generic (4.15.0.140.127) ... 2021-04-03T02:57:33.158 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up linux-generic (4.15.0.140.127) ... 2021-04-03T02:57:33.268 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:33.403 INFO:teuthology.orchestra.run.gibba011.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-04-03T02:57:33.403 INFO:teuthology.orchestra.run.gibba011.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:57:41.960 INFO:teuthology.orchestra.run.gibba011.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-04-03T02:57:41.961 INFO:teuthology.orchestra.run.gibba011.stdout:Sourcing file `/etc/default/grub' 2021-04-03T02:57:41.996 INFO:teuthology.orchestra.run.gibba011.stdout:Generating grub configuration file ... 2021-04-03T02:57:42.359 INFO:teuthology.orchestra.run.gibba011.stdout:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:57:42.395 INFO:teuthology.orchestra.run.gibba011.stdout:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:57:42.768 INFO:teuthology.orchestra.run.gibba011.stdout:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T02:57:42.803 INFO:teuthology.orchestra.run.gibba011.stdout:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T02:57:43.989 INFO:teuthology.orchestra.run.gibba011.stdout:done 2021-04-03T02:57:44.264 DEBUG:teuthology.orchestra.run.gibba011:> dpkg -s linux-image-generic 2021-04-03T02:57:44.275 INFO:teuthology.orchestra.run.gibba011.stdout:Package: linux-image-generic 2021-04-03T02:57:44.276 INFO:teuthology.orchestra.run.gibba011.stdout:Status: install ok installed 2021-04-03T02:57:44.276 INFO:teuthology.orchestra.run.gibba011.stdout:Priority: optional 2021-04-03T02:57:44.276 INFO:teuthology.orchestra.run.gibba011.stdout:Section: kernel 2021-04-03T02:57:44.276 INFO:teuthology.orchestra.run.gibba011.stdout:Installed-Size: 17 2021-04-03T02:57:44.277 INFO:teuthology.orchestra.run.gibba011.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T02:57:44.277 INFO:teuthology.orchestra.run.gibba011.stdout:Architecture: amd64 2021-04-03T02:57:44.277 INFO:teuthology.orchestra.run.gibba011.stdout:Source: linux-meta 2021-04-03T02:57:44.277 INFO:teuthology.orchestra.run.gibba011.stdout:Version: 4.15.0.140.127 2021-04-03T02:57:44.277 INFO:teuthology.orchestra.run.gibba011.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T02:57:44.278 INFO:teuthology.orchestra.run.gibba011.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T02:57:44.278 INFO:teuthology.orchestra.run.gibba011.stdout:Recommends: thermald 2021-04-03T02:57:44.278 INFO:teuthology.orchestra.run.gibba011.stdout:Description: Generic Linux kernel image 2021-04-03T02:57:44.278 INFO:teuthology.orchestra.run.gibba011.stdout: This package will always depend on the latest generic kernel image 2021-04-03T02:57:44.278 INFO:teuthology.orchestra.run.gibba011.stdout: available. 2021-04-03T02:57:44.279 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-122-generic Expected: 4.15.0-140-generic 2021-04-03T02:57:44.279 DEBUG:teuthology.orchestra.run.gibba015:> uname -r 2021-04-03T02:57:44.287 INFO:teuthology.orchestra.run.gibba015.stdout:4.15.0-122-generic 2021-04-03T02:57:44.288 INFO:teuthology.task.kernel:Running kernel on gibba015: 4.15.0-122-generic 2021-04-03T02:57:44.288 DEBUG:teuthology.orchestra.run.gibba015:> sudo apt-get clean 2021-04-03T02:57:44.363 DEBUG:teuthology.orchestra.run.gibba015:> sudo apt-get update 2021-04-03T02:57:44.619 INFO:teuthology.orchestra.run.gibba015.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T02:57:44.619 INFO:teuthology.orchestra.run.gibba015.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T02:57:44.692 INFO:teuthology.orchestra.run.gibba015.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T02:57:44.919 INFO:teuthology.orchestra.run.gibba015.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,639 kB] 2021-04-03T02:57:44.959 INFO:teuthology.orchestra.run.gibba015.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T02:57:45.041 INFO:teuthology.orchestra.run.gibba015.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,978 kB] 2021-04-03T02:57:45.342 INFO:teuthology.orchestra.run.gibba015.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,253 kB] 2021-04-03T02:57:45.393 INFO:teuthology.orchestra.run.gibba015.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [402 kB] 2021-04-03T02:57:45.393 INFO:teuthology.orchestra.run.gibba015.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [950 kB] 2021-04-03T02:57:45.407 INFO:teuthology.orchestra.run.gibba015.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-04-03T02:57:45.408 INFO:teuthology.orchestra.run.gibba015.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [286 kB] 2021-04-03T02:57:45.413 INFO:teuthology.orchestra.run.gibba015.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [310 kB] 2021-04-03T02:57:45.416 INFO:teuthology.orchestra.run.gibba015.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [38.2 kB] 2021-04-03T02:57:45.418 INFO:teuthology.orchestra.run.gibba015.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,724 kB] 2021-04-03T02:57:45.419 INFO:teuthology.orchestra.run.gibba015.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-04-03T02:57:45.419 INFO:teuthology.orchestra.run.gibba015.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [265 kB] 2021-04-03T02:57:45.423 INFO:teuthology.orchestra.run.gibba015.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [34.6 kB] 2021-04-03T02:57:45.424 INFO:teuthology.orchestra.run.gibba015.stdout:Get:18 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [980 kB] 2021-04-03T02:57:45.458 INFO:teuthology.orchestra.run.gibba015.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,562 kB] 2021-04-03T02:57:45.489 INFO:teuthology.orchestra.run.gibba015.stdout:Get:20 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,118 kB] 2021-04-03T02:57:45.511 INFO:teuthology.orchestra.run.gibba015.stdout:Get:21 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [251 kB] 2021-04-03T02:57:45.513 INFO:teuthology.orchestra.run.gibba015.stdout:Get:22 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-04-03T02:57:45.513 INFO:teuthology.orchestra.run.gibba015.stdout:Get:23 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-04-03T02:57:45.513 INFO:teuthology.orchestra.run.gibba015.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [365 kB] 2021-04-03T02:57:45.514 INFO:teuthology.orchestra.run.gibba015.stdout:Get:25 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-04-03T02:57:45.516 INFO:teuthology.orchestra.run.gibba015.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-04-03T02:57:45.517 INFO:teuthology.orchestra.run.gibba015.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-04-03T02:57:45.518 INFO:teuthology.orchestra.run.gibba015.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-04-03T02:57:46.719 INFO:teuthology.orchestra.run.gibba015.stdout:Fetched 13.5 MB in 2s (6,249 kB/s) 2021-04-03T02:57:47.762 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T02:57:47.788 DEBUG:teuthology.orchestra.run.gibba015:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T02:57:47.833 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T02:57:48.006 INFO:teuthology.orchestra.run.gibba015.stdout:Building dependency tree... 2021-04-03T02:57:48.006 INFO:teuthology.orchestra.run.gibba015.stdout:Reading state information... 2021-04-03T02:57:48.058 INFO:teuthology.orchestra.run.gibba015.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T02:57:48.059 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T02:57:48.060 DEBUG:teuthology.orchestra.run.gibba015:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T02:57:48.157 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T02:57:48.332 INFO:teuthology.orchestra.run.gibba015.stdout:Building dependency tree... 2021-04-03T02:57:48.333 INFO:teuthology.orchestra.run.gibba015.stdout:Reading state information... 2021-04-03T02:57:48.435 INFO:teuthology.orchestra.run.gibba015.stdout:The following additional packages will be installed: 2021-04-03T02:57:48.436 INFO:teuthology.orchestra.run.gibba015.stdout: linux-generic linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:57:48.436 INFO:teuthology.orchestra.run.gibba015.stdout: linux-headers-generic linux-image-4.15.0-140-generic 2021-04-03T02:57:48.436 INFO:teuthology.orchestra.run.gibba015.stdout: linux-modules-4.15.0-140-generic linux-modules-extra-4.15.0-140-generic 2021-04-03T02:57:48.437 INFO:teuthology.orchestra.run.gibba015.stdout:Suggested packages: 2021-04-03T02:57:48.437 INFO:teuthology.orchestra.run.gibba015.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-04-03T02:57:48.437 INFO:teuthology.orchestra.run.gibba015.stdout:Recommended packages: 2021-04-03T02:57:48.437 INFO:teuthology.orchestra.run.gibba015.stdout: thermald 2021-04-03T02:57:48.451 INFO:teuthology.orchestra.run.gibba015.stdout:The following NEW packages will be installed: 2021-04-03T02:57:48.451 INFO:teuthology.orchestra.run.gibba015.stdout: linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:57:48.452 INFO:teuthology.orchestra.run.gibba015.stdout: linux-image-4.15.0-140-generic linux-modules-4.15.0-140-generic 2021-04-03T02:57:48.452 INFO:teuthology.orchestra.run.gibba015.stdout: linux-modules-extra-4.15.0-140-generic 2021-04-03T02:57:48.452 INFO:teuthology.orchestra.run.gibba015.stdout:The following packages will be upgraded: 2021-04-03T02:57:48.453 INFO:teuthology.orchestra.run.gibba015.stdout: linux-generic linux-headers-generic linux-image-generic 2021-04-03T02:57:48.620 INFO:teuthology.orchestra.run.gibba015.stdout:3 upgraded, 5 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T02:57:48.620 INFO:teuthology.orchestra.run.gibba015.stdout:Need to get 66.0 MB of archives. 2021-04-03T02:57:48.620 INFO:teuthology.orchestra.run.gibba015.stdout:After this operation, 334 MB of additional disk space will be used. 2021-04-03T02:57:48.621 INFO:teuthology.orchestra.run.gibba015.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-140-generic amd64 4.15.0-140.144 [13.1 MB] 2021-04-03T02:57:49.659 INFO:teuthology.orchestra.run.gibba015.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-140-generic amd64 4.15.0-140.144 [8,021 kB] 2021-04-03T02:57:49.940 INFO:teuthology.orchestra.run.gibba015.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-140-generic amd64 4.15.0-140.144 [32.8 MB] 2021-04-03T02:57:50.944 INFO:teuthology.orchestra.run.gibba015.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.140.127 [1,868 B] 2021-04-03T02:57:50.944 INFO:teuthology.orchestra.run.gibba015.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.140.127 [2,580 B] 2021-04-03T02:57:50.944 INFO:teuthology.orchestra.run.gibba015.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140 all 4.15.0-140.144 [10.9 MB] 2021-04-03T02:57:51.249 INFO:teuthology.orchestra.run.gibba015.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140-generic amd64 4.15.0-140.144 [1,101 kB] 2021-04-03T02:57:51.270 INFO:teuthology.orchestra.run.gibba015.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.140.127 [2,484 B] 2021-04-03T02:57:51.602 INFO:teuthology.orchestra.run.gibba015.stdout:Fetched 66.0 MB in 3s (23.5 MB/s) 2021-04-03T02:57:51.738 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package linux-modules-4.15.0-140-generic. 2021-04-03T02:57:51.754 INFO:teuthology.orchestra.run.gibba015.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102071 files and directories currently installed.) 2021-04-03T02:57:51.756 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../0-linux-modules-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:51.802 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:52.994 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package linux-image-4.15.0-140-generic. 2021-04-03T02:57:52.997 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../1-linux-image-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:53.083 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:53.488 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-140-generic. 2021-04-03T02:57:53.491 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:53.530 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:57:55.965 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../3-linux-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:57:56.094 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:57:56.368 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:57:56.473 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-image-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:57:56.767 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package linux-headers-4.15.0-140. 2021-04-03T02:57:56.770 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../5-linux-headers-4.15.0-140_4.15.0-140.144_all.deb ... 2021-04-03T02:57:56.809 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:57:59.213 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package linux-headers-4.15.0-140-generic. 2021-04-03T02:57:59.220 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../6-linux-headers-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:57:59.263 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:00.545 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:58:00.655 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking linux-headers-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:58:00.975 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:58:01.101 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:01.535 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:01.837 INFO:teuthology.orchestra.run.gibba015.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:58:01.837 INFO:teuthology.orchestra.run.gibba015.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-140-generic 2021-04-03T02:58:01.962 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:02.363 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-image-generic (4.15.0.140.127) ... 2021-04-03T02:58:02.481 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:02.582 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-headers-generic (4.15.0.140.127) ... 2021-04-03T02:58:02.682 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up linux-generic (4.15.0.140.127) ... 2021-04-03T02:58:02.784 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:02.940 INFO:teuthology.orchestra.run.gibba015.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-04-03T02:58:02.940 INFO:teuthology.orchestra.run.gibba015.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:58:12.097 INFO:teuthology.orchestra.run.gibba015.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-04-03T02:58:12.097 INFO:teuthology.orchestra.run.gibba015.stdout:Sourcing file `/etc/default/grub' 2021-04-03T02:58:12.131 INFO:teuthology.orchestra.run.gibba015.stdout:Generating grub configuration file ... 2021-04-03T02:58:12.500 INFO:teuthology.orchestra.run.gibba015.stdout:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:58:12.533 INFO:teuthology.orchestra.run.gibba015.stdout:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:58:12.905 INFO:teuthology.orchestra.run.gibba015.stdout:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T02:58:12.967 INFO:teuthology.orchestra.run.gibba015.stdout:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T02:58:14.187 INFO:teuthology.orchestra.run.gibba015.stdout:done 2021-04-03T02:58:14.512 DEBUG:teuthology.orchestra.run.gibba015:> dpkg -s linux-image-generic 2021-04-03T02:58:14.524 INFO:teuthology.orchestra.run.gibba015.stdout:Package: linux-image-generic 2021-04-03T02:58:14.525 INFO:teuthology.orchestra.run.gibba015.stdout:Status: install ok installed 2021-04-03T02:58:14.525 INFO:teuthology.orchestra.run.gibba015.stdout:Priority: optional 2021-04-03T02:58:14.525 INFO:teuthology.orchestra.run.gibba015.stdout:Section: kernel 2021-04-03T02:58:14.525 INFO:teuthology.orchestra.run.gibba015.stdout:Installed-Size: 17 2021-04-03T02:58:14.526 INFO:teuthology.orchestra.run.gibba015.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T02:58:14.526 INFO:teuthology.orchestra.run.gibba015.stdout:Architecture: amd64 2021-04-03T02:58:14.526 INFO:teuthology.orchestra.run.gibba015.stdout:Source: linux-meta 2021-04-03T02:58:14.526 INFO:teuthology.orchestra.run.gibba015.stdout:Version: 4.15.0.140.127 2021-04-03T02:58:14.526 INFO:teuthology.orchestra.run.gibba015.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T02:58:14.527 INFO:teuthology.orchestra.run.gibba015.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T02:58:14.527 INFO:teuthology.orchestra.run.gibba015.stdout:Recommends: thermald 2021-04-03T02:58:14.527 INFO:teuthology.orchestra.run.gibba015.stdout:Description: Generic Linux kernel image 2021-04-03T02:58:14.527 INFO:teuthology.orchestra.run.gibba015.stdout: This package will always depend on the latest generic kernel image 2021-04-03T02:58:14.527 INFO:teuthology.orchestra.run.gibba015.stdout: available. 2021-04-03T02:58:14.528 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-122-generic Expected: 4.15.0-140-generic 2021-04-03T02:58:14.528 DEBUG:teuthology.orchestra.run.gibba026:> uname -r 2021-04-03T02:58:14.536 INFO:teuthology.orchestra.run.gibba026.stdout:4.15.0-122-generic 2021-04-03T02:58:14.537 INFO:teuthology.task.kernel:Running kernel on gibba026: 4.15.0-122-generic 2021-04-03T02:58:14.537 DEBUG:teuthology.orchestra.run.gibba026:> sudo apt-get clean 2021-04-03T02:58:14.613 DEBUG:teuthology.orchestra.run.gibba026:> sudo apt-get update 2021-04-03T02:58:14.877 INFO:teuthology.orchestra.run.gibba026.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T02:58:14.888 INFO:teuthology.orchestra.run.gibba026.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T02:58:14.958 INFO:teuthology.orchestra.run.gibba026.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T02:58:15.191 INFO:teuthology.orchestra.run.gibba026.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,639 kB] 2021-04-03T02:58:15.264 INFO:teuthology.orchestra.run.gibba026.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T02:58:15.361 INFO:teuthology.orchestra.run.gibba026.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,253 kB] 2021-04-03T02:58:15.630 INFO:teuthology.orchestra.run.gibba026.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [950 kB] 2021-04-03T02:58:15.633 INFO:teuthology.orchestra.run.gibba026.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,978 kB] 2021-04-03T02:58:15.644 INFO:teuthology.orchestra.run.gibba026.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [310 kB] 2021-04-03T02:58:15.650 INFO:teuthology.orchestra.run.gibba026.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [265 kB] 2021-04-03T02:58:15.654 INFO:teuthology.orchestra.run.gibba026.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-04-03T02:58:15.654 INFO:teuthology.orchestra.run.gibba026.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [34.6 kB] 2021-04-03T02:58:15.655 INFO:teuthology.orchestra.run.gibba026.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [980 kB] 2021-04-03T02:58:15.722 INFO:teuthology.orchestra.run.gibba026.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,118 kB] 2021-04-03T02:58:15.738 INFO:teuthology.orchestra.run.gibba026.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [402 kB] 2021-04-03T02:58:15.742 INFO:teuthology.orchestra.run.gibba026.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [251 kB] 2021-04-03T02:58:15.744 INFO:teuthology.orchestra.run.gibba026.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-04-03T02:58:15.744 INFO:teuthology.orchestra.run.gibba026.stdout:Get:18 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-04-03T02:58:15.744 INFO:teuthology.orchestra.run.gibba026.stdout:Get:19 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-04-03T02:58:15.755 INFO:teuthology.orchestra.run.gibba026.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [286 kB] 2021-04-03T02:58:15.765 INFO:teuthology.orchestra.run.gibba026.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-04-03T02:58:15.765 INFO:teuthology.orchestra.run.gibba026.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [38.2 kB] 2021-04-03T02:58:15.766 INFO:teuthology.orchestra.run.gibba026.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,724 kB] 2021-04-03T02:58:15.806 INFO:teuthology.orchestra.run.gibba026.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,562 kB] 2021-04-03T02:58:15.851 INFO:teuthology.orchestra.run.gibba026.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [365 kB] 2021-04-03T02:58:15.866 INFO:teuthology.orchestra.run.gibba026.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-04-03T02:58:15.867 INFO:teuthology.orchestra.run.gibba026.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-04-03T02:58:15.867 INFO:teuthology.orchestra.run.gibba026.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-04-03T02:58:17.068 INFO:teuthology.orchestra.run.gibba026.stdout:Fetched 13.5 MB in 2s (6,077 kB/s) 2021-04-03T02:58:18.134 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T02:58:18.160 DEBUG:teuthology.orchestra.run.gibba026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T02:58:18.210 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T02:58:18.406 INFO:teuthology.orchestra.run.gibba026.stdout:Building dependency tree... 2021-04-03T02:58:18.407 INFO:teuthology.orchestra.run.gibba026.stdout:Reading state information... 2021-04-03T02:58:18.468 INFO:teuthology.orchestra.run.gibba026.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T02:58:18.468 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T02:58:18.469 DEBUG:teuthology.orchestra.run.gibba026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T02:58:18.562 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T02:58:18.733 INFO:teuthology.orchestra.run.gibba026.stdout:Building dependency tree... 2021-04-03T02:58:18.734 INFO:teuthology.orchestra.run.gibba026.stdout:Reading state information... 2021-04-03T02:58:18.835 INFO:teuthology.orchestra.run.gibba026.stdout:The following additional packages will be installed: 2021-04-03T02:58:18.835 INFO:teuthology.orchestra.run.gibba026.stdout: linux-generic linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:58:18.836 INFO:teuthology.orchestra.run.gibba026.stdout: linux-headers-generic linux-image-4.15.0-140-generic 2021-04-03T02:58:18.836 INFO:teuthology.orchestra.run.gibba026.stdout: linux-modules-4.15.0-140-generic linux-modules-extra-4.15.0-140-generic 2021-04-03T02:58:18.836 INFO:teuthology.orchestra.run.gibba026.stdout:Suggested packages: 2021-04-03T02:58:18.837 INFO:teuthology.orchestra.run.gibba026.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-04-03T02:58:18.837 INFO:teuthology.orchestra.run.gibba026.stdout:Recommended packages: 2021-04-03T02:58:18.837 INFO:teuthology.orchestra.run.gibba026.stdout: thermald 2021-04-03T02:58:18.851 INFO:teuthology.orchestra.run.gibba026.stdout:The following NEW packages will be installed: 2021-04-03T02:58:18.851 INFO:teuthology.orchestra.run.gibba026.stdout: linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:58:18.851 INFO:teuthology.orchestra.run.gibba026.stdout: linux-image-4.15.0-140-generic linux-modules-4.15.0-140-generic 2021-04-03T02:58:18.852 INFO:teuthology.orchestra.run.gibba026.stdout: linux-modules-extra-4.15.0-140-generic 2021-04-03T02:58:18.852 INFO:teuthology.orchestra.run.gibba026.stdout:The following packages will be upgraded: 2021-04-03T02:58:18.852 INFO:teuthology.orchestra.run.gibba026.stdout: linux-generic linux-headers-generic linux-image-generic 2021-04-03T02:58:19.019 INFO:teuthology.orchestra.run.gibba026.stdout:3 upgraded, 5 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T02:58:19.020 INFO:teuthology.orchestra.run.gibba026.stdout:Need to get 66.0 MB of archives. 2021-04-03T02:58:19.020 INFO:teuthology.orchestra.run.gibba026.stdout:After this operation, 334 MB of additional disk space will be used. 2021-04-03T02:58:19.020 INFO:teuthology.orchestra.run.gibba026.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-140-generic amd64 4.15.0-140.144 [13.1 MB] 2021-04-03T02:58:20.040 INFO:teuthology.orchestra.run.gibba026.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-140-generic amd64 4.15.0-140.144 [8,021 kB] 2021-04-03T02:58:20.272 INFO:teuthology.orchestra.run.gibba026.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-140-generic amd64 4.15.0-140.144 [32.8 MB] 2021-04-03T02:58:21.170 INFO:teuthology.orchestra.run.gibba026.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.140.127 [1,868 B] 2021-04-03T02:58:21.170 INFO:teuthology.orchestra.run.gibba026.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.140.127 [2,580 B] 2021-04-03T02:58:21.170 INFO:teuthology.orchestra.run.gibba026.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140 all 4.15.0-140.144 [10.9 MB] 2021-04-03T02:58:21.484 INFO:teuthology.orchestra.run.gibba026.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140-generic amd64 4.15.0-140.144 [1,101 kB] 2021-04-03T02:58:21.502 INFO:teuthology.orchestra.run.gibba026.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.140.127 [2,484 B] 2021-04-03T02:58:21.811 INFO:teuthology.orchestra.run.gibba026.stdout:Fetched 66.0 MB in 3s (25.0 MB/s) 2021-04-03T02:58:21.949 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package linux-modules-4.15.0-140-generic. 2021-04-03T02:58:21.965 INFO:teuthology.orchestra.run.gibba026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102071 files and directories currently installed.) 2021-04-03T02:58:21.966 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../0-linux-modules-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:22.014 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:23.205 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package linux-image-4.15.0-140-generic. 2021-04-03T02:58:23.208 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../1-linux-image-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:23.294 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:23.742 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-140-generic. 2021-04-03T02:58:23.745 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:23.783 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:26.263 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../3-linux-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:58:26.393 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:58:26.683 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:58:26.788 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-image-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:58:27.107 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package linux-headers-4.15.0-140. 2021-04-03T02:58:27.111 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../5-linux-headers-4.15.0-140_4.15.0-140.144_all.deb ... 2021-04-03T02:58:27.149 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:58:29.566 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package linux-headers-4.15.0-140-generic. 2021-04-03T02:58:29.573 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../6-linux-headers-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:29.615 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:30.931 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:58:31.058 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking linux-headers-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:58:31.347 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:58:31.453 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:31.838 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:32.121 INFO:teuthology.orchestra.run.gibba026.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:58:32.121 INFO:teuthology.orchestra.run.gibba026.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-140-generic 2021-04-03T02:58:32.265 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:32.616 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-image-generic (4.15.0.140.127) ... 2021-04-03T02:58:32.742 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:32.868 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-headers-generic (4.15.0.140.127) ... 2021-04-03T02:58:32.994 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up linux-generic (4.15.0.140.127) ... 2021-04-03T02:58:33.121 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:33.252 INFO:teuthology.orchestra.run.gibba026.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-04-03T02:58:33.252 INFO:teuthology.orchestra.run.gibba026.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:58:42.704 INFO:teuthology.orchestra.run.gibba026.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-04-03T02:58:42.704 INFO:teuthology.orchestra.run.gibba026.stdout:Sourcing file `/etc/default/grub' 2021-04-03T02:58:42.730 INFO:teuthology.orchestra.run.gibba026.stdout:Generating grub configuration file ... 2021-04-03T02:58:43.023 INFO:teuthology.orchestra.run.gibba026.stdout:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:58:43.053 INFO:teuthology.orchestra.run.gibba026.stdout:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:58:43.208 INFO:teuthology.orchestra.run.gibba026.stdout:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T02:58:43.253 INFO:teuthology.orchestra.run.gibba026.stdout:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T02:58:44.184 INFO:teuthology.orchestra.run.gibba026.stdout:done 2021-04-03T02:58:44.499 DEBUG:teuthology.orchestra.run.gibba026:> dpkg -s linux-image-generic 2021-04-03T02:58:44.510 INFO:teuthology.orchestra.run.gibba026.stdout:Package: linux-image-generic 2021-04-03T02:58:44.511 INFO:teuthology.orchestra.run.gibba026.stdout:Status: install ok installed 2021-04-03T02:58:44.511 INFO:teuthology.orchestra.run.gibba026.stdout:Priority: optional 2021-04-03T02:58:44.511 INFO:teuthology.orchestra.run.gibba026.stdout:Section: kernel 2021-04-03T02:58:44.512 INFO:teuthology.orchestra.run.gibba026.stdout:Installed-Size: 17 2021-04-03T02:58:44.512 INFO:teuthology.orchestra.run.gibba026.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T02:58:44.512 INFO:teuthology.orchestra.run.gibba026.stdout:Architecture: amd64 2021-04-03T02:58:44.513 INFO:teuthology.orchestra.run.gibba026.stdout:Source: linux-meta 2021-04-03T02:58:44.513 INFO:teuthology.orchestra.run.gibba026.stdout:Version: 4.15.0.140.127 2021-04-03T02:58:44.513 INFO:teuthology.orchestra.run.gibba026.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T02:58:44.513 INFO:teuthology.orchestra.run.gibba026.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T02:58:44.514 INFO:teuthology.orchestra.run.gibba026.stdout:Recommends: thermald 2021-04-03T02:58:44.514 INFO:teuthology.orchestra.run.gibba026.stdout:Description: Generic Linux kernel image 2021-04-03T02:58:44.514 INFO:teuthology.orchestra.run.gibba026.stdout: This package will always depend on the latest generic kernel image 2021-04-03T02:58:44.514 INFO:teuthology.orchestra.run.gibba026.stdout: available. 2021-04-03T02:58:44.515 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-122-generic Expected: 4.15.0-140-generic 2021-04-03T02:58:44.515 DEBUG:teuthology.orchestra.run.gibba028:> uname -r 2021-04-03T02:58:44.524 INFO:teuthology.orchestra.run.gibba028.stdout:4.15.0-122-generic 2021-04-03T02:58:44.524 INFO:teuthology.task.kernel:Running kernel on gibba028: 4.15.0-122-generic 2021-04-03T02:58:44.524 DEBUG:teuthology.orchestra.run.gibba028:> sudo apt-get clean 2021-04-03T02:58:44.607 DEBUG:teuthology.orchestra.run.gibba028:> sudo apt-get update 2021-04-03T02:58:44.733 INFO:teuthology.orchestra.run.gibba028.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T02:58:44.858 INFO:teuthology.orchestra.run.gibba028.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T02:58:44.943 INFO:teuthology.orchestra.run.gibba028.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T02:58:45.002 INFO:teuthology.orchestra.run.gibba028.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,639 kB] 2021-04-03T02:58:45.110 INFO:teuthology.orchestra.run.gibba028.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [950 kB] 2021-04-03T02:58:45.116 INFO:teuthology.orchestra.run.gibba028.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [310 kB] 2021-04-03T02:58:45.119 INFO:teuthology.orchestra.run.gibba028.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-04-03T02:58:45.119 INFO:teuthology.orchestra.run.gibba028.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [265 kB] 2021-04-03T02:58:45.121 INFO:teuthology.orchestra.run.gibba028.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [34.6 kB] 2021-04-03T02:58:45.122 INFO:teuthology.orchestra.run.gibba028.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [980 kB] 2021-04-03T02:58:45.133 INFO:teuthology.orchestra.run.gibba028.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,118 kB] 2021-04-03T02:58:45.141 INFO:teuthology.orchestra.run.gibba028.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [251 kB] 2021-04-03T02:58:45.145 INFO:teuthology.orchestra.run.gibba028.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-04-03T02:58:45.145 INFO:teuthology.orchestra.run.gibba028.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-04-03T02:58:45.145 INFO:teuthology.orchestra.run.gibba028.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-04-03T02:58:45.194 INFO:teuthology.orchestra.run.gibba028.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T02:58:45.276 INFO:teuthology.orchestra.run.gibba028.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,978 kB] 2021-04-03T02:58:45.581 INFO:teuthology.orchestra.run.gibba028.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,253 kB] 2021-04-03T02:58:45.637 INFO:teuthology.orchestra.run.gibba028.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [402 kB] 2021-04-03T02:58:45.649 INFO:teuthology.orchestra.run.gibba028.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-04-03T02:58:45.650 INFO:teuthology.orchestra.run.gibba028.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [286 kB] 2021-04-03T02:58:45.658 INFO:teuthology.orchestra.run.gibba028.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [38.2 kB] 2021-04-03T02:58:45.659 INFO:teuthology.orchestra.run.gibba028.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,724 kB] 2021-04-03T02:58:45.702 INFO:teuthology.orchestra.run.gibba028.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,562 kB] 2021-04-03T02:58:45.743 INFO:teuthology.orchestra.run.gibba028.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [365 kB] 2021-04-03T02:58:45.762 INFO:teuthology.orchestra.run.gibba028.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-04-03T02:58:45.762 INFO:teuthology.orchestra.run.gibba028.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-04-03T02:58:45.763 INFO:teuthology.orchestra.run.gibba028.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-04-03T02:58:46.546 INFO:teuthology.orchestra.run.gibba028.stdout:Fetched 13.5 MB in 2s (7,945 kB/s) 2021-04-03T02:58:47.674 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T02:58:47.700 DEBUG:teuthology.orchestra.run.gibba028:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T02:58:47.744 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T02:58:47.910 INFO:teuthology.orchestra.run.gibba028.stdout:Building dependency tree... 2021-04-03T02:58:47.910 INFO:teuthology.orchestra.run.gibba028.stdout:Reading state information... 2021-04-03T02:58:47.960 INFO:teuthology.orchestra.run.gibba028.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T02:58:47.960 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T02:58:47.961 DEBUG:teuthology.orchestra.run.gibba028:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T02:58:48.051 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T02:58:48.222 INFO:teuthology.orchestra.run.gibba028.stdout:Building dependency tree... 2021-04-03T02:58:48.223 INFO:teuthology.orchestra.run.gibba028.stdout:Reading state information... 2021-04-03T02:58:48.330 INFO:teuthology.orchestra.run.gibba028.stdout:The following additional packages will be installed: 2021-04-03T02:58:48.330 INFO:teuthology.orchestra.run.gibba028.stdout: linux-generic linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:58:48.330 INFO:teuthology.orchestra.run.gibba028.stdout: linux-headers-generic linux-image-4.15.0-140-generic 2021-04-03T02:58:48.330 INFO:teuthology.orchestra.run.gibba028.stdout: linux-modules-4.15.0-140-generic linux-modules-extra-4.15.0-140-generic 2021-04-03T02:58:48.331 INFO:teuthology.orchestra.run.gibba028.stdout:Suggested packages: 2021-04-03T02:58:48.331 INFO:teuthology.orchestra.run.gibba028.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-04-03T02:58:48.332 INFO:teuthology.orchestra.run.gibba028.stdout:Recommended packages: 2021-04-03T02:58:48.332 INFO:teuthology.orchestra.run.gibba028.stdout: thermald 2021-04-03T02:58:48.346 INFO:teuthology.orchestra.run.gibba028.stdout:The following NEW packages will be installed: 2021-04-03T02:58:48.346 INFO:teuthology.orchestra.run.gibba028.stdout: linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:58:48.346 INFO:teuthology.orchestra.run.gibba028.stdout: linux-image-4.15.0-140-generic linux-modules-4.15.0-140-generic 2021-04-03T02:58:48.346 INFO:teuthology.orchestra.run.gibba028.stdout: linux-modules-extra-4.15.0-140-generic 2021-04-03T02:58:48.347 INFO:teuthology.orchestra.run.gibba028.stdout:The following packages will be upgraded: 2021-04-03T02:58:48.347 INFO:teuthology.orchestra.run.gibba028.stdout: linux-generic linux-headers-generic linux-image-generic 2021-04-03T02:58:48.514 INFO:teuthology.orchestra.run.gibba028.stdout:3 upgraded, 5 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T02:58:48.515 INFO:teuthology.orchestra.run.gibba028.stdout:Need to get 66.0 MB of archives. 2021-04-03T02:58:48.515 INFO:teuthology.orchestra.run.gibba028.stdout:After this operation, 334 MB of additional disk space will be used. 2021-04-03T02:58:48.515 INFO:teuthology.orchestra.run.gibba028.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-140-generic amd64 4.15.0-140.144 [13.1 MB] 2021-04-03T02:58:49.500 INFO:teuthology.orchestra.run.gibba028.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-140-generic amd64 4.15.0-140.144 [8,021 kB] 2021-04-03T02:58:49.731 INFO:teuthology.orchestra.run.gibba028.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-140-generic amd64 4.15.0-140.144 [32.8 MB] 2021-04-03T02:58:50.634 INFO:teuthology.orchestra.run.gibba028.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.140.127 [1,868 B] 2021-04-03T02:58:50.634 INFO:teuthology.orchestra.run.gibba028.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.140.127 [2,580 B] 2021-04-03T02:58:50.635 INFO:teuthology.orchestra.run.gibba028.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140 all 4.15.0-140.144 [10.9 MB] 2021-04-03T02:58:50.948 INFO:teuthology.orchestra.run.gibba028.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140-generic amd64 4.15.0-140.144 [1,101 kB] 2021-04-03T02:58:50.964 INFO:teuthology.orchestra.run.gibba028.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.140.127 [2,484 B] 2021-04-03T02:58:51.266 INFO:teuthology.orchestra.run.gibba028.stdout:Fetched 66.0 MB in 3s (25.3 MB/s) 2021-04-03T02:58:51.412 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package linux-modules-4.15.0-140-generic. 2021-04-03T02:58:51.467 INFO:teuthology.orchestra.run.gibba028.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102071 files and directories currently installed.) 2021-04-03T02:58:51.469 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../0-linux-modules-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:51.516 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:52.717 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package linux-image-4.15.0-140-generic. 2021-04-03T02:58:52.733 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../1-linux-image-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:52.824 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:53.320 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-140-generic. 2021-04-03T02:58:53.326 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:53.370 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:58:55.905 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../3-linux-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:58:56.029 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:58:56.355 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:58:56.482 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-image-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:58:56.801 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package linux-headers-4.15.0-140. 2021-04-03T02:58:56.815 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../5-linux-headers-4.15.0-140_4.15.0-140.144_all.deb ... 2021-04-03T02:58:56.859 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:58:59.836 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package linux-headers-4.15.0-140-generic. 2021-04-03T02:58:59.861 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../6-linux-headers-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:58:59.903 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:01.632 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:59:01.758 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking linux-headers-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:59:02.103 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:59:02.229 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:02.671 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:02.978 INFO:teuthology.orchestra.run.gibba028.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:59:02.978 INFO:teuthology.orchestra.run.gibba028.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-140-generic 2021-04-03T02:59:03.115 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:03.500 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-image-generic (4.15.0.140.127) ... 2021-04-03T02:59:03.618 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:03.735 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-headers-generic (4.15.0.140.127) ... 2021-04-03T02:59:03.853 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up linux-generic (4.15.0.140.127) ... 2021-04-03T02:59:03.979 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:04.134 INFO:teuthology.orchestra.run.gibba028.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-04-03T02:59:04.134 INFO:teuthology.orchestra.run.gibba028.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:59:18.529 INFO:teuthology.orchestra.run.gibba028.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-04-03T02:59:18.529 INFO:teuthology.orchestra.run.gibba028.stdout:Sourcing file `/etc/default/grub' 2021-04-03T02:59:18.562 INFO:teuthology.orchestra.run.gibba028.stdout:Generating grub configuration file ... 2021-04-03T02:59:18.908 INFO:teuthology.orchestra.run.gibba028.stdout:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:59:18.946 INFO:teuthology.orchestra.run.gibba028.stdout:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:59:19.319 INFO:teuthology.orchestra.run.gibba028.stdout:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T02:59:19.359 INFO:teuthology.orchestra.run.gibba028.stdout:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T02:59:21.710 INFO:teuthology.orchestra.run.gibba028.stdout:done 2021-04-03T02:59:22.087 DEBUG:teuthology.orchestra.run.gibba028:> dpkg -s linux-image-generic 2021-04-03T02:59:22.098 INFO:teuthology.orchestra.run.gibba028.stdout:Package: linux-image-generic 2021-04-03T02:59:22.098 INFO:teuthology.orchestra.run.gibba028.stdout:Status: install ok installed 2021-04-03T02:59:22.098 INFO:teuthology.orchestra.run.gibba028.stdout:Priority: optional 2021-04-03T02:59:22.099 INFO:teuthology.orchestra.run.gibba028.stdout:Section: kernel 2021-04-03T02:59:22.099 INFO:teuthology.orchestra.run.gibba028.stdout:Installed-Size: 17 2021-04-03T02:59:22.099 INFO:teuthology.orchestra.run.gibba028.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T02:59:22.099 INFO:teuthology.orchestra.run.gibba028.stdout:Architecture: amd64 2021-04-03T02:59:22.099 INFO:teuthology.orchestra.run.gibba028.stdout:Source: linux-meta 2021-04-03T02:59:22.100 INFO:teuthology.orchestra.run.gibba028.stdout:Version: 4.15.0.140.127 2021-04-03T02:59:22.100 INFO:teuthology.orchestra.run.gibba028.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T02:59:22.100 INFO:teuthology.orchestra.run.gibba028.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T02:59:22.100 INFO:teuthology.orchestra.run.gibba028.stdout:Recommends: thermald 2021-04-03T02:59:22.101 INFO:teuthology.orchestra.run.gibba028.stdout:Description: Generic Linux kernel image 2021-04-03T02:59:22.101 INFO:teuthology.orchestra.run.gibba028.stdout: This package will always depend on the latest generic kernel image 2021-04-03T02:59:22.101 INFO:teuthology.orchestra.run.gibba028.stdout: available. 2021-04-03T02:59:22.101 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-122-generic Expected: 4.15.0-140-generic 2021-04-03T02:59:22.102 DEBUG:teuthology.orchestra.run.gibba031:> uname -r 2021-04-03T02:59:22.111 INFO:teuthology.orchestra.run.gibba031.stdout:4.15.0-122-generic 2021-04-03T02:59:22.111 INFO:teuthology.task.kernel:Running kernel on gibba031: 4.15.0-122-generic 2021-04-03T02:59:22.111 DEBUG:teuthology.orchestra.run.gibba031:> sudo apt-get clean 2021-04-03T02:59:22.189 DEBUG:teuthology.orchestra.run.gibba031:> sudo apt-get update 2021-04-03T02:59:22.460 INFO:teuthology.orchestra.run.gibba031.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T02:59:22.460 INFO:teuthology.orchestra.run.gibba031.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T02:59:22.527 INFO:teuthology.orchestra.run.gibba031.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T02:59:22.780 INFO:teuthology.orchestra.run.gibba031.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T02:59:22.786 INFO:teuthology.orchestra.run.gibba031.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [950 kB] 2021-04-03T02:59:22.879 INFO:teuthology.orchestra.run.gibba031.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,253 kB] 2021-04-03T02:59:23.148 INFO:teuthology.orchestra.run.gibba031.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,978 kB] 2021-04-03T02:59:23.174 INFO:teuthology.orchestra.run.gibba031.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,639 kB] 2021-04-03T02:59:23.255 INFO:teuthology.orchestra.run.gibba031.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [402 kB] 2021-04-03T02:59:23.267 INFO:teuthology.orchestra.run.gibba031.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-04-03T02:59:23.268 INFO:teuthology.orchestra.run.gibba031.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [286 kB] 2021-04-03T02:59:23.288 INFO:teuthology.orchestra.run.gibba031.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [38.2 kB] 2021-04-03T02:59:23.289 INFO:teuthology.orchestra.run.gibba031.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,724 kB] 2021-04-03T02:59:23.289 INFO:teuthology.orchestra.run.gibba031.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [310 kB] 2021-04-03T02:59:23.307 INFO:teuthology.orchestra.run.gibba031.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [265 kB] 2021-04-03T02:59:23.319 INFO:teuthology.orchestra.run.gibba031.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-04-03T02:59:23.319 INFO:teuthology.orchestra.run.gibba031.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [34.6 kB] 2021-04-03T02:59:23.320 INFO:teuthology.orchestra.run.gibba031.stdout:Get:18 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [980 kB] 2021-04-03T02:59:23.336 INFO:teuthology.orchestra.run.gibba031.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,562 kB] 2021-04-03T02:59:23.352 INFO:teuthology.orchestra.run.gibba031.stdout:Get:20 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,118 kB] 2021-04-03T02:59:23.382 INFO:teuthology.orchestra.run.gibba031.stdout:Get:21 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [251 kB] 2021-04-03T02:59:23.386 INFO:teuthology.orchestra.run.gibba031.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [365 kB] 2021-04-03T02:59:23.389 INFO:teuthology.orchestra.run.gibba031.stdout:Get:23 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-04-03T02:59:23.390 INFO:teuthology.orchestra.run.gibba031.stdout:Get:24 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-04-03T02:59:23.390 INFO:teuthology.orchestra.run.gibba031.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-04-03T02:59:23.390 INFO:teuthology.orchestra.run.gibba031.stdout:Get:26 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-04-03T02:59:23.391 INFO:teuthology.orchestra.run.gibba031.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-04-03T02:59:23.392 INFO:teuthology.orchestra.run.gibba031.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-04-03T02:59:24.518 INFO:teuthology.orchestra.run.gibba031.stdout:Fetched 13.5 MB in 2s (6,427 kB/s) 2021-04-03T02:59:25.649 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T02:59:25.675 DEBUG:teuthology.orchestra.run.gibba031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T02:59:25.719 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T02:59:25.887 INFO:teuthology.orchestra.run.gibba031.stdout:Building dependency tree... 2021-04-03T02:59:25.888 INFO:teuthology.orchestra.run.gibba031.stdout:Reading state information... 2021-04-03T02:59:25.937 INFO:teuthology.orchestra.run.gibba031.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T02:59:25.937 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T02:59:25.938 DEBUG:teuthology.orchestra.run.gibba031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T02:59:26.031 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T02:59:26.193 INFO:teuthology.orchestra.run.gibba031.stdout:Building dependency tree... 2021-04-03T02:59:26.194 INFO:teuthology.orchestra.run.gibba031.stdout:Reading state information... 2021-04-03T02:59:26.295 INFO:teuthology.orchestra.run.gibba031.stdout:The following additional packages will be installed: 2021-04-03T02:59:26.295 INFO:teuthology.orchestra.run.gibba031.stdout: linux-generic linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:59:26.295 INFO:teuthology.orchestra.run.gibba031.stdout: linux-headers-generic linux-image-4.15.0-140-generic 2021-04-03T02:59:26.295 INFO:teuthology.orchestra.run.gibba031.stdout: linux-modules-4.15.0-140-generic linux-modules-extra-4.15.0-140-generic 2021-04-03T02:59:26.296 INFO:teuthology.orchestra.run.gibba031.stdout:Suggested packages: 2021-04-03T02:59:26.296 INFO:teuthology.orchestra.run.gibba031.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-04-03T02:59:26.296 INFO:teuthology.orchestra.run.gibba031.stdout:Recommended packages: 2021-04-03T02:59:26.297 INFO:teuthology.orchestra.run.gibba031.stdout: thermald 2021-04-03T02:59:26.310 INFO:teuthology.orchestra.run.gibba031.stdout:The following NEW packages will be installed: 2021-04-03T02:59:26.311 INFO:teuthology.orchestra.run.gibba031.stdout: linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic 2021-04-03T02:59:26.311 INFO:teuthology.orchestra.run.gibba031.stdout: linux-image-4.15.0-140-generic linux-modules-4.15.0-140-generic 2021-04-03T02:59:26.311 INFO:teuthology.orchestra.run.gibba031.stdout: linux-modules-extra-4.15.0-140-generic 2021-04-03T02:59:26.312 INFO:teuthology.orchestra.run.gibba031.stdout:The following packages will be upgraded: 2021-04-03T02:59:26.312 INFO:teuthology.orchestra.run.gibba031.stdout: linux-generic linux-headers-generic linux-image-generic 2021-04-03T02:59:26.480 INFO:teuthology.orchestra.run.gibba031.stdout:3 upgraded, 5 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T02:59:26.480 INFO:teuthology.orchestra.run.gibba031.stdout:Need to get 66.0 MB of archives. 2021-04-03T02:59:26.480 INFO:teuthology.orchestra.run.gibba031.stdout:After this operation, 334 MB of additional disk space will be used. 2021-04-03T02:59:26.481 INFO:teuthology.orchestra.run.gibba031.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-140-generic amd64 4.15.0-140.144 [13.1 MB] 2021-04-03T02:59:27.500 INFO:teuthology.orchestra.run.gibba031.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-140-generic amd64 4.15.0-140.144 [8,021 kB] 2021-04-03T02:59:27.731 INFO:teuthology.orchestra.run.gibba031.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-140-generic amd64 4.15.0-140.144 [32.8 MB] 2021-04-03T02:59:28.642 INFO:teuthology.orchestra.run.gibba031.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.140.127 [1,868 B] 2021-04-03T02:59:28.643 INFO:teuthology.orchestra.run.gibba031.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.140.127 [2,580 B] 2021-04-03T02:59:28.643 INFO:teuthology.orchestra.run.gibba031.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140 all 4.15.0-140.144 [10.9 MB] 2021-04-03T02:59:28.947 INFO:teuthology.orchestra.run.gibba031.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-140-generic amd64 4.15.0-140.144 [1,101 kB] 2021-04-03T02:59:29.004 INFO:teuthology.orchestra.run.gibba031.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.140.127 [2,484 B] 2021-04-03T02:59:29.284 INFO:teuthology.orchestra.run.gibba031.stdout:Fetched 66.0 MB in 3s (24.5 MB/s) 2021-04-03T02:59:29.399 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package linux-modules-4.15.0-140-generic. 2021-04-03T02:59:29.452 INFO:teuthology.orchestra.run.gibba031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102071 files and directories currently installed.) 2021-04-03T02:59:29.454 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../0-linux-modules-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:59:29.494 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:30.702 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package linux-image-4.15.0-140-generic. 2021-04-03T02:59:30.708 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../1-linux-image-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:59:30.806 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:31.272 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-140-generic. 2021-04-03T02:59:31.278 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:59:31.313 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:33.738 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../3-linux-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:59:33.845 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:59:34.137 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:59:34.257 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-image-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:59:34.601 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package linux-headers-4.15.0-140. 2021-04-03T02:59:34.615 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../5-linux-headers-4.15.0-140_4.15.0-140.144_all.deb ... 2021-04-03T02:59:34.659 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:59:37.479 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package linux-headers-4.15.0-140-generic. 2021-04-03T02:59:37.512 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../6-linux-headers-4.15.0-140-generic_4.15.0-140.144_amd64.deb ... 2021-04-03T02:59:37.545 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:39.143 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.140.127_amd64.deb ... 2021-04-03T02:59:39.270 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking linux-headers-generic (4.15.0.140.127) over (4.15.0.122.109) ... 2021-04-03T02:59:39.589 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-headers-4.15.0-140 (4.15.0-140.144) ... 2021-04-03T02:59:39.716 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-modules-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:40.141 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:40.449 INFO:teuthology.orchestra.run.gibba031.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:59:40.449 INFO:teuthology.orchestra.run.gibba031.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-140-generic 2021-04-03T02:59:40.585 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-modules-extra-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:40.944 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-image-generic (4.15.0.140.127) ... 2021-04-03T02:59:41.054 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-headers-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:41.163 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-headers-generic (4.15.0.140.127) ... 2021-04-03T02:59:41.264 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up linux-generic (4.15.0.140.127) ... 2021-04-03T02:59:41.373 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for linux-image-4.15.0-140-generic (4.15.0-140.144) ... 2021-04-03T02:59:41.509 INFO:teuthology.orchestra.run.gibba031.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-04-03T02:59:41.510 INFO:teuthology.orchestra.run.gibba031.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:59:54.608 INFO:teuthology.orchestra.run.gibba031.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-04-03T02:59:54.609 INFO:teuthology.orchestra.run.gibba031.stdout:Sourcing file `/etc/default/grub' 2021-04-03T02:59:54.638 INFO:teuthology.orchestra.run.gibba031.stdout:Generating grub configuration file ... 2021-04-03T02:59:54.981 INFO:teuthology.orchestra.run.gibba031.stdout:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T02:59:55.020 INFO:teuthology.orchestra.run.gibba031.stdout:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T02:59:55.407 INFO:teuthology.orchestra.run.gibba031.stdout:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T02:59:55.451 INFO:teuthology.orchestra.run.gibba031.stdout:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T02:59:56.867 INFO:teuthology.orchestra.run.gibba031.stdout:done 2021-04-03T02:59:57.248 DEBUG:teuthology.orchestra.run.gibba031:> dpkg -s linux-image-generic 2021-04-03T02:59:57.259 INFO:teuthology.orchestra.run.gibba031.stdout:Package: linux-image-generic 2021-04-03T02:59:57.259 INFO:teuthology.orchestra.run.gibba031.stdout:Status: install ok installed 2021-04-03T02:59:57.259 INFO:teuthology.orchestra.run.gibba031.stdout:Priority: optional 2021-04-03T02:59:57.259 INFO:teuthology.orchestra.run.gibba031.stdout:Section: kernel 2021-04-03T02:59:57.260 INFO:teuthology.orchestra.run.gibba031.stdout:Installed-Size: 17 2021-04-03T02:59:57.260 INFO:teuthology.orchestra.run.gibba031.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T02:59:57.260 INFO:teuthology.orchestra.run.gibba031.stdout:Architecture: amd64 2021-04-03T02:59:57.260 INFO:teuthology.orchestra.run.gibba031.stdout:Source: linux-meta 2021-04-03T02:59:57.260 INFO:teuthology.orchestra.run.gibba031.stdout:Version: 4.15.0.140.127 2021-04-03T02:59:57.261 INFO:teuthology.orchestra.run.gibba031.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T02:59:57.261 INFO:teuthology.orchestra.run.gibba031.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T02:59:57.261 INFO:teuthology.orchestra.run.gibba031.stdout:Recommends: thermald 2021-04-03T02:59:57.261 INFO:teuthology.orchestra.run.gibba031.stdout:Description: Generic Linux kernel image 2021-04-03T02:59:57.261 INFO:teuthology.orchestra.run.gibba031.stdout: This package will always depend on the latest generic kernel image 2021-04-03T02:59:57.262 INFO:teuthology.orchestra.run.gibba031.stdout: available. 2021-04-03T02:59:57.262 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-122-generic Expected: 4.15.0-140-generic 2021-04-03T02:59:57.262 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-04-03T02:59:57.263 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-04-03T02:59:57.263 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-04-03T02:59:57.263 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-04-03T02:59:57.263 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-04-03T02:59:57.264 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-04-03T02:59:57.264 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2021-04-03T02:59:57.264 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba011.front.sepia.ceph.com, path=None, version=distro) 2021-04-03T02:59:57.265 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-get clean 2021-04-03T02:59:57.315 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-get update 2021-04-03T02:59:57.389 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-04-03T02:59:57.511 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T02:59:57.591 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-04-03T02:59:57.671 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-04-03T02:59:58.820 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T02:59:58.846 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T02:59:58.892 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T02:59:59.059 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-04-03T02:59:59.059 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-04-03T02:59:59.109 INFO:teuthology.orchestra.run.gibba011.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T02:59:59.110 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T02:59:59.111 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T02:59:59.210 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T02:59:59.375 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-04-03T02:59:59.375 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-04-03T02:59:59.489 INFO:teuthology.orchestra.run.gibba011.stdout:linux-image-generic is already the newest version (4.15.0.140.127). 2021-04-03T02:59:59.489 INFO:teuthology.orchestra.run.gibba011.stdout:0 upgraded, 0 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T02:59:59.490 DEBUG:teuthology.orchestra.run.gibba011:> dpkg -s linux-image-generic 2021-04-03T02:59:59.545 INFO:teuthology.orchestra.run.gibba011.stdout:Package: linux-image-generic 2021-04-03T02:59:59.545 INFO:teuthology.orchestra.run.gibba011.stdout:Status: install ok installed 2021-04-03T02:59:59.545 INFO:teuthology.orchestra.run.gibba011.stdout:Priority: optional 2021-04-03T02:59:59.546 INFO:teuthology.orchestra.run.gibba011.stdout:Section: kernel 2021-04-03T02:59:59.546 INFO:teuthology.orchestra.run.gibba011.stdout:Installed-Size: 17 2021-04-03T02:59:59.546 INFO:teuthology.orchestra.run.gibba011.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T02:59:59.546 INFO:teuthology.orchestra.run.gibba011.stdout:Architecture: amd64 2021-04-03T02:59:59.546 INFO:teuthology.orchestra.run.gibba011.stdout:Source: linux-meta 2021-04-03T02:59:59.547 INFO:teuthology.orchestra.run.gibba011.stdout:Version: 4.15.0.140.127 2021-04-03T02:59:59.547 INFO:teuthology.orchestra.run.gibba011.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T02:59:59.547 INFO:teuthology.orchestra.run.gibba011.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T02:59:59.547 INFO:teuthology.orchestra.run.gibba011.stdout:Recommends: thermald 2021-04-03T02:59:59.548 INFO:teuthology.orchestra.run.gibba011.stdout:Description: Generic Linux kernel image 2021-04-03T02:59:59.548 INFO:teuthology.orchestra.run.gibba011.stdout: This package will always depend on the latest generic kernel image 2021-04-03T02:59:59.548 INFO:teuthology.orchestra.run.gibba011.stdout: available. 2021-04-03T02:59:59.548 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T02:59:59.589 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.vqrFfl97Dd 2021-04-03T02:59:59.590 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /boot/grub/grub.cfg /tmp/tmp.vqrFfl97Dd 2021-04-03T02:59:59.651 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.vqrFfl97Dd 2021-04-03T02:59:59.767 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.vqrFfl97Dd is 8KB 2021-04-03T02:59:59.819 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.vqrFfl97Dd 2021-04-03T02:59:59.824 DEBUG:teuthology.orchestra.run.gibba011:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-04-03T02:59:59.883 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T02:59:59.883 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-04-03T02:59:59.883 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-04-03T02:59:59.950 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-140-generic 2021-04-03T02:59:59.950 DEBUG:teuthology.orchestra.run.gibba011:> sudo update-grub 2021-04-03T03:00:01.244 INFO:teuthology.orchestra.run.gibba011.stderr:Sourcing file `/etc/default/grub' 2021-04-03T03:00:01.286 INFO:teuthology.orchestra.run.gibba011.stderr:Generating grub configuration file ... 2021-04-03T03:00:01.800 INFO:teuthology.orchestra.run.gibba011.stderr:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T03:00:01.816 INFO:teuthology.orchestra.run.gibba011.stderr:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T03:00:02.277 INFO:teuthology.orchestra.run.gibba011.stderr:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T03:00:02.293 INFO:teuthology.orchestra.run.gibba011.stderr:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T03:00:03.905 INFO:teuthology.orchestra.run.gibba011.stderr:done 2021-04-03T03:00:03.907 DEBUG:teuthology.orchestra.run.gibba011:> sudo shutdown -r now 2021-04-03T03:00:03.911 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2021-04-03T03:00:03.911 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba015.front.sepia.ceph.com, path=None, version=distro) 2021-04-03T03:00:03.911 DEBUG:teuthology.orchestra.run.gibba015:> sudo apt-get clean 2021-04-03T03:00:03.970 DEBUG:teuthology.orchestra.run.gibba015:> sudo apt-get update 2021-04-03T03:00:05.008 INFO:teuthology.orchestra.run.gibba015.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-04-03T03:00:05.457 INFO:teuthology.orchestra.run.gibba015.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:00:05.537 INFO:teuthology.orchestra.run.gibba015.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-04-03T03:00:05.617 INFO:teuthology.orchestra.run.gibba015.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-04-03T03:00:06.648 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T03:00:06.675 DEBUG:teuthology.orchestra.run.gibba015:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T03:00:06.720 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T03:00:06.885 INFO:teuthology.orchestra.run.gibba015.stdout:Building dependency tree... 2021-04-03T03:00:06.885 INFO:teuthology.orchestra.run.gibba015.stdout:Reading state information... 2021-04-03T03:00:06.935 INFO:teuthology.orchestra.run.gibba015.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T03:00:06.936 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T03:00:06.937 DEBUG:teuthology.orchestra.run.gibba015:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T03:00:07.033 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T03:00:07.202 INFO:teuthology.orchestra.run.gibba015.stdout:Building dependency tree... 2021-04-03T03:00:07.203 INFO:teuthology.orchestra.run.gibba015.stdout:Reading state information... 2021-04-03T03:00:07.321 INFO:teuthology.orchestra.run.gibba015.stdout:linux-image-generic is already the newest version (4.15.0.140.127). 2021-04-03T03:00:07.322 INFO:teuthology.orchestra.run.gibba015.stdout:0 upgraded, 0 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T03:00:07.322 DEBUG:teuthology.orchestra.run.gibba015:> dpkg -s linux-image-generic 2021-04-03T03:00:07.376 INFO:teuthology.orchestra.run.gibba015.stdout:Package: linux-image-generic 2021-04-03T03:00:07.377 INFO:teuthology.orchestra.run.gibba015.stdout:Status: install ok installed 2021-04-03T03:00:07.377 INFO:teuthology.orchestra.run.gibba015.stdout:Priority: optional 2021-04-03T03:00:07.377 INFO:teuthology.orchestra.run.gibba015.stdout:Section: kernel 2021-04-03T03:00:07.377 INFO:teuthology.orchestra.run.gibba015.stdout:Installed-Size: 17 2021-04-03T03:00:07.378 INFO:teuthology.orchestra.run.gibba015.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T03:00:07.378 INFO:teuthology.orchestra.run.gibba015.stdout:Architecture: amd64 2021-04-03T03:00:07.378 INFO:teuthology.orchestra.run.gibba015.stdout:Source: linux-meta 2021-04-03T03:00:07.378 INFO:teuthology.orchestra.run.gibba015.stdout:Version: 4.15.0.140.127 2021-04-03T03:00:07.379 INFO:teuthology.orchestra.run.gibba015.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T03:00:07.379 INFO:teuthology.orchestra.run.gibba015.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T03:00:07.379 INFO:teuthology.orchestra.run.gibba015.stdout:Recommends: thermald 2021-04-03T03:00:07.379 INFO:teuthology.orchestra.run.gibba015.stdout:Description: Generic Linux kernel image 2021-04-03T03:00:07.380 INFO:teuthology.orchestra.run.gibba015.stdout: This package will always depend on the latest generic kernel image 2021-04-03T03:00:07.380 INFO:teuthology.orchestra.run.gibba015.stdout: available. 2021-04-03T03:00:07.380 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:00:07.421 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.QHui5DayR3 2021-04-03T03:00:07.422 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /boot/grub/grub.cfg /tmp/tmp.QHui5DayR3 2021-04-03T03:00:07.483 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.QHui5DayR3 2021-04-03T03:00:07.602 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.QHui5DayR3 is 8KB 2021-04-03T03:00:07.616 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.QHui5DayR3 2021-04-03T03:00:07.623 DEBUG:teuthology.orchestra.run.gibba015:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:07.682 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:00:07.682 DEBUG:teuthology.orchestra.run.gibba015:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-04-03T03:00:07.683 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:07.744 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-140-generic 2021-04-03T03:00:07.745 DEBUG:teuthology.orchestra.run.gibba015:> sudo update-grub 2021-04-03T03:00:08.895 INFO:teuthology.orchestra.run.gibba015.stderr:Sourcing file `/etc/default/grub' 2021-04-03T03:00:08.916 INFO:teuthology.orchestra.run.gibba015.stderr:Generating grub configuration file ... 2021-04-03T03:00:09.197 INFO:teuthology.orchestra.run.gibba015.stderr:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T03:00:09.212 INFO:teuthology.orchestra.run.gibba015.stderr:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T03:00:09.610 INFO:teuthology.orchestra.run.gibba015.stderr:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T03:00:09.626 INFO:teuthology.orchestra.run.gibba015.stderr:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T03:00:10.976 INFO:teuthology.orchestra.run.gibba015.stderr:done 2021-04-03T03:00:10.978 DEBUG:teuthology.orchestra.run.gibba015:> sudo shutdown -r now 2021-04-03T03:00:10.981 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2021-04-03T03:00:10.982 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba026.front.sepia.ceph.com, path=None, version=distro) 2021-04-03T03:00:10.982 DEBUG:teuthology.orchestra.run.gibba026:> sudo apt-get clean 2021-04-03T03:00:11.067 DEBUG:teuthology.orchestra.run.gibba026:> sudo apt-get update 2021-04-03T03:00:11.296 INFO:teuthology.orchestra.run.gibba026.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-04-03T03:00:11.297 INFO:teuthology.orchestra.run.gibba026.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:00:11.368 INFO:teuthology.orchestra.run.gibba026.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-04-03T03:00:11.447 INFO:teuthology.orchestra.run.gibba026.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-04-03T03:00:12.521 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T03:00:12.547 DEBUG:teuthology.orchestra.run.gibba026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T03:00:12.593 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T03:00:12.765 INFO:teuthology.orchestra.run.gibba026.stdout:Building dependency tree... 2021-04-03T03:00:12.765 INFO:teuthology.orchestra.run.gibba026.stdout:Reading state information... 2021-04-03T03:00:12.817 INFO:teuthology.orchestra.run.gibba026.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T03:00:12.818 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T03:00:12.818 DEBUG:teuthology.orchestra.run.gibba026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T03:00:12.906 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T03:00:13.071 INFO:teuthology.orchestra.run.gibba026.stdout:Building dependency tree... 2021-04-03T03:00:13.072 INFO:teuthology.orchestra.run.gibba026.stdout:Reading state information... 2021-04-03T03:00:13.187 INFO:teuthology.orchestra.run.gibba026.stdout:linux-image-generic is already the newest version (4.15.0.140.127). 2021-04-03T03:00:13.187 INFO:teuthology.orchestra.run.gibba026.stdout:0 upgraded, 0 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T03:00:13.188 DEBUG:teuthology.orchestra.run.gibba026:> dpkg -s linux-image-generic 2021-04-03T03:00:13.238 INFO:teuthology.orchestra.run.gibba026.stdout:Package: linux-image-generic 2021-04-03T03:00:13.238 INFO:teuthology.orchestra.run.gibba026.stdout:Status: install ok installed 2021-04-03T03:00:13.239 INFO:teuthology.orchestra.run.gibba026.stdout:Priority: optional 2021-04-03T03:00:13.239 INFO:teuthology.orchestra.run.gibba026.stdout:Section: kernel 2021-04-03T03:00:13.239 INFO:teuthology.orchestra.run.gibba026.stdout:Installed-Size: 17 2021-04-03T03:00:13.239 INFO:teuthology.orchestra.run.gibba026.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T03:00:13.240 INFO:teuthology.orchestra.run.gibba026.stdout:Architecture: amd64 2021-04-03T03:00:13.240 INFO:teuthology.orchestra.run.gibba026.stdout:Source: linux-meta 2021-04-03T03:00:13.240 INFO:teuthology.orchestra.run.gibba026.stdout:Version: 4.15.0.140.127 2021-04-03T03:00:13.240 INFO:teuthology.orchestra.run.gibba026.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T03:00:13.241 INFO:teuthology.orchestra.run.gibba026.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T03:00:13.241 INFO:teuthology.orchestra.run.gibba026.stdout:Recommends: thermald 2021-04-03T03:00:13.241 INFO:teuthology.orchestra.run.gibba026.stdout:Description: Generic Linux kernel image 2021-04-03T03:00:13.241 INFO:teuthology.orchestra.run.gibba026.stdout: This package will always depend on the latest generic kernel image 2021-04-03T03:00:13.242 INFO:teuthology.orchestra.run.gibba026.stdout: available. 2021-04-03T03:00:13.242 DEBUG:teuthology.orchestra.run.gibba026:> mktemp 2021-04-03T03:00:13.283 INFO:teuthology.orchestra.run.gibba026.stdout:/tmp/tmp.lDvwcK6nnt 2021-04-03T03:00:13.283 DEBUG:teuthology.orchestra.run.gibba026:> sudo cp /boot/grub/grub.cfg /tmp/tmp.lDvwcK6nnt 2021-04-03T03:00:13.329 DEBUG:teuthology.orchestra.run.gibba026:> sudo chmod 0666 /tmp/tmp.lDvwcK6nnt 2021-04-03T03:00:13.432 DEBUG:teuthology.orchestra.remote:gibba026:/tmp/tmp.lDvwcK6nnt is 8KB 2021-04-03T03:00:13.481 DEBUG:teuthology.orchestra.run.gibba026:> rm -fr /tmp/tmp.lDvwcK6nnt 2021-04-03T03:00:13.484 DEBUG:teuthology.orchestra.run.gibba026:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:13.529 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:00:13.530 DEBUG:teuthology.orchestra.run.gibba026:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-04-03T03:00:13.530 DEBUG:teuthology.orchestra.run.gibba026:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:13.582 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-140-generic 2021-04-03T03:00:13.582 DEBUG:teuthology.orchestra.run.gibba026:> sudo update-grub 2021-04-03T03:00:14.927 INFO:teuthology.orchestra.run.gibba026.stderr:Sourcing file `/etc/default/grub' 2021-04-03T03:00:14.964 INFO:teuthology.orchestra.run.gibba026.stderr:Generating grub configuration file ... 2021-04-03T03:00:15.215 INFO:teuthology.orchestra.run.gibba026.stderr:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T03:00:15.227 INFO:teuthology.orchestra.run.gibba026.stderr:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T03:00:15.530 INFO:teuthology.orchestra.run.gibba026.stderr:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T03:00:15.542 INFO:teuthology.orchestra.run.gibba026.stderr:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T03:00:16.366 INFO:teuthology.orchestra.run.gibba026.stderr:done 2021-04-03T03:00:16.367 DEBUG:teuthology.orchestra.run.gibba026:> sudo shutdown -r now 2021-04-03T03:00:16.368 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2021-04-03T03:00:16.369 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba028.front.sepia.ceph.com, path=None, version=distro) 2021-04-03T03:00:16.369 DEBUG:teuthology.orchestra.run.gibba028:> sudo apt-get clean 2021-04-03T03:00:16.406 DEBUG:teuthology.orchestra.run.gibba028:> sudo apt-get update 2021-04-03T03:00:16.561 INFO:teuthology.orchestra.run.gibba028.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-04-03T03:00:16.662 INFO:teuthology.orchestra.run.gibba028.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:00:16.742 INFO:teuthology.orchestra.run.gibba028.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-04-03T03:00:16.822 INFO:teuthology.orchestra.run.gibba028.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-04-03T03:00:18.094 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T03:00:18.120 DEBUG:teuthology.orchestra.run.gibba028:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T03:00:18.165 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T03:00:18.337 INFO:teuthology.orchestra.run.gibba028.stdout:Building dependency tree... 2021-04-03T03:00:18.337 INFO:teuthology.orchestra.run.gibba028.stdout:Reading state information... 2021-04-03T03:00:18.387 INFO:teuthology.orchestra.run.gibba028.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T03:00:18.388 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T03:00:18.389 DEBUG:teuthology.orchestra.run.gibba028:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T03:00:18.489 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T03:00:18.659 INFO:teuthology.orchestra.run.gibba028.stdout:Building dependency tree... 2021-04-03T03:00:18.660 INFO:teuthology.orchestra.run.gibba028.stdout:Reading state information... 2021-04-03T03:00:18.779 INFO:teuthology.orchestra.run.gibba028.stdout:linux-image-generic is already the newest version (4.15.0.140.127). 2021-04-03T03:00:18.779 INFO:teuthology.orchestra.run.gibba028.stdout:0 upgraded, 0 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T03:00:18.780 DEBUG:teuthology.orchestra.run.gibba028:> dpkg -s linux-image-generic 2021-04-03T03:00:18.832 INFO:teuthology.orchestra.run.gibba028.stdout:Package: linux-image-generic 2021-04-03T03:00:18.833 INFO:teuthology.orchestra.run.gibba028.stdout:Status: install ok installed 2021-04-03T03:00:18.833 INFO:teuthology.orchestra.run.gibba028.stdout:Priority: optional 2021-04-03T03:00:18.833 INFO:teuthology.orchestra.run.gibba028.stdout:Section: kernel 2021-04-03T03:00:18.833 INFO:teuthology.orchestra.run.gibba028.stdout:Installed-Size: 17 2021-04-03T03:00:18.834 INFO:teuthology.orchestra.run.gibba028.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T03:00:18.834 INFO:teuthology.orchestra.run.gibba028.stdout:Architecture: amd64 2021-04-03T03:00:18.834 INFO:teuthology.orchestra.run.gibba028.stdout:Source: linux-meta 2021-04-03T03:00:18.834 INFO:teuthology.orchestra.run.gibba028.stdout:Version: 4.15.0.140.127 2021-04-03T03:00:18.835 INFO:teuthology.orchestra.run.gibba028.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T03:00:18.835 INFO:teuthology.orchestra.run.gibba028.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T03:00:18.835 INFO:teuthology.orchestra.run.gibba028.stdout:Recommends: thermald 2021-04-03T03:00:18.835 INFO:teuthology.orchestra.run.gibba028.stdout:Description: Generic Linux kernel image 2021-04-03T03:00:18.835 INFO:teuthology.orchestra.run.gibba028.stdout: This package will always depend on the latest generic kernel image 2021-04-03T03:00:18.836 INFO:teuthology.orchestra.run.gibba028.stdout: available. 2021-04-03T03:00:18.836 DEBUG:teuthology.orchestra.run.gibba028:> mktemp 2021-04-03T03:00:18.876 INFO:teuthology.orchestra.run.gibba028.stdout:/tmp/tmp.U2v6sssdWH 2021-04-03T03:00:18.877 DEBUG:teuthology.orchestra.run.gibba028:> sudo cp /boot/grub/grub.cfg /tmp/tmp.U2v6sssdWH 2021-04-03T03:00:18.932 DEBUG:teuthology.orchestra.run.gibba028:> sudo chmod 0666 /tmp/tmp.U2v6sssdWH 2021-04-03T03:00:19.034 DEBUG:teuthology.orchestra.remote:gibba028:/tmp/tmp.U2v6sssdWH is 8KB 2021-04-03T03:00:19.089 DEBUG:teuthology.orchestra.run.gibba028:> rm -fr /tmp/tmp.U2v6sssdWH 2021-04-03T03:00:19.097 DEBUG:teuthology.orchestra.run.gibba028:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:19.160 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:00:19.160 DEBUG:teuthology.orchestra.run.gibba028:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-04-03T03:00:19.160 DEBUG:teuthology.orchestra.run.gibba028:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:19.225 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-140-generic 2021-04-03T03:00:19.225 DEBUG:teuthology.orchestra.run.gibba028:> sudo update-grub 2021-04-03T03:00:20.444 INFO:teuthology.orchestra.run.gibba028.stderr:Sourcing file `/etc/default/grub' 2021-04-03T03:00:20.464 INFO:teuthology.orchestra.run.gibba028.stderr:Generating grub configuration file ... 2021-04-03T03:00:20.737 INFO:teuthology.orchestra.run.gibba028.stderr:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T03:00:20.754 INFO:teuthology.orchestra.run.gibba028.stderr:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T03:00:21.139 INFO:teuthology.orchestra.run.gibba028.stderr:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T03:00:21.156 INFO:teuthology.orchestra.run.gibba028.stderr:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T03:00:23.067 INFO:teuthology.orchestra.run.gibba028.stderr:done 2021-04-03T03:00:23.070 DEBUG:teuthology.orchestra.run.gibba028:> sudo shutdown -r now 2021-04-03T03:00:23.073 INFO:teuthology.task.kernel:Installing distro kernel on client.2... 2021-04-03T03:00:23.073 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba031.front.sepia.ceph.com, path=None, version=distro) 2021-04-03T03:00:23.074 DEBUG:teuthology.orchestra.run.gibba031:> sudo apt-get clean 2021-04-03T03:00:23.109 DEBUG:teuthology.orchestra.run.gibba031:> sudo apt-get update 2021-04-03T03:00:23.252 INFO:teuthology.orchestra.run.gibba031.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-04-03T03:00:23.375 INFO:teuthology.orchestra.run.gibba031.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:00:23.455 INFO:teuthology.orchestra.run.gibba031.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-04-03T03:00:23.535 INFO:teuthology.orchestra.run.gibba031.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-04-03T03:00:24.693 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T03:00:24.719 DEBUG:teuthology.orchestra.run.gibba031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-04-03T03:00:24.765 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T03:00:24.929 INFO:teuthology.orchestra.run.gibba031.stdout:Building dependency tree... 2021-04-03T03:00:24.930 INFO:teuthology.orchestra.run.gibba031.stdout:Reading state information... 2021-04-03T03:00:24.979 INFO:teuthology.orchestra.run.gibba031.stderr:E: Unable to locate package linux-image-current-generic 2021-04-03T03:00:24.980 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-04-03T03:00:24.981 DEBUG:teuthology.orchestra.run.gibba031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-04-03T03:00:25.078 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T03:00:25.246 INFO:teuthology.orchestra.run.gibba031.stdout:Building dependency tree... 2021-04-03T03:00:25.247 INFO:teuthology.orchestra.run.gibba031.stdout:Reading state information... 2021-04-03T03:00:25.356 INFO:teuthology.orchestra.run.gibba031.stdout:linux-image-generic is already the newest version (4.15.0.140.127). 2021-04-03T03:00:25.356 INFO:teuthology.orchestra.run.gibba031.stdout:0 upgraded, 0 newly installed, 0 to remove and 139 not upgraded. 2021-04-03T03:00:25.357 DEBUG:teuthology.orchestra.run.gibba031:> dpkg -s linux-image-generic 2021-04-03T03:00:25.410 INFO:teuthology.orchestra.run.gibba031.stdout:Package: linux-image-generic 2021-04-03T03:00:25.411 INFO:teuthology.orchestra.run.gibba031.stdout:Status: install ok installed 2021-04-03T03:00:25.411 INFO:teuthology.orchestra.run.gibba031.stdout:Priority: optional 2021-04-03T03:00:25.411 INFO:teuthology.orchestra.run.gibba031.stdout:Section: kernel 2021-04-03T03:00:25.411 INFO:teuthology.orchestra.run.gibba031.stdout:Installed-Size: 17 2021-04-03T03:00:25.412 INFO:teuthology.orchestra.run.gibba031.stdout:Maintainer: Ubuntu Kernel Team 2021-04-03T03:00:25.412 INFO:teuthology.orchestra.run.gibba031.stdout:Architecture: amd64 2021-04-03T03:00:25.412 INFO:teuthology.orchestra.run.gibba031.stdout:Source: linux-meta 2021-04-03T03:00:25.412 INFO:teuthology.orchestra.run.gibba031.stdout:Version: 4.15.0.140.127 2021-04-03T03:00:25.413 INFO:teuthology.orchestra.run.gibba031.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-04-03T03:00:25.413 INFO:teuthology.orchestra.run.gibba031.stdout:Depends: linux-image-4.15.0-140-generic, linux-modules-extra-4.15.0-140-generic, linux-firmware, intel-microcode, amd64-microcode 2021-04-03T03:00:25.413 INFO:teuthology.orchestra.run.gibba031.stdout:Recommends: thermald 2021-04-03T03:00:25.413 INFO:teuthology.orchestra.run.gibba031.stdout:Description: Generic Linux kernel image 2021-04-03T03:00:25.414 INFO:teuthology.orchestra.run.gibba031.stdout: This package will always depend on the latest generic kernel image 2021-04-03T03:00:25.414 INFO:teuthology.orchestra.run.gibba031.stdout: available. 2021-04-03T03:00:25.414 DEBUG:teuthology.orchestra.run.gibba031:> mktemp 2021-04-03T03:00:25.454 INFO:teuthology.orchestra.run.gibba031.stdout:/tmp/tmp.apUyDAG37x 2021-04-03T03:00:25.455 DEBUG:teuthology.orchestra.run.gibba031:> sudo cp /boot/grub/grub.cfg /tmp/tmp.apUyDAG37x 2021-04-03T03:00:25.510 DEBUG:teuthology.orchestra.run.gibba031:> sudo chmod 0666 /tmp/tmp.apUyDAG37x 2021-04-03T03:00:25.621 DEBUG:teuthology.orchestra.remote:gibba031:/tmp/tmp.apUyDAG37x is 8KB 2021-04-03T03:00:25.634 DEBUG:teuthology.orchestra.run.gibba031:> rm -fr /tmp/tmp.apUyDAG37x 2021-04-03T03:00:25.641 DEBUG:teuthology.orchestra.run.gibba031:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:25.693 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:00:25.693 DEBUG:teuthology.orchestra.run.gibba031:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-04-03T03:00:25.693 DEBUG:teuthology.orchestra.run.gibba031:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-04-03T03:00:25.750 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-140-generic 2021-04-03T03:00:25.751 DEBUG:teuthology.orchestra.run.gibba031:> sudo update-grub 2021-04-03T03:00:26.989 INFO:teuthology.orchestra.run.gibba031.stderr:Sourcing file `/etc/default/grub' 2021-04-03T03:00:27.022 INFO:teuthology.orchestra.run.gibba031.stderr:Generating grub configuration file ... 2021-04-03T03:00:27.245 INFO:teuthology.orchestra.run.gibba031.stderr:Found linux image: /boot/vmlinuz-4.15.0-140-generic 2021-04-03T03:00:27.259 INFO:teuthology.orchestra.run.gibba031.stderr:Found initrd image: /boot/initrd.img-4.15.0-140-generic 2021-04-03T03:00:27.607 INFO:teuthology.orchestra.run.gibba031.stderr:Found linux image: /boot/vmlinuz-4.15.0-122-generic 2021-04-03T03:00:27.621 INFO:teuthology.orchestra.run.gibba031.stderr:Found initrd image: /boot/initrd.img-4.15.0-122-generic 2021-04-03T03:00:28.744 INFO:teuthology.orchestra.run.gibba031.stderr:done 2021-04-03T03:00:28.747 DEBUG:teuthology.orchestra.run.gibba031:> sudo shutdown -r now 2021-04-03T03:00:58.752 INFO:teuthology.misc:Re-opening connections... 2021-04-03T03:00:58.754 INFO:teuthology.misc:trying to connect to ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T03:00:58.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:00:58.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba011.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:01:17.275 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.111 2021-04-03T03:01:17.276 INFO:teuthology.misc:trying to connect to ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T03:01:17.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:01:17.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba015.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:17.309 DEBUG:teuthology.orchestra.remote:timed out 2021-04-03T03:02:17.318 INFO:teuthology.misc:trying to connect to ubuntu@gibba026.front.sepia.ceph.com 2021-04-03T03:02:17.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:02:17.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:17.438 DEBUG:teuthology.orchestra.run.gibba026:> true 2021-04-03T03:02:17.801 INFO:teuthology.misc:trying to connect to ubuntu@gibba031.front.sepia.ceph.com 2021-04-03T03:02:17.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:02:17.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:20.827 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.131 2021-04-03T03:02:20.828 DEBUG:teuthology.misc:waited 82.07381629943848 2021-04-03T03:02:21.829 INFO:teuthology.misc:trying to connect to ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T03:02:21.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:02:21.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba011.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:21.939 DEBUG:teuthology.orchestra.run.gibba011:> true 2021-04-03T03:02:22.181 INFO:teuthology.misc:trying to connect to ubuntu@gibba028.front.sepia.ceph.com 2021-04-03T03:02:22.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:02:22.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:22.297 DEBUG:teuthology.orchestra.run.gibba028:> true 2021-04-03T03:02:22.654 DEBUG:teuthology.misc:waited 83.9001533985138 2021-04-03T03:02:23.655 INFO:teuthology.misc:trying to connect to ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T03:02:23.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:02:23.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba015.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:23.768 DEBUG:teuthology.orchestra.run.gibba015:> true 2021-04-03T03:02:24.025 DEBUG:teuthology.misc:waited 85.27134132385254 2021-04-03T03:02:25.026 INFO:teuthology.misc:trying to connect to ubuntu@gibba031.front.sepia.ceph.com 2021-04-03T03:02:25.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:02:25.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:25.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.131 2021-04-03T03:02:25.029 DEBUG:teuthology.misc:waited 86.27517104148865 2021-04-03T03:02:26.030 INFO:teuthology.misc:trying to connect to ubuntu@gibba031.front.sepia.ceph.com 2021-04-03T03:02:26.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:02:26.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:02:26.230 DEBUG:teuthology.orchestra.run.gibba031:> true 2021-04-03T03:02:26.994 DEBUG:teuthology.misc:waited 88.23972630500793 2021-04-03T03:02:27.995 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2021-04-03T03:02:27.996 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.15.0-140-generic"... 2021-04-03T03:02:27.996 DEBUG:teuthology.orchestra.run.gibba011:> uname -r 2021-04-03T03:02:28.005 INFO:teuthology.orchestra.run.gibba011.stdout:4.15.0-140-generic 2021-04-03T03:02:28.005 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-140-generic vs 4.15.0-140-generic 2021-04-03T03:02:28.006 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-04-03T03:02:28.006 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-04-03T03:02:28.006 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2021-04-03T03:02:28.007 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.15.0-140-generic"... 2021-04-03T03:02:28.007 DEBUG:teuthology.orchestra.run.gibba015:> uname -r 2021-04-03T03:02:28.015 INFO:teuthology.orchestra.run.gibba015.stdout:4.15.0-140-generic 2021-04-03T03:02:28.016 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-140-generic vs 4.15.0-140-generic 2021-04-03T03:02:28.016 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-04-03T03:02:28.016 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-04-03T03:02:28.016 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2021-04-03T03:02:28.017 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.15.0-140-generic"... 2021-04-03T03:02:28.017 DEBUG:teuthology.orchestra.run.gibba026:> uname -r 2021-04-03T03:02:28.025 INFO:teuthology.orchestra.run.gibba026.stdout:4.15.0-140-generic 2021-04-03T03:02:28.026 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-140-generic vs 4.15.0-140-generic 2021-04-03T03:02:28.026 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-04-03T03:02:28.026 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-04-03T03:02:28.026 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2021-04-03T03:02:28.027 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.15.0-140-generic"... 2021-04-03T03:02:28.027 DEBUG:teuthology.orchestra.run.gibba028:> uname -r 2021-04-03T03:02:28.035 INFO:teuthology.orchestra.run.gibba028.stdout:4.15.0-140-generic 2021-04-03T03:02:28.036 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-140-generic vs 4.15.0-140-generic 2021-04-03T03:02:28.036 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-04-03T03:02:28.036 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-04-03T03:02:28.036 INFO:teuthology.task.kernel:Checking client client.2 for new kernel version... 2021-04-03T03:02:28.037 INFO:teuthology.task.kernel:Checking kernel version of client.2, want "4.15.0-140-generic"... 2021-04-03T03:02:28.037 DEBUG:teuthology.orchestra.run.gibba031:> uname -r 2021-04-03T03:02:28.045 INFO:teuthology.orchestra.run.gibba031.stdout:4.15.0-140-generic 2021-04-03T03:02:28.046 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-140-generic vs 4.15.0-140-generic 2021-04-03T03:02:28.046 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-04-03T03:02:28.046 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-04-03T03:02:29.047 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2021-04-03T03:02:29.048 DEBUG:teuthology.orchestra.run.gibba011:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-04-03T03:02:29.075 INFO:teuthology.orchestra.run.gibba011.stdout:ttyS1 2021-04-03T03:02:29.091 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2021-04-03T03:02:29.092 DEBUG:teuthology.orchestra.run.gibba015:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-04-03T03:02:29.132 INFO:teuthology.orchestra.run.gibba015.stdout:ttyS1 2021-04-03T03:02:29.152 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2021-04-03T03:02:29.153 DEBUG:teuthology.orchestra.run.gibba026:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-04-03T03:02:29.181 INFO:teuthology.orchestra.run.gibba026.stdout:ttyS1 2021-04-03T03:02:29.199 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2021-04-03T03:02:29.200 DEBUG:teuthology.orchestra.run.gibba028:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-04-03T03:02:29.232 INFO:teuthology.orchestra.run.gibba028.stdout:ttyS1 2021-04-03T03:02:29.262 INFO:teuthology.task.kernel:Enabling kdb on client.2... 2021-04-03T03:02:29.262 DEBUG:teuthology.orchestra.run.gibba031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-04-03T03:02:29.296 INFO:teuthology.orchestra.run.gibba031.stdout:ttyS1 2021-04-03T03:02:29.334 INFO:teuthology.run_tasks:Running task internal.base... 2021-04-03T03:02:29.342 INFO:teuthology.task.internal:Creating test directory... 2021-04-03T03:02:29.343 DEBUG:teuthology.orchestra.run.gibba011:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-04-03T03:02:29.346 DEBUG:teuthology.orchestra.run.gibba015:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-04-03T03:02:29.347 DEBUG:teuthology.orchestra.run.gibba026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-04-03T03:02:29.351 DEBUG:teuthology.orchestra.run.gibba028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-04-03T03:02:29.353 DEBUG:teuthology.orchestra.run.gibba031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-04-03T03:02:29.384 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-04-03T03:02:29.442 INFO:teuthology.run_tasks:Running task internal.archive... 2021-04-03T03:02:29.541 INFO:teuthology.task.internal:Creating archive directory... 2021-04-03T03:02:29.542 DEBUG:teuthology.orchestra.run.gibba011:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-04-03T03:02:29.545 DEBUG:teuthology.orchestra.run.gibba015:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-04-03T03:02:29.547 DEBUG:teuthology.orchestra.run.gibba026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-04-03T03:02:29.550 DEBUG:teuthology.orchestra.run.gibba028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-04-03T03:02:29.552 DEBUG:teuthology.orchestra.run.gibba031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-04-03T03:02:29.573 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-04-03T03:02:29.670 INFO:teuthology.task.internal:Enabling coredump saving... 2021-04-03T03:02:29.671 DEBUG:teuthology.orchestra.run.gibba011:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-04-03T03:02:29.675 DEBUG:teuthology.orchestra.run.gibba015:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-04-03T03:02:29.676 DEBUG:teuthology.orchestra.run.gibba026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-04-03T03:02:29.679 DEBUG:teuthology.orchestra.run.gibba028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-04-03T03:02:29.681 DEBUG:teuthology.orchestra.run.gibba031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-04-03T03:02:29.685 INFO:teuthology.orchestra.run.gibba015.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.688 INFO:teuthology.orchestra.run.gibba015.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.690 INFO:teuthology.orchestra.run.gibba011.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.690 INFO:teuthology.orchestra.run.gibba028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.696 INFO:teuthology.orchestra.run.gibba011.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.696 INFO:teuthology.orchestra.run.gibba028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.701 INFO:teuthology.orchestra.run.gibba031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.702 INFO:teuthology.orchestra.run.gibba026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.704 INFO:teuthology.orchestra.run.gibba031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.706 INFO:teuthology.orchestra.run.gibba026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-04-03T03:02:29.706 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-04-03T03:02:29.782 INFO:teuthology.task.internal:Configuring sudo... 2021-04-03T03:02:29.783 DEBUG:teuthology.orchestra.run.gibba011:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-04-03T03:02:29.786 DEBUG:teuthology.orchestra.run.gibba015:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-04-03T03:02:29.788 DEBUG:teuthology.orchestra.run.gibba026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-04-03T03:02:29.791 DEBUG:teuthology.orchestra.run.gibba028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-04-03T03:02:29.793 DEBUG:teuthology.orchestra.run.gibba031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-04-03T03:02:29.804 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-04-03T03:02:29.905 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-04-03T03:02:29.906 DEBUG:teuthology.orchestra.run.gibba011:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-04-03T03:02:29.909 DEBUG:teuthology.orchestra.run.gibba015:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-04-03T03:02:29.910 DEBUG:teuthology.orchestra.run.gibba026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-04-03T03:02:29.914 DEBUG:teuthology.orchestra.run.gibba028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-04-03T03:02:29.916 DEBUG:teuthology.orchestra.run.gibba031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-04-03T03:02:29.925 DEBUG:teuthology.orchestra.run.gibba011:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-04-03T03:02:29.966 DEBUG:teuthology.orchestra.run.gibba011:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-04-03T03:02:30.014 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:02:30.015 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-04-03T03:02:30.082 DEBUG:teuthology.orchestra.run.gibba015:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-04-03T03:02:30.085 DEBUG:teuthology.orchestra.run.gibba015:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-04-03T03:02:30.130 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:02:30.130 DEBUG:teuthology.orchestra.run.gibba015:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-04-03T03:02:30.185 DEBUG:teuthology.orchestra.run.gibba026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-04-03T03:02:30.194 DEBUG:teuthology.orchestra.run.gibba026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-04-03T03:02:30.241 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:02:30.242 DEBUG:teuthology.orchestra.run.gibba026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-04-03T03:02:30.305 DEBUG:teuthology.orchestra.run.gibba028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-04-03T03:02:30.314 DEBUG:teuthology.orchestra.run.gibba028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-04-03T03:02:30.359 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:02:30.360 DEBUG:teuthology.orchestra.run.gibba028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-04-03T03:02:30.414 DEBUG:teuthology.orchestra.run.gibba031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-04-03T03:02:30.423 DEBUG:teuthology.orchestra.run.gibba031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-04-03T03:02:30.467 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:02:30.468 DEBUG:teuthology.orchestra.run.gibba031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-04-03T03:02:30.539 DEBUG:teuthology.orchestra.run.gibba011:> sudo service rsyslog restart 2021-04-03T03:02:30.543 DEBUG:teuthology.orchestra.run.gibba015:> sudo service rsyslog restart 2021-04-03T03:02:30.546 DEBUG:teuthology.orchestra.run.gibba026:> sudo service rsyslog restart 2021-04-03T03:02:30.548 DEBUG:teuthology.orchestra.run.gibba028:> sudo service rsyslog restart 2021-04-03T03:02:30.551 DEBUG:teuthology.orchestra.run.gibba031:> sudo service rsyslog restart 2021-04-03T03:02:30.694 INFO:teuthology.run_tasks:Running task internal.timer... 2021-04-03T03:02:30.703 INFO:teuthology.task.internal:Starting timer... 2021-04-03T03:02:30.703 INFO:teuthology.run_tasks:Running task pcp... 2021-04-03T03:02:30.782 INFO:teuthology.run_tasks:Running task selinux... 2021-04-03T03:02:30.895 DEBUG:teuthology.task.selinux:Excluding gibba011: OS 'ubuntu' does not support SELinux 2021-04-03T03:02:30.896 DEBUG:teuthology.task.selinux:Excluding gibba015: OS 'ubuntu' does not support SELinux 2021-04-03T03:02:30.896 DEBUG:teuthology.task.selinux:Excluding gibba026: OS 'ubuntu' does not support SELinux 2021-04-03T03:02:30.896 DEBUG:teuthology.task.selinux:Excluding gibba028: OS 'ubuntu' does not support SELinux 2021-04-03T03:02:30.897 DEBUG:teuthology.task.selinux:Excluding gibba031: OS 'ubuntu' does not support SELinux 2021-04-03T03:02:30.897 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-04-03T03:02:30.897 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-04-03T03:02:30.897 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-04-03T03:02:30.898 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-04-03T03:02:31.060 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2021-04-03T03:02:31.079 INFO:teuthology.repo_utils:Fetching master from origin 2021-04-03T03:02:31.172 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-04-03T03:02:31.326 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-04-03T03:02:31.338 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit gibba011.front.sepia.ceph.com,gibba015.front.sepia.ceph.com,gibba026.front.sepia.ceph.com,gibba028.front.sepia.ceph.com,gibba031.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-04-03T03:02:32.106 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-04-03T03:02:33.670 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-04-03T03:02:35.872 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-04-03T03:02:35.918 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:35 +0000 (0:00:00.072) 0:00:00.072 ******** 2021-04-03T03:02:35.930 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:35 +0000 (0:00:00.011) 0:00:00.084 ******** 2021-04-03T03:02:35.940 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:35 +0000 (0:00:00.010) 0:00:00.094 ******** 2021-04-03T03:02:35.951 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:35 +0000 (0:00:00.011) 0:00:00.105 ******** 2021-04-03T03:02:35.963 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:35 +0000 (0:00:00.011) 0:00:00.117 ******** 2021-04-03T03:02:37.200 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-04-03T03:02:37.202 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host gibba031.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba031.front.sepia.ceph.com] 2021-04-03T03:02:37.236 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host gibba015.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba015.front.sepia.ceph.com] 2021-04-03T03:02:37.246 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host gibba011.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba011.front.sepia.ceph.com] 2021-04-03T03:02:37.297 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:01.333) 0:00:01.451 ******** 2021-04-03T03:02:37.327 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.029) 0:00:01.481 ******** 2021-04-03T03:02:37.349 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.022) 0:00:01.503 ******** 2021-04-03T03:02:37.372 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host gibba028.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba028.front.sepia.ceph.com] 2021-04-03T03:02:37.424 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.074) 0:00:01.578 ******** 2021-04-03T03:02:37.455 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host gibba026.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. 2021-04-03T03:02:37.457 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] 2021-04-03T03:02:37.507 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.083) 0:00:01.661 ******** 2021-04-03T03:02:37.525 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-04-03T03:02:37.527 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-04-03T03:02:37.550 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.042) 0:00:01.704 ******** 2021-04-03T03:02:37.574 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.023) 0:00:01.728 ******** 2021-04-03T03:02:37.595 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.021) 0:00:01.749 ******** 2021-04-03T03:02:37.603 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-04-03T03:02:37.629 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.033) 0:00:01.783 ******** 2021-04-03T03:02:37.636 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-04-03T03:02:37.661 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:37 +0000 (0:00:00.032) 0:00:01.815 ******** 2021-04-03T03:02:38.159 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-04-03T03:02:38.161 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-04-03T03:02:38.184 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.522) 0:00:02.338 ******** 2021-04-03T03:02:38.191 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-04-03T03:02:38.192 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-04-03T03:02:38.217 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.032) 0:00:02.371 ******** 2021-04-03T03:02:38.237 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.019) 0:00:02.391 ******** 2021-04-03T03:02:38.243 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-04-03T03:02:38.269 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.032) 0:00:02.423 ******** 2021-04-03T03:02:38.317 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-04-03T03:02:38.341 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.072) 0:00:02.495 ******** 2021-04-03T03:02:38.577 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-04-03T03:02:38.579 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.048910", "end": "2021-04-03 03:02:38.386923", "rc": 0, "start": "2021-04-03 03:02:38.338013", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.049520", "end": "2021-04-03 03:02:38.914280", "rc": 0, "start": "2021-04-03 03:02:38.864760", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-04-03T03:02:38.604 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.263) 0:00:02.759 ******** 2021-04-03T03:02:38.625 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.020) 0:00:02.779 ******** 2021-04-03T03:02:38.632 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.055504", "end": "2021-04-03 03:02:38.796303", "rc": 0, "start": "2021-04-03 03:02:38.740799", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-04-03T03:02:38.633 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.056780", "end": "2021-04-03 03:02:38.574378", "rc": 0, "start": "2021-04-03 03:02:38.517598", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-04-03T03:02:38.634 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.067217", "end": "2021-04-03 03:02:38.883609", "rc": 0, "start": "2021-04-03 03:02:38.816392", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-04-03T03:02:38.660 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.034) 0:00:02.814 ******** 2021-04-03T03:02:38.681 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.020) 0:00:02.835 ******** 2021-04-03T03:02:38.702 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.021) 0:00:02.856 ******** 2021-04-03T03:02:38.906 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-04-03T03:02:38.907 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [gibba031.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:38.908 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:38.931 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.229) 0:00:03.085 ******** 2021-04-03T03:02:38.950 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.018) 0:00:03.104 ******** 2021-04-03T03:02:38.969 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.019) 0:00:03.123 ******** 2021-04-03T03:02:38.975 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:38.976 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:39.001 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:38 +0000 (0:00:00.031) 0:00:03.155 ******** 2021-04-03T03:02:39.020 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:39 +0000 (0:00:00.019) 0:00:03.174 ******** 2021-04-03T03:02:39.727 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-04-03T03:02:39.729 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617418959.0348952-19436-162978293968277/source", "state": "file", "uid": 0} changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617418959.0014343-19433-188328347033298/source", "state": "file", "uid": 0} 2021-04-03T03:02:39.750 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:39 +0000 (0:00:00.730) 0:00:03.905 ******** 2021-04-03T03:02:39.770 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:39 +0000 (0:00:00.019) 0:00:03.924 ******** 2021-04-03T03:02:39.778 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617418958.975304-19430-18247937653247/source", "state": "file", "uid": 0} 2021-04-03T03:02:39.779 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617418958.9585667-19428-230197958394714/source", "state": "file", "uid": 0} 2021-04-03T03:02:39.780 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617418959.0552394-19439-269446458147955/source", "state": "file", "uid": 0} 2021-04-03T03:02:39.805 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:39 +0000 (0:00:00.034) 0:00:03.959 ******** 2021-04-03T03:02:39.824 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:39 +0000 (0:00:00.019) 0:00:03.978 ******** 2021-04-03T03:02:39.844 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:39 +0000 (0:00:00.019) 0:00:03.998 ******** 2021-04-03T03:02:40.132 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-04-03T03:02:40.134 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-04-03T03:02:40.136 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.c 2021-04-03T03:02:40.137 INFO:teuthology.task.ansible.out:om\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ak 2021-04-03T03:02:40.139 INFO:teuthology.task.ansible.out:lap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-04-03T03:02:40.169 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-04-03T03:02:40.171 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.c 2021-04-03T03:02:40.173 INFO:teuthology.task.ansible.out:om\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ak 2021-04-03T03:02:40.174 INFO:teuthology.task.ansible.out:lap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKL 2021-04-03T03:02:40.176 INFO:teuthology.task.ansible.out:wtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-04-03T03:02:40.296 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-04-03T03:02:40.299 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-04-03T03:02:40.439 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:40 +0000 (0:00:00.595) 0:00:04.593 ******** 2021-04-03T03:02:40.459 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:40 +0000 (0:00:00.019) 0:00:04.613 ******** 2021-04-03T03:02:40.478 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:40 +0000 (0:00:00.019) 0:00:04.632 ******** 2021-04-03T03:02:40.497 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:40 +0000 (0:00:00.019) 0:00:04.651 ******** 2021-04-03T03:02:40.524 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:40 +0000 (0:00:00.026) 0:00:04.678 ******** 2021-04-03T03:02:40.884 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba011.front.sepia.ceph.com] 2021-04-03T03:02:40.918 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] 2021-04-03T03:02:41.047 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-04-03T03:02:41.049 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.523) 0:00:05.201 ******** 2021-04-03T03:02:41.071 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] 2021-04-03T03:02:41.082 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] 2021-04-03T03:02:41.092 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] 2021-04-03T03:02:41.123 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.075) 0:00:05.277 ******** 2021-04-03T03:02:41.192 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.069) 0:00:05.346 ******** 2021-04-03T03:02:41.214 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.021) 0:00:05.368 ******** 2021-04-03T03:02:41.237 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.022) 0:00:05.391 ******** 2021-04-03T03:02:41.609 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-04-03T03:02:41.611 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.178996", "end": "2021-04-03 03:02:41.587032", "rc": 0, "start": "2021-04-03 03:02:41.408036", "stderr": "", "stderr_lines": [], "stdout": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "stdout_lines": ["5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232"]} changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.204825", "end": "2021-04-03 03:02:41.590190", "rc": 0, "start": "2021-04-03 03:02:41.385365", "stderr": "", "stderr_lines": [], "stdout": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "stdout_lines": ["5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232"]} 2021-04-03T03:02:41.612 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.263629", "end": "2021-04-03 03:02:41.587041", "rc": 0, "start": "2021-04-03 03:02:41.323412", "stderr": "", "stderr_lines": [], "stdout": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "stdout_lines": ["5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232"]} 2021-04-03T03:02:41.613 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.152781", "end": "2021-04-03 03:02:41.588199", "rc": 0, "start": "2021-04-03 03:02:41.435418", "stderr": "", "stderr_lines": [], "stdout": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "stdout_lines": ["5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232"]} 2021-04-03T03:02:41.634 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.396) 0:00:05.788 ******** 2021-04-03T03:02:41.653 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.018) 0:00:05.807 ******** 2021-04-03T03:02:41.674 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.020) 0:00:05.828 ******** 2021-04-03T03:02:41.693 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.019) 0:00:05.847 ******** 2021-04-03T03:02:41.699 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.320492", "end": "2021-04-03 03:02:41.608507", "rc": 0, "start": "2021-04-03 03:02:41.288015", "stderr": "", "stderr_lines": [], "stdout": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "stdout_lines": ["5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232"]} 2021-04-03T03:02:41.725 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.031) 0:00:05.879 ******** 2021-04-03T03:02:41.731 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-04-03T03:02:41.732 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.009264", "end": "2021-04-03 03:02:41.941347", "failed_when_result": false, "rc": 0, "start": "2021-04-03 03:02:41.932083", "stderr": "", "stderr_lines": [], "stdout": "e549618f811c1ef59d08997aa55e495f75d8d5e6", "stdout_lines": ["e549618f811c1ef59d08997aa55e495f75d8d5e6"]} 2021-04-03T03:02:41.758 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.033) 0:00:05.912 ******** 2021-04-03T03:02:41.764 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008845", "end": "2021-04-03 03:02:41.738187", "failed_when_result": false, "rc": 0, "start": "2021-04-03 03:02:41.729342", "stderr": "", "stderr_lines": [], "stdout": "e549618f811c1ef59d08997aa55e495f75d8d5e6", "stdout_lines": ["e549618f811c1ef59d08997aa55e495f75d8d5e6"]} 2021-04-03T03:02:41.791 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.033) 0:00:05.945 ******** 2021-04-03T03:02:41.798 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.009410", "end": "2021-04-03 03:02:41.586148", "failed_when_result": false, "rc": 0, "start": "2021-04-03 03:02:41.576738", "stderr": "", "stderr_lines": [], "stdout": "e549618f811c1ef59d08997aa55e495f75d8d5e6", "stdout_lines": ["e549618f811c1ef59d08997aa55e495f75d8d5e6"]} 2021-04-03T03:02:41.799 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.010719", "end": "2021-04-03 03:02:42.117104", "failed_when_result": false, "rc": 0, "start": "2021-04-03 03:02:42.106385", "stderr": "", "stderr_lines": [], "stdout": "e549618f811c1ef59d08997aa55e495f75d8d5e6", "stdout_lines": ["e549618f811c1ef59d08997aa55e495f75d8d5e6"]} 2021-04-03T03:02:41.826 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.035) 0:00:05.980 ******** 2021-04-03T03:02:41.847 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.020) 0:00:06.001 ******** 2021-04-03T03:02:41.855 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008156", "end": "2021-04-03 03:02:42.094899", "failed_when_result": false, "rc": 0, "start": "2021-04-03 03:02:42.086743", "stderr": "", "stderr_lines": [], "stdout": "e549618f811c1ef59d08997aa55e495f75d8d5e6", "stdout_lines": ["e549618f811c1ef59d08997aa55e495f75d8d5e6"]} 2021-04-03T03:02:41.856 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-04-03T03:02:41.856 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:41.857 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:41.881 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.033) 0:00:06.035 ******** 2021-04-03T03:02:41.903 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.021) 0:00:06.057 ******** 2021-04-03T03:02:41.924 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.021) 0:00:06.078 ******** 2021-04-03T03:02:41.930 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:41.932 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:41.957 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.032) 0:00:06.111 ******** 2021-04-03T03:02:41.977 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:41 +0000 (0:00:00.019) 0:00:06.131 ******** 2021-04-03T03:02:41.982 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:41.984 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.007 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.030) 0:00:06.161 ******** 2021-04-03T03:02:42.028 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.020) 0:00:06.182 ******** 2021-04-03T03:02:42.034 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.036 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.059 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.031) 0:00:06.214 ******** 2021-04-03T03:02:42.080 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.019) 0:00:06.234 ******** 2021-04-03T03:02:42.085 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.087 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.110 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.030) 0:00:06.264 ******** 2021-04-03T03:02:42.137 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.026) 0:00:06.291 ******** 2021-04-03T03:02:42.144 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-04-03T03:02:42.146 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.170 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.032) 0:00:06.324 ******** 2021-04-03T03:02:42.191 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.021) 0:00:06.345 ******** 2021-04-03T03:02:42.198 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.200 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.224 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.032) 0:00:06.378 ******** 2021-04-03T03:02:42.253 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.028) 0:00:06.407 ******** 2021-04-03T03:02:42.258 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.260 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.284 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.031) 0:00:06.438 ******** 2021-04-03T03:02:42.306 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.021) 0:00:06.460 ******** 2021-04-03T03:02:42.332 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.026) 0:00:06.486 ******** 2021-04-03T03:02:42.342 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.343 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.371 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.035) 0:00:06.522 ******** 2021-04-03T03:02:42.390 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.022) 0:00:06.544 ******** 2021-04-03T03:02:42.397 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-04-03T03:02:42.398 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.422 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.031) 0:00:06.576 ******** 2021-04-03T03:02:42.442 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.020) 0:00:06.596 ******** 2021-04-03T03:02:42.449 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-04-03T03:02:42.450 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [Filter the managed_admin_users list] ************************************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.477 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.034) 0:00:06.631 ******** 2021-04-03T03:02:42.498 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.021) 0:00:06.652 ******** 2021-04-03T03:02:42.507 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.536 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.037) 0:00:06.690 ******** 2021-04-03T03:02:42.555 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-04-03T03:02:42.557 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [Filter the managed_admin_users list] ************************************* 2021-04-03T03:02:42.557 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:02:42.584 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.047) 0:00:06.738 ******** 2021-04-03T03:02:42.614 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.029) 0:00:06.768 ******** 2021-04-03T03:02:42.623 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-04-03T03:02:42.624 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:42.626 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:42.652 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.037) 0:00:06.806 ******** 2021-04-03T03:02:42.674 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.022) 0:00:06.828 ******** 2021-04-03T03:02:42.681 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:42.707 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.032) 0:00:06.861 ******** 2021-04-03T03:02:42.745 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:02:42.769 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:42 +0000 (0:00:00.061) 0:00:06.923 ******** 2021-04-03T03:02:42.775 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-04-03T03:02:42.776 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-04-03T03:02:42.830 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-04-03T03:02:42.863 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-04-03T03:02:42.876 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-04-03T03:02:42.888 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-04-03T03:02:42.920 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-04-03T03:02:42.943 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-04-03T03:02:42.954 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-04-03T03:02:42.966 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-04-03T03:02:42.978 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-04-03T03:02:43.011 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-04-03T03:02:43.043 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-04-03T03:02:43.044 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-04-03T03:02:43.056 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-04-03T03:02:43.067 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-04-03T03:02:43.111 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-04-03T03:02:43.133 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-04-03T03:02:43.144 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-04-03T03:02:43.156 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-04-03T03:02:43.157 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-04-03T03:02:43.211 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-04-03T03:02:43.223 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-04-03T03:02:43.256 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-04-03T03:02:43.257 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-04-03T03:02:43.258 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-04-03T03:02:43.301 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-04-03T03:02:43.323 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-04-03T03:02:43.345 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-04-03T03:02:43.346 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-04-03T03:02:43.358 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-04-03T03:02:43.390 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-04-03T03:02:43.412 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-04-03T03:02:43.435 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-04-03T03:02:43.447 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-04-03T03:02:43.448 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-04-03T03:02:43.491 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-04-03T03:02:43.503 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-04-03T03:02:43.525 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-04-03T03:02:43.537 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-04-03T03:02:43.538 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-04-03T03:02:43.591 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-04-03T03:02:43.603 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-04-03T03:02:43.614 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-04-03T03:02:43.626 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-04-03T03:02:43.638 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-04-03T03:02:43.681 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-04-03T03:02:43.692 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-04-03T03:02:43.704 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-04-03T03:02:43.716 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-04-03T03:02:43.727 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-04-03T03:02:43.781 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-04-03T03:02:43.803 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-04-03T03:02:43.808 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-04-03T03:02:43.810 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-04-03T03:02:43.820 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-04-03T03:02:43.874 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-04-03T03:02:43.896 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-04-03T03:02:43.897 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-04-03T03:02:43.908 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-04-03T03:02:43.920 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-04-03T03:02:43.974 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-04-03T03:02:43.996 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-04-03T03:02:44.001 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-04-03T03:02:44.013 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-04-03T03:02:44.014 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-04-03T03:02:44.067 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-04-03T03:02:44.079 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-04-03T03:02:44.101 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-04-03T03:02:44.113 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-04-03T03:02:44.114 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-04-03T03:02:44.167 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-04-03T03:02:44.168 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-04-03T03:02:44.191 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-04-03T03:02:44.202 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-04-03T03:02:44.214 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-04-03T03:02:44.257 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-04-03T03:02:44.268 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-04-03T03:02:44.291 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-04-03T03:02:44.302 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-04-03T03:02:44.303 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-04-03T03:02:44.357 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-04-03T03:02:44.369 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-04-03T03:02:44.391 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-04-03T03:02:44.402 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-04-03T03:02:44.403 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-04-03T03:02:44.457 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-04-03T03:02:44.469 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-04-03T03:02:44.491 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-04-03T03:02:44.492 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-04-03T03:02:44.493 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-04-03T03:02:44.536 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-04-03T03:02:44.569 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-04-03T03:02:44.580 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-04-03T03:02:44.592 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-04-03T03:02:44.593 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-04-03T03:02:44.636 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-04-03T03:02:44.668 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-04-03T03:02:44.680 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-04-03T03:02:44.681 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-04-03T03:02:44.683 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-04-03T03:02:44.725 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-04-03T03:02:44.768 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-04-03T03:02:44.769 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-04-03T03:02:44.781 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-04-03T03:02:44.792 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-04-03T03:02:44.814 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-04-03T03:02:44.857 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-04-03T03:02:44.870 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-04-03T03:02:44.872 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-04-03T03:02:44.895 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-04-03T03:02:44.906 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-04-03T03:02:44.949 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-04-03T03:02:44.961 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-04-03T03:02:44.972 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-04-03T03:02:44.994 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-04-03T03:02:45.006 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-04-03T03:02:45.049 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-04-03T03:02:45.050 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-04-03T03:02:45.072 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-04-03T03:02:45.084 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-04-03T03:02:45.095 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-04-03T03:02:45.138 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-04-03T03:02:45.149 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-04-03T03:02:45.161 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-04-03T03:02:45.183 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-04-03T03:02:45.195 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-04-03T03:02:45.228 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-04-03T03:02:45.239 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-04-03T03:02:45.251 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-04-03T03:02:45.283 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-04-03T03:02:45.284 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-04-03T03:02:45.317 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-04-03T03:02:45.340 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-04-03T03:02:45.341 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-04-03T03:02:45.363 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-04-03T03:02:45.375 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-04-03T03:02:45.418 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-04-03T03:02:45.429 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-04-03T03:02:45.441 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-04-03T03:02:45.473 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-04-03T03:02:45.474 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-04-03T03:02:45.506 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-04-03T03:02:45.528 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-04-03T03:02:45.551 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-04-03T03:02:45.563 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-04-03T03:02:45.565 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-04-03T03:02:45.608 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-04-03T03:02:45.630 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-04-03T03:02:45.641 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-04-03T03:02:45.653 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-04-03T03:02:45.654 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-04-03T03:02:45.697 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-04-03T03:02:45.730 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-04-03T03:02:45.741 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-04-03T03:02:45.752 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-04-03T03:02:45.753 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-04-03T03:02:45.786 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-04-03T03:02:45.819 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-04-03T03:02:45.841 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-04-03T03:02:45.843 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-04-03T03:02:45.854 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-04-03T03:02:45.865 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-04-03T03:02:45.919 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-04-03T03:02:45.930 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-04-03T03:02:45.942 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-04-03T03:02:45.943 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-04-03T03:02:45.954 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-04-03T03:02:46.008 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-04-03T03:02:46.020 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-04-03T03:02:46.033 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-04-03T03:02:46.035 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-04-03T03:02:46.046 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-04-03T03:02:46.099 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-04-03T03:02:46.111 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-04-03T03:02:46.123 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-04-03T03:02:46.124 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-04-03T03:02:46.135 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-04-03T03:02:46.199 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-04-03T03:02:46.211 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-04-03T03:02:46.223 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-04-03T03:02:46.225 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-04-03T03:02:46.227 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-04-03T03:02:46.301 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-04-03T03:02:46.302 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:46.325 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-04-03T03:02:46.326 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:46.328 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:46.401 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-04-03T03:02:46.402 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-04-03T03:02:46.414 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-04-03T03:02:46.415 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-04-03T03:02:46.417 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:46.491 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:46.492 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-04-03T03:02:46.515 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-04-03T03:02:46.516 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-04-03T03:02:46.518 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-04-03T03:02:46.583 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-04-03T03:02:46.584 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-04-03T03:02:46.608 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-04-03T03:02:46.610 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-04-03T03:02:46.611 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-04-03T03:02:46.675 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-04-03T03:02:46.687 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-04-03T03:02:46.698 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-04-03T03:02:46.700 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-04-03T03:02:46.711 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-04-03T03:02:46.775 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-04-03T03:02:46.776 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-04-03T03:02:46.798 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-04-03T03:02:46.799 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-04-03T03:02:46.811 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-04-03T03:02:46.864 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-04-03T03:02:46.875 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-04-03T03:02:46.897 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-04-03T03:02:46.898 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-04-03T03:02:46.900 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-04-03T03:02:46.953 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-04-03T03:02:46.975 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-04-03T03:02:46.986 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-04-03T03:02:46.998 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-04-03T03:02:46.999 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-04-03T03:02:47.042 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:47.064 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-04-03T03:02:47.077 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:47.088 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-04-03T03:02:47.100 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:47.143 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-04-03T03:02:47.165 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-04-03T03:02:47.166 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-04-03T03:02:47.188 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:47.200 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-04-03T03:02:47.232 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-04-03T03:02:47.254 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-04-03T03:02:47.255 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:47.287 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-04-03T03:02:47.288 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-04-03T03:02:47.321 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-04-03T03:02:47.343 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-04-03T03:02:47.355 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-04-03T03:02:47.378 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-04-03T03:02:47.389 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-04-03T03:02:47.421 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-04-03T03:02:47.444 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-04-03T03:02:47.455 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-04-03T03:02:47.479 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-04-03T03:02:47.480 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-04-03T03:02:47.513 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-04-03T03:02:47.545 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-04-03T03:02:47.546 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-04-03T03:02:47.568 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-04-03T03:02:47.580 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-04-03T03:02:47.592 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-04-03T03:02:47.635 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-04-03T03:02:47.646 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-04-03T03:02:47.669 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-04-03T03:02:47.680 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-04-03T03:02:47.681 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-04-03T03:02:47.735 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-04-03T03:02:47.737 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-04-03T03:02:47.758 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-04-03T03:02:47.780 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-04-03T03:02:47.781 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-04-03T03:02:47.824 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-04-03T03:02:47.838 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-04-03T03:02:47.850 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-04-03T03:02:47.872 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-04-03T03:02:47.884 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-04-03T03:02:47.927 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-04-03T03:02:47.928 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-04-03T03:02:47.950 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-04-03T03:02:47.962 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-04-03T03:02:47.973 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-04-03T03:02:48.027 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-04-03T03:02:48.028 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-04-03T03:02:48.051 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-04-03T03:02:48.052 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-04-03T03:02:48.063 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-04-03T03:02:48.117 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-04-03T03:02:48.118 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-04-03T03:02:48.140 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-04-03T03:02:48.152 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-04-03T03:02:48.164 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-04-03T03:02:48.207 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-04-03T03:02:48.218 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-04-03T03:02:48.231 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-04-03T03:02:48.242 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-04-03T03:02:48.264 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-04-03T03:02:48.308 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-04-03T03:02:48.309 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-04-03T03:02:48.332 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-04-03T03:02:48.334 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-04-03T03:02:48.354 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-04-03T03:02:48.397 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-04-03T03:02:48.408 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-04-03T03:02:48.420 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-04-03T03:02:48.431 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-04-03T03:02:48.443 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-04-03T03:02:48.506 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-04-03T03:02:48.518 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-04-03T03:02:48.519 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-04-03T03:02:48.521 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-04-03T03:02:48.532 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-04-03T03:02:48.606 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:48.607 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-04-03T03:02:48.609 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-04-03T03:02:48.611 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-04-03T03:02:48.643 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-04-03T03:02:48.707 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:48.708 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-04-03T03:02:48.709 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:48.711 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-04-03T03:02:48.732 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-04-03T03:02:48.796 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-04-03T03:02:48.797 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-04-03T03:02:48.808 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-04-03T03:02:48.809 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-04-03T03:02:48.831 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:48.896 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-04-03T03:02:48.897 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-04-03T03:02:48.898 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-04-03T03:02:48.899 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-04-03T03:02:48.931 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-04-03T03:02:48.984 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-04-03T03:02:48.996 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-04-03T03:02:48.997 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-04-03T03:02:48.998 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-04-03T03:02:49.020 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-04-03T03:02:49.073 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-04-03T03:02:49.086 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-04-03T03:02:49.087 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-04-03T03:02:49.089 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-04-03T03:02:49.122 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-04-03T03:02:49.176 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-04-03T03:02:49.188 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-04-03T03:02:49.189 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-04-03T03:02:49.191 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-04-03T03:02:49.212 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-04-03T03:02:49.266 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-04-03T03:02:49.277 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-04-03T03:02:49.289 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-04-03T03:02:49.290 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-04-03T03:02:49.312 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-04-03T03:02:49.355 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-04-03T03:02:49.366 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-04-03T03:02:49.378 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-04-03T03:02:49.379 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-04-03T03:02:49.411 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-04-03T03:02:49.444 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-04-03T03:02:49.466 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-04-03T03:02:49.468 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-04-03T03:02:49.479 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-04-03T03:02:49.501 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-04-03T03:02:49.533 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-04-03T03:02:49.565 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-04-03T03:02:49.566 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-04-03T03:02:49.568 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-04-03T03:02:49.600 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-04-03T03:02:49.622 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-04-03T03:02:49.655 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-04-03T03:02:49.667 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-04-03T03:02:49.668 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-04-03T03:02:49.701 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-04-03T03:02:49.712 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-04-03T03:02:49.755 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-04-03T03:02:49.767 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-04-03T03:02:49.768 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-04-03T03:02:49.800 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-04-03T03:02:49.801 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-04-03T03:02:49.845 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-04-03T03:02:49.857 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-04-03T03:02:49.869 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-04-03T03:02:49.891 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-04-03T03:02:49.893 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-04-03T03:02:49.945 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-04-03T03:02:49.956 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-04-03T03:02:49.968 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-04-03T03:02:49.979 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-04-03T03:02:49.991 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-04-03T03:02:50.044 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-04-03T03:02:50.045 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-04-03T03:02:50.067 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-04-03T03:02:50.079 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-04-03T03:02:50.091 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-04-03T03:02:50.134 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-04-03T03:02:50.135 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-04-03T03:02:50.167 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-04-03T03:02:50.168 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-04-03T03:02:50.180 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-04-03T03:02:50.223 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-04-03T03:02:50.235 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-04-03T03:02:50.257 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-04-03T03:02:50.258 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-04-03T03:02:50.280 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-04-03T03:02:50.313 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-04-03T03:02:50.325 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-04-03T03:02:50.347 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-04-03T03:02:50.359 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-04-03T03:02:50.371 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-04-03T03:02:50.414 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-04-03T03:02:50.415 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-04-03T03:02:50.439 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-04-03T03:02:50.451 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-04-03T03:02:50.473 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-04-03T03:02:50.516 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-04-03T03:02:50.517 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-04-03T03:02:50.528 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-04-03T03:02:50.551 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-04-03T03:02:50.563 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-04-03T03:02:50.606 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-04-03T03:02:50.607 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-04-03T03:02:50.630 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-04-03T03:02:50.641 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-04-03T03:02:50.663 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-04-03T03:02:50.696 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-04-03T03:02:50.697 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-04-03T03:02:50.720 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-04-03T03:02:50.743 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-04-03T03:02:50.756 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-04-03T03:02:50.789 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-04-03T03:02:50.790 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-04-03T03:02:50.812 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-04-03T03:02:50.834 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-04-03T03:02:50.846 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-04-03T03:02:50.879 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-04-03T03:02:50.891 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-04-03T03:02:50.893 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-04-03T03:02:50.936 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-04-03T03:02:50.948 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-04-03T03:02:50.970 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-04-03T03:02:50.982 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-04-03T03:02:50.984 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-04-03T03:02:51.028 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-04-03T03:02:51.039 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-04-03T03:02:51.061 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-04-03T03:02:51.073 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-04-03T03:02:51.076 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-04-03T03:02:51.129 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-04-03T03:02:51.130 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-04-03T03:02:51.152 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-04-03T03:02:51.165 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-04-03T03:02:51.167 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-04-03T03:02:51.220 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-04-03T03:02:51.232 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-04-03T03:02:51.243 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-04-03T03:02:51.256 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-04-03T03:02:51.267 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-04-03T03:02:51.321 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-04-03T03:02:51.322 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-04-03T03:02:51.345 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-04-03T03:02:51.346 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-04-03T03:02:51.358 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-04-03T03:02:51.423 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-04-03T03:02:51.436 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-04-03T03:02:51.438 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-04-03T03:02:51.449 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-04-03T03:02:51.461 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-04-03T03:02:51.514 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-04-03T03:02:51.526 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-04-03T03:02:51.527 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-04-03T03:02:51.538 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-04-03T03:02:51.561 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-04-03T03:02:51.615 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-04-03T03:02:51.616 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-04-03T03:02:51.618 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-04-03T03:02:51.640 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-04-03T03:02:51.662 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-04-03T03:02:51.705 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-04-03T03:02:51.706 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-04-03T03:02:51.707 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-04-03T03:02:51.740 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-04-03T03:02:51.751 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-04-03T03:02:51.794 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-04-03T03:02:51.806 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-04-03T03:02:51.818 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-04-03T03:02:51.830 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-04-03T03:02:51.842 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-04-03T03:02:51.885 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-04-03T03:02:51.907 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-04-03T03:02:51.909 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-04-03T03:02:51.941 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-04-03T03:02:51.942 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-04-03T03:02:51.974 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-04-03T03:02:51.996 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-04-03T03:02:52.008 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-04-03T03:02:52.030 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-04-03T03:02:52.031 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-04-03T03:02:52.074 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:52.097 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-04-03T03:02:52.109 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-04-03T03:02:52.121 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-04-03T03:02:52.122 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-04-03T03:02:52.186 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-04-03T03:02:52.197 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-04-03T03:02:52.210 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-04-03T03:02:52.222 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-04-03T03:02:52.224 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-04-03T03:02:52.290 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-04-03T03:02:52.292 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-04-03T03:02:52.304 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-04-03T03:02:52.316 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:52.317 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:52.392 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-04-03T03:02:52.393 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-04-03T03:02:52.404 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-04-03T03:02:52.405 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-04-03T03:02:52.427 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-04-03T03:02:52.491 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-04-03T03:02:52.492 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:52.494 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-04-03T03:02:52.504 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-04-03T03:02:52.537 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-04-03T03:02:52.580 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-04-03T03:02:52.592 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-04-03T03:02:52.593 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-04-03T03:02:52.595 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-04-03T03:02:52.668 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-04-03T03:02:52.680 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-04-03T03:02:52.681 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-04-03T03:02:52.693 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-04-03T03:02:52.694 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-04-03T03:02:52.768 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-04-03T03:02:52.780 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-04-03T03:02:52.781 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-04-03T03:02:52.782 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-04-03T03:02:52.793 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-04-03T03:02:52.858 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-04-03T03:02:52.870 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-04-03T03:02:52.871 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-04-03T03:02:52.883 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-04-03T03:02:52.894 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-04-03T03:02:52.937 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-04-03T03:02:52.959 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-04-03T03:02:52.982 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-04-03T03:02:52.984 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-04-03T03:02:52.986 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-04-03T03:02:53.029 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-04-03T03:02:53.051 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-04-03T03:02:53.074 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-04-03T03:02:53.076 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-04-03T03:02:53.078 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-04-03T03:02:53.123 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-04-03T03:02:53.146 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-04-03T03:02:53.168 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-04-03T03:02:53.169 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-04-03T03:02:53.171 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-04-03T03:02:53.223 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-04-03T03:02:53.234 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-04-03T03:02:53.256 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-04-03T03:02:53.257 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-04-03T03:02:53.269 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-04-03T03:02:53.322 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-04-03T03:02:53.334 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-04-03T03:02:53.356 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-04-03T03:02:53.357 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-04-03T03:02:53.370 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-04-03T03:02:53.424 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-04-03T03:02:53.426 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-04-03T03:02:53.449 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-04-03T03:02:53.450 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-04-03T03:02:53.451 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-04-03T03:02:53.505 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-04-03T03:02:53.516 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-04-03T03:02:53.549 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-04-03T03:02:53.550 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-04-03T03:02:53.561 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-04-03T03:02:53.594 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-04-03T03:02:53.605 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-04-03T03:02:53.649 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-04-03T03:02:53.662 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-04-03T03:02:53.665 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-04-03T03:02:53.688 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-04-03T03:02:53.690 INFO:teuthology.task.ansible.out: 2021-04-03T03:02:53.732 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-04-03T03:02:53.744 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-04-03T03:02:53.755 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-04-03T03:02:53.757 INFO:teuthology.task.ansible.out: ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-04-03T03:02:53.759 INFO:teuthology.task.ansible.out: 2021-04-03T03:02:53.780 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-04-03T03:02:53.823 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-04-03T03:02:53.835 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-04-03T03:02:53.837 INFO:teuthology.task.ansible.out: 2021-04-03T03:02:53.847 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-04-03T03:02:53.858 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-04-03T03:02:53.870 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-04-03T03:02:53.924 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-04-03T03:02:53.937 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-04-03T03:02:53.938 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-04-03T03:02:53.949 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-04-03T03:02:53.971 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-04-03T03:02:54.014 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-04-03T03:02:54.026 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-04-03T03:02:54.037 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-04-03T03:02:54.049 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-04-03T03:02:54.060 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-04-03T03:02:54.113 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-04-03T03:02:54.114 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-04-03T03:02:54.126 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-04-03T03:02:54.148 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-04-03T03:02:54.149 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-04-03T03:02:54.203 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-04-03T03:02:54.214 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-04-03T03:02:54.225 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-04-03T03:02:54.237 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-04-03T03:02:54.238 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-04-03T03:02:54.302 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-04-03T03:02:54.303 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-04-03T03:02:54.326 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-04-03T03:02:54.328 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-04-03T03:02:54.339 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-04-03T03:02:54.382 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-04-03T03:02:54.404 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-04-03T03:02:54.416 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-04-03T03:02:54.417 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-04-03T03:02:54.429 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-04-03T03:02:54.493 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-04-03T03:02:54.494 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-04-03T03:02:54.506 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-04-03T03:02:54.518 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-04-03T03:02:54.519 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-04-03T03:02:54.583 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-04-03T03:02:54.592 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-04-03T03:02:54.596 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-04-03T03:02:54.607 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-04-03T03:02:54.620 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-04-03T03:02:54.673 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-04-03T03:02:54.685 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-04-03T03:02:54.686 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-04-03T03:02:54.709 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-04-03T03:02:54.722 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-04-03T03:02:54.775 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-04-03T03:02:54.776 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-04-03T03:02:54.778 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-04-03T03:02:54.809 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-04-03T03:02:54.810 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-04-03T03:02:54.864 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-04-03T03:02:54.865 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-04-03T03:02:54.876 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-04-03T03:02:54.909 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-04-03T03:02:54.910 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-04-03T03:02:54.963 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-04-03T03:02:54.965 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-04-03T03:02:54.975 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-04-03T03:02:54.997 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-04-03T03:02:54.998 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-04-03T03:02:55.051 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-04-03T03:02:55.063 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-04-03T03:02:55.074 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-04-03T03:02:55.096 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-04-03T03:02:55.108 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-04-03T03:02:55.151 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-04-03T03:02:55.162 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-04-03T03:02:55.194 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-04-03T03:02:55.196 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-04-03T03:02:55.198 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-04-03T03:02:55.240 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-04-03T03:02:55.252 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-04-03T03:02:55.285 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-04-03T03:02:55.297 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-04-03T03:02:55.329 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-04-03T03:02:55.382 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-04-03T03:02:55.394 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-04-03T03:02:55.437 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-04-03T03:02:55.480 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-04-03T03:02:55.481 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-04-03T03:02:55.576 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-04-03T03:02:55.577 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-04-03T03:02:55.672 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-04-03T03:02:55.798 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "name": "pdhange", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-04-03T03:02:55.800 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "name": "pdhange", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-04-03T03:02:55.936 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "name": "pdhange", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-04-03T03:02:56.063 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "name": "pdhange", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-04-03T03:02:56.148 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "name": "pdhange", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-04-03T03:02:56.201 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-04-03T03:02:56.275 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-04-03T03:02:56.370 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-04-03T03:02:56.517 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-04-03T03:02:56.518 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-04-03T03:02:56.624 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-04-03T03:02:56.708 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-04-03T03:02:56.772 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-04-03T03:02:56.887 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-04-03T03:02:56.971 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-04-03T03:02:57.076 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-04-03T03:02:57.150 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-04-03T03:02:57.193 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-04-03T03:02:57.258 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-04-03T03:02:57.332 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-04-03T03:02:57.490 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-04-03T03:02:57.628 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-04-03T03:02:57.650 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-04-03T03:02:57.693 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-04-03T03:02:57.715 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-04-03T03:02:57.913 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-04-03T03:02:58.061 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:15.292) 0:00:22.215 ******** 2021-04-03T03:02:58.074 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-04-03T03:02:58.076 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-04-03T03:02:58.296 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-04-03T03:02:58.435 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-04-03T03:02:58.436 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-04-03T03:02:58.526 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.464) 0:00:22.680 ******** 2021-04-03T03:02:58.546 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.020) 0:00:22.700 ******** 2021-04-03T03:02:58.568 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.021) 0:00:22.722 ******** 2021-04-03T03:02:58.590 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.021) 0:00:22.744 ******** 2021-04-03T03:02:58.611 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.021) 0:00:22.765 ******** 2021-04-03T03:02:58.617 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-04-03T03:02:58.653 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.041) 0:00:22.807 ******** 2021-04-03T03:02:58.674 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.020) 0:00:22.828 ******** 2021-04-03T03:02:58.682 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-04-03T03:02:58.684 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-04-03T03:02:58.686 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-04-03T03:02:58.687 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-04-03T03:02:58.689 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-04-03T03:02:58.690 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5m 2021-04-03T03:02:58.691 INFO:teuthology.task.ansible.out:Rahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de5 2021-04-03T03:02:58.692 INFO:teuthology.task.ansible.out:3c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e81 2021-04-03T03:02:58.694 INFO:teuthology.task.ansible.out:6f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d 2021-04-03T03:02:58.696 INFO:teuthology.task.ansible.out:2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCp 2021-04-03T03:02:58.698 INFO:teuthology.task.ansible.out:UnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5 2021-04-03T03:02:58.699 INFO:teuthology.task.ansible.out:e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n 2021-04-03T03:02:58.700 INFO:teuthology.task.ansible.out:7X0iJXA cc646e47c8b503eb38d 2021-04-03T03:02:58.701 INFO:teuthology.task.ansible.out:30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullstea 2021-04-03T03:02:58.703 INFO:teuthology.task.ansible.out:m 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9 2021-04-03T03:02:58.705 INFO:teuthology.task.ansible.out:dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}]}, "changed": false} 2021-04-03T03:02:58.728 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.054) 0:00:22.882 ******** 2021-04-03T03:02:58.757 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.028) 0:00:22.911 ******** 2021-04-03T03:02:58.778 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.021) 0:00:22.932 ******** 2021-04-03T03:02:58.791 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-04-03T03:02:58.793 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-04-03T03:02:58.794 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-04-03T03:02:58.796 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-04-03T03:02:58.797 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name" 2021-04-03T03:02:58.798 INFO:teuthology.task.ansible.out:: "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de5 2021-04-03T03:02:58.799 INFO:teuthology.task.ansible.out:3c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e81 2021-04-03T03:02:58.801 INFO:teuthology.task.ansible.out:6f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d 2021-04-03T03:02:58.803 INFO:teuthology.task.ansible.out:2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCp 2021-04-03T03:02:58.804 INFO:teuthology.task.ansible.out:UnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5 2021-04-03T03:02:58.806 INFO:teuthology.task.ansible.out:e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d 2021-04-03T03:02:58.807 INFO:teuthology.task.ansible.out:30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullstea 2021-04-03T03:02:58.809 INFO:teuthology.task.ansible.out:m 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9 2021-04-03T03:02:58.810 INFO:teuthology.task.ansible.out:dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}]}, "changed": false} ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home 2021-04-03T03:02:58.812 INFO:teuthology.task.ansible.out:o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9 2021-04-03T03:02:58.813 INFO:teuthology.task.ansible.out:517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai 2021-04-03T03:02:58.815 INFO:teuthology.task.ansible.out: YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinst 2021-04-03T03:02:58.817 INFO:teuthology.task.ansible.out:ein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\n 2021-04-03T03:02:58.817 INFO:teuthology.task.ansible.out:yehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@times 2021-04-03T03:02:58.818 INFO:teuthology.task.ansible.out:ink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 6891 2021-04-03T03:02:58.819 INFO:teuthology.task.ansible.out:5d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn" 2021-04-03T03:02:58.821 INFO:teuthology.task.ansible.out:: "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura 2021-04-03T03:02:58.822 INFO:teuthology.task.ansible.out:@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83 2021-04-03T03:02:58.824 INFO:teuthology.task.ansible.out:dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksiriva 2021-04-03T03:02:58.826 INFO:teuthology.task.ansible.out:d@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@la 2021-04-03T03:02:58.826 INFO:teuthology.task.ansible.out:ptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name" 2021-04-03T03:02:58.827 INFO:teuthology.task.ansible.out:: "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e0 2021-04-03T03:02:58.829 INFO:teuthology.task.ansible.out:5ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, { 2021-04-03T03:02:58.831 INFO:teuthology.task.ansible.out:"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}]}, "changed": false} 2021-04-03T03:02:58.832 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-04-03T03:02:58.834 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-04-03T03:02:58.835 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-04-03T03:02:58.837 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-04-03T03:02:58.838 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name" 2021-04-03T03:02:58.839 INFO:teuthology.task.ansible.out:: "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de5 2021-04-03T03:02:58.840 INFO:teuthology.task.ansible.out:3c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e81 2021-04-03T03:02:58.842 INFO:teuthology.task.ansible.out:6f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d 2021-04-03T03:02:58.844 INFO:teuthology.task.ansible.out:2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCp 2021-04-03T03:02:58.845 INFO:teuthology.task.ansible.out:UnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5 2021-04-03T03:02:58.847 INFO:teuthology.task.ansible.out:e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d 2021-04-03T03:02:58.848 INFO:teuthology.task.ansible.out:30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": 2021-04-03T03:02:58.849 INFO:teuthology.task.ansible.out: "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullstea 2021-04-03T03:02:58.850 INFO:teuthology.task.ansible.out:m 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9 2021-04-03T03:02:58.852 INFO:teuthology.task.ansible.out:dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}]}, "changed": false} 2021-04-03T03:02:58.877 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.099) 0:00:23.031 ******** 2021-04-03T03:02:58.901 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.023) 0:00:23.055 ******** 2021-04-03T03:02:58.925 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.023) 0:00:23.079 ******** 2021-04-03T03:02:58.940 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-04-03T03:02:58.942 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-04-03T03:02:58.944 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-04-03T03:02:58.945 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-04-03T03:02:58.947 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name" 2021-04-03T03:02:58.948 INFO:teuthology.task.ansible.out:: "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de5 2021-04-03T03:02:58.950 INFO:teuthology.task.ansible.out:3c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e81 2021-04-03T03:02:58.951 INFO:teuthology.task.ansible.out:6f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d 2021-04-03T03:02:58.953 INFO:teuthology.task.ansible.out:2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCp 2021-04-03T03:02:58.955 INFO:teuthology.task.ansible.out:UnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5 2021-04-03T03:02:58.957 INFO:teuthology.task.ansible.out:e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d 2021-04-03T03:02:58.959 INFO:teuthology.task.ansible.out:30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": 2021-04-03T03:02:58.960 INFO:teuthology.task.ansible.out: "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullstea 2021-04-03T03:02:58.961 INFO:teuthology.task.ansible.out:m 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9 2021-04-03T03:02:58.963 INFO:teuthology.task.ansible.out:dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}]}, "changed": false} 2021-04-03T03:02:58.990 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:58 +0000 (0:00:00.065) 0:00:23.144 ******** 2021-04-03T03:02:59.853 INFO:teuthology.task.ansible.out:FAILED - RETRYING: Clone the keys repo (5 retries left). 2021-04-03T03:02:59.855 INFO:teuthology.task.ansible.out:FAILED - RETRYING: Clone the keys repo (5 retries left). 2021-04-03T03:02:59.865 INFO:teuthology.task.ansible.out:FAILED - RETRYING: Clone the keys repo (5 retries left). 2021-04-03T03:02:59.865 INFO:teuthology.task.ansible.out:FAILED - RETRYING: Clone the keys repo (5 retries left). 2021-04-03T03:02:59.918 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-04-03T03:02:59.920 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"after": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "attempts": 1, "before": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "changed": false, "remote_url_changed": false} 2021-04-03T03:02:59.945 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:02:59 +0000 (0:00:00.954) 0:00:24.099 ******** 2021-04-03T03:03:00.128 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:00.130 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-04-03T03:03:00.233 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-04-03T03:03:00.338 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-04-03T03:03:00.454 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-04-03T03:03:00.456 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-04-03T03:03:00.549 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-04-03T03:03:00.647 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-04-03T03:03:00.649 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-04-03T03:03:00.753 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-04-03T03:03:00.858 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-04-03T03:03:00.963 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-04-03T03:03:00.975 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:01.079 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-04-03T03:03:01.081 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-04-03T03:03:01.164 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-04-03T03:03:01.165 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-04-03T03:03:01.175 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:01.186 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:01.291 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-04-03T03:03:01.406 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-04-03T03:03:01.407 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-04-03T03:03:01.500 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-04-03T03:03:01.605 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-04-03T03:03:01.607 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-04-03T03:03:01.689 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-04-03T03:03:01.784 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-04-03T03:03:01.878 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-04-03T03:03:01.973 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-04-03T03:03:01.975 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-04-03T03:03:02.068 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-04-03T03:03:02.069 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-04-03T03:03:02.162 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-04-03T03:03:02.257 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-04-03T03:03:02.362 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46T 2021-04-03T03:03:02.364 INFO:teuthology.task.ansible.out:RJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-04-03T03:03:02.457 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-04-03T03:03:02.552 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-04-03T03:03:02.647 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-04-03T03:03:02.732 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-04-03T03:03:02.826 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-04-03T03:03:02.931 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-04-03T03:03:02.933 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-04-03T03:03:03.036 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-04-03T03:03:03.131 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-04-03T03:03:03.226 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-04-03T03:03:03.310 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-04-03T03:03:03.405 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-04-03T03:03:03.499 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:03.584 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-04-03T03:03:03.678 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-04-03T03:03:03.773 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-04-03T03:03:03.775 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-04-03T03:03:03.877 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-04-03T03:03:03.879 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-04-03T03:03:03.984 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-04-03T03:03:04.069 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-04-03T03:03:04.164 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-04-03T03:03:04.259 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-04-03T03:03:04.261 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:04.355 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-04-03T03:03:04.356 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-04-03T03:03:04.358 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-04-03T03:03:04.453 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-04-03T03:03:04.550 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-04-03T03:03:04.646 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-04-03T03:03:04.731 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-04-03T03:03:04.816 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-04-03T03:03:04.911 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-04-03T03:03:04.913 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-04-03T03:03:05.006 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-04-03T03:03:05.100 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-04-03T03:03:05.195 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-04-03T03:03:05.196 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-04-03T03:03:05.289 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-04-03T03:03:05.383 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-04-03T03:03:05.478 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-04-03T03:03:05.479 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-04-03T03:03:05.574 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-04-03T03:03:05.671 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-04-03T03:03:05.758 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-04-03T03:03:05.855 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:05.951 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-04-03T03:03:06.046 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-04-03T03:03:06.131 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-04-03T03:03:06.227 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-04-03T03:03:06.323 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-04-03T03:03:06.408 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-04-03T03:03:06.492 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-04-03T03:03:06.588 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-04-03T03:03:06.664 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-04-03T03:03:06.760 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-04-03T03:03:06.845 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-04-03T03:03:06.941 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-04-03T03:03:07.037 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-04-03T03:03:07.131 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-04-03T03:03:07.215 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-04-03T03:03:07.310 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-04-03T03:03:07.414 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-04-03T03:03:07.416 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-04-03T03:03:07.498 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-04-03T03:03:07.593 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-04-03T03:03:07.688 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-04-03T03:03:07.783 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-04-03T03:03:07.878 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-04-03T03:03:07.880 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-04-03T03:03:07.963 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-04-03T03:03:07.965 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-04-03T03:03:08.058 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-04-03T03:03:08.174 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-04-03T03:03:08.268 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-04-03T03:03:08.363 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-04-03T03:03:08.457 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-04-03T03:03:08.552 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-04-03T03:03:08.647 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-04-03T03:03:08.743 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-04-03T03:03:08.745 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-04-03T03:03:08.838 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-04-03T03:03:08.922 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-04-03T03:03:09.016 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-04-03T03:03:09.111 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-04-03T03:03:09.195 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-04-03T03:03:09.290 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-04-03T03:03:09.374 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-04-03T03:03:09.376 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:09.469 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-04-03T03:03:09.563 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-04-03T03:03:09.647 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-04-03T03:03:09.742 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-04-03T03:03:09.826 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-04-03T03:03:09.921 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-04-03T03:03:10.017 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-04-03T03:03:10.112 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-04-03T03:03:10.197 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-04-03T03:03:10.293 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-04-03T03:03:10.389 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-04-03T03:03:10.400 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-04-03T03:03:10.402 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"after": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "attempts": 2, "before": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "changed": false, "remote_url_changed": false} 2021-04-03T03:03:10.424 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:10 +0000 (0:00:10.479) 0:00:34.578 ******** 2021-04-03T03:03:10.431 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"after": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "attempts": 2, "before": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "changed": false, "remote_url_changed": false} 2021-04-03T03:03:10.458 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:10 +0000 (0:00:00.034) 0:00:34.612 ******** 2021-04-03T03:03:10.468 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"after": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "attempts": 2, "before": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "changed": false, "remote_url_changed": false} 2021-04-03T03:03:10.494 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:10 +0000 (0:00:00.035) 0:00:34.648 ******** 2021-04-03T03:03:10.502 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:10.504 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-04-03T03:03:10.515 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-04-03T03:03:10.516 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"after": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "attempts": 2, "before": "5bb5aa76bc3fd5b5b3f6df845aec3bd1f9062232", "changed": false, "remote_url_changed": false} 2021-04-03T03:03:10.540 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:10 +0000 (0:00:00.045) 0:00:34.694 ******** 2021-04-03T03:03:10.590 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:10.592 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-04-03T03:03:10.623 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-04-03T03:03:10.655 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-04-03T03:03:10.677 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-04-03T03:03:10.688 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-04-03T03:03:10.721 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-04-03T03:03:10.733 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-04-03T03:03:10.756 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-04-03T03:03:10.767 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-04-03T03:03:10.789 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-04-03T03:03:10.821 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-04-03T03:03:10.843 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-04-03T03:03:10.854 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-04-03T03:03:10.865 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-04-03T03:03:10.887 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-04-03T03:03:10.941 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-04-03T03:03:10.942 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc 2021-04-03T03:03:10.944 INFO:teuthology.task.ansible.out:2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-04-03T03:03:10.953 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-04-03T03:03:10.955 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-04-03T03:03:10.986 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-04-03T03:03:10.988 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-04-03T03:03:11.039 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-04-03T03:03:11.040 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-04-03T03:03:11.062 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-04-03T03:03:11.064 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-04-03T03:03:11.065 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-04-03T03:03:11.094 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-04-03T03:03:11.127 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-04-03T03:03:11.128 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-04-03T03:03:11.138 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-04-03T03:03:11.140 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-04-03T03:03:11.160 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-04-03T03:03:11.162 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-04-03T03:03:11.172 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-04-03T03:03:11.216 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-04-03T03:03:11.218 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-04-03T03:03:11.228 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-04-03T03:03:11.250 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-04-03T03:03:11.272 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-04-03T03:03:11.273 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-04-03T03:03:11.315 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-04-03T03:03:11.337 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-04-03T03:03:11.339 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-04-03T03:03:11.370 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-04-03T03:03:11.391 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-04-03T03:03:11.413 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-04-03T03:03:11.445 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-04-03T03:03:11.446 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-04-03T03:03:11.448 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.479 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-04-03T03:03:11.500 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-04-03T03:03:11.501 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-04-03T03:03:11.512 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.535 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-04-03T03:03:11.547 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.548 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-04-03T03:03:11.550 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-04-03T03:03:11.581 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-04-03T03:03:11.592 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-04-03T03:03:11.593 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.604 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-04-03T03:03:11.606 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-04-03T03:03:11.647 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-04-03T03:03:11.649 INFO:teuthology.task.ansible.out:on", "validate_certs": true} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.659 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-04-03T03:03:11.660 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.682 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-04-03T03:03:11.704 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-04-03T03:03:11.705 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-04-03T03:03:11.726 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-04-03T03:03:11.727 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-04-03T03:03:11.737 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.739 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.749 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-04-03T03:03:11.771 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-04-03T03:03:11.771 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-04-03T03:03:11.773 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-04-03T03:03:11.782 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.794 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.805 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-04-03T03:03:11.807 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-04-03T03:03:11.816 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.827 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:11.838 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-04-03T03:03:11.840 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-04-03T03:03:11.849 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-04-03T03:03:11.871 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-04-03T03:03:11.893 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-04-03T03:03:11.927 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-04-03T03:03:11.938 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-04-03T03:03:11.970 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-04-03T03:03:11.972 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47 2021-04-03T03:03:11.974 INFO:teuthology.task.ansible.out:v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-04-03T03:03:12.014 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-04-03T03:03:12.016 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-04-03T03:03:12.036 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-04-03T03:03:12.038 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Nj 2021-04-03T03:03:12.039 INFO:teuthology.task.ansible.out:k0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-04-03T03:03:12.059 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-04-03T03:03:12.091 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-04-03T03:03:12.113 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-04-03T03:03:12.125 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-04-03T03:03:12.147 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-04-03T03:03:12.148 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-04-03T03:03:12.202 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-04-03T03:03:12.204 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-04-03T03:03:12.213 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-04-03T03:03:12.224 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-04-03T03:03:12.226 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-04-03T03:03:12.236 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-04-03T03:03:12.247 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-04-03T03:03:12.249 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-04-03T03:03:12.291 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-04-03T03:03:12.313 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-04-03T03:03:12.335 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-04-03T03:03:12.336 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-04-03T03:03:12.349 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-04-03T03:03:12.381 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-04-03T03:03:12.414 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-04-03T03:03:12.416 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-04-03T03:03:12.426 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-04-03T03:03:12.437 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-04-03T03:03:12.438 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-04-03T03:03:12.481 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-04-03T03:03:12.503 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-04-03T03:03:12.504 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-04-03T03:03:12.514 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-04-03T03:03:12.526 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-04-03T03:03:12.528 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-04-03T03:03:12.612 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-04-03T03:03:12.613 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-04-03T03:03:12.615 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-04-03T03:03:12.617 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-04-03T03:03:12.626 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-04-03T03:03:12.628 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, 2021-04-03T03:03:12.629 INFO:teuthology.task.ansible.out:"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-04-03T03:03:12.702 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-04-03T03:03:12.713 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "k 2021-04-03T03:03:12.714 INFO:teuthology.task.ansible.out:ey_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i 2021-04-03T03:03:12.716 INFO:teuthology.task.ansible.out:0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-04-03T03:03:12.725 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-04-03T03:03:12.727 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSr 2021-04-03T03:03:12.729 INFO:teuthology.task.ansible.out:HklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-04-03T03:03:12.800 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-04-03T03:03:12.800 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46T 2021-04-03T03:03:12.805 INFO:teuthology.task.ansible.out:RJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-04-03T03:03:12.812 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-04-03T03:03:12.824 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-04-03T03:03:12.835 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-04-03T03:03:12.899 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-04-03T03:03:12.910 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-04-03T03:03:12.911 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-04-03T03:03:12.923 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-04-03T03:03:12.979 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-04-03T03:03:13.012 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:13 +0000 (0:00:02.472) 0:00:37.167 ******** 2021-04-03T03:03:13.022 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-04-03T03:03:13.023 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46T 2021-04-03T03:03:13.025 INFO:teuthology.task.ansible.out:RJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1y 2021-04-03T03:03:13.027 INFO:teuthology.task.ansible.out:c2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020d 2021-04-03T03:03:13.029 INFO:teuthology.task.ansible.out:af852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state" 2021-04-03T03:03:13.030 INFO:teuthology.task.ansible.out:: "present", "user": "yehudasa", "validate_certs": true} 2021-04-03T03:03:13.103 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-04-03T03:03:13.104 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-04-03T03:03:13.116 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:13.117 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-04-03T03:03:13.119 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-04-03T03:03:13.120 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.131 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.142 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.153 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.165 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.176 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.187 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.188 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-04-03T03:03:13.199 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-04-03T03:03:13.211 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-04-03T03:03:13.222 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-04-03T03:03:13.276 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:13.278 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-04-03T03:03:13.287 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.289 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5 2021-04-03T03:03:13.291 INFO:teuthology.task.ansible.out:f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state 2021-04-03T03:03:13.293 INFO:teuthology.task.ansible.out:": "present", "user": "joao", "validate_certs": true} 2021-04-03T03:03:13.300 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:13.302 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.311 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.313 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-04-03T03:03:13.323 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-04-03T03:03:13.376 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-04-03T03:03:13.388 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-04-03T03:03:13.389 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn 2021-04-03T03:03:13.391 INFO:teuthology.task.ansible.out:": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-04-03T03:03:13.421 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.423 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-04-03T03:03:13.433 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-04-03T03:03:13.466 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-04-03T03:03:13.467 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:13.467 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-04-03T03:03:13.478 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.480 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d1 2021-04-03T03:03:13.482 INFO:teuthology.task.ansible.out:3dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.490 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.492 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.502 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.503 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.513 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.515 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] 2021-04-03T03:03:13.516 INFO:teuthology.task.ansible.out: => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.525 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.527 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] 2021-04-03T03:03:13.529 INFO:teuthology.task.ansible.out: => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.538 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.539 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.550 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.550 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.551 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSY 2021-04-03T03:03:13.552 INFO:teuthology.task.ansible.out:oiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.564 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.566 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.576 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.578 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.588 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.589 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.600 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.611 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.612 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.614 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSY 2021-04-03T03:03:13.615 INFO:teuthology.task.ansible.out:oiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrx 2021-04-03T03:03:13.617 INFO:teuthology.task.ansible.out:TyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-04-03T03:03:13.626 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.628 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.638 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.651 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEo 2021-04-03T03:03:13.652 INFO:teuthology.task.ansible.out:K7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-04-03T03:03:13.662 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.664 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-04-03T03:03:13.674 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:13.675 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.685 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.687 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.697 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.698 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.708 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.710 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.720 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.722 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee 2021-04-03T03:03:13.724 INFO:teuthology.task.ansible.out:754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu 2021-04-03T03:03:13.725 INFO:teuthology.task.ansible.out:/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.733 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.734 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.746 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.747 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.748 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-04-03T03:03:13.748 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:13.748 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.759 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.761 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.771 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.772 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.783 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.784 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.795 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.806 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.808 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.817 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.819 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32ce 2021-04-03T03:03:13.820 INFO:teuthology.task.ansible.out:f29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.831 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.832 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.842 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.844 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.855 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.856 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.867 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.869 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.878 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.880 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.890 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.902 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.913 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.914 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1 2021-04-03T03:03:13.916 INFO:teuthology.task.ansible.out:eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.926 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.927 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.937 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.939 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:13.940 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"nam 2021-04-03T03:03:13.942 INFO:teuthology.task.ansible.out:e": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:13.950 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:13.952 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.962 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.964 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.973 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.975 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.985 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.996 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:13.997 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.008 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.010 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.021 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.021 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.032 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.034 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.034 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.034 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.036 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-04-03T03:03:14.046 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:14.047 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.057 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.068 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.070 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.080 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.082 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.091 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.093 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.093 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, 2021-04-03T03:03:14.095 INFO:teuthology.task.ansible.out: "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.105 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.107 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.116 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.118 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn 2021-04-03T03:03:14.120 INFO:teuthology.task.ansible.out:': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.129 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.130 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d7 2021-04-03T03:03:14.132 INFO:teuthology.task.ansible.out:3728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.141 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.142 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.152 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.164 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.165 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.176 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.177 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "ski 2021-04-03T03:03:14.179 INFO:teuthology.task.ansible.out:p_reason": "Conditional result was False"} 2021-04-03T03:03:14.188 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.190 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:14.200 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:14.202 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.212 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.213 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 4715 2021-04-03T03:03:14.215 INFO:teuthology.task.ansible.out:38eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf94609 2021-04-03T03:03:14.217 INFO:teuthology.task.ansible.out:80985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.225 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.227 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.236 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.247 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.249 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.259 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.260 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.270 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.302 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.304 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10 2021-04-03T03:03:14.305 INFO:teuthology.task.ansible.out:aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-04-03T03:03:14.327 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:14 +0000 (0:00:01.314) 0:00:38.481 ******** 2021-04-03T03:03:14.333 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-04-03T03:03:14.335 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-04-03T03:03:14.377 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-04-03T03:03:14.378 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-04-03T03:03:14.389 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-04-03T03:03:14.391 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-04-03T03:03:14.400 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-04-03T03:03:14.402 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:14.424 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:14 +0000 (0:00:00.096) 0:00:38.578 ******** 2021-04-03T03:03:14.430 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.431 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-04-03T03:03:14.473 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-04-03T03:03:14.475 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-04-03T03:03:14.485 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-04-03T03:03:14.486 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB 2021-04-03T03:03:14.488 INFO:teuthology.task.ansible.out:5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-04-03T03:03:14.497 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-04-03T03:03:14.582 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-04-03T03:03:14.584 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.584 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJ 2021-04-03T03:03:14.586 INFO:teuthology.task.ansible.out:Fjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-04-03T03:03:14.596 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-04-03T03:03:14.671 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-04-03T03:03:14.673 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-04-03T03:03:14.683 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.685 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-04-03T03:03:14.687 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-04-03T03:03:14.695 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-04-03T03:03:14.759 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-04-03T03:03:14.771 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:14.772 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vX 2021-04-03T03:03:14.774 INFO:teuthology.task.ansible.out:ArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+r 2021-04-03T03:03:14.776 INFO:teuthology.task.ansible.out:fGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-04-03T03:03:14.783 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-04-03T03:03:14.795 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-04-03T03:03:14.848 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-04-03T03:03:14.860 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:14.862 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-04-03T03:03:14.873 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-04-03T03:03:14.884 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-04-03T03:03:14.886 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:14.937 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nul 2021-04-03T03:03:14.939 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:14.948 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-04-03T03:03:14.949 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:14.949 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-04-03T03:03:14.971 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:14.972 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-04-03T03:03:14.974 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-04-03T03:03:14.975 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNH 2021-04-03T03:03:14.977 INFO:teuthology.task.ansible.out:ORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:15.035 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-04-03T03:03:15.063 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-04-03T03:03:15.065 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB 2021-04-03T03:03:15.066 INFO:teuthology.task.ansible.out:3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XO 2021-04-03T03:03:15.068 INFO:teuthology.task.ansible.out:Vj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJ 2021-04-03T03:03:15.070 INFO:teuthology.task.ansible.out:R61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-04-03T03:03:15.135 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-04-03T03:03:15.136 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-04-03T03:03:15.147 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-04-03T03:03:15.148 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-04-03T03:03:15.222 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-04-03T03:03:15.233 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-04-03T03:03:15.234 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-04-03T03:03:15.236 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-04-03T03:03:15.246 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:15.248 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-04-03T03:03:15.321 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-04-03T03:03:15.322 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-04-03T03:03:15.333 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-04-03T03:03:15.334 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-04-03T03:03:15.345 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:15.347 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-04-03T03:03:15.420 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-04-03T03:03:15.420 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-04-03T03:03:15.422 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-04-03T03:03:15.433 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-04-03T03:03:15.435 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-04-03T03:03:15.519 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-04-03T03:03:15.521 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-04-03T03:03:15.532 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:15.534 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-04-03T03:03:15.536 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-04-03T03:03:15.608 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-04-03T03:03:15.610 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-04-03T03:03:15.620 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-04-03T03:03:15.622 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1K 2021-04-03T03:03:15.623 INFO:teuthology.task.ansible.out:h9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-04-03T03:03:15.633 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-04-03T03:03:15.635 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-04-03T03:03:15.707 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-04-03T03:03:15.718 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-04-03T03:03:15.720 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-04-03T03:03:15.721 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-04-03T03:03:15.794 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-04-03T03:03:15.806 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-04-03T03:03:15.807 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-04-03T03:03:15.819 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-04-03T03:03:15.882 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-04-03T03:03:15.884 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-04-03T03:03:15.894 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-04-03T03:03:15.895 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-04-03T03:03:15.905 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:15.907 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-04-03T03:03:15.909 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-04-03T03:03:15.917 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-04-03T03:03:15.919 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-04-03T03:03:15.970 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-04-03T03:03:15.992 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-04-03T03:03:15.994 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-04-03T03:03:16.004 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:16.005 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-04-03T03:03:16.078 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-04-03T03:03:16.090 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.091 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/e 2021-04-03T03:03:16.093 INFO:teuthology.task.ansible.out:AA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvb 2021-04-03T03:03:16.095 INFO:teuthology.task.ansible.out:BudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-04-03T03:03:16.103 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-04-03T03:03:16.166 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-04-03T03:03:16.168 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-04-03T03:03:16.178 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.180 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:16.180 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-04-03T03:03:16.192 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-04-03T03:03:16.204 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-04-03T03:03:16.205 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-04-03T03:03:16.257 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-04-03T03:03:16.268 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.270 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-04-03T03:03:16.280 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:16.292 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-04-03T03:03:16.345 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-04-03T03:03:16.356 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.358 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-04-03T03:03:16.369 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-04-03T03:03:16.380 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-04-03T03:03:16.433 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-04-03T03:03:16.444 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.446 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-04-03T03:03:16.456 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:16.457 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4h 2021-04-03T03:03:16.459 INFO:teuthology.task.ansible.out:LmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:16.468 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-04-03T03:03:16.521 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:16.543 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.544 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp 2021-04-03T03:03:16.546 INFO:teuthology.task.ansible.out:2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-04-03T03:03:16.566 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-04-03T03:03:16.609 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-04-03T03:03:16.642 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.644 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:16.644 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-04-03T03:03:16.646 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-04-03T03:03:16.657 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-04-03T03:03:16.699 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-04-03T03:03:16.732 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.734 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnci 2021-04-03T03:03:16.737 INFO:teuthology.task.ansible.out:v8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-04-03T03:03:16.756 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-04-03T03:03:16.778 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-04-03T03:03:16.820 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.822 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-04-03T03:03:16.823 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-04-03T03:03:16.835 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-04-03T03:03:16.846 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-04-03T03:03:16.879 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-04-03T03:03:16.913 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:16.914 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-04-03T03:03:16.924 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:16.926 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-04-03T03:03:16.936 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-04-03T03:03:16.937 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-04-03T03:03:16.969 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-04-03T03:03:17.002 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-04-03T03:03:17.013 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.015 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-04-03T03:03:17.035 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.037 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-04-03T03:03:17.048 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-04-03T03:03:17.101 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-04-03T03:03:17.112 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.114 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-04-03T03:03:17.125 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.127 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-04-03T03:03:17.138 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-04-03T03:03:17.202 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-04-03T03:03:17.214 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.215 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-04-03T03:03:17.227 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-04-03T03:03:17.228 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-04-03T03:03:17.291 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-04-03T03:03:17.302 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.304 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-04-03T03:03:17.314 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.316 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_cer 2021-04-03T03:03:17.318 INFO:teuthology.task.ansible.out:ts": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-04-03T03:03:17.391 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-04-03T03:03:17.403 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.404 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-04-03T03:03:17.415 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-04-03T03:03:17.427 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-04-03T03:03:17.490 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-04-03T03:03:17.501 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.503 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-04-03T03:03:17.514 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-04-03T03:03:17.568 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-04-03T03:03:17.579 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-04-03T03:03:17.581 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-04-03T03:03:17.591 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.593 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-04-03T03:03:17.613 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.615 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-04-03T03:03:17.667 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-04-03T03:03:17.667 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-04-03T03:03:17.689 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-04-03T03:03:17.691 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-04-03T03:03:17.712 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.713 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-04-03T03:03:17.754 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-04-03T03:03:17.766 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-04-03T03:03:17.777 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-04-03T03:03:17.779 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-04-03T03:03:17.800 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.801 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-04-03T03:03:17.842 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-04-03T03:03:17.844 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-04-03T03:03:17.865 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-04-03T03:03:17.865 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-04-03T03:03:17.877 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.879 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-04-03T03:03:17.888 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.890 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-04-03T03:03:17.942 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-04-03T03:03:17.953 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-04-03T03:03:17.965 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-04-03T03:03:17.965 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:17.966 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-04-03T03:03:17.988 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:17.990 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-04-03T03:03:18.031 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-04-03T03:03:18.043 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-04-03T03:03:18.044 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-04-03T03:03:18.054 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-04-03T03:03:18.077 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-04-03T03:03:18.080 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-04-03T03:03:18.121 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-04-03T03:03:18.133 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-04-03T03:03:18.155 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-04-03T03:03:18.178 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-04-03T03:03:18.221 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-04-03T03:03:18.232 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-04-03T03:03:18.254 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-04-03T03:03:18.256 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-04-03T03:03:18.265 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-04-03T03:03:18.307 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:18.309 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-04-03T03:03:18.319 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:18.321 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-04-03T03:03:18.323 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-04-03T03:03:18.332 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-04-03T03:03:18.354 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-04-03T03:03:18.366 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-04-03T03:03:18.410 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-04-03T03:03:18.412 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-04-03T03:03:18.432 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-04-03T03:03:18.443 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-04-03T03:03:18.465 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-04-03T03:03:18.467 INFO:teuthology.task.ansible.out: 2021-04-03T03:03:18.498 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-04-03T03:03:18.500 INFO:teuthology.task.ansible.out: 2021-04-03T03:03:18.521 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-04-03T03:03:18.523 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-04-03T03:03:18.532 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-04-03T03:03:18.534 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-04-03T03:03:18.554 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-04-03T03:03:18.587 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-04-03T03:03:18.618 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-04-03T03:03:18.620 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9f 2021-04-03T03:03:18.622 INFO:teuthology.task.ansible.out:pE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-04-03T03:03:18.641 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-04-03T03:03:18.683 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-04-03T03:03:18.705 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-04-03T03:03:18.716 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-04-03T03:03:18.718 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** changed: [gibba011.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-04-03T03:03:18.728 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:18.730 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-04-03T03:03:18.732 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-04-03T03:03:18.771 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-04-03T03:03:18.793 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-04-03T03:03:18.804 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-04-03T03:03:18.804 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:18.805 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-04-03T03:03:18.805 INFO:teuthology.task.ansible.out: 2021-04-03T03:03:18.816 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:18.818 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-04-03T03:03:18.820 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-04-03T03:03:18.860 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-04-03T03:03:18.883 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-04-03T03:03:18.894 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-04-03T03:03:18.896 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-04-03T03:03:18.916 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:18.918 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-04-03T03:03:18.949 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-04-03T03:03:18.982 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-04-03T03:03:18.993 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:18.995 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-04-03T03:03:19.006 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-04-03T03:03:19.049 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-04-03T03:03:19.071 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-04-03T03:03:19.082 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-04-03T03:03:19.084 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-04-03T03:03:19.105 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:19.107 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-04-03T03:03:19.137 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-04-03T03:03:19.139 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-04-03T03:03:19.169 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-04-03T03:03:19.171 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-04-03T03:03:19.181 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:19.183 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-04-03T03:03:19.204 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:19.206 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-04-03T03:03:19.237 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-04-03T03:03:19.259 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-04-03T03:03:19.270 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-04-03T03:03:19.281 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:19.283 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-04-03T03:03:19.292 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:19.294 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-04-03T03:03:19.314 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-04-03T03:03:19.347 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-04-03T03:03:19.349 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-04-03T03:03:19.358 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-04-03T03:03:19.360 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-04-03T03:03:19.380 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:19.382 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-04-03T03:03:19.391 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:19.393 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-04-03T03:03:19.414 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-04-03T03:03:19.436 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-04-03T03:03:19.447 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-04-03T03:03:19.469 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:19.470 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-04-03T03:03:19.480 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:19.482 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-04-03T03:03:19.502 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-04-03T03:03:19.525 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-04-03T03:03:19.546 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-04-03T03:03:19.568 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:19.570 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9 2021-04-03T03:03:19.571 INFO:teuthology.task.ansible.out:EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.local", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-04-03T03:03:19.591 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-04-03T03:03:19.623 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-04-03T03:03:19.644 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-04-03T03:03:19.646 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba011.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-04-03T03:03:19.676 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:19.678 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-04-03T03:03:19.700 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:19 +0000 (0:00:05.276) 0:00:43.854 ******** 2021-04-03T03:03:19.706 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-04-03T03:03:19.719 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-04-03T03:03:19.740 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-04-03T03:03:19.752 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-04-03T03:03:19.784 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-04-03T03:03:19.786 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:19.796 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-04-03T03:03:19.828 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-04-03T03:03:19.860 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-04-03T03:03:19.861 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-04-03T03:03:19.883 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-04-03T03:03:19.925 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-04-03T03:03:19.958 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-04-03T03:03:19.960 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-04-03T03:03:19.970 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-04-03T03:03:19.971 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "checksum": "53b31205b3cc9729fea6ce5ceab6366ba2fdabc0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "6cd40c867036c949fadb741414d81285", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617418999.7703006-22109-63820257021062/source", "state": "file", "uid": 0} 2021-04-03T03:03:19.995 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:19 +0000 (0:00:00.294) 0:00:44.149 ******** 2021-04-03T03:03:20.002 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.002 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-04-03T03:03:20.004 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:20.014 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-04-03T03:03:20.016 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:20.047 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-04-03T03:03:20.048 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-04-03T03:03:20.059 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-04-03T03:03:20.061 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => { "msg": "Host gibba011.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-04-03T03:03:20.084 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.089) 0:00:44.238 ******** 2021-04-03T03:03:20.090 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.092 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-04-03T03:03:20.102 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-04-03T03:03:20.135 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-04-03T03:03:20.146 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-04-03T03:03:20.168 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-04-03T03:03:20.169 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-04-03T03:03:20.193 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.109) 0:00:44.347 ******** 2021-04-03T03:03:20.200 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-04-03T03:03:20.233 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-04-03T03:03:20.235 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", " 2021-04-03T03:03:20.237 INFO:teuthology.task.ansible.out:manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-04-03T03:03:20.256 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-04-03T03:03:20.258 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.280 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.087) 0:00:44.434 ******** 2021-04-03T03:03:20.297 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.299 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-04-03T03:03:20.319 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-04-03T03:03:20.321 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-04-03T03:03:20.342 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-04-03T03:03:20.344 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.369 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.088) 0:00:44.523 ******** 2021-04-03T03:03:20.407 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.409 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-04-03T03:03:20.419 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-04-03T03:03:20.420 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-04-03T03:03:20.431 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-04-03T03:03:20.453 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-04-03T03:03:20.455 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.478 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.108) 0:00:44.632 ******** 2021-04-03T03:03:20.506 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.508 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-04-03T03:03:20.518 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-04-03T03:03:20.519 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-04-03T03:03:20.531 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-04-03T03:03:20.542 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.564 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.086) 0:00:44.718 ******** 2021-04-03T03:03:20.591 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.593 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-04-03T03:03:20.603 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-04-03T03:03:20.604 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-04-03T03:03:20.626 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-04-03T03:03:20.637 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-04-03T03:03:20.639 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.661 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.097) 0:00:44.815 ******** 2021-04-03T03:03:20.689 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-04-03T03:03:20.700 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-04-03T03:03:20.701 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-04-03T03:03:20.723 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.746 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.084) 0:00:44.900 ******** 2021-04-03T03:03:20.783 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.785 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qV 2021-04-03T03:03:20.786 INFO:teuthology.task.ansible.out:E8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-04-03T03:03:20.795 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-04-03T03:03:20.807 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.808 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-04-03T03:03:20.833 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.087) 0:00:44.987 ******** 2021-04-03T03:03:20.870 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-04-03T03:03:20.881 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-04-03T03:03:20.883 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-04-03T03:03:20.893 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:20.916 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.083) 0:00:45.070 ******** 2021-04-03T03:03:20.922 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:20.924 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-04-03T03:03:20.965 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-04-03T03:03:20.967 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-04-03T03:03:20.977 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-04-03T03:03:20.979 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [gibba011.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:21.001 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:20 +0000 (0:00:00.085) 0:00:45.155 ******** 2021-04-03T03:03:21.007 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.010 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-04-03T03:03:21.061 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:21.063 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "ke 2021-04-03T03:03:21.065 INFO:teuthology.task.ansible.out:yfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-04-03T03:03:21.088 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.086) 0:00:45.242 ******** 2021-04-03T03:03:21.105 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-04-03T03:03:21.147 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-04-03T03:03:21.158 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-04-03T03:03:21.160 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-04-03T03:03:21.182 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.094) 0:00:45.336 ******** 2021-04-03T03:03:21.189 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-04-03T03:03:21.190 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-04-03T03:03:21.243 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-04-03T03:03:21.245 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:21.267 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.084) 0:00:45.421 ******** 2021-04-03T03:03:21.273 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.275 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2021-04-03T03:03:21.276 INFO:teuthology.task.ansible.out:": "shraddhaag", "validate_certs": true} 2021-04-03T03:03:21.286 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-04-03T03:03:21.328 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:21.330 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-04-03T03:03:21.352 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.085) 0:00:45.506 ******** 2021-04-03T03:03:21.358 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-04-03T03:03:21.359 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-04-03T03:03:21.371 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-04-03T03:03:21.414 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-04-03T03:03:21.416 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:21.437 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.084) 0:00:45.591 ******** 2021-04-03T03:03:21.443 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.445 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-04-03T03:03:21.456 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-04-03T03:03:21.457 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-04-03T03:03:21.458 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-04-03T03:03:21.501 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-04-03T03:03:21.503 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:21.523 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.086) 0:00:45.677 ******** 2021-04-03T03:03:21.530 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.532 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-04-03T03:03:21.533 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-04-03T03:03:21.555 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-04-03T03:03:21.556 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-04-03T03:03:21.589 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-04-03T03:03:21.590 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:21.611 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.088) 0:00:45.766 ******** 2021-04-03T03:03:21.617 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.618 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-04-03T03:03:21.629 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-04-03T03:03:21.641 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-04-03T03:03:21.655 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-04-03T03:03:21.677 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:21.699 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.087) 0:00:45.853 ******** 2021-04-03T03:03:21.708 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.708 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-04-03T03:03:21.721 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-04-03T03:03:21.733 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-04-03T03:03:21.735 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-04-03T03:03:21.746 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-04-03T03:03:21.768 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:21.801 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.101) 0:00:45.955 ******** 2021-04-03T03:03:21.808 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.810 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-04-03T03:03:21.820 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-04-03T03:03:21.822 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-04-03T03:03:21.831 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-04-03T03:03:21.843 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-04-03T03:03:21.876 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-04-03T03:03:21.888 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-04-03T03:03:21.890 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:21.928 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:21 +0000 (0:00:00.127) 0:00:46.082 ******** 2021-04-03T03:03:21.935 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:21.936 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-04-03T03:03:21.938 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-04-03T03:03:21.970 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-04-03T03:03:21.992 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.015 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.087) 0:00:46.169 ******** 2021-04-03T03:03:22.023 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.024 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJd 2021-04-03T03:03:22.026 INFO:teuthology.task.ansible.out:Rzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomai 2021-04-03T03:03:22.028 INFO:teuthology.task.ansible.out:n\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-04-03T03:03:22.058 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-04-03T03:03:22.080 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.103 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.087) 0:00:46.257 ******** 2021-04-03T03:03:22.114 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.115 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-04-03T03:03:22.126 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-04-03T03:03:22.138 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-04-03T03:03:22.149 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-04-03T03:03:22.171 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-04-03T03:03:22.173 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.197 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.093) 0:00:46.351 ******** 2021-04-03T03:03:22.203 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.205 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-04-03T03:03:22.216 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-04-03T03:03:22.227 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-04-03T03:03:22.239 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-04-03T03:03:22.261 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-04-03T03:03:22.263 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.284 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.087) 0:00:46.438 ******** 2021-04-03T03:03:22.301 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.302 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-04-03T03:03:22.303 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-04-03T03:03:22.314 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-04-03T03:03:22.326 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-04-03T03:03:22.348 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.370 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.086) 0:00:46.524 ******** 2021-04-03T03:03:22.398 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.400 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-04-03T03:03:22.413 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-04-03T03:03:22.414 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-04-03T03:03:22.427 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-04-03T03:03:22.438 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-04-03T03:03:22.440 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.461 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.090) 0:00:46.615 ******** 2021-04-03T03:03:22.498 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.500 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-04-03T03:03:22.510 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-04-03T03:03:22.512 INFO:teuthology.task.ansible.out: ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-04-03T03:03:22.513 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-04-03T03:03:22.535 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-04-03T03:03:22.537 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:22.558 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.097) 0:00:46.712 ******** 2021-04-03T03:03:22.596 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.598 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-04-03T03:03:22.609 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-04-03T03:03:22.610 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-04-03T03:03:22.612 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-04-03T03:03:22.633 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.657 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.098) 0:00:46.811 ******** 2021-04-03T03:03:22.684 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.686 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-04-03T03:03:22.696 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-04-03T03:03:22.709 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-04-03T03:03:22.710 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-04-03T03:03:22.732 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-04-03T03:03:22.734 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.755 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.098) 0:00:46.909 ******** 2021-04-03T03:03:22.792 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.794 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6 2021-04-03T03:03:22.797 INFO:teuthology.task.ansible.out:IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-04-03T03:03:22.807 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-04-03T03:03:22.829 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-04-03T03:03:22.830 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.854 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.099) 0:00:47.009 ******** 2021-04-03T03:03:22.882 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.884 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-04-03T03:03:22.894 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-04-03T03:03:22.895 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-04-03T03:03:22.928 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-04-03T03:03:22.930 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:22.959 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:22 +0000 (0:00:00.104) 0:00:47.113 ******** 2021-04-03T03:03:22.966 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:22.967 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-04-03T03:03:22.979 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-04-03T03:03:22.979 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-04-03T03:03:22.981 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-04-03T03:03:23.033 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-04-03T03:03:23.035 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.072 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.113) 0:00:47.226 ******** 2021-04-03T03:03:23.079 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:23.081 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-04-03T03:03:23.082 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYsl 2021-04-03T03:03:23.083 INFO:teuthology.task.ansible.out:I0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZ 2021-04-03T03:03:23.085 INFO:teuthology.task.ansible.out:Yr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-04-03T03:03:23.148 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.150 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-04-03T03:03:23.174 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.101) 0:00:47.328 ******** 2021-04-03T03:03:23.180 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-04-03T03:03:23.182 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-04-03T03:03:23.183 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-04-03T03:03:23.247 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-04-03T03:03:23.270 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-04-03T03:03:23.282 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-04-03T03:03:23.283 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-04-03T03:03:23.367 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.193) 0:00:47.521 ******** 2021-04-03T03:03:23.375 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-04-03T03:03:23.376 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-04-03T03:03:23.376 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.031458", "end": "2021-04-03 03:03:23.597599", "rc": 0, "start": "2021-04-03 03:03:23.566141", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-04-03T03:03:23.378 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslO 2021-04-03T03:03:23.379 INFO:teuthology.task.ansible.out:KeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-04-03T03:03:23.404 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.036) 0:00:47.558 ******** 2021-04-03T03:03:23.412 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-04-03T03:03:23.445 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-04-03T03:03:23.467 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "k 2021-04-03T03:03:23.468 INFO:teuthology.task.ansible.out:ey_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-04-03T03:03:23.533 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-04-03T03:03:23.555 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-04-03T03:03:23.556 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:23.556 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.567 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.569 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.712 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.308) 0:00:47.866 ******** 2021-04-03T03:03:23.719 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:23.721 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.722 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.749 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:23.779 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:23.781 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62 2021-04-03T03:03:23.782 INFO:teuthology.task.ansible.out:e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9re 2021-04-03T03:03:23.784 INFO:teuthology.task.ansible.out:G4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} changed: [gibba031.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-04-03T03:03:23.830 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:23.832 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e19 2021-04-03T03:03:23.834 INFO:teuthology.task.ansible.out:02a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "commen 2021-04-03T03:03:23.835 INFO:teuthology.task.ansible.out:t": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False 2021-04-03T03:03:23.837 INFO:teuthology.task.ansible.out:"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": nu 2021-04-03T03:03:23.838 INFO:teuthology.task.ansible.out:ll, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-04-03T03:03:23.858 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.145) 0:00:48.012 ******** 2021-04-03T03:03:23.878 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.020) 0:00:48.032 ******** 2021-04-03T03:03:23.900 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:23 +0000 (0:00:00.021) 0:00:48.054 ******** 2021-04-03T03:03:23.907 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-04-03T03:03:23.924 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.926 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba026.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fed 2021-04-03T03:03:23.928 INFO:teuthology.task.ansible.out:oragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} TASK [common : Mask sleep units] *********************************************** skipping: [gibba011.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.955 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-04-03T03:03:23.987 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:23.988 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} TASK [common : Mask sleep units] *********************************************** skipping: [gibba011.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e20 2021-04-03T03:03:23.990 INFO:teuthology.task.ansible.out:10c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} TASK [common : Mask sleep units] *********************************************** TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.990 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.991 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:23.991 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c7 2021-04-03T03:03:23.993 INFO:teuthology.task.ansible.out:6885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'tamil', 'ov 2021-04-03T03:03:23.994 INFO:teuthology.task.ansible.out:pn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5 2021-04-03T03:03:23.996 INFO:teuthology.task.ansible.out:f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.018 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.118) 0:00:48.172 ******** 2021-04-03T03:03:24.040 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.021) 0:00:48.194 ******** 2021-04-03T03:03:24.048 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.048 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.050 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.050 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.051 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@a 2021-04-03T03:03:24.052 INFO:teuthology.task.ansible.out:klap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.053 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0Rv 2021-04-03T03:03:24.054 INFO:teuthology.task.ansible.out:okJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result 2021-04-03T03:03:24.056 INFO:teuthology.task.ansible.out: was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac 2021-04-03T03:03:24.057 INFO:teuthology.task.ansible.out:0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36Hh 2021-04-03T03:03:24.059 INFO:teuthology.task.ansible.out:eVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.073 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.074 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.075 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.076 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.101 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.061) 0:00:48.255 ******** 2021-04-03T03:03:24.108 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.110 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.122 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-04-03T03:03:24.127 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 2021-04-03T03:03:24.129 INFO:teuthology.task.ansible.out:'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b 2021-04-03T03:03:24.131 INFO:teuthology.task.ansible.out:64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.140 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.142 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/ 2021-04-03T03:03:24.144 INFO:teuthology.task.ansible.out:v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.154 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.156 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.166 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.168 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.168 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.191 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.090) 0:00:48.345 ******** 2021-04-03T03:03:24.202 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.203 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.205 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.205 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.207 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.207 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.209 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.210 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-04-03T03:03:24.212 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.223 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.235 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.236 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-04-03T03:03:24.238 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.238 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.250 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.252 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.262 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.264 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.291 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.099) 0:00:48.445 ******** 2021-04-03T03:03:24.298 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.300 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.301 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr 2021-04-03T03:03:24.303 INFO:teuthology.task.ansible.out:1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} ok: [gibba028.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "p 2021-04-03T03:03:24.305 INFO:teuthology.task.ansible.out:ath": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.315 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.316 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7O 2021-04-03T03:03:24.318 INFO:teuthology.task.ansible.out:k9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.318 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsI 2021-04-03T03:03:24.320 INFO:teuthology.task.ansible.out:tN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+ 2021-04-03T03:03:24.321 INFO:teuthology.task.ansible.out:IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.331 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.332 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 2021-04-03T03:03:24.334 INFO:teuthology.task.ansible.out:50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.343 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.345 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.345 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovp 2021-04-03T03:03:24.347 INFO:teuthology.task.ansible.out:n": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.356 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.357 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.358 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-04-03T03:03:24.358 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:24.358 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.359 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.359 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) = 2021-04-03T03:03:24.361 INFO:teuthology.task.ansible.out:> {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.383 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.092) 0:00:48.538 ******** 2021-04-03T03:03:24.393 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.395 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.396 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.397 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => { 2021-04-03T03:03:24.399 INFO:teuthology.task.ansible.out:"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTi 2021-04-03T03:03:24.400 INFO:teuthology.task.ansible.out:uIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d1 2021-04-03T03:03:24.402 INFO:teuthology.task.ansible.out:7c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.412 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.414 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aM 2021-04-03T03:03:24.416 INFO:teuthology.task.ansible.out:f33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e595 2021-04-03T03:03:24.417 INFO:teuthology.task.ansible.out:3db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacf 2021-04-03T03:03:24.419 INFO:teuthology.task.ansible.out:fd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.429 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.430 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.431 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.431 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1L 2021-04-03T03:03:24.433 INFO:teuthology.task.ansible.out:jLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.442 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.444 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.445 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.445 INFO:teuthology.task.ansible.out:skipping: [gibba031. 2021-04-03T03:03:24.446 INFO:teuthology.task.ansible.out:front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.469 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.085) 0:00:48.623 ******** 2021-04-03T03:03:24.476 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.477 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.479 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.481 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.481 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.482 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZh 2021-04-03T03:03:24.483 INFO:teuthology.task.ansible.out:mBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.494 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.495 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.497 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.498 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.500 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-04-03T03:03:24.502 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.502 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.503 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.503 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "it 2021-04-03T03:03:24.505 INFO:teuthology.task.ansible.out:em", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.514 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.516 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8 2021-04-03T03:03:24.517 INFO:teuthology.task.ansible.out:oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791c 2021-04-03T03:03:24.519 INFO:teuthology.task.ansible.out:ec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.529 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.531 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.531 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.532 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop K 2021-04-03T03:03:24.533 INFO:teuthology.task.ansible.out:D0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.534 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.544 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.546 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.546 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.547 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbs 2021-04-03T03:03:24.548 INFO:teuthology.task.ansible.out:slJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQ 2021-04-03T03:03:24.550 INFO:teuthology.task.ansible.out:Tw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.559 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.561 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.561 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.562 INFO:teuthology.task.ansible.out:skipping: [gibba031.front. 2021-04-03T03:03:24.564 INFO:teuthology.task.ansible.out:sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4J 2021-04-03T03:03:24.566 INFO:teuthology.task.ansible.out:oonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.576 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.577 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.578 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, " 2021-04-03T03:03:24.579 INFO:teuthology.task.ansible.out:item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.590 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.592 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.592 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.593 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJ 2021-04-03T03:03:24.594 INFO:teuthology.task.ansible.out:J7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.595 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df43 2021-04-03T03:03:24.596 INFO:teuthology.task.ansible.out:3b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.607 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.609 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.609 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba 2021-04-03T03:03:24.611 INFO:teuthology.task.ansible.out:0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.620 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.622 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.622 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.624 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.635 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.636 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.637 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.637 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/ 2021-04-03T03:03:24.639 INFO:teuthology.task.ansible.out:1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.639 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0 2021-04-03T03:03:24.641 INFO:teuthology.task.ansible.out:135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.651 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.652 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.653 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.653 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item 2021-04-03T03:03:24.655 INFO:teuthology.task.ansible.out:": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.666 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.667 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.668 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.669 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAA 2021-04-03T03:03:24.670 INFO:teuthology.task.ansible.out:RY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.672 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.682 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.684 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.684 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.685 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", 2021-04-03T03:03:24.686 INFO:teuthology.task.ansible.out: "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.687 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.697 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.698 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.698 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.699 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.699 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.700 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsriv 2021-04-03T03:03:24.701 INFO:teuthology.task.ansible.out:as", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.702 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.702 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn' 2021-04-03T03:03:24.703 INFO:teuthology.task.ansible.out:: 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.713 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.714 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.715 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.716 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.717 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.717 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name 2021-04-03T03:03:24.719 INFO:teuthology.task.ansible.out:": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.728 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.730 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.730 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.730 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph 2021-04-03T03:03:24.732 INFO:teuthology.task.ansible.out:.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.743 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.744 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.745 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.746 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.746 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.764 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.765 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.766 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.766 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop 2021-04-03T03:03:24.768 INFO:teuthology.task.ansible.out:_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.791 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.322) 0:00:48.946 ******** 2021-04-03T03:03:24.800 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.800 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.802 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.802 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.803 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.803 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sC 2021-04-03T03:03:24.805 INFO:teuthology.task.ansible.out:d6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607 2021-04-03T03:03:24.806 INFO:teuthology.task.ansible.out:b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.807 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_ 2021-04-03T03:03:24.808 INFO:teuthology.task.ansible.out:loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.809 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.809 INFO:teuthology.task.ansible.out:sk 2021-04-03T03:03:24.811 INFO:teuthology.task.ansible.out:ipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.821 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.822 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.823 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.823 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c73895166 2021-04-03T03:03:24.825 INFO:teuthology.task.ansible.out:64692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87 2021-04-03T03:03:24.826 INFO:teuthology.task.ansible.out:e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.836 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.838 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.838 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.839 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHI 2021-04-03T03:03:24.840 INFO:teuthology.task.ansible.out:AxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.850 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.851 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.852 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.852 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedo 2021-04-03T03:03:24.854 INFO:teuthology.task.ansible.out:ra CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.863 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.865 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.865 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.865 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rf 2021-04-03T03:03:24.867 INFO:teuthology.task.ansible.out:lap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.876 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.878 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.878 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-04-03T03:03:24.879 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.879 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.880 INFO:teuthology.task.ansible.out:skipping: [gibba03 2021-04-03T03:03:24.881 INFO:teuthology.task.ansible.out:1.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.904 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:24 +0000 (0:00:00.112) 0:00:49.058 ******** 2021-04-03T03:03:24.911 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.912 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.914 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.914 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.916 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.917 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.918 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.929 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.930 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.931 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.931 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", 2021-04-03T03:03:24.933 INFO:teuthology.task.ansible.out:"ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.944 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.945 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.946 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.947 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.958 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.960 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.960 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.961 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jf 2021-04-03T03:03:24.963 INFO:teuthology.task.ansible.out:suselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.972 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.973 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.974 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.974 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7 2021-04-03T03:03:24.976 INFO:teuthology.task.ansible.out:be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.986 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.987 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.988 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:24.988 INFO:teuthology.task.ansible.out:skipping: [gib 2021-04-03T03:03:24.990 INFO:teuthology.task.ansible.out:ba026.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.000 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.001 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.002 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 2021-04-03T03:03:25.003 INFO:teuthology.task.ansible.out:'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.013 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.015 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.015 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.016 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "ch 2021-04-03T03:03:25.017 INFO:teuthology.task.ansible.out:anged": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.027 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.029 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.040 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.041 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.042 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.042 INFO:teuthology.task.ansible.out:skipping: [gibba028.fr 2021-04-03T03:03:25.044 INFO:teuthology.task.ansible.out:ont.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.053 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.054 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.055 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.055 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh 2021-04-03T03:03:25.057 INFO:teuthology.task.ansible.out:1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.066 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.068 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.068 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:25.069 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-04-03T03:03:25.069 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-04-03T03:03:25.069 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb0 2021-04-03T03:03:25.071 INFO:teuthology.task.ansible.out:98dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.081 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.083 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.083 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.084 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adki 2021-04-03T03:03:25.085 INFO:teuthology.task.ansible.out:ng@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.095 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.096 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.097 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Cond 2021-04-03T03:03:25.098 INFO:teuthology.task.ansible.out:itional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.099 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.127 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.129 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.129 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.130 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark 2021-04-03T03:03:25.131 INFO:teuthology.task.ansible.out:", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.153 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.248) 0:00:49.307 ******** 2021-04-03T03:03:25.161 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.163 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.163 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.165 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.165 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.167 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba015.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-04-03T03:03:25.168 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.199 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.201 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.225 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.071) 0:00:49.379 ******** 2021-04-03T03:03:25.231 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.231 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.233 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.233 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.235 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} TASK [Filter the revoked_users list] ******************************************* skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.235 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.236 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.267 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.041) 0:00:49.421 ******** 2021-04-03T03:03:25.273 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.275 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.685914", "end": "2021-04-03 03:03:25.507338", "rc": 0, "start": "2021-04-03 03:03:24.821424", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:25.276 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:25.277 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-04-03T03:03:25.277 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.278 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.279 INFO:teuthology.task.ansible.out:skipping: [gibba026 2021-04-03T03:03:25.281 INFO:teuthology.task.ansible.out:.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.306 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.039) 0:00:49.460 ******** 2021-04-03T03:03:25.314 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.315 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.333 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-04-03T03:03:25.335 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.358 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.051) 0:00:49.512 ******** 2021-04-03T03:03:25.396 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.037) 0:00:49.550 ******** 2021-04-03T03:03:25.406 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:25.407 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-04-03T03:03:25.409 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-04-03T03:03:25.409 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.442 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.046) 0:00:49.596 ******** 2021-04-03T03:03:25.449 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:25.451 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} ok: [gibba015.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-04-03T03:03:25.452 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:25.476 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.033) 0:00:49.630 ******** 2021-04-03T03:03:25.537 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:25.539 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} ok: [gibba015.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-04-03T03:03:25.571 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-04-03T03:03:25.624 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-04-03T03:03:25.625 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-04-03T03:03:25.647 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-04-03T03:03:25.659 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-04-03T03:03:25.660 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.057609", "end": "2021-04-03 03:03:25.895387", "rc": 0, "start": "2021-04-03 03:03:25.837778", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [Remove revoked users] **************************************************** ok: [gibba031.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-04-03T03:03:25.684 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:25 +0000 (0:00:00.207) 0:00:49.838 ******** 2021-04-03T03:03:25.722 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-04-03T03:03:25.733 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-04-03T03:03:25.745 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-04-03T03:03:25.756 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-04-03T03:03:25.821 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} ok: [gibba015.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-04-03T03:03:25.843 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-04-03T03:03:25.844 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-04-03T03:03:25.921 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-04-03T03:03:25.933 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-04-03T03:03:25.934 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-04-03T03:03:25.957 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-04-03T03:03:26.011 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-04-03T03:03:26.023 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-04-03T03:03:26.025 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-04-03T03:03:26.056 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-04-03T03:03:26.109 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-04-03T03:03:26.121 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-04-03T03:03:26.123 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-04-03T03:03:26.145 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-04-03T03:03:26.199 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-04-03T03:03:26.210 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-04-03T03:03:26.211 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-04-03T03:03:26.243 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-04-03T03:03:26.297 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-04-03T03:03:26.298 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-04-03T03:03:26.309 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-04-03T03:03:26.331 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-04-03T03:03:26.395 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-04-03T03:03:26.396 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-04-03T03:03:26.428 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-04-03T03:03:26.439 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-04-03T03:03:26.482 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-04-03T03:03:26.493 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-04-03T03:03:26.516 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-04-03T03:03:26.569 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-04-03T03:03:26.580 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-04-03T03:03:26.581 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-04-03T03:03:26.604 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-04-03T03:03:26.678 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-04-03T03:03:26.679 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-04-03T03:03:26.680 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-04-03T03:03:26.734 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-04-03T03:03:26.766 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-04-03T03:03:26.767 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-04-03T03:03:26.768 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-04-03T03:03:26.821 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-04-03T03:03:26.853 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-04-03T03:03:26.865 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-04-03T03:03:26.866 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-04-03T03:03:26.909 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-04-03T03:03:26.941 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-04-03T03:03:26.953 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-04-03T03:03:26.964 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-04-03T03:03:26.997 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-04-03T03:03:27.050 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-04-03T03:03:27.061 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-04-03T03:03:27.062 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-04-03T03:03:27.095 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-04-03T03:03:27.139 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-04-03T03:03:27.151 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-04-03T03:03:27.163 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-04-03T03:03:27.185 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-04-03T03:03:27.238 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-04-03T03:03:27.239 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-04-03T03:03:27.251 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-04-03T03:03:27.284 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-04-03T03:03:27.338 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-04-03T03:03:27.339 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-04-03T03:03:27.351 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-04-03T03:03:27.383 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-04-03T03:03:27.426 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-04-03T03:03:27.438 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-04-03T03:03:27.448 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-04-03T03:03:27.471 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-04-03T03:03:27.525 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} ok: [gibba026.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-04-03T03:03:27.547 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-04-03T03:03:27.558 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-04-03T03:03:27.611 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-04-03T03:03:27.612 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-04-03T03:03:27.645 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-04-03T03:03:27.647 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-04-03T03:03:27.709 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-04-03T03:03:27.711 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-04-03T03:03:27.733 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-04-03T03:03:27.735 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-04-03T03:03:27.808 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-04-03T03:03:27.809 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-04-03T03:03:27.820 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-04-03T03:03:27.832 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-04-03T03:03:27.907 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-04-03T03:03:27.909 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} ok: [gibba028.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-04-03T03:03:27.930 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-04-03T03:03:27.994 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-04-03T03:03:28.005 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-04-03T03:03:28.006 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-04-03T03:03:28.017 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-04-03T03:03:28.092 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-04-03T03:03:28.103 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-04-03T03:03:28.104 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-04-03T03:03:28.115 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-04-03T03:03:28.180 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-04-03T03:03:28.202 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-04-03T03:03:28.204 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} ok: [gibba015.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-04-03T03:03:28.288 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-04-03T03:03:28.299 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-04-03T03:03:28.300 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-04-03T03:03:28.323 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-04-03T03:03:28.386 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-04-03T03:03:28.387 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-04-03T03:03:28.420 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-04-03T03:03:28.431 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-04-03T03:03:28.484 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-04-03T03:03:28.485 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-04-03T03:03:28.517 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-04-03T03:03:28.529 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-04-03T03:03:28.582 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-04-03T03:03:28.614 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-04-03T03:03:28.615 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-04-03T03:03:28.669 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-04-03T03:03:28.712 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-04-03T03:03:28.723 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-04-03T03:03:28.765 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-04-03T03:03:28.798 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-04-03T03:03:28.810 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-04-03T03:03:28.812 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-04-03T03:03:28.866 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-04-03T03:03:28.898 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-04-03T03:03:28.899 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:03:28.920 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:28 +0000 (0:00:03.236) 0:00:53.075 ******** 2021-04-03T03:03:28.927 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:28.928 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-04-03T03:03:28.950 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-04-03T03:03:28.961 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-04-03T03:03:29.046 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-04-03T03:03:29.152 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-04-03T03:03:29.206 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-04-03T03:03:29.248 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-04-03T03:03:29.270 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-04-03T03:03:29.292 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-04-03T03:03:29.325 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-04-03T03:03:29.388 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-04-03T03:03:29.484 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-04-03T03:03:29.537 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-04-03T03:03:29.539 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:03:29.560 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:29 +0000 (0:00:00.639) 0:00:53.714 ******** 2021-04-03T03:03:29.577 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:29.578 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-04-03T03:03:29.599 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-04-03T03:03:29.642 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:29.664 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:29 +0000 (0:00:00.104) 0:00:53.818 ******** 2021-04-03T03:03:29.681 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:29.683 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} changed: [gibba028.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-04-03T03:03:29.694 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-04-03T03:03:29.726 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-04-03T03:03:29.769 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-04-03T03:03:29.770 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-04-03T03:03:29.792 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-04-03T03:03:29.856 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-04-03T03:03:29.857 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-04-03T03:03:29.880 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-04-03T03:03:29.955 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-04-03T03:03:29.967 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-04-03T03:03:29.978 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-04-03T03:03:30.043 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-04-03T03:03:30.044 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-04-03T03:03:30.067 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-04-03T03:03:30.069 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-04-03T03:03:30.133 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-04-03T03:03:30.144 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-04-03T03:03:30.177 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-04-03T03:03:30.177 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-04-03T03:03:30.178 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-04-03T03:03:30.202 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.537) 0:00:54.356 ******** 2021-04-03T03:03:30.226 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.024) 0:00:54.380 ******** 2021-04-03T03:03:30.233 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:30.235 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-04-03T03:03:30.247 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-04-03T03:03:30.259 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-04-03T03:03:30.302 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-04-03T03:03:30.345 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-04-03T03:03:30.357 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-04-03T03:03:30.400 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-04-03T03:03:30.444 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-04-03T03:03:30.455 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-04-03T03:03:30.499 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-04-03T03:03:30.501 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "checksum": "53b31205b3cc9729fea6ce5ceab6366ba2fdabc0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "6cd40c867036c949fadb741414d81285", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419010.2979918-22950-26007726559545/source", "state": "file", "uid": 0} 2021-04-03T03:03:30.528 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.301) 0:00:54.682 ******** 2021-04-03T03:03:30.535 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:30.537 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-04-03T03:03:30.548 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-04-03T03:03:30.549 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-04-03T03:03:30.603 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.075) 0:00:54.757 ******** 2021-04-03T03:03:30.609 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-04-03T03:03:30.611 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => { "msg": "Host gibba015.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-04-03T03:03:30.637 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.033) 0:00:54.791 ******** 2021-04-03T03:03:30.664 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:30.666 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-04-03T03:03:30.687 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-04-03T03:03:30.728 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-04-03T03:03:30.730 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-04-03T03:03:30.760 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.123) 0:00:54.914 ******** 2021-04-03T03:03:30.782 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.021) 0:00:54.936 ******** 2021-04-03T03:03:30.788 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:30.790 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-04-03T03:03:30.823 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-04-03T03:03:30.824 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:03:30.847 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.065) 0:00:55.001 ******** 2021-04-03T03:03:30.855 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-04-03T03:03:30.857 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:30.881 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.033) 0:00:55.035 ******** 2021-04-03T03:03:30.887 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:30.889 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-04-03T03:03:30.900 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-04-03T03:03:30.901 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "checksum": "53b31205b3cc9729fea6ce5ceab6366ba2fdabc0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "6cd40c867036c949fadb741414d81285", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419010.6903949-22990-246016209366137/source", "state": "file", "uid": 0} 2021-04-03T03:03:30.929 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.048) 0:00:55.083 ******** 2021-04-03T03:03:30.946 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-04-03T03:03:30.947 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:30.973 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:30 +0000 (0:00:00.043) 0:00:55.127 ******** 2021-04-03T03:03:30.983 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:30.985 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-04-03T03:03:31.006 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-04-03T03:03:31.008 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => { "msg": "Host gibba031.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-04-03T03:03:31.042 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.068) 0:00:55.196 ******** 2021-04-03T03:03:31.048 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-04-03T03:03:31.049 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.074 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.032) 0:00:55.228 ******** 2021-04-03T03:03:31.084 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:31.086 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-04-03T03:03:31.119 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-04-03T03:03:31.121 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "checksum": "53b31205b3cc9729fea6ce5ceab6366ba2fdabc0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "6cd40c867036c949fadb741414d81285", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419010.8583953-23011-226644604403289/source", "state": "file", "uid": 0} 2021-04-03T03:03:31.145 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.070) 0:00:55.299 ******** 2021-04-03T03:03:31.151 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-04-03T03:03:31.153 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-04-03T03:03:31.154 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.189 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.044) 0:00:55.343 ******** 2021-04-03T03:03:31.210 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.020) 0:00:55.364 ******** 2021-04-03T03:03:31.216 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-04-03T03:03:31.218 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-04-03T03:03:31.240 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:31.242 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} TASK [common : Log the OS name, version and release] *************************** ok: [gibba028.front.sepia.ceph.com] => { "msg": "Host gibba028.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-04-03T03:03:31.267 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.057) 0:00:55.421 ******** 2021-04-03T03:03:31.288 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.021) 0:00:55.442 ******** 2021-04-03T03:03:31.295 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-04-03T03:03:31.297 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.323 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.034) 0:00:55.477 ******** 2021-04-03T03:03:31.344 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.021) 0:00:55.498 ******** 2021-04-03T03:03:31.360 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-04-03T03:03:31.362 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-04-03T03:03:31.386 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.042) 0:00:55.540 ******** 2021-04-03T03:03:31.392 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-04-03T03:03:31.394 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.418 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.031) 0:00:55.572 ******** 2021-04-03T03:03:31.424 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:31.426 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.451 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.033) 0:00:55.606 ******** 2021-04-03T03:03:31.457 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-04-03T03:03:31.459 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.483 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.031) 0:00:55.637 ******** 2021-04-03T03:03:31.490 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-04-03T03:03:31.492 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.519 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.035) 0:00:55.673 ******** 2021-04-03T03:03:31.527 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-04-03T03:03:31.528 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.556 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.037) 0:00:55.710 ******** 2021-04-03T03:03:31.573 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-04-03T03:03:31.575 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.599 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.042) 0:00:55.753 ******** 2021-04-03T03:03:31.605 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-04-03T03:03:31.606 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "checksum": "53b31205b3cc9729fea6ce5ceab6366ba2fdabc0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "6cd40c867036c949fadb741414d81285", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419011.3548229-23080-144158806556231/source", "state": "file", "uid": 0} TASK [common : Set entitlements_path] ****************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.632 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.033) 0:00:55.786 ******** 2021-04-03T03:03:31.653 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.020) 0:00:55.807 ******** 2021-04-03T03:03:31.660 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:31.662 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-04-03T03:03:31.663 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-04-03T03:03:31.665 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.690 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.037) 0:00:55.844 ******** 2021-04-03T03:03:31.713 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.022) 0:00:55.867 ******** 2021-04-03T03:03:31.718 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-04-03T03:03:31.720 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.745 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.032) 0:00:55.899 ******** 2021-04-03T03:03:31.747 INFO:teuthology.task.ansible.out: 2021-04-03T03:03:31.753 INFO:teuthology.task.ansible.out: 2021-04-03T03:03:31.755 INFO:teuthology.task.ansible.out:TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [gibba015.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba015.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:31.756 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [gibba026.front.sepia.ceph.com] => { "msg": "Host gibba026.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-04-03T03:03:31.781 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.035) 0:00:55.935 ******** 2021-04-03T03:03:31.807 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.026) 0:00:55.961 ******** 2021-04-03T03:03:31.813 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-04-03T03:03:31.815 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Create /usr/libexec.] ******************************************* ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:03:31.815 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-04-03T03:03:31.816 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.842 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.034) 0:00:55.996 ******** 2021-04-03T03:03:31.863 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.020) 0:00:56.017 ******** 2021-04-03T03:03:31.883 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.020) 0:00:56.037 ******** 2021-04-03T03:03:31.889 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-04-03T03:03:31.891 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Including major version specific variables.] ******************** ok: [gibba026.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-04-03T03:03:31.921 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.037) 0:00:56.075 ******** 2021-04-03T03:03:31.942 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.021) 0:00:56.096 ******** 2021-04-03T03:03:31.951 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-04-03T03:03:31.953 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:31.980 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.037) 0:00:56.134 ******** 2021-04-03T03:03:32.001 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:31 +0000 (0:00:00.020) 0:00:56.155 ******** 2021-04-03T03:03:32.007 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-04-03T03:03:32.009 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.035 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.034) 0:00:56.190 ******** 2021-04-03T03:03:32.057 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.021) 0:00:56.211 ******** 2021-04-03T03:03:32.063 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-04-03T03:03:32.064 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.089 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.032) 0:00:56.243 ******** 2021-04-03T03:03:32.109 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.020) 0:00:56.263 ******** 2021-04-03T03:03:32.115 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-04-03T03:03:32.117 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:32.142 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.032) 0:00:56.296 ******** 2021-04-03T03:03:32.149 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-04-03T03:03:32.151 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.151 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:32.152 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419011.9177542-23170-6881955639544/source", "state": "file", "uid": 0} 2021-04-03T03:03:32.180 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.038) 0:00:56.334 ******** 2021-04-03T03:03:32.186 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-04-03T03:03:32.187 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.221 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.041) 0:00:56.375 ******** 2021-04-03T03:03:32.243 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.021) 0:00:56.397 ******** 2021-04-03T03:03:32.251 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-04-03T03:03:32.252 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.283 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.040) 0:00:56.437 ******** 2021-04-03T03:03:32.315 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.032) 0:00:56.469 ******** 2021-04-03T03:03:32.324 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-04-03T03:03:32.326 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [gibba011.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-04-03T03:03:32.326 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-04-03T03:03:32.326 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:32.327 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:32.352 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.036) 0:00:56.506 ******** 2021-04-03T03:03:32.373 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.020) 0:00:56.527 ******** 2021-04-03T03:03:32.380 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-04-03T03:03:32.381 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:32.407 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.034) 0:00:56.561 ******** 2021-04-03T03:03:32.414 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-04-03T03:03:32.416 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.440 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.032) 0:00:56.594 ******** 2021-04-03T03:03:32.446 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-04-03T03:03:32.447 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba028.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set have_entitlements] ****************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.473 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.032) 0:00:56.627 ******** 2021-04-03T03:03:32.493 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.019) 0:00:56.647 ******** 2021-04-03T03:03:32.499 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-04-03T03:03:32.501 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.524 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.031) 0:00:56.678 ******** 2021-04-03T03:03:32.531 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:32.533 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-04-03T03:03:32.535 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-04-03T03:03:32.536 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.560 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.035) 0:00:56.714 ******** 2021-04-03T03:03:32.581 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.020) 0:00:56.735 ******** 2021-04-03T03:03:32.642 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-04-03T03:03:32.644 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.668 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.086) 0:00:56.822 ******** 2021-04-03T03:03:32.674 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-04-03T03:03:32.676 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.703 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.035) 0:00:56.857 ******** 2021-04-03T03:03:32.710 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.837 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.133) 0:00:56.991 ******** 2021-04-03T03:03:32.845 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-04-03T03:03:32.847 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.872 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.034) 0:00:57.026 ******** 2021-04-03T03:03:32.892 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.020) 0:00:57.046 ******** 2021-04-03T03:03:32.913 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.021) 0:00:57.067 ******** 2021-04-03T03:03:32.920 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-04-03T03:03:32.922 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:32.947 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.033) 0:00:57.101 ******** 2021-04-03T03:03:32.968 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.020) 0:00:57.122 ******** 2021-04-03T03:03:32.974 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-04-03T03:03:32.976 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.000 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:32 +0000 (0:00:00.031) 0:00:57.154 ******** 2021-04-03T03:03:33.021 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.175 ******** 2021-04-03T03:03:33.027 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-04-03T03:03:33.029 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.053 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.032) 0:00:57.207 ******** 2021-04-03T03:03:33.074 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.228 ******** 2021-04-03T03:03:33.079 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-04-03T03:03:33.081 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Check for epel] ************************************************* skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:33.105 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.031) 0:00:57.259 ******** 2021-04-03T03:03:33.126 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.280 ******** 2021-04-03T03:03:33.132 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-04-03T03:03:33.134 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [gibba015.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:33.162 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.035) 0:00:57.316 ******** 2021-04-03T03:03:33.183 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.021) 0:00:57.337 ******** 2021-04-03T03:03:33.191 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-04-03T03:03:33.193 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install nrpe without epel] ************************************** skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.218 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.034) 0:00:57.372 ******** 2021-04-03T03:03:33.224 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.226 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [gibba031.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install nrpe without epel] ************************************** skipping: [gibba011.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.228 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.252 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.034) 0:00:57.406 ******** 2021-04-03T03:03:33.273 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.427 ******** 2021-04-03T03:03:33.294 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.448 ******** 2021-04-03T03:03:33.299 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-04-03T03:03:33.301 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba026.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Register with subscription-manager.] **************************** skipping: [gibba028.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:33.327 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.033) 0:00:57.481 ******** 2021-04-03T03:03:33.348 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.501 ******** 2021-04-03T03:03:33.354 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-04-03T03:03:33.355 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.380 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.032) 0:00:57.534 ******** 2021-04-03T03:03:33.401 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.555 ******** 2021-04-03T03:03:33.407 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-04-03T03:03:33.408 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.434 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.033) 0:00:57.588 ******** 2021-04-03T03:03:33.455 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.609 ******** 2021-04-03T03:03:33.482 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.027) 0:00:57.636 ******** 2021-04-03T03:03:33.489 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-04-03T03:03:33.490 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.515 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.032) 0:00:57.669 ******** 2021-04-03T03:03:33.521 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-04-03T03:03:33.523 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.548 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.033) 0:00:57.702 ******** 2021-04-03T03:03:33.569 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.021) 0:00:57.723 ******** 2021-04-03T03:03:33.575 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.601 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.032) 0:00:57.755 ******** 2021-04-03T03:03:33.607 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-04-03T03:03:33.609 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.633 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.031) 0:00:57.787 ******** 2021-04-03T03:03:33.639 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-04-03T03:03:33.640 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.665 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.031) 0:00:57.819 ******** 2021-04-03T03:03:33.684 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.019) 0:00:57.838 ******** 2021-04-03T03:03:33.691 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.716 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.031) 0:00:57.870 ******** 2021-04-03T03:03:33.722 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-04-03T03:03:33.724 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.748 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.031) 0:00:57.902 ******** 2021-04-03T03:03:33.754 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-04-03T03:03:33.756 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.780 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.032) 0:00:57.934 ******** 2021-04-03T03:03:33.801 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.020) 0:00:57.955 ******** 2021-04-03T03:03:33.808 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.837 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.035) 0:00:57.991 ******** 2021-04-03T03:03:33.843 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-04-03T03:03:33.845 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:33.869 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.032) 0:00:58.023 ******** 2021-04-03T03:03:33.876 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-04-03T03:03:33.878 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [gibba028.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:33.903 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.033) 0:00:58.057 ******** 2021-04-03T03:03:33.925 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.021) 0:00:58.079 ******** 2021-04-03T03:03:33.931 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:33.957 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.032) 0:00:58.111 ******** 2021-04-03T03:03:33.963 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-04-03T03:03:33.965 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:33.989 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:33 +0000 (0:00:00.031) 0:00:58.143 ******** 2021-04-03T03:03:33.994 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-04-03T03:03:33.996 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.021 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:58.175 ******** 2021-04-03T03:03:34.040 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.019) 0:00:58.194 ******** 2021-04-03T03:03:34.047 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.048 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-04-03T03:03:34.048 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:03:34.050 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.074 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.033) 0:00:58.228 ******** 2021-04-03T03:03:34.095 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.020) 0:00:58.249 ******** 2021-04-03T03:03:34.117 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.022) 0:00:58.271 ******** 2021-04-03T03:03:34.123 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-04-03T03:03:34.125 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.149 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:58.303 ******** 2021-04-03T03:03:34.177 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.027) 0:00:58.331 ******** 2021-04-03T03:03:34.184 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-04-03T03:03:34.185 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.211 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.034) 0:00:58.365 ******** 2021-04-03T03:03:34.218 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-04-03T03:03:34.220 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.245 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.033) 0:00:58.399 ******** 2021-04-03T03:03:34.251 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-04-03T03:03:34.253 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.278 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:58.432 ******** 2021-04-03T03:03:34.298 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.019) 0:00:58.452 ******** 2021-04-03T03:03:34.304 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-04-03T03:03:34.306 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.330 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:58.484 ******** 2021-04-03T03:03:34.336 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-04-03T03:03:34.338 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.362 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.031) 0:00:58.516 ******** 2021-04-03T03:03:34.368 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-04-03T03:03:34.369 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-04-03T03:03:34.394 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:58.548 ******** 2021-04-03T03:03:34.416 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.022) 0:00:58.570 ******** 2021-04-03T03:03:34.423 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-04-03T03:03:34.424 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.425 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.450 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.034) 0:00:58.604 ******** 2021-04-03T03:03:34.472 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.021) 0:00:58.626 ******** 2021-04-03T03:03:34.478 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-04-03T03:03:34.480 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.504 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.031) 0:00:58.658 ******** 2021-04-03T03:03:34.509 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-04-03T03:03:34.511 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:34.540 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.036) 0:00:58.694 ******** 2021-04-03T03:03:34.547 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-04-03T03:03:34.549 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.549 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-04-03T03:03:34.574 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.033) 0:00:58.728 ******** 2021-04-03T03:03:34.594 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.020) 0:00:58.748 ******** 2021-04-03T03:03:34.602 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-04-03T03:03:34.603 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.628 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.033) 0:00:58.782 ******** 2021-04-03T03:03:34.649 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.021) 0:00:58.803 ******** 2021-04-03T03:03:34.655 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-04-03T03:03:34.657 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios check_mem script] ********************************* ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-04-03T03:03:34.682 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:58.836 ******** 2021-04-03T03:03:34.703 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.020) 0:00:58.857 ******** 2021-04-03T03:03:34.709 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-04-03T03:03:34.710 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.735 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.031) 0:00:58.889 ******** 2021-04-03T03:03:34.755 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.020) 0:00:58.909 ******** 2021-04-03T03:03:34.761 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-04-03T03:03:34.763 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.787 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:58.941 ******** 2021-04-03T03:03:34.803 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-04-03T03:03:34.805 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.828 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.040) 0:00:58.982 ******** 2021-04-03T03:03:34.834 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-04-03T03:03:34.836 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:34.859 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.031) 0:00:59.013 ******** 2021-04-03T03:03:34.865 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:34.867 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} TASK [common : Get the current timezone] *************************************** ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.003102", "end": "2021-04-03 03:03:35.033840", "rc": 0, "start": "2021-04-03 03:03:35.030738", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-04-03T03:03:34.868 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [gibba031.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:34.897 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.037) 0:00:59.051 ******** 2021-04-03T03:03:34.918 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.020) 0:00:59.072 ******** 2021-04-03T03:03:34.925 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-04-03T03:03:34.927 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.951 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.032) 0:00:59.105 ******** 2021-04-03T03:03:34.958 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:34.960 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-04-03T03:03:34.961 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:34.986 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.034) 0:00:59.140 ******** 2021-04-03T03:03:35.005 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:34 +0000 (0:00:00.019) 0:00:59.159 ******** 2021-04-03T03:03:35.012 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-04-03T03:03:35.013 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.038 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.033) 0:00:59.192 ******** 2021-04-03T03:03:35.060 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.021) 0:00:59.214 ******** 2021-04-03T03:03:35.081 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.021) 0:00:59.235 ******** 2021-04-03T03:03:35.087 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-04-03T03:03:35.089 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.114 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.032) 0:00:59.268 ******** 2021-04-03T03:03:35.135 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.020) 0:00:59.289 ******** 2021-04-03T03:03:35.141 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-04-03T03:03:35.143 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.166 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.031) 0:00:59.320 ******** 2021-04-03T03:03:35.186 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.019) 0:00:59.340 ******** 2021-04-03T03:03:35.193 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-04-03T03:03:35.194 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Check firewalld status] ***************************************** skipping: [gibba011.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:35.222 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.035) 0:00:59.376 ******** 2021-04-03T03:03:35.243 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.021) 0:00:59.397 ******** 2021-04-03T03:03:35.250 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-04-03T03:03:35.252 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Mask sleep units] *********************************************** skipping: [gibba015.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.277 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.034) 0:00:59.431 ******** 2021-04-03T03:03:35.284 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-04-03T03:03:35.286 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Mask sleep units] *********************************************** skipping: [gibba015.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.287 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.288 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-04-03T03:03:35.288 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.312 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.034) 0:00:59.466 ******** 2021-04-03T03:03:35.333 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.020) 0:00:59.487 ******** 2021-04-03T03:03:35.354 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.021) 0:00:59.508 ******** 2021-04-03T03:03:35.360 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-04-03T03:03:35.362 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.362 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.388 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.034) 0:00:59.542 ******** 2021-04-03T03:03:35.420 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.031) 0:00:59.574 ******** 2021-04-03T03:03:35.426 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.428 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.452 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.032) 0:00:59.606 ******** 2021-04-03T03:03:35.474 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.021) 0:00:59.628 ******** 2021-04-03T03:03:35.480 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-04-03T03:03:35.482 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.508 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.034) 0:00:59.662 ******** 2021-04-03T03:03:35.514 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-04-03T03:03:35.516 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.541 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.032) 0:00:59.695 ******** 2021-04-03T03:03:35.547 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-04-03T03:03:35.549 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.578 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.037) 0:00:59.732 ******** 2021-04-03T03:03:35.599 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.020) 0:00:59.752 ******** 2021-04-03T03:03:35.607 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-04-03T03:03:35.608 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [gibba031.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.610 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.610 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.638 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.039) 0:00:59.792 ******** 2021-04-03T03:03:35.659 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.021) 0:00:59.813 ******** 2021-04-03T03:03:35.681 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.021) 0:00:59.835 ******** 2021-04-03T03:03:35.687 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-04-03T03:03:35.688 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.714 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.032) 0:00:59.868 ******** 2021-04-03T03:03:35.734 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.020) 0:00:59.888 ******** 2021-04-03T03:03:35.752 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-04-03T03:03:35.753 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.777 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.043) 0:00:59.931 ******** 2021-04-03T03:03:35.783 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-04-03T03:03:35.785 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.809 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.032) 0:00:59.964 ******** 2021-04-03T03:03:35.817 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-04-03T03:03:35.818 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.008045", "end": "2021-04-03 03:03:35.647286", "rc": 0, "start": "2021-04-03 03:03:35.639241", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.820 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.846 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.036) 0:01:00.000 ******** 2021-04-03T03:03:35.866 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.019) 0:01:00.020 ******** 2021-04-03T03:03:35.883 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-04-03T03:03:35.885 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.907 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.041) 0:01:00.061 ******** 2021-04-03T03:03:35.924 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-04-03T03:03:35.925 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.956 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.049) 0:01:00.110 ******** 2021-04-03T03:03:35.963 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-04-03T03:03:35.965 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.004789", "end": "2021-04-03 03:03:36.234160", "rc": 0, "start": "2021-04-03 03:03:36.229371", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:35.990 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:35 +0000 (0:00:00.033) 0:01:00.144 ******** 2021-04-03T03:03:36.011 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.021) 0:01:00.165 ******** 2021-04-03T03:03:36.018 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-04-03T03:03:36.019 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.044 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.032) 0:01:00.198 ******** 2021-04-03T03:03:36.060 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-04-03T03:03:36.062 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.085 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.041) 0:01:00.239 ******** 2021-04-03T03:03:36.091 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-04-03T03:03:36.093 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.117 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.031) 0:01:00.271 ******** 2021-04-03T03:03:36.153 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-04-03T03:03:36.155 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.164 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.166 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.190 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.073) 0:01:00.344 ******** 2021-04-03T03:03:36.196 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-04-03T03:03:36.198 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.006921", "end": "2021-04-03 03:03:36.173991", "rc": 0, "start": "2021-04-03 03:03:36.167070", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-04-03T03:03:36.200 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-04-03T03:03:36.201 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:00 UTC", "ActiveEnterTimestampMonotonic": "18152385", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount systemd-tmpfiles-setup.service remote-fs.target systemd-journald.socket time-sync.target basic.target system.slice -.mount sysinit.target nss-lookup.target local-fs.target network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:00 UTC", "AssertTimestampMonotonic": "18151854", "Before": "getty@tty1.service xdm.service plymouth-quit.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:00 UTC", "ConditionTimestampMonotonic": "18151854", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-serv 2021-04-03T03:03:36.203 INFO:teuthology.task.ansible.out:er.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1370", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:00 UTC", "ExecMainStartTimestampMonotonic": "18152372", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-04-03 03:02:00 UTC] ; stop_time=[n/a] ; pid=1370 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:00 UTC", "InactiveExitTimestampMonotonic": "18152385", "InvocationID": "4d5f4ad1a29440a7b89ff26b296521b6", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "in 2021-04-03T03:03:36.204 INFO:teuthology.task.ansible.out:finity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1370", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-04-03T03:03:36.206 INFO:teuthology.task.ansible.out:"RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:00 UTC", "StateChangeTimestampMonotonic": "18152385", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:00 UTC", "WatchdogTimestampMonotonic": "18152385", "WatchdogUSec": "0"}} 2021-04-03T03:03:36.227 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.036) 0:01:00.381 ******** 2021-04-03T03:03:36.248 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.021) 0:01:00.402 ******** 2021-04-03T03:03:36.270 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.021) 0:01:00.424 ******** 2021-04-03T03:03:36.275 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-04-03T03:03:36.277 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.301 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.031) 0:01:00.455 ******** 2021-04-03T03:03:36.307 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-04-03T03:03:36.309 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.338 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.036) 0:01:00.492 ******** 2021-04-03T03:03:36.345 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-04-03T03:03:36.347 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.322123", "end": "2021-04-03 03:03:36.509476", "rc": 0, "start": "2021-04-03 03:03:36.187353", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:36.347 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** skipping: [gibba031.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.372 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.034) 0:01:00.526 ******** 2021-04-03T03:03:36.393 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.020) 0:01:00.547 ******** 2021-04-03T03:03:36.399 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.401 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.402 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.428 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.035) 0:01:00.582 ******** 2021-04-03T03:03:36.449 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.020) 0:01:00.603 ******** 2021-04-03T03:03:36.455 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-04-03T03:03:36.456 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.481 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.031) 0:01:00.635 ******** 2021-04-03T03:03:36.501 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.020) 0:01:00.655 ******** 2021-04-03T03:03:36.508 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-04-03T03:03:36.509 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.534 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.032) 0:01:00.688 ******** 2021-04-03T03:03:36.557 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.022) 0:01:00.711 ******** 2021-04-03T03:03:36.564 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-04-03T03:03:36.565 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.591 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.033) 0:01:00.745 ******** 2021-04-03T03:03:36.614 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.022) 0:01:00.768 ******** 2021-04-03T03:03:36.621 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-04-03T03:03:36.622 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.634 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.635 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-04-03T03:03:36.635 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.660 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.046) 0:01:00.814 ******** 2021-04-03T03:03:36.689 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.029) 0:01:00.843 ******** 2021-04-03T03:03:36.697 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-04-03T03:03:36.698 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.724 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.034) 0:01:00.878 ******** 2021-04-03T03:03:36.747 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.023) 0:01:00.901 ******** 2021-04-03T03:03:36.754 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-04-03T03:03:36.755 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.046564", "end": "2021-04-03 03:03:36.871654", "rc": 0, "start": "2021-04-03 03:03:36.825090", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:36.757 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.783 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.036) 0:01:00.938 ******** 2021-04-03T03:03:36.806 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.022) 0:01:00.960 ******** 2021-04-03T03:03:36.829 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.022) 0:01:00.983 ******** 2021-04-03T03:03:36.835 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-04-03T03:03:36.837 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.863 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.034) 0:01:01.017 ******** 2021-04-03T03:03:36.886 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.022) 0:01:01.040 ******** 2021-04-03T03:03:36.892 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-04-03T03:03:36.894 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.919 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.032) 0:01:01.073 ******** 2021-04-03T03:03:36.946 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-04-03T03:03:36.947 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:36.971 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.052) 0:01:01.125 ******** 2021-04-03T03:03:36.992 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:36 +0000 (0:00:00.021) 0:01:01.146 ******** 2021-04-03T03:03:36.998 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-04-03T03:03:36.999 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.023 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.030) 0:01:01.177 ******** 2021-04-03T03:03:37.040 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-04-03T03:03:37.041 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.068 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.045) 0:01:01.222 ******** 2021-04-03T03:03:37.075 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-04-03T03:03:37.077 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.101 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.032) 0:01:01.255 ******** 2021-04-03T03:03:37.107 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-04-03T03:03:37.109 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.133 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.032) 0:01:01.287 ******** 2021-04-03T03:03:37.139 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-04-03T03:03:37.140 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.163 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.030) 0:01:01.317 ******** 2021-04-03T03:03:37.190 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-04-03T03:03:37.192 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.214 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.051) 0:01:01.369 ******** 2021-04-03T03:03:37.221 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-04-03T03:03:37.222 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.005779", "end": "2021-04-03 03:03:37.504687", "rc": 0, "start": "2021-04-03 03:03:37.498908", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:37.246 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.031) 0:01:01.400 ******** 2021-04-03T03:03:37.251 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-04-03T03:03:37.253 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.277 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.030) 0:01:01.431 ******** 2021-04-03T03:03:37.314 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-04-03T03:03:37.316 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.339 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.061) 0:01:01.493 ******** 2021-04-03T03:03:37.344 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-04-03T03:03:37.345 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.369 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.029) 0:01:01.523 ******** 2021-04-03T03:03:37.447 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-04-03T03:03:37.449 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:37.470 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.101) 0:01:01.624 ******** 2021-04-03T03:03:37.517 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-04-03T03:03:37.519 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.216792", "end": "2021-04-03 03:03:37.505180", "rc": 0, "start": "2021-04-03 03:03:37.288388", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [common : Clean apt cache] ************************************************ changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.049474", "end": "2021-04-03 03:03:37.825481", "rc": 0, "start": "2021-04-03 03:03:37.776007", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:37.542 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.071) 0:01:01.696 ******** 2021-04-03T03:03:37.562 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.020) 0:01:01.716 ******** 2021-04-03T03:03:37.568 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-04-03T03:03:37.570 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-04-03T03:03:37.598 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.036) 0:01:01.753 ******** 2021-04-03T03:03:37.615 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-04-03T03:03:37.617 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.640 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.041) 0:01:01.794 ******** 2021-04-03T03:03:37.667 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-04-03T03:03:37.669 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.748354", "end": "2021-04-03 03:03:37.507341", "rc": 0, "start": "2021-04-03 03:03:36.758987", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:37.692 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.051) 0:01:01.846 ******** 2021-04-03T03:03:37.699 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-04-03T03:03:37.701 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-04-03T03:03:37.726 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.033) 0:01:01.880 ******** 2021-04-03T03:03:37.753 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-04-03T03:03:37.754 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:37.777 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.051) 0:01:01.931 ******** 2021-04-03T03:03:37.804 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-04-03T03:03:37.806 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-04-03T03:03:37.829 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.051) 0:01:01.983 ******** 2021-04-03T03:03:37.856 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-04-03T03:03:37.858 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.047742", "end": "2021-04-03 03:03:37.820151", "rc": 0, "start": "2021-04-03 03:03:37.772409", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:37.882 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.053) 0:01:02.036 ******** 2021-04-03T03:03:37.920 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-04-03T03:03:37.922 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-04-03T03:03:37.944 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:37 +0000 (0:00:00.062) 0:01:02.098 ******** 2021-04-03T03:03:37.983 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-04-03T03:03:37.984 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.047538", "end": "2021-04-03 03:03:37.808237", "rc": 0, "start": "2021-04-03 03:03:37.760699", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:38.008 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:38 +0000 (0:00:00.063) 0:01:02.162 ******** 2021-04-03T03:03:38.098 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-04-03T03:03:38.099 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "name": "ubuntu", "state": "present", "system": false} 2021-04-03T03:03:38.121 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:38 +0000 (0:00:00.113) 0:01:02.276 ******** 2021-04-03T03:03:38.190 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-04-03T03:03:38.192 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:38.212 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:38 +0000 (0:00:00.090) 0:01:02.366 ******** 2021-04-03T03:03:38.281 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-04-03T03:03:38.283 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:38.303 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:38 +0000 (0:00:00.091) 0:01:02.457 ******** 2021-04-03T03:03:38.465 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-04-03T03:03:38.467 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-04-03T03:03:38.492 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:38 +0000 (0:00:00.189) 0:01:02.647 ******** 2021-04-03T03:03:38.821 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-04-03T03:03:38.823 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-04-03T03:03:38.845 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:38 +0000 (0:00:00.352) 0:01:02.999 ******** 2021-04-03T03:03:38.913 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-04-03T03:03:38.915 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba011"}, "changed": false} 2021-04-03T03:03:38.937 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:38 +0000 (0:00:00.092) 0:01:03.091 ******** 2021-04-03T03:03:39.592 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-04-03T03:03:39.594 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:03:39.616 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:39 +0000 (0:00:00.678) 0:01:03.770 ******** 2021-04-03T03:03:39.657 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-04-03T03:03:39.659 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba011.front.sepia.ceph.com", "ansible_hostname": "gibba011", "ansible_nodename": "gibba011"}, "changed": true, "name": "gibba011"} 2021-04-03T03:03:39.693 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:39 +0000 (0:00:00.077) 0:01:03.847 ******** 2021-04-03T03:03:39.953 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-04-03T03:03:39.955 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-04-03T03:03:39.977 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:39 +0000 (0:00:00.283) 0:01:04.131 ******** 2021-04-03T03:03:40.243 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-04-03T03:03:40.245 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-04-03T03:03:40.267 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.290) 0:01:04.421 ******** 2021-04-03T03:03:40.274 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-04-03T03:03:40.275 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:03:40.304 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.036) 0:01:04.458 ******** 2021-04-03T03:03:40.311 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-04-03T03:03:40.313 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:03:40.337 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.032) 0:01:04.491 ******** 2021-04-03T03:03:40.407 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:40.440 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.103) 0:01:04.594 ******** 2021-04-03T03:03:40.714 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-04-03T03:03:40.715 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-04-03T03:03:40.738 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.297) 0:01:04.892 ******** 2021-04-03T03:03:40.744 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-04-03T03:03:40.746 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-04-03T03:03:40.770 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.032) 0:01:04.924 ******** 2021-04-03T03:03:40.799 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-04-03T03:03:40.801 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:40.826 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.055) 0:01:04.980 ******** 2021-04-03T03:03:40.895 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:03:40.896 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:40.918 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.092) 0:01:05.072 ******** 2021-04-03T03:03:40.946 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-04-03T03:03:40.947 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:03:40.971 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.052) 0:01:05.125 ******** 2021-04-03T03:03:40.977 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-04-03T03:03:40.979 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:03:41.003 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:40 +0000 (0:00:00.032) 0:01:05.157 ******** 2021-04-03T03:03:41.012 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-04-03T03:03:41.014 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.039 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.035) 0:01:05.193 ******** 2021-04-03T03:03:41.077 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-04-03T03:03:41.078 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.101 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.061) 0:01:05.255 ******** 2021-04-03T03:03:41.117 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:03:41.119 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.148 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.046) 0:01:05.302 ******** 2021-04-03T03:03:41.155 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-04-03T03:03:41.157 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:03:41.180 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.032) 0:01:05.335 ******** 2021-04-03T03:03:41.208 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:03:41.210 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.234 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.053) 0:01:05.388 ******** 2021-04-03T03:03:41.295 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-04-03T03:03:41.297 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.326 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.091) 0:01:05.480 ******** 2021-04-03T03:03:41.353 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-04-03T03:03:41.354 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-04-03T03:03:41.378 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.052) 0:01:05.532 ******** 2021-04-03T03:03:41.394 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-04-03T03:03:41.396 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.420 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.041) 0:01:05.574 ******** 2021-04-03T03:03:41.426 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-04-03T03:03:41.427 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:03:41.452 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.031) 0:01:05.606 ******** 2021-04-03T03:03:41.586 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-04-03T03:03:41.587 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} TASK [testnode : Configure local mirrorlists] ********************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.610 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.158) 0:01:05.764 ******** 2021-04-03T03:03:41.631 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.020) 0:01:05.785 ******** 2021-04-03T03:03:41.780 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-04-03T03:03:41.805 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.174) 0:01:05.959 ******** 2021-04-03T03:03:41.812 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-04-03T03:03:41.814 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:41.840 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.034) 0:01:05.994 ******** 2021-04-03T03:03:41.861 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.021) 0:01:06.015 ******** 2021-04-03T03:03:41.868 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:41.869 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-04-03T03:03:41.975 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-04-03T03:03:42.005 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:41 +0000 (0:00:00.144) 0:01:06.159 ******** 2021-04-03T03:03:42.011 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-04-03T03:03:42.014 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:03:42.135 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.130) 0:01:06.289 ******** 2021-04-03T03:03:42.144 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-04-03T03:03:42.145 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [gibba015.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-04-03T03:03:42.171 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.035) 0:01:06.325 ******** 2021-04-03T03:03:42.178 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-04-03T03:03:42.204 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.032) 0:01:06.358 ******** 2021-04-03T03:03:42.233 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-04-03T03:03:42.235 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-04-03T03:03:42.261 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.056) 0:01:06.415 ******** 2021-04-03T03:03:42.267 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:42.269 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-04-03T03:03:42.296 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.035) 0:01:06.450 ******** 2021-04-03T03:03:42.304 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-04-03T03:03:42.306 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:42.333 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.037) 0:01:06.487 ******** 2021-04-03T03:03:42.384 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-04-03T03:03:42.386 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:03:42.412 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.078) 0:01:06.566 ******** 2021-04-03T03:03:42.418 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-04-03T03:03:42.420 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:42.448 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.036) 0:01:06.602 ******** 2021-04-03T03:03:42.454 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:42.456 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} TASK [common : Add krb5 config file] ******************************************* 2021-04-03T03:03:42.456 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-04-03T03:03:42.481 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.032) 0:01:06.635 ******** 2021-04-03T03:03:42.518 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-04-03T03:03:42.520 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:42.543 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.062) 0:01:06.697 ******** 2021-04-03T03:03:42.613 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:42.615 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [gibba031.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-04-03T03:03:42.638 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.094) 0:01:06.792 ******** 2021-04-03T03:03:42.697 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:42.699 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419022.4919224-24054-166423878883832/source", "state": "file", "uid": 0} 2021-04-03T03:03:42.708 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [gibba 2021-04-03T03:03:42.710 INFO:teuthology.task.ansible.out:011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:42.736 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.097) 0:01:06.890 ******** 2021-04-03T03:03:42.795 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:42.796 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-04-03T03:03:42.821 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.085) 0:01:06.975 ******** 2021-04-03T03:03:42.828 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-04-03T03:03:42.830 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:42.854 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.032) 0:01:07.008 ******** 2021-04-03T03:03:42.881 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:42.883 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-04-03T03:03:42.893 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-04-03T03:03:42.894 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:03:42.917 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.063) 0:01:07.071 ******** 2021-04-03T03:03:42.924 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-04-03T03:03:42.925 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:03:42.951 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.033) 0:01:07.105 ******** 2021-04-03T03:03:42.957 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-04-03T03:03:42.959 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:42.984 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:42 +0000 (0:00:00.033) 0:01:07.138 ******** 2021-04-03T03:03:43.054 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-04-03T03:03:43.056 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.078 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.093) 0:01:07.232 ******** 2021-04-03T03:03:43.085 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:43.087 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-04-03T03:03:43.114 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.035) 0:01:07.268 ******** 2021-04-03T03:03:43.130 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-04-03T03:03:43.132 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:03:43.156 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.042) 0:01:07.310 ******** 2021-04-03T03:03:43.163 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:03:43.165 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.189 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.032) 0:01:07.343 ******** 2021-04-03T03:03:43.195 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-04-03T03:03:43.197 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:43.221 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.032) 0:01:07.375 ******** 2021-04-03T03:03:43.238 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:43.240 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** changed: [gibba031.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419023.0318716-24115-81426642668168/source", "state": "file", "uid": 0} 2021-04-03T03:03:43.262 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.286 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.065) 0:01:07.440 ******** 2021-04-03T03:03:43.292 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-04-03T03:03:43.294 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.322 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.035) 0:01:07.476 ******** 2021-04-03T03:03:43.349 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:03:43.351 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.375 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.052) 0:01:07.529 ******** 2021-04-03T03:03:43.391 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-04-03T03:03:43.393 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:43.417 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.042) 0:01:07.571 ******** 2021-04-03T03:03:43.424 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:43.425 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-04-03T03:03:43.436 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-04-03T03:03:43.438 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.462 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.044) 0:01:07.616 ******** 2021-04-03T03:03:43.468 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:43.470 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-04-03T03:03:43.492 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-04-03T03:03:43.494 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [gibba028.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-04-03T03:03:43.504 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-04-03T03:03:43.506 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.516 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.540 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.078) 0:01:07.694 ******** 2021-04-03T03:03:43.546 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:03:43.548 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.572 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.031) 0:01:07.726 ******** 2021-04-03T03:03:43.612 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:43.614 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-04-03T03:03:43.617 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-04-03T03:03:43.619 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.644 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.072) 0:01:07.798 ******** 2021-04-03T03:03:43.664 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.019) 0:01:07.818 ******** 2021-04-03T03:03:43.670 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:43.672 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-04-03T03:03:43.673 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:03:43.675 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.699 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.035) 0:01:07.853 ******** 2021-04-03T03:03:43.726 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.026) 0:01:07.880 ******** 2021-04-03T03:03:43.735 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:43.736 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-04-03T03:03:43.738 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-04-03T03:03:43.738 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:43.765 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.039) 0:01:07.919 ******** 2021-04-03T03:03:43.781 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-04-03T03:03:43.784 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.812 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.046) 0:01:07.966 ******** 2021-04-03T03:03:43.829 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-04-03T03:03:43.831 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} TASK [common : Include yum_systems vars] *************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.854 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.042) 0:01:08.008 ******** 2021-04-03T03:03:43.877 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.022) 0:01:08.031 ******** 2021-04-03T03:03:43.884 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-04-03T03:03:43.886 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:43.910 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.033) 0:01:08.064 ******** 2021-04-03T03:03:43.917 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-04-03T03:03:43.919 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-04-03T03:03:43.947 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.036) 0:01:08.101 ******** 2021-04-03T03:03:43.955 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-04-03T03:03:43.957 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:43.983 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:43 +0000 (0:00:00.035) 0:01:08.137 ******** 2021-04-03T03:03:43.988 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-04-03T03:03:43.990 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.015 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.032) 0:01:08.169 ******** 2021-04-03T03:03:44.052 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-04-03T03:03:44.054 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} TASK [common : Install nrpe without epel] ************************************** skipping: [gibba031.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.079 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.063) 0:01:08.233 ******** 2021-04-03T03:03:44.085 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.087 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.113 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.033) 0:01:08.267 ******** 2021-04-03T03:03:44.119 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-04-03T03:03:44.121 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.147 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.033) 0:01:08.301 ******** 2021-04-03T03:03:44.175 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:44.176 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419023.951225-24238-94075348555104/source", "state": "file", "uid": 0} 2021-04-03T03:03:44.187 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-04-03T03:03:44.188 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.213 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.066) 0:01:08.367 ******** 2021-04-03T03:03:44.229 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-04-03T03:03:44.231 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.253 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.040) 0:01:08.408 ******** 2021-04-03T03:03:44.270 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-04-03T03:03:44.272 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:03:44.313 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.059) 0:01:08.467 ******** 2021-04-03T03:03:44.330 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-04-03T03:03:44.331 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.355 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.042) 0:01:08.509 ******** 2021-04-03T03:03:44.383 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:44.385 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-04-03T03:03:44.417 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419024.170858-24268-12527027593079/source", "state": "file", "uid": 0} 2021-04-03T03:03:44.428 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-04-03T03:03:44.430 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.452 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.096) 0:01:08.606 ******** 2021-04-03T03:03:44.510 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-04-03T03:03:44.511 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:44.534 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.082) 0:01:08.688 ******** 2021-04-03T03:03:44.593 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-04-03T03:03:44.595 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-04-03T03:03:44.619 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.084) 0:01:08.773 ******** 2021-04-03T03:03:44.626 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-04-03T03:03:44.627 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-04-03T03:03:44.629 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-04-03T03:03:44.656 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.036) 0:01:08.810 ******** 2021-04-03T03:03:44.673 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-04-03T03:03:44.675 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001328", "end": "2021-04-03 03:03:44.922684", "rc": 0, "start": "2021-04-03 03:03:44.921356", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:44.700 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.044) 0:01:08.855 ******** 2021-04-03T03:03:44.728 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-04-03T03:03:44.730 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:44.753 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.052) 0:01:08.907 ******** 2021-04-03T03:03:44.780 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-04-03T03:03:44.782 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [gibba028.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-04-03T03:03:44.809 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.055) 0:01:08.963 ******** 2021-04-03T03:03:44.834 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.024) 0:01:08.988 ******** 2021-04-03T03:03:44.841 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-04-03T03:03:44.843 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:03:44.868 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.033) 0:01:09.022 ******** 2021-04-03T03:03:44.896 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.027) 0:01:09.050 ******** 2021-04-03T03:03:44.903 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-04-03T03:03:44.904 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-04-03T03:03:44.929 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.033) 0:01:09.083 ******** 2021-04-03T03:03:44.935 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-04-03T03:03:44.937 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} TASK [common : Check for epel] ************************************************* skipping: [gibba026.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:44.963 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.033) 0:01:09.117 ******** 2021-04-03T03:03:44.984 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:44 +0000 (0:00:00.021) 0:01:09.138 ******** 2021-04-03T03:03:45.034 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-04-03T03:03:45.036 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.045 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.056 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.058 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install nrpe without epel] ************************************** skipping: [gibba026.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.081 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.096) 0:01:09.235 ******** 2021-04-03T03:03:45.102 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.020) 0:01:09.256 ******** 2021-04-03T03:03:45.108 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:45.110 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-04-03T03:03:45.121 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-04-03T03:03:45.123 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-04-03T03:03:45.148 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.046) 0:01:09.302 ******** 2021-04-03T03:03:45.155 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-04-03T03:03:45.157 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.181 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.033) 0:01:09.335 ******** 2021-04-03T03:03:45.187 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-04-03T03:03:45.189 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [gibba015.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-04-03T03:03:45.191 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [gibba028.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:45.216 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.035) 0:01:09.370 ******** 2021-04-03T03:03:45.236 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.019) 0:01:09.390 ******** 2021-04-03T03:03:45.257 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.020) 0:01:09.411 ******** 2021-04-03T03:03:45.262 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-04-03T03:03:45.264 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.288 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.031) 0:01:09.442 ******** 2021-04-03T03:03:45.305 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-04-03T03:03:45.307 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.335 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.046) 0:01:09.489 ******** 2021-04-03T03:03:45.341 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-04-03T03:03:45.343 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.343 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.345 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.370 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.035) 0:01:09.524 ******** 2021-04-03T03:03:45.408 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-04-03T03:03:45.410 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:45.433 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.062) 0:01:09.587 ******** 2021-04-03T03:03:45.439 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-04-03T03:03:45.441 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.467 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.033) 0:01:09.621 ******** 2021-04-03T03:03:45.494 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-04-03T03:03:45.496 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.520 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.053) 0:01:09.674 ******** 2021-04-03T03:03:45.528 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-04-03T03:03:45.529 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-04-03T03:03:45.556 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.035) 0:01:09.710 ******** 2021-04-03T03:03:45.709 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:45.711 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-04-03T03:03:45.784 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-04-03T03:03:45.809 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.252) 0:01:09.963 ******** 2021-04-03T03:03:45.815 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-04-03T03:03:45.816 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:03:45.842 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.032) 0:01:09.996 ******** 2021-04-03T03:03:45.863 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.021) 0:01:10.017 ******** 2021-04-03T03:03:45.880 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-04-03T03:03:45.882 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:45.905 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:45 +0000 (0:00:00.041) 0:01:10.059 ******** 2021-04-03T03:03:46.005 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-04-03T03:03:46.007 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:46.034 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.128) 0:01:10.188 ******** 2021-04-03T03:03:46.042 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:46.044 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-04-03T03:03:46.046 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-04-03T03:03:46.047 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ActiveEnterTimestampMonotonic": "18546813", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "time-sync.target nss-lookup.target systemd-tmpfiles-setup.service system.slice basic.target remote-fs.target network.target local-fs.target sysinit.target var-run.mount systemd-journald.socket -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:06 UTC", "AssertTimestampMonotonic": "18546276", "Before": "multi-user.target xdm.service getty@tty1.service plymouth-quit.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ConditionTimestampMonotonic": "18546276", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-serv 2021-04-03T03:03:46.049 INFO:teuthology.task.ansible.out:er.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1461", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ExecMainStartTimestampMonotonic": "18546798", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-04-03 03:02:06 UTC] ; stop_time=[n/a] ; pid=1461 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:06 UTC", "InactiveExitTimestampMonotonic": "18546813", "InvocationID": "51fbfa35a68446fd9323c94d8b8ab99d", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "in 2021-04-03T03:03:46.050 INFO:teuthology.task.ansible.out:finity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1461", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-04-03T03:03:46.052 INFO:teuthology.task.ansible.out:"RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:06 UTC", "StateChangeTimestampMonotonic": "18546813", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:06 UTC", "WatchdogTimestampMonotonic": "18546813", "WatchdogUSec": "0"}} 2021-04-03T03:03:46.074 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.039) 0:01:10.228 ******** 2021-04-03T03:03:46.101 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-04-03T03:03:46.103 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.130 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.056) 0:01:10.284 ******** 2021-04-03T03:03:46.152 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.021) 0:01:10.306 ******** 2021-04-03T03:03:46.158 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-04-03T03:03:46.160 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-04-03T03:03:46.186 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.034) 0:01:10.340 ******** 2021-04-03T03:03:46.215 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.028) 0:01:10.369 ******** 2021-04-03T03:03:46.254 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-04-03T03:03:46.255 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.284 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.069) 0:01:10.438 ******** 2021-04-03T03:03:46.357 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-04-03T03:03:46.359 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.382 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.098) 0:01:10.536 ******** 2021-04-03T03:03:46.391 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:46.393 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-04-03T03:03:46.418 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.035) 0:01:10.572 ******** 2021-04-03T03:03:46.424 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-04-03T03:03:46.426 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-04-03T03:03:46.451 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.032) 0:01:10.605 ******** 2021-04-03T03:03:46.457 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-04-03T03:03:46.459 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.490 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.039) 0:01:10.644 ******** 2021-04-03T03:03:46.513 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.023) 0:01:10.667 ******** 2021-04-03T03:03:46.519 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-04-03T03:03:46.521 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-04-03T03:03:46.550 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.037) 0:01:10.704 ******** 2021-04-03T03:03:46.556 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-04-03T03:03:46.558 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.583 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.033) 0:01:10.738 ******** 2021-04-03T03:03:46.653 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-04-03T03:03:46.655 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.677 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.093) 0:01:10.831 ******** 2021-04-03T03:03:46.716 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:46.717 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-04-03T03:03:46.729 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-04-03T03:03:46.730 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:25 UTC", "ActiveEnterTimestampMonotonic": "18426139", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target sysinit.target -.mount systemd-journald.socket systemd-tmpfiles-setup.service var-run.mount system.slice local-fs.target nss-lookup.target network.target remote-fs.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:25 UTC", "AssertTimestampMonotonic": "18425106", "Before": "plymouth-quit.service getty@tty1.service xdm.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:25 UTC", "ConditionTimestampMonotonic": "18425105", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-serv 2021-04-03T03:03:46.732 INFO:teuthology.task.ansible.out:er.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1240", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:25 UTC", "ExecMainStartTimestampMonotonic": "18426121", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-04-03 03:02:25 UTC] ; stop_time=[n/a] ; pid=1240 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:25 UTC", "InactiveExitTimestampMonotonic": "18426139", "InvocationID": "aad2c82be0564596a5b40774d1b88484", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "in 2021-04-03T03:03:46.733 INFO:teuthology.task.ansible.out:finity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1240", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-04-03T03:03:46.735 INFO:teuthology.task.ansible.out:"RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:25 UTC", "StateChangeTimestampMonotonic": "18426139", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:25 UTC", "WatchdogTimestampMonotonic": "18426138", "WatchdogUSec": "0"}} 2021-04-03T03:03:46.756 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.078) 0:01:10.910 ******** 2021-04-03T03:03:46.762 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-04-03T03:03:46.763 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.789 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.032) 0:01:10.942 ******** 2021-04-03T03:03:46.795 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-04-03T03:03:46.797 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [gibba026.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-04-03T03:03:46.824 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.035) 0:01:10.978 ******** 2021-04-03T03:03:46.849 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.025) 0:01:11.003 ******** 2021-04-03T03:03:46.858 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-04-03T03:03:46.859 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.897 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.047) 0:01:11.051 ******** 2021-04-03T03:03:46.918 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.020) 0:01:11.072 ******** 2021-04-03T03:03:46.924 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-04-03T03:03:46.925 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.949 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.031) 0:01:11.103 ******** 2021-04-03T03:03:46.966 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-04-03T03:03:46.968 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:46.990 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:46 +0000 (0:00:00.040) 0:01:11.144 ******** 2021-04-03T03:03:46.996 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-04-03T03:03:46.998 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.028 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.038) 0:01:11.182 ******** 2021-04-03T03:03:47.038 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:47.040 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} TASK [common : Check firewalld status] ***************************************** skipping: [gibba026.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:47.065 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.036) 0:01:11.219 ******** 2021-04-03T03:03:47.072 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-04-03T03:03:47.073 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.099 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.033) 0:01:11.253 ******** 2021-04-03T03:03:47.105 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:47.107 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} TASK [testnode : Install python-apt] ******************************************* ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:03:47.109 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-04-03T03:03:47.109 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-04-03T03:03:47.110 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.137 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.038) 0:01:11.291 ******** 2021-04-03T03:03:47.158 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.021) 0:01:11.312 ******** 2021-04-03T03:03:47.180 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.021) 0:01:11.334 ******** 2021-04-03T03:03:47.187 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-04-03T03:03:47.189 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include package type specific vars.] ************************** ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:47.215 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.034) 0:01:11.369 ******** 2021-04-03T03:03:47.237 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.021) 0:01:11.391 ******** 2021-04-03T03:03:47.243 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-04-03T03:03:47.245 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.270 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.033) 0:01:11.424 ******** 2021-04-03T03:03:47.291 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.021) 0:01:11.445 ******** 2021-04-03T03:03:47.297 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-04-03T03:03:47.299 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.323 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.032) 0:01:11.478 ******** 2021-04-03T03:03:47.330 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-04-03T03:03:47.332 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-04-03T03:03:47.357 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.033) 0:01:11.511 ******** 2021-04-03T03:03:47.363 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-04-03T03:03:47.365 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-04-03T03:03:47.393 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.035) 0:01:11.547 ******** 2021-04-03T03:03:47.411 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-04-03T03:03:47.412 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.439 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.045) 0:01:11.592 ******** 2021-04-03T03:03:47.446 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-04-03T03:03:47.448 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-04-03T03:03:47.448 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-04-03T03:03:47.449 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo 2021-04-03T03:03:47.449 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.013830", "end": "2021-04-03 03:03:47.680405", "rc": 0, "start": "2021-04-03 03:03:47.666575", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:47.475 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.036) 0:01:11.629 ******** 2021-04-03T03:03:47.498 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.022) 0:01:11.652 ******** 2021-04-03T03:03:47.507 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-04-03T03:03:47.509 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.535 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.036) 0:01:11.689 ******** 2021-04-03T03:03:47.558 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.022) 0:01:11.711 ******** 2021-04-03T03:03:47.565 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-04-03T03:03:47.566 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-04-03T03:03:47.592 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.034) 0:01:11.746 ******** 2021-04-03T03:03:47.601 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-04-03T03:03:47.603 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-04-03T03:03:47.628 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.035) 0:01:11.782 ******** 2021-04-03T03:03:47.634 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-04-03T03:03:47.635 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.660 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.031) 0:01:11.814 ******** 2021-04-03T03:03:47.731 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-04-03T03:03:47.733 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.755 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.094) 0:01:11.909 ******** 2021-04-03T03:03:47.775 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.020) 0:01:11.929 ******** 2021-04-03T03:03:47.792 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-04-03T03:03:47.794 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:12 UTC", "ActiveEnterTimestampMonotonic": "18487865", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "local-fs.target nss-lookup.target remote-fs.target basic.target var-run.mount -.mount time-sync.target sysinit.target network.target system.slice systemd-tmpfiles-setup.service systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:12 UTC", "AssertTimestampMonotonic": "18487108", "Before": "plymouth-quit.service xdm.service shutdown.target getty@tty1.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:12 UTC", "ConditionTimestampMonotonic": "18487108", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-serv 2021-04-03T03:03:47.795 INFO:teuthology.task.ansible.out:er.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1338", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:12 UTC", "ExecMainStartTimestampMonotonic": "18487851", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-04-03 03:02:12 UTC] ; stop_time=[n/a] ; pid=1338 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:12 UTC", "InactiveExitTimestampMonotonic": "18487865", "InvocationID": "88357260a38241309361568b8c96366c", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "in 2021-04-03T03:03:47.797 INFO:teuthology.task.ansible.out:finity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1338", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-04-03T03:03:47.798 INFO:teuthology.task.ansible.out:"RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:12 UTC", "StateChangeTimestampMonotonic": "18487865", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:12 UTC", "WatchdogTimestampMonotonic": "18487865", "WatchdogUSec": "0"}} 2021-04-03T03:03:47.823 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.048) 0:01:11.978 ******** 2021-04-03T03:03:47.831 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-04-03T03:03:47.832 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-04-03T03:03:47.857 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.033) 0:01:12.011 ******** 2021-04-03T03:03:47.864 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-04-03T03:03:47.866 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.894 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.036) 0:01:12.048 ******** 2021-04-03T03:03:47.900 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-04-03T03:03:47.901 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "name": "ubuntu", "state": "present", "system": false} 2021-04-03T03:03:47.926 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.032) 0:01:12.080 ******** 2021-04-03T03:03:47.934 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-04-03T03:03:47.936 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:47.971 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:47 +0000 (0:00:00.044) 0:01:12.125 ******** 2021-04-03T03:03:47.981 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-04-03T03:03:47.983 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.008 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.037) 0:01:12.162 ******** 2021-04-03T03:03:48.014 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-04-03T03:03:48.015 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.040 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.031) 0:01:12.194 ******** 2021-04-03T03:03:48.046 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-04-03T03:03:48.047 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.071 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.030) 0:01:12.225 ******** 2021-04-03T03:03:48.078 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-04-03T03:03:48.079 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:19 UTC", "ActiveEnterTimestampMonotonic": "18105076", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target nss-lookup.target systemd-tmpfiles-setup.service -.mount local-fs.target basic.target sysinit.target network.target var-run.mount time-sync.target systemd-journald.socket system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:19 UTC", "AssertTimestampMonotonic": "18103868", "Before": "getty@tty1.service shutdown.target xdm.service multi-user.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:19 UTC", "ConditionTimestampMonotonic": "18103867", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-serv 2021-04-03T03:03:48.081 INFO:teuthology.task.ansible.out:er.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1369", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:19 UTC", "ExecMainStartTimestampMonotonic": "18105056", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-04-03 03:02:19 UTC] ; stop_time=[n/a] ; pid=1369 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:19 UTC", "InactiveExitTimestampMonotonic": "18105076", "InvocationID": "2a28e0e1a31c4156bef2f68bf24638b4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "in 2021-04-03T03:03:48.082 INFO:teuthology.task.ansible.out:finity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1369", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-04-03T03:03:48.084 INFO:teuthology.task.ansible.out:"RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:19 UTC", "StateChangeTimestampMonotonic": "18105076", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:19 UTC", "WatchdogTimestampMonotonic": "18105074", "WatchdogUSec": "0"}} TASK [testnode : Include package type specific vars.] ************************** ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_ser 2021-04-03T03:03:48.085 INFO:teuthology.task.ansible.out:vice_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:48.106 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.035) 0:01:12.260 ******** 2021-04-03T03:03:48.126 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.019) 0:01:12.280 ******** 2021-04-03T03:03:48.132 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-04-03T03:03:48.134 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.158 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.032) 0:01:12.312 ******** 2021-04-03T03:03:48.165 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-04-03T03:03:48.167 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.190 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.031) 0:01:12.344 ******** 2021-04-03T03:03:48.197 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-04-03T03:03:48.198 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.226 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.036) 0:01:12.380 ******** 2021-04-03T03:03:48.234 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-04-03T03:03:48.235 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-04-03T03:03:48.264 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.037) 0:01:12.418 ******** 2021-04-03T03:03:48.270 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-04-03T03:03:48.271 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.296 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.032) 0:01:12.450 ******** 2021-04-03T03:03:48.302 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-04-03T03:03:48.304 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.330 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.033) 0:01:12.484 ******** 2021-04-03T03:03:48.347 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-04-03T03:03:48.349 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} TASK [testnode : Including major version specific variables.] ****************** ok: [gibba031.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-04-03T03:03:48.373 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.043) 0:01:12.527 ******** 2021-04-03T03:03:48.394 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.021) 0:01:12.548 ******** 2021-04-03T03:03:48.401 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-04-03T03:03:48.403 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.428 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.033) 0:01:12.582 ******** 2021-04-03T03:03:48.449 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.021) 0:01:12.603 ******** 2021-04-03T03:03:48.465 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-04-03T03:03:48.467 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-04-03T03:03:48.493 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.043) 0:01:12.647 ******** 2021-04-03T03:03:48.498 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-04-03T03:03:48.500 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.524 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.031) 0:01:12.678 ******** 2021-04-03T03:03:48.531 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.556 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.031) 0:01:12.710 ******** 2021-04-03T03:03:48.583 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-04-03T03:03:48.585 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-04-03T03:03:48.608 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.052) 0:01:12.762 ******** 2021-04-03T03:03:48.614 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-04-03T03:03:48.616 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.644 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.035) 0:01:12.798 ******** 2021-04-03T03:03:48.650 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-04-03T03:03:48.652 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.677 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.033) 0:01:12.831 ******** 2021-04-03T03:03:48.704 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-04-03T03:03:48.706 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-04-03T03:03:48.731 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.054) 0:01:12.885 ******** 2021-04-03T03:03:48.754 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.022) 0:01:12.908 ******** 2021-04-03T03:03:48.760 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-04-03T03:03:48.762 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "name": "ubuntu", "state": "present", "system": false} 2021-04-03T03:03:48.787 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.033) 0:01:12.941 ******** 2021-04-03T03:03:48.808 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.021) 0:01:12.962 ******** 2021-04-03T03:03:48.814 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-04-03T03:03:48.815 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba015"}, "changed": false} 2021-04-03T03:03:48.840 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.031) 0:01:12.994 ******** 2021-04-03T03:03:48.846 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.872 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.031) 0:01:13.026 ******** 2021-04-03T03:03:48.878 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-04-03T03:03:48.880 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the teuthology user.] ********************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.904 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.032) 0:01:13.058 ******** 2021-04-03T03:03:48.925 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.020) 0:01:13.079 ******** 2021-04-03T03:03:48.941 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-04-03T03:03:48.943 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.966 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.041) 0:01:13.120 ******** 2021-04-03T03:03:48.972 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-04-03T03:03:48.974 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:48.998 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:48 +0000 (0:00:00.031) 0:01:13.152 ******** 2021-04-03T03:03:49.004 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-04-03T03:03:49.006 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:49.034 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.036) 0:01:13.188 ******** 2021-04-03T03:03:49.041 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-04-03T03:03:49.043 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:49.067 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.032) 0:01:13.221 ******** 2021-04-03T03:03:49.074 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-04-03T03:03:49.076 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:49.100 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.032) 0:01:13.254 ******** 2021-04-03T03:03:49.138 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-04-03T03:03:49.140 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-04-03T03:03:49.164 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.064) 0:01:13.318 ******** 2021-04-03T03:03:49.170 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-04-03T03:03:49.171 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:49.196 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.031) 0:01:13.350 ******** 2021-04-03T03:03:49.202 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-04-03T03:03:49.204 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-04-03T03:03:49.229 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.032) 0:01:13.383 ******** 2021-04-03T03:03:49.235 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-04-03T03:03:49.237 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba015.front.sepia.ceph.com", "ansible_hostname": "gibba015", "ansible_nodename": "gibba015"}, "changed": true, "name": "gibba015"} 2021-04-03T03:03:49.263 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.033) 0:01:13.417 ******** 2021-04-03T03:03:49.271 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-04-03T03:03:49.273 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-04-03T03:03:49.274 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-04-03T03:03:49.299 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.035) 0:01:13.453 ******** 2021-04-03T03:03:49.319 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.020) 0:01:13.473 ******** 2021-04-03T03:03:49.380 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-04-03T03:03:49.382 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-04-03T03:03:49.405 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.085) 0:01:13.559 ******** 2021-04-03T03:03:49.411 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-04-03T03:03:49.413 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-04-03T03:03:49.439 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.033) 0:01:13.593 ******** 2021-04-03T03:03:49.457 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-04-03T03:03:49.459 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:03:49.482 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.043) 0:01:13.636 ******** 2021-04-03T03:03:49.499 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-04-03T03:03:49.501 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-04-03T03:03:49.530 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.047) 0:01:13.684 ******** 2021-04-03T03:03:49.539 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-04-03T03:03:49.540 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-04-03T03:03:49.572 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.042) 0:01:13.726 ******** 2021-04-03T03:03:49.581 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-04-03T03:03:49.583 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-04-03T03:03:49.584 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-04-03T03:03:49.584 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-04-03T03:03:49.610 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.038) 0:01:13.764 ******** 2021-04-03T03:03:49.634 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.023) 0:01:13.788 ******** 2021-04-03T03:03:49.652 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-04-03T03:03:49.654 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-04-03T03:03:49.678 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.043) 0:01:13.832 ******** 2021-04-03T03:03:49.684 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-04-03T03:03:49.686 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "name": "ubuntu", "state": "present", "system": false} 2021-04-03T03:03:49.711 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.032) 0:01:13.865 ******** 2021-04-03T03:03:49.717 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-04-03T03:03:49.719 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba031"}, "changed": false} 2021-04-03T03:03:49.744 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.032) 0:01:13.898 ******** 2021-04-03T03:03:49.781 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-04-03T03:03:49.783 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [testnode : Create the teuthology user.] ********************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:49.809 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.065) 0:01:13.963 ******** 2021-04-03T03:03:49.831 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.021) 0:01:13.985 ******** 2021-04-03T03:03:49.871 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-04-03T03:03:49.873 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-04-03T03:03:49.897 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.065) 0:01:14.051 ******** 2021-04-03T03:03:49.904 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-04-03T03:03:49.905 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:49.930 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.033) 0:01:14.084 ******** 2021-04-03T03:03:49.980 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-04-03T03:03:49.982 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "name": "ubuntu", "state": "present", "system": false} 2021-04-03T03:03:50.005 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:49 +0000 (0:00:00.074) 0:01:14.159 ******** 2021-04-03T03:03:50.075 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:50.104 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.098) 0:01:14.258 ******** 2021-04-03T03:03:50.111 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-04-03T03:03:50.113 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-04-03T03:03:50.139 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.035) 0:01:14.293 ******** 2021-04-03T03:03:50.146 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-04-03T03:03:50.148 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba031.front.sepia.ceph.com", "ansible_hostname": "gibba031", "ansible_nodename": "gibba031"}, "changed": true, "name": "gibba031"} 2021-04-03T03:03:50.174 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.035) 0:01:14.328 ******** 2021-04-03T03:03:50.182 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-04-03T03:03:50.183 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:50.209 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.034) 0:01:14.363 ******** 2021-04-03T03:03:50.346 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-04-03T03:03:50.348 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-04-03T03:03:50.371 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.162) 0:01:14.525 ******** 2021-04-03T03:03:50.378 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-04-03T03:03:50.380 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-04-03T03:03:50.405 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.033) 0:01:14.559 ******** 2021-04-03T03:03:50.432 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-04-03T03:03:50.434 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-04-03T03:03:50.459 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.053) 0:01:14.613 ******** 2021-04-03T03:03:50.465 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-04-03T03:03:50.466 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:50.491 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.032) 0:01:14.645 ******** 2021-04-03T03:03:50.656 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:03:50.657 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:50.681 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.190) 0:01:14.835 ******** 2021-04-03T03:03:50.689 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-04-03T03:03:50.691 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-04-03T03:03:50.716 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.034) 0:01:14.870 ******** 2021-04-03T03:03:50.851 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-04-03T03:03:50.875 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.159) 0:01:15.029 ******** 2021-04-03T03:03:50.896 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:50.909 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.033) 0:01:15.063 ******** 2021-04-03T03:03:50.917 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-04-03T03:03:50.918 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba026"}, "changed": false} TASK [testnode : Upload ssh_config] ******************************************** ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-04-03T03:03:50.949 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:50 +0000 (0:00:00.039) 0:01:15.103 ******** 2021-04-03T03:03:51.109 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.159) 0:01:15.263 ******** 2021-04-03T03:03:51.117 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-04-03T03:03:51.118 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba028"}, "changed": false} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.144 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.035) 0:01:15.298 ******** 2021-04-03T03:03:51.167 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.023) 0:01:15.321 ******** 2021-04-03T03:03:51.207 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:03:51.209 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.233 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.065) 0:01:15.387 ******** 2021-04-03T03:03:51.303 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-04-03T03:03:51.304 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.327 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.093) 0:01:15.481 ******** 2021-04-03T03:03:51.409 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-04-03T03:03:51.410 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.443 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.115) 0:01:15.597 ******** 2021-04-03T03:03:51.483 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-04-03T03:03:51.485 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba026.front.sepia.ceph.com", "ansible_hostname": "gibba026", "ansible_nodename": "gibba026"}, "changed": true, "name": "gibba026"} 2021-04-03T03:03:51.509 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.066) 0:01:15.663 ******** 2021-04-03T03:03:51.516 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-04-03T03:03:51.518 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-04-03T03:03:51.544 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.034) 0:01:15.698 ******** 2021-04-03T03:03:51.550 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-04-03T03:03:51.552 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.586 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.042) 0:01:15.740 ******** 2021-04-03T03:03:51.595 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-04-03T03:03:51.597 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba028.front.sepia.ceph.com", "ansible_hostname": "gibba028", "ansible_nodename": "gibba028"}, "changed": true, "name": "gibba028"} 2021-04-03T03:03:51.633 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.046) 0:01:15.787 ******** 2021-04-03T03:03:51.640 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-04-03T03:03:51.641 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.666 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.033) 0:01:15.820 ******** 2021-04-03T03:03:51.672 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-04-03T03:03:51.697 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.031) 0:01:15.851 ******** 2021-04-03T03:03:51.735 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:03:51.737 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.769 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.071) 0:01:15.923 ******** 2021-04-03T03:03:51.776 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-04-03T03:03:51.803 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.034) 0:01:15.957 ******** 2021-04-03T03:03:51.832 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-04-03T03:03:51.834 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-04-03T03:03:51.835 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:51.871 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.067) 0:01:16.025 ******** 2021-04-03T03:03:51.902 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.031) 0:01:16.056 ******** 2021-04-03T03:03:51.907 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-04-03T03:03:51.933 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.031) 0:01:16.087 ******** 2021-04-03T03:03:51.972 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:03:51.973 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-04-03T03:03:51.999 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:51 +0000 (0:00:00.065) 0:01:16.153 ******** 2021-04-03T03:03:52.021 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.021) 0:01:16.175 ******** 2021-04-03T03:03:52.026 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-04-03T03:03:52.028 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.053 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.031) 0:01:16.207 ******** 2021-04-03T03:03:52.091 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:03:52.092 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.121 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.067) 0:01:16.275 ******** 2021-04-03T03:03:52.138 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-04-03T03:03:52.139 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.141 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-04-03T03:03:52.167 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.046) 0:01:16.321 ******** 2021-04-03T03:03:52.198 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.030) 0:01:16.352 ******** 2021-04-03T03:03:52.205 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-04-03T03:03:52.207 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.232 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.034) 0:01:16.386 ******** 2021-04-03T03:03:52.239 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-04-03T03:03:52.241 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.276 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.043) 0:01:16.430 ******** 2021-04-03T03:03:52.285 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-04-03T03:03:52.286 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-04-03T03:03:52.312 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.035) 0:01:16.466 ******** 2021-04-03T03:03:52.320 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-04-03T03:03:52.322 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.347 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.035) 0:01:16.501 ******** 2021-04-03T03:03:52.354 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:03:52.356 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.380 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.032) 0:01:16.534 ******** 2021-04-03T03:03:52.412 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-04-03T03:03:52.413 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.449 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.068) 0:01:16.603 ******** 2021-04-03T03:03:52.469 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-04-03T03:03:52.471 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.495 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.046) 0:01:16.649 ******** 2021-04-03T03:03:52.523 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-04-03T03:03:52.548 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.052) 0:01:16.702 ******** 2021-04-03T03:03:52.564 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-04-03T03:03:52.566 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.592 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.043) 0:01:16.746 ******** 2021-04-03T03:03:52.610 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-04-03T03:03:52.612 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-04-03T03:03:52.637 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.044) 0:01:16.791 ******** 2021-04-03T03:03:52.659 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.022) 0:01:16.813 ******** 2021-04-03T03:03:52.665 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-04-03T03:03:52.667 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.692 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.032) 0:01:16.846 ******** 2021-04-03T03:03:52.719 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-04-03T03:03:52.721 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.748 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.056) 0:01:16.902 ******** 2021-04-03T03:03:52.755 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-04-03T03:03:52.756 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-04-03T03:03:52.757 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.789 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.040) 0:01:16.943 ******** 2021-04-03T03:03:52.812 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.022) 0:01:16.966 ******** 2021-04-03T03:03:52.819 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-04-03T03:03:52.821 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-04-03T03:03:52.851 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.039) 0:01:17.005 ******** 2021-04-03T03:03:52.858 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:03:52.860 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.884 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.032) 0:01:17.038 ******** 2021-04-03T03:03:52.889 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:03:52.891 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG] ************************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.915 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.031) 0:01:17.069 ******** 2021-04-03T03:03:52.937 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.021) 0:01:17.091 ******** 2021-04-03T03:03:52.945 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-04-03T03:03:52.947 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:52.983 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:52 +0000 (0:00:00.046) 0:01:17.137 ******** 2021-04-03T03:03:52.989 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-04-03T03:03:52.991 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.016 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.170 ******** 2021-04-03T03:03:53.038 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.022) 0:01:17.192 ******** 2021-04-03T03:03:53.046 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-04-03T03:03:53.047 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.076 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.037) 0:01:17.230 ******** 2021-04-03T03:03:53.082 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:03:53.083 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.108 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.262 ******** 2021-04-03T03:03:53.115 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:03:53.117 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.141 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.295 ******** 2021-04-03T03:03:53.168 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.026) 0:01:17.322 ******** 2021-04-03T03:03:53.175 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-04-03T03:03:53.177 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.201 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.033) 0:01:17.356 ******** 2021-04-03T03:03:53.208 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-04-03T03:03:53.209 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install epel packages] **************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.234 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.388 ******** 2021-04-03T03:03:53.256 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.021) 0:01:17.410 ******** 2021-04-03T03:03:53.262 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:03:53.264 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.289 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.443 ******** 2021-04-03T03:03:53.310 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.021) 0:01:17.464 ******** 2021-04-03T03:03:53.316 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:03:53.318 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.341 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.031) 0:01:17.495 ******** 2021-04-03T03:03:53.348 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:03:53.350 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.373 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.527 ******** 2021-04-03T03:03:53.380 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-04-03T03:03:53.382 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.406 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.560 ******** 2021-04-03T03:03:53.427 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.020) 0:01:17.581 ******** 2021-04-03T03:03:53.433 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:03:53.434 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.459 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.613 ******** 2021-04-03T03:03:53.479 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.019) 0:01:17.633 ******** 2021-04-03T03:03:53.496 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-04-03T03:03:53.498 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.524 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.044) 0:01:17.678 ******** 2021-04-03T03:03:53.545 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.020) 0:01:17.699 ******** 2021-04-03T03:03:53.552 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-04-03T03:03:53.553 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.578 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.732 ******** 2021-04-03T03:03:53.600 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.021) 0:01:17.753 ******** 2021-04-03T03:03:53.606 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-04-03T03:03:53.607 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.632 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.786 ******** 2021-04-03T03:03:53.638 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-04-03T03:03:53.640 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.665 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.033) 0:01:17.819 ******** 2021-04-03T03:03:53.671 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-04-03T03:03:53.672 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.697 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.851 ******** 2021-04-03T03:03:53.718 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.020) 0:01:17.872 ******** 2021-04-03T03:03:53.724 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-04-03T03:03:53.750 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.031) 0:01:17.904 ******** 2021-04-03T03:03:53.756 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-04-03T03:03:53.758 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.782 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.936 ******** 2021-04-03T03:03:53.788 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-04-03T03:03:53.790 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure local mirrorlists] ********************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.815 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:17.969 ******** 2021-04-03T03:03:53.835 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.019) 0:01:17.989 ******** 2021-04-03T03:03:53.841 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-04-03T03:03:53.873 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.038) 0:01:18.027 ******** 2021-04-03T03:03:53.880 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:03:53.881 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.881 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-04-03T03:03:53.881 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:53.906 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.033) 0:01:18.060 ******** 2021-04-03T03:03:53.929 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.022) 0:01:18.083 ******** 2021-04-03T03:03:53.935 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-04-03T03:03:53.936 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-04-03T03:03:53.961 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.032) 0:01:18.115 ******** 2021-04-03T03:03:53.982 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:53 +0000 (0:00:00.020) 0:01:18.136 ******** 2021-04-03T03:03:53.988 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-04-03T03:03:53.990 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.014 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:18.168 ******** 2021-04-03T03:03:54.021 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-04-03T03:03:54.023 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.048 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.033) 0:01:18.202 ******** 2021-04-03T03:03:54.055 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-04-03T03:03:54.057 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-04-03T03:03:54.093 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.044) 0:01:18.246 ******** 2021-04-03T03:03:54.126 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.033) 0:01:18.280 ******** 2021-04-03T03:03:54.134 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-04-03T03:03:54.136 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.160 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.033) 0:01:18.314 ******** 2021-04-03T03:03:54.167 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:03:54.169 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG keys] ********************************************* skipping: [gibba026.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.170 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.195 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.034) 0:01:18.349 ******** 2021-04-03T03:03:54.201 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-04-03T03:03:54.228 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.033) 0:01:18.382 ******** 2021-04-03T03:03:54.256 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.027) 0:01:18.410 ******** 2021-04-03T03:03:54.262 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-04-03T03:03:54.264 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install epel packages] **************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.290 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.034) 0:01:18.444 ******** 2021-04-03T03:03:54.321 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.030) 0:01:18.475 ******** 2021-04-03T03:03:54.328 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-04-03T03:03:54.329 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG] ************************************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.355 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.033) 0:01:18.509 ******** 2021-04-03T03:03:54.378 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.023) 0:01:18.532 ******** 2021-04-03T03:03:54.385 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-04-03T03:03:54.387 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.419 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.040) 0:01:18.573 ******** 2021-04-03T03:03:54.425 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:03:54.427 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.452 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:18.606 ******** 2021-04-03T03:03:54.474 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.021) 0:01:18.628 ******** 2021-04-03T03:03:54.480 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-04-03T03:03:54.482 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.511 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.036) 0:01:18.665 ******** 2021-04-03T03:03:54.539 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-04-03T03:03:54.540 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.564 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.053) 0:01:18.718 ******** 2021-04-03T03:03:54.570 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:03:54.571 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001251", "end": "2021-04-03 03:03:54.736013", "rc": 0, "start": "2021-04-03 03:03:54.734762", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:54.596 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.031) 0:01:18.750 ******** 2021-04-03T03:03:54.616 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.020) 0:01:18.770 ******** 2021-04-03T03:03:54.622 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-04-03T03:03:54.624 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.651 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.035) 0:01:18.805 ******** 2021-04-03T03:03:54.658 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-04-03T03:03:54.660 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.684 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:18.838 ******** 2021-04-03T03:03:54.691 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-04-03T03:03:54.692 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.717 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:18.871 ******** 2021-04-03T03:03:54.739 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.021) 0:01:18.893 ******** 2021-04-03T03:03:54.745 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:03:54.746 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.771 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:18.925 ******** 2021-04-03T03:03:54.777 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-04-03T03:03:54.779 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.803 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:18.957 ******** 2021-04-03T03:03:54.820 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-04-03T03:03:54.821 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.845 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.042) 0:01:18.999 ******** 2021-04-03T03:03:54.861 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-04-03T03:03:54.863 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.887 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.041) 0:01:19.041 ******** 2021-04-03T03:03:54.893 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-04-03T03:03:54.894 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.919 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:19.073 ******** 2021-04-03T03:03:54.925 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-04-03T03:03:54.927 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.951 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:19.105 ******** 2021-04-03T03:03:54.957 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-04-03T03:03:54.959 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:54.984 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:54 +0000 (0:00:00.032) 0:01:19.138 ******** 2021-04-03T03:03:54.991 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-04-03T03:03:54.993 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.018 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.033) 0:01:19.172 ******** 2021-04-03T03:03:55.047 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.028) 0:01:19.201 ******** 2021-04-03T03:03:55.054 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-04-03T03:03:55.056 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.081 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.034) 0:01:19.235 ******** 2021-04-03T03:03:55.088 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-04-03T03:03:55.089 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add sources list] ********************************************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.114 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.268 ******** 2021-04-03T03:03:55.135 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.021) 0:01:19.289 ******** 2021-04-03T03:03:55.142 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-04-03T03:03:55.143 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.168 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.322 ******** 2021-04-03T03:03:55.174 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-04-03T03:03:55.176 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.201 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.355 ******** 2021-04-03T03:03:55.206 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-04-03T03:03:55.208 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.233 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.387 ******** 2021-04-03T03:03:55.239 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:03:55.241 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.276 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.042) 0:01:19.430 ******** 2021-04-03T03:03:55.282 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-04-03T03:03:55.284 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.309 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.033) 0:01:19.463 ******** 2021-04-03T03:03:55.315 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:03:55.317 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.341 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.495 ******** 2021-04-03T03:03:55.347 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-04-03T03:03:55.349 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.373 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.527 ******** 2021-04-03T03:03:55.379 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-04-03T03:03:55.381 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.405 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.559 ******** 2021-04-03T03:03:55.422 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-04-03T03:03:55.424 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.451 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.045) 0:01:19.605 ******** 2021-04-03T03:03:55.458 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:03:55.460 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.485 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.034) 0:01:19.639 ******** 2021-04-03T03:03:55.492 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-04-03T03:03:55.494 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.519 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.033) 0:01:19.673 ******** 2021-04-03T03:03:55.524 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:03:55.526 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.551 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.705 ******** 2021-04-03T03:03:55.557 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:03:55.559 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.583 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.737 ******** 2021-04-03T03:03:55.590 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:55.592 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-04-03T03:03:55.612 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-04-03T03:03:55.614 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.637 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.053) 0:01:19.791 ******** 2021-04-03T03:03:55.654 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-04-03T03:03:55.656 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.679 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.041) 0:01:19.833 ******** 2021-04-03T03:03:55.685 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-04-03T03:03:55.687 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001345", "end": "2021-04-03 03:03:55.959993", "rc": 0, "start": "2021-04-03 03:03:55.958648", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:55.711 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.032) 0:01:19.865 ******** 2021-04-03T03:03:55.717 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:03:55.719 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.743 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.031) 0:01:19.897 ******** 2021-04-03T03:03:55.749 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-04-03T03:03:55.750 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.774 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.030) 0:01:19.928 ******** 2021-04-03T03:03:55.780 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-04-03T03:03:55.806 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.031) 0:01:19.960 ******** 2021-04-03T03:03:55.811 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:03:55.813 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.837 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.031) 0:01:19.991 ******** 2021-04-03T03:03:55.843 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-04-03T03:03:55.845 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.871 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.034) 0:01:20.025 ******** 2021-04-03T03:03:55.899 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-04-03T03:03:55.900 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:55.923 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.051) 0:01:20.077 ******** 2021-04-03T03:03:55.939 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-04-03T03:03:55.941 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install apt keys] ********************************************* ok: [gibba015.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-04-03T03:03:55.964 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.041) 0:01:20.118 ******** 2021-04-03T03:03:55.986 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:55 +0000 (0:00:00.021) 0:01:20.140 ******** 2021-04-03T03:03:55.993 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-04-03T03:03:55.994 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.019 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.032) 0:01:20.173 ******** 2021-04-03T03:03:56.056 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-04-03T03:03:56.058 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.081 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.062) 0:01:20.235 ******** 2021-04-03T03:03:56.087 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-04-03T03:03:56.089 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.113 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.032) 0:01:20.267 ******** 2021-04-03T03:03:56.120 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-04-03T03:03:56.121 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-04-03T03:03:56.146 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.033) 0:01:20.300 ******** 2021-04-03T03:03:56.163 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-04-03T03:03:56.165 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.188 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.041) 0:01:20.342 ******** 2021-04-03T03:03:56.194 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-04-03T03:03:56.196 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.220 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.032) 0:01:20.374 ******** 2021-04-03T03:03:56.226 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-04-03T03:03:56.228 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.253 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.032) 0:01:20.407 ******** 2021-04-03T03:03:56.258 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-04-03T03:03:56.260 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.284 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.030) 0:01:20.438 ******** 2021-04-03T03:03:56.301 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-04-03T03:03:56.303 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.333 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.049) 0:01:20.487 ******** 2021-04-03T03:03:56.372 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-04-03T03:03:56.374 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.397 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.063) 0:01:20.551 ******** 2021-04-03T03:03:56.403 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-04-03T03:03:56.405 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.430 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.032) 0:01:20.584 ******** 2021-04-03T03:03:56.488 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-04-03T03:03:56.490 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.512 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.082) 0:01:20.666 ******** 2021-04-03T03:03:56.539 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-04-03T03:03:56.541 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001314", "end": "2021-04-03 03:03:56.520896", "rc": 0, "start": "2021-04-03 03:03:56.519582", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:56.565 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.052) 0:01:20.719 ******** 2021-04-03T03:03:56.582 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-04-03T03:03:56.584 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:56.615 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.049) 0:01:20.769 ******** 2021-04-03T03:03:56.621 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-04-03T03:03:56.623 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:03:56.647 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.032) 0:01:20.801 ******** 2021-04-03T03:03:56.656 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-04-03T03:03:56.694 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.046) 0:01:20.848 ******** 2021-04-03T03:03:56.725 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:56.727 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} TASK [testnode : Add local apt repos.] ***************************************** 2021-04-03T03:03:56.758 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.064) 0:01:20.912 ******** 2021-04-03T03:03:56.765 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-04-03T03:03:56.766 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001305", "end": "2021-04-03 03:03:56.609399", "rc": 0, "start": "2021-04-03 03:03:56.608094", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:56.792 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.034) 0:01:20.946 ******** 2021-04-03T03:03:56.862 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-04-03T03:03:56.885 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.093) 0:01:21.039 ******** 2021-04-03T03:03:56.912 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-04-03T03:03:56.914 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.013874", "end": "2021-04-03 03:03:57.080591", "rc": 0, "start": "2021-04-03 03:03:57.066717", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:56.937 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.051) 0:01:21.091 ******** 2021-04-03T03:03:56.954 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-04-03T03:03:56.956 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-04-03T03:03:56.988 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:56 +0000 (0:00:00.051) 0:01:21.142 ******** 2021-04-03T03:03:56.997 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-04-03T03:03:56.999 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137601 files and directorie 2021-04-03T03:03:57.001 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Rea 2021-04-03T03:03:57.002 INFO:teuthology.task.ansible.out:ding database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137601 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:03:57.027 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:57 +0000 (0:00:00.038) 0:01:21.181 ******** 2021-04-03T03:03:57.055 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-04-03T03:03:57.057 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:57.081 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:57 +0000 (0:00:00.054) 0:01:21.235 ******** 2021-04-03T03:03:57.098 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:57.100 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-04-03T03:03:57.126 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:57 +0000 (0:00:00.044) 0:01:21.280 ******** 2021-04-03T03:03:57.142 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-04-03T03:03:57.144 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-04-03T03:03:57.168 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:57 +0000 (0:00:00.042) 0:01:21.322 ******** 2021-04-03T03:03:57.238 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-04-03T03:03:57.240 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:03:57.263 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:57 +0000 (0:00:00.094) 0:01:21.417 ******** 2021-04-03T03:03:57.793 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-04-03T03:03:57.794 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:03:57.828 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:57 +0000 (0:00:00.564) 0:01:21.982 ******** 2021-04-03T03:03:57.887 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:57.889 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-04-03T03:03:57.909 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-04-03T03:03:57.933 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:57 +0000 (0:00:00.104) 0:01:22.087 ******** 2021-04-03T03:03:57.980 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:57.982 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-04-03T03:03:58.076 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-04-03T03:03:58.077 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.013858", "end": "2021-04-03 03:03:58.369655", "rc": 0, "start": "2021-04-03 03:03:58.355797", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:58.099 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:58 +0000 (0:00:00.166) 0:01:22.253 ******** 2021-04-03T03:03:58.577 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-04-03T03:03:58.579 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-04-03T03:03:58.606 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:58 +0000 (0:00:00.507) 0:01:22.760 ******** 2021-04-03T03:03:58.698 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-04-03T03:03:58.723 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:58 +0000 (0:00:00.116) 0:01:22.877 ******** 2021-04-03T03:03:58.909 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-04-03T03:03:58.911 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:03:58.933 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:58 +0000 (0:00:00.210) 0:01:23.087 ******** 2021-04-03T03:03:59.268 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-04-03T03:03:59.270 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:03:59.291 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:59 +0000 (0:00:00.358) 0:01:23.446 ******** 2021-04-03T03:03:59.350 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:03:59.352 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-04-03T03:03:59.374 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:59 +0000 (0:00:00.082) 0:01:23.528 ******** 2021-04-03T03:03:59.395 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:59 +0000 (0:00:00.020) 0:01:23.549 ******** 2021-04-03T03:03:59.486 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:59 +0000 (0:00:00.091) 0:01:23.640 ******** 2021-04-03T03:03:59.522 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-04-03T03:03:59.524 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.013974", "end": "2021-04-03 03:03:59.510948", "rc": 0, "start": "2021-04-03 03:03:59.496974", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:59.547 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:59 +0000 (0:00:00.061) 0:01:23.701 ******** 2021-04-03T03:03:59.638 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.013694", "end": "2021-04-03 03:03:59.480939", "rc": 0, "start": "2021-04-03 03:03:59.467245", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:03:59.661 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:03:59 +0000 (0:00:00.113) 0:01:23.815 ******** 2021-04-03T03:04:00.054 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-04-03T03:04:00.056 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:04:00.078 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:00 +0000 (0:00:00.417) 0:01:24.232 ******** 2021-04-03T03:04:01.554 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:04:01.577 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:01 +0000 (0:00:01.498) 0:01:25.731 ******** 2021-04-03T03:04:01.647 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:04:01.671 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:01 +0000 (0:00:00.094) 0:01:25.825 ******** 2021-04-03T03:04:03.667 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:03.669 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apt-utils libapt-pkg5.0\nSuggested packages:\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apt apt-utils libapt-pkg5.0\n3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.\nNeed to get 2214 kB of archives.\nAfter this operation, 0 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt amd64 1.6.12ubuntu0.2 [1201 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]\nFetched 2214 kB in 1s (2627 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 2021-04-03T03:04:03.671 INFO:teuthology.task.ansible.out:10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 7 2021-04-03T03:04:03.672 INFO:teuthology.task.ansible.out:5%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt-utils (1.6.12ubuntu0.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apt-utils libapt-pkg5.0", "Suggested packages:", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apt apt-utils libapt-pkg5.0", "3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.", "Need to get 2214 kB of archives.", "After this operation, 0 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt 2021-04-03T03:04:03.674 INFO:teuthology.task.ansible.out:amd64 1.6.12ubuntu0.2 [1201 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]", "Fetched 2214 kB in 1s (2627 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Rea 2021-04-03T03:04:03.676 INFO:teuthology.task.ansible.out:ding database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt-utils (1.6.12ubuntu0.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:03.692 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:03 +0000 (0:00:02.021) 0:01:27.847 ******** 2021-04-03T03:04:03.764 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-04-03T03:04:03.766 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:03.788 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:03 +0000 (0:00:00.095) 0:01:27.942 ******** 2021-04-03T03:04:05.993 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-04-03T03:04:05.995 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137601 files and directorie 2021-04-03T03:04:05.996 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Rea 2021-04-03T03:04:05.998 INFO:teuthology.task.ansible.out:ding database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137601 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:06.018 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:06 +0000 (0:00:02.230) 0:01:30.172 ******** 2021-04-03T03:04:07.241 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137601 files and directorie 2021-04-03T03:04:07.243 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Rea 2021-04-03T03:04:07.245 INFO:teuthology.task.ansible.out:ding database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137601 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:07.266 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:07 +0000 (0:00:01.248) 0:01:31.420 ******** 2021-04-03T03:04:09.763 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137601 files and directorie 2021-04-03T03:04:09.764 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Rea 2021-04-03T03:04:09.766 INFO:teuthology.task.ansible.out:ding database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137601 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:09.789 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:09 +0000 (0:00:02.522) 0:01:33.943 ******** 2021-04-03T03:04:10.394 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137601 files and directorie 2021-04-03T03:04:10.396 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 133 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Rea 2021-04-03T03:04:10.397 INFO:teuthology.task.ansible.out:ding database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137601 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:10.418 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:10 +0000 (0:00:00.629) 0:01:34.572 ******** 2021-04-03T03:04:11.864 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:11.865 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 14.3 MB in 1s (12.6 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database .. 2021-04-03T03:04:11.866 INFO:teuthology.task.ansible.out:. \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nPreparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...\r\nUnpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up git (1:2.17.1-1ubuntu0.8) ...\r\nSetti 2021-04-03T03:04:11.868 INFO:teuthology.task.ansible.out:ng up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2. 2021-04-03T03:04:11.870 INFO:teuthology.task.ansible.out:11+dfsg-1ubuntu7.36 [870 kB]", "Fetched 14.3 MB in 1s (12.6 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Preparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...", "Unpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1u 2021-04-03T03:04:11.871 INFO:teuthology.task.ansible.out:buntu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up git (1:2.17.1-1ubuntu0.8) ...", "Setting up l 2021-04-03T03:04:11.872 INFO:teuthology.task.ansible.out:ibrbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:11.896 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:11 +0000 (0:00:01.477) 0:01:36.050 ******** 2021-04-03T03:04:12.521 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-04-03T03:04:12.522 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:04:12.544 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:12 +0000 (0:00:00.647) 0:01:36.698 ******** 2021-04-03T03:04:12.801 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:12.803 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apt-utils libapt-pkg5.0\nSuggested packages:\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apt apt-utils libapt-pkg5.0\n3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.\nNeed to get 2214 kB of archives.\nAfter this operation, 0 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt amd64 1.6.12ubuntu0.2 [1201 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]\nFetched 2214 kB in 1s (2616 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 2021-04-03T03:04:12.804 INFO:teuthology.task.ansible.out:10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 7 2021-04-03T03:04:12.806 INFO:teuthology.task.ansible.out:5%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt-utils (1.6.12ubuntu0.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apt-utils libapt-pkg5.0", "Suggested packages:", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apt apt-utils libapt-pkg5.0", "3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.", "Need to get 2214 kB of archives.", "After this operation, 0 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt 2021-04-03T03:04:12.807 INFO:teuthology.task.ansible.out:amd64 1.6.12ubuntu0.2 [1201 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]", "Fetched 2214 kB in 1s (2616 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Rea 2021-04-03T03:04:12.809 INFO:teuthology.task.ansible.out:ding database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt-utils (1.6.12ubuntu0.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:12.825 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:12 +0000 (0:00:00.281) 0:01:36.980 ******** 2021-04-03T03:04:12.894 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-04-03T03:04:12.896 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:12.917 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:12 +0000 (0:00:00.091) 0:01:37.071 ******** 2021-04-03T03:04:13.164 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-04-03T03:04:13.166 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:04:13.188 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:13 +0000 (0:00:00.270) 0:01:37.342 ******** 2021-04-03T03:04:13.320 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-04-03T03:04:13.321 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-04-03T03:04:13.345 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:13 +0000 (0:00:00.156) 0:01:37.499 ******** 2021-04-03T03:04:13.403 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-04-03T03:04:13.405 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:13.425 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:13 +0000 (0:00:00.080) 0:01:37.579 ******** 2021-04-03T03:04:13.556 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-04-03T03:04:13.558 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-04-03T03:04:13.579 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:13 +0000 (0:00:00.154) 0:01:37.733 ******** 2021-04-03T03:04:13.816 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:13.818 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apt-utils libapt-pkg5.0\nSuggested packages:\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apt apt-utils libapt-pkg5.0\n3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.\nNeed to get 2214 kB of archives.\nAfter this operation, 0 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt amd64 1.6.12ubuntu0.2 [1201 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]\nFetched 2214 kB in 1s (2630 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 2021-04-03T03:04:13.820 INFO:teuthology.task.ansible.out:10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 7 2021-04-03T03:04:13.822 INFO:teuthology.task.ansible.out:5%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt-utils (1.6.12ubuntu0.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apt-utils libapt-pkg5.0", "Suggested packages:", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apt apt-utils libapt-pkg5.0", "3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.", "Need to get 2214 kB of archives.", "After this operation, 0 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt 2021-04-03T03:04:13.823 INFO:teuthology.task.ansible.out:amd64 1.6.12ubuntu0.2 [1201 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]", "Fetched 2214 kB in 1s (2630 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Rea 2021-04-03T03:04:13.825 INFO:teuthology.task.ansible.out:ding database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt-utils (1.6.12ubuntu0.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:13.841 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:13 +0000 (0:00:00.261) 0:01:37.995 ******** 2021-04-03T03:04:13.847 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-04-03T03:04:13.848 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-04-03T03:04:13.873 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:13 +0000 (0:00:00.032) 0:01:38.027 ******** 2021-04-03T03:04:14.017 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-04-03T03:04:14.019 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:14.042 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.168) 0:01:38.196 ******** 2021-04-03T03:04:14.048 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-04-03T03:04:14.049 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1234, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-04-03T03:04:14.075 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.032) 0:01:38.229 ******** 2021-04-03T03:04:14.218 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-04-03T03:04:14.220 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:14.241 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.166) 0:01:38.395 ******** 2021-04-03T03:04:14.489 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-04-03T03:04:14.490 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:00 UTC", "ActiveEnterTimestampMonotonic": "18393168", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target system.slice nss-lookup.target systemd-tmpfiles-setup.service systemd-journald.socket -.mount basic.target sysinit.target remote-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:00 UTC", "AssertTimestampMonotonic": "18151065", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:00 UTC", "ConditionTimestampMonotonic": "18151065", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "D 2021-04-03T03:04:14.491 INFO:teuthology.task.ansible.out:ropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1552", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:00 UTC", "ExecMainStartTimestampMonotonic": "18393161", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:00 UTC", "InactiveExitTimestampMonotonic": "18151486", "InvocationID": "4c3ac44d6c91484598926f1fe3e3be1f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft 2021-04-03T03:04:14.493 INFO:teuthology.task.ansible.out:": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1552", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGH 2021-04-03T03:04:14.494 INFO:teuthology.task.ansible.out:UP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:00 UTC", "StateChangeTimestampMonotonic": "18393168", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:00 UTC", "WatchdogTimestampMonotonic": "18393167", "WatchdogUSec": "0"}} 2021-04-03T03:04:14.513 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.271) 0:01:38.667 ******** 2021-04-03T03:04:14.581 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:04:14.583 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:14.607 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.093) 0:01:38.761 ******** 2021-04-03T03:04:14.675 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-04-03T03:04:14.677 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:14.698 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.091) 0:01:38.852 ******** 2021-04-03T03:04:14.766 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:04:14.767 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:14.788 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.090) 0:01:38.943 ******** 2021-04-03T03:04:14.846 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:04:14.848 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:14.869 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.080) 0:01:39.023 ******** 2021-04-03T03:04:14.926 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-04-03T03:04:14.928 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:14.950 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:14 +0000 (0:00:00.081) 0:01:39.104 ******** 2021-04-03T03:04:15.018 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-04-03T03:04:15.020 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.041 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.090) 0:01:39.195 ******** 2021-04-03T03:04:15.215 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:04:15.217 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.238 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.197) 0:01:39.392 ******** 2021-04-03T03:04:15.413 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-04-03T03:04:15.414 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.436 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.197) 0:01:39.590 ******** 2021-04-03T03:04:15.599 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:15.601 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.622 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.186) 0:01:39.777 ******** 2021-04-03T03:04:15.691 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:04:15.692 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.714 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.091) 0:01:39.868 ******** 2021-04-03T03:04:15.783 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:15.785 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.805 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.090) 0:01:39.959 ******** 2021-04-03T03:04:15.865 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:15.866 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.897 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.092) 0:01:40.051 ******** 2021-04-03T03:04:15.956 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:15.958 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:15.978 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:15 +0000 (0:00:00.080) 0:01:40.132 ******** 2021-04-03T03:04:16.035 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:16.037 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:16.058 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:16 +0000 (0:00:00.080) 0:01:40.212 ******** 2021-04-03T03:04:16.116 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:16.118 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:16.138 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:16 +0000 (0:00:00.079) 0:01:40.292 ******** 2021-04-03T03:04:16.196 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:16.198 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:16.219 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:16 +0000 (0:00:00.080) 0:01:40.373 ******** 2021-04-03T03:04:16.287 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:16.289 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:16.310 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:16 +0000 (0:00:00.091) 0:01:40.464 ******** 2021-04-03T03:04:16.475 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:16.546 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 03 April 2021 03:04:16 +0000 (0:00:00.187) 0:01:40.652 ******** 2021-04-03T03:04:16.673 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:16.674 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:16.695 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:16 +0000 (0:00:00.197) 0:01:40.849 ******** 2021-04-03T03:04:16.858 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:16.860 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:16.886 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:16 +0000 (0:00:00.191) 0:01:41.040 ******** 2021-04-03T03:04:17.051 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:17.053 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:17.073 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.186) 0:01:41.227 ******** 2021-04-03T03:04:17.238 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:17.240 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:17.261 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.187) 0:01:41.415 ******** 2021-04-03T03:04:17.446 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:17.447 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:17.472 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.211) 0:01:41.626 ******** 2021-04-03T03:04:17.530 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:17.532 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:17.553 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.081) 0:01:41.707 ******** 2021-04-03T03:04:17.581 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:17.581 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apt-utils libapt-pkg5.0\nSuggested packages:\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apt apt-utils libapt-pkg5.0\n3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.\nNeed to get 2214 kB of archives.\nAfter this operation, 0 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt amd64 1.6.12ubuntu0.2 [1201 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]\nFetched 2214 kB in 1s (2620 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 2021-04-03T03:04:17.583 INFO:teuthology.task.ansible.out:10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 7 2021-04-03T03:04:17.585 INFO:teuthology.task.ansible.out:5%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt-utils (1.6.12ubuntu0.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apt-utils libapt-pkg5.0", "Suggested packages:", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apt apt-utils libapt-pkg5.0", "3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.", "Need to get 2214 kB of archives.", "After this operation, 0 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt 2021-04-03T03:04:17.586 INFO:teuthology.task.ansible.out:amd64 1.6.12ubuntu0.2 [1201 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]", "Fetched 2214 kB in 1s (2620 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Rea 2021-04-03T03:04:17.588 INFO:teuthology.task.ansible.out:ding database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt-utils (1.6.12ubuntu0.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:17.608 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.054) 0:01:41.762 ******** 2021-04-03T03:04:17.626 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:17.628 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:17.651 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.043) 0:01:41.805 ******** 2021-04-03T03:04:17.679 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-04-03T03:04:17.681 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:17.705 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.053) 0:01:41.859 ******** 2021-04-03T03:04:17.722 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-04-03T03:04:17.723 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:17.746 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:17 +0000 (0:00:00.041) 0:01:41.901 ******** 2021-04-03T03:04:17.985 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-04-03T03:04:17.987 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-04-03T03:04:18.009 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:18 +0000 (0:00:00.262) 0:01:42.163 ******** 2021-04-03T03:04:18.368 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:18.369 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-04-03T03:04:18.371 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:18.371 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apt-utils libapt-pkg5.0\nSuggested packages:\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apt apt-utils libapt-pkg5.0\n3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.\nNeed to get 2214 kB of archives.\nAfter this operation, 0 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0.2 [806 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt amd64 1.6.12ubuntu0.2 [1201 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]\nFetched 2214 kB in 1s ( 2021-04-03T03:04:18.373 INFO:teuthology.task.ansible.out:2593 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt (1.6.12ubuntu0.2) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database .. 2021-04-03T03:04:18.375 INFO:teuthology.task.ansible.out:. 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...\r\nUnpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...\r\nSetting up apt-utils (1.6.12ubuntu0.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apt-utils libapt-pkg5.0", "Suggested packages:", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apt apt-utils libapt-pkg5.0", "3 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.", "Need to get 2214 kB of archives.", "After this operation, 0 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-pkg5.0 amd64 1.6.12ubuntu0 2021-04-03T03:04:18.376 INFO:teuthology.task.ansible.out:.2 [806 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt amd64 1.6.12ubuntu0.2 [1201 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.12ubuntu0.2 [207 kB]", "Fetched 2214 kB in 1s (2593 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg5.0_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up libapt-pkg5.0:amd64 (1.6.12ubuntu0.2) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt (1.6.12ubuntu0.2) ...", "(Re 2021-04-03T03:04:18.378 INFO:teuthology.task.ansible.out:ading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../apt-utils_1.6.12ubuntu0.2_amd64.deb ...", "Unpacking apt-utils (1.6.12ubuntu0.2) over (1.6.12ubuntu0.1) ...", "Setting up apt-utils (1.6.12ubuntu0.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:18.408 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:18 +0000 (0:00:00.398) 0:01:42.562 ******** 2021-04-03T03:04:18.429 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:18 +0000 (0:00:00.021) 0:01:42.583 ******** 2021-04-03T03:04:18.585 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:18.587 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:18.611 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:18 +0000 (0:00:00.181) 0:01:42.765 ******** 2021-04-03T03:04:18.617 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-04-03T03:04:18.619 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:18.760 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:18 +0000 (0:00:00.148) 0:01:42.914 ******** 2021-04-03T03:04:19.030 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-04-03T03:04:19.032 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-04-03T03:04:19.054 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:19 +0000 (0:00:00.293) 0:01:43.208 ******** 2021-04-03T03:04:19.301 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-04-03T03:04:19.303 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-04-03T03:04:19.325 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:19 +0000 (0:00:00.271) 0:01:43.479 ******** 2021-04-03T03:04:19.582 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-04-03T03:04:19.584 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-04-03T03:04:19.606 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:19 +0000 (0:00:00.280) 0:01:43.760 ******** 2021-04-03T03:04:19.737 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-04-03T03:04:19.739 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:19.761 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:19 +0000 (0:00:00.155) 0:01:43.915 ******** 2021-04-03T03:04:19.831 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-04-03T03:04:19.832 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:19.853 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:19 +0000 (0:00:00.092) 0:01:44.007 ******** 2021-04-03T03:04:19.921 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-04-03T03:04:19.923 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:19.944 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:19 +0000 (0:00:00.090) 0:01:44.098 ******** 2021-04-03T03:04:20.012 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-04-03T03:04:20.013 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.034 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.090) 0:01:44.188 ******** 2021-04-03T03:04:20.103 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-04-03T03:04:20.105 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.125 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.091) 0:01:44.280 ******** 2021-04-03T03:04:20.194 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-04-03T03:04:20.196 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.219 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.093) 0:01:44.374 ******** 2021-04-03T03:04:20.298 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:20.300 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-04-03T03:04:20.320 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.100) 0:01:44.474 ******** 2021-04-03T03:04:20.401 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-04-03T03:04:20.403 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.414 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.415 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.425 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.427 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.436 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-04-03T03:04:20.447 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.449 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.459 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.459 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.470 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.472 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.482 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.483 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.510 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.189) 0:01:44.664 ******** 2021-04-03T03:04:20.736 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-04-03T03:04:20.738 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005478", "end": "2021-04-03 03:04:20.709504", "rc": 0, "start": "2021-04-03 03:04:20.704026", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock"]} 2021-04-03T03:04:20.761 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.251) 0:01:44.915 ******** 2021-04-03T03:04:20.829 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-04-03T03:04:20.831 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.854 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.092) 0:01:45.008 ******** 2021-04-03T03:04:20.942 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-04-03T03:04:20.944 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:20.969 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:20 +0000 (0:00:00.115) 0:01:45.123 ******** 2021-04-03T03:04:21.037 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-04-03T03:04:21.039 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:21.061 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:21 +0000 (0:00:00.091) 0:01:45.215 ******** 2021-04-03T03:04:21.590 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:21.591 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 14.3 MB in 1s (12.1 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database .. 2021-04-03T03:04:21.592 INFO:teuthology.task.ansible.out:. \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nPreparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...\r\nUnpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up git (1:2.17.1-1ubuntu0.8) ...\r\nSetti 2021-04-03T03:04:21.594 INFO:teuthology.task.ansible.out:ng up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2. 2021-04-03T03:04:21.596 INFO:teuthology.task.ansible.out:11+dfsg-1ubuntu7.36 [870 kB]", "Fetched 14.3 MB in 1s (12.1 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Preparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...", "Unpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1u 2021-04-03T03:04:21.597 INFO:teuthology.task.ansible.out:buntu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up git (1:2.17.1-1ubuntu0.8) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:21.616 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:21 +0000 (0:00:00.555) 0:01:45.770 ******** 2021-04-03T03:04:21.708 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-04-03T03:04:21.709 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:04:21.731 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:21 +0000 (0:00:00.114) 0:01:45.885 ******** 2021-04-03T03:04:21.800 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-04-03T03:04:21.802 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:21.828 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:21 +0000 (0:00:00.096) 0:01:45.982 ******** 2021-04-03T03:04:21.899 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:21.901 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item={'block_used': 13759007, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336101, 'size_available': 927072669696, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336101, "block_size": 4096, "block_total": 240095108, "block_used": 13759007, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072669696, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}} 2021-04-03T03:04:21.922 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:21 +0000 (0:00:00.094) 0:01:46.076 ******** 2021-04-03T03:04:22.064 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-04-03T03:04:22.066 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002492", "end": "2021-04-03 03:04:22.291306", "rc": 0, "start": "2021-04-03 03:04:22.288814", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:22.088 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:22 +0000 (0:00:00.165) 0:01:46.242 ******** 2021-04-03T03:04:22.167 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-04-03T03:04:22.169 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'block_used': 13759007, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336101, 'size_available': 927072669696, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336101, "block_size": 4096, "block_total": 240095108, "block_used": 13759007, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072669696, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:22.191 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:22 +0000 (0:00:00.103) 0:01:46.345 ******** 2021-04-03T03:04:22.260 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-04-03T03:04:22.262 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-04-03T03:04:22.284 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:22 +0000 (0:00:00.092) 0:01:46.438 ******** 2021-04-03T03:04:22.305 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:22 +0000 (0:00:00.020) 0:01:46.459 ******** 2021-04-03T03:04:22.364 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:22.366 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:22.503 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:22.504 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 14.3 MB in 1s (12.0 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database .. 2021-04-03T03:04:22.506 INFO:teuthology.task.ansible.out:. \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nPreparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...\r\nUnpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up git (1:2.17.1-1ubuntu0.8) ...\r\nSetti 2021-04-03T03:04:22.508 INFO:teuthology.task.ansible.out:ng up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2. 2021-04-03T03:04:22.509 INFO:teuthology.task.ansible.out:11+dfsg-1ubuntu7.36 [870 kB]", "Fetched 14.3 MB in 1s (12.0 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Preparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...", "Unpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu 2021-04-03T03:04:22.510 INFO:teuthology.task.ansible.out:-utils.", "Preparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1u 2021-04-03T03:04:22.512 INFO:teuthology.task.ansible.out:buntu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up git (1:2.17.1-1ubuntu0.8) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:22.535 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:22 +0000 (0:00:00.230) 0:01:46.689 ******** 2021-04-03T03:04:22.972 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-04-03T03:04:22.974 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:04:22.996 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:22 +0000 (0:00:00.460) 0:01:47.150 ******** 2021-04-03T03:04:23.127 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-04-03T03:04:23.129 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-04-03T03:04:23.150 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:23 +0000 (0:00:00.154) 0:01:47.304 ******** 2021-04-03T03:04:23.188 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-04-03T03:04:23.190 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:04:23.213 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:23 +0000 (0:00:00.062) 0:01:47.367 ******** 2021-04-03T03:04:23.218 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-04-03T03:04:23.220 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:23.248 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:23 +0000 (0:00:00.034) 0:01:47.402 ******** 2021-04-03T03:04:23.380 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-04-03T03:04:23.382 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-04-03T03:04:23.403 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:23 +0000 (0:00:00.155) 0:01:47.557 ******** 2021-04-03T03:04:23.660 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-04-03T03:04:23.662 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-04-03T03:04:23.684 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:23 +0000 (0:00:00.280) 0:01:47.838 ******** 2021-04-03T03:04:23.826 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-04-03T03:04:23.828 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:04:23.850 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:23 +0000 (0:00:00.166) 0:01:48.004 ******** 2021-04-03T03:04:23.856 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-04-03T03:04:23.858 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1234, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-04-03T03:04:23.883 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:23 +0000 (0:00:00.032) 0:01:48.037 ******** 2021-04-03T03:04:24.004 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-04-03T03:04:24.006 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-04-03T03:04:24.030 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.147) 0:01:48.184 ******** 2021-04-03T03:04:24.036 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-04-03T03:04:24.037 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:24.066 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.036) 0:01:48.220 ******** 2021-04-03T03:04:24.086 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:24.088 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tA2bzXNfVBksPjsTXtAqtq0BdPmYQSf2B'], 'uuids': ['b11b26a9-dfb6-4358-b54c-f53a10c26e5e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.617068", "end": "2021-04-03 03:04:24.288992", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tA2bzXNfVBksPjsTXtAqtq0BdPmYQSf2B"], "labels": [], "masters": [], "uuids": ["b11b26a9-dfb6-4358-b54c-f53a10c26e5e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:22.671924", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:24.088 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (i 2021-04-03T03:04:24.090 INFO:teuthology.task.ansible.out:tem={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.101 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-04-03T03:04:24.103 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.127 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.060) 0:01:48.281 ******** 2021-04-03T03:04:24.258 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-04-03T03:04:24.260 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-04-03T03:04:24.288 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.160) 0:01:48.442 ******** 2021-04-03T03:04:24.315 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-04-03T03:04:24.317 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ActiveEnterTimestampMonotonic": "18904750", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target basic.target -.mount systemd-tmpfiles-setup.service sysinit.target systemd-journald.socket nss-lookup.target system.slice network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:06 UTC", "AssertTimestampMonotonic": "18547266", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ConditionTimestampMonotonic": "18547266", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "D 2021-04-03T03:04:24.318 INFO:teuthology.task.ansible.out:ropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1572", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ExecMainStartTimestampMonotonic": "18904743", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:06 UTC", "InactiveExitTimestampMonotonic": "18547889", "InvocationID": "52dd6ba87f534eab8c47e71715aedb86", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft 2021-04-03T03:04:24.320 INFO:teuthology.task.ansible.out:": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1572", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGH 2021-04-03T03:04:24.321 INFO:teuthology.task.ansible.out:UP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:06 UTC", "StateChangeTimestampMonotonic": "18904750", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:06 UTC", "WatchdogTimestampMonotonic": "18904749", "WatchdogUSec": "0"}} 2021-04-03T03:04:24.341 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.053) 0:01:48.495 ******** 2021-04-03T03:04:24.411 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:04:24.413 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.437 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.095) 0:01:48.591 ******** 2021-04-03T03:04:24.506 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-04-03T03:04:24.508 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.529 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.091) 0:01:48.683 ******** 2021-04-03T03:04:24.577 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-04-03T03:04:24.579 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-04-03T03:04:24.601 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.072) 0:01:48.755 ******** 2021-04-03T03:04:24.607 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:04:24.610 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.635 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.033) 0:01:48.789 ******** 2021-04-03T03:04:24.694 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:04:24.695 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.717 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.081) 0:01:48.871 ******** 2021-04-03T03:04:24.765 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-04-03T03:04:24.766 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1234, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-04-03T03:04:24.789 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.072) 0:01:48.943 ******** 2021-04-03T03:04:24.794 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-04-03T03:04:24.795 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.820 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.030) 0:01:48.974 ******** 2021-04-03T03:04:24.889 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-04-03T03:04:24.891 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:24.912 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.092) 0:01:49.066 ******** 2021-04-03T03:04:24.929 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-04-03T03:04:24.931 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:24.955 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:24 +0000 (0:00:00.042) 0:01:49.109 ******** 2021-04-03T03:04:24.982 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:04:24.983 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.006 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.051) 0:01:49.160 ******** 2021-04-03T03:04:25.076 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-04-03T03:04:25.077 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.099 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.092) 0:01:49.253 ******** 2021-04-03T03:04:25.168 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:25.169 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.191 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.092) 0:01:49.345 ******** 2021-04-03T03:04:25.209 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-04-03T03:04:25.210 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:26 UTC", "ActiveEnterTimestampMonotonic": "18812490", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target nss-lookup.target system.slice sysinit.target remote-fs.target network.target -.mount systemd-tmpfiles-setup.service systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:25 UTC", "AssertTimestampMonotonic": "18424374", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:25 UTC", "ConditionTimestampMonotonic": "18424374", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "D 2021-04-03T03:04:25.211 INFO:teuthology.task.ansible.out:ropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1400", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:26 UTC", "ExecMainStartTimestampMonotonic": "18812436", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:25 UTC", "InactiveExitTimestampMonotonic": "18425057", "InvocationID": "2ca1e81246ad4a759de919455a27a186", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft 2021-04-03T03:04:25.213 INFO:teuthology.task.ansible.out:": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1400", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGH 2021-04-03T03:04:25.214 INFO:teuthology.task.ansible.out:UP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:26 UTC", "StateChangeTimestampMonotonic": "18812490", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:26 UTC", "WatchdogTimestampMonotonic": "18812487", "WatchdogUSec": "0"}} 2021-04-03T03:04:25.239 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.048) 0:01:49.394 ******** 2021-04-03T03:04:25.258 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:25.259 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6trKGyX7x9Qrh6UIuflL3GLz4njgUcxwAf'], 'uuids': ['4fedb8ed-0ef2-4bec-a955-cb7600150c24']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.087954", "end": "2021-04-03 03:04:25.477771", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6trKGyX7x9Qrh6UIuflL3GLz4njgUcxwAf"], "labels": [], "masters": [], "uuids": ["4fedb8ed-0ef2-4bec-a955-cb7600150c24"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:24.389817", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:25.261 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.287 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.047) 0:01:49.441 ******** 2021-04-03T03:04:25.294 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:25.295 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF', 'wwn-0x50000399ab9813ca'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1', 'wwn-0x50000399ab9813ca-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813ca', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JWF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF", "wwn-0x50000399ab9813ca"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1", "wwn-0x50000399ab9813ca-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JWF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813ca"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:25.296 INFO:teuthology.task.ansible.out:as False"} 2021-04-03T03:04:25.317 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:04:25.319 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.355 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.068) 0:01:49.509 ******** 2021-04-03T03:04:25.361 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:25.363 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.387 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.032) 0:01:49.541 ******** 2021-04-03T03:04:25.425 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-04-03T03:04:25.427 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.451 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.063) 0:01:49.605 ******** 2021-04-03T03:04:25.456 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:25.457 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.483 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.032) 0:01:49.637 ******** 2021-04-03T03:04:25.521 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:04:25.523 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.546 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.062) 0:01:49.700 ******** 2021-04-03T03:04:25.551 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:25.553 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.581 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.035) 0:01:49.735 ******** 2021-04-03T03:04:25.620 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:04:25.622 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.644 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.063) 0:01:49.799 ******** 2021-04-03T03:04:25.650 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:25.651 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.676 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.031) 0:01:49.830 ******** 2021-04-03T03:04:25.714 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-04-03T03:04:25.715 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.738 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.062) 0:01:49.892 ******** 2021-04-03T03:04:25.754 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:25.756 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.779 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.040) 0:01:49.933 ******** 2021-04-03T03:04:25.806 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-04-03T03:04:25.808 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.834 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.055) 0:01:49.988 ******** 2021-04-03T03:04:25.852 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.879 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.044) 0:01:50.033 ******** 2021-04-03T03:04:25.896 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:04:25.898 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.922 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.042) 0:01:50.076 ******** 2021-04-03T03:04:25.948 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:25.950 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:25.973 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:25 +0000 (0:00:00.051) 0:01:50.127 ******** 2021-04-03T03:04:25.990 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-04-03T03:04:25.992 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.016 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.042) 0:01:50.170 ******** 2021-04-03T03:04:26.064 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:26.066 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.089 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.072) 0:01:50.243 ******** 2021-04-03T03:04:26.095 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:26.096 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.121 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.032) 0:01:50.275 ******** 2021-04-03T03:04:26.180 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:26.182 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.204 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.082) 0:01:50.358 ******** 2021-04-03T03:04:26.210 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:04:26.212 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.236 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.032) 0:01:50.390 ******** 2021-04-03T03:04:26.274 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:26.276 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.299 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.062) 0:01:50.453 ******** 2021-04-03T03:04:26.316 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:26.317 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.341 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.042) 0:01:50.495 ******** 2021-04-03T03:04:26.378 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:26.380 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.403 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.061) 0:01:50.557 ******** 2021-04-03T03:04:26.420 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.449 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.046) 0:01:50.603 ******** 2021-04-03T03:04:26.466 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:26.467 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ty3t43-H5WP-jAk7-FnrH-dsOO-ASmD-DKUP0c', 'nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A', 'nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.108271", "end": "2021-04-03 03:04:26.682438", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-ty3t43-H5WP-jAk7-FnrH-dsOO-ASmD-DKUP0c", "nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A", "nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:25.574167", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now parti 2021-04-03T03:04:26.468 INFO:teuthology.task.ansible.out:tion the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:26.479 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.480 INFO:teuthology.task.ansible.out: 2021-04-03T03:04:26.480 INFO:teuthology.task.ansible.out:TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.481 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:26.481 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.507 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.058) 0:01:50.661 ******** 2021-04-03T03:04:26.525 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:26.527 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.550 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.042) 0:01:50.704 ******** 2021-04-03T03:04:26.577 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:26.579 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.602 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.051) 0:01:50.756 ******** 2021-04-03T03:04:26.618 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:26.620 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.645 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.042) 0:01:50.799 ******** 2021-04-03T03:04:26.661 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:26.663 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.687 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.042) 0:01:50.841 ******** 2021-04-03T03:04:26.696 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:26.697 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 14.3 MB in 1s (12.0 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database .. 2021-04-03T03:04:26.698 INFO:teuthology.task.ansible.out:. \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nPreparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...\r\nUnpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up git (1:2.17.1-1ubuntu0.8) ...\r\nSetti 2021-04-03T03:04:26.700 INFO:teuthology.task.ansible.out:ng up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2. 2021-04-03T03:04:26.701 INFO:teuthology.task.ansible.out:11+dfsg-1ubuntu7.36 [870 kB]", "Fetched 14.3 MB in 1s (12.0 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Preparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...", "Unpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1u 2021-04-03T03:04:26.703 INFO:teuthology.task.ansible.out:buntu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up git (1:2.17.1-1ubuntu0.8) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:26.724 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.036) 0:01:50.878 ******** 2021-04-03T03:04:26.742 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:26.744 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.768 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.043) 0:01:50.922 ******** 2021-04-03T03:04:26.774 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:26.775 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.800 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.031) 0:01:50.954 ******** 2021-04-03T03:04:26.838 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:26.839 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.863 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.063) 0:01:51.017 ******** 2021-04-03T03:04:26.869 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-04-03T03:04:26.871 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.895 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.032) 0:01:51.049 ******** 2021-04-03T03:04:26.934 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:26.935 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:26.958 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:26 +0000 (0:00:00.063) 0:01:51.112 ******** 2021-04-03T03:04:27.027 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:27.029 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.055 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.097) 0:01:51.209 ******** 2021-04-03T03:04:27.063 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-04-03T03:04:27.065 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-04-03T03:04:27.101 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.045) 0:01:51.255 ******** 2021-04-03T03:04:27.128 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:27.130 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.154 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.053) 0:01:51.308 ******** 2021-04-03T03:04:27.223 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:27.225 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.246 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.092) 0:01:51.401 ******** 2021-04-03T03:04:27.306 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:27.307 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.330 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.083) 0:01:51.484 ******** 2021-04-03T03:04:27.358 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:27.360 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} TASK [testnode : Install non aarch64 packages] ********************************* ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:04:27.385 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.054) 0:01:51.539 ******** 2021-04-03T03:04:27.407 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.022) 0:01:51.561 ******** 2021-04-03T03:04:27.413 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:27.414 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.442 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.035) 0:01:51.596 ******** 2021-04-03T03:04:27.459 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:27.461 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.485 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.042) 0:01:51.639 ******** 2021-04-03T03:04:27.513 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:27.515 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.543 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.057) 0:01:51.697 ******** 2021-04-03T03:04:27.614 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:27.616 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.638 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.095) 0:01:51.792 ******** 2021-04-03T03:04:27.655 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:27.656 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tdN2MOigqjNwFwdKZsXHLb6GxmugFivQg'], 'uuids': ['96ffdeec-5115-409d-bf32-ae24b1d14342']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.095875", "end": "2021-04-03 03:04:27.877012", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tdN2MOigqjNwFwdKZsXHLb6GxmugFivQg"], "labels": [], "masters": [], "uuids": ["96ffdeec-5115-409d-bf32-ae24b1d14342"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:26.781137", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:27.667 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.679 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.701 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:27.703 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.724 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.086) 0:01:51.879 ******** 2021-04-03T03:04:27.804 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-04-03T03:04:27.806 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:27.831 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.106) 0:01:51.985 ******** 2021-04-03T03:04:27.837 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-04-03T03:04:27.839 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-04-03T03:04:27.864 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:27 +0000 (0:00:00.032) 0:01:52.018 ******** 2021-04-03T03:04:27.987 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-04-03T03:04:27.989 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-04-03T03:04:28.012 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.147) 0:01:52.166 ******** 2021-04-03T03:04:28.071 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-04-03T03:04:28.072 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:04:28.096 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.083) 0:01:52.250 ******** 2021-04-03T03:04:28.134 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-04-03T03:04:28.135 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:28.137 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 14.3 MB in 1s (11.3 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database .. 2021-04-03T03:04:28.139 INFO:teuthology.task.ansible.out:. \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137459 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nPreparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...\r\nUnpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up git (1:2.17.1-1ubuntu0.8) ...\r\nSetti 2021-04-03T03:04:28.140 INFO:teuthology.task.ansible.out:ng up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 129 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2. 2021-04-03T03:04:28.142 INFO:teuthology.task.ansible.out:11+dfsg-1ubuntu7.36 [870 kB]", "Fetched 14.3 MB in 1s (11.3 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137459 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Preparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...", "Unpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1u 2021-04-03T03:04:28.143 INFO:teuthology.task.ansible.out:buntu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up git (1:2.17.1-1ubuntu0.8) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2021-04-03T03:04:28.162 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.066) 0:01:52.316 ******** 2021-04-03T03:04:28.183 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.021) 0:01:52.337 ******** 2021-04-03T03:04:28.233 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-04-03T03:04:28.235 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-04-03T03:04:28.260 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.076) 0:01:52.414 ******** 2021-04-03T03:04:28.266 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:28.268 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-04-03T03:04:28.293 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.033) 0:01:52.447 ******** 2021-04-03T03:04:28.331 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-04-03T03:04:28.333 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:28.356 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.063) 0:01:52.510 ******** 2021-04-03T03:04:28.373 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:28.375 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:28.398 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.041) 0:01:52.552 ******** 2021-04-03T03:04:28.415 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-04-03T03:04:28.417 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-04-03T03:04:28.441 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.042) 0:01:52.595 ******** 2021-04-03T03:04:28.512 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-04-03T03:04:28.513 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-04-03T03:04:28.540 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.099) 0:01:52.695 ******** 2021-04-03T03:04:28.600 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-04-03T03:04:28.601 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:28.629 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.088) 0:01:52.783 ******** 2021-04-03T03:04:28.666 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-04-03T03:04:28.668 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-04-03T03:04:28.692 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.063) 0:01:52.846 ******** 2021-04-03T03:04:28.698 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-04-03T03:04:28.699 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:28.724 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.032) 0:01:52.878 ******** 2021-04-03T03:04:28.783 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-04-03T03:04:28.785 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:28.807 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.083) 0:01:52.961 ******** 2021-04-03T03:04:28.824 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:28.826 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tljAp5NHRZHZvemA3soC0gnmcQR0wa6Ut'], 'uuids': ['d751d67d-349b-407f-b7be-ee88f2cdde3d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.070017", "end": "2021-04-03 03:04:29.042587", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tljAp5NHRZHZvemA3soC0gnmcQR0wa6Ut"], "labels": [], "masters": [], "uuids": ["d751d67d-349b-407f-b7be-ee88f2cdde3d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:27.972570", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:28.837 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-04-03T03:04:28.839 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:28.839 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-04-03T03:04:28.864 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.057) 0:01:53.018 ******** 2021-04-03T03:04:28.887 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.023) 0:01:53.041 ******** 2021-04-03T03:04:28.893 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-04-03T03:04:28.895 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:28.919 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.031) 0:01:53.073 ******** 2021-04-03T03:04:28.957 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-04-03T03:04:28.959 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-04-03T03:04:28.982 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:28 +0000 (0:00:00.063) 0:01:53.136 ******** 2021-04-03T03:04:28.998 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-04-03T03:04:29.000 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.024 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.041) 0:01:53.178 ******** 2021-04-03T03:04:29.031 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-04-03T03:04:29.033 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1234, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-04-03T03:04:29.057 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.033) 0:01:53.211 ******** 2021-04-03T03:04:29.085 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-04-03T03:04:29.087 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.109 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.051) 0:01:53.263 ******** 2021-04-03T03:04:29.188 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-04-03T03:04:29.211 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.101) 0:01:53.365 ******** 2021-04-03T03:04:29.216 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-04-03T03:04:29.218 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:29.246 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.035) 0:01:53.400 ******** 2021-04-03T03:04:29.263 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-04-03T03:04:29.265 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-04-03T03:04:29.289 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.042) 0:01:53.443 ******** 2021-04-03T03:04:29.296 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-04-03T03:04:29.297 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.308 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.310 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.320 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.322 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-04-03T03:04:29.331 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.342 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.344 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.353 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.355 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.365 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.367 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.376 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.404 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.114) 0:01:53.558 ******** 2021-04-03T03:04:29.431 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-04-03T03:04:29.433 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:29.457 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.052) 0:01:53.611 ******** 2021-04-03T03:04:29.514 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-04-03T03:04:29.516 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.537 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.080) 0:01:53.691 ******** 2021-04-03T03:04:29.544 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-04-03T03:04:29.545 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:13 UTC", "ActiveEnterTimestampMonotonic": "18740380", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target remote-fs.target systemd-tmpfiles-setup.service nss-lookup.target basic.target -.mount system.slice network.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:12 UTC", "AssertTimestampMonotonic": "18486657", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:12 UTC", "ConditionTimestampMonotonic": "18486657", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "D 2021-04-03T03:04:29.547 INFO:teuthology.task.ansible.out:ropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1401", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:13 UTC", "ExecMainStartTimestampMonotonic": "18740374", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:12 UTC", "InactiveExitTimestampMonotonic": "18487083", "InvocationID": "d92b190b424b4c72980ee8a2b79c9d52", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft 2021-04-03T03:04:29.548 INFO:teuthology.task.ansible.out:": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1401", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGH 2021-04-03T03:04:29.550 INFO:teuthology.task.ansible.out:UP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:13 UTC", "StateChangeTimestampMonotonic": "18740380", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:13 UTC", "WatchdogTimestampMonotonic": "18740379", "WatchdogUSec": "0"}} TASK [testnode : Install packages with --no-install-recommends] **************** ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:04:29.573 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.035) 0:01:53.727 ******** 2021-04-03T03:04:29.594 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.020) 0:01:53.747 ******** 2021-04-03T03:04:29.611 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-04-03T03:04:29.614 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.637 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.043) 0:01:53.791 ******** 2021-04-03T03:04:29.642 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-04-03T03:04:29.644 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.002751", "end": "2021-04-03 03:04:29.613436", "rc": 0, "start": "2021-04-03 03:04:29.610685", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock"]} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.670 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.033) 0:01:53.824 ******** 2021-04-03T03:04:29.690 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.020) 0:01:53.844 ******** 2021-04-03T03:04:29.707 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-04-03T03:04:29.709 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.732 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.041) 0:01:53.886 ******** 2021-04-03T03:04:29.737 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-04-03T03:04:29.739 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.765 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.033) 0:01:53.919 ******** 2021-04-03T03:04:29.771 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-04-03T03:04:29.773 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.801 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.036) 0:01:53.955 ******** 2021-04-03T03:04:29.822 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.020) 0:01:53.976 ******** 2021-04-03T03:04:29.829 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-04-03T03:04:29.831 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.855 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.033) 0:01:54.009 ******** 2021-04-03T03:04:29.862 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-04-03T03:04:29.864 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.887 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.032) 0:01:54.041 ******** 2021-04-03T03:04:29.892 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:04:29.894 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.919 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.032) 0:01:54.073 ******** 2021-04-03T03:04:29.940 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.020) 0:01:54.094 ******** 2021-04-03T03:04:29.946 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-04-03T03:04:29.948 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:29.972 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:29 +0000 (0:00:00.032) 0:01:54.126 ******** 2021-04-03T03:04:29.978 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-04-03T03:04:29.980 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.004 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.031) 0:01:54.158 ******** 2021-04-03T03:04:30.010 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:30.012 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tot9JNWfI41IbkGbDNU9aWPb4OrbM1Dnk'], 'uuids': ['7f0a1060-ca82-4d00-aa30-e6f2c9dccc3c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.067421", "end": "2021-04-03 03:04:30.203911", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tot9JNWfI41IbkGbDNU9aWPb4OrbM1Dnk"], "labels": [], "masters": [], "uuids": ["7f0a1060-ca82-4d00-aa30-e6f2c9dccc3c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:29.136490", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:30.013 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] *** 2021-04-03T03:04:30.014 INFO:teuthology.task.ansible.out:*********************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.016 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:30.047 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.043) 0:01:54.201 ******** 2021-04-03T03:04:30.074 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.026) 0:01:54.228 ******** 2021-04-03T03:04:30.081 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:30.083 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-04-03T03:04:30.108 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.034) 0:01:54.262 ******** 2021-04-03T03:04:30.114 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-04-03T03:04:30.115 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-04-03T03:04:30.141 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.032) 0:01:54.295 ******** 2021-04-03T03:04:30.148 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:30.149 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.160 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.184 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.043) 0:01:54.338 ******** 2021-04-03T03:04:30.191 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-04-03T03:04:30.193 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.203 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.205 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.215 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.226 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-04-03T03:04:30.228 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.237 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.239 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.248 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.250 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba031.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.273 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.088) 0:01:54.427 ******** 2021-04-03T03:04:30.278 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.281 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.282 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:30.283 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tA2bzXNfVBksPjsTXtAqtq0BdPmYQSf2B'], 'uuids': ['b11b26a9-dfb6-4358-b54c-f53a10c26e5e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.076732", "end": "2021-04-03 03:04:30.500030", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tA2bzXNfVBksPjsTXtAqtq0BdPmYQSf2B"], "labels": [], "masters": [], "uuids": ["b11b26a9-dfb6-4358-b54c-f53a10c26e5e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:30.423298", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0754687 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0754687 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:30.311 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.038) 0:01:54.465 ******** 2021-04-03T03:04:30.317 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.318 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.330 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-04-03T03:04:30.332 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.355 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.043) 0:01:54.509 ******** 2021-04-03T03:04:30.414 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-04-03T03:04:30.415 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-04-03T03:04:30.443 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.088) 0:01:54.597 ******** 2021-04-03T03:04:30.450 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-04-03T03:04:30.452 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.475 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.032) 0:01:54.630 ******** 2021-04-03T03:04:30.482 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:30.483 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6trKGyX7x9Qrh6UIuflL3GLz4njgUcxwAf'], 'uuids': ['4fedb8ed-0ef2-4bec-a955-cb7600150c24']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.076278", "end": "2021-04-03 03:04:30.670889", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6trKGyX7x9Qrh6UIuflL3GLz4njgUcxwAf"], "labels": [], "masters": [], "uuids": ["4fedb8ed-0ef2-4bec-a955-cb7600150c24"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:30.594611", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0750707 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0750707 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:30.484 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF', 'wwn-0x50000399ab9813ca'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1', 'wwn-0x50000399ab9813ca-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813ca', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JWF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF", "wwn-0x50000399ab9813ca"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JWF6XF-part1", "wwn-0x50000399ab9813ca-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JWF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813ca"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:30.486 INFO:teuthology.task.ansible.out:as False"} 2021-04-03T03:04:30.549 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-04-03T03:04:30.550 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003093", "end": "2021-04-03 03:04:30.521052", "rc": 0, "start": "2021-04-03 03:04:30.517959", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock"]} TASK [testnode : set_fact] ***************************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.572 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.096) 0:01:54.726 ******** 2021-04-03T03:04:30.595 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.022) 0:01:54.749 ******** 2021-04-03T03:04:30.602 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:30.604 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ty3t43-H5WP-jAk7-FnrH-dsOO-ASmD-DKUP0c', 'nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A', 'nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.023893", "end": "2021-04-03 03:04:30.779062", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-ty3t43-H5WP-jAk7-FnrH-dsOO-ASmD-DKUP0c", "nvme-INTEL_SSDPEL1K375GA_PHKM920000BW375A", "nvme-nvme.8086-50484b4d393230303030425733373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:30.755169", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.022752 s, 5.1 GB/s", "stderr_lines": ["110+ 2021-04-03T03:04:30.606 INFO:teuthology.task.ansible.out:0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.022752 s, 5.1 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.628 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-04-03T03:04:30.630 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:04:30.655 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.059) 0:01:54.809 ******** 2021-04-03T03:04:30.662 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:04:30.664 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [gibba028.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1234, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-04-03T03:04:30.691 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.036) 0:01:54.845 ******** 2021-04-03T03:04:30.712 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.020) 0:01:54.866 ******** 2021-04-03T03:04:30.719 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-04-03T03:04:30.721 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.749 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.036) 0:01:54.903 ******** 2021-04-03T03:04:30.756 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-04-03T03:04:30.758 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:30.758 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tdN2MOigqjNwFwdKZsXHLb6GxmugFivQg'], 'uuids': ['96ffdeec-5115-409d-bf32-ae24b1d14342']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.080452", "end": "2021-04-03 03:04:30.956332", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tdN2MOigqjNwFwdKZsXHLb6GxmugFivQg"], "labels": [], "masters": [], "uuids": ["96ffdeec-5115-409d-bf32-ae24b1d14342"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:30.875880", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.079166 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.079166 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:30.760 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.785 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.035) 0:01:54.939 ******** 2021-04-03T03:04:30.791 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:30.793 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.819 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.033) 0:01:54.973 ******** 2021-04-03T03:04:30.824 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-04-03T03:04:30.826 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.851 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.031) 0:01:55.005 ******** 2021-04-03T03:04:30.867 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:30.869 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item={'block_used': 13759007, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336101, 'size_available': 927072669696, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336101, "block_size": 4096, "block_total": 240095108, "block_used": 13759007, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072669696, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}} 2021-04-03T03:04:30.869 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [gibba028.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:30.894 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.043) 0:01:55.048 ******** 2021-04-03T03:04:30.915 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.020) 0:01:55.069 ******** 2021-04-03T03:04:30.921 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:30.922 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:30.947 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.032) 0:01:55.101 ******** 2021-04-03T03:04:30.968 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:30 +0000 (0:00:00.020) 0:01:55.122 ******** 2021-04-03T03:04:30.975 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:30.977 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tljAp5NHRZHZvemA3soC0gnmcQR0wa6Ut'], 'uuids': ['d751d67d-349b-407f-b7be-ee88f2cdde3d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.080728", "end": "2021-04-03 03:04:31.156051", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tljAp5NHRZHZvemA3soC0gnmcQR0wa6Ut"], "labels": [], "masters": [], "uuids": ["d751d67d-349b-407f-b7be-ee88f2cdde3d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:31.075323", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0795111 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0795111 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:30.977 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': 2021-04-03T03:04:30.979 INFO:teuthology.task.ansible.out: '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.021 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.044 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.075) 0:01:55.198 ******** 2021-04-03T03:04:31.050 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-04-03T03:04:31.052 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002479", "end": "2021-04-03 03:04:31.185276", "rc": 0, "start": "2021-04-03 03:04:31.182797", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:31.076 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.032) 0:01:55.230 ******** 2021-04-03T03:04:31.114 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:31.116 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tot9JNWfI41IbkGbDNU9aWPb4OrbM1Dnk'], 'uuids': ['7f0a1060-ca82-4d00-aa30-e6f2c9dccc3c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.075806", "end": "2021-04-03 03:04:31.329025", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-AqaGIunYse3CgzgGL4lqCnk3lpBfMb6tot9JNWfI41IbkGbDNU9aWPb4OrbM1Dnk"], "labels": [], "masters": [], "uuids": ["7f0a1060-ca82-4d00-aa30-e6f2c9dccc3c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:31.253219", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0745815 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0745815 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:31.140 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.063) 0:01:55.294 ******** 2021-04-03T03:04:31.192 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.051) 0:01:55.346 ******** 2021-04-03T03:04:31.201 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-04-03T03:04:31.203 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'block_used': 13759007, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336101, 'size_available': 927072669696, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336101, "block_size": 4096, "block_total": 240095108, "block_used": 13759007, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072669696, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.230 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.038) 0:01:55.384 ******** 2021-04-03T03:04:31.237 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:31.238 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.263 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.033) 0:01:55.417 ******** 2021-04-03T03:04:31.271 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-04-03T03:04:31.273 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:19 UTC", "ActiveEnterTimestampMonotonic": "18410815", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target network.target sysinit.target systemd-tmpfiles-setup.service -.mount system.slice systemd-journald.socket remote-fs.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:19 UTC", "AssertTimestampMonotonic": "18102272", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:19 UTC", "ConditionTimestampMonotonic": "18102272", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "D 2021-04-03T03:04:31.275 INFO:teuthology.task.ansible.out:ropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1427", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:19 UTC", "ExecMainStartTimestampMonotonic": "18410804", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:19 UTC", "InactiveExitTimestampMonotonic": "18102685", "InvocationID": "723c22edb2dc44c7a77d1533405ae0f0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft 2021-04-03T03:04:31.276 INFO:teuthology.task.ansible.out:": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1427", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGH 2021-04-03T03:04:31.278 INFO:teuthology.task.ansible.out:UP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:19 UTC", "StateChangeTimestampMonotonic": "18410815", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:19 UTC", "WatchdogTimestampMonotonic": "18410814", "WatchdogUSec": "0"}} 2021-04-03T03:04:31.305 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.042) 0:01:55.459 ******** 2021-04-03T03:04:31.313 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-04-03T03:04:31.340 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.034) 0:01:55.494 ******** 2021-04-03T03:04:31.346 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:31.347 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.375 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.035) 0:01:55.529 ******** 2021-04-03T03:04:31.383 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-04-03T03:04:31.385 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.411 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.036) 0:01:55.565 ******** 2021-04-03T03:04:31.428 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:31.430 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.441 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.468 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.056) 0:01:55.621 ******** 2021-04-03T03:04:31.478 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-04-03T03:04:31.480 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.126357", "end": "2021-04-03 03:04:31.626243", "rc": 0, "start": "2021-04-03 03:04:31.499886", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:31.505 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.037) 0:01:55.659 ******** 2021-04-03T03:04:31.512 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-04-03T03:04:31.513 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.544 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.039) 0:01:55.698 ******** 2021-04-03T03:04:31.551 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:31.552 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.576 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.032) 0:01:55.730 ******** 2021-04-03T03:04:31.604 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-04-03T03:04:31.606 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.629 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.052) 0:01:55.783 ******** 2021-04-03T03:04:31.635 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-04-03T03:04:31.637 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} TASK [testnode : Check for physical volumes] *********************************** changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.032254", "end": "2021-04-03 03:04:31.853161", "rc": 0, "start": "2021-04-03 03:04:31.820907", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:31.662 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.032) 0:01:55.816 ******** 2021-04-03T03:04:31.685 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.022) 0:01:55.839 ******** 2021-04-03T03:04:31.691 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:31.693 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.718 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.033) 0:01:55.872 ******** 2021-04-03T03:04:31.725 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-04-03T03:04:31.726 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.751 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.032) 0:01:55.905 ******** 2021-04-03T03:04:31.757 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-04-03T03:04:31.758 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.822 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.071) 0:01:55.976 ******** 2021-04-03T03:04:31.844 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.021) 0:01:55.998 ******** 2021-04-03T03:04:31.850 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:31.851 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.879 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.034) 0:01:56.033 ******** 2021-04-03T03:04:31.899 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.020) 0:01:56.053 ******** 2021-04-03T03:04:31.920 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.020) 0:01:56.074 ******** 2021-04-03T03:04:31.926 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:31.928 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item={'block_used': 13759008, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336100, 'size_available': 927072665600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149514, 'block_size': 4096, 'inode_available': 60905462}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336100, "block_size": 4096, "block_total": 240095108, "block_used": 13759008, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905462, "inode_total": 61054976, "inode_used": 149514, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072665600, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}} 2021-04-03T03:04:31.928 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.928 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:31.929 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:31.952 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.032) 0:01:56.106 ******** 2021-04-03T03:04:31.971 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.019) 0:01:56.125 ******** 2021-04-03T03:04:31.990 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:31 +0000 (0:00:00.019) 0:01:56.144 ******** 2021-04-03T03:04:31.996 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-04-03T03:04:31.997 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.025 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.034) 0:01:56.179 ******** 2021-04-03T03:04:32.046 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.021) 0:01:56.200 ******** 2021-04-03T03:04:32.075 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.028) 0:01:56.229 ******** 2021-04-03T03:04:32.083 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-04-03T03:04:32.085 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.109 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.033) 0:01:56.263 ******** 2021-04-03T03:04:32.131 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.022) 0:01:56.285 ******** 2021-04-03T03:04:32.140 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-04-03T03:04:32.141 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002499", "end": "2021-04-03 03:04:32.370820", "rc": 0, "start": "2021-04-03 03:04:32.368321", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.167 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.036) 0:01:56.321 ******** 2021-04-03T03:04:32.189 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.022) 0:01:56.343 ******** 2021-04-03T03:04:32.196 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:32.198 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.198 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:32.225 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.035) 0:01:56.379 ******** 2021-04-03T03:04:32.247 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.022) 0:01:56.401 ******** 2021-04-03T03:04:32.270 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.022) 0:01:56.424 ******** 2021-04-03T03:04:32.277 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-04-03T03:04:32.277 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'block_used': 13759008, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336100, 'size_available': 927072665600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149514, 'block_size': 4096, 'inode_available': 60905462}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336100, "block_size": 4096, "block_total": 240095108, "block_used": 13759008, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905462, "inode_total": 61054976, "inode_used": 149514, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072665600, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.279 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:32.281 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'block_used': 13759007, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336101, 'size_available': 927072669696, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336101, "block_size": 4096, "block_total": 240095108, "block_used": 13759007, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072669696, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:32.281 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.305 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.034) 0:01:56.459 ******** 2021-04-03T03:04:32.326 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.020) 0:01:56.480 ******** 2021-04-03T03:04:32.347 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.021) 0:01:56.501 ******** 2021-04-03T03:04:32.353 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:32.355 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Combine list of non-root disks] ******************************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set grub config.] ********************************************* skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.379 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.032) 0:01:56.533 ******** 2021-04-03T03:04:32.404 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.024) 0:01:56.558 ******** 2021-04-03T03:04:32.426 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.021) 0:01:56.580 ******** 2021-04-03T03:04:32.436 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-04-03T03:04:32.437 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.462 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.036) 0:01:56.616 ******** 2021-04-03T03:04:32.485 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.022) 0:01:56.639 ******** 2021-04-03T03:04:32.491 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-04-03T03:04:32.493 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.518 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.033) 0:01:56.673 ******** 2021-04-03T03:04:32.542 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.023) 0:01:56.696 ******** 2021-04-03T03:04:32.548 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-04-03T03:04:32.550 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.552 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.577 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.035) 0:01:56.731 ******** 2021-04-03T03:04:32.599 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.021) 0:01:56.753 ******** 2021-04-03T03:04:32.605 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-04-03T03:04:32.607 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.632 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.033) 0:01:56.786 ******** 2021-04-03T03:04:32.654 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.021) 0:01:56.808 ******** 2021-04-03T03:04:32.660 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-04-03T03:04:32.662 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.685 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.031) 0:01:56.839 ******** 2021-04-03T03:04:32.692 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-04-03T03:04:32.694 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.718 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.033) 0:01:56.872 ******** 2021-04-03T03:04:32.746 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:32.748 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.776 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.057) 0:01:56.930 ******** 2021-04-03T03:04:32.804 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:32.806 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.930 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.153) 0:01:57.084 ******** 2021-04-03T03:04:32.937 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:32.938 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-04-03T03:04:32.964 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.033) 0:01:57.118 ******** 2021-04-03T03:04:32.971 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:32.972 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:32.998 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:32 +0000 (0:00:00.033) 0:01:57.152 ******** 2021-04-03T03:04:33.005 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:33.006 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.029 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.031) 0:01:57.183 ******** 2021-04-03T03:04:33.036 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:33.038 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.063 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.033) 0:01:57.217 ******** 2021-04-03T03:04:33.090 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:33.092 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.115 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.051) 0:01:57.269 ******** 2021-04-03T03:04:33.143 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:33.144 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.167 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.052) 0:01:57.321 ******** 2021-04-03T03:04:33.195 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-04-03T03:04:33.197 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.219 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.051) 0:01:57.373 ******** 2021-04-03T03:04:33.246 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:33.248 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.271 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.051) 0:01:57.425 ******** 2021-04-03T03:04:33.277 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-04-03T03:04:33.279 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} TASK [testnode : Create volume group(s)] *************************************** changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-04-03T03:04:33.303 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.031) 0:01:57.457 ******** 2021-04-03T03:04:33.322 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.019) 0:01:57.476 ******** 2021-04-03T03:04:33.332 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:33.334 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.363 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.040) 0:01:57.517 ******** 2021-04-03T03:04:33.389 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:33.391 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.421 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.057) 0:01:57.575 ******** 2021-04-03T03:04:33.438 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:33.439 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpILiI8nEeN6SehkzpksQLJ61YJbPe75Iq'], 'uuids': ['eee40ada-2c82-4a0b-85be-d30abf560de9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.926290", "end": "2021-04-03 03:04:33.580238", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpILiI8nEeN6SehkzpksQLJ61YJbPe75Iq"], "labels": [], "masters": [], "uuids": ["eee40ada-2c82-4a0b-85be-d30abf560de9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:31.653948", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:33.451 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.452 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-04-03T03:04:33.452 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.452 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.476 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.055) 0:01:57.630 ******** 2021-04-03T03:04:33.503 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-04-03T03:04:33.505 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.527 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.050) 0:01:57.681 ******** 2021-04-03T03:04:33.554 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-04-03T03:04:33.555 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.578 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.050) 0:01:57.732 ******** 2021-04-03T03:04:33.584 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-04-03T03:04:33.586 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-04-03T03:04:33.610 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.032) 0:01:57.764 ******** 2021-04-03T03:04:33.617 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:33.645 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.035) 0:01:57.799 ******** 2021-04-03T03:04:33.652 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:33.654 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:33.749 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-04-03T03:04:33.750 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-04-03T03:04:33.772 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.127) 0:01:57.926 ******** 2021-04-03T03:04:33.779 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:33.780 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:33.897 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-04-03T03:04:33.898 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-04-03T03:04:33.926 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:33 +0000 (0:00:00.154) 0:01:58.081 ******** 2021-04-03T03:04:33.933 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:33.935 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:33.988 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-04-03T03:04:33.989 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-04-03T03:04:34.013 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.086) 0:01:58.167 ******** 2021-04-03T03:04:34.030 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-04-03T03:04:34.032 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:34.055 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.041) 0:01:58.209 ******** 2021-04-03T03:04:34.062 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-04-03T03:04:34.063 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:34.087 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.031) 0:01:58.241 ******** 2021-04-03T03:04:34.093 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-04-03T03:04:34.094 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.118 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.031) 0:01:58.273 ******** 2021-04-03T03:04:34.145 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-04-03T03:04:34.147 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.176 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.057) 0:01:58.330 ******** 2021-04-03T03:04:34.193 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:34.195 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:34.220 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.044) 0:01:58.374 ******** 2021-04-03T03:04:34.229 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-04-03T03:04:34.230 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:34.231 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmqP2taNqkPN10BPfYoF1bclpluY1jOTtJ'], 'uuids': ['b1485987-4a60-43c5-801e-e74185106c09']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.592180", "end": "2021-04-03 03:04:34.480261", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmqP2taNqkPN10BPfYoF1bclpluY1jOTtJ"], "labels": [], "masters": [], "uuids": ["b1485987-4a60-43c5-801e-e74185106c09"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:32.888081", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot o 2021-04-03T03:04:34.232 INFO:teuthology.task.ansible.out:r after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.234 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.261 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.040) 0:01:58.415 ******** 2021-04-03T03:04:34.288 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-04-03T03:04:34.290 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.313 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.052) 0:01:58.467 ******** 2021-04-03T03:04:34.330 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-04-03T03:04:34.332 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-04-03T03:04:34.354 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.041) 0:01:58.508 ******** 2021-04-03T03:04:34.361 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-04-03T03:04:34.361 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.362 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-04-03T03:04:34.362 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-04-03T03:04:34.388 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.033) 0:01:58.542 ******** 2021-04-03T03:04:34.411 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.022) 0:01:58.565 ******** 2021-04-03T03:04:34.417 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:34.418 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-04-03T03:04:34.443 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.032) 0:01:58.597 ******** 2021-04-03T03:04:34.460 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:34.462 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:34.483 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-04-03T03:04:34.484 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.494 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.495 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.505 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.516 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.518 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.527 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-04-03T03:04:34.538 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.540 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba026.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.551 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.552 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.562 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.562 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.573 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.575 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.601 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.158) 0:01:58.755 ******** 2021-04-03T03:04:34.618 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:34.620 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:34.620 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:34.621 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpjYRcCEUdv7wDLlWKJrjueoeXtD6075iU'], 'uuids': ['389d7121-154b-44ec-aaee-58b53d75ba64']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.088065", "end": "2021-04-03 03:04:34.767359", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpjYRcCEUdv7wDLlWKJrjueoeXtD6075iU"], "labels": [], "masters": [], "uuids": ["389d7121-154b-44ec-aaee-58b53d75ba64"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:33.679294", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk 2021-04-03T03:04:34.622 INFO:teuthology.task.ansible.out: using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:34.633 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF', 'wwn-0x50000399ab801e93'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF-part1', 'wwn-0x50000399ab801e93-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801e93', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K3PZF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF", "wwn-0x50000399ab801e93"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF-part1", "wwn-0x50000399ab801e93-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K3PZF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801e93"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:34.635 INFO:teuthology.task.ansible.out:as False"} 2021-04-03T03:04:34.655 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-04-03T03:04:34.657 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-04-03T03:04:34.682 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.081) 0:01:58.837 ******** 2021-04-03T03:04:34.699 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:34.702 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:34.711 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.739 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.056) 0:01:58.893 ******** 2021-04-03T03:04:34.776 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-04-03T03:04:34.778 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.805 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.066) 0:01:58.959 ******** 2021-04-03T03:04:34.811 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-04-03T03:04:34.813 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003284", "end": "2021-04-03 03:04:34.775056", "rc": 0, "start": "2021-04-03 03:04:34.771772", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock"]} 2021-04-03T03:04:34.839 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.033) 0:01:58.993 ******** 2021-04-03T03:04:34.845 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-04-03T03:04:34.847 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.879 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.040) 0:01:59.034 ******** 2021-04-03T03:04:34.885 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-04-03T03:04:34.887 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.914 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.034) 0:01:59.068 ******** 2021-04-03T03:04:34.930 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-04-03T03:04:34.932 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-04-03T03:04:34.954 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.040) 0:01:59.108 ******** 2021-04-03T03:04:34.960 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-04-03T03:04:34.962 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:34.992 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:34 +0000 (0:00:00.037) 0:01:59.146 ******** 2021-04-03T03:04:34.998 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-04-03T03:04:35.000 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:35.023 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.031) 0:01:59.177 ******** 2021-04-03T03:04:35.029 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-04-03T03:04:35.030 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.053 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.030) 0:01:59.207 ******** 2021-04-03T03:04:35.059 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-04-03T03:04:35.060 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-04-03T03:04:35.093 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.039) 0:01:59.247 ******** 2021-04-03T03:04:35.100 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-04-03T03:04:35.102 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:35.126 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.032) 0:01:59.280 ******** 2021-04-03T03:04:35.152 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-04-03T03:04:35.154 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.176 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.050) 0:01:59.330 ******** 2021-04-03T03:04:35.202 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-04-03T03:04:35.204 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.224 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.048) 0:01:59.378 ******** 2021-04-03T03:04:35.250 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-04-03T03:04:35.253 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.283 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.058) 0:01:59.437 ******** 2021-04-03T03:04:35.310 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.336 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.052) 0:01:59.490 ******** 2021-04-03T03:04:35.342 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-04-03T03:04:35.344 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true} 2021-04-03T03:04:35.381 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.045) 0:01:59.535 ******** 2021-04-03T03:04:35.388 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-04-03T03:04:35.390 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.414 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.033) 0:01:59.568 ******** 2021-04-03T03:04:35.421 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:35.421 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmNJXAHqOys5KTkXFiDQquruvfScjeF9Rm'], 'uuids': ['7616791d-8cb3-4fcd-96e6-8a265a40b827']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.076262", "end": "2021-04-03 03:04:35.657958", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmNJXAHqOys5KTkXFiDQquruvfScjeF9Rm"], "labels": [], "masters": [], "uuids": ["7616791d-8cb3-4fcd-96e6-8a265a40b827"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:34.581696", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:35.423 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF', 'wwn-0x50000399ab801e8e'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF-part1', 'wwn-0x50000399ab801e8e-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801e8e', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K3PUF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF", "wwn-0x50000399ab801e8e"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF-part1", "wwn-0x50000399ab801e8e-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K3PUF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801e8e"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:35.425 INFO:teuthology.task.ansible.out:as False"} 2021-04-03T03:04:35.445 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-04-03T03:04:35.468 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.053) 0:01:59.622 ******** 2021-04-03T03:04:35.506 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-04-03T03:04:35.508 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.518 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.520 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.530 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.531 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.542 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.553 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-04-03T03:04:35.555 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.564 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.566 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.575 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.577 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.587 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.588 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.598 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.625 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.157) 0:01:59.779 ******** 2021-04-03T03:04:35.631 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-04-03T03:04:35.633 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:04:35.657 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.031) 0:01:59.811 ******** 2021-04-03T03:04:35.663 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-04-03T03:04:35.665 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-04-03T03:04:35.687 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.029) 0:01:59.841 ******** 2021-04-03T03:04:35.692 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-04-03T03:04:35.694 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.715 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.028) 0:01:59.869 ******** 2021-04-03T03:04:35.721 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-04-03T03:04:35.722 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.746 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.030) 0:01:59.900 ******** 2021-04-03T03:04:35.752 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:35.754 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item={'block_used': 13759006, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336102, 'size_available': 927072673792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336102, "block_size": 4096, "block_total": 240095108, "block_used": 13759006, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072673792, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}} 2021-04-03T03:04:35.781 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.035) 0:01:59.935 ******** 2021-04-03T03:04:35.787 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-04-03T03:04:35.789 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.817 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.035) 0:01:59.971 ******** 2021-04-03T03:04:35.823 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-04-03T03:04:35.824 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003506", "end": "2021-04-03 03:04:35.791734", "rc": 0, "start": "2021-04-03 03:04:35.788228", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-lock"]} 2021-04-03T03:04:35.862 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.044) 0:02:00.016 ******** 2021-04-03T03:04:35.870 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:35.872 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.873 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:35.874 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-rW1Ev1-c28g-ic05-CB3E-xrd2-PbeL-Pvc1qh', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400GH375A', 'nvme-nvme.8086-50484b4d393133343030474833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.130868", "end": "2021-04-03 03:04:36.001007", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-rW1Ev1-c28g-ic05-CB3E-xrd2-PbeL-Pvc1qh", "nvme-INTEL_SSDPEL1K375GA_PHKM913400GH375A", "nvme-nvme.8086-50484b4d393133343030474833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:34.870139", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now parti 2021-04-03T03:04:35.876 INFO:teuthology.task.ansible.out:tion the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:35.877 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.902 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.040) 0:02:00.056 ******** 2021-04-03T03:04:35.909 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.910 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-04-03T03:04:35.910 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002472", "end": "2021-04-03 03:04:35.869669", "rc": 0, "start": "2021-04-03 03:04:35.867197", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:35.912 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:35.935 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.033) 0:02:00.089 ******** 2021-04-03T03:04:35.968 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.033) 0:02:00.122 ******** 2021-04-03T03:04:35.974 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:35.976 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [gibba026.front.sepia.ceph.com] => (item={'block_used': 13759006, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336102, 'size_available': 927072673792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336102, "block_size": 4096, "block_total": 240095108, "block_used": 13759006, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072673792, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:36.001 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:35 +0000 (0:00:00.033) 0:02:00.155 ******** 2021-04-03T03:04:36.022 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.020) 0:02:00.176 ******** 2021-04-03T03:04:36.029 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-04-03T03:04:36.030 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:36.053 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.031) 0:02:00.207 ******** 2021-04-03T03:04:36.059 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-04-03T03:04:36.083 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.029) 0:02:00.237 ******** 2021-04-03T03:04:36.088 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-04-03T03:04:36.090 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:36.113 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.029) 0:02:00.267 ******** 2021-04-03T03:04:36.119 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-04-03T03:04:36.120 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:36.145 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.031) 0:02:00.299 ******** 2021-04-03T03:04:36.151 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:36.152 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:36.241 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-04-03T03:04:36.243 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:36.263 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.118) 0:02:00.418 ******** 2021-04-03T03:04:36.488 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-04-03T03:04:36.489 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-04-03T03:04:36.516 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.252) 0:02:00.670 ******** 2021-04-03T03:04:36.605 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:36.607 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-PMzt6e-ie2V-lf7Y-s2Ed-Ue8S-Rs8g-Udppg4', 'nvme-INTEL_SSDPEL1K375GA_PHKM8522008J375A', 'nvme-nvme.8086-50484b4d383532323030384a33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.121472", "end": "2021-04-03 03:04:36.875766", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-PMzt6e-ie2V-lf7Y-s2Ed-Ue8S-Rs8g-Udppg4", "nvme-INTEL_SSDPEL1K375GA_PHKM8522008J375A", "nvme-nvme.8086-50484b4d383532323030384a33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:35.754294", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now parti 2021-04-03T03:04:36.609 INFO:teuthology.task.ansible.out:tion the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:36.618 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:36.630 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:36.693 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-04-03T03:04:36.695 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:04:36.718 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.201) 0:02:00.872 ******** 2021-04-03T03:04:36.735 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-04-03T03:04:36.737 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-04-03T03:04:36.758 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.040) 0:02:00.913 ******** 2021-04-03T03:04:36.765 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-04-03T03:04:36.766 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:36.790 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.031) 0:02:00.944 ******** 2021-04-03T03:04:36.827 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:36.829 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item={'block_used': 13759008, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336100, 'size_available': 927072665600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336100, "block_size": 4096, "block_total": 240095108, "block_used": 13759008, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072665600, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}} 2021-04-03T03:04:36.852 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.061) 0:02:01.006 ******** 2021-04-03T03:04:36.951 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-04-03T03:04:36.953 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002508", "end": "2021-04-03 03:04:36.819743", "rc": 0, "start": "2021-04-03 03:04:36.817235", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:36.975 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:36 +0000 (0:00:00.123) 0:02:01.129 ******** 2021-04-03T03:04:37.012 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-04-03T03:04:37.014 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'block_used': 13759008, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336100, 'size_available': 927072665600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336100, "block_size": 4096, "block_total": 240095108, "block_used": 13759008, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072665600, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.038 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:37 +0000 (0:00:00.062) 0:02:01.192 ******** 2021-04-03T03:04:37.066 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:37.068 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dpibv7vB4KZQZYc68t8kDeDlditqNZAkY7'], 'uuids': ['22912828-621f-4e2c-91b5-a6577d1fbd90']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.089984", "end": "2021-04-03 03:04:37.206300", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dpibv7vB4KZQZYc68t8kDeDlditqNZAkY7"], "labels": [], "masters": [], "uuids": ["22912828-621f-4e2c-91b5-a6577d1fbd90"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:36.116316", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [gibba015.front.sepia.ceph.com] => (ite 2021-04-03T03:04:37.069 INFO:teuthology.task.ansible.out:m={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Wipe filesystems on non-root partitions] ********************** TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:37.069 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.094 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:37 +0000 (0:00:00.055) 0:02:01.248 ******** 2021-04-03T03:04:37.164 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.394 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-04-03T03:04:37.395 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:04:37.418 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:37 +0000 (0:00:00.323) 0:02:01.572 ******** 2021-04-03T03:04:37.542 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-04-03T03:04:37.544 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.025173", "end": "2021-04-03 03:04:37.755754", "rc": 0, "start": "2021-04-03 03:04:37.730581", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-04-03T03:04:37.565 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:37 +0000 (0:00:00.147) 0:02:01.719 ******** 2021-04-03T03:04:37.593 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-04-03T03:04:37.595 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.616 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:37 +0000 (0:00:00.050) 0:02:01.770 ******** 2021-04-03T03:04:37.641 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-04-03T03:04:37.643 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.673 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:37 +0000 (0:00:00.057) 0:02:01.827 ******** 2021-04-03T03:04:37.794 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:37.796 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecm8IKR6KBbvwdJXznY3WvkB8Coa3J4ejuz'], 'uuids': ['47babc2f-5128-4cb8-bbc3-42f0b2d7bbf3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.079841", "end": "2021-04-03 03:04:38.057861", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecm8IKR6KBbvwdJXznY3WvkB8Coa3J4ejuz"], "labels": [], "masters": [], "uuids": ["47babc2f-5128-4cb8-bbc3-42f0b2d7bbf3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:36.978020", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:37.796 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (ite 2021-04-03T03:04:37.798 INFO:teuthology.task.ansible.out:m={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.807 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.905 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsvOYM6lBYzGAkCBd1cLIrhieivVfYm7Y8'], 'uuids': ['16a1bc70-4fc6-4d96-ad7d-309cf92e343d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.661791", "end": "2021-04-03 03:04:37.882527", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsvOYM6lBYzGAkCBd1cLIrhieivVfYm7Y8"], "labels": [], "masters": [], "uuids": ["16a1bc70-4fc6-4d96-ad7d-309cf92e343d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:36.220736", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:37.917 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:37.929 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:38.245 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dp3Q2G2mSSlhFGRvw8HCZeZxy01Po6nPL3'], 'uuids': ['4170e064-87bf-47cb-be16-dcde13eb9c66']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.083775", "end": "2021-04-03 03:04:38.382143", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dp3Q2G2mSSlhFGRvw8HCZeZxy01Po6nPL3"], "labels": [], "masters": [], "uuids": ["4170e064-87bf-47cb-be16-dcde13eb9c66"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:37.298368", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:38.246 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:38.320 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-04-03T03:04:38.321 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:04:38.342 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:38 +0000 (0:00:00.669) 0:02:02.496 ******** 2021-04-03T03:04:38.369 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-04-03T03:04:38.371 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:38.392 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:38 +0000 (0:00:00.049) 0:02:02.546 ******** 2021-04-03T03:04:38.618 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-04-03T03:04:38.620 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-04-03T03:04:38.642 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:38 +0000 (0:00:00.250) 0:02:02.796 ******** 2021-04-03T03:04:38.669 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-04-03T03:04:38.670 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:38.692 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:38 +0000 (0:00:00.050) 0:02:02.846 ******** 2021-04-03T03:04:38.835 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-04-03T03:04:38.836 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ActiveEnterTimestampMonotonic": "24477768", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-journald.socket -.mount system.slice basic.target sysinit.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:00 UTC", "AssertTimestampMonotonic": "17896241", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:00 UTC", "ConditionTimestampMonotonic": "17896240", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Document 2021-04-03T03:04:38.838 INFO:teuthology.task.ansible.out:ation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1990", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:06 UTC", "ExecMainStartTimestampMonotonic": "24477731", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:00 UTC", "InactiveExitTimestampMonotonic": "17896665", "InvocationID": "34deec6dc8524301b71a6fb74f3ea4d7", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limit 2021-04-03T03:04:38.839 INFO:teuthology.task.ansible.out:RTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1990", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:06 UTC", "St 2021-04-03T03:04:38.841 INFO:teuthology.task.ansible.out:ateChangeTimestampMonotonic": "24477768", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:06 UTC", "WatchdogTimestampMonotonic": "24477766", "WatchdogUSec": "0"}} 2021-04-03T03:04:38.860 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:38 +0000 (0:00:00.168) 0:02:03.014 ******** 2021-04-03T03:04:38.887 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:38.889 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-04-03T03:04:38.911 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:38 +0000 (0:00:00.050) 0:02:03.065 ******** 2021-04-03T03:04:38.937 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:38.940 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:38.964 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:38 +0000 (0:00:00.053) 0:02:03.118 ******** 2021-04-03T03:04:38.973 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:38.974 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOR64p7i7F1K4dSiCwfXUTjFyFdeZOsv'], 'uuids': ['0ed49394-2471-4983-b581-24bd62fef3a4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.679825", "end": "2021-04-03 03:04:38.800176", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOR64p7i7F1K4dSiCwfXUTjFyFdeZOsv"], "labels": [], "masters": [], "uuids": ["0ed49394-2471-4983-b581-24bd62fef3a4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:37.120351", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:38.976 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:38.978 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:38.978 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmJgMm0f9JQjdcVI2Qki5qRyNfBRKwIeWp'], 'uuids': ['87344de7-317a-47ce-bfab-41bf3b82fac3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.087152", "end": "2021-04-03 03:04:39.240826", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmJgMm0f9JQjdcVI2Qki5qRyNfBRKwIeWp"], "labels": [], "masters": [], "uuids": ["87344de7-317a-47ce-bfab-41bf3b82fac3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sa 2021-04-03T03:04:38.980 INFO:teuthology.task.ansible.out:s_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:38.153674", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:38.991 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.002 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-04-03T03:04:39.004 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.029 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:39 +0000 (0:00:00.064) 0:02:03.183 ******** 2021-04-03T03:04:39.099 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:39.101 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsdfxTXbjl6nvaX9pbaUKG5SQLLt5hfEZO'], 'uuids': ['e1c1fc40-503c-4c13-b6c4-9ef53d6ed1e8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.082782", "end": "2021-04-03 03:04:39.069627", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsdfxTXbjl6nvaX9pbaUKG5SQLLt5hfEZO"], "labels": [], "masters": [], "uuids": ["e1c1fc40-503c-4c13-b6c4-9ef53d6ed1e8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:37.986845", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:39.102 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (i 2021-04-03T03:04:39.104 INFO:teuthology.task.ansible.out:tem={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF', 'wwn-0x50000399ab901d74'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF-part1', 'wwn-0x50000399ab901d74-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab901d74', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I5K314F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF", "wwn-0x50000399ab901d74"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF-part1", "wwn-0x50000399ab901d74-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I5K314F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab901d74"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.420 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DphssW7Nksn9Tib9eG4q9qztQWiet6imDW'], 'uuids': ['6128732f-e9af-4005-bb11-6044cbde3d53']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.089536", "end": "2021-04-03 03:04:39.566424", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DphssW7Nksn9Tib9eG4q9qztQWiet6imDW"], "labels": [], "masters": [], "uuids": ["6128732f-e9af-4005-bb11-6044cbde3d53"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:38.476888", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:39.467 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:39 +0000 (0:00:00.437) 0:02:03.621 ******** 2021-04-03T03:04:39.517 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:39.519 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.592 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-04-03T03:04:39.593 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": false} 2021-04-03T03:04:39.618 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:39 +0000 (0:00:00.151) 0:02:03.772 ******** 2021-04-03T03:04:39.646 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-04-03T03:04:39.648 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.670 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:39 +0000 (0:00:00.052) 0:02:03.825 ******** 2021-04-03T03:04:39.678 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:39.680 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpILiI8nEeN6SehkzpksQLJ61YJbPe75Iq'], 'uuids': ['eee40ada-2c82-4a0b-85be-d30abf560de9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.075896", "end": "2021-04-03 03:04:39.816264", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpILiI8nEeN6SehkzpksQLJ61YJbPe75Iq"], "labels": [], "masters": [], "uuids": ["eee40ada-2c82-4a0b-85be-d30abf560de9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:39.740368", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.074731 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.074731 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:39.681 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.683 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.779 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-04-03T03:04:39.781 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:39.803 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:39 +0000 (0:00:00.132) 0:02:03.957 ******** 2021-04-03T03:04:39.842 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:39.844 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpjYRcCEUdv7wDLlWKJrjueoeXtD6075iU'], 'uuids': ['389d7121-154b-44ec-aaee-58b53d75ba64']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.076417", "end": "2021-04-03 03:04:39.984602", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DpjYRcCEUdv7wDLlWKJrjueoeXtD6075iU"], "labels": [], "masters": [], "uuids": ["389d7121-154b-44ec-aaee-58b53d75ba64"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:39.908185", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0752067 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0752067 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:39.844 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF', 'wwn-0x50000399ab801e93'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labe 2021-04-03T03:04:39.846 INFO:teuthology.task.ansible.out:ls': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF-part1', 'wwn-0x50000399ab801e93-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801e93', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K3PZF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF", "wwn-0x50000399ab801e93"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PZF6XF-part1", "wwn-0x50000399ab801e93-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K3PZF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801e93"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.961 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-rW1Ev1-c28g-ic05-CB3E-xrd2-PbeL-Pvc1qh', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400GH375A', 'nvme-nvme.8086-50484b4d393133343030474833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.028098", "end": "2021-04-03 03:04:40.101444", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-rW1Ev1-c28g-ic05-CB3E-xrd2-PbeL-Pvc1qh", "nvme-INTEL_SSDPEL1K375GA_PHKM913400GH375A", "nvme-nvme.8086-50484b4d393133343030474833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.073346", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0268889 s, 4.3 GB/s", "stderr_lines": ["110 2021-04-03T03:04:39.963 INFO:teuthology.task.ansible.out:+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0268889 s, 4.3 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:39.974 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.038 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-04-03T03:04:40.040 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-04-03T03:04:40.061 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:40 +0000 (0:00:00.258) 0:02:04.215 ******** 2021-04-03T03:04:40.141 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:40.142 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dpibv7vB4KZQZYc68t8kDeDlditqNZAkY7'], 'uuids': ['22912828-621f-4e2c-91b5-a6577d1fbd90']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.081133", "end": "2021-04-03 03:04:40.285069", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dpibv7vB4KZQZYc68t8kDeDlditqNZAkY7"], "labels": [], "masters": [], "uuids": ["22912828-621f-4e2c-91b5-a6577d1fbd90"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.203936", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0798179 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0798179 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:40.143 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmrbGsIKskmjzJciwpgKfxdW4VFFhSCiJm'], 'uuids': ['264999d1-9415-4ab8-9b98-47a9c105ccaf']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.078342", "end": "2021-04-03 03:04:40.407753", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmrbGsIKskmjzJciwpgKfxdW4VFFhSCiJm"], "labels": [], "masters": [], "uuids": ["264999d1-9415-4ab8-9b98-47a9c105ccaf"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:39.329411", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk us 2021-04-03T03:04:40.145 INFO:teuthology.task.ansible.out:ing fdisk or", "other utilities."]} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.157 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.158 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:40.159 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzTlgWNzn4ZXHSg4GXdOMEQp3l2n919OZ7'], 'uuids': ['4a5b8710-6c97-4de6-a752-0bee6baaf1da']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.104137", "end": "2021-04-03 03:04:40.025319", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzTlgWNzn4ZXHSg4GXdOMEQp3l2n919OZ7"], "labels": [], "masters": [], "uuids": ["4a5b8710-6c97-4de6-a752-0bee6baaf1da"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:38.921182", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk us 2021-04-03T03:04:40.160 INFO:teuthology.task.ansible.out:ing fdisk or", "other utilities."]} 2021-04-03T03:04:40.171 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF', 'wwn-0x50000399aba81669'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF-part1', 'wwn-0x50000399aba81669-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399aba81669', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I8K38JF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF", "wwn-0x50000399aba81669"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF-part1", "wwn-0x50000399aba81669-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I8K38JF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399aba81669"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:40.173 INFO:teuthology.task.ansible.out:as False"} 2021-04-03T03:04:40.270 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:40 +0000 (0:00:00.208) 0:02:04.424 ******** 2021-04-03T03:04:40.312 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-lm0WGu-LoIt-lnq6-IumI-LxNU-kMxi-LMwkVB', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200003J375A', 'nvme-nvme.8086-50484b4d393230303030334a33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.107887", "end": "2021-04-03 03:04:40.281523", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-lm0WGu-LoIt-lnq6-IumI-LxNU-kMxi-LMwkVB", "nvme-INTEL_SSDPEL1K375GA_PHKM9200003J375A", "nvme-nvme.8086-50484b4d393230303030334a33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:39.173636", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now parti 2021-04-03T03:04:40.314 INFO:teuthology.task.ansible.out:tion the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.325 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:40.327 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dp3Q2G2mSSlhFGRvw8HCZeZxy01Po6nPL3'], 'uuids': ['4170e064-87bf-47cb-be16-dcde13eb9c66']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.080281", "end": "2021-04-03 03:04:40.463773", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5Dp3Q2G2mSSlhFGRvw8HCZeZxy01Po6nPL3"], "labels": [], "masters": [], "uuids": ["4170e064-87bf-47cb-be16-dcde13eb9c66"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.383492", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0790677 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0790677 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': 2021-04-03T03:04:40.328 INFO:teuthology.task.ansible.out:'0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.340 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.488 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DphssW7Nksn9Tib9eG4q9qztQWiet6imDW'], 'uuids': ['6128732f-e9af-4005-bb11-6044cbde3d53']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.080786", "end": "2021-04-03 03:04:40.633574", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-JOCpjJw1x1ZAwTT8zCPhtwtGVKfdL5DphssW7Nksn9Tib9eG4q9qztQWiet6imDW"], "labels": [], "masters": [], "uuids": ["6128732f-e9af-4005-bb11-6044cbde3d53"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.552788", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0795111 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0795111 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:40.500 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmqP2taNqkPN10BPfYoF1bclpluY1jOTtJ'], 'uuids': ['b1485987-4a60-43c5-801e-e74185106c09']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.075994", "end": "2021-04-03 03:04:40.766397", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmqP2taNqkPN10BPfYoF1bclpluY1jOTtJ"], "labels": [], "masters": [], "uuids": ["b1485987-4a60-43c5-801e-e74185106c09"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.690403", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0747261 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0747261 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:40.524 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:40 +0000 (0:00:00.254) 0:02:04.678 ******** 2021-04-03T03:04:40.532 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.533 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.671 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmNJXAHqOys5KTkXFiDQquruvfScjeF9Rm'], 'uuids': ['7616791d-8cb3-4fcd-96e6-8a265a40b827']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.076605", "end": "2021-04-03 03:04:40.938082", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmNJXAHqOys5KTkXFiDQquruvfScjeF9Rm"], "labels": [], "masters": [], "uuids": ["7616791d-8cb3-4fcd-96e6-8a265a40b827"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.861477", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0753869 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0753869 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:40.683 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF', 'wwn-0x50000399ab801e8e'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF-part1', 'wwn-0x50000399ab801e8e-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab801e8e', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I3K3PUF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF", "wwn-0x50000399ab801e8e"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I3K3PUF6XF-part1", "wwn-0x50000399ab801e8e-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I3K3PUF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab801e8e"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:40.685 INFO:teuthology.task.ansible.out:as False"} 2021-04-03T03:04:40.738 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-04-03T03:04:40.740 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.115526", "end": "2021-04-03 03:04:40.875378", "rc": 0, "start": "2021-04-03 03:04:40.759852", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:40.762 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:40 +0000 (0:00:00.237) 0:02:04.916 ******** 2021-04-03T03:04:40.791 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:40.793 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-PMzt6e-ie2V-lf7Y-s2Ed-Ue8S-Rs8g-Udppg4', 'nvme-INTEL_SSDPEL1K375GA_PHKM8522008J375A', 'nvme-nvme.8086-50484b4d383532323030384a33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.027235", "end": "2021-04-03 03:04:41.054205", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-PMzt6e-ie2V-lf7Y-s2Ed-Ue8S-Rs8g-Udppg4", "nvme-INTEL_SSDPEL1K375GA_PHKM8522008J375A", "nvme-nvme.8086-50484b4d383532323030384a33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:41.026970", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0259677 s, 4.4 GB/s", "stderr_lines": ["110 2021-04-03T03:04:40.795 INFO:teuthology.task.ansible.out:+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0259677 s, 4.4 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.805 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.817 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-04-03T03:04:40.818 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-04-03T03:04:40.844 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:40 +0000 (0:00:00.081) 0:02:04.998 ******** 2021-04-03T03:04:40.893 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-04-03T03:04:40.895 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.038526", "end": "2021-04-03 03:04:41.039698", "rc": 0, "start": "2021-04-03 03:04:41.001172", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:40.919 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:40 +0000 (0:00:00.075) 0:02:05.073 ******** 2021-04-03T03:04:40.948 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-04-03T03:04:40.975 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:40 +0000 (0:00:00.055) 0:02:05.129 ******** 2021-04-03T03:04:40.983 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:40.984 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecm8IKR6KBbvwdJXznY3WvkB8Coa3J4ejuz'], 'uuids': ['47babc2f-5128-4cb8-bbc3-42f0b2d7bbf3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.080284", "end": "2021-04-03 03:04:41.230666", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecm8IKR6KBbvwdJXznY3WvkB8Coa3J4ejuz"], "labels": [], "masters": [], "uuids": ["47babc2f-5128-4cb8-bbc3-42f0b2d7bbf3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:41.150382", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0790796 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0790796 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:40.985 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.987 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:40.998 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.025 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.049) 0:02:05.179 ******** 2021-04-03T03:04:41.053 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-04-03T03:04:41.054 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.077 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.052) 0:02:05.231 ******** 2021-04-03T03:04:41.105 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-04-03T03:04:41.106 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.135 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.058) 0:02:05.290 ******** 2021-04-03T03:04:41.143 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:41.144 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmJgMm0f9JQjdcVI2Qki5qRyNfBRKwIeWp'], 'uuids': ['87344de7-317a-47ce-bfab-41bf3b82fac3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.080495", "end": "2021-04-03 03:04:41.407700", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmJgMm0f9JQjdcVI2Qki5qRyNfBRKwIeWp"], "labels": [], "masters": [], "uuids": ["87344de7-317a-47ce-bfab-41bf3b82fac3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:41.327205", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0792246 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0792246 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:41.157 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.168 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.192 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.056) 0:02:05.346 ******** 2021-04-03T03:04:41.210 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-04-03T03:04:41.211 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false} 2021-04-03T03:04:41.236 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.043) 0:02:05.390 ******** 2021-04-03T03:04:41.266 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-04-03T03:04:41.268 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-04-03T03:04:41.290 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.053) 0:02:05.444 ******** 2021-04-03T03:04:41.297 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-04-03T03:04:41.299 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:41.328 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.038) 0:02:05.482 ******** 2021-04-03T03:04:41.336 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:41.337 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmrbGsIKskmjzJciwpgKfxdW4VFFhSCiJm'], 'uuids': ['264999d1-9415-4ab8-9b98-47a9c105ccaf']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.076840", "end": "2021-04-03 03:04:41.585728", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-vB2VA68heEw1dO6cVWJhV214QEjbLecmrbGsIKskmjzJciwpgKfxdW4VFFhSCiJm"], "labels": [], "masters": [], "uuids": ["264999d1-9415-4ab8-9b98-47a9c105ccaf"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:41.508888", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0756348 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0756348 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:41.375 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.046) 0:02:05.529 ******** 2021-04-03T03:04:41.382 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:41.383 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'block_used': 13759007, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336101, 'size_available': 927072669696, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336101, "block_size": 4096, "block_total": 240095108, "block_used": 13759007, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072669696, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.411 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.035) 0:02:05.565 ******** 2021-04-03T03:04:41.421 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-04-03T03:04:41.423 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1617418606.5859957, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1617418606.5819976, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 10060, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1617418606.5819976, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3104356462", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-04-03T03:04:41.423 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:41.425 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-XIJfjr-f5kZ-hems-h6uq-TLgR-LsDv-eLxmoK', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400L8375A', 'nvme-nvme.8086-50484b4d3931333430304c3833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.144110", "end": "2021-04-03 03:04:41.260623", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-XIJfjr-f5kZ-hems-h6uq-TLgR-LsDv-eLxmoK", "nvme-INTEL_SSDPEL1K375GA_PHKM913400L8375A", "nvme-nvme.8086-50484b4d3931333430304c3833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.116513", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now parti 2021-04-03T03:04:41.426 INFO:teuthology.task.ansible.out:tion the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional 2021-04-03T03:04:41.428 INFO:teuthology.task.ansible.out: result was False"} 2021-04-03T03:04:41.451 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.040) 0:02:05.605 ******** 2021-04-03T03:04:41.459 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-04-03T03:04:41.461 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.485 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.034) 0:02:05.639 ******** 2021-04-03T03:04:41.503 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:41.505 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcszz7UkqTHwjAGfvzLGSaVmdmUPjwS0AJa'], 'uuids': ['85ff5d16-042c-4593-bf1f-1b6c449460dd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.089733", "end": "2021-04-03 03:04:41.470606", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcszz7UkqTHwjAGfvzLGSaVmdmUPjwS0AJa"], "labels": [], "masters": [], "uuids": ["85ff5d16-042c-4593-bf1f-1b6c449460dd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:40.380873", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:41.516 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.517 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:41.517 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.518 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:41.518 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.542 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.056) 0:02:05.696 ******** 2021-04-03T03:04:41.570 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:41.572 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.600 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.058) 0:02:05.754 ******** 2021-04-03T03:04:41.629 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-04-03T03:04:41.631 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.653 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.052) 0:02:05.807 ******** 2021-04-03T03:04:41.661 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-04-03T03:04:41.663 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.157406", "end": "2021-04-03 03:04:41.895210", "rc": 0, "start": "2021-04-03 03:04:41.737804", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:41.688 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.035) 0:02:05.842 ******** 2021-04-03T03:04:41.695 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-04-03T03:04:41.697 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.722 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.033) 0:02:05.876 ******** 2021-04-03T03:04:41.760 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-04-03T03:04:41.762 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.785 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.062) 0:02:05.939 ******** 2021-04-03T03:04:41.803 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-04-03T03:04:41.805 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-04-03T03:04:41.829 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.044) 0:02:05.983 ******** 2021-04-03T03:04:41.838 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-04-03T03:04:41.839 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.033973", "end": "2021-04-03 03:04:42.089197", "rc": 0, "start": "2021-04-03 03:04:42.055224", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:41.865 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.035) 0:02:06.019 ******** 2021-04-03T03:04:41.872 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-04-03T03:04:41.874 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-04-03T03:04:41.897 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.032) 0:02:06.051 ******** 2021-04-03T03:04:41.904 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-04-03T03:04:41.932 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.035) 0:02:06.086 ******** 2021-04-03T03:04:41.939 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-04-03T03:04:41.941 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:41.964 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.031) 0:02:06.118 ******** 2021-04-03T03:04:41.971 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-04-03T03:04:41.973 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.000 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:41 +0000 (0:00:00.035) 0:02:06.154 ******** 2021-04-03T03:04:42.028 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-04-03T03:04:42.030 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.052 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.051) 0:02:06.206 ******** 2021-04-03T03:04:42.070 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-04-03T03:04:42.072 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.105 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.052) 0:02:06.259 ******** 2021-04-03T03:04:42.132 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.027) 0:02:06.286 ******** 2021-04-03T03:04:42.160 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-04-03T03:04:42.161 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create volume group(s)] *************************************** changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-04-03T03:04:42.185 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.053) 0:02:06.339 ******** 2021-04-03T03:04:42.210 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.024) 0:02:06.364 ******** 2021-04-03T03:04:42.217 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-04-03T03:04:42.218 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-04-03T03:04:42.243 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.032) 0:02:06.397 ******** 2021-04-03T03:04:42.273 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-04-03T03:04:42.275 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.309 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.065) 0:02:06.463 ******** 2021-04-03T03:04:42.319 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-04-03T03:04:42.320 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:42.348 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.039) 0:02:06.502 ******** 2021-04-03T03:04:42.359 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-04-03T03:04:42.360 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.362 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.364 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.393 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.044) 0:02:06.547 ******** 2021-04-03T03:04:42.400 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:42.401 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Set root disk] ************************************************ skipping: [gibba031.front.sepia.ceph.com] => (item={'block_used': 13759008, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336100, 'size_available': 927072665600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149514, 'block_size': 4096, 'inode_available': 60905462}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336100, "block_size": 4096, "block_total": 240095108, "block_used": 13759008, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905462, "inode_total": 61054976, "inode_used": 149514, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072665600, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.428 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.034) 0:02:06.582 ******** 2021-04-03T03:04:42.434 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-04-03T03:04:42.436 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.450 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.452 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba011.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.476 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.048) 0:02:06.631 ******** 2021-04-03T03:04:42.483 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-04-03T03:04:42.485 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.506 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.029) 0:02:06.660 ******** 2021-04-03T03:04:42.512 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:42.514 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.539 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.032) 0:02:06.693 ******** 2021-04-03T03:04:42.545 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:42.547 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : set_fact] ***************************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.574 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.035) 0:02:06.728 ******** 2021-04-03T03:04:42.580 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-04-03T03:04:42.582 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.605 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.030) 0:02:06.759 ******** 2021-04-03T03:04:42.611 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:42.613 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOU6erj0KacFR61BsHLRGAvLTVq6tszt'], 'uuids': ['2b196a9d-85fb-4bf8-b606-9cd65e1348d8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.093149", "end": "2021-04-03 03:04:42.453024", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOU6erj0KacFR61BsHLRGAvLTVq6tszt"], "labels": [], "masters": [], "uuids": ["2b196a9d-85fb-4bf8-b606-9cd65e1348d8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:41.359875", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:42.613 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (ite 2021-04-03T03:04:42.615 INFO:teuthology.task.ansible.out:m={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.615 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual" 2021-04-03T03:04:42.617 INFO:teuthology.task.ansible.out:: 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.642 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.037) 0:02:06.796 ******** 2021-04-03T03:04:42.648 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-04-03T03:04:42.650 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.675 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.032) 0:02:06.829 ******** 2021-04-03T03:04:42.681 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-04-03T03:04:42.683 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.705 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.030) 0:02:06.859 ******** 2021-04-03T03:04:42.711 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:42.712 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsM6cqtIzGVxNj0XUBhs8dF7TXU9uYkeYF'], 'uuids': ['4ec264d0-7718-4afd-867c-404bf1dec257']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.080933", "end": "2021-04-03 03:04:42.662630", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsM6cqtIzGVxNj0XUBhs8dF7TXU9uYkeYF"], "labels": [], "masters": [], "uuids": ["4ec264d0-7718-4afd-867c-404bf1dec257"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:41.581697", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:42.713 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.715 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:42.715 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.740 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.035) 0:02:06.894 ******** 2021-04-03T03:04:42.747 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-04-03T03:04:42.751 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.775 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.034) 0:02:06.929 ******** 2021-04-03T03:04:42.780 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-04-03T03:04:42.781 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.806 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.031) 0:02:06.960 ******** 2021-04-03T03:04:42.813 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-04-03T03:04:42.815 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.837 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.031) 0:02:06.992 ******** 2021-04-03T03:04:42.843 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-04-03T03:04:42.845 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.870 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.032) 0:02:07.025 ******** 2021-04-03T03:04:42.877 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:42.879 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:42.900 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.925 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.054) 0:02:07.079 ******** 2021-04-03T03:04:42.953 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-04-03T03:04:42.955 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:42.976 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:42 +0000 (0:00:00.051) 0:02:07.130 ******** 2021-04-03T03:04:42.995 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:42.997 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:43.031 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.054) 0:02:07.185 ******** 2021-04-03T03:04:43.101 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-04-03T03:04:43.103 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002303", "end": "2021-04-03 03:04:43.299708", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-04-03 03:04:43.297405", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:43.113 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-04-03T03:04:43.114 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-04-03T03:04:43.137 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.106) 0:02:07.291 ******** 2021-04-03T03:04:43.144 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-04-03T03:04:43.146 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-04-03T03:04:43.170 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.032) 0:02:07.324 ******** 2021-04-03T03:04:43.187 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-04-03T03:04:43.189 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001476", "end": "2021-04-03 03:04:43.385961", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-04-03 03:04:43.384485", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:43.252 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001333", "end": "2021-04-03 03:04:43.464475", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-04-03 03:04:43.463142", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:43.284 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:43.286 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:43.327 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:43.329 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [gibba011.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.002046", "end": "2021-04-03 03:04:43.542398", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-04-03 03:04:43.540352", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:43.356 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.185) 0:02:07.510 ******** 2021-04-03T03:04:43.363 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:43.364 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:43.385 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-04-03T03:04:43.409 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.052) 0:02:07.563 ******** 2021-04-03T03:04:43.447 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:43.448 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:43.459 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:43.461 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:43.502 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-04-03T03:04:43.504 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-04-03T03:04:43.528 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:43.530 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:43.530 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:43.565 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.156) 0:02:07.719 ******** 2021-04-03T03:04:43.592 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-04-03T03:04:43.594 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:43.615 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.049) 0:02:07.769 ******** 2021-04-03T03:04:43.642 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-04-03T03:04:43.643 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:43.665 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.050) 0:02:07.819 ******** 2021-04-03T03:04:43.671 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:43.673 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:43.767 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-04-03T03:04:43.769 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:43.791 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.125) 0:02:07.945 ******** 2021-04-03T03:04:43.808 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:43.810 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzcMXDmUr43QfkARenp5sR7caqWPxopJmM'], 'uuids': ['86d9d391-2116-4d91-830f-3cfbdc4bba18']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.102346", "end": "2021-04-03 03:04:43.663102", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzcMXDmUr43QfkARenp5sR7caqWPxopJmM"], "labels": [], "masters": [], "uuids": ["86d9d391-2116-4d91-830f-3cfbdc4bba18"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:42.560756", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Create logical volume(s)] ** 2021-04-03T03:04:43.812 INFO:teuthology.task.ansible.out:*********************************** changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:43.823 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-04-03T03:04:43.825 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-04-03T03:04:43.846 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.055) 0:02:08.000 ******** 2021-04-03T03:04:43.871 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:43.873 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcs3sOHE9oJ7Vch7U8q021C4bg1Azk9PrWZ'], 'uuids': ['eae84607-f876-42ab-a043-5ea034fd583f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.079575", "end": "2021-04-03 03:04:43.837548", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcs3sOHE9oJ7Vch7U8q021C4bg1Azk9PrWZ"], "labels": [], "masters": [], "uuids": ["eae84607-f876-42ab-a043-5ea034fd583f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:42.757973", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:43.907 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.060) 0:02:08.061 ******** 2021-04-03T03:04:43.955 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:43.958 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:43.985 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:43 +0000 (0:00:00.078) 0:02:08.139 ******** 2021-04-03T03:04:43.992 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-04-03T03:04:43.994 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true} 2021-04-03T03:04:44.025 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.040) 0:02:08.179 ******** 2021-04-03T03:04:44.086 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-04-03T03:04:44.087 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-04-03T03:04:44.111 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.086) 0:02:08.265 ******** 2021-04-03T03:04:44.128 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:44.129 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsvOYM6lBYzGAkCBd1cLIrhieivVfYm7Y8'], 'uuids': ['16a1bc70-4fc6-4d96-ad7d-309cf92e343d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.075524", "end": "2021-04-03 03:04:44.097366", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsvOYM6lBYzGAkCBd1cLIrhieivVfYm7Y8"], "labels": [], "masters": [], "uuids": ["16a1bc70-4fc6-4d96-ad7d-309cf92e343d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:44.021842", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0742576 s, 1.6 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0742576 s, 1.6 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:44.130 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': 2021-04-03T03:04:44.132 INFO:teuthology.task.ansible.out: '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.185 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-04-03T03:04:44.187 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-04-03T03:04:44.209 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.098) 0:02:08.363 ******** 2021-04-03T03:04:44.236 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-04-03T03:04:44.238 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.260 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.050) 0:02:08.414 ******** 2021-04-03T03:04:44.267 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:44.269 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:44.279 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-04-03T03:04:44.281 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.303 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.042) 0:02:08.457 ******** 2021-04-03T03:04:44.321 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:44.322 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsdfxTXbjl6nvaX9pbaUKG5SQLLt5hfEZO'], 'uuids': ['e1c1fc40-503c-4c13-b6c4-9ef53d6ed1e8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.076138", "end": "2021-04-03 03:04:44.292229", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsdfxTXbjl6nvaX9pbaUKG5SQLLt5hfEZO"], "labels": [], "masters": [], "uuids": ["e1c1fc40-503c-4c13-b6c4-9ef53d6ed1e8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:44.216091", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.074919 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.074919 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:44.324 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF', 'wwn-0x50000399ab901d74'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF-part1', 'wwn-0x50000399ab901d74-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab901d74', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I5K314F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF", "wwn-0x50000399ab901d74"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I5K314F6XF-part1", "wwn-0x50000399ab901d74-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I5K314F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab901d74"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:44.326 INFO:teuthology.task.ansible.out:as False"} TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:44.327 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.350 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.046) 0:02:08.504 ******** 2021-04-03T03:04:44.356 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:44.358 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:44.368 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:44.370 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.392 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.042) 0:02:08.546 ******** 2021-04-03T03:04:44.429 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:44.431 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:44.473 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:44.475 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-lm0WGu-LoIt-lnq6-IumI-LxNU-kMxi-LMwkVB', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200003J375A', 'nvme-nvme.8086-50484b4d393230303030334a33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.023725", "end": "2021-04-03 03:04:44.449347", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-lm0WGu-LoIt-lnq6-IumI-LxNU-kMxi-LMwkVB", "nvme-INTEL_SSDPEL1K375GA_PHKM9200003J375A", "nvme-nvme.8086-50484b4d393230303030334a33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:44.425622", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0225393 s, 5.1 GB/s", "stderr_lines": ["110 2021-04-03T03:04:44.476 INFO:teuthology.task.ansible.out:+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0225393 s, 5.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:44.485 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.486 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.509 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-04-03T03:04:44.511 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:44.512 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:44.512 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:44.539 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.146) 0:02:08.693 ******** 2021-04-03T03:04:44.547 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.577 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.037) 0:02:08.731 ******** 2021-04-03T03:04:44.605 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-04-03T03:04:44.606 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.629 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.052) 0:02:08.784 ******** 2021-04-03T03:04:44.647 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-04-03T03:04:44.649 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:44.650 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcszz7UkqTHwjAGfvzLGSaVmdmUPjwS0AJa'], 'uuids': ['85ff5d16-042c-4593-bf1f-1b6c449460dd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.076725", "end": "2021-04-03 03:04:44.622918", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcszz7UkqTHwjAGfvzLGSaVmdmUPjwS0AJa"], "labels": [], "masters": [], "uuids": ["85ff5d16-042c-4593-bf1f-1b6c449460dd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:44.546193", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0755147 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0755147 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:44.673 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.043) 0:02:08.827 ******** 2021-04-03T03:04:44.680 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.681 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.683 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.710 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.036) 0:02:08.864 ******** 2021-04-03T03:04:44.811 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-04-03T03:04:44.813 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:44.837 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.127) 0:02:08.991 ******** 2021-04-03T03:04:44.844 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:44.845 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsM6cqtIzGVxNj0XUBhs8dF7TXU9uYkeYF'], 'uuids': ['4ec264d0-7718-4afd-867c-404bf1dec257']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.076800", "end": "2021-04-03 03:04:44.809545", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcsM6cqtIzGVxNj0XUBhs8dF7TXU9uYkeYF"], "labels": [], "masters": [], "uuids": ["4ec264d0-7718-4afd-867c-404bf1dec257"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:44.732745", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0755822 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0755822 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:44.847 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:44.869 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-04-03T03:04:44.871 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-04-03T03:04:44.894 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.056) 0:02:09.048 ******** 2021-04-03T03:04:44.910 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-04-03T03:04:44.912 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-04-03T03:04:44.936 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:44 +0000 (0:00:00.042) 0:02:09.090 ******** 2021-04-03T03:04:44.976 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-04-03T03:04:44.977 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzUdsc2jPrMOWKA7j4y2CaTVRI0MShrkND'], 'uuids': ['e8b0dbf1-1261-4039-bcc2-e401c35c12f3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.084401", "end": "2021-04-03 03:04:44.840277", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzUdsc2jPrMOWKA7j4y2CaTVRI0MShrkND"], "labels": [], "masters": [], "uuids": ["e8b0dbf1-1261-4039-bcc2-e401c35c12f3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:43.755876", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-04-03T03:04:45.020 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.083) 0:02:09.174 ******** 2021-04-03T03:04:45.035 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-04-03T03:04:45.037 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:45.038 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcs3sOHE9oJ7Vch7U8q021C4bg1Azk9PrWZ'], 'uuids': ['eae84607-f876-42ab-a043-5ea034fd583f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.075831", "end": "2021-04-03 03:04:44.996807", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-IBzDqB2prn7hB27rq5TkEQKJVJ8npxcs3sOHE9oJ7Vch7U8q021C4bg1Azk9PrWZ"], "labels": [], "masters": [], "uuids": ["eae84607-f876-42ab-a043-5ea034fd583f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:44.920976", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0746564 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0746564 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:45.081 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.061) 0:02:09.235 ******** 2021-04-03T03:04:45.105 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.023) 0:02:09.259 ******** 2021-04-03T03:04:45.113 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.169 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-04-03T03:04:45.171 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-04-03T03:04:45.200 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.095) 0:02:09.354 ******** 2021-04-03T03:04:45.241 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:45.243 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOR64p7i7F1K4dSiCwfXUTjFyFdeZOsv'], 'uuids': ['0ed49394-2471-4983-b581-24bd62fef3a4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.075509", "end": "2021-04-03 03:04:45.111183", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOR64p7i7F1K4dSiCwfXUTjFyFdeZOsv"], "labels": [], "masters": [], "uuids": ["0ed49394-2471-4983-b581-24bd62fef3a4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:45.035674", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.074249 s, 1.6 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.074249 s, 1.6 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:45.254 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.265 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.278 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-04-03T03:04:45.279 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-04-03T03:04:45.303 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.102) 0:02:09.457 ******** 2021-04-03T03:04:45.330 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-04-03T03:04:45.332 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.354 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.051) 0:02:09.508 ******** 2021-04-03T03:04:45.370 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-04-03T03:04:45.372 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.167901", "end": "2021-04-03 03:04:45.333219", "rc": 0, "start": "2021-04-03 03:04:45.165318", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:45.395 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.041) 0:02:09.549 ******** 2021-04-03T03:04:45.402 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-04-03T03:04:45.403 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.428 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.032) 0:02:09.582 ******** 2021-04-03T03:04:45.436 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:45.437 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzTlgWNzn4ZXHSg4GXdOMEQp3l2n919OZ7'], 'uuids': ['4a5b8710-6c97-4de6-a752-0bee6baaf1da']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.075817", "end": "2021-04-03 03:04:45.296701", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzTlgWNzn4ZXHSg4GXdOMEQp3l2n919OZ7"], "labels": [], "masters": [], "uuids": ["4a5b8710-6c97-4de6-a752-0bee6baaf1da"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:45.220884", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0746184 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0746184 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:45.439 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF', 'wwn-0x50000399aba81669'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF-part1', 'wwn-0x50000399aba81669-part1'], 'uuids': ['544fff2a-e246-4637-96d0-94b4cd4bcfea']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399aba81669', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I8K38JF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF", "wwn-0x50000399aba81669"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38JF6XF-part1", "wwn-0x50000399aba81669-part1"], "labels": [], "masters": [], "uuids": ["544fff2a-e246-4637-96d0-94b4cd4bcfea"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I8K38JF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399aba81669"}}, "skip_reason": "Conditional result w 2021-04-03T03:04:45.440 INFO:teuthology.task.ansible.out:as False"} 2021-04-03T03:04:45.451 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:45.453 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.475 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.047) 0:02:09.629 ******** 2021-04-03T03:04:45.502 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:45.504 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.526 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.050) 0:02:09.680 ******** 2021-04-03T03:04:45.544 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-04-03T03:04:45.546 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.035123", "end": "2021-04-03 03:04:45.509393", "rc": 0, "start": "2021-04-03 03:04:45.474270", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:45.570 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.044) 0:02:09.724 ******** 2021-04-03T03:04:45.580 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:45.581 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-XIJfjr-f5kZ-hems-h6uq-TLgR-LsDv-eLxmoK', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400L8375A', 'nvme-nvme.8086-50484b4d3931333430304c3833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.023695", "end": "2021-04-03 03:04:45.423480", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane DC P4800X Series SSD", "links": {"ids": ["lvm-pv-uuid-XIJfjr-f5kZ-hems-h6uq-TLgR-LsDv-eLxmoK", "nvme-INTEL_SSDPEL1K375GA_PHKM913400L8375A", "nvme-nvme.8086-50484b4d3931333430304c3833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:45.399785", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0224408 s, 5.1 GB/s", "stderr_lines": ["110 2021-04-03T03:04:45.582 INFO:teuthology.task.ansible.out:+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0224408 s, 5.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:45.584 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.596 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-04-03T03:04:45.623 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.050) 0:02:09.775 ******** 2021-04-03T03:04:45.641 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-04-03T03:04:45.641 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:45.669 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.048) 0:02:09.823 ******** 2021-04-03T03:04:45.675 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-04-03T03:04:45.677 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.706 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.037) 0:02:09.861 ******** 2021-04-03T03:04:45.734 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-04-03T03:04:45.736 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.762 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.055) 0:02:09.916 ******** 2021-04-03T03:04:45.768 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:45.769 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOU6erj0KacFR61BsHLRGAvLTVq6tszt'], 'uuids': ['2b196a9d-85fb-4bf8-b606-9cd65e1348d8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.075945", "end": "2021-04-03 03:04:45.621383", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzEOU6erj0KacFR61BsHLRGAvLTVq6tszt"], "labels": [], "masters": [], "uuids": ["2b196a9d-85fb-4bf8-b606-9cd65e1348d8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:45.545438", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.074793 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.074793 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:45.771 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:45.772 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:45.773 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.774 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.798 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.035) 0:02:09.952 ******** 2021-04-03T03:04:45.803 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-04-03T03:04:45.805 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.833 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.035) 0:02:09.987 ******** 2021-04-03T03:04:45.839 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-04-03T03:04:45.840 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:04:45.867 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.033) 0:02:10.021 ******** 2021-04-03T03:04:45.876 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-04-03T03:04:45.878 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.902 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:45 +0000 (0:00:00.035) 0:02:10.056 ******** 2021-04-03T03:04:45.992 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:45.994 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzcMXDmUr43QfkARenp5sR7caqWPxopJmM'], 'uuids': ['86d9d391-2116-4d91-830f-3cfbdc4bba18']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.076153", "end": "2021-04-03 03:04:45.853305", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzcMXDmUr43QfkARenp5sR7caqWPxopJmM"], "labels": [], "masters": [], "uuids": ["86d9d391-2116-4d91-830f-3cfbdc4bba18"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:45.777152", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0749831 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0749831 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-04-03T03:04:45.994 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.025528", "end": "2021-04-03 03:04:46.139740", "rc": 0, "start": "2021-04-03 03:04:46.114212", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local 2021-04-03T03:04:45.996 INFO:teuthology.task.ansible.out:/share/perl/5.26.1/Amazon/S3.pm"]} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:45.998 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [gibba026.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:46.024 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.121) 0:02:10.178 ******** 2021-04-03T03:04:46.048 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.024) 0:02:10.202 ******** 2021-04-03T03:04:46.054 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-04-03T03:04:46.056 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.081 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.032) 0:02:10.235 ******** 2021-04-03T03:04:46.101 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.020) 0:02:10.255 ******** 2021-04-03T03:04:46.107 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:46.108 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'block_used': 13759006, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336102, 'size_available': 927072673792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336102, "block_size": 4096, "block_total": 240095108, "block_used": 13759006, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072673792, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.109 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.133 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.032) 0:02:10.287 ******** 2021-04-03T03:04:46.162 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.028) 0:02:10.316 ******** 2021-04-03T03:04:46.170 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-04-03T03:04:46.171 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzUdsc2jPrMOWKA7j4y2CaTVRI0MShrkND'], 'uuids': ['e8b0dbf1-1261-4039-bcc2-e401c35c12f3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.076439", "end": "2021-04-03 03:04:46.021583", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-BqzdYSLiFueXgRNaZBfEsFMeziMxy3IzUdsc2jPrMOWKA7j4y2CaTVRI0MShrkND"], "labels": [], "masters": [], "uuids": ["e8b0dbf1-1261-4039-bcc2-e401c35c12f3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-04-03 03:04:45.945144", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0752735 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0752735 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:46.206 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.044) 0:02:10.360 ******** 2021-04-03T03:04:46.213 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-04-03T03:04:46.214 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.240 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.033) 0:02:10.394 ******** 2021-04-03T03:04:46.266 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:46.268 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.290 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.050) 0:02:10.444 ******** 2021-04-03T03:04:46.317 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:46.319 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload CPAN config for root.] ********************************* ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-04-03T03:04:46.347 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.056) 0:02:10.501 ******** 2021-04-03T03:04:46.368 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.021) 0:02:10.522 ******** 2021-04-03T03:04:46.386 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-04-03T03:04:46.388 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.411 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.042) 0:02:10.565 ******** 2021-04-03T03:04:46.440 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-04-03T03:04:46.442 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.464 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.053) 0:02:10.618 ******** 2021-04-03T03:04:46.492 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-04-03T03:04:46.493 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.515 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.050) 0:02:10.669 ******** 2021-04-03T03:04:46.521 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-04-03T03:04:46.523 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.174551", "end": "2021-04-03 03:04:46.352194", "rc": 0, "start": "2021-04-03 03:04:46.177643", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:46.547 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.032) 0:02:10.701 ******** 2021-04-03T03:04:46.701 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-04-03T03:04:46.702 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.037286", "end": "2021-04-03 03:04:46.551645", "rc": 0, "start": "2021-04-03 03:04:46.514359", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:46.730 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.182) 0:02:10.884 ******** 2021-04-03T03:04:46.747 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-04-03T03:04:46.749 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:04:46.771 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.040) 0:02:10.925 ******** 2021-04-03T03:04:46.777 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-04-03T03:04:46.804 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.032) 0:02:10.958 ******** 2021-04-03T03:04:46.810 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-04-03T03:04:46.811 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.836 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.032) 0:02:10.990 ******** 2021-04-03T03:04:46.844 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-04-03T03:04:46.844 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:46.873 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.036) 0:02:11.027 ******** 2021-04-03T03:04:46.901 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-04-03T03:04:46.902 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create volume group(s)] *************************************** changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-04-03T03:04:46.928 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.055) 0:02:11.082 ******** 2021-04-03T03:04:46.950 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:46 +0000 (0:00:00.021) 0:02:11.104 ******** 2021-04-03T03:04:46.978 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-04-03T03:04:46.980 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.005 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.054) 0:02:11.159 ******** 2021-04-03T03:04:47.012 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-04-03T03:04:47.013 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} 2021-04-03T03:04:47.049 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.044) 0:02:11.203 ******** 2021-04-03T03:04:47.056 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-04-03T03:04:47.058 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.084 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.034) 0:02:11.238 ******** 2021-04-03T03:04:47.092 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-04-03T03:04:47.093 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-04-03T03:04:47.119 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.035) 0:02:11.273 ******** 2021-04-03T03:04:47.146 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-04-03T03:04:47.147 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.170 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.050) 0:02:11.324 ******** 2021-04-03T03:04:47.176 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-04-03T03:04:47.177 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.024961", "end": "2021-04-03 03:04:47.435252", "rc": 0, "start": "2021-04-03 03:04:47.410291", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:47.179 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [gibba028.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-04-03T03:04:47.211 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.040) 0:02:11.365 ******** 2021-04-03T03:04:47.339 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.128) 0:02:11.493 ******** 2021-04-03T03:04:47.346 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-04-03T03:04:47.347 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'block_used': 13759008, 'uuid': '544fff2a-e246-4637-96d0-94b4cd4bcfea', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226336100, 'size_available': 927072665600, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149513, 'block_size': 4096, 'inode_available': 60905463}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226336100, "block_size": 4096, "block_total": 240095108, "block_used": 13759008, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905463, "inode_total": 61054976, "inode_used": 149513, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927072665600, "size_total": 983429562368, "uuid": "544fff2a-e246-4637-96d0-94b4cd4bcfea"}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.349 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-04-03T03:04:47.350 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:12 UTC", "ActiveEnterTimestampMonotonic": "24952620", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice systemd-journald.socket -.mount basic.target network.target systemd-tmpfiles-setup.service sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:05 UTC", "AssertTimestampMonotonic": "18380128", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:05 UTC", "ConditionTimestampMonotonic": "18380128", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Document 2021-04-03T03:04:47.352 INFO:teuthology.task.ansible.out:ation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1942", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:12 UTC", "ExecMainStartTimestampMonotonic": "24952582", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:05 UTC", "InactiveExitTimestampMonotonic": "18380536", "InvocationID": "4660a44086c649f6b39cd6c548bd7db3", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limit 2021-04-03T03:04:47.353 INFO:teuthology.task.ansible.out:RTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1942", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:12 UTC", "St 2021-04-03T03:04:47.355 INFO:teuthology.task.ansible.out:ateChangeTimestampMonotonic": "24952620", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:12 UTC", "WatchdogTimestampMonotonic": "24952618", "WatchdogUSec": "0"}} TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:47.377 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.038) 0:02:11.531 ******** 2021-04-03T03:04:47.397 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.019) 0:02:11.551 ******** 2021-04-03T03:04:47.403 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-04-03T03:04:47.405 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-04-03T03:04:47.429 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.031) 0:02:11.583 ******** 2021-04-03T03:04:47.452 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.022) 0:02:11.606 ******** 2021-04-03T03:04:47.457 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-04-03T03:04:47.459 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.482 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.030) 0:02:11.636 ******** 2021-04-03T03:04:47.502 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.019) 0:02:11.656 ******** 2021-04-03T03:04:47.507 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-04-03T03:04:47.509 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:47.509 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.533 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.031) 0:02:11.687 ******** 2021-04-03T03:04:47.553 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.019) 0:02:11.707 ******** 2021-04-03T03:04:47.559 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:47.561 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.586 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.032) 0:02:11.740 ******** 2021-04-03T03:04:47.612 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:47.614 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.635 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.049) 0:02:11.789 ******** 2021-04-03T03:04:47.651 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:47.652 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:47.662 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-04-03T03:04:47.664 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.691 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.056) 0:02:11.845 ******** 2021-04-03T03:04:47.718 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-04-03T03:04:47.719 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.740 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.048) 0:02:11.894 ******** 2021-04-03T03:04:47.766 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-04-03T03:04:47.767 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:47.789 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.048) 0:02:11.943 ******** 2021-04-03T03:04:47.805 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:47.807 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:47.832 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.043) 0:02:11.986 ******** 2021-04-03T03:04:47.943 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-04-03T03:04:47.945 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-04-03T03:04:47.967 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:47 +0000 (0:00:00.134) 0:02:12.121 ******** 2021-04-03T03:04:48.111 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:48.112 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:48.155 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": false} 2021-04-03T03:04:48.179 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.212) 0:02:12.333 ******** 2021-04-03T03:04:48.186 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-04-03T03:04:48.188 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:48.214 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.034) 0:02:12.368 ******** 2021-04-03T03:04:48.220 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-04-03T03:04:48.222 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:48.245 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.030) 0:02:12.399 ******** 2021-04-03T03:04:48.252 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-04-03T03:04:48.254 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:48.279 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.034) 0:02:12.433 ******** 2021-04-03T03:04:48.305 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.025) 0:02:12.459 ******** 2021-04-03T03:04:48.312 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:48.313 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:48.346 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-04-03T03:04:48.347 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:48.348 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:48.373 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.067) 0:02:12.527 ******** 2021-04-03T03:04:48.379 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:48.408 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.034) 0:02:12.562 ******** 2021-04-03T03:04:48.435 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-04-03T03:04:48.437 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:48.472 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.063) 0:02:12.626 ******** 2021-04-03T03:04:48.502 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-04-03T03:04:48.504 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:48.526 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.054) 0:02:12.680 ******** 2021-04-03T03:04:48.533 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-04-03T03:04:48.535 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-04-03T03:04:48.560 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.033) 0:02:12.714 ******** 2021-04-03T03:04:48.586 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-04-03T03:04:48.588 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:48.610 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.050) 0:02:12.764 ******** 2021-04-03T03:04:48.637 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-04-03T03:04:48.639 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:48.663 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.052) 0:02:12.817 ******** 2021-04-03T03:04:48.669 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-04-03T03:04:48.671 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-04-03T03:04:48.695 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.032) 0:02:12.849 ******** 2021-04-03T03:04:48.703 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:48.705 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-04-03T03:04:48.730 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.034) 0:02:12.884 ******** 2021-04-03T03:04:48.757 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-04-03T03:04:48.758 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:32 UTC", "ActiveEnterTimestampMonotonic": "24906594", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice sysinit.target -.mount systemd-journald.socket network.target basic.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:25 UTC", "AssertTimestampMonotonic": "18260127", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:25 UTC", "ConditionTimestampMonotonic": "18260127", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Document 2021-04-03T03:04:48.760 INFO:teuthology.task.ansible.out:ation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1922", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:32 UTC", "ExecMainStartTimestampMonotonic": "24906556", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:25 UTC", "InactiveExitTimestampMonotonic": "18260535", "InvocationID": "69c3d31896b24d399885fc62fea6a9a4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limit 2021-04-03T03:04:48.761 INFO:teuthology.task.ansible.out:RTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1922", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:32 UTC", "St 2021-04-03T03:04:48.762 INFO:teuthology.task.ansible.out:ateChangeTimestampMonotonic": "24906594", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:32 UTC", "WatchdogTimestampMonotonic": "24906592", "WatchdogUSec": "0"}} 2021-04-03T03:04:48.784 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.054) 0:02:12.939 ******** 2021-04-03T03:04:48.811 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:48.813 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-04-03T03:04:48.835 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.050) 0:02:12.989 ******** 2021-04-03T03:04:48.852 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-04-03T03:04:48.853 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true} 2021-04-03T03:04:48.875 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.039) 0:02:13.029 ******** 2021-04-03T03:04:48.882 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:48.884 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:48.908 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.033) 0:02:13.063 ******** 2021-04-03T03:04:48.935 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-04-03T03:04:48.937 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:48.959 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:48 +0000 (0:00:00.050) 0:02:13.113 ******** 2021-04-03T03:04:49.059 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-04-03T03:04:49.060 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-04-03T03:04:49.091 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.131) 0:02:13.245 ******** 2021-04-03T03:04:49.097 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:49.099 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:49.120 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:49.143 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.052) 0:02:13.297 ******** 2021-04-03T03:04:49.173 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-04-03T03:04:49.175 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:49.203 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.060) 0:02:13.357 ******** 2021-04-03T03:04:49.230 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:49.232 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:49.264 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.060) 0:02:13.418 ******** 2021-04-03T03:04:49.274 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-04-03T03:04:49.275 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-04-03T03:04:49.299 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.035) 0:02:13.453 ******** 2021-04-03T03:04:49.305 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-04-03T03:04:49.307 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-04-03T03:04:49.309 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:49.310 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:49.336 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.036) 0:02:13.490 ******** 2021-04-03T03:04:49.359 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.022) 0:02:13.513 ******** 2021-04-03T03:04:49.459 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-04-03T03:04:49.461 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-04-03T03:04:49.484 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.124) 0:02:13.638 ******** 2021-04-03T03:04:49.504 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.020) 0:02:13.658 ******** 2021-04-03T03:04:49.585 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-04-03T03:04:49.586 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:49.611 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.106) 0:02:13.765 ******** 2021-04-03T03:04:49.619 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-04-03T03:04:49.621 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": false} TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:49.649 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.037) 0:02:13.803 ******** 2021-04-03T03:04:49.667 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-04-03T03:04:49.669 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false} 2021-04-03T03:04:49.691 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.042) 0:02:13.845 ******** 2021-04-03T03:04:49.697 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-04-03T03:04:49.699 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:49.729 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.037) 0:02:13.883 ******** 2021-04-03T03:04:49.735 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:49.736 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : Set interface] ************************************************ 2021-04-03T03:04:49.737 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-04-03T03:04:49.762 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.033) 0:02:13.916 ******** 2021-04-03T03:04:49.780 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-04-03T03:04:49.782 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:49.823 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-04-03T03:04:49.825 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:49.850 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.087) 0:02:14.004 ******** 2021-04-03T03:04:49.856 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-04-03T03:04:49.857 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-04-03T03:04:49.884 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.033) 0:02:14.038 ******** 2021-04-03T03:04:49.891 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-04-03T03:04:49.893 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1617418609.3380544, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1617418609.3340564, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 10060, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1617418609.3340564, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3104356462", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-04-03T03:04:49.893 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-04-03T03:04:49.895 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:49.923 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.039) 0:02:14.077 ******** 2021-04-03T03:04:49.956 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:49 +0000 (0:00:00.032) 0:02:14.110 ******** 2021-04-03T03:04:49.986 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-04-03T03:04:49.989 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.013 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.057) 0:02:14.167 ******** 2021-04-03T03:04:50.021 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-04-03T03:04:50.022 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-04-03T03:04:50.046 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.033) 0:02:14.200 ******** 2021-04-03T03:04:50.054 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-04-03T03:04:50.057 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.082 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.035) 0:02:14.236 ******** 2021-04-03T03:04:50.088 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-04-03T03:04:50.090 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-04-03T03:04:50.122 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.040) 0:02:14.276 ******** 2021-04-03T03:04:50.145 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.022) 0:02:14.299 ******** 2021-04-03T03:04:50.150 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-04-03T03:04:50.152 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} TASK [testnode : ifdown and ifup] ********************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.182 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.037) 0:02:14.336 ******** 2021-04-03T03:04:50.204 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.021) 0:02:14.358 ******** 2021-04-03T03:04:50.212 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-04-03T03:04:50.214 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:50.238 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.033) 0:02:14.392 ******** 2021-04-03T03:04:50.276 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-04-03T03:04:50.277 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [gibba015.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-04-03T03:04:50.300 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.062) 0:02:14.455 ******** 2021-04-03T03:04:50.322 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.021) 0:02:14.476 ******** 2021-04-03T03:04:50.391 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-04-03T03:04:50.393 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-04-03T03:04:50.419 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.096) 0:02:14.573 ******** 2021-04-03T03:04:50.439 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-04-03T03:04:50.441 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true} TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.463 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.044) 0:02:14.617 ******** 2021-04-03T03:04:50.484 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.020) 0:02:14.638 ******** 2021-04-03T03:04:50.504 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-04-03T03:04:50.505 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.506 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.508 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} skipping: [gibba015.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.534 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.049) 0:02:14.688 ******** 2021-04-03T03:04:50.564 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-04-03T03:04:50.565 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.576 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.577 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.588 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.589 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.613 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.078) 0:02:14.767 ******** 2021-04-03T03:04:50.640 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:50.641 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.677 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.064) 0:02:14.831 ******** 2021-04-03T03:04:50.684 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-04-03T03:04:50.686 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-04-03T03:04:50.710 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.033) 0:02:14.864 ******** 2021-04-03T03:04:50.717 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-04-03T03:04:50.717 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-04-03T03:04:50.719 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.755 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.044) 0:02:14.909 ******** 2021-04-03T03:04:50.778 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.023) 0:02:14.932 ******** 2021-04-03T03:04:50.785 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-04-03T03:04:50.787 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.814 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.036) 0:02:14.969 ******** 2021-04-03T03:04:50.822 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-04-03T03:04:50.824 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} TASK [testnode : Install cachefilesd] ****************************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.848 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.033) 0:02:15.002 ******** 2021-04-03T03:04:50.869 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.021) 0:02:15.023 ******** 2021-04-03T03:04:50.876 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-04-03T03:04:50.878 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.901 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.032) 0:02:15.056 ******** 2021-04-03T03:04:50.907 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-04-03T03:04:50.909 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.930 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.028) 0:02:15.084 ******** 2021-04-03T03:04:50.935 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-04-03T03:04:50.937 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.962 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.031) 0:02:15.116 ******** 2021-04-03T03:04:50.968 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:50.970 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:50.990 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:50 +0000 (0:00:00.028) 0:02:15.144 ******** 2021-04-03T03:04:50.996 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-04-03T03:04:50.997 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.024682", "end": "2021-04-03 03:04:50.952175", "rc": 0, "start": "2021-04-03 03:04:50.927493", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:51.022 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.031) 0:02:15.176 ******** 2021-04-03T03:04:51.053 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.030) 0:02:15.207 ******** 2021-04-03T03:04:51.058 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-04-03T03:04:51.060 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:51.085 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.031) 0:02:15.239 ******** 2021-04-03T03:04:51.108 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.022) 0:02:15.262 ******** 2021-04-03T03:04:51.113 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-04-03T03:04:51.115 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false} TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:51.115 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:51.139 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.031) 0:02:15.293 ******** 2021-04-03T03:04:51.172 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.032) 0:02:15.326 ******** 2021-04-03T03:04:51.197 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.024) 0:02:15.351 ******** 2021-04-03T03:04:51.202 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-04-03T03:04:51.204 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:51.228 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.031) 0:02:15.382 ******** 2021-04-03T03:04:51.234 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-04-03T03:04:51.236 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} TASK [testnode : Add CPAN config directory for the teuthology user.] *********** ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:51.263 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.034) 0:02:15.417 ******** 2021-04-03T03:04:51.285 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.021) 0:02:15.439 ******** 2021-04-03T03:04:51.291 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-04-03T03:04:51.293 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002249", "end": "2021-04-03 03:04:51.388305", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-04-03 03:04:51.386056", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:51.325 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001472", "end": "2021-04-03 03:04:51.466208", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-04-03 03:04:51.464736", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:51.368 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-04-03T03:04:51.370 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:04:51.395 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.110) 0:02:15.549 ******** 2021-04-03T03:04:51.401 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-04-03T03:04:51.402 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1617418605.468469, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1617418605.4644668, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 10060, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1617418605.4644668, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3104356462", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-04-03T03:04:51.428 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.033) 0:02:15.582 ******** 2021-04-03T03:04:51.434 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-04-03T03:04:51.435 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001358", "end": "2021-04-03 03:04:51.543170", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-04-03 03:04:51.541812", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:51.500 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.016701", "end": "2021-04-03 03:04:51.639699", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-04-03 03:04:51.622998", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:51.524 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.096) 0:02:15.678 ******** 2021-04-03T03:04:51.530 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-04-03T03:04:51.531 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-04-03T03:04:51.554 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.029) 0:02:15.708 ******** 2021-04-03T03:04:51.559 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-04-03T03:04:51.582 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.028) 0:02:15.736 ******** 2021-04-03T03:04:51.587 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-04-03T03:04:51.589 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-04-03T03:04:51.620 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.038) 0:02:15.774 ******** 2021-04-03T03:04:51.627 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-04-03T03:04:51.629 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1234, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-04-03T03:04:51.653 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.032) 0:02:15.807 ******** 2021-04-03T03:04:51.659 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-04-03T03:04:51.661 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:51.686 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.032) 0:02:15.840 ******** 2021-04-03T03:04:51.694 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-04-03T03:04:51.695 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-04-03T03:04:51.796 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-04-03T03:04:51.798 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-04-03T03:04:51.820 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.134) 0:02:15.974 ******** 2021-04-03T03:04:51.826 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-04-03T03:04:51.828 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:04:51.857 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.037) 0:02:16.011 ******** 2021-04-03T03:04:51.885 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-04-03T03:04:51.886 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:51.909 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.052) 0:02:16.064 ******** 2021-04-03T03:04:51.929 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.019) 0:02:16.083 ******** 2021-04-03T03:04:51.937 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-04-03T03:04:51.938 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-04-03T03:04:51.965 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:51 +0000 (0:00:00.035) 0:02:16.119 ******** 2021-04-03T03:04:51.992 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-04-03T03:04:51.994 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.016 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.051) 0:02:16.170 ******** 2021-04-03T03:04:52.043 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-04-03T03:04:52.045 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.055 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.057 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [gibba031.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.067 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.092 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.075) 0:02:16.246 ******** 2021-04-03T03:04:52.129 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-04-03T03:04:52.131 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.140 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.141 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.152 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.152 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-04-03T03:04:52.153 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-04-03T03:04:52.154 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba031.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.182 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.090) 0:02:16.337 ******** 2021-04-03T03:04:52.203 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.020) 0:02:16.357 ******** 2021-04-03T03:04:52.209 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-04-03T03:04:52.211 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.233 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.029) 0:02:16.387 ******** 2021-04-03T03:04:52.239 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:52.241 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.268 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.034) 0:02:16.422 ******** 2021-04-03T03:04:52.294 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-04-03T03:04:52.296 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.320 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.051) 0:02:16.474 ******** 2021-04-03T03:04:52.346 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-04-03T03:04:52.348 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.372 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.051) 0:02:16.526 ******** 2021-04-03T03:04:52.388 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-04-03T03:04:52.390 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:18 UTC", "ActiveEnterTimestampMonotonic": "24726099", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service system.slice basic.target sysinit.target systemd-journald.socket network.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:12 UTC", "AssertTimestampMonotonic": "18192160", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:12 UTC", "ConditionTimestampMonotonic": "18192160", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Document 2021-04-03T03:04:52.392 INFO:teuthology.task.ansible.out:ation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1993", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:18 UTC", "ExecMainStartTimestampMonotonic": "24726061", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:12 UTC", "InactiveExitTimestampMonotonic": "18192580", "InvocationID": "86ddbea5278c44698bdf21af3acb7b3b", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limit 2021-04-03T03:04:52.393 INFO:teuthology.task.ansible.out:RTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1993", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:18 UTC", "St 2021-04-03T03:04:52.395 INFO:teuthology.task.ansible.out:ateChangeTimestampMonotonic": "24726099", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:18 UTC", "WatchdogTimestampMonotonic": "24726097", "WatchdogUSec": "0"}} TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.415 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.043) 0:02:16.569 ******** 2021-04-03T03:04:52.435 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.019) 0:02:16.589 ******** 2021-04-03T03:04:52.440 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:52.445 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-04-03T03:04:52.465 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.030) 0:02:16.619 ******** 2021-04-03T03:04:52.471 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-04-03T03:04:52.473 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.499 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.034) 0:02:16.654 ******** 2021-04-03T03:04:52.506 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:52.507 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.529 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.029) 0:02:16.683 ******** 2021-04-03T03:04:52.536 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-04-03T03:04:52.537 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.563 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.033) 0:02:16.717 ******** 2021-04-03T03:04:52.584 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.021) 0:02:16.738 ******** 2021-04-03T03:04:52.591 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-04-03T03:04:52.593 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.617 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.033) 0:02:16.771 ******** 2021-04-03T03:04:52.624 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-04-03T03:04:52.625 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.652 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.035) 0:02:16.806 ******** 2021-04-03T03:04:52.679 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-04-03T03:04:52.681 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.703 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.050) 0:02:16.857 ******** 2021-04-03T03:04:52.709 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-04-03T03:04:52.711 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.024873", "end": "2021-04-03 03:04:52.561338", "rc": 0, "start": "2021-04-03 03:04:52.536465", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-04-03T03:04:52.737 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.034) 0:02:16.891 ******** 2021-04-03T03:04:52.765 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-04-03T03:04:52.767 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:52.789 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.051) 0:02:16.943 ******** 2021-04-03T03:04:52.815 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-04-03T03:04:52.816 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [gibba031.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002409", "end": "2021-04-03 03:04:53.070973", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-04-03 03:04:53.068564", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:52.843 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:52 +0000 (0:00:00.054) 0:02:16.997 ******** 2021-04-03T03:04:52.892 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001525", "end": "2021-04-03 03:04:53.154433", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-04-03 03:04:53.152908", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:52.976 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001414", "end": "2021-04-03 03:04:53.234097", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-04-03 03:04:53.232683", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:53.061 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.001964", "end": "2021-04-03 03:04:53.315886", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-04-03 03:04:53.313922", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:53.087 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.244) 0:02:17.241 ******** 2021-04-03T03:04:53.114 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-04-03T03:04:53.141 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.053) 0:02:17.295 ******** 2021-04-03T03:04:53.200 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-04-03T03:04:53.202 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": false} 2021-04-03T03:04:53.229 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.087) 0:02:17.383 ******** 2021-04-03T03:04:53.235 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-04-03T03:04:53.236 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-04-03T03:04:53.264 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-04-03T03:04:53.266 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:53.287 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.058) 0:02:17.441 ******** 2021-04-03T03:04:53.393 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:53.415 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.128) 0:02:17.569 ******** 2021-04-03T03:04:53.453 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-04-03T03:04:53.455 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:04:53.479 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.063) 0:02:17.633 ******** 2021-04-03T03:04:53.506 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-04-03T03:04:53.508 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:53.529 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.050) 0:02:17.684 ******** 2021-04-03T03:04:53.664 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-04-03T03:04:53.666 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-04-03T03:04:53.693 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.163) 0:02:17.847 ******** 2021-04-03T03:04:53.762 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-04-03T03:04:53.763 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-04-03T03:04:53.787 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.093) 0:02:17.941 ******** 2021-04-03T03:04:53.814 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-04-03T03:04:53.816 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:53.837 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:53 +0000 (0:00:00.050) 0:02:17.991 ******** 2021-04-03T03:04:54.001 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-04-03T03:04:54.002 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-04-03 03:02:25 UTC", "ActiveEnterTimestampMonotonic": "24376933", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target sysinit.target -.mount system.slice systemd-tmpfiles-setup.service network.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-04-03 03:02:19 UTC", "AssertTimestampMonotonic": "17936139", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-04-03 03:02:19 UTC", "ConditionTimestampMonotonic": "17936139", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Document 2021-04-03T03:04:54.004 INFO:teuthology.task.ansible.out:ation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2033", "ExecMainStartTimestamp": "Sat 2021-04-03 03:02:25 UTC", "ExecMainStartTimestampMonotonic": "24376896", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-04-03 03:02:19 UTC", "InactiveExitTimestampMonotonic": "17936700", "InvocationID": "6f69862d2d6a4741ac54a398e7f0c6ac", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128079", "LimitNPROCSoft": "128079", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limit 2021-04-03T03:04:54.005 INFO:teuthology.task.ansible.out:RTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2033", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:02:25 UTC", "St 2021-04-03T03:04:54.007 INFO:teuthology.task.ansible.out:ateChangeTimestampMonotonic": "24376933", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-04-03 03:02:25 UTC", "WatchdogTimestampMonotonic": "24376931", "WatchdogUSec": "0"}} 2021-04-03T03:04:54.029 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.191) 0:02:18.183 ******** 2021-04-03T03:04:54.058 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:54.060 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-04-03T03:04:54.090 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.061) 0:02:18.244 ******** 2021-04-03T03:04:54.119 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:54.121 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:54.157 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.067) 0:02:18.311 ******** 2021-04-03T03:04:54.184 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-04-03T03:04:54.186 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:54.216 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.058) 0:02:18.370 ******** 2021-04-03T03:04:54.227 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-04-03T03:04:54.228 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-04-03T03:04:54.254 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.038) 0:02:18.408 ******** 2021-04-03T03:04:54.619 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-04-03T03:04:54.621 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false} 2021-04-03T03:04:54.643 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.388) 0:02:18.797 ******** 2021-04-03T03:04:54.669 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-04-03T03:04:54.671 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-04-03T03:04:54.693 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.050) 0:02:18.847 ******** 2021-04-03T03:04:54.793 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-04-03T03:04:54.795 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1617418615.5920575, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1617418615.5880594, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 10060, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1617418615.5880594, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3104356462", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": false} 2021-04-03T03:04:54.818 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.124) 0:02:18.972 ******** 2021-04-03T03:04:54.840 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.021) 0:02:18.994 ******** 2021-04-03T03:04:54.871 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-04-03T03:04:54.872 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:54.894 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.054) 0:02:19.048 ******** 2021-04-03T03:04:54.932 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-04-03T03:04:54.933 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-04-03T03:04:54.960 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:54 +0000 (0:00:00.065) 0:02:19.114 ******** 2021-04-03T03:04:54.986 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-04-03T03:04:54.988 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-04-03T03:04:55.010 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.050) 0:02:19.164 ******** 2021-04-03T03:04:55.016 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-04-03T03:04:55.018 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-04-03T03:04:55.051 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.040) 0:02:19.205 ******** 2021-04-03T03:04:55.057 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-04-03T03:04:55.059 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.089 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.038) 0:02:19.243 ******** 2021-04-03T03:04:55.242 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-04-03T03:04:55.243 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-04-03T03:04:55.265 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.175) 0:02:19.419 ******** 2021-04-03T03:04:55.325 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-04-03T03:04:55.327 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1234, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-04-03T03:04:55.351 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.085) 0:02:19.505 ******** 2021-04-03T03:04:55.359 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-04-03T03:04:55.361 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-04-03T03:04:55.386 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.035) 0:02:19.540 ******** 2021-04-03T03:04:55.414 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-04-03T03:04:55.416 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.438 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.051) 0:02:19.592 ******** 2021-04-03T03:04:55.466 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-04-03T03:04:55.468 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.477 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.479 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.489 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.490 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.514 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.075) 0:02:19.668 ******** 2021-04-03T03:04:55.551 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-04-03T03:04:55.553 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba026.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.564 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.565 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.588 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.074) 0:02:19.742 ******** 2021-04-03T03:04:55.614 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:55.616 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.639 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.050) 0:02:19.793 ******** 2021-04-03T03:04:55.666 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-04-03T03:04:55.667 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.689 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.049) 0:02:19.843 ******** 2021-04-03T03:04:55.715 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-04-03T03:04:55.717 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.741 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.051) 0:02:19.895 ******** 2021-04-03T03:04:55.767 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-04-03T03:04:55.769 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.790 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.049) 0:02:19.944 ******** 2021-04-03T03:04:55.816 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-04-03T03:04:55.818 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.841 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.050) 0:02:19.995 ******** 2021-04-03T03:04:55.867 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-04-03T03:04:55.869 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.896 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.055) 0:02:20.050 ******** 2021-04-03T03:04:55.913 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-04-03T03:04:55.914 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-04-03T03:04:55.937 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.040) 0:02:20.091 ******** 2021-04-03T03:04:55.943 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-04-03T03:04:55.945 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:55.972 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:55 +0000 (0:00:00.034) 0:02:20.126 ******** 2021-04-03T03:04:55.999 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-04-03T03:04:56.000 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:56.022 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.049) 0:02:20.176 ******** 2021-04-03T03:04:56.132 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-04-03T03:04:56.134 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002300", "end": "2021-04-03 03:04:56.108451", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-04-03 03:04:56.106151", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:56.217 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001430", "end": "2021-04-03 03:04:56.187084", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-04-03 03:04:56.185654", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:56.301 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001305", "end": "2021-04-03 03:04:56.274156", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-04-03 03:04:56.272851", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:56.303 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [gibba028.front.sepia.ceph.com] => {"changed": false} 2021-04-03T03:04:56.328 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.306) 0:02:20.482 ******** 2021-04-03T03:04:56.356 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-04-03T03:04:56.357 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-04-03T03:04:56.379 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.050) 0:02:20.533 ******** 2021-04-03T03:04:56.395 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-04-03T03:04:56.397 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.013052", "end": "2021-04-03 03:04:56.361360", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-04-03 03:04:56.348308", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:56.422 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.043) 0:02:20.576 ******** 2021-04-03T03:04:56.449 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-04-03T03:04:56.472 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.049) 0:02:20.626 ******** 2021-04-03T03:04:56.488 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-04-03T03:04:56.490 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1617418609.0861053, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1617418609.0821033, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 10060, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1617418609.0821033, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3104356462", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-04-03T03:04:56.514 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.042) 0:02:20.668 ******** 2021-04-03T03:04:56.572 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-04-03T03:04:56.574 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-04-03T03:04:56.633 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-04-03T03:04:56.635 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-04-03T03:04:56.659 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.144) 0:02:20.813 ******** 2021-04-03T03:04:56.685 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-04-03T03:04:56.687 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-04-03T03:04:56.708 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.049) 0:02:20.862 ******** 2021-04-03T03:04:56.734 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-04-03T03:04:56.736 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:56.757 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.049) 0:02:20.911 ******** 2021-04-03T03:04:56.856 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-04-03T03:04:56.858 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-04-03T03:04:56.880 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:56 +0000 (0:00:00.123) 0:02:21.034 ******** 2021-04-03T03:04:56.990 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-04-03T03:04:56.991 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-04-03T03:04:57.013 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.132) 0:02:21.167 ******** 2021-04-03T03:04:57.040 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-04-03T03:04:57.042 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.063 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.049) 0:02:21.217 ******** 2021-04-03T03:04:57.089 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-04-03T03:04:57.091 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.101 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.102 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.112 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.136 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.073) 0:02:21.290 ******** 2021-04-03T03:04:57.173 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-04-03T03:04:57.175 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.185 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.186 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba028.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.211 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.074) 0:02:21.365 ******** 2021-04-03T03:04:57.237 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-04-03T03:04:57.239 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.263 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.051) 0:02:21.417 ******** 2021-04-03T03:04:57.289 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-04-03T03:04:57.291 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.312 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.049) 0:02:21.466 ******** 2021-04-03T03:04:57.340 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-04-03T03:04:57.342 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.367 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.054) 0:02:21.521 ******** 2021-04-03T03:04:57.393 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-04-03T03:04:57.395 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.416 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.049) 0:02:21.570 ******** 2021-04-03T03:04:57.442 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-04-03T03:04:57.466 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.470 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.054) 0:02:21.624 ******** 2021-04-03T03:04:57.497 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.523 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.051) 0:02:21.676 ******** 2021-04-03T03:04:57.549 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-04-03T03:04:57.550 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.573 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.051) 0:02:21.727 ******** 2021-04-03T03:04:57.600 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-04-03T03:04:57.601 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:57.623 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:57 +0000 (0:00:00.049) 0:02:21.777 ******** 2021-04-03T03:04:57.732 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-04-03T03:04:57.734 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002345", "end": "2021-04-03 03:04:57.603352", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-04-03 03:04:57.601007", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:57.806 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001419", "end": "2021-04-03 03:04:57.680985", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-04-03 03:04:57.679566", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:57.890 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001408", "end": "2021-04-03 03:04:57.765486", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-04-03 03:04:57.764078", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:57.995 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.013625", "end": "2021-04-03 03:04:57.851793", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-04-03 03:04:57.838168", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:04:58.019 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.396) 0:02:22.173 ******** 2021-04-03T03:04:58.044 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-04-03T03:04:58.067 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.048) 0:02:22.221 ******** 2021-04-03T03:04:58.166 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-04-03T03:04:58.167 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-04-03T03:04:58.234 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-04-03T03:04:58.252 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.184) 0:02:22.406 ******** 2021-04-03T03:04:58.274 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.021) 0:02:22.428 ******** 2021-04-03T03:04:58.312 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.037) 0:02:22.466 ******** 2021-04-03T03:04:58.451 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.139) 0:02:22.605 ******** 2021-04-03T03:04:58.471 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.019) 0:02:22.625 ******** 2021-04-03T03:04:58.815 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba011.front.sepia.ceph.com] 2021-04-03T03:04:58.868 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.397) 0:02:23.022 ******** 2021-04-03T03:04:58.881 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] 2021-04-03T03:04:58.914 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:04:58.916 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-04-03T03:04:58.918 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba026.front.sepia.ceph.com] 2021-04-03T03:04:58.931 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] 2021-04-03T03:04:58.941 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] 2021-04-03T03:04:58.969 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.100) 0:02:23.123 ******** 2021-04-03T03:04:58.988 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:58 +0000 (0:00:00.019) 0:02:23.142 ******** 2021-04-03T03:04:59.031 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:59.032 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-04-03T03:04:59.053 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.065) 0:02:23.208 ******** 2021-04-03T03:04:59.074 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.020) 0:02:23.228 ******** 2021-04-03T03:04:59.095 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.021) 0:02:23.249 ******** 2021-04-03T03:04:59.115 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.019) 0:02:23.269 ******** 2021-04-03T03:04:59.135 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.020) 0:02:23.289 ******** 2021-04-03T03:04:59.142 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-04-03T03:04:59.143 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-04-03T03:04:59.144 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:59.144 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-04-03T03:04:59.146 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-04-03T03:04:59.169 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.034) 0:02:23.323 ******** 2021-04-03T03:04:59.188 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.018) 0:02:23.342 ******** 2021-04-03T03:04:59.209 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.020) 0:02:23.363 ******** 2021-04-03T03:04:59.229 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.020) 0:02:23.383 ******** 2021-04-03T03:04:59.235 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-04-03T03:04:59.236 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-04-03T03:04:59.237 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-04-03T03:04:59.238 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:59.260 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.031) 0:02:23.414 ******** 2021-04-03T03:04:59.279 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.018) 0:02:23.433 ******** 2021-04-03T03:04:59.313 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.033) 0:02:23.467 ******** 2021-04-03T03:04:59.319 INFO:teuthology.task.ansible.out:skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:59.320 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-04-03T03:04:59.320 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-04-03T03:04:59.344 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.031) 0:02:23.498 ******** 2021-04-03T03:04:59.367 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.023) 0:02:23.521 ******** 2021-04-03T03:04:59.374 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:04:59.375 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:04:59.398 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.031) 0:02:23.553 ******** 2021-04-03T03:04:59.414 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-04-03T03:04:59.416 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-04-03T03:04:59.438 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.039) 0:02:23.592 ******** 2021-04-03T03:04:59.447 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-04-03T03:04:59.482 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:04:59 +0000 (0:00:00.043) 0:02:23.636 ******** 2021-04-03T03:05:07.834 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-04-03T03:05:07.836 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"cache_update_time": 1617418636, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 128 not upgraded.\nNeed to get 39.9 MB of archives.\nAfter this operation, 1024 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]\nPreconfiguring packages ...\nFetched 39.9 MB in 2s (21.7 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137602 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...\r\nUnpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...\r\nSetting up docker.io (19.03.6-0ubuntu1~18.04.3) ...\r\ndocker.service is a disabled or a static unit, not starting it.\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newl 2021-04-03T03:05:07.837 INFO:teuthology.task.ansible.out:y installed, 0 to remove and 128 not upgraded.", "Need to get 39.9 MB of archives.", "After this operation, 1024 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]", "Preconfiguring packages ...", "Fetched 39.9 MB in 2s (21.7 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137602 files and directories currently installed.)", "Preparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...", "Unpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...", "Setting up docker.io (19.03.6-0ubuntu1~18.04.3) ...", "docker.service is a disabled or a static unit, not starting it.", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ..."]} 2021-04-03T03:05:07.858 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:07 +0000 (0:00:08.376) 0:02:32.012 ******** 2021-04-03T03:05:07.886 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:05:07.888 INFO:teuthology.task.ansible.out:skipping: [gibba011.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:05:07.910 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:07 +0000 (0:00:00.051) 0:02:32.064 ******** 2021-04-03T03:05:07.938 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:05:07.939 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-04-03T03:05:07.963 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:07 +0000 (0:00:00.052) 0:02:32.117 ******** 2021-04-03T03:05:07.970 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-04-03T03:05:07.971 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"cache_update_time": 1617418666, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 128 not upgraded.\nNeed to get 39.9 MB of archives.\nAfter this operation, 1024 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]\nPreconfiguring packages ...\nFetched 39.9 MB in 2s (21.2 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137602 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...\r\nUnpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...\r\nSetting up docker.io (19.03.6-0ubuntu1~18.04.3) ...\r\ndocker.service is a disabled or a static unit, not starting it.\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newl 2021-04-03T03:05:07.972 INFO:teuthology.task.ansible.out:y installed, 0 to remove and 128 not upgraded.", "Need to get 39.9 MB of archives.", "After this operation, 1024 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]", "Preconfiguring packages ...", "Fetched 39.9 MB in 2s (21.2 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137602 files and directories currently installed.)", "Preparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...", "Unpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...", "Setting up docker.io (19.03.6-0ubuntu1~18.04.3) ...", "docker.service is a disabled or a static unit, not starting it.", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ..."]} 2021-04-03T03:05:07.998 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:07 +0000 (0:00:00.035) 0:02:32.152 ******** 2021-04-03T03:05:08.025 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:05:08.027 INFO:teuthology.task.ansible.out:skipping: [gibba015.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:05:08.050 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.051) 0:02:32.204 ******** 2021-04-03T03:05:08.078 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.080 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} TASK [container-host : set_fact] *********************************************** ok: [gibba015.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-04-03T03:05:08.104 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.053) 0:02:32.258 ******** 2021-04-03T03:05:08.152 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.154 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.183 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.079) 0:02:32.337 ******** 2021-04-03T03:05:08.190 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-04-03T03:05:08.191 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"cache_update_time": 1617418764, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 128 not upgraded.\nNeed to get 39.9 MB of archives.\nAfter this operation, 1024 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]\nPreconfiguring packages ...\nFetched 39.9 MB in 2s (22.0 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137602 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...\r\nUnpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...\r\nSetting up docker.io (19.03.6-0ubuntu1~18.04.3) ...\r\ndocker.service is a disabled or a static unit, not starting it.\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newl 2021-04-03T03:05:08.193 INFO:teuthology.task.ansible.out:y installed, 0 to remove and 128 not upgraded.", "Need to get 39.9 MB of archives.", "After this operation, 1024 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]", "Preconfiguring packages ...", "Fetched 39.9 MB in 2s (22.0 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137602 files and directories currently installed.)", "Preparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...", "Unpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...", "Setting up docker.io (19.03.6-0ubuntu1~18.04.3) ...", "docker.service is a disabled or a static unit, not starting it.", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ..."]} 2021-04-03T03:05:08.218 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.035) 0:02:32.372 ******** 2021-04-03T03:05:08.227 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.228 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.239 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:05:08.241 INFO:teuthology.task.ansible.out:skipping: [gibba031.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:05:08.262 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.044) 0:02:32.416 ******** 2021-04-03T03:05:08.288 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:05:08.290 INFO:teuthology.task.ansible.out:ok: [gibba031.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-04-03T03:05:08.312 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.049) 0:02:32.466 ******** 2021-04-03T03:05:08.318 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.320 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.322 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-04-03T03:05:08.322 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"cache_update_time": 1617418697, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 128 not upgraded.\nNeed to get 39.9 MB of archives.\nAfter this operation, 1024 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]\nPreconfiguring packages ...\nFetched 39.9 MB in 2s (21.6 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137602 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...\r\nUnpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...\r\nSetting up docker.io (19.03.6-0ubuntu1~18.04.3) ...\r\ndocker.service is a disabled or a static unit, not starting it.\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsut 2021-04-03T03:05:08.324 INFO:teuthology.task.ansible.out:ils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 128 not upgraded.", "Need to get 39.9 MB of archives.", "After this operation, 1024 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]", "Preconfiguring packages ...", "Fetched 39.9 MB in 2s (21.6 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137602 files and directories currently installed.)", "Preparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...", "Unpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...", "Setting up docker.io (19.03.6-0ubuntu1~18.04.3) ...", "docker.service is a disabled or a static unit, not starting it.", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ..."]} 2021-04-03T03:05:08.347 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.035) 0:02:32.501 ******** 2021-04-03T03:05:08.369 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.021) 0:02:32.523 ******** 2021-04-03T03:05:08.396 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-04-03T03:05:08.398 INFO:teuthology.task.ansible.out:skipping: [gibba026.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:05:08.420 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.050) 0:02:32.574 ******** 2021-04-03T03:05:08.436 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.438 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.448 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-04-03T03:05:08.450 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419108.2329197-29932-107014511283608/source", "state": "file", "uid": 0} TASK [container-host : set_fact] *********************************************** ok: [gibba026.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-04-03T03:05:08.474 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.053) 0:02:32.628 ******** 2021-04-03T03:05:08.514 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.515 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.545 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.070) 0:02:32.699 ******** 2021-04-03T03:05:08.592 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-04-03T03:05:08.594 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"cache_update_time": 1617418726, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 128 not upgraded.\nNeed to get 39.9 MB of archives.\nAfter this operation, 1024 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]\nPreconfiguring packages ...\nFetched 39.9 MB in 2s (21.2 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137602 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...\r\nUnpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...\r\nSetting up docker.io (19.03.6-0ubuntu1~18.04.3) ...\r\ndocker.service is a disabled or a static unit, not starting it.\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newl 2021-04-03T03:05:08.596 INFO:teuthology.task.ansible.out:y installed, 0 to remove and 128 not upgraded.", "Need to get 39.9 MB of archives.", "After this operation, 1024 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.3 [39.9 MB]", "Preconfiguring packages ...", "Fetched 39.9 MB in 2s (21.2 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137602 files and directories currently installed.)", "Preparing to unpack .../docker.io_19.03.6-0ubuntu1~18.04.3_amd64.deb ...", "Unpacking docker.io (19.03.6-0ubuntu1~18.04.3) over (19.03.6-0ubuntu1~18.04.2) ...", "Setting up docker.io (19.03.6-0ubuntu1~18.04.3) ...", "docker.service is a disabled or a static unit, not starting it.", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ..."]} 2021-04-03T03:05:08.618 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.073) 0:02:32.772 ******** 2021-04-03T03:05:08.625 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-04-03T03:05:08.627 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419108.3974245-29954-113330722987705/source", "state": "file", "uid": 0} TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.627 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.639 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba028.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-04-03T03:05:08.662 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.044) 0:02:32.816 ******** 2021-04-03T03:05:08.669 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-04-03T03:05:08.670 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419108.4631588-29932-16081607558558/source", "state": "file", "uid": 0} 2021-04-03T03:05:08.696 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.033) 0:02:32.850 ******** 2021-04-03T03:05:08.703 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.704 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.706 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [gibba028.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-04-03T03:05:08.737 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.041) 0:02:32.891 ******** 2021-04-03T03:05:08.758 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.020) 0:02:32.912 ******** 2021-04-03T03:05:08.786 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-04-03T03:05:08.788 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419108.5904818-29985-223539175748194/source", "state": "file", "uid": 0} 2021-04-03T03:05:08.819 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419108.6181223-29954-54468493188055/source", "state": "file", "uid": 0} 2021-04-03T03:05:08.846 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:08 +0000 (0:00:00.087) 0:02:33.000 ******** 2021-04-03T03:05:08.894 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-04-03T03:05:08.896 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:08.980 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-04-03T03:05:09.014 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:09 +0000 (0:00:00.168) 0:02:33.168 ******** 2021-04-03T03:05:09.021 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-04-03T03:05:09.022 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419108.7866402-30016-238990754458012/source", "state": "file", "uid": 0} 2021-04-03T03:05:09.023 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419108.8062015-29985-51259560491454/source", "state": "file", "uid": 0} 2021-04-03T03:05:09.051 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:09 +0000 (0:00:00.036) 0:02:33.205 ******** 2021-04-03T03:05:09.224 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419109.0172913-30016-49460900321372/source", "state": "file", "uid": 0} 2021-04-03T03:05:09.250 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:09 +0000 (0:00:00.199) 0:02:33.404 ******** 2021-04-03T03:05:09.279 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419109.0618832-30046-213447821436159/source", "state": "file", "uid": 0} 2021-04-03T03:05:09.500 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419109.2941911-30046-50582986843002/source", "state": "file", "uid": 0} 2021-04-03T03:05:09.530 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:09 +0000 (0:00:00.279) 0:02:33.684 ******** 2021-04-03T03:05:11.176 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-04-03T03:05:11.178 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-hGuURR-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-hGuURR-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/package 2021-04-03T03:05:11.179 INFO:teuthology.task.ansible.out:s/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-04-03T03:05:11.199 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:01.669) 0:02:35.353 ******** 2021-04-03T03:05:11.298 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-04-03T03:05:11.300 INFO:teuthology.task.ansible.out:ok: [gibba011.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-04-03T03:05:11.324 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.124) 0:02:35.478 ******** 2021-04-03T03:05:11.413 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-04-03T03:05:11.415 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-P029BX-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-P029BX-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/package 2021-04-03T03:05:11.417 INFO:teuthology.task.ansible.out:s/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-ma3KxP-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d 2021-04-03T03:05:11.418 INFO:teuthology.task.ansible.out:8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-ma3KxP-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt 2021-04-03T03:05:11.420 INFO:teuthology.task.ansible.out:", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-04-03T03:05:11.440 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.116) 0:02:35.594 ******** 2021-04-03T03:05:11.463 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.022) 0:02:35.616 ******** 2021-04-03T03:05:11.543 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-04-03T03:05:11.544 INFO:teuthology.task.ansible.out:ok: [gibba015.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-04-03T03:05:11.567 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.104) 0:02:35.721 ******** 2021-04-03T03:05:11.572 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-04-03T03:05:11.574 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419111.3694715-30118-157414160449431/source", "state": "file", "uid": 0} TASK [container-host : Check for docker's daemon.json] ************************* ok: [gibba031.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-04-03T03:05:11.599 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.031) 0:02:35.753 ******** 2021-04-03T03:05:11.620 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.020) 0:02:35.774 ******** 2021-04-03T03:05:11.803 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-04-03T03:05:11.805 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-Q7XFLk-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-Q7XFLk-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/package 2021-04-03T03:05:11.806 INFO:teuthology.task.ansible.out:s/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-04-03T03:05:11.827 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.207) 0:02:35.981 ******** 2021-04-03T03:05:11.834 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-04-03T03:05:11.835 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419111.6091833-30139-134695482362340/source", "state": "file", "uid": 0} TASK [container-host : Add local docker.io registry mirror] ******************** changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.085383", "end": "2021-04-03 03:05:12.012943", "rc": 0, "start": "2021-04-03 03:05:11.927560", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:05:11.861 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.033) 0:02:36.015 ******** 2021-04-03T03:05:11.887 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.026) 0:02:36.041 ******** 2021-04-03T03:05:11.895 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-04-03T03:05:11.896 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419111.6660879-30143-78532497143003/source", "state": "file", "uid": 0} 2021-04-03T03:05:11.921 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.033) 0:02:36.075 ******** 2021-04-03T03:05:11.947 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-04-03T03:05:11.949 INFO:teuthology.task.ansible.out:ok: [gibba026.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-04-03T03:05:11.973 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:11 +0000 (0:00:00.051) 0:02:36.127 ******** 2021-04-03T03:05:12.063 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-04-03T03:05:12.065 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.084039", "end": "2021-04-03 03:05:12.196092", "rc": 0, "start": "2021-04-03 03:05:12.112053", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:05:12.087 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.114) 0:02:36.241 ******** 2021-04-03T03:05:12.104 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.083713", "end": "2021-04-03 03:05:12.355294", "rc": 0, "start": "2021-04-03 03:05:12.271581", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:05:12.131 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.043) 0:02:36.285 ******** 2021-04-03T03:05:12.137 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-04-03T03:05:12.138 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-eVomMt-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-eVomMt-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/package 2021-04-03T03:05:12.139 INFO:teuthology.task.ansible.out:s/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-04-03T03:05:12.167 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.035) 0:02:36.321 ******** 2021-04-03T03:05:12.236 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-04-03T03:05:12.237 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419112.021631-30173-243511410964118/source", "state": "file", "uid": 0} 2021-04-03T03:05:12.261 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.093) 0:02:36.415 ******** 2021-04-03T03:05:12.309 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-04-03T03:05:12.310 INFO:teuthology.task.ansible.out:ok: [gibba028.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-04-03T03:05:12.333 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.072) 0:02:36.487 ******** 2021-04-03T03:05:12.475 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-04-03T03:05:12.476 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.084441", "end": "2021-04-03 03:05:12.453909", "rc": 0, "start": "2021-04-03 03:05:12.369468", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:05:12.501 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.167) 0:02:36.655 ******** 2021-04-03T03:05:12.621 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-04-03T03:05:12.623 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1617419112.3792439-30199-245543092785134/source", "state": "file", "uid": 0} 2021-04-03T03:05:12.645 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.144) 0:02:36.799 ******** 2021-04-03T03:05:12.859 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-04-03T03:05:12.861 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.081864", "end": "2021-04-03 03:05:12.741702", "rc": 0, "start": "2021-04-03 03:05:12.659838", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-04-03T03:05:12.881 INFO:teuthology.task.ansible.out:Saturday 03 April 2021 03:05:12 +0000 (0:00:00.236) 0:02:37.035 ******** 2021-04-03T03:05:14.071 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-04-03T03:05:14.072 INFO:teuthology.task.ansible.out:changed: [gibba011.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket containerd.service docker.socket network-online.target firewalld.service system.slice sysinit.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMai 2021-04-03T03:05:14.074 INFO:teuthology.task.ansible.out:nStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079 2021-04-03T03:05:14.075 INFO:teuthology.task.ansible.out:", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target docker.socket", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:03:51 UTC", "StateChangeTimestampMonotonic": "129416364", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-04-03T03:05:14.077 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-04-03T03:05:14.315 INFO:teuthology.task.ansible.out:changed: [gibba031.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice containerd.service docker.socket network-online.target firewalld.service sysinit.target systemd-journald.socket basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMai 2021-04-03T03:05:14.317 INFO:teuthology.task.ansible.out:nStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079 2021-04-03T03:05:14.319 INFO:teuthology.task.ansible.out:", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice docker.socket", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:04:02 UTC", "StateChangeTimestampMonotonic": "114971511", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-04-03T03:05:14.320 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-04-03T03:05:14.339 INFO:teuthology.task.ansible.out:changed: [gibba015.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "docker.socket sysinit.target network-online.target system.slice firewalld.service containerd.service basic.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMai 2021-04-03T03:05:14.341 INFO:teuthology.task.ansible.out:nStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079 2021-04-03T03:05:14.342 INFO:teuthology.task.ansible.out:", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target docker.socket system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:04:01 UTC", "StateChangeTimestampMonotonic": "133654298", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-04-03T03:05:14.344 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-04-03T03:05:15.054 INFO:teuthology.task.ansible.out:changed: [gibba026.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket sysinit.target docker.socket basic.target containerd.service network-online.target firewalld.service system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMai 2021-04-03T03:05:15.056 INFO:teuthology.task.ansible.out:nStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079 2021-04-03T03:05:15.057 INFO:teuthology.task.ansible.out:", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target docker.socket system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:04:03 UTC", "StateChangeTimestampMonotonic": "129818316", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-04-03T03:05:15.059 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-04-03T03:05:15.489 INFO:teuthology.task.ansible.out:changed: [gibba028.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sysinit.target systemd-journald.socket network-online.target containerd.service firewalld.service docker.socket system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMai 2021-04-03T03:05:15.491 INFO:teuthology.task.ansible.out:nStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128079", "LimitSIGPENDINGSoft": "128079 2021-04-03T03:05:15.493 INFO:teuthology.task.ansible.out:", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target docker.socket system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-04-03 03:04:04 UTC", "StateChangeTimestampMonotonic": "122960228", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-04-03T03:05:15.494 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-04-03T03:05:15.555 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-04-03T03:05:15.556 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-04-03T03:05:15.558 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* gibba011.front.sepia.ceph.com : ok=127 changed=41 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 gibba015.front.sepia.ceph.com : ok=127 changed=41 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 gibba026.front.sepia.ceph.com : ok=127 changed=41 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 gibba028.front.sepia.ceph.com : ok=127 changed=41 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 gibba031.front.sepia.ceph.com : ok=127 changed=41 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 Saturday 03 April 2021 03:05:15 +0000 (0:00:02.676) 0:02:39.712 ******** =============================================================================== 2021-04-03T03:05:15.558 INFO:teuthology.task.ansible.out:Create all admin users with sudo access. ------------------------------- 15.29s container-host : Install container packages ----------------------------- 8.38s container-host : Restart docker service --------------------------------- 2.68s users : Update authorized_keys using the keys repo ---------------------- 2.47s testnode : Ensure packages are not present. ----------------------------- 2.02s container-host : Install registries-conf-ctl ---------------------------- 1.67s testnode : Upgrade packages --------------------------------------------- 1.48s Gathering Facts --------------------------------------------------------- 1.33s users : Clone the keys repo --------------------------------------------- 0.95s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.73s ansible-managed : Add authorized keys for the ansible user. ------------- 0.60s testnode : Install apt keys --------------------------------------------- 0.57s Gathering Facts ---------------------------------------- 2021-04-03T03:05:15.560 INFO:teuthology.task.ansible.out:----------------- 0.52s ansible-managed : Create the ansible user. ------------------------------ 0.52s testnode : Update apt cache. -------------------------------------------- 0.42s Gathering Facts --------------------------------------------------------- 0.40s users : Check keys_repo HEAD sha1 --------------------------------------- 0.40s testnode : List any leftover Ceph artifacts from previous jobs ---------- 0.40s testnode : Zap all non-root disks --------------------------------------- 0.32s testnode : Install packages --------------------------------------------- 0.29s 2021-04-03T03:05:15.825 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@gibba011.front.sepia.ceph.com'), Remote(name='ubuntu@gibba015.front.sepia.ceph.com'), Remote(name='ubuntu@gibba026.front.sepia.ceph.com'), Remote(name='ubuntu@gibba028.front.sepia.ceph.com'), Remote(name='ubuntu@gibba031.front.sepia.ceph.com')] 2021-04-03T03:05:15.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:05:15.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba011.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:05:15.890 DEBUG:teuthology.orchestra.run.gibba011:> true 2021-04-03T03:05:16.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:05:16.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba015.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:05:16.074 DEBUG:teuthology.orchestra.run.gibba015:> true 2021-04-03T03:05:16.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:05:16.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:05:16.260 DEBUG:teuthology.orchestra.run.gibba026:> true 2021-04-03T03:05:16.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:05:16.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:05:16.448 DEBUG:teuthology.orchestra.run.gibba028:> true 2021-04-03T03:05:16.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T03:05:16.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-04-03T03:05:16.680 DEBUG:teuthology.orchestra.run.gibba031:> true 2021-04-03T03:05:16.814 INFO:teuthology.run_tasks:Running task clock... 2021-04-03T03:05:16.826 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-04-03T03:05:16.826 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-04-03T03:05:16.828 DEBUG:teuthology.orchestra.run.gibba011:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T03:05:16.854 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-04-03T03:05:16.855 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Command line: ntpd -gq 2021-04-03T03:05:16.856 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: proto: precision = 0.031 usec (-25) 2021-04-03T03:05:16.856 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:16.856 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:16.857 INFO:teuthology.orchestra.run.gibba011.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:16.857 INFO:teuthology.orchestra.run.gibba011.stderr:restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:16.858 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Listen and drop on 0 v6wildcard [::]:123 2021-04-03T03:05:16.858 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-04-03T03:05:16.858 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Listen normally on 2 lo 127.0.0.1:123 2021-04-03T03:05:16.858 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Listen normally on 3 enp1s0f0 172.21.2.111:123 2021-04-03T03:05:16.859 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Listen normally on 4 lo [::1]:123 2021-04-03T03:05:16.859 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8e0a%2]:123 2021-04-03T03:05:16.859 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:17 ntpd[10662]: Listening on routing socket on fd #22 for interface updates 2021-04-03T03:05:18.857 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:19 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:18.857 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:19 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:18.858 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:19 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:18.858 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:19 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:20.858 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:20.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:20.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:20.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:20.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:20.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:20.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:20.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:21 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:21.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:21.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:21.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:21.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:21.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:21.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:21.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:21.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:22 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:22.859 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:22.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:22.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:22.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:22.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:22.861 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:22.861 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:22.861 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:23 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:23.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:23.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:23.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:23.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:23.861 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:23.861 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:23.861 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:23.861 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:24.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:25 ntpd[10662]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:24.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:25 ntpd[10662]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:24.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:25 ntpd[10662]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:24.860 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:25 ntpd[10662]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:24.861 INFO:teuthology.orchestra.run.gibba011.stdout: 3 Apr 03:05:24 ntpd[10662]: ntpd: time set -0.207233 s 2021-04-03T03:05:24.861 INFO:teuthology.orchestra.run.gibba011.stdout:ntpd: time set -0.207233s 2021-04-03T03:05:24.862 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-04-03T03:05:24.862 INFO:teuthology.orchestra.run.gibba011.stderr: 3 Apr 03:05:24 ntpd[10662]: can't open /var/log/ntpstats/loopstats.20210403: Permission denied 2021-04-03T03:05:24.920 INFO:teuthology.orchestra.run.gibba011.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T03:05:24.920 INFO:teuthology.orchestra.run.gibba011.stdout:============================================================================== 2021-04-03T03:05:24.921 INFO:teuthology.orchestra.run.gibba011.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:24.921 INFO:teuthology.orchestra.run.gibba011.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:24.921 INFO:teuthology.orchestra.run.gibba011.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:24.921 INFO:teuthology.orchestra.run.gibba011.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:24.922 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-04-03T03:05:24.922 DEBUG:teuthology.orchestra.run.gibba015:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T03:05:24.964 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-04-03T03:05:24.965 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Command line: ntpd -gq 2021-04-03T03:05:24.965 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: proto: precision = 0.042 usec (-24) 2021-04-03T03:05:24.966 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:24.966 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:24.966 INFO:teuthology.orchestra.run.gibba015.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:24.966 INFO:teuthology.orchestra.run.gibba015.stderr:restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:24.967 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Listen and drop on 0 v6wildcard [::]:123 2021-04-03T03:05:24.967 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-04-03T03:05:24.968 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Listen normally on 2 lo 127.0.0.1:123 2021-04-03T03:05:24.968 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Listen normally on 3 enp1s0f0 172.21.2.115:123 2021-04-03T03:05:24.968 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Listen normally on 4 lo [::1]:123 2021-04-03T03:05:24.968 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:9252%2]:123 2021-04-03T03:05:24.968 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:25 ntpd[10588]: Listening on routing socket on fd #22 for interface updates 2021-04-03T03:05:25.965 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:26 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:25.966 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:26 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:25.966 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:26 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:25.966 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:26 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:27.968 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:27.968 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:27.968 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:27.968 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:27.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:27.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:27.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:27.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:28 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:28.968 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:29 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:28.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:29 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:28.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:29 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:28.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:29 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:29.968 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:29.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:29.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:29.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:29.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:29.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:29.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:29.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:29.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:29.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:29.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:29.971 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:30 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:30.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:31 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:30.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:31 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:30.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:31 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:30.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:31 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:31.969 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:32 ntpd[10588]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:31.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:32 ntpd[10588]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:31.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:32 ntpd[10588]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:31.970 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:32 ntpd[10588]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:31.971 INFO:teuthology.orchestra.run.gibba015.stdout: 3 Apr 03:05:31 ntpd[10588]: ntpd: time set -0.136218 s 2021-04-03T03:05:31.971 INFO:teuthology.orchestra.run.gibba015.stdout:ntpd: time set -0.136218s 2021-04-03T03:05:31.971 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:31 ntpd[10588]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-04-03T03:05:31.972 INFO:teuthology.orchestra.run.gibba015.stderr: 3 Apr 03:05:31 ntpd[10588]: can't open /var/log/ntpstats/loopstats.20210403: Permission denied 2021-04-03T03:05:32.035 INFO:teuthology.orchestra.run.gibba015.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T03:05:32.035 INFO:teuthology.orchestra.run.gibba015.stdout:============================================================================== 2021-04-03T03:05:32.036 INFO:teuthology.orchestra.run.gibba015.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:32.036 INFO:teuthology.orchestra.run.gibba015.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:32.036 INFO:teuthology.orchestra.run.gibba015.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:32.036 INFO:teuthology.orchestra.run.gibba015.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:32.036 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-04-03T03:05:32.037 DEBUG:teuthology.orchestra.run.gibba026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T03:05:32.088 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-04-03T03:05:32.089 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Command line: ntpd -gq 2021-04-03T03:05:32.090 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: proto: precision = 0.049 usec (-24) 2021-04-03T03:05:32.090 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:32.090 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:32.090 INFO:teuthology.orchestra.run.gibba026.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:32.091 INFO:teuthology.orchestra.run.gibba026.stderr:restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:32.092 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Listen and drop on 0 v6wildcard [::]:123 2021-04-03T03:05:32.092 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-04-03T03:05:32.092 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Listen normally on 2 lo 127.0.0.1:123 2021-04-03T03:05:32.092 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Listen normally on 3 enp1s0f0 172.21.2.126:123 2021-04-03T03:05:32.092 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Listen normally on 4 lo [::1]:123 2021-04-03T03:05:32.093 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8e04%2]:123 2021-04-03T03:05:32.093 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:32 ntpd[10601]: Listening on routing socket on fd #22 for interface updates 2021-04-03T03:05:33.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:33.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:33.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:33.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:33.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:33.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:33.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:33.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:33 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:34.089 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:34 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:34.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:34 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:34.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:34 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:34.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:34 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:35.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:35.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:35.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:35.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:35.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:35.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:35.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:35.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:35 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:36.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:36.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:36.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:36.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:36.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:36.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:36.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:36.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:36 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:37.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:37.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:37.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:37.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:37.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:37.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:37.093 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:37.093 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:37 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:38.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:38.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:38.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:38.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:38.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:38.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:38.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:38.092 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:38 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:39.090 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:39 ntpd[10601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:39.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:39 ntpd[10601]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:39.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:39 ntpd[10601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:39.091 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:39 ntpd[10601]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:39.092 INFO:teuthology.orchestra.run.gibba026.stdout: 3 Apr 03:05:39 ntpd[10601]: ntpd: time slew +0.002734 s 2021-04-03T03:05:39.092 INFO:teuthology.orchestra.run.gibba026.stdout:ntpd: time slew +0.002734s 2021-04-03T03:05:39.093 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:39 ntpd[10601]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-04-03T03:05:39.093 INFO:teuthology.orchestra.run.gibba026.stderr: 3 Apr 03:05:39 ntpd[10601]: can't open /var/log/ntpstats/loopstats.20210403: Permission denied 2021-04-03T03:05:39.142 INFO:teuthology.orchestra.run.gibba026.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T03:05:39.142 INFO:teuthology.orchestra.run.gibba026.stdout:============================================================================== 2021-04-03T03:05:39.143 INFO:teuthology.orchestra.run.gibba026.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:39.143 INFO:teuthology.orchestra.run.gibba026.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:39.143 INFO:teuthology.orchestra.run.gibba026.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:39.143 INFO:teuthology.orchestra.run.gibba026.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:39.144 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-04-03T03:05:39.145 DEBUG:teuthology.orchestra.run.gibba028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T03:05:39.205 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-04-03T03:05:39.205 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Command line: ntpd -gq 2021-04-03T03:05:39.206 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: proto: precision = 0.049 usec (-24) 2021-04-03T03:05:39.206 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:39.207 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:39.207 INFO:teuthology.orchestra.run.gibba028.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:39.207 INFO:teuthology.orchestra.run.gibba028.stderr:restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:39.208 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Listen and drop on 0 v6wildcard [::]:123 2021-04-03T03:05:39.209 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-04-03T03:05:39.209 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Listen normally on 2 lo 127.0.0.1:123 2021-04-03T03:05:39.209 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Listen normally on 3 enp1s0f0 172.21.2.128:123 2021-04-03T03:05:39.209 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Listen normally on 4 lo [::1]:123 2021-04-03T03:05:39.210 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:9190%2]:123 2021-04-03T03:05:39.210 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:39 ntpd[10621]: Listening on routing socket on fd #22 for interface updates 2021-04-03T03:05:40.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:40.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:40.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:40.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:40.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:40.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:40.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:40.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:40 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:41.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:41 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:41.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:41 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:41.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:41 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:41.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:41 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:42.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:42.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:42.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:42.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:42.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:42.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:42.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:42.208 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:42.208 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:42.208 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:42.208 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:42.208 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:42 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:43.204 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:43 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:43.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:43 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:43.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:43 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:43.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:43 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:44.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:44.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:44.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:44.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:44.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:44.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:44.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:44.206 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:44.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:44.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:44.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:44.207 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:44 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:45.203 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:45 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:45.203 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:45 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:45.204 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:45 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:45.204 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:45 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:46.203 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:46 ntpd[10621]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:46.203 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:46 ntpd[10621]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:46.203 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:46 ntpd[10621]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:46.204 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:46 ntpd[10621]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:46.204 INFO:teuthology.orchestra.run.gibba028.stdout: 3 Apr 03:05:46 ntpd[10621]: ntpd: time set +0.085139 s 2021-04-03T03:05:46.204 INFO:teuthology.orchestra.run.gibba028.stdout:ntpd: time set +0.085139s 2021-04-03T03:05:46.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:46 ntpd[10621]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-04-03T03:05:46.205 INFO:teuthology.orchestra.run.gibba028.stderr: 3 Apr 03:05:46 ntpd[10621]: can't open /var/log/ntpstats/loopstats.20210403: Permission denied 2021-04-03T03:05:46.237 INFO:teuthology.orchestra.run.gibba028.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T03:05:46.237 INFO:teuthology.orchestra.run.gibba028.stdout:============================================================================== 2021-04-03T03:05:46.237 INFO:teuthology.orchestra.run.gibba028.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:46.238 INFO:teuthology.orchestra.run.gibba028.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:46.238 INFO:teuthology.orchestra.run.gibba028.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:46.238 INFO:teuthology.orchestra.run.gibba028.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:46.238 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-04-03T03:05:46.239 DEBUG:teuthology.orchestra.run.gibba031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T03:05:46.290 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-04-03T03:05:46.290 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Command line: ntpd -gq 2021-04-03T03:05:46.291 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: proto: precision = 0.045 usec (-24) 2021-04-03T03:05:46.291 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:46.292 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:46.292 INFO:teuthology.orchestra.run.gibba031.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-04-03T03:05:46.293 INFO:teuthology.orchestra.run.gibba031.stderr:restrict ::: KOD does nothing without LIMITED. 2021-04-03T03:05:46.294 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Listen and drop on 0 v6wildcard [::]:123 2021-04-03T03:05:46.294 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-04-03T03:05:46.295 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Listen normally on 2 lo 127.0.0.1:123 2021-04-03T03:05:46.295 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Listen normally on 3 enp1s0f0 172.21.2.131:123 2021-04-03T03:05:46.295 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Listen normally on 4 lo [::1]:123 2021-04-03T03:05:46.295 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:9188%2]:123 2021-04-03T03:05:46.296 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:46 ntpd[10478]: Listening on routing socket on fd #22 for interface updates 2021-04-03T03:05:48.292 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:48.292 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:48.293 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:48.293 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:48.293 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:48.293 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:48.294 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:48.294 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:48 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:50.294 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:50.294 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:50.294 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:50.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:50.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:50.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:50.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:50.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:50.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:50.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:50.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:50.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:50.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:50.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:50.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:50.298 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:50 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:52.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:52.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:52.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:52.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:52.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:52.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:52.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:52.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:52.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:52.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:52.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:52.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:52.298 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:52.298 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:52.298 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:52.298 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:52 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:54.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:54.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:54.295 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:54.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:54.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-04-03T03:05:54.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: can't open /var/log/ntpstats/rawstats.20210403: Permission denied 2021-04-03T03:05:54.296 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-04-03T03:05:54.297 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: can't open /var/log/ntpstats/peerstats.20210403: Permission denied 2021-04-03T03:05:54.297 INFO:teuthology.orchestra.run.gibba031.stdout: 3 Apr 03:05:54 ntpd[10478]: ntpd: time set -0.247466 s 2021-04-03T03:05:54.297 INFO:teuthology.orchestra.run.gibba031.stdout:ntpd: time set -0.247466s 2021-04-03T03:05:54.298 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-04-03T03:05:54.298 INFO:teuthology.orchestra.run.gibba031.stderr: 3 Apr 03:05:54 ntpd[10478]: can't open /var/log/ntpstats/loopstats.20210403: Permission denied 2021-04-03T03:05:54.364 INFO:teuthology.orchestra.run.gibba031.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T03:05:54.364 INFO:teuthology.orchestra.run.gibba031.stdout:============================================================================== 2021-04-03T03:05:54.364 INFO:teuthology.orchestra.run.gibba031.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:54.364 INFO:teuthology.orchestra.run.gibba031.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:54.365 INFO:teuthology.orchestra.run.gibba031.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:54.365 INFO:teuthology.orchestra.run.gibba031.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-04-03T03:05:54.366 INFO:teuthology.run_tasks:Running task install... 2021-04-03T03:05:54.377 DEBUG:teuthology.task.install:project ceph 2021-04-03T03:05:54.378 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '5feb67b5bda898b8c739180363606432d4f39831'}} 2021-04-03T03:05:54.378 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '5feb67b5bda898b8c739180363606432d4f39831'} 2021-04-03T03:05:54.378 INFO:teuthology.task.install:Using flavor: basic 2021-04-03T03:05:54.384 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-04-03T03:05:54.385 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2021-04-03T03:05:54.385 DEBUG:teuthology.orchestra.run.gibba011:> sudo lsb_release -is 2021-04-03T03:05:54.414 INFO:teuthology.orchestra.run.gibba011.stdout:Ubuntu 2021-04-03T03:05:54.417 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.417 DEBUG:teuthology.orchestra.run.gibba015:> sudo lsb_release -is 2021-04-03T03:05:54.418 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-key list | grep Ceph 2021-04-03T03:05:54.446 INFO:teuthology.orchestra.run.gibba015.stdout:Ubuntu 2021-04-03T03:05:54.448 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.449 DEBUG:teuthology.orchestra.run.gibba026:> sudo lsb_release -is 2021-04-03T03:05:54.449 DEBUG:teuthology.orchestra.run.gibba015:> sudo apt-key list | grep Ceph 2021-04-03T03:05:54.477 INFO:teuthology.orchestra.run.gibba026.stdout:Ubuntu 2021-04-03T03:05:54.480 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.480 DEBUG:teuthology.orchestra.run.gibba028:> sudo lsb_release -is 2021-04-03T03:05:54.481 DEBUG:teuthology.orchestra.run.gibba026:> sudo apt-key list | grep Ceph 2021-04-03T03:05:54.509 INFO:teuthology.orchestra.run.gibba028.stdout:Ubuntu 2021-04-03T03:05:54.512 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.512 DEBUG:teuthology.orchestra.run.gibba031:> sudo lsb_release -is 2021-04-03T03:05:54.513 DEBUG:teuthology.orchestra.run.gibba028:> sudo apt-key list | grep Ceph 2021-04-03T03:05:54.524 INFO:teuthology.orchestra.run.gibba011.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-04-03T03:05:54.542 INFO:teuthology.orchestra.run.gibba031.stdout:Ubuntu 2021-04-03T03:05:54.543 INFO:teuthology.orchestra.run.gibba015.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-04-03T03:05:54.545 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.546 DEBUG:teuthology.orchestra.run.gibba031:> sudo apt-key list | grep Ceph 2021-04-03T03:05:54.557 INFO:teuthology.orchestra.run.gibba011.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-04-03T03:05:54.558 INFO:teuthology.orchestra.run.gibba011.stdout:uid [ unknown] Ceph.com (release key) 2021-04-03T03:05:54.559 DEBUG:teuthology.orchestra.run.gibba011:> sudo lsb_release -is 2021-04-03T03:05:54.564 INFO:teuthology.orchestra.run.gibba015.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-04-03T03:05:54.565 INFO:teuthology.orchestra.run.gibba015.stdout:uid [ unknown] Ceph.com (release key) 2021-04-03T03:05:54.566 DEBUG:teuthology.orchestra.run.gibba015:> sudo lsb_release -is 2021-04-03T03:05:54.585 INFO:teuthology.orchestra.run.gibba026.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-04-03T03:05:54.593 INFO:teuthology.orchestra.run.gibba011.stdout:Ubuntu 2021-04-03T03:05:54.594 INFO:teuthology.orchestra.run.gibba015.stdout:Ubuntu 2021-04-03T03:05:54.598 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.598 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-04-03T03:05:54.599 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-04-03T03:05:54.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:05:54.603 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.604 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-04-03T03:05:54.604 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-04-03T03:05:54.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:05:54.609 INFO:teuthology.orchestra.run.gibba026.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-04-03T03:05:54.609 INFO:teuthology.orchestra.run.gibba026.stdout:uid [ unknown] Ceph.com (release key) 2021-04-03T03:05:54.610 DEBUG:teuthology.orchestra.run.gibba026:> sudo lsb_release -is 2021-04-03T03:05:54.613 INFO:teuthology.orchestra.run.gibba028.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-04-03T03:05:54.644 INFO:teuthology.orchestra.run.gibba028.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-04-03T03:05:54.644 INFO:teuthology.orchestra.run.gibba028.stdout:uid [ unknown] Ceph.com (release key) 2021-04-03T03:05:54.644 DEBUG:teuthology.orchestra.run.gibba028:> sudo lsb_release -is 2021-04-03T03:05:54.646 INFO:teuthology.orchestra.run.gibba026.stdout:Ubuntu 2021-04-03T03:05:54.646 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.646 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-04-03T03:05:54.647 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-04-03T03:05:54.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:05:54.650 INFO:teuthology.orchestra.run.gibba031.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-04-03T03:05:54.678 INFO:teuthology.orchestra.run.gibba028.stdout:Ubuntu 2021-04-03T03:05:54.680 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.681 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-04-03T03:05:54.681 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-04-03T03:05:54.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:05:54.718 INFO:teuthology.orchestra.run.gibba031.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-04-03T03:05:54.719 INFO:teuthology.orchestra.run.gibba031.stdout:uid [ unknown] Ceph.com (release key) 2021-04-03T03:05:54.720 DEBUG:teuthology.orchestra.run.gibba031:> sudo lsb_release -is 2021-04-03T03:05:54.752 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default/ 2021-04-03T03:05:54.753 INFO:teuthology.task.install.deb:Package version is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:54.759 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default/ 2021-04-03T03:05:54.760 INFO:teuthology.task.install.deb:Package version is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:54.776 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default/ 2021-04-03T03:05:54.776 INFO:teuthology.task.install.deb:Package version is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:54.793 INFO:teuthology.orchestra.run.gibba031.stdout:Ubuntu 2021-04-03T03:05:54.796 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:05:54.797 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-04-03T03:05:54.797 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-04-03T03:05:54.797 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:05:54.830 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default/ 2021-04-03T03:05:54.830 INFO:teuthology.task.install.deb:Package version is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:54.866 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:05:54.867 DEBUG:teuthology.orchestra.run.gibba015:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-04-03T03:05:54.877 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:05:54.877 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-04-03T03:05:54.881 DEBUG:teuthology.orchestra.run.gibba015:> sudo apt-get update 2021-04-03T03:05:54.881 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:05:54.882 DEBUG:teuthology.orchestra.run.gibba026:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-04-03T03:05:54.893 DEBUG:teuthology.orchestra.run.gibba026:> sudo apt-get update 2021-04-03T03:05:54.902 DEBUG:teuthology.orchestra.run.gibba011:> sudo apt-get update 2021-04-03T03:05:54.933 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default/ 2021-04-03T03:05:54.934 INFO:teuthology.task.install.deb:Package version is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:54.938 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:05:54.938 DEBUG:teuthology.orchestra.run.gibba028:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-04-03T03:05:54.949 DEBUG:teuthology.orchestra.run.gibba028:> sudo apt-get update 2021-04-03T03:05:55.024 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-04-03T03:05:55.040 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:05:55.040 DEBUG:teuthology.orchestra.run.gibba031:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-04-03T03:05:55.051 DEBUG:teuthology.orchestra.run.gibba031:> sudo apt-get update 2021-04-03T03:05:55.073 INFO:teuthology.orchestra.run.gibba028.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-04-03T03:05:55.076 INFO:teuthology.orchestra.run.gibba011.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic InRelease 2021-04-03T03:05:55.082 INFO:teuthology.orchestra.run.gibba015.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic InRelease 2021-04-03T03:05:55.094 INFO:teuthology.orchestra.run.gibba026.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic InRelease 2021-04-03T03:05:55.098 INFO:teuthology.orchestra.run.gibba011.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-04-03T03:05:55.103 INFO:teuthology.orchestra.run.gibba015.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-04-03T03:05:55.115 INFO:teuthology.orchestra.run.gibba026.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-04-03T03:05:55.120 INFO:teuthology.orchestra.run.gibba011.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release.gpg 2021-04-03T03:05:55.125 INFO:teuthology.orchestra.run.gibba028.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic InRelease 2021-04-03T03:05:55.127 INFO:teuthology.orchestra.run.gibba015.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release.gpg 2021-04-03T03:05:55.139 INFO:teuthology.orchestra.run.gibba026.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release.gpg 2021-04-03T03:05:55.140 INFO:teuthology.orchestra.run.gibba011.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,085 B] 2021-04-03T03:05:55.145 INFO:teuthology.orchestra.run.gibba028.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-04-03T03:05:55.146 INFO:teuthology.orchestra.run.gibba011.stdout:Hit:6 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:05:55.148 INFO:teuthology.orchestra.run.gibba015.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.7 kB] 2021-04-03T03:05:55.160 INFO:teuthology.orchestra.run.gibba015.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:05:55.160 INFO:teuthology.orchestra.run.gibba015.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T03:05:55.161 INFO:teuthology.orchestra.run.gibba026.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,085 B] 2021-04-03T03:05:55.165 INFO:teuthology.orchestra.run.gibba011.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.7 kB] 2021-04-03T03:05:55.167 INFO:teuthology.orchestra.run.gibba028.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release.gpg 2021-04-03T03:05:55.171 INFO:teuthology.orchestra.run.gibba026.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:05:55.172 INFO:teuthology.orchestra.run.gibba026.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T03:05:55.181 INFO:teuthology.orchestra.run.gibba015.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,085 B] 2021-04-03T03:05:55.188 INFO:teuthology.orchestra.run.gibba028.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.7 kB] 2021-04-03T03:05:55.191 INFO:teuthology.orchestra.run.gibba026.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.7 kB] 2021-04-03T03:05:55.197 INFO:teuthology.orchestra.run.gibba028.stdout:Hit:6 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:05:55.218 INFO:teuthology.orchestra.run.gibba028.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,085 B] 2021-04-03T03:05:55.226 INFO:teuthology.orchestra.run.gibba011.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T03:05:55.231 INFO:teuthology.orchestra.run.gibba031.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic InRelease 2021-04-03T03:05:55.232 INFO:teuthology.orchestra.run.gibba015.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T03:05:55.244 INFO:teuthology.orchestra.run.gibba026.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T03:05:55.252 INFO:teuthology.orchestra.run.gibba031.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-04-03T03:05:55.275 INFO:teuthology.orchestra.run.gibba031.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic Release.gpg 2021-04-03T03:05:55.276 INFO:teuthology.orchestra.run.gibba028.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T03:05:55.296 INFO:teuthology.orchestra.run.gibba031.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,085 B] 2021-04-03T03:05:55.310 INFO:teuthology.orchestra.run.gibba031.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-04-03T03:05:55.313 INFO:teuthology.orchestra.run.gibba031.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-04-03T03:05:55.333 INFO:teuthology.orchestra.run.gibba031.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.7 kB] 2021-04-03T03:05:55.376 INFO:teuthology.orchestra.run.gibba031.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-04-03T03:05:55.507 INFO:teuthology.orchestra.run.gibba015.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T03:05:55.508 INFO:teuthology.orchestra.run.gibba026.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T03:05:55.519 INFO:teuthology.orchestra.run.gibba011.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T03:05:55.531 INFO:teuthology.orchestra.run.gibba028.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T03:05:55.668 INFO:teuthology.orchestra.run.gibba015.stdout:Fetched 279 kB in 1s (456 kB/s) 2021-04-03T03:05:55.678 INFO:teuthology.orchestra.run.gibba011.stdout:Fetched 190 kB in 1s (306 kB/s) 2021-04-03T03:05:55.681 INFO:teuthology.orchestra.run.gibba031.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-04-03T03:05:55.700 INFO:teuthology.orchestra.run.gibba028.stdout:Fetched 190 kB in 1s (326 kB/s) 2021-04-03T03:05:55.762 INFO:teuthology.orchestra.run.gibba026.stdout:Fetched 279 kB in 1s (411 kB/s) 2021-04-03T03:05:55.847 INFO:teuthology.orchestra.run.gibba031.stdout:Fetched 279 kB in 1s (414 kB/s) 2021-04-03T03:05:56.508 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T03:05:56.518 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T03:05:56.533 DEBUG:teuthology.orchestra.run.gibba015:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.10-137-g5feb67b5-1bionic cephadm=15.2.10-137-g5feb67b5-1bionic ceph-mds=15.2.10-137-g5feb67b5-1bionic ceph-mgr=15.2.10-137-g5feb67b5-1bionic ceph-common=15.2.10-137-g5feb67b5-1bionic ceph-fuse=15.2.10-137-g5feb67b5-1bionic ceph-test=15.2.10-137-g5feb67b5-1bionic radosgw=15.2.10-137-g5feb67b5-1bionic python3-rados=15.2.10-137-g5feb67b5-1bionic python3-rgw=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic python3-rbd=15.2.10-137-g5feb67b5-1bionic libcephfs2=15.2.10-137-g5feb67b5-1bionic libcephfs-dev=15.2.10-137-g5feb67b5-1bionic librados2=15.2.10-137-g5feb67b5-1bionic librbd1=15.2.10-137-g5feb67b5-1bionic rbd-fuse=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic cephfs-shell=15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:56.541 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T03:05:56.544 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.10-137-g5feb67b5-1bionic cephadm=15.2.10-137-g5feb67b5-1bionic ceph-mds=15.2.10-137-g5feb67b5-1bionic ceph-mgr=15.2.10-137-g5feb67b5-1bionic ceph-common=15.2.10-137-g5feb67b5-1bionic ceph-fuse=15.2.10-137-g5feb67b5-1bionic ceph-test=15.2.10-137-g5feb67b5-1bionic radosgw=15.2.10-137-g5feb67b5-1bionic python3-rados=15.2.10-137-g5feb67b5-1bionic python3-rgw=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic python3-rbd=15.2.10-137-g5feb67b5-1bionic libcephfs2=15.2.10-137-g5feb67b5-1bionic libcephfs-dev=15.2.10-137-g5feb67b5-1bionic librados2=15.2.10-137-g5feb67b5-1bionic librbd1=15.2.10-137-g5feb67b5-1bionic rbd-fuse=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic cephfs-shell=15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:56.567 DEBUG:teuthology.orchestra.run.gibba028:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.10-137-g5feb67b5-1bionic cephadm=15.2.10-137-g5feb67b5-1bionic ceph-mds=15.2.10-137-g5feb67b5-1bionic ceph-mgr=15.2.10-137-g5feb67b5-1bionic ceph-common=15.2.10-137-g5feb67b5-1bionic ceph-fuse=15.2.10-137-g5feb67b5-1bionic ceph-test=15.2.10-137-g5feb67b5-1bionic radosgw=15.2.10-137-g5feb67b5-1bionic python3-rados=15.2.10-137-g5feb67b5-1bionic python3-rgw=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic python3-rbd=15.2.10-137-g5feb67b5-1bionic libcephfs2=15.2.10-137-g5feb67b5-1bionic libcephfs-dev=15.2.10-137-g5feb67b5-1bionic librados2=15.2.10-137-g5feb67b5-1bionic librbd1=15.2.10-137-g5feb67b5-1bionic rbd-fuse=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic cephfs-shell=15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:56.578 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T03:05:56.588 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T03:05:56.600 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T03:05:56.612 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T03:05:56.626 DEBUG:teuthology.orchestra.run.gibba026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.10-137-g5feb67b5-1bionic cephadm=15.2.10-137-g5feb67b5-1bionic ceph-mds=15.2.10-137-g5feb67b5-1bionic ceph-mgr=15.2.10-137-g5feb67b5-1bionic ceph-common=15.2.10-137-g5feb67b5-1bionic ceph-fuse=15.2.10-137-g5feb67b5-1bionic ceph-test=15.2.10-137-g5feb67b5-1bionic radosgw=15.2.10-137-g5feb67b5-1bionic python3-rados=15.2.10-137-g5feb67b5-1bionic python3-rgw=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic python3-rbd=15.2.10-137-g5feb67b5-1bionic libcephfs2=15.2.10-137-g5feb67b5-1bionic libcephfs-dev=15.2.10-137-g5feb67b5-1bionic librados2=15.2.10-137-g5feb67b5-1bionic librbd1=15.2.10-137-g5feb67b5-1bionic rbd-fuse=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic cephfs-shell=15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:56.671 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T03:05:56.684 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T03:05:56.709 DEBUG:teuthology.orchestra.run.gibba031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.10-137-g5feb67b5-1bionic cephadm=15.2.10-137-g5feb67b5-1bionic ceph-mds=15.2.10-137-g5feb67b5-1bionic ceph-mgr=15.2.10-137-g5feb67b5-1bionic ceph-common=15.2.10-137-g5feb67b5-1bionic ceph-fuse=15.2.10-137-g5feb67b5-1bionic ceph-test=15.2.10-137-g5feb67b5-1bionic radosgw=15.2.10-137-g5feb67b5-1bionic python3-rados=15.2.10-137-g5feb67b5-1bionic python3-rgw=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic python3-rbd=15.2.10-137-g5feb67b5-1bionic libcephfs2=15.2.10-137-g5feb67b5-1bionic libcephfs-dev=15.2.10-137-g5feb67b5-1bionic librados2=15.2.10-137-g5feb67b5-1bionic librbd1=15.2.10-137-g5feb67b5-1bionic rbd-fuse=15.2.10-137-g5feb67b5-1bionic python3-cephfs=15.2.10-137-g5feb67b5-1bionic cephfs-shell=15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:05:56.740 INFO:teuthology.orchestra.run.gibba015.stdout:Building dependency tree... 2021-04-03T03:05:56.741 INFO:teuthology.orchestra.run.gibba015.stdout:Reading state information... 2021-04-03T03:05:56.753 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-04-03T03:05:56.753 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-04-03T03:05:56.755 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T03:05:56.776 INFO:teuthology.orchestra.run.gibba028.stdout:Building dependency tree... 2021-04-03T03:05:56.777 INFO:teuthology.orchestra.run.gibba028.stdout:Reading state information... 2021-04-03T03:05:56.838 INFO:teuthology.orchestra.run.gibba015.stdout:The following additional packages will be installed: 2021-04-03T03:05:56.838 INFO:teuthology.orchestra.run.gibba015.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:56.839 INFO:teuthology.orchestra.run.gibba015.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:56.839 INFO:teuthology.orchestra.run.gibba015.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2021-04-03T03:05:56.839 INFO:teuthology.orchestra.run.gibba015.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-04-03T03:05:56.839 INFO:teuthology.orchestra.run.gibba015.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-04-03T03:05:56.839 INFO:teuthology.orchestra.run.gibba015.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-04-03T03:05:56.840 INFO:teuthology.orchestra.run.gibba015.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-04-03T03:05:56.840 INFO:teuthology.orchestra.run.gibba015.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-04-03T03:05:56.840 INFO:teuthology.orchestra.run.gibba015.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-04-03T03:05:56.840 INFO:teuthology.orchestra.run.gibba015.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2021-04-03T03:05:56.841 INFO:teuthology.orchestra.run.gibba015.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2021-04-03T03:05:56.841 INFO:teuthology.orchestra.run.gibba015.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2021-04-03T03:05:56.841 INFO:teuthology.orchestra.run.gibba015.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2021-04-03T03:05:56.841 INFO:teuthology.orchestra.run.gibba015.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2021-04-03T03:05:56.842 INFO:teuthology.orchestra.run.gibba015.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2021-04-03T03:05:56.842 INFO:teuthology.orchestra.run.gibba015.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2021-04-03T03:05:56.842 INFO:teuthology.orchestra.run.gibba015.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2021-04-03T03:05:56.842 INFO:teuthology.orchestra.run.gibba015.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2021-04-03T03:05:56.842 INFO:teuthology.orchestra.run.gibba015.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-04-03T03:05:56.843 INFO:teuthology.orchestra.run.gibba015.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-04-03T03:05:56.843 INFO:teuthology.orchestra.run.gibba015.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-04-03T03:05:56.843 INFO:teuthology.orchestra.run.gibba015.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-04-03T03:05:56.843 INFO:teuthology.orchestra.run.gibba015.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-04-03T03:05:56.844 INFO:teuthology.orchestra.run.gibba015.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-04-03T03:05:56.844 INFO:teuthology.orchestra.run.gibba015.stdout:Suggested packages: 2021-04-03T03:05:56.844 INFO:teuthology.orchestra.run.gibba015.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-04-03T03:05:56.844 INFO:teuthology.orchestra.run.gibba015.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-04-03T03:05:56.844 INFO:teuthology.orchestra.run.gibba015.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-04-03T03:05:56.845 INFO:teuthology.orchestra.run.gibba015.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-04-03T03:05:56.845 INFO:teuthology.orchestra.run.gibba015.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-04-03T03:05:56.845 INFO:teuthology.orchestra.run.gibba015.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-04-03T03:05:56.845 INFO:teuthology.orchestra.run.gibba015.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-04-03T03:05:56.845 INFO:teuthology.orchestra.run.gibba015.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-04-03T03:05:56.846 INFO:teuthology.orchestra.run.gibba015.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-04-03T03:05:56.846 INFO:teuthology.orchestra.run.gibba015.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-04-03T03:05:56.846 INFO:teuthology.orchestra.run.gibba015.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-04-03T03:05:56.846 INFO:teuthology.orchestra.run.gibba015.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-04-03T03:05:56.847 INFO:teuthology.orchestra.run.gibba015.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-04-03T03:05:56.847 INFO:teuthology.orchestra.run.gibba015.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-04-03T03:05:56.847 INFO:teuthology.orchestra.run.gibba015.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-04-03T03:05:56.848 INFO:teuthology.orchestra.run.gibba026.stdout:Building dependency tree... 2021-04-03T03:05:56.848 INFO:teuthology.orchestra.run.gibba026.stdout:Reading state information... 2021-04-03T03:05:56.853 INFO:teuthology.orchestra.run.gibba011.stdout:The following additional packages will be installed: 2021-04-03T03:05:56.853 INFO:teuthology.orchestra.run.gibba011.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:56.853 INFO:teuthology.orchestra.run.gibba011.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:56.854 INFO:teuthology.orchestra.run.gibba011.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2021-04-03T03:05:56.854 INFO:teuthology.orchestra.run.gibba011.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-04-03T03:05:56.855 INFO:teuthology.orchestra.run.gibba011.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-04-03T03:05:56.855 INFO:teuthology.orchestra.run.gibba011.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-04-03T03:05:56.855 INFO:teuthology.orchestra.run.gibba011.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-04-03T03:05:56.855 INFO:teuthology.orchestra.run.gibba011.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-04-03T03:05:56.856 INFO:teuthology.orchestra.run.gibba011.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-04-03T03:05:56.856 INFO:teuthology.orchestra.run.gibba011.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2021-04-03T03:05:56.856 INFO:teuthology.orchestra.run.gibba011.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2021-04-03T03:05:56.856 INFO:teuthology.orchestra.run.gibba011.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2021-04-03T03:05:56.856 INFO:teuthology.orchestra.run.gibba011.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2021-04-03T03:05:56.857 INFO:teuthology.orchestra.run.gibba011.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2021-04-03T03:05:56.857 INFO:teuthology.orchestra.run.gibba011.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2021-04-03T03:05:56.857 INFO:teuthology.orchestra.run.gibba011.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2021-04-03T03:05:56.857 INFO:teuthology.orchestra.run.gibba011.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2021-04-03T03:05:56.857 INFO:teuthology.orchestra.run.gibba011.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2021-04-03T03:05:56.858 INFO:teuthology.orchestra.run.gibba011.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-04-03T03:05:56.858 INFO:teuthology.orchestra.run.gibba011.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-04-03T03:05:56.858 INFO:teuthology.orchestra.run.gibba011.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-04-03T03:05:56.858 INFO:teuthology.orchestra.run.gibba011.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-04-03T03:05:56.859 INFO:teuthology.orchestra.run.gibba011.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-04-03T03:05:56.859 INFO:teuthology.orchestra.run.gibba011.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-04-03T03:05:56.859 INFO:teuthology.orchestra.run.gibba011.stdout:Suggested packages: 2021-04-03T03:05:56.859 INFO:teuthology.orchestra.run.gibba011.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-04-03T03:05:56.859 INFO:teuthology.orchestra.run.gibba011.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-04-03T03:05:56.860 INFO:teuthology.orchestra.run.gibba011.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-04-03T03:05:56.860 INFO:teuthology.orchestra.run.gibba011.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-04-03T03:05:56.860 INFO:teuthology.orchestra.run.gibba011.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-04-03T03:05:56.860 INFO:teuthology.orchestra.run.gibba011.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-04-03T03:05:56.860 INFO:teuthology.orchestra.run.gibba011.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-04-03T03:05:56.861 INFO:teuthology.orchestra.run.gibba011.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-04-03T03:05:56.861 INFO:teuthology.orchestra.run.gibba011.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-04-03T03:05:56.861 INFO:teuthology.orchestra.run.gibba011.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-04-03T03:05:56.861 INFO:teuthology.orchestra.run.gibba011.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-04-03T03:05:56.861 INFO:teuthology.orchestra.run.gibba011.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-04-03T03:05:56.862 INFO:teuthology.orchestra.run.gibba011.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-04-03T03:05:56.862 INFO:teuthology.orchestra.run.gibba011.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-04-03T03:05:56.862 INFO:teuthology.orchestra.run.gibba011.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-04-03T03:05:56.874 INFO:teuthology.orchestra.run.gibba028.stdout:The following additional packages will be installed: 2021-04-03T03:05:56.874 INFO:teuthology.orchestra.run.gibba028.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:56.874 INFO:teuthology.orchestra.run.gibba028.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:56.877 INFO:teuthology.orchestra.run.gibba028.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2021-04-03T03:05:56.877 INFO:teuthology.orchestra.run.gibba028.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-04-03T03:05:56.877 INFO:teuthology.orchestra.run.gibba028.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-04-03T03:05:56.877 INFO:teuthology.orchestra.run.gibba028.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-04-03T03:05:56.877 INFO:teuthology.orchestra.run.gibba028.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-04-03T03:05:56.878 INFO:teuthology.orchestra.run.gibba028.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-04-03T03:05:56.878 INFO:teuthology.orchestra.run.gibba028.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-04-03T03:05:56.878 INFO:teuthology.orchestra.run.gibba028.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2021-04-03T03:05:56.878 INFO:teuthology.orchestra.run.gibba028.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2021-04-03T03:05:56.879 INFO:teuthology.orchestra.run.gibba028.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2021-04-03T03:05:56.879 INFO:teuthology.orchestra.run.gibba028.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2021-04-03T03:05:56.879 INFO:teuthology.orchestra.run.gibba028.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2021-04-03T03:05:56.879 INFO:teuthology.orchestra.run.gibba028.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2021-04-03T03:05:56.880 INFO:teuthology.orchestra.run.gibba028.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2021-04-03T03:05:56.880 INFO:teuthology.orchestra.run.gibba028.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2021-04-03T03:05:56.880 INFO:teuthology.orchestra.run.gibba028.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2021-04-03T03:05:56.880 INFO:teuthology.orchestra.run.gibba028.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-04-03T03:05:56.880 INFO:teuthology.orchestra.run.gibba028.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-04-03T03:05:56.881 INFO:teuthology.orchestra.run.gibba028.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-04-03T03:05:56.881 INFO:teuthology.orchestra.run.gibba028.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-04-03T03:05:56.881 INFO:teuthology.orchestra.run.gibba028.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-04-03T03:05:56.881 INFO:teuthology.orchestra.run.gibba028.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-04-03T03:05:56.881 INFO:teuthology.orchestra.run.gibba028.stdout:Suggested packages: 2021-04-03T03:05:56.882 INFO:teuthology.orchestra.run.gibba028.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-04-03T03:05:56.882 INFO:teuthology.orchestra.run.gibba028.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-04-03T03:05:56.882 INFO:teuthology.orchestra.run.gibba028.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-04-03T03:05:56.882 INFO:teuthology.orchestra.run.gibba028.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-04-03T03:05:56.882 INFO:teuthology.orchestra.run.gibba028.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-04-03T03:05:56.883 INFO:teuthology.orchestra.run.gibba028.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-04-03T03:05:56.883 INFO:teuthology.orchestra.run.gibba028.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-04-03T03:05:56.883 INFO:teuthology.orchestra.run.gibba028.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-04-03T03:05:56.883 INFO:teuthology.orchestra.run.gibba028.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-04-03T03:05:56.884 INFO:teuthology.orchestra.run.gibba028.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-04-03T03:05:56.884 INFO:teuthology.orchestra.run.gibba028.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-04-03T03:05:56.884 INFO:teuthology.orchestra.run.gibba028.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-04-03T03:05:56.884 INFO:teuthology.orchestra.run.gibba028.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-04-03T03:05:56.884 INFO:teuthology.orchestra.run.gibba028.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-04-03T03:05:56.885 INFO:teuthology.orchestra.run.gibba028.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-04-03T03:05:56.894 INFO:teuthology.orchestra.run.gibba015.stdout:The following NEW packages will be installed: 2021-04-03T03:05:56.894 INFO:teuthology.orchestra.run.gibba015.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-04-03T03:05:56.895 INFO:teuthology.orchestra.run.gibba015.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:56.895 INFO:teuthology.orchestra.run.gibba015.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:56.895 INFO:teuthology.orchestra.run.gibba015.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2021-04-03T03:05:56.895 INFO:teuthology.orchestra.run.gibba015.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2021-04-03T03:05:56.895 INFO:teuthology.orchestra.run.gibba015.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2021-04-03T03:05:56.896 INFO:teuthology.orchestra.run.gibba015.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2021-04-03T03:05:56.896 INFO:teuthology.orchestra.run.gibba015.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2021-04-03T03:05:56.896 INFO:teuthology.orchestra.run.gibba015.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2021-04-03T03:05:56.896 INFO:teuthology.orchestra.run.gibba015.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2021-04-03T03:05:56.897 INFO:teuthology.orchestra.run.gibba015.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2021-04-03T03:05:56.897 INFO:teuthology.orchestra.run.gibba015.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-04-03T03:05:56.897 INFO:teuthology.orchestra.run.gibba015.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-04-03T03:05:56.897 INFO:teuthology.orchestra.run.gibba015.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-04-03T03:05:56.897 INFO:teuthology.orchestra.run.gibba015.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-04-03T03:05:56.898 INFO:teuthology.orchestra.run.gibba015.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-04-03T03:05:56.898 INFO:teuthology.orchestra.run.gibba015.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-04-03T03:05:56.898 INFO:teuthology.orchestra.run.gibba015.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-04-03T03:05:56.898 INFO:teuthology.orchestra.run.gibba015.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2021-04-03T03:05:56.898 INFO:teuthology.orchestra.run.gibba015.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-04-03T03:05:56.899 INFO:teuthology.orchestra.run.gibba015.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-04-03T03:05:56.899 INFO:teuthology.orchestra.run.gibba015.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-04-03T03:05:56.899 INFO:teuthology.orchestra.run.gibba015.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-04-03T03:05:56.899 INFO:teuthology.orchestra.run.gibba015.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-04-03T03:05:56.900 INFO:teuthology.orchestra.run.gibba015.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-04-03T03:05:56.900 INFO:teuthology.orchestra.run.gibba015.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-04-03T03:05:56.900 INFO:teuthology.orchestra.run.gibba015.stdout:The following packages will be upgraded: 2021-04-03T03:05:56.900 INFO:teuthology.orchestra.run.gibba015.stdout: librados2 librbd1 2021-04-03T03:05:56.910 INFO:teuthology.orchestra.run.gibba011.stdout:The following NEW packages will be installed: 2021-04-03T03:05:56.910 INFO:teuthology.orchestra.run.gibba011.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-04-03T03:05:56.910 INFO:teuthology.orchestra.run.gibba011.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:56.910 INFO:teuthology.orchestra.run.gibba011.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:56.911 INFO:teuthology.orchestra.run.gibba011.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2021-04-03T03:05:56.911 INFO:teuthology.orchestra.run.gibba011.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2021-04-03T03:05:56.911 INFO:teuthology.orchestra.run.gibba011.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2021-04-03T03:05:56.912 INFO:teuthology.orchestra.run.gibba011.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2021-04-03T03:05:56.912 INFO:teuthology.orchestra.run.gibba011.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2021-04-03T03:05:56.913 INFO:teuthology.orchestra.run.gibba011.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2021-04-03T03:05:56.913 INFO:teuthology.orchestra.run.gibba011.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2021-04-03T03:05:56.913 INFO:teuthology.orchestra.run.gibba011.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2021-04-03T03:05:56.913 INFO:teuthology.orchestra.run.gibba011.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-04-03T03:05:56.914 INFO:teuthology.orchestra.run.gibba011.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-04-03T03:05:56.914 INFO:teuthology.orchestra.run.gibba011.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-04-03T03:05:56.914 INFO:teuthology.orchestra.run.gibba011.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-04-03T03:05:56.914 INFO:teuthology.orchestra.run.gibba011.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-04-03T03:05:56.914 INFO:teuthology.orchestra.run.gibba011.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-04-03T03:05:56.915 INFO:teuthology.orchestra.run.gibba011.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-04-03T03:05:56.915 INFO:teuthology.orchestra.run.gibba011.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2021-04-03T03:05:56.915 INFO:teuthology.orchestra.run.gibba011.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-04-03T03:05:56.915 INFO:teuthology.orchestra.run.gibba011.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-04-03T03:05:56.916 INFO:teuthology.orchestra.run.gibba011.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-04-03T03:05:56.916 INFO:teuthology.orchestra.run.gibba011.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-04-03T03:05:56.916 INFO:teuthology.orchestra.run.gibba011.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-04-03T03:05:56.916 INFO:teuthology.orchestra.run.gibba011.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-04-03T03:05:56.916 INFO:teuthology.orchestra.run.gibba011.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-04-03T03:05:56.917 INFO:teuthology.orchestra.run.gibba011.stdout:The following packages will be upgraded: 2021-04-03T03:05:56.917 INFO:teuthology.orchestra.run.gibba011.stdout: librados2 librbd1 2021-04-03T03:05:56.926 INFO:teuthology.orchestra.run.gibba031.stdout:Building dependency tree... 2021-04-03T03:05:56.926 INFO:teuthology.orchestra.run.gibba031.stdout:Reading state information... 2021-04-03T03:05:56.932 INFO:teuthology.orchestra.run.gibba028.stdout:The following NEW packages will be installed: 2021-04-03T03:05:56.932 INFO:teuthology.orchestra.run.gibba028.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-04-03T03:05:56.932 INFO:teuthology.orchestra.run.gibba028.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:56.933 INFO:teuthology.orchestra.run.gibba028.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:56.933 INFO:teuthology.orchestra.run.gibba028.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2021-04-03T03:05:56.933 INFO:teuthology.orchestra.run.gibba028.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2021-04-03T03:05:56.933 INFO:teuthology.orchestra.run.gibba028.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2021-04-03T03:05:56.933 INFO:teuthology.orchestra.run.gibba028.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2021-04-03T03:05:56.934 INFO:teuthology.orchestra.run.gibba028.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2021-04-03T03:05:56.934 INFO:teuthology.orchestra.run.gibba028.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2021-04-03T03:05:56.934 INFO:teuthology.orchestra.run.gibba028.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2021-04-03T03:05:56.934 INFO:teuthology.orchestra.run.gibba028.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2021-04-03T03:05:56.934 INFO:teuthology.orchestra.run.gibba028.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-04-03T03:05:56.935 INFO:teuthology.orchestra.run.gibba028.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-04-03T03:05:56.935 INFO:teuthology.orchestra.run.gibba028.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-04-03T03:05:56.935 INFO:teuthology.orchestra.run.gibba028.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-04-03T03:05:56.935 INFO:teuthology.orchestra.run.gibba028.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-04-03T03:05:56.936 INFO:teuthology.orchestra.run.gibba028.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-04-03T03:05:56.936 INFO:teuthology.orchestra.run.gibba028.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-04-03T03:05:56.936 INFO:teuthology.orchestra.run.gibba028.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2021-04-03T03:05:56.936 INFO:teuthology.orchestra.run.gibba028.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-04-03T03:05:56.936 INFO:teuthology.orchestra.run.gibba028.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-04-03T03:05:56.937 INFO:teuthology.orchestra.run.gibba028.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-04-03T03:05:56.937 INFO:teuthology.orchestra.run.gibba028.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-04-03T03:05:56.937 INFO:teuthology.orchestra.run.gibba028.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-04-03T03:05:56.937 INFO:teuthology.orchestra.run.gibba028.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-04-03T03:05:56.937 INFO:teuthology.orchestra.run.gibba028.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-04-03T03:05:56.938 INFO:teuthology.orchestra.run.gibba028.stdout:The following packages will be upgraded: 2021-04-03T03:05:56.938 INFO:teuthology.orchestra.run.gibba028.stdout: librados2 librbd1 2021-04-03T03:05:56.942 INFO:teuthology.orchestra.run.gibba026.stdout:The following additional packages will be installed: 2021-04-03T03:05:56.942 INFO:teuthology.orchestra.run.gibba026.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:56.942 INFO:teuthology.orchestra.run.gibba026.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:56.943 INFO:teuthology.orchestra.run.gibba026.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2021-04-03T03:05:56.943 INFO:teuthology.orchestra.run.gibba026.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-04-03T03:05:56.943 INFO:teuthology.orchestra.run.gibba026.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-04-03T03:05:56.943 INFO:teuthology.orchestra.run.gibba026.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-04-03T03:05:56.943 INFO:teuthology.orchestra.run.gibba026.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-04-03T03:05:56.944 INFO:teuthology.orchestra.run.gibba026.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-04-03T03:05:56.944 INFO:teuthology.orchestra.run.gibba026.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-04-03T03:05:56.944 INFO:teuthology.orchestra.run.gibba026.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2021-04-03T03:05:56.944 INFO:teuthology.orchestra.run.gibba026.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2021-04-03T03:05:56.945 INFO:teuthology.orchestra.run.gibba026.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2021-04-03T03:05:56.945 INFO:teuthology.orchestra.run.gibba026.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2021-04-03T03:05:56.945 INFO:teuthology.orchestra.run.gibba026.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2021-04-03T03:05:56.945 INFO:teuthology.orchestra.run.gibba026.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2021-04-03T03:05:56.945 INFO:teuthology.orchestra.run.gibba026.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2021-04-03T03:05:56.946 INFO:teuthology.orchestra.run.gibba026.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2021-04-03T03:05:56.946 INFO:teuthology.orchestra.run.gibba026.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2021-04-03T03:05:56.946 INFO:teuthology.orchestra.run.gibba026.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-04-03T03:05:56.946 INFO:teuthology.orchestra.run.gibba026.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-04-03T03:05:56.946 INFO:teuthology.orchestra.run.gibba026.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-04-03T03:05:56.947 INFO:teuthology.orchestra.run.gibba026.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-04-03T03:05:56.947 INFO:teuthology.orchestra.run.gibba026.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-04-03T03:05:56.947 INFO:teuthology.orchestra.run.gibba026.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-04-03T03:05:56.948 INFO:teuthology.orchestra.run.gibba026.stdout:Suggested packages: 2021-04-03T03:05:56.948 INFO:teuthology.orchestra.run.gibba026.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-04-03T03:05:56.948 INFO:teuthology.orchestra.run.gibba026.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-04-03T03:05:56.948 INFO:teuthology.orchestra.run.gibba026.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-04-03T03:05:56.949 INFO:teuthology.orchestra.run.gibba026.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-04-03T03:05:56.949 INFO:teuthology.orchestra.run.gibba026.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-04-03T03:05:56.949 INFO:teuthology.orchestra.run.gibba026.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-04-03T03:05:56.949 INFO:teuthology.orchestra.run.gibba026.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-04-03T03:05:56.950 INFO:teuthology.orchestra.run.gibba026.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-04-03T03:05:56.950 INFO:teuthology.orchestra.run.gibba026.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-04-03T03:05:56.950 INFO:teuthology.orchestra.run.gibba026.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-04-03T03:05:56.950 INFO:teuthology.orchestra.run.gibba026.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-04-03T03:05:56.950 INFO:teuthology.orchestra.run.gibba026.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-04-03T03:05:56.951 INFO:teuthology.orchestra.run.gibba026.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-04-03T03:05:56.951 INFO:teuthology.orchestra.run.gibba026.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-04-03T03:05:56.951 INFO:teuthology.orchestra.run.gibba026.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-04-03T03:05:56.999 INFO:teuthology.orchestra.run.gibba026.stdout:The following NEW packages will be installed: 2021-04-03T03:05:56.999 INFO:teuthology.orchestra.run.gibba026.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-04-03T03:05:57.000 INFO:teuthology.orchestra.run.gibba026.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:57.000 INFO:teuthology.orchestra.run.gibba026.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:57.000 INFO:teuthology.orchestra.run.gibba026.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2021-04-03T03:05:57.000 INFO:teuthology.orchestra.run.gibba026.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2021-04-03T03:05:57.000 INFO:teuthology.orchestra.run.gibba026.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2021-04-03T03:05:57.001 INFO:teuthology.orchestra.run.gibba026.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2021-04-03T03:05:57.001 INFO:teuthology.orchestra.run.gibba026.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2021-04-03T03:05:57.001 INFO:teuthology.orchestra.run.gibba026.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2021-04-03T03:05:57.001 INFO:teuthology.orchestra.run.gibba026.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2021-04-03T03:05:57.001 INFO:teuthology.orchestra.run.gibba026.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2021-04-03T03:05:57.002 INFO:teuthology.orchestra.run.gibba026.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-04-03T03:05:57.002 INFO:teuthology.orchestra.run.gibba026.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-04-03T03:05:57.002 INFO:teuthology.orchestra.run.gibba026.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-04-03T03:05:57.002 INFO:teuthology.orchestra.run.gibba026.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-04-03T03:05:57.003 INFO:teuthology.orchestra.run.gibba026.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-04-03T03:05:57.003 INFO:teuthology.orchestra.run.gibba026.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-04-03T03:05:57.003 INFO:teuthology.orchestra.run.gibba026.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-04-03T03:05:57.003 INFO:teuthology.orchestra.run.gibba026.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2021-04-03T03:05:57.003 INFO:teuthology.orchestra.run.gibba026.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-04-03T03:05:57.004 INFO:teuthology.orchestra.run.gibba026.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-04-03T03:05:57.004 INFO:teuthology.orchestra.run.gibba026.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-04-03T03:05:57.004 INFO:teuthology.orchestra.run.gibba026.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-04-03T03:05:57.004 INFO:teuthology.orchestra.run.gibba026.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-04-03T03:05:57.005 INFO:teuthology.orchestra.run.gibba026.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-04-03T03:05:57.005 INFO:teuthology.orchestra.run.gibba026.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-04-03T03:05:57.005 INFO:teuthology.orchestra.run.gibba026.stdout:The following packages will be upgraded: 2021-04-03T03:05:57.006 INFO:teuthology.orchestra.run.gibba026.stdout: librados2 librbd1 2021-04-03T03:05:57.018 INFO:teuthology.orchestra.run.gibba015.stdout:2 upgraded, 115 newly installed, 0 to remove and 128 not upgraded. 2021-04-03T03:05:57.018 INFO:teuthology.orchestra.run.gibba015.stdout:Need to get 164 MB of archives. 2021-04-03T03:05:57.018 INFO:teuthology.orchestra.run.gibba015.stdout:After this operation, 775 MB of additional disk space will be used. 2021-04-03T03:05:57.019 INFO:teuthology.orchestra.run.gibba015.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.10-137-g5feb67b5-1bionic [1,703 kB] 2021-04-03T03:05:57.027 INFO:teuthology.orchestra.run.gibba031.stdout:The following additional packages will be installed: 2021-04-03T03:05:57.028 INFO:teuthology.orchestra.run.gibba031.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:57.028 INFO:teuthology.orchestra.run.gibba031.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:57.030 INFO:teuthology.orchestra.run.gibba031.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2021-04-03T03:05:57.030 INFO:teuthology.orchestra.run.gibba031.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-04-03T03:05:57.030 INFO:teuthology.orchestra.run.gibba031.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-04-03T03:05:57.031 INFO:teuthology.orchestra.run.gibba031.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-04-03T03:05:57.031 INFO:teuthology.orchestra.run.gibba031.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-04-03T03:05:57.031 INFO:teuthology.orchestra.run.gibba031.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-04-03T03:05:57.031 INFO:teuthology.orchestra.run.gibba031.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-04-03T03:05:57.032 INFO:teuthology.orchestra.run.gibba031.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2021-04-03T03:05:57.032 INFO:teuthology.orchestra.run.gibba031.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2021-04-03T03:05:57.032 INFO:teuthology.orchestra.run.gibba031.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2021-04-03T03:05:57.032 INFO:teuthology.orchestra.run.gibba031.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2021-04-03T03:05:57.032 INFO:teuthology.orchestra.run.gibba031.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2021-04-03T03:05:57.033 INFO:teuthology.orchestra.run.gibba031.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2021-04-03T03:05:57.033 INFO:teuthology.orchestra.run.gibba031.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2021-04-03T03:05:57.033 INFO:teuthology.orchestra.run.gibba031.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2021-04-03T03:05:57.033 INFO:teuthology.orchestra.run.gibba031.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2021-04-03T03:05:57.033 INFO:teuthology.orchestra.run.gibba031.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-04-03T03:05:57.034 INFO:teuthology.orchestra.run.gibba031.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-04-03T03:05:57.034 INFO:teuthology.orchestra.run.gibba031.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-04-03T03:05:57.034 INFO:teuthology.orchestra.run.gibba031.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-04-03T03:05:57.034 INFO:teuthology.orchestra.run.gibba031.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-04-03T03:05:57.035 INFO:teuthology.orchestra.run.gibba031.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-04-03T03:05:57.035 INFO:teuthology.orchestra.run.gibba031.stdout:Suggested packages: 2021-04-03T03:05:57.035 INFO:teuthology.orchestra.run.gibba031.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-04-03T03:05:57.035 INFO:teuthology.orchestra.run.gibba031.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-04-03T03:05:57.035 INFO:teuthology.orchestra.run.gibba031.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-04-03T03:05:57.036 INFO:teuthology.orchestra.run.gibba031.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-04-03T03:05:57.036 INFO:teuthology.orchestra.run.gibba031.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-04-03T03:05:57.036 INFO:teuthology.orchestra.run.gibba031.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-04-03T03:05:57.036 INFO:teuthology.orchestra.run.gibba031.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-04-03T03:05:57.037 INFO:teuthology.orchestra.run.gibba031.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-04-03T03:05:57.037 INFO:teuthology.orchestra.run.gibba031.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-04-03T03:05:57.037 INFO:teuthology.orchestra.run.gibba031.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-04-03T03:05:57.037 INFO:teuthology.orchestra.run.gibba031.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-04-03T03:05:57.037 INFO:teuthology.orchestra.run.gibba031.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-04-03T03:05:57.038 INFO:teuthology.orchestra.run.gibba031.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-04-03T03:05:57.038 INFO:teuthology.orchestra.run.gibba031.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-04-03T03:05:57.038 INFO:teuthology.orchestra.run.gibba031.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-04-03T03:05:57.038 INFO:teuthology.orchestra.run.gibba011.stdout:2 upgraded, 115 newly installed, 0 to remove and 128 not upgraded. 2021-04-03T03:05:57.039 INFO:teuthology.orchestra.run.gibba011.stdout:Need to get 164 MB of archives. 2021-04-03T03:05:57.039 INFO:teuthology.orchestra.run.gibba011.stdout:After this operation, 775 MB of additional disk space will be used. 2021-04-03T03:05:57.039 INFO:teuthology.orchestra.run.gibba011.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.10-137-g5feb67b5-1bionic [1,703 kB] 2021-04-03T03:05:57.056 INFO:teuthology.orchestra.run.gibba028.stdout:2 upgraded, 115 newly installed, 0 to remove and 128 not upgraded. 2021-04-03T03:05:57.056 INFO:teuthology.orchestra.run.gibba028.stdout:Need to get 164 MB of archives. 2021-04-03T03:05:57.057 INFO:teuthology.orchestra.run.gibba028.stdout:After this operation, 775 MB of additional disk space will be used. 2021-04-03T03:05:57.057 INFO:teuthology.orchestra.run.gibba028.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.10-137-g5feb67b5-1bionic [1,703 kB] 2021-04-03T03:05:57.066 INFO:teuthology.orchestra.run.gibba015.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-04-03T03:05:57.083 INFO:teuthology.orchestra.run.gibba011.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-04-03T03:05:57.084 INFO:teuthology.orchestra.run.gibba031.stdout:The following NEW packages will be installed: 2021-04-03T03:05:57.084 INFO:teuthology.orchestra.run.gibba031.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-04-03T03:05:57.085 INFO:teuthology.orchestra.run.gibba031.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-04-03T03:05:57.085 INFO:teuthology.orchestra.run.gibba031.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-04-03T03:05:57.085 INFO:teuthology.orchestra.run.gibba031.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2021-04-03T03:05:57.087 INFO:teuthology.orchestra.run.gibba031.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2021-04-03T03:05:57.087 INFO:teuthology.orchestra.run.gibba031.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2021-04-03T03:05:57.088 INFO:teuthology.orchestra.run.gibba031.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2021-04-03T03:05:57.088 INFO:teuthology.orchestra.run.gibba031.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2021-04-03T03:05:57.088 INFO:teuthology.orchestra.run.gibba031.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2021-04-03T03:05:57.088 INFO:teuthology.orchestra.run.gibba031.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2021-04-03T03:05:57.089 INFO:teuthology.orchestra.run.gibba031.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2021-04-03T03:05:57.089 INFO:teuthology.orchestra.run.gibba031.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-04-03T03:05:57.089 INFO:teuthology.orchestra.run.gibba031.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-04-03T03:05:57.090 INFO:teuthology.orchestra.run.gibba031.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-04-03T03:05:57.091 INFO:teuthology.orchestra.run.gibba031.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-04-03T03:05:57.091 INFO:teuthology.orchestra.run.gibba031.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-04-03T03:05:57.091 INFO:teuthology.orchestra.run.gibba031.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-04-03T03:05:57.092 INFO:teuthology.orchestra.run.gibba031.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-04-03T03:05:57.092 INFO:teuthology.orchestra.run.gibba031.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2021-04-03T03:05:57.093 INFO:teuthology.orchestra.run.gibba031.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-04-03T03:05:57.093 INFO:teuthology.orchestra.run.gibba031.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-04-03T03:05:57.093 INFO:teuthology.orchestra.run.gibba031.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-04-03T03:05:57.094 INFO:teuthology.orchestra.run.gibba031.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-04-03T03:05:57.094 INFO:teuthology.orchestra.run.gibba031.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-04-03T03:05:57.095 INFO:teuthology.orchestra.run.gibba031.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-04-03T03:05:57.095 INFO:teuthology.orchestra.run.gibba031.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-04-03T03:05:57.095 INFO:teuthology.orchestra.run.gibba031.stdout:The following packages will be upgraded: 2021-04-03T03:05:57.095 INFO:teuthology.orchestra.run.gibba031.stdout: librados2 librbd1 2021-04-03T03:05:57.104 INFO:teuthology.orchestra.run.gibba028.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-04-03T03:05:57.124 INFO:teuthology.orchestra.run.gibba026.stdout:2 upgraded, 115 newly installed, 0 to remove and 128 not upgraded. 2021-04-03T03:05:57.124 INFO:teuthology.orchestra.run.gibba026.stdout:Need to get 164 MB of archives. 2021-04-03T03:05:57.125 INFO:teuthology.orchestra.run.gibba026.stdout:After this operation, 775 MB of additional disk space will be used. 2021-04-03T03:05:57.125 INFO:teuthology.orchestra.run.gibba026.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.10-137-g5feb67b5-1bionic [1,703 kB] 2021-04-03T03:05:57.172 INFO:teuthology.orchestra.run.gibba026.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-04-03T03:05:57.258 INFO:teuthology.orchestra.run.gibba031.stdout:2 upgraded, 115 newly installed, 0 to remove and 128 not upgraded. 2021-04-03T03:05:57.258 INFO:teuthology.orchestra.run.gibba031.stdout:Need to get 164 MB of archives. 2021-04-03T03:05:57.258 INFO:teuthology.orchestra.run.gibba031.stdout:After this operation, 775 MB of additional disk space will be used. 2021-04-03T03:05:57.258 INFO:teuthology.orchestra.run.gibba031.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-04-03T03:05:57.260 INFO:teuthology.orchestra.run.gibba015.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.10-137-g5feb67b5-1bionic [3,624 kB] 2021-04-03T03:05:57.281 INFO:teuthology.orchestra.run.gibba031.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.10-137-g5feb67b5-1bionic [1,703 kB] 2021-04-03T03:05:57.282 INFO:teuthology.orchestra.run.gibba011.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.10-137-g5feb67b5-1bionic [3,624 kB] 2021-04-03T03:05:57.300 INFO:teuthology.orchestra.run.gibba028.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.10-137-g5feb67b5-1bionic [3,624 kB] 2021-04-03T03:05:57.320 INFO:teuthology.orchestra.run.gibba015.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2021-04-03T03:05:57.327 INFO:teuthology.orchestra.run.gibba015.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2021-04-03T03:05:57.336 INFO:teuthology.orchestra.run.gibba015.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-04-03T03:05:57.336 INFO:teuthology.orchestra.run.gibba015.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-04-03T03:05:57.342 INFO:teuthology.orchestra.run.gibba011.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2021-04-03T03:05:57.344 INFO:teuthology.orchestra.run.gibba028.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2021-04-03T03:05:57.351 INFO:teuthology.orchestra.run.gibba011.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2021-04-03T03:05:57.351 INFO:teuthology.orchestra.run.gibba028.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2021-04-03T03:05:57.361 INFO:teuthology.orchestra.run.gibba011.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-04-03T03:05:57.362 INFO:teuthology.orchestra.run.gibba011.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-04-03T03:05:57.379 INFO:teuthology.orchestra.run.gibba028.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-04-03T03:05:57.380 INFO:teuthology.orchestra.run.gibba028.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-04-03T03:05:57.412 INFO:teuthology.orchestra.run.gibba026.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2021-04-03T03:05:57.421 INFO:teuthology.orchestra.run.gibba026.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2021-04-03T03:05:57.432 INFO:teuthology.orchestra.run.gibba026.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-04-03T03:05:57.432 INFO:teuthology.orchestra.run.gibba026.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-04-03T03:05:57.484 INFO:teuthology.orchestra.run.gibba015.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-04-03T03:05:57.492 INFO:teuthology.orchestra.run.gibba015.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-04-03T03:05:57.498 INFO:teuthology.orchestra.run.gibba031.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2021-04-03T03:05:57.505 INFO:teuthology.orchestra.run.gibba031.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2021-04-03T03:05:57.505 INFO:teuthology.orchestra.run.gibba015.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-04-03T03:05:57.514 INFO:teuthology.orchestra.run.gibba028.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-04-03T03:05:57.527 INFO:teuthology.orchestra.run.gibba028.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-04-03T03:05:57.533 INFO:teuthology.orchestra.run.gibba031.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-04-03T03:05:57.533 INFO:teuthology.orchestra.run.gibba031.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-04-03T03:05:57.538 INFO:teuthology.orchestra.run.gibba028.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-04-03T03:05:57.544 INFO:teuthology.orchestra.run.gibba011.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-04-03T03:05:57.553 INFO:teuthology.orchestra.run.gibba011.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-04-03T03:05:57.568 INFO:teuthology.orchestra.run.gibba011.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-04-03T03:05:57.577 INFO:teuthology.orchestra.run.gibba026.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-04-03T03:05:57.583 INFO:teuthology.orchestra.run.gibba015.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-04-03T03:05:57.586 INFO:teuthology.orchestra.run.gibba015.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-04-03T03:05:57.588 INFO:teuthology.orchestra.run.gibba026.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-04-03T03:05:57.601 INFO:teuthology.orchestra.run.gibba026.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-04-03T03:05:57.608 INFO:teuthology.orchestra.run.gibba015.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-04-03T03:05:57.609 INFO:teuthology.orchestra.run.gibba015.stdout:Get:14 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.10-137-g5feb67b5-1bionic [471 kB] 2021-04-03T03:05:57.617 INFO:teuthology.orchestra.run.gibba015.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-04-03T03:05:57.618 INFO:teuthology.orchestra.run.gibba028.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-04-03T03:05:57.621 INFO:teuthology.orchestra.run.gibba028.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-04-03T03:05:57.624 INFO:teuthology.orchestra.run.gibba015.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-04-03T03:05:57.630 INFO:teuthology.orchestra.run.gibba015.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-04-03T03:05:57.635 INFO:teuthology.orchestra.run.gibba015.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-04-03T03:05:57.637 INFO:teuthology.orchestra.run.gibba011.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.10-137-g5feb67b5-1bionic [471 kB] 2021-04-03T03:05:57.645 INFO:teuthology.orchestra.run.gibba028.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-04-03T03:05:57.651 INFO:teuthology.orchestra.run.gibba028.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-04-03T03:05:57.660 INFO:teuthology.orchestra.run.gibba028.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-04-03T03:05:57.661 INFO:teuthology.orchestra.run.gibba028.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.10-137-g5feb67b5-1bionic [471 kB] 2021-04-03T03:05:57.662 INFO:teuthology.orchestra.run.gibba015.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-04-03T03:05:57.665 INFO:teuthology.orchestra.run.gibba015.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-04-03T03:05:57.666 INFO:teuthology.orchestra.run.gibba011.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-04-03T03:05:57.668 INFO:teuthology.orchestra.run.gibba011.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-04-03T03:05:57.669 INFO:teuthology.orchestra.run.gibba031.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-04-03T03:05:57.671 INFO:teuthology.orchestra.run.gibba028.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-04-03T03:05:57.677 INFO:teuthology.orchestra.run.gibba015.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.10-137-g5feb67b5-1bionic [397 kB] 2021-04-03T03:05:57.678 INFO:teuthology.orchestra.run.gibba026.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-04-03T03:05:57.678 INFO:teuthology.orchestra.run.gibba028.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-04-03T03:05:57.681 INFO:teuthology.orchestra.run.gibba026.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-04-03T03:05:57.681 INFO:teuthology.orchestra.run.gibba031.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-04-03T03:05:57.690 INFO:teuthology.orchestra.run.gibba011.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-04-03T03:05:57.694 INFO:teuthology.orchestra.run.gibba031.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-04-03T03:05:57.697 INFO:teuthology.orchestra.run.gibba011.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-04-03T03:05:57.699 INFO:teuthology.orchestra.run.gibba028.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-04-03T03:05:57.703 INFO:teuthology.orchestra.run.gibba028.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-04-03T03:05:57.708 INFO:teuthology.orchestra.run.gibba026.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-04-03T03:05:57.708 INFO:teuthology.orchestra.run.gibba011.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.10-137-g5feb67b5-1bionic [397 kB] 2021-04-03T03:05:57.710 INFO:teuthology.orchestra.run.gibba011.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-04-03T03:05:57.715 INFO:teuthology.orchestra.run.gibba026.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-04-03T03:05:57.716 INFO:teuthology.orchestra.run.gibba011.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-04-03T03:05:57.723 INFO:teuthology.orchestra.run.gibba026.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-04-03T03:05:57.724 INFO:teuthology.orchestra.run.gibba011.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-04-03T03:05:57.726 INFO:teuthology.orchestra.run.gibba015.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-04-03T03:05:57.732 INFO:teuthology.orchestra.run.gibba026.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-04-03T03:05:57.733 INFO:teuthology.orchestra.run.gibba015.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-04-03T03:05:57.734 INFO:teuthology.orchestra.run.gibba015.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-04-03T03:05:57.735 INFO:teuthology.orchestra.run.gibba028.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.10-137-g5feb67b5-1bionic [397 kB] 2021-04-03T03:05:57.736 INFO:teuthology.orchestra.run.gibba015.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-04-03T03:05:57.736 INFO:teuthology.orchestra.run.gibba015.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-04-03T03:05:57.737 INFO:teuthology.orchestra.run.gibba015.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-04-03T03:05:57.737 INFO:teuthology.orchestra.run.gibba015.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-04-03T03:05:57.740 INFO:teuthology.orchestra.run.gibba026.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-04-03T03:05:57.745 INFO:teuthology.orchestra.run.gibba015.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.10-137-g5feb67b5-1bionic [20.7 kB] 2021-04-03T03:05:57.746 INFO:teuthology.orchestra.run.gibba015.stdout:Get:30 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.10-137-g5feb67b5-1bionic [150 kB] 2021-04-03T03:05:57.750 INFO:teuthology.orchestra.run.gibba015.stdout:Get:31 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.10-137-g5feb67b5-1bionic [29.1 kB] 2021-04-03T03:05:57.751 INFO:teuthology.orchestra.run.gibba011.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-04-03T03:05:57.752 INFO:teuthology.orchestra.run.gibba015.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.10-137-g5feb67b5-1bionic [374 kB] 2021-04-03T03:05:57.755 INFO:teuthology.orchestra.run.gibba011.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-04-03T03:05:57.758 INFO:teuthology.orchestra.run.gibba026.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-04-03T03:05:57.759 INFO:teuthology.orchestra.run.gibba028.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-04-03T03:05:57.762 INFO:teuthology.orchestra.run.gibba026.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-04-03T03:05:57.768 INFO:teuthology.orchestra.run.gibba028.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-04-03T03:05:57.769 INFO:teuthology.orchestra.run.gibba028.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-04-03T03:05:57.770 INFO:teuthology.orchestra.run.gibba028.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-04-03T03:05:57.771 INFO:teuthology.orchestra.run.gibba028.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-04-03T03:05:57.772 INFO:teuthology.orchestra.run.gibba031.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-04-03T03:05:57.774 INFO:teuthology.orchestra.run.gibba028.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-04-03T03:05:57.774 INFO:teuthology.orchestra.run.gibba028.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-04-03T03:05:57.775 INFO:teuthology.orchestra.run.gibba031.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-04-03T03:05:57.776 INFO:teuthology.orchestra.run.gibba011.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.10-137-g5feb67b5-1bionic [20.7 kB] 2021-04-03T03:05:57.777 INFO:teuthology.orchestra.run.gibba011.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.10-137-g5feb67b5-1bionic [150 kB] 2021-04-03T03:05:57.789 INFO:teuthology.orchestra.run.gibba011.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-04-03T03:05:57.798 INFO:teuthology.orchestra.run.gibba011.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-04-03T03:05:57.798 INFO:teuthology.orchestra.run.gibba031.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-04-03T03:05:57.799 INFO:teuthology.orchestra.run.gibba011.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-04-03T03:05:57.800 INFO:teuthology.orchestra.run.gibba011.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-04-03T03:05:57.805 INFO:teuthology.orchestra.run.gibba031.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-04-03T03:05:57.806 INFO:teuthology.orchestra.run.gibba015.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-04-03T03:05:57.808 INFO:teuthology.orchestra.run.gibba015.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-04-03T03:05:57.811 INFO:teuthology.orchestra.run.gibba015.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-04-03T03:05:57.813 INFO:teuthology.orchestra.run.gibba031.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-04-03T03:05:57.819 INFO:teuthology.orchestra.run.gibba026.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-04-03T03:05:57.823 INFO:teuthology.orchestra.run.gibba031.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-04-03T03:05:57.828 INFO:teuthology.orchestra.run.gibba026.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-04-03T03:05:57.829 INFO:teuthology.orchestra.run.gibba026.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-04-03T03:05:57.830 INFO:teuthology.orchestra.run.gibba026.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-04-03T03:05:57.830 INFO:teuthology.orchestra.run.gibba031.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-04-03T03:05:57.831 INFO:teuthology.orchestra.run.gibba026.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-04-03T03:05:57.834 INFO:teuthology.orchestra.run.gibba026.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-04-03T03:05:57.839 INFO:teuthology.orchestra.run.gibba028.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-04-03T03:05:57.842 INFO:teuthology.orchestra.run.gibba028.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-04-03T03:05:57.845 INFO:teuthology.orchestra.run.gibba028.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-04-03T03:05:57.853 INFO:teuthology.orchestra.run.gibba031.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-04-03T03:05:57.856 INFO:teuthology.orchestra.run.gibba031.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-04-03T03:05:57.869 INFO:teuthology.orchestra.run.gibba011.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-04-03T03:05:57.871 INFO:teuthology.orchestra.run.gibba011.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-04-03T03:05:57.872 INFO:teuthology.orchestra.run.gibba011.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-04-03T03:05:57.872 INFO:teuthology.orchestra.run.gibba011.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-04-03T03:05:57.874 INFO:teuthology.orchestra.run.gibba011.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.10-137-g5feb67b5-1bionic [29.1 kB] 2021-04-03T03:05:57.875 INFO:teuthology.orchestra.run.gibba011.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-04-03T03:05:57.876 INFO:teuthology.orchestra.run.gibba011.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.10-137-g5feb67b5-1bionic [374 kB] 2021-04-03T03:05:57.878 INFO:teuthology.orchestra.run.gibba011.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-04-03T03:05:57.880 INFO:teuthology.orchestra.run.gibba011.stdout:Get:36 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.10-137-g5feb67b5-1bionic [3,163 kB] 2021-04-03T03:05:57.886 INFO:teuthology.orchestra.run.gibba015.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-04-03T03:05:57.887 INFO:teuthology.orchestra.run.gibba015.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-04-03T03:05:57.892 INFO:teuthology.orchestra.run.gibba028.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.10-137-g5feb67b5-1bionic [20.7 kB] 2021-04-03T03:05:57.893 INFO:teuthology.orchestra.run.gibba028.stdout:Get:33 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.10-137-g5feb67b5-1bionic [150 kB] 2021-04-03T03:05:57.893 INFO:teuthology.orchestra.run.gibba028.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.10-137-g5feb67b5-1bionic [29.1 kB] 2021-04-03T03:05:57.894 INFO:teuthology.orchestra.run.gibba028.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.10-137-g5feb67b5-1bionic [374 kB] 2021-04-03T03:05:57.896 INFO:teuthology.orchestra.run.gibba028.stdout:Get:36 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.10-137-g5feb67b5-1bionic [3,163 kB] 2021-04-03T03:05:57.899 INFO:teuthology.orchestra.run.gibba026.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-04-03T03:05:57.900 INFO:teuthology.orchestra.run.gibba015.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.10-137-g5feb67b5-1bionic [3,163 kB] 2021-04-03T03:05:57.900 INFO:teuthology.orchestra.run.gibba026.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-04-03T03:05:57.902 INFO:teuthology.orchestra.run.gibba026.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-04-03T03:05:57.905 INFO:teuthology.orchestra.run.gibba026.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-04-03T03:05:57.905 INFO:teuthology.orchestra.run.gibba015.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-04-03T03:05:57.906 INFO:teuthology.orchestra.run.gibba015.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-04-03T03:05:57.906 INFO:teuthology.orchestra.run.gibba015.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-04-03T03:05:57.907 INFO:teuthology.orchestra.run.gibba015.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-04-03T03:05:57.911 INFO:teuthology.orchestra.run.gibba015.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-04-03T03:05:57.920 INFO:teuthology.orchestra.run.gibba028.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-04-03T03:05:57.921 INFO:teuthology.orchestra.run.gibba028.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-04-03T03:05:57.941 INFO:teuthology.orchestra.run.gibba028.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-04-03T03:05:57.941 INFO:teuthology.orchestra.run.gibba028.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-04-03T03:05:57.942 INFO:teuthology.orchestra.run.gibba028.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-04-03T03:05:57.944 INFO:teuthology.orchestra.run.gibba028.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-04-03T03:05:57.946 INFO:teuthology.orchestra.run.gibba028.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-04-03T03:05:57.948 INFO:teuthology.orchestra.run.gibba031.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-04-03T03:05:57.949 INFO:teuthology.orchestra.run.gibba011.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-04-03T03:05:57.950 INFO:teuthology.orchestra.run.gibba011.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-04-03T03:05:57.956 INFO:teuthology.orchestra.run.gibba031.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-04-03T03:05:57.956 INFO:teuthology.orchestra.run.gibba031.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-04-03T03:05:57.957 INFO:teuthology.orchestra.run.gibba031.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-04-03T03:05:57.958 INFO:teuthology.orchestra.run.gibba031.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-04-03T03:05:57.960 INFO:teuthology.orchestra.run.gibba031.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-04-03T03:05:57.965 INFO:teuthology.orchestra.run.gibba015.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-04-03T03:05:57.969 INFO:teuthology.orchestra.run.gibba015.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-04-03T03:05:57.969 INFO:teuthology.orchestra.run.gibba015.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-04-03T03:05:57.971 INFO:teuthology.orchestra.run.gibba011.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-04-03T03:05:57.972 INFO:teuthology.orchestra.run.gibba011.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-04-03T03:05:57.979 INFO:teuthology.orchestra.run.gibba026.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-04-03T03:05:57.979 INFO:teuthology.orchestra.run.gibba026.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-04-03T03:05:58.000 INFO:teuthology.orchestra.run.gibba028.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-04-03T03:05:58.000 INFO:teuthology.orchestra.run.gibba026.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-04-03T03:05:58.001 INFO:teuthology.orchestra.run.gibba026.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-04-03T03:05:58.001 INFO:teuthology.orchestra.run.gibba026.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-04-03T03:05:58.002 INFO:teuthology.orchestra.run.gibba026.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-04-03T03:05:58.005 INFO:teuthology.orchestra.run.gibba028.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-04-03T03:05:58.006 INFO:teuthology.orchestra.run.gibba028.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-04-03T03:05:58.028 INFO:teuthology.orchestra.run.gibba031.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-04-03T03:05:58.029 INFO:teuthology.orchestra.run.gibba031.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-04-03T03:05:58.029 INFO:teuthology.orchestra.run.gibba011.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-04-03T03:05:58.030 INFO:teuthology.orchestra.run.gibba011.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-04-03T03:05:58.031 INFO:teuthology.orchestra.run.gibba031.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-04-03T03:05:58.033 INFO:teuthology.orchestra.run.gibba011.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-04-03T03:05:58.034 INFO:teuthology.orchestra.run.gibba031.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-04-03T03:05:58.037 INFO:teuthology.orchestra.run.gibba026.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.10-137-g5feb67b5-1bionic [3,624 kB] 2021-04-03T03:05:58.042 INFO:teuthology.orchestra.run.gibba011.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-04-03T03:05:58.045 INFO:teuthology.orchestra.run.gibba015.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-04-03T03:05:58.046 INFO:teuthology.orchestra.run.gibba011.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-04-03T03:05:58.047 INFO:teuthology.orchestra.run.gibba011.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-04-03T03:05:58.058 INFO:teuthology.orchestra.run.gibba026.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-04-03T03:05:58.066 INFO:teuthology.orchestra.run.gibba026.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-04-03T03:05:58.070 INFO:teuthology.orchestra.run.gibba026.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-04-03T03:05:58.070 INFO:teuthology.orchestra.run.gibba026.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-04-03T03:05:58.080 INFO:teuthology.orchestra.run.gibba028.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-04-03T03:05:58.096 INFO:teuthology.orchestra.run.gibba015.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-04-03T03:05:58.096 INFO:teuthology.orchestra.run.gibba015.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-04-03T03:05:58.108 INFO:teuthology.orchestra.run.gibba031.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-04-03T03:05:58.108 INFO:teuthology.orchestra.run.gibba031.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-04-03T03:05:58.109 INFO:teuthology.orchestra.run.gibba011.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-04-03T03:05:58.118 INFO:teuthology.orchestra.run.gibba028.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-04-03T03:05:58.118 INFO:teuthology.orchestra.run.gibba028.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-04-03T03:05:58.129 INFO:teuthology.orchestra.run.gibba031.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-04-03T03:05:58.129 INFO:teuthology.orchestra.run.gibba031.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-04-03T03:05:58.129 INFO:teuthology.orchestra.run.gibba031.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-04-03T03:05:58.131 INFO:teuthology.orchestra.run.gibba031.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-04-03T03:05:58.138 INFO:teuthology.orchestra.run.gibba026.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-04-03T03:05:58.143 INFO:teuthology.orchestra.run.gibba015.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.10-137-g5feb67b5-1bionic [107 kB] 2021-04-03T03:05:58.146 INFO:teuthology.orchestra.run.gibba015.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.10-137-g5feb67b5-1bionic [372 kB] 2021-04-03T03:05:58.173 INFO:teuthology.orchestra.run.gibba011.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.10-137-g5feb67b5-1bionic [107 kB] 2021-04-03T03:05:58.177 INFO:teuthology.orchestra.run.gibba011.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.10-137-g5feb67b5-1bionic [372 kB] 2021-04-03T03:05:58.188 INFO:teuthology.orchestra.run.gibba031.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-04-03T03:05:58.195 INFO:teuthology.orchestra.run.gibba031.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-04-03T03:05:58.199 INFO:teuthology.orchestra.run.gibba028.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.10-137-g5feb67b5-1bionic [107 kB] 2021-04-03T03:05:58.200 INFO:teuthology.orchestra.run.gibba011.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-04-03T03:05:58.200 INFO:teuthology.orchestra.run.gibba011.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-04-03T03:05:58.201 INFO:teuthology.orchestra.run.gibba031.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-04-03T03:05:58.201 INFO:teuthology.orchestra.run.gibba031.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-04-03T03:05:58.203 INFO:teuthology.orchestra.run.gibba028.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.10-137-g5feb67b5-1bionic [372 kB] 2021-04-03T03:05:58.204 INFO:teuthology.orchestra.run.gibba015.stdout:Get:52 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.10-137-g5feb67b5-1bionic [20.8 MB] 2021-04-03T03:05:58.222 INFO:teuthology.orchestra.run.gibba026.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-04-03T03:05:58.223 INFO:teuthology.orchestra.run.gibba026.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-04-03T03:05:58.236 INFO:teuthology.orchestra.run.gibba011.stdout:Get:52 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.10-137-g5feb67b5-1bionic [20.8 MB] 2021-04-03T03:05:58.261 INFO:teuthology.orchestra.run.gibba028.stdout:Get:52 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.10-137-g5feb67b5-1bionic [20.8 MB] 2021-04-03T03:05:58.268 INFO:teuthology.orchestra.run.gibba031.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-04-03T03:05:58.351 INFO:teuthology.orchestra.run.gibba031.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-04-03T03:05:58.352 INFO:teuthology.orchestra.run.gibba031.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-04-03T03:05:58.412 INFO:teuthology.orchestra.run.gibba015.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-04-03T03:05:58.431 INFO:teuthology.orchestra.run.gibba028.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-04-03T03:05:58.464 INFO:teuthology.orchestra.run.gibba015.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-04-03T03:05:58.465 INFO:teuthology.orchestra.run.gibba015.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-04-03T03:05:58.481 INFO:teuthology.orchestra.run.gibba015.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-04-03T03:05:58.481 INFO:teuthology.orchestra.run.gibba015.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-04-03T03:05:58.482 INFO:teuthology.orchestra.run.gibba015.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-04-03T03:05:58.482 INFO:teuthology.orchestra.run.gibba015.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-04-03T03:05:58.483 INFO:teuthology.orchestra.run.gibba015.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-04-03T03:05:58.483 INFO:teuthology.orchestra.run.gibba011.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-04-03T03:05:58.484 INFO:teuthology.orchestra.run.gibba015.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-04-03T03:05:58.484 INFO:teuthology.orchestra.run.gibba028.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-04-03T03:05:58.487 INFO:teuthology.orchestra.run.gibba028.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-04-03T03:05:58.493 INFO:teuthology.orchestra.run.gibba015.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-04-03T03:05:58.507 INFO:teuthology.orchestra.run.gibba026.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-04-03T03:05:58.509 INFO:teuthology.orchestra.run.gibba028.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-04-03T03:05:58.509 INFO:teuthology.orchestra.run.gibba028.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-04-03T03:05:58.510 INFO:teuthology.orchestra.run.gibba028.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-04-03T03:05:58.510 INFO:teuthology.orchestra.run.gibba028.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-04-03T03:05:58.511 INFO:teuthology.orchestra.run.gibba028.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-04-03T03:05:58.512 INFO:teuthology.orchestra.run.gibba028.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-04-03T03:05:58.512 INFO:teuthology.orchestra.run.gibba028.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-04-03T03:05:58.517 INFO:teuthology.orchestra.run.gibba011.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-04-03T03:05:58.518 INFO:teuthology.orchestra.run.gibba011.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-04-03T03:05:58.540 INFO:teuthology.orchestra.run.gibba026.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-04-03T03:05:58.542 INFO:teuthology.orchestra.run.gibba026.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-04-03T03:05:58.556 INFO:teuthology.orchestra.run.gibba011.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-04-03T03:05:58.556 INFO:teuthology.orchestra.run.gibba011.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-04-03T03:05:58.556 INFO:teuthology.orchestra.run.gibba011.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-04-03T03:05:58.557 INFO:teuthology.orchestra.run.gibba011.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-04-03T03:05:58.557 INFO:teuthology.orchestra.run.gibba011.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-04-03T03:05:58.558 INFO:teuthology.orchestra.run.gibba011.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-04-03T03:05:58.565 INFO:teuthology.orchestra.run.gibba011.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-04-03T03:05:58.574 INFO:teuthology.orchestra.run.gibba015.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-04-03T03:05:58.576 INFO:teuthology.orchestra.run.gibba015.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-04-03T03:05:58.577 INFO:teuthology.orchestra.run.gibba026.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-04-03T03:05:58.578 INFO:teuthology.orchestra.run.gibba026.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-04-03T03:05:58.578 INFO:teuthology.orchestra.run.gibba026.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-04-03T03:05:58.580 INFO:teuthology.orchestra.run.gibba026.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-04-03T03:05:58.581 INFO:teuthology.orchestra.run.gibba026.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-04-03T03:05:58.581 INFO:teuthology.orchestra.run.gibba026.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-04-03T03:05:58.586 INFO:teuthology.orchestra.run.gibba015.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-04-03T03:05:58.587 INFO:teuthology.orchestra.run.gibba015.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-04-03T03:05:58.588 INFO:teuthology.orchestra.run.gibba026.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-04-03T03:05:58.592 INFO:teuthology.orchestra.run.gibba028.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-04-03T03:05:58.594 INFO:teuthology.orchestra.run.gibba015.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-04-03T03:05:58.594 INFO:teuthology.orchestra.run.gibba028.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-04-03T03:05:58.595 INFO:teuthology.orchestra.run.gibba015.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-04-03T03:05:58.595 INFO:teuthology.orchestra.run.gibba015.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-04-03T03:05:58.597 INFO:teuthology.orchestra.run.gibba028.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-04-03T03:05:58.598 INFO:teuthology.orchestra.run.gibba028.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-04-03T03:05:58.599 INFO:teuthology.orchestra.run.gibba028.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-04-03T03:05:58.599 INFO:teuthology.orchestra.run.gibba028.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-04-03T03:05:58.600 INFO:teuthology.orchestra.run.gibba015.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-04-03T03:05:58.629 INFO:teuthology.orchestra.run.gibba031.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-04-03T03:05:58.638 INFO:teuthology.orchestra.run.gibba011.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-04-03T03:05:58.641 INFO:teuthology.orchestra.run.gibba011.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-04-03T03:05:58.649 INFO:teuthology.orchestra.run.gibba011.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-04-03T03:05:58.650 INFO:teuthology.orchestra.run.gibba011.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-04-03T03:05:58.651 INFO:teuthology.orchestra.run.gibba011.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-04-03T03:05:58.651 INFO:teuthology.orchestra.run.gibba011.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-04-03T03:05:58.651 INFO:teuthology.orchestra.run.gibba011.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-04-03T03:05:58.652 INFO:teuthology.orchestra.run.gibba011.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-04-03T03:05:58.652 INFO:teuthology.orchestra.run.gibba011.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-04-03T03:05:58.653 INFO:teuthology.orchestra.run.gibba015.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-04-03T03:05:58.655 INFO:teuthology.orchestra.run.gibba015.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-04-03T03:05:58.662 INFO:teuthology.orchestra.run.gibba026.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-04-03T03:05:58.665 INFO:teuthology.orchestra.run.gibba026.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-04-03T03:05:58.671 INFO:teuthology.orchestra.run.gibba031.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-04-03T03:05:58.672 INFO:teuthology.orchestra.run.gibba028.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-04-03T03:05:58.673 INFO:teuthology.orchestra.run.gibba031.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-04-03T03:05:58.674 INFO:teuthology.orchestra.run.gibba026.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-04-03T03:05:58.675 INFO:teuthology.orchestra.run.gibba028.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-04-03T03:05:58.675 INFO:teuthology.orchestra.run.gibba026.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-04-03T03:05:58.678 INFO:teuthology.orchestra.run.gibba028.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-04-03T03:05:58.680 INFO:teuthology.orchestra.run.gibba028.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-04-03T03:05:58.681 INFO:teuthology.orchestra.run.gibba026.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-04-03T03:05:58.682 INFO:teuthology.orchestra.run.gibba026.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-04-03T03:05:58.683 INFO:teuthology.orchestra.run.gibba026.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-04-03T03:05:58.687 INFO:teuthology.orchestra.run.gibba026.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-04-03T03:05:58.691 INFO:teuthology.orchestra.run.gibba026.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-04-03T03:05:58.697 INFO:teuthology.orchestra.run.gibba031.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.10-137-g5feb67b5-1bionic [3,624 kB] 2021-04-03T03:05:58.715 INFO:teuthology.orchestra.run.gibba031.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-04-03T03:05:58.715 INFO:teuthology.orchestra.run.gibba031.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-04-03T03:05:58.716 INFO:teuthology.orchestra.run.gibba031.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-04-03T03:05:58.717 INFO:teuthology.orchestra.run.gibba031.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-04-03T03:05:58.719 INFO:teuthology.orchestra.run.gibba031.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-04-03T03:05:58.720 INFO:teuthology.orchestra.run.gibba031.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-04-03T03:05:58.721 INFO:teuthology.orchestra.run.gibba031.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-04-03T03:05:58.724 INFO:teuthology.orchestra.run.gibba011.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-04-03T03:05:58.727 INFO:teuthology.orchestra.run.gibba011.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-04-03T03:05:58.733 INFO:teuthology.orchestra.run.gibba015.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-04-03T03:05:58.741 INFO:teuthology.orchestra.run.gibba015.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-04-03T03:05:58.748 INFO:teuthology.orchestra.run.gibba026.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-04-03T03:05:58.751 INFO:teuthology.orchestra.run.gibba026.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-04-03T03:05:58.753 INFO:teuthology.orchestra.run.gibba028.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-04-03T03:05:58.754 INFO:teuthology.orchestra.run.gibba015.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2021-04-03T03:05:58.755 INFO:teuthology.orchestra.run.gibba015.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-04-03T03:05:58.761 INFO:teuthology.orchestra.run.gibba028.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-04-03T03:05:58.764 INFO:teuthology.orchestra.run.gibba015.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-04-03T03:05:58.764 INFO:teuthology.orchestra.run.gibba015.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-04-03T03:05:58.765 INFO:teuthology.orchestra.run.gibba015.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-04-03T03:05:58.766 INFO:teuthology.orchestra.run.gibba015.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-04-03T03:05:58.774 INFO:teuthology.orchestra.run.gibba028.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2021-04-03T03:05:58.775 INFO:teuthology.orchestra.run.gibba028.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-04-03T03:05:58.783 INFO:teuthology.orchestra.run.gibba028.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-04-03T03:05:58.784 INFO:teuthology.orchestra.run.gibba028.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-04-03T03:05:58.789 INFO:teuthology.orchestra.run.gibba031.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-04-03T03:05:58.790 INFO:teuthology.orchestra.run.gibba031.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-04-03T03:05:58.804 INFO:teuthology.orchestra.run.gibba011.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-04-03T03:05:58.807 INFO:teuthology.orchestra.run.gibba011.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2021-04-03T03:05:58.808 INFO:teuthology.orchestra.run.gibba011.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-04-03T03:05:58.810 INFO:teuthology.orchestra.run.gibba011.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-04-03T03:05:58.810 INFO:teuthology.orchestra.run.gibba011.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-04-03T03:05:58.810 INFO:teuthology.orchestra.run.gibba011.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-04-03T03:05:58.811 INFO:teuthology.orchestra.run.gibba011.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-04-03T03:05:58.811 INFO:teuthology.orchestra.run.gibba011.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:58.814 INFO:teuthology.orchestra.run.gibba015.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:58.819 INFO:teuthology.orchestra.run.gibba015.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-04-03T03:05:58.828 INFO:teuthology.orchestra.run.gibba026.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-04-03T03:05:58.833 INFO:teuthology.orchestra.run.gibba028.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-04-03T03:05:58.834 INFO:teuthology.orchestra.run.gibba028.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-04-03T03:05:58.834 INFO:teuthology.orchestra.run.gibba026.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2021-04-03T03:05:58.835 INFO:teuthology.orchestra.run.gibba026.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-04-03T03:05:58.836 INFO:teuthology.orchestra.run.gibba028.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:58.837 INFO:teuthology.orchestra.run.gibba026.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-04-03T03:05:58.837 INFO:teuthology.orchestra.run.gibba026.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-04-03T03:05:58.838 INFO:teuthology.orchestra.run.gibba026.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-04-03T03:05:58.838 INFO:teuthology.orchestra.run.gibba026.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-04-03T03:05:58.838 INFO:teuthology.orchestra.run.gibba026.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:58.842 INFO:teuthology.orchestra.run.gibba028.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-04-03T03:05:58.869 INFO:teuthology.orchestra.run.gibba031.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-04-03T03:05:58.871 INFO:teuthology.orchestra.run.gibba031.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-04-03T03:05:58.875 INFO:teuthology.orchestra.run.gibba031.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-04-03T03:05:58.875 INFO:teuthology.orchestra.run.gibba031.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-04-03T03:05:58.876 INFO:teuthology.orchestra.run.gibba031.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-04-03T03:05:58.877 INFO:teuthology.orchestra.run.gibba031.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-04-03T03:05:58.877 INFO:teuthology.orchestra.run.gibba031.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-04-03T03:05:58.878 INFO:teuthology.orchestra.run.gibba031.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-04-03T03:05:58.885 INFO:teuthology.orchestra.run.gibba011.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-04-03T03:05:58.886 INFO:teuthology.orchestra.run.gibba011.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-04-03T03:05:58.893 INFO:teuthology.orchestra.run.gibba015.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-04-03T03:05:58.894 INFO:teuthology.orchestra.run.gibba015.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-04-03T03:05:58.894 INFO:teuthology.orchestra.run.gibba015.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-04-03T03:05:58.895 INFO:teuthology.orchestra.run.gibba015.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.4 [1,097 kB] 2021-04-03T03:05:58.908 INFO:teuthology.orchestra.run.gibba026.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-04-03T03:05:58.909 INFO:teuthology.orchestra.run.gibba026.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-04-03T03:05:58.913 INFO:teuthology.orchestra.run.gibba028.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-04-03T03:05:58.913 INFO:teuthology.orchestra.run.gibba028.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-04-03T03:05:58.913 INFO:teuthology.orchestra.run.gibba028.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-04-03T03:05:58.914 INFO:teuthology.orchestra.run.gibba028.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.4 [1,097 kB] 2021-04-03T03:05:58.924 INFO:teuthology.orchestra.run.gibba015.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-04-03T03:05:58.924 INFO:teuthology.orchestra.run.gibba015.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-04-03T03:05:58.925 INFO:teuthology.orchestra.run.gibba028.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-04-03T03:05:58.925 INFO:teuthology.orchestra.run.gibba028.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-04-03T03:05:58.949 INFO:teuthology.orchestra.run.gibba031.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-04-03T03:05:58.955 INFO:teuthology.orchestra.run.gibba031.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-04-03T03:05:58.965 INFO:teuthology.orchestra.run.gibba011.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-04-03T03:05:58.965 INFO:teuthology.orchestra.run.gibba011.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-04-03T03:05:58.965 INFO:teuthology.orchestra.run.gibba011.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.4 [1,097 kB] 2021-04-03T03:05:58.975 INFO:teuthology.orchestra.run.gibba011.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-04-03T03:05:58.975 INFO:teuthology.orchestra.run.gibba011.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-04-03T03:05:58.988 INFO:teuthology.orchestra.run.gibba026.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-04-03T03:05:58.989 INFO:teuthology.orchestra.run.gibba026.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-04-03T03:05:58.992 INFO:teuthology.orchestra.run.gibba026.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.4 [1,097 kB] 2021-04-03T03:05:58.999 INFO:teuthology.orchestra.run.gibba026.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.10-137-g5feb67b5-1bionic [471 kB] 2021-04-03T03:05:59.003 INFO:teuthology.orchestra.run.gibba026.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-04-03T03:05:59.003 INFO:teuthology.orchestra.run.gibba026.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-04-03T03:05:59.029 INFO:teuthology.orchestra.run.gibba031.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2021-04-03T03:05:59.030 INFO:teuthology.orchestra.run.gibba031.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-04-03T03:05:59.037 INFO:teuthology.orchestra.run.gibba028.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-04-03T03:05:59.037 INFO:teuthology.orchestra.run.gibba028.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-04-03T03:05:59.038 INFO:teuthology.orchestra.run.gibba028.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.5 [330 kB] 2021-04-03T03:05:59.038 INFO:teuthology.orchestra.run.gibba031.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-04-03T03:05:59.039 INFO:teuthology.orchestra.run.gibba031.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-04-03T03:05:59.039 INFO:teuthology.orchestra.run.gibba031.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-04-03T03:05:59.040 INFO:teuthology.orchestra.run.gibba031.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-04-03T03:05:59.040 INFO:teuthology.orchestra.run.gibba028.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-04-03T03:05:59.042 INFO:teuthology.orchestra.run.gibba031.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:59.047 INFO:teuthology.orchestra.run.gibba031.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-04-03T03:05:59.060 INFO:teuthology.orchestra.run.gibba015.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-04-03T03:05:59.061 INFO:teuthology.orchestra.run.gibba015.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-04-03T03:05:59.061 INFO:teuthology.orchestra.run.gibba015.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.5 [330 kB] 2021-04-03T03:05:59.064 INFO:teuthology.orchestra.run.gibba015.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-04-03T03:05:59.064 INFO:teuthology.orchestra.run.gibba015.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-04-03T03:05:59.065 INFO:teuthology.orchestra.run.gibba015.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-04-03T03:05:59.065 INFO:teuthology.orchestra.run.gibba015.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:59.066 INFO:teuthology.orchestra.run.gibba015.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-04-03T03:05:59.067 INFO:teuthology.orchestra.run.gibba015.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-04-03T03:05:59.073 INFO:teuthology.orchestra.run.gibba028.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-04-03T03:05:59.073 INFO:teuthology.orchestra.run.gibba028.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-04-03T03:05:59.074 INFO:teuthology.orchestra.run.gibba028.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:59.074 INFO:teuthology.orchestra.run.gibba028.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-04-03T03:05:59.075 INFO:teuthology.orchestra.run.gibba028.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-04-03T03:05:59.078 INFO:teuthology.orchestra.run.gibba026.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.10-137-g5feb67b5-1bionic [397 kB] 2021-04-03T03:05:59.089 INFO:teuthology.orchestra.run.gibba026.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-04-03T03:05:59.090 INFO:teuthology.orchestra.run.gibba026.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-04-03T03:05:59.090 INFO:teuthology.orchestra.run.gibba026.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.5 [330 kB] 2021-04-03T03:05:59.093 INFO:teuthology.orchestra.run.gibba026.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-04-03T03:05:59.093 INFO:teuthology.orchestra.run.gibba026.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-04-03T03:05:59.096 INFO:teuthology.orchestra.run.gibba011.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-04-03T03:05:59.096 INFO:teuthology.orchestra.run.gibba011.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-04-03T03:05:59.097 INFO:teuthology.orchestra.run.gibba011.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.5 [330 kB] 2021-04-03T03:05:59.102 INFO:teuthology.orchestra.run.gibba011.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-04-03T03:05:59.102 INFO:teuthology.orchestra.run.gibba011.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-04-03T03:05:59.109 INFO:teuthology.orchestra.run.gibba031.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-04-03T03:05:59.110 INFO:teuthology.orchestra.run.gibba031.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-04-03T03:05:59.124 INFO:teuthology.orchestra.run.gibba011.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-04-03T03:05:59.126 INFO:teuthology.orchestra.run.gibba011.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:59.129 INFO:teuthology.orchestra.run.gibba011.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-04-03T03:05:59.130 INFO:teuthology.orchestra.run.gibba011.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-04-03T03:05:59.147 INFO:teuthology.orchestra.run.gibba026.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-04-03T03:05:59.149 INFO:teuthology.orchestra.run.gibba026.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:59.157 INFO:teuthology.orchestra.run.gibba026.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-04-03T03:05:59.160 INFO:teuthology.orchestra.run.gibba026.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-04-03T03:05:59.164 INFO:teuthology.orchestra.run.gibba026.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.10-137-g5feb67b5-1bionic [20.7 kB] 2021-04-03T03:05:59.166 INFO:teuthology.orchestra.run.gibba026.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.10-137-g5feb67b5-1bionic [150 kB] 2021-04-03T03:05:59.189 INFO:teuthology.orchestra.run.gibba031.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-04-03T03:05:59.192 INFO:teuthology.orchestra.run.gibba031.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.4 [1,097 kB] 2021-04-03T03:05:59.203 INFO:teuthology.orchestra.run.gibba031.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-04-03T03:05:59.203 INFO:teuthology.orchestra.run.gibba031.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-04-03T03:05:59.230 INFO:teuthology.orchestra.run.gibba026.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.10-137-g5feb67b5-1bionic [29.1 kB] 2021-04-03T03:05:59.231 INFO:teuthology.orchestra.run.gibba026.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.10-137-g5feb67b5-1bionic [374 kB] 2021-04-03T03:05:59.290 INFO:teuthology.orchestra.run.gibba031.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-04-03T03:05:59.290 INFO:teuthology.orchestra.run.gibba031.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-04-03T03:05:59.291 INFO:teuthology.orchestra.run.gibba031.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.5 [330 kB] 2021-04-03T03:05:59.293 INFO:teuthology.orchestra.run.gibba031.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-04-03T03:05:59.293 INFO:teuthology.orchestra.run.gibba031.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-04-03T03:05:59.294 INFO:teuthology.orchestra.run.gibba031.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-04-03T03:05:59.309 INFO:teuthology.orchestra.run.gibba026.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.10-137-g5feb67b5-1bionic [3,163 kB] 2021-04-03T03:05:59.348 INFO:teuthology.orchestra.run.gibba031.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-04-03T03:05:59.352 INFO:teuthology.orchestra.run.gibba031.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-04-03T03:05:59.353 INFO:teuthology.orchestra.run.gibba031.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-04-03T03:05:59.884 INFO:teuthology.orchestra.run.gibba026.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.10-137-g5feb67b5-1bionic [107 kB] 2021-04-03T03:05:59.887 INFO:teuthology.orchestra.run.gibba026.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.10-137-g5feb67b5-1bionic [372 kB] 2021-04-03T03:05:59.963 INFO:teuthology.orchestra.run.gibba026.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.10-137-g5feb67b5-1bionic [20.8 MB] 2021-04-03T03:06:00.695 INFO:teuthology.orchestra.run.gibba015.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.10-137-g5feb67b5-1bionic [9,145 kB] 2021-04-03T03:06:00.850 INFO:teuthology.orchestra.run.gibba011.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.10-137-g5feb67b5-1bionic [9,145 kB] 2021-04-03T03:06:00.937 INFO:teuthology.orchestra.run.gibba028.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.10-137-g5feb67b5-1bionic [9,145 kB] 2021-04-03T03:06:01.019 INFO:teuthology.orchestra.run.gibba031.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.10-137-g5feb67b5-1bionic [471 kB] 2021-04-03T03:06:01.289 INFO:teuthology.orchestra.run.gibba031.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.10-137-g5feb67b5-1bionic [397 kB] 2021-04-03T03:06:01.496 INFO:teuthology.orchestra.run.gibba031.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.10-137-g5feb67b5-1bionic [20.7 kB] 2021-04-03T03:06:01.546 INFO:teuthology.orchestra.run.gibba031.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.10-137-g5feb67b5-1bionic [150 kB] 2021-04-03T03:06:01.628 INFO:teuthology.orchestra.run.gibba031.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.10-137-g5feb67b5-1bionic [29.1 kB] 2021-04-03T03:06:01.631 INFO:teuthology.orchestra.run.gibba031.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.10-137-g5feb67b5-1bionic [374 kB] 2021-04-03T03:06:01.840 INFO:teuthology.orchestra.run.gibba031.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.10-137-g5feb67b5-1bionic [3,163 kB] 2021-04-03T03:06:01.900 INFO:teuthology.orchestra.run.gibba015.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.10-137-g5feb67b5-1bionic [161 kB] 2021-04-03T03:06:01.907 INFO:teuthology.orchestra.run.gibba015.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.10-137-g5feb67b5-1bionic [1,377 kB] 2021-04-03T03:06:02.015 INFO:teuthology.orchestra.run.gibba011.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.10-137-g5feb67b5-1bionic [161 kB] 2021-04-03T03:06:02.068 INFO:teuthology.orchestra.run.gibba011.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.10-137-g5feb67b5-1bionic [1,377 kB] 2021-04-03T03:06:02.103 INFO:teuthology.orchestra.run.gibba015.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.10-137-g5feb67b5-1bionic [6,163 kB] 2021-04-03T03:06:02.170 INFO:teuthology.orchestra.run.gibba028.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.10-137-g5feb67b5-1bionic [161 kB] 2021-04-03T03:06:02.204 INFO:teuthology.orchestra.run.gibba028.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.10-137-g5feb67b5-1bionic [1,377 kB] 2021-04-03T03:06:02.221 INFO:teuthology.orchestra.run.gibba011.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.10-137-g5feb67b5-1bionic [6,163 kB] 2021-04-03T03:06:02.380 INFO:teuthology.orchestra.run.gibba028.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.10-137-g5feb67b5-1bionic [6,163 kB] 2021-04-03T03:06:02.894 INFO:teuthology.orchestra.run.gibba015.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.10-137-g5feb67b5-1bionic [24.2 MB] 2021-04-03T03:06:03.064 INFO:teuthology.orchestra.run.gibba011.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.10-137-g5feb67b5-1bionic [24.2 MB] 2021-04-03T03:06:03.236 INFO:teuthology.orchestra.run.gibba028.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.10-137-g5feb67b5-1bionic [24.2 MB] 2021-04-03T03:06:03.486 INFO:teuthology.orchestra.run.gibba026.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.10-137-g5feb67b5-1bionic [9,145 kB] 2021-04-03T03:06:03.674 INFO:teuthology.orchestra.run.gibba031.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.10-137-g5feb67b5-1bionic [107 kB] 2021-04-03T03:06:03.695 INFO:teuthology.orchestra.run.gibba031.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.10-137-g5feb67b5-1bionic [372 kB] 2021-04-03T03:06:03.908 INFO:teuthology.orchestra.run.gibba031.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.10-137-g5feb67b5-1bionic [20.8 MB] 2021-04-03T03:06:04.966 INFO:teuthology.orchestra.run.gibba026.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.10-137-g5feb67b5-1bionic [161 kB] 2021-04-03T03:06:04.972 INFO:teuthology.orchestra.run.gibba026.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.10-137-g5feb67b5-1bionic [1,377 kB] 2021-04-03T03:06:05.194 INFO:teuthology.orchestra.run.gibba026.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.10-137-g5feb67b5-1bionic [6,163 kB] 2021-04-03T03:06:06.059 INFO:teuthology.orchestra.run.gibba015.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.10-137-g5feb67b5-1bionic [3,948 B] 2021-04-03T03:06:06.060 INFO:teuthology.orchestra.run.gibba015.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.10-137-g5feb67b5-1bionic [546 kB] 2021-04-03T03:06:06.123 INFO:teuthology.orchestra.run.gibba015.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.10-137-g5feb67b5-1bionic [2,012 kB] 2021-04-03T03:06:06.218 INFO:teuthology.orchestra.run.gibba026.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.10-137-g5feb67b5-1bionic [24.2 MB] 2021-04-03T03:06:06.308 INFO:teuthology.orchestra.run.gibba011.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.10-137-g5feb67b5-1bionic [3,948 B] 2021-04-03T03:06:06.308 INFO:teuthology.orchestra.run.gibba011.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.10-137-g5feb67b5-1bionic [546 kB] 2021-04-03T03:06:06.363 INFO:teuthology.orchestra.run.gibba015.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.10-137-g5feb67b5-1bionic [52.6 kB] 2021-04-03T03:06:06.365 INFO:teuthology.orchestra.run.gibba015.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.10-137-g5feb67b5-1bionic [58.0 kB] 2021-04-03T03:06:06.377 INFO:teuthology.orchestra.run.gibba011.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.10-137-g5feb67b5-1bionic [2,012 kB] 2021-04-03T03:06:06.414 INFO:teuthology.orchestra.run.gibba015.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.10-137-g5feb67b5-1bionic [3,197 kB] 2021-04-03T03:06:06.561 INFO:teuthology.orchestra.run.gibba028.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.10-137-g5feb67b5-1bionic [3,948 B] 2021-04-03T03:06:06.562 INFO:teuthology.orchestra.run.gibba028.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.10-137-g5feb67b5-1bionic [546 kB] 2021-04-03T03:06:06.618 INFO:teuthology.orchestra.run.gibba011.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.10-137-g5feb67b5-1bionic [52.6 kB] 2021-04-03T03:06:06.621 INFO:teuthology.orchestra.run.gibba011.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.10-137-g5feb67b5-1bionic [58.0 kB] 2021-04-03T03:06:06.629 INFO:teuthology.orchestra.run.gibba028.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.10-137-g5feb67b5-1bionic [2,012 kB] 2021-04-03T03:06:06.666 INFO:teuthology.orchestra.run.gibba011.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.10-137-g5feb67b5-1bionic [3,197 kB] 2021-04-03T03:06:06.803 INFO:teuthology.orchestra.run.gibba015.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.10-137-g5feb67b5-1bionic [34.3 kB] 2021-04-03T03:06:06.804 INFO:teuthology.orchestra.run.gibba015.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.10-137-g5feb67b5-1bionic [7,866 kB] 2021-04-03T03:06:06.919 INFO:teuthology.orchestra.run.gibba028.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.10-137-g5feb67b5-1bionic [52.6 kB] 2021-04-03T03:06:06.921 INFO:teuthology.orchestra.run.gibba028.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.10-137-g5feb67b5-1bionic [58.0 kB] 2021-04-03T03:06:06.923 INFO:teuthology.orchestra.run.gibba028.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.10-137-g5feb67b5-1bionic [3,197 kB] 2021-04-03T03:06:07.065 INFO:teuthology.orchestra.run.gibba011.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.10-137-g5feb67b5-1bionic [34.3 kB] 2021-04-03T03:06:07.067 INFO:teuthology.orchestra.run.gibba011.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.10-137-g5feb67b5-1bionic [7,866 kB] 2021-04-03T03:06:07.372 INFO:teuthology.orchestra.run.gibba028.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.10-137-g5feb67b5-1bionic [34.3 kB] 2021-04-03T03:06:07.373 INFO:teuthology.orchestra.run.gibba028.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.10-137-g5feb67b5-1bionic [7,866 kB] 2021-04-03T03:06:07.862 INFO:teuthology.orchestra.run.gibba015.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.10-137-g5feb67b5-1bionic [4,064 B] 2021-04-03T03:06:07.862 INFO:teuthology.orchestra.run.gibba015.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.10-137-g5feb67b5-1bionic [22.5 kB] 2021-04-03T03:06:07.864 INFO:teuthology.orchestra.run.gibba015.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.10-137-g5feb67b5-1bionic [41.8 MB] 2021-04-03T03:06:08.181 INFO:teuthology.orchestra.run.gibba011.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.10-137-g5feb67b5-1bionic [4,064 B] 2021-04-03T03:06:08.182 INFO:teuthology.orchestra.run.gibba011.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.10-137-g5feb67b5-1bionic [22.5 kB] 2021-04-03T03:06:08.183 INFO:teuthology.orchestra.run.gibba011.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.10-137-g5feb67b5-1bionic [41.8 MB] 2021-04-03T03:06:08.484 INFO:teuthology.orchestra.run.gibba028.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.10-137-g5feb67b5-1bionic [4,064 B] 2021-04-03T03:06:08.484 INFO:teuthology.orchestra.run.gibba028.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.10-137-g5feb67b5-1bionic [22.5 kB] 2021-04-03T03:06:08.489 INFO:teuthology.orchestra.run.gibba028.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.10-137-g5feb67b5-1bionic [41.8 MB] 2021-04-03T03:06:10.166 INFO:teuthology.orchestra.run.gibba026.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.10-137-g5feb67b5-1bionic [3,948 B] 2021-04-03T03:06:10.166 INFO:teuthology.orchestra.run.gibba026.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.10-137-g5feb67b5-1bionic [546 kB] 2021-04-03T03:06:10.243 INFO:teuthology.orchestra.run.gibba026.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.10-137-g5feb67b5-1bionic [2,012 kB] 2021-04-03T03:06:10.601 INFO:teuthology.orchestra.run.gibba026.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.10-137-g5feb67b5-1bionic [52.6 kB] 2021-04-03T03:06:10.603 INFO:teuthology.orchestra.run.gibba026.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.10-137-g5feb67b5-1bionic [58.0 kB] 2021-04-03T03:06:10.606 INFO:teuthology.orchestra.run.gibba026.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.10-137-g5feb67b5-1bionic [3,197 kB] 2021-04-03T03:06:11.056 INFO:teuthology.orchestra.run.gibba031.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.10-137-g5feb67b5-1bionic [9,145 kB] 2021-04-03T03:06:11.218 INFO:teuthology.orchestra.run.gibba026.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.10-137-g5feb67b5-1bionic [34.3 kB] 2021-04-03T03:06:11.241 INFO:teuthology.orchestra.run.gibba026.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.10-137-g5feb67b5-1bionic [7,866 kB] 2021-04-03T03:06:12.614 INFO:teuthology.orchestra.run.gibba026.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.10-137-g5feb67b5-1bionic [4,064 B] 2021-04-03T03:06:12.615 INFO:teuthology.orchestra.run.gibba026.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.10-137-g5feb67b5-1bionic [22.5 kB] 2021-04-03T03:06:12.616 INFO:teuthology.orchestra.run.gibba026.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.10-137-g5feb67b5-1bionic [41.8 MB] 2021-04-03T03:06:12.746 INFO:teuthology.orchestra.run.gibba015.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.10-137-g5feb67b5-1bionic [15.8 kB] 2021-04-03T03:06:12.747 INFO:teuthology.orchestra.run.gibba015.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.10-137-g5feb67b5-1bionic [18.0 kB] 2021-04-03T03:06:12.748 INFO:teuthology.orchestra.run.gibba015.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.10-137-g5feb67b5-1bionic [10.3 MB] 2021-04-03T03:06:13.442 INFO:teuthology.orchestra.run.gibba031.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.10-137-g5feb67b5-1bionic [161 kB] 2021-04-03T03:06:13.446 INFO:teuthology.orchestra.run.gibba031.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.10-137-g5feb67b5-1bionic [1,377 kB] 2021-04-03T03:06:13.643 INFO:teuthology.orchestra.run.gibba015.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.10-137-g5feb67b5-1bionic [70.6 kB] 2021-04-03T03:06:13.769 INFO:teuthology.orchestra.run.gibba031.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.10-137-g5feb67b5-1bionic [6,163 kB] 2021-04-03T03:06:14.052 INFO:teuthology.orchestra.run.gibba015.stdout:Fetched 164 MB in 17s (9,818 kB/s) 2021-04-03T03:06:14.175 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pyparsing. 2021-04-03T03:06:14.235 INFO:teuthology.orchestra.run.gibba015.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137602 files and directories currently installed.) 2021-04-03T03:06:14.237 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-04-03T03:06:14.273 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:06:14.559 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pyperclip. 2021-04-03T03:06:14.578 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2021-04-03T03:06:14.609 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2021-04-03T03:06:14.774 INFO:teuthology.orchestra.run.gibba028.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.10-137-g5feb67b5-1bionic [15.8 kB] 2021-04-03T03:06:14.775 INFO:teuthology.orchestra.run.gibba028.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.10-137-g5feb67b5-1bionic [18.0 kB] 2021-04-03T03:06:14.776 INFO:teuthology.orchestra.run.gibba028.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.10-137-g5feb67b5-1bionic [10.3 MB] 2021-04-03T03:06:14.870 INFO:teuthology.orchestra.run.gibba031.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.10-137-g5feb67b5-1bionic [24.2 MB] 2021-04-03T03:06:14.895 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-cmd2. 2021-04-03T03:06:14.914 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2021-04-03T03:06:14.953 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:06:15.223 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-uritemplate. 2021-04-03T03:06:15.242 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-04-03T03:06:15.281 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:06:15.592 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-04-03T03:06:15.599 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-04-03T03:06:15.683 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:06:15.734 INFO:teuthology.orchestra.run.gibba011.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.10-137-g5feb67b5-1bionic [15.8 kB] 2021-04-03T03:06:15.735 INFO:teuthology.orchestra.run.gibba011.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.10-137-g5feb67b5-1bionic [18.0 kB] 2021-04-03T03:06:15.736 INFO:teuthology.orchestra.run.gibba011.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.10-137-g5feb67b5-1bionic [10.3 MB] 2021-04-03T03:06:15.933 INFO:teuthology.orchestra.run.gibba028.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.10-137-g5feb67b5-1bionic [70.6 kB] 2021-04-03T03:06:16.005 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../005-librbd1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:16.103 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking librbd1 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:16.350 INFO:teuthology.orchestra.run.gibba028.stdout:Fetched 164 MB in 19s (8,648 kB/s) 2021-04-03T03:06:16.469 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pyparsing. 2021-04-03T03:06:16.530 INFO:teuthology.orchestra.run.gibba028.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137602 files and directories currently installed.) 2021-04-03T03:06:16.533 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-04-03T03:06:16.576 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:06:16.604 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../006-librados2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:16.751 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking librados2 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:16.853 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pyperclip. 2021-04-03T03:06:16.873 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2021-04-03T03:06:16.903 INFO:teuthology.orchestra.run.gibba026.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.10-137-g5feb67b5-1bionic [15.8 kB] 2021-04-03T03:06:16.904 INFO:teuthology.orchestra.run.gibba026.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.10-137-g5feb67b5-1bionic [18.0 kB] 2021-04-03T03:06:16.905 INFO:teuthology.orchestra.run.gibba026.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.10-137-g5feb67b5-1bionic [10.3 MB] 2021-04-03T03:06:16.912 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2021-04-03T03:06:16.936 INFO:teuthology.orchestra.run.gibba011.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.10-137-g5feb67b5-1bionic [70.6 kB] 2021-04-03T03:06:17.214 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-cmd2. 2021-04-03T03:06:17.221 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2021-04-03T03:06:17.289 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:06:17.304 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libcephfs2. 2021-04-03T03:06:17.324 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../007-libcephfs2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:17.363 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:17.381 INFO:teuthology.orchestra.run.gibba011.stdout:Fetched 164 MB in 20s (8,209 kB/s) 2021-04-03T03:06:17.464 INFO:teuthology.orchestra.run.gibba026.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.10-137-g5feb67b5-1bionic [70.6 kB] 2021-04-03T03:06:17.490 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pyparsing. 2021-04-03T03:06:17.553 INFO:teuthology.orchestra.run.gibba011.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137602 files and directories currently installed.) 2021-04-03T03:06:17.556 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-04-03T03:06:17.583 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-uritemplate. 2021-04-03T03:06:17.597 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:06:17.603 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-04-03T03:06:17.616 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-rados. 2021-04-03T03:06:17.635 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../008-python3-rados_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:17.641 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:06:17.674 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:17.716 INFO:teuthology.orchestra.run.gibba031.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.10-137-g5feb67b5-1bionic [3,948 B] 2021-04-03T03:06:17.716 INFO:teuthology.orchestra.run.gibba031.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.10-137-g5feb67b5-1bionic [546 kB] 2021-04-03T03:06:17.746 INFO:teuthology.orchestra.run.gibba031.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.10-137-g5feb67b5-1bionic [2,012 kB] 2021-04-03T03:06:17.871 INFO:teuthology.orchestra.run.gibba031.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.10-137-g5feb67b5-1bionic [52.6 kB] 2021-04-03T03:06:17.874 INFO:teuthology.orchestra.run.gibba026.stdout:Fetched 164 MB in 20s (8,034 kB/s) 2021-04-03T03:06:17.879 INFO:teuthology.orchestra.run.gibba031.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.10-137-g5feb67b5-1bionic [58.0 kB] 2021-04-03T03:06:17.881 INFO:teuthology.orchestra.run.gibba031.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.10-137-g5feb67b5-1bionic [3,197 kB] 2021-04-03T03:06:17.891 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pyperclip. 2021-04-03T03:06:17.910 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2021-04-03T03:06:17.949 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2021-04-03T03:06:17.953 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-04-03T03:06:17.961 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-04-03T03:06:17.976 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pyparsing. 2021-04-03T03:06:18.010 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-04-03T03:06:18.017 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:18.028 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:06:18.038 INFO:teuthology.orchestra.run.gibba026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137602 files and directories currently installed.) 2021-04-03T03:06:18.040 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-04-03T03:06:18.059 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:18.072 INFO:teuthology.orchestra.run.gibba031.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.10-137-g5feb67b5-1bionic [34.3 kB] 2021-04-03T03:06:18.072 INFO:teuthology.orchestra.run.gibba031.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.10-137-g5feb67b5-1bionic [7,866 kB] 2021-04-03T03:06:18.074 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:06:18.268 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-cmd2. 2021-04-03T03:06:18.276 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2021-04-03T03:06:18.309 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../005-librbd1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:18.318 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:06:18.360 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pyperclip. 2021-04-03T03:06:18.379 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2021-04-03T03:06:18.395 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-cephfs. 2021-04-03T03:06:18.415 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../010-python3-cephfs_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:18.418 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2021-04-03T03:06:18.442 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking librbd1 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:18.453 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:18.541 INFO:teuthology.orchestra.run.gibba031.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.10-137-g5feb67b5-1bionic [4,064 B] 2021-04-03T03:06:18.542 INFO:teuthology.orchestra.run.gibba031.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.10-137-g5feb67b5-1bionic [22.5 kB] 2021-04-03T03:06:18.542 INFO:teuthology.orchestra.run.gibba031.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.10-137-g5feb67b5-1bionic [41.8 MB] 2021-04-03T03:06:18.638 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-uritemplate. 2021-04-03T03:06:18.657 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-04-03T03:06:18.687 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-cmd2. 2021-04-03T03:06:18.706 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2021-04-03T03:06:18.738 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:06:18.739 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-ceph-common. 2021-04-03T03:06:18.745 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:06:18.758 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:18.789 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:18.974 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../006-librados2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:19.017 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-prettytable. 2021-04-03T03:06:19.037 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2021-04-03T03:06:19.065 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-uritemplate. 2021-04-03T03:06:19.074 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-04-03T03:06:19.075 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-04-03T03:06:19.082 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-04-03T03:06:19.085 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-04-03T03:06:19.096 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking librados2 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:19.115 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:06:19.165 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:06:19.328 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-rbd. 2021-04-03T03:06:19.347 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../013-python3-rbd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:19.386 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:19.401 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-04-03T03:06:19.420 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-04-03T03:06:19.484 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:06:19.496 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../005-librbd1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:19.630 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking librbd1 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:19.700 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libcephfs2. 2021-04-03T03:06:19.719 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../007-libcephfs2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:19.755 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-04-03T03:06:19.757 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../005-librbd1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:19.758 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:19.760 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-04-03T03:06:19.805 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:06:19.882 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking librbd1 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:20.078 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-rados. 2021-04-03T03:06:20.082 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-04-03T03:06:20.084 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../008-python3-rados_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:20.101 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-04-03T03:06:20.112 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../006-librados2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:20.127 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:20.150 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:06:20.234 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking librados2 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:20.355 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../006-librados2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:20.435 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package librgw2. 2021-04-03T03:06:20.452 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking librados2 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:20.455 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../016-librgw2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:20.456 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-04-03T03:06:20.475 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:20.494 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:20.514 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:20.735 INFO:teuthology.orchestra.run.gibba031.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.10-137-g5feb67b5-1bionic [15.8 kB] 2021-04-03T03:06:20.736 INFO:teuthology.orchestra.run.gibba031.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.10-137-g5feb67b5-1bionic [18.0 kB] 2021-04-03T03:06:20.736 INFO:teuthology.orchestra.run.gibba031.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.10-137-g5feb67b5-1bionic [10.3 MB] 2021-04-03T03:06:20.791 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-cephfs. 2021-04-03T03:06:20.798 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../010-python3-cephfs_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:20.832 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:20.854 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libcephfs2. 2021-04-03T03:06:20.862 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../007-libcephfs2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:20.904 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:20.945 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-rgw. 2021-04-03T03:06:20.965 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../017-python3-rgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:20.997 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libcephfs2. 2021-04-03T03:06:21.004 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.017 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../007-libcephfs2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:21.055 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.135 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-ceph-common. 2021-04-03T03:06:21.154 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:21.182 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-rados. 2021-04-03T03:06:21.193 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.202 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../008-python3-rados_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:21.207 INFO:teuthology.orchestra.run.gibba031.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/octopus/5feb67b5bda898b8c739180363606432d4f39831/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.10-137-g5feb67b5-1bionic [70.6 kB] 2021-04-03T03:06:21.249 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.306 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-04-03T03:06:21.326 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-04-03T03:06:21.333 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-rados. 2021-04-03T03:06:21.353 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../008-python3-rados_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:21.364 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:06:21.391 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.538 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-prettytable. 2021-04-03T03:06:21.546 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2021-04-03T03:06:21.588 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-04-03T03:06:21.610 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-04-03T03:06:21.613 INFO:teuthology.orchestra.run.gibba031.stdout:Fetched 164 MB in 24s (6,817 kB/s) 2021-04-03T03:06:21.617 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:21.626 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-04-03T03:06:21.634 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2021-04-03T03:06:21.659 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.667 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:06:21.692 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pyparsing. 2021-04-03T03:06:21.716 INFO:teuthology.orchestra.run.gibba031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137602 files and directories currently installed.) 2021-04-03T03:06:21.718 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-04-03T03:06:21.750 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:06:21.760 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-04-03T03:06:21.777 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:21.810 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.832 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-rbd. 2021-04-03T03:06:21.851 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../013-python3-rbd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:21.899 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:21.979 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package liboath0. 2021-04-03T03:06:21.979 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-cephfs. 2021-04-03T03:06:21.998 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2021-04-03T03:06:21.999 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../010-python3-cephfs_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:22.003 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pyperclip. 2021-04-03T03:06:22.022 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2021-04-03T03:06:22.037 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-04-03T03:06:22.037 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:22.061 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2021-04-03T03:06:22.062 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-cephfs. 2021-04-03T03:06:22.070 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../010-python3-cephfs_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:22.104 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:22.226 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-04-03T03:06:22.234 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-04-03T03:06:22.276 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:06:22.313 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-cmd2. 2021-04-03T03:06:22.332 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2021-04-03T03:06:22.339 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libradosstriper1. 2021-04-03T03:06:22.347 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../021-libradosstriper1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:22.371 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:06:22.373 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-ceph-common. 2021-04-03T03:06:22.389 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:22.390 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-ceph-common. 2021-04-03T03:06:22.393 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:22.398 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:22.431 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:22.440 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:22.604 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-04-03T03:06:22.612 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-04-03T03:06:22.646 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:06:22.666 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-uritemplate. 2021-04-03T03:06:22.674 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-04-03T03:06:22.716 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:06:22.735 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-prettytable. 2021-04-03T03:06:22.743 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-prettytable. 2021-04-03T03:06:22.754 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2021-04-03T03:06:22.762 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2021-04-03T03:06:22.775 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-common. 2021-04-03T03:06:22.793 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-04-03T03:06:22.795 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../022-ceph-common_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:22.809 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-04-03T03:06:22.825 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:23.002 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-04-03T03:06:23.006 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package librgw2. 2021-04-03T03:06:23.021 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-04-03T03:06:23.026 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../016-librgw2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:23.062 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-rbd. 2021-04-03T03:06:23.065 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:23.070 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-rbd. 2021-04-03T03:06:23.070 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../013-python3-rbd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:23.085 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:06:23.089 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../013-python3-rbd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:23.111 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:23.128 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:23.382 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../005-librbd1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:23.456 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-04-03T03:06:23.475 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-04-03T03:06:23.488 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking librbd1 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:23.498 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-04-03T03:06:23.505 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-04-03T03:06:23.509 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-rgw. 2021-04-03T03:06:23.514 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:06:23.528 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../017-python3-rgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:23.548 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:06:23.567 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:23.792 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-04-03T03:06:23.799 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-04-03T03:06:23.833 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:06:23.878 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-04-03T03:06:23.897 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-04-03T03:06:23.901 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-04-03T03:06:23.920 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-04-03T03:06:23.936 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:06:23.956 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../006-librados2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:23.967 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:06:24.078 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking librados2 (15.2.10-137-g5feb67b5-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-04-03T03:06:24.169 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package librgw2. 2021-04-03T03:06:24.189 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../016-librgw2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:24.228 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:24.265 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-04-03T03:06:24.270 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2021-04-03T03:06:24.296 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-base. 2021-04-03T03:06:24.303 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package librgw2. 2021-04-03T03:06:24.310 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../016-librgw2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:24.314 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../023-ceph-base_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:24.314 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:06:24.344 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:24.367 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:24.639 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libcephfs2. 2021-04-03T03:06:24.657 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../007-libcephfs2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:24.675 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package liboath0. 2021-04-03T03:06:24.695 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2021-04-03T03:06:24.696 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-rgw. 2021-04-03T03:06:24.698 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:24.714 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../017-python3-rgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:24.742 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-04-03T03:06:24.746 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:24.805 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-rgw. 2021-04-03T03:06:24.825 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../017-python3-rgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:24.863 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:24.984 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-rados. 2021-04-03T03:06:25.003 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../008-python3-rados_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:25.019 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libradosstriper1. 2021-04-03T03:06:25.027 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../021-libradosstriper1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:25.042 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:25.057 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-04-03T03:06:25.069 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:25.077 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-04-03T03:06:25.107 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:06:25.167 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-04-03T03:06:25.174 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-04-03T03:06:25.216 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:06:25.351 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-dateutil. 2021-04-03T03:06:25.359 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-04-03T03:06:25.361 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-04-03T03:06:25.367 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2021-04-03T03:06:25.368 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:25.370 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2021-04-03T03:06:25.401 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:06:25.409 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-04-03T03:06:25.411 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:25.586 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-04-03T03:06:25.598 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-common. 2021-04-03T03:06:25.605 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2021-04-03T03:06:25.606 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../022-ceph-common_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:25.648 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:25.653 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:06:25.672 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-cephfs. 2021-04-03T03:06:25.691 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../010-python3-cephfs_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:25.722 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:25.737 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-asn1crypto. 2021-04-03T03:06:25.745 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2021-04-03T03:06:25.771 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package liboath0. 2021-04-03T03:06:25.787 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:06:25.790 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2021-04-03T03:06:25.829 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-04-03T03:06:25.966 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-ceph-common. 2021-04-03T03:06:25.985 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:25.997 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package liboath0. 2021-04-03T03:06:26.004 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2021-04-03T03:06:26.046 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-04-03T03:06:26.048 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:26.073 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-cffi-backend. 2021-04-03T03:06:26.092 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-04-03T03:06:26.115 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libradosstriper1. 2021-04-03T03:06:26.123 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../021-libradosstriper1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:26.131 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:06:26.156 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:26.369 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-prettytable. 2021-04-03T03:06:26.374 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2021-04-03T03:06:26.375 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libradosstriper1. 2021-04-03T03:06:26.394 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../021-libradosstriper1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:26.410 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-04-03T03:06:26.433 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:26.450 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-cryptography. 2021-04-03T03:06:26.458 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-04-03T03:06:26.499 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:06:26.518 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-common. 2021-04-03T03:06:26.537 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../022-ceph-common_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:26.576 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:26.655 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-rbd. 2021-04-03T03:06:26.674 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../013-python3-rbd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:26.713 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:26.837 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-openssl. 2021-04-03T03:06:26.856 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-04-03T03:06:26.870 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-common. 2021-04-03T03:06:26.877 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../022-ceph-common_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:26.895 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:06:26.919 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:27.040 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-04-03T03:06:27.047 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-04-03T03:06:27.081 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:06:27.156 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-04-03T03:06:27.169 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-base. 2021-04-03T03:06:27.175 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:27.177 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../023-ceph-base_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:27.214 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:27.215 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:27.334 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-04-03T03:06:27.349 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-04-03T03:06:27.384 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:06:27.543 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-bcrypt. 2021-04-03T03:06:27.550 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-04-03T03:06:27.592 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-04-03T03:06:27.671 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package librgw2. 2021-04-03T03:06:27.678 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../016-librgw2_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:27.720 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:27.895 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-cherrypy3. 2021-04-03T03:06:27.915 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2021-04-03T03:06:27.962 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:06:28.039 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-base. 2021-04-03T03:06:28.058 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../023-ceph-base_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:28.110 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:28.222 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-rgw. 2021-04-03T03:06:28.241 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../017-python3-rgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:28.281 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:28.298 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-jwt. 2021-04-03T03:06:28.305 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-04-03T03:06:28.308 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-dateutil. 2021-04-03T03:06:28.315 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2021-04-03T03:06:28.348 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:06:28.357 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-04-03T03:06:28.449 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-base. 2021-04-03T03:06:28.456 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../023-ceph-base_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:28.502 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:28.558 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-04-03T03:06:28.566 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-04-03T03:06:28.599 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:06:28.642 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-logutils. 2021-04-03T03:06:28.652 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-asn1crypto. 2021-04-03T03:06:28.662 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2021-04-03T03:06:28.671 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2021-04-03T03:06:28.702 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:06:28.709 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-04-03T03:06:28.836 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-04-03T03:06:28.855 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2021-04-03T03:06:28.894 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:06:28.937 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-cffi-backend. 2021-04-03T03:06:28.957 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-04-03T03:06:28.987 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:06:29.036 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-markupsafe. 2021-04-03T03:06:29.043 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-04-03T03:06:29.086 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-04-03T03:06:29.151 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-dateutil. 2021-04-03T03:06:29.171 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2021-04-03T03:06:29.210 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-04-03T03:06:29.247 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package liboath0. 2021-04-03T03:06:29.248 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-cryptography. 2021-04-03T03:06:29.255 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2021-04-03T03:06:29.268 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-04-03T03:06:29.296 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-04-03T03:06:29.306 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:06:29.406 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-mako. 2021-04-03T03:06:29.425 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2021-04-03T03:06:29.464 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:06:29.505 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-asn1crypto. 2021-04-03T03:06:29.524 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2021-04-03T03:06:29.555 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:06:29.579 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-dateutil. 2021-04-03T03:06:29.593 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-openssl. 2021-04-03T03:06:29.598 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2021-04-03T03:06:29.613 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-04-03T03:06:29.624 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libradosstriper1. 2021-04-03T03:06:29.637 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-04-03T03:06:29.643 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../021-libradosstriper1_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:29.682 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:29.685 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:06:29.758 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-simplegeneric. 2021-04-03T03:06:29.766 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2021-04-03T03:06:29.808 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:06:29.815 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-cffi-backend. 2021-04-03T03:06:29.835 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-04-03T03:06:29.873 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:06:29.898 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-asn1crypto. 2021-04-03T03:06:29.917 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2021-04-03T03:06:29.954 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-04-03T03:06:29.956 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:06:29.973 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:30.012 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:30.043 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-common. 2021-04-03T03:06:30.047 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../022-ceph-common_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:30.084 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:30.111 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-singledispatch. 2021-04-03T03:06:30.130 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-04-03T03:06:30.167 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-cryptography. 2021-04-03T03:06:30.175 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-04-03T03:06:30.177 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:06:30.185 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-cffi-backend. 2021-04-03T03:06:30.204 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-04-03T03:06:30.209 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:06:30.243 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:06:30.325 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-bcrypt. 2021-04-03T03:06:30.332 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-04-03T03:06:30.497 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-webob. 2021-04-03T03:06:30.503 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-04-03T03:06:30.538 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:06:30.553 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-openssl. 2021-04-03T03:06:30.573 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-04-03T03:06:30.612 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:06:30.857 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-04-03T03:06:30.876 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:30.915 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:31.167 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-04-03T03:06:31.170 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-cryptography. 2021-04-03T03:06:31.177 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-04-03T03:06:31.220 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:06:31.235 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-bcrypt. 2021-04-03T03:06:31.242 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-04-03T03:06:31.284 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-04-03T03:06:31.458 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-bs4. 2021-04-03T03:06:31.465 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2021-04-03T03:06:31.486 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-cherrypy3. 2021-04-03T03:06:31.505 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2021-04-03T03:06:31.507 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-04-03T03:06:31.544 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:06:31.573 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-openssl. 2021-04-03T03:06:31.593 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-04-03T03:06:31.632 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:06:31.818 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-waitress. 2021-04-03T03:06:31.838 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2021-04-03T03:06:31.847 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-jwt. 2021-04-03T03:06:31.854 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-04-03T03:06:31.876 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-04-03T03:06:31.896 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:06:31.900 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-04-03T03:06:31.920 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:31.959 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:32.182 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-base. 2021-04-03T03:06:32.199 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../023-ceph-base_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:32.204 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-tempita. 2021-04-03T03:06:32.210 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2021-04-03T03:06:32.216 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-logutils. 2021-04-03T03:06:32.236 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2021-04-03T03:06:32.252 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:32.254 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-04-03T03:06:32.274 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-04-03T03:06:32.305 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-bcrypt. 2021-04-03T03:06:32.312 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-04-03T03:06:32.354 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-04-03T03:06:32.462 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-cherrypy3. 2021-04-03T03:06:32.469 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2021-04-03T03:06:32.512 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:06:32.590 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-paste. 2021-04-03T03:06:32.610 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-04-03T03:06:32.611 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-markupsafe. 2021-04-03T03:06:32.617 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-04-03T03:06:32.649 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:06:32.660 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-04-03T03:06:32.682 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-cherrypy3. 2021-04-03T03:06:32.701 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2021-04-03T03:06:32.749 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:06:32.841 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-jwt. 2021-04-03T03:06:32.860 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-04-03T03:06:32.907 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:06:32.951 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-04-03T03:06:32.959 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-04-03T03:06:32.963 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-mako. 2021-04-03T03:06:32.983 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2021-04-03T03:06:33.001 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:06:33.021 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:06:33.085 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-jwt. 2021-04-03T03:06:33.093 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-04-03T03:06:33.135 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:06:33.202 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-logutils. 2021-04-03T03:06:33.221 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2021-04-03T03:06:33.273 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-04-03T03:06:33.277 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-dateutil. 2021-04-03T03:06:33.295 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2021-04-03T03:06:33.327 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-04-03T03:06:33.328 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pastedeploy. 2021-04-03T03:06:33.332 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-simplegeneric. 2021-04-03T03:06:33.339 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2021-04-03T03:06:33.348 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2021-04-03T03:06:33.382 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:06:33.387 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:06:33.455 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-logutils. 2021-04-03T03:06:33.475 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2021-04-03T03:06:33.521 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-04-03T03:06:33.571 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-markupsafe. 2021-04-03T03:06:33.578 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-04-03T03:06:33.581 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-asn1crypto. 2021-04-03T03:06:33.599 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2021-04-03T03:06:33.620 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-04-03T03:06:33.630 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:06:33.710 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-singledispatch. 2021-04-03T03:06:33.714 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-webtest. 2021-04-03T03:06:33.721 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-04-03T03:06:33.730 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-04-03T03:06:33.764 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:06:33.769 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:06:33.849 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-markupsafe. 2021-04-03T03:06:33.856 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-04-03T03:06:33.874 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-cffi-backend. 2021-04-03T03:06:33.892 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-04-03T03:06:33.898 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-04-03T03:06:33.924 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-mako. 2021-04-03T03:06:33.932 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:06:33.943 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2021-04-03T03:06:33.982 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:06:34.113 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-webob. 2021-04-03T03:06:34.119 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-04-03T03:06:34.126 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pecan. 2021-04-03T03:06:34.145 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2021-04-03T03:06:34.162 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:06:34.192 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-04-03T03:06:34.201 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-cryptography. 2021-04-03T03:06:34.208 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-04-03T03:06:34.226 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-mako. 2021-04-03T03:06:34.243 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:06:34.246 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2021-04-03T03:06:34.285 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:06:34.301 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-simplegeneric. 2021-04-03T03:06:34.308 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2021-04-03T03:06:34.350 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:06:34.473 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-bs4. 2021-04-03T03:06:34.493 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2021-04-03T03:06:34.531 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-04-03T03:06:34.538 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-werkzeug. 2021-04-03T03:06:34.545 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-04-03T03:06:34.580 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-simplegeneric. 2021-04-03T03:06:34.587 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2021-04-03T03:06:34.587 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:06:34.588 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-openssl. 2021-04-03T03:06:34.608 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-04-03T03:06:34.620 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-singledispatch. 2021-04-03T03:06:34.629 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:06:34.638 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:06:34.639 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-04-03T03:06:34.678 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:06:34.842 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-waitress. 2021-04-03T03:06:34.850 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2021-04-03T03:06:34.890 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-04-03T03:06:34.892 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-04-03T03:06:34.910 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:34.923 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-singledispatch. 2021-04-03T03:06:34.933 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr. 2021-04-03T03:06:34.943 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-04-03T03:06:34.952 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../048-ceph-mgr_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:34.957 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:34.990 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:06:34.991 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:35.005 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-webob. 2021-04-03T03:06:35.012 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-04-03T03:06:35.055 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:06:35.179 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-tempita. 2021-04-03T03:06:35.199 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2021-04-03T03:06:35.245 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-04-03T03:06:35.294 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-bcrypt. 2021-04-03T03:06:35.302 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-04-03T03:06:35.318 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-webob. 2021-04-03T03:06:35.325 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-04-03T03:06:35.343 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-04-03T03:06:35.367 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-bs4. 2021-04-03T03:06:35.367 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:06:35.386 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2021-04-03T03:06:35.419 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mon. 2021-04-03T03:06:35.425 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-04-03T03:06:35.425 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../049-ceph-mon_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:35.468 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:35.573 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-paste. 2021-04-03T03:06:35.581 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-04-03T03:06:35.622 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:06:35.646 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-cherrypy3. 2021-04-03T03:06:35.666 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2021-04-03T03:06:35.679 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-bs4. 2021-04-03T03:06:35.698 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2021-04-03T03:06:35.704 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:06:35.737 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-04-03T03:06:35.753 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-waitress. 2021-04-03T03:06:35.760 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2021-04-03T03:06:35.802 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-04-03T03:06:35.909 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-04-03T03:06:35.929 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-04-03T03:06:35.975 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:06:36.033 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-jwt. 2021-04-03T03:06:36.039 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-04-03T03:06:36.081 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-waitress. 2021-04-03T03:06:36.082 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:06:36.087 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2021-04-03T03:06:36.122 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-tempita. 2021-04-03T03:06:36.131 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-04-03T03:06:36.141 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2021-04-03T03:06:36.180 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-04-03T03:06:36.255 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-osd. 2021-04-03T03:06:36.263 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../050-ceph-osd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:36.279 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pastedeploy. 2021-04-03T03:06:36.286 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2021-04-03T03:06:36.304 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:36.328 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:06:36.394 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-logutils. 2021-04-03T03:06:36.413 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2021-04-03T03:06:36.451 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-tempita. 2021-04-03T03:06:36.460 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-04-03T03:06:36.470 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2021-04-03T03:06:36.482 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-paste. 2021-04-03T03:06:36.490 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-04-03T03:06:36.509 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-04-03T03:06:36.532 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:06:36.623 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-webtest. 2021-04-03T03:06:36.642 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-04-03T03:06:36.681 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:06:36.763 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-markupsafe. 2021-04-03T03:06:36.770 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-04-03T03:06:36.803 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-paste. 2021-04-03T03:06:36.810 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-04-03T03:06:36.812 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-04-03T03:06:36.853 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:06:36.861 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-04-03T03:06:36.880 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-04-03T03:06:36.919 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:06:37.042 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pecan. 2021-04-03T03:06:37.049 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2021-04-03T03:06:37.091 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-04-03T03:06:37.116 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-mako. 2021-04-03T03:06:37.135 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2021-04-03T03:06:37.148 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-04-03T03:06:37.168 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-04-03T03:06:37.174 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:06:37.206 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:06:37.246 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pastedeploy. 2021-04-03T03:06:37.253 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2021-04-03T03:06:37.296 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:06:37.422 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-werkzeug. 2021-04-03T03:06:37.442 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-04-03T03:06:37.485 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-simplegeneric. 2021-04-03T03:06:37.489 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:06:37.491 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2021-04-03T03:06:37.534 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pastedeploy. 2021-04-03T03:06:37.534 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:06:37.553 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2021-04-03T03:06:37.599 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-webtest. 2021-04-03T03:06:37.600 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:06:37.618 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-04-03T03:06:37.665 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:06:37.816 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr. 2021-04-03T03:06:37.823 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../048-ceph-mgr_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:37.845 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-singledispatch. 2021-04-03T03:06:37.852 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph. 2021-04-03T03:06:37.865 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-04-03T03:06:37.865 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:37.869 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../051-ceph_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:37.903 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:06:37.910 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:37.919 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-webtest. 2021-04-03T03:06:37.927 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-04-03T03:06:37.969 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:06:38.010 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pecan. 2021-04-03T03:06:38.018 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2021-04-03T03:06:38.060 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-04-03T03:06:38.222 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-webob. 2021-04-03T03:06:38.229 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-fuse. 2021-04-03T03:06:38.230 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-04-03T03:06:38.236 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../052-ceph-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:38.272 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:06:38.278 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:38.314 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pecan. 2021-04-03T03:06:38.319 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mon. 2021-04-03T03:06:38.326 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../049-ceph-mon_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:38.334 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2021-04-03T03:06:38.368 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:38.373 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-04-03T03:06:38.431 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-werkzeug. 2021-04-03T03:06:38.450 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-04-03T03:06:38.497 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:06:38.606 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mds. 2021-04-03T03:06:38.609 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-bs4. 2021-04-03T03:06:38.626 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../053-ceph-mds_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:38.628 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2021-04-03T03:06:38.664 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:38.675 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-04-03T03:06:38.735 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-werkzeug. 2021-04-03T03:06:38.742 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-04-03T03:06:38.784 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:06:38.867 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr. 2021-04-03T03:06:38.874 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../048-ceph-mgr_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:38.916 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:38.969 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-waitress. 2021-04-03T03:06:38.975 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2021-04-03T03:06:39.010 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-04-03T03:06:39.059 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package cephadm. 2021-04-03T03:06:39.066 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../054-cephadm_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:39.101 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:39.130 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr. 2021-04-03T03:06:39.147 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-osd. 2021-04-03T03:06:39.149 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../048-ceph-mgr_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:39.155 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../050-ceph-osd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:39.188 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:39.197 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:39.305 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-tempita. 2021-04-03T03:06:39.325 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2021-04-03T03:06:39.363 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-04-03T03:06:39.370 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mon. 2021-04-03T03:06:39.390 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../049-ceph-mon_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:39.436 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:39.445 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-jinja2. 2021-04-03T03:06:39.465 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-04-03T03:06:39.503 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:06:39.616 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mon. 2021-04-03T03:06:39.624 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../049-ceph-mon_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:39.665 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:39.674 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-paste. 2021-04-03T03:06:39.680 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-04-03T03:06:39.716 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:06:39.823 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-04-03T03:06:39.831 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:39.872 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:40.010 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-04-03T03:06:40.014 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-04-03T03:06:40.052 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:06:40.209 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-repoze.lru. 2021-04-03T03:06:40.223 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-osd. 2021-04-03T03:06:40.229 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2021-04-03T03:06:40.243 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../050-ceph-osd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:40.275 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-04-03T03:06:40.281 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:40.346 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pastedeploy. 2021-04-03T03:06:40.353 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2021-04-03T03:06:40.396 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:06:40.444 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-osd. 2021-04-03T03:06:40.452 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../050-ceph-osd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:40.493 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:40.578 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-routes. 2021-04-03T03:06:40.585 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2021-04-03T03:06:40.619 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-04-03T03:06:40.698 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-webtest. 2021-04-03T03:06:40.718 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-04-03T03:06:40.757 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:06:40.802 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph. 2021-04-03T03:06:40.822 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../051-ceph_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:40.868 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:40.939 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-04-03T03:06:40.959 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:41.085 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pecan. 2021-04-03T03:06:41.092 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2021-04-03T03:06:41.134 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-04-03T03:06:41.212 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-fuse. 2021-04-03T03:06:41.220 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../052-ceph-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:41.230 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:41.262 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:41.481 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-werkzeug. 2021-04-03T03:06:41.500 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-04-03T03:06:41.547 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:06:41.649 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mds. 2021-04-03T03:06:41.669 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../053-ceph-mds_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:41.716 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:41.869 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph. 2021-04-03T03:06:41.887 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../051-ceph_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:41.900 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr. 2021-04-03T03:06:41.903 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-04-03T03:06:41.907 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../048-ceph-mgr_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:41.925 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:41.936 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:41.949 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:41.961 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.041 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph. 2021-04-03T03:06:42.059 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../051-ceph_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:42.099 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.177 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package cephadm. 2021-04-03T03:06:42.197 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../054-cephadm_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:42.243 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.255 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-fuse. 2021-04-03T03:06:42.262 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../052-ceph-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:42.289 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-numpy. 2021-04-03T03:06:42.294 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-04-03T03:06:42.305 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.338 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:06:42.386 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mon. 2021-04-03T03:06:42.406 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../049-ceph-mon_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:42.435 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-fuse. 2021-04-03T03:06:42.442 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../052-ceph-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:42.445 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.485 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.579 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-jinja2. 2021-04-03T03:06:42.586 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-04-03T03:06:42.629 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:06:42.674 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mds. 2021-04-03T03:06:42.692 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../053-ceph-mds_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:42.733 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.836 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-decorator. 2021-04-03T03:06:42.843 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2021-04-03T03:06:42.880 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mds. 2021-04-03T03:06:42.885 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-04-03T03:06:42.900 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../053-ceph-mds_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:42.938 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:42.949 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-04-03T03:06:42.969 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:43.007 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:43.202 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package cephadm. 2021-04-03T03:06:43.205 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-osd. 2021-04-03T03:06:43.220 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../054-cephadm_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:43.221 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../050-ceph-osd_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:43.222 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-scipy. 2021-04-03T03:06:43.242 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-04-03T03:06:43.264 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:43.268 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:43.280 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:06:43.318 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-repoze.lru. 2021-04-03T03:06:43.326 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2021-04-03T03:06:43.368 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-04-03T03:06:43.391 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package cephadm. 2021-04-03T03:06:43.399 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../054-cephadm_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:43.441 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:43.622 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-jinja2. 2021-04-03T03:06:43.628 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-04-03T03:06:43.671 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:06:43.704 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-routes. 2021-04-03T03:06:43.724 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2021-04-03T03:06:43.735 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-jinja2. 2021-04-03T03:06:43.755 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-04-03T03:06:43.770 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-04-03T03:06:43.802 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:06:43.999 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-04-03T03:06:44.019 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:44.057 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:44.090 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-04-03T03:06:44.097 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:44.129 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-04-03T03:06:44.135 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:44.140 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:44.171 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:44.205 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-04-03T03:06:44.226 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-04-03T03:06:44.271 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:06:44.385 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-repoze.lru. 2021-04-03T03:06:44.390 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2021-04-03T03:06:44.426 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-04-03T03:06:44.516 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-repoze.lru. 2021-04-03T03:06:44.536 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2021-04-03T03:06:44.574 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-04-03T03:06:44.683 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-joblib. 2021-04-03T03:06:44.690 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2021-04-03T03:06:44.746 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-routes. 2021-04-03T03:06:44.749 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-joblib (0.11-1) ... 2021-04-03T03:06:44.766 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2021-04-03T03:06:44.805 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-04-03T03:06:44.813 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-04-03T03:06:44.821 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:44.836 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph. 2021-04-03T03:06:44.853 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../051-ceph_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:44.862 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:44.894 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:44.902 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-routes. 2021-04-03T03:06:44.909 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2021-04-03T03:06:44.951 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-04-03T03:06:45.094 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-sklearn. 2021-04-03T03:06:45.102 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2021-04-03T03:06:45.143 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-04-03T03:06:45.149 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-04-03T03:06:45.156 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:45.182 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-numpy. 2021-04-03T03:06:45.197 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-fuse. 2021-04-03T03:06:45.197 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-04-03T03:06:45.198 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:45.204 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../052-ceph-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:45.240 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:06:45.246 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:45.262 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-04-03T03:06:45.282 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:45.329 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:45.575 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-04-03T03:06:45.583 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:45.624 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:45.632 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mds. 2021-04-03T03:06:45.650 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../053-ceph-mds_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:45.690 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:45.804 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-decorator. 2021-04-03T03:06:45.825 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2021-04-03T03:06:45.862 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-04-03T03:06:45.887 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-04-03T03:06:45.893 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:45.937 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:46.086 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-04-03T03:06:46.105 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:46.135 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package cephadm. 2021-04-03T03:06:46.143 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../054-cephadm_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:46.144 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:46.165 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-scipy. 2021-04-03T03:06:46.172 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-04-03T03:06:46.184 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:46.206 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:06:46.282 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-numpy. 2021-04-03T03:06:46.302 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-04-03T03:06:46.349 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:06:46.464 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-numpy. 2021-04-03T03:06:46.471 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-04-03T03:06:46.480 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-jinja2. 2021-04-03T03:06:46.498 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-04-03T03:06:46.513 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:06:46.546 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:06:46.686 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-monotonic. 2021-04-03T03:06:46.694 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2021-04-03T03:06:46.727 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-04-03T03:06:46.853 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-decorator. 2021-04-03T03:06:46.865 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-04-03T03:06:46.871 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2021-04-03T03:06:46.873 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:46.912 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-04-03T03:06:46.915 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:47.044 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-decorator. 2021-04-03T03:06:47.052 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2021-04-03T03:06:47.055 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-fasteners. 2021-04-03T03:06:47.062 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2021-04-03T03:06:47.065 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-04-03T03:06:47.086 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-04-03T03:06:47.093 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-04-03T03:06:47.104 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-04-03T03:06:47.123 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:06:47.209 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-repoze.lru. 2021-04-03T03:06:47.228 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2021-04-03T03:06:47.264 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-scipy. 2021-04-03T03:06:47.268 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-04-03T03:06:47.271 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-04-03T03:06:47.313 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:06:47.413 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-scipy. 2021-04-03T03:06:47.434 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-04-03T03:06:47.441 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-httplib2. 2021-04-03T03:06:47.462 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-04-03T03:06:47.471 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:06:47.499 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:06:47.510 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-joblib. 2021-04-03T03:06:47.531 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2021-04-03T03:06:47.568 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-joblib (0.11-1) ... 2021-04-03T03:06:47.571 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-routes. 2021-04-03T03:06:47.579 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2021-04-03T03:06:47.620 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-04-03T03:06:47.827 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pyasn1. 2021-04-03T03:06:47.833 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2021-04-03T03:06:47.876 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:06:47.880 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-sklearn. 2021-04-03T03:06:47.887 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2021-04-03T03:06:47.921 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-04-03T03:06:47.931 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-04-03T03:06:47.950 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:47.989 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:48.213 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-04-03T03:06:48.234 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-04-03T03:06:48.288 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:06:48.306 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-04-03T03:06:48.327 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-04-03T03:06:48.352 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-04-03T03:06:48.364 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:06:48.373 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:48.411 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:48.413 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-04-03T03:06:48.434 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-04-03T03:06:48.471 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:06:48.607 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-rsa. 2021-04-03T03:06:48.611 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2021-04-03T03:06:48.653 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-04-03T03:06:48.657 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-04-03T03:06:48.661 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:48.703 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:48.767 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-joblib. 2021-04-03T03:06:48.775 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2021-04-03T03:06:48.817 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-joblib (0.11-1) ... 2021-04-03T03:06:48.899 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-joblib. 2021-04-03T03:06:48.907 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2021-04-03T03:06:48.949 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-joblib (0.11-1) ... 2021-04-03T03:06:48.977 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-04-03T03:06:48.998 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-04-03T03:06:48.998 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-numpy. 2021-04-03T03:06:49.019 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-04-03T03:06:49.043 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:06:49.065 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:06:49.153 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-sklearn. 2021-04-03T03:06:49.174 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2021-04-03T03:06:49.212 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-04-03T03:06:49.286 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-sklearn. 2021-04-03T03:06:49.307 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2021-04-03T03:06:49.344 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-04-03T03:06:49.347 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-monotonic. 2021-04-03T03:06:49.368 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2021-04-03T03:06:49.413 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-04-03T03:06:49.439 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-oauth2client. 2021-04-03T03:06:49.446 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2021-04-03T03:06:49.488 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-04-03T03:06:49.553 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-decorator. 2021-04-03T03:06:49.574 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2021-04-03T03:06:49.611 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-04-03T03:06:49.642 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-04-03T03:06:49.649 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:49.691 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:49.741 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-fasteners. 2021-04-03T03:06:49.749 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2021-04-03T03:06:49.790 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-04-03T03:06:49.834 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-04-03T03:06:49.834 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-setuptools. 2021-04-03T03:06:49.840 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:49.854 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2021-04-03T03:06:49.875 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:49.897 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-scipy. 2021-04-03T03:06:49.900 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-04-03T03:06:49.917 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-04-03T03:06:49.955 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:06:50.118 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-httplib2. 2021-04-03T03:06:50.139 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-04-03T03:06:50.185 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:06:50.278 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-websocket. 2021-04-03T03:06:50.299 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-04-03T03:06:50.344 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:06:50.555 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pyasn1. 2021-04-03T03:06:50.563 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2021-04-03T03:06:50.604 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:06:50.612 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-monotonic. 2021-04-03T03:06:50.633 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2021-04-03T03:06:50.672 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-kubernetes. 2021-04-03T03:06:50.678 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-04-03T03:06:50.680 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-04-03T03:06:50.722 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:06:50.787 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-monotonic. 2021-04-03T03:06:50.807 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2021-04-03T03:06:50.838 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-04-03T03:06:50.845 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-04-03T03:06:50.859 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-04-03T03:06:50.904 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:06:50.983 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-04-03T03:06:51.004 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-04-03T03:06:51.006 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-fasteners. 2021-04-03T03:06:51.026 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2021-04-03T03:06:51.041 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:06:51.072 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-04-03T03:06:51.085 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-04-03T03:06:51.106 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:51.142 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:51.248 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-fasteners. 2021-04-03T03:06:51.255 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2021-04-03T03:06:51.297 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-04-03T03:06:51.333 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-joblib. 2021-04-03T03:06:51.341 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2021-04-03T03:06:51.360 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-rsa. 2021-04-03T03:06:51.367 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2021-04-03T03:06:51.383 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-joblib (0.11-1) ... 2021-04-03T03:06:51.400 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-httplib2. 2021-04-03T03:06:51.407 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-04-03T03:06:51.410 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-04-03T03:06:51.441 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:06:51.487 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-json-pointer. 2021-04-03T03:06:51.495 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2021-04-03T03:06:51.613 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-04-03T03:06:51.634 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-httplib2. 2021-04-03T03:06:51.654 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-04-03T03:06:51.700 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:06:51.739 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-04-03T03:06:51.753 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-sklearn. 2021-04-03T03:06:51.759 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-04-03T03:06:51.774 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2021-04-03T03:06:51.778 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pyasn1. 2021-04-03T03:06:51.799 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2021-04-03T03:06:51.805 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:06:51.819 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-04-03T03:06:51.836 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:06:52.016 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-jsonpatch. 2021-04-03T03:06:52.019 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pyasn1. 2021-04-03T03:06:52.024 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-04-03T03:06:52.027 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2021-04-03T03:06:52.066 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:06:52.069 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:06:52.231 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-04-03T03:06:52.234 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-04-03T03:06:52.238 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-04-03T03:06:52.241 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:52.259 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-oauth2client. 2021-04-03T03:06:52.267 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2021-04-03T03:06:52.280 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:06:52.283 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:52.308 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-04-03T03:06:52.394 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-04-03T03:06:52.398 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-04-03T03:06:52.415 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:52.418 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-04-03T03:06:52.452 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:52.456 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:06:52.583 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-rsa. 2021-04-03T03:06:52.603 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2021-04-03T03:06:52.628 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-setuptools. 2021-04-03T03:06:52.641 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-04-03T03:06:52.649 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2021-04-03T03:06:52.694 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-04-03T03:06:52.780 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libonig4:amd64. 2021-04-03T03:06:52.783 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-rsa. 2021-04-03T03:06:52.787 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2021-04-03T03:06:52.791 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2021-04-03T03:06:52.829 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:06:52.850 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-04-03T03:06:52.960 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-04-03T03:06:52.967 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-04-03T03:06:53.010 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:06:53.082 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-websocket. 2021-04-03T03:06:53.090 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-04-03T03:06:53.131 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:06:53.145 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-monotonic. 2021-04-03T03:06:53.161 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-04-03T03:06:53.165 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2021-04-03T03:06:53.182 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-04-03T03:06:53.182 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libjq1:amd64. 2021-04-03T03:06:53.203 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-04-03T03:06:53.203 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:53.227 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:06:53.248 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:06:53.415 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-oauth2client. 2021-04-03T03:06:53.433 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2021-04-03T03:06:53.451 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-kubernetes. 2021-04-03T03:06:53.472 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-04-03T03:06:53.473 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-04-03T03:06:53.510 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package jq. 2021-04-03T03:06:53.513 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-fasteners. 2021-04-03T03:06:53.517 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:06:53.521 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2021-04-03T03:06:53.531 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:53.563 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-04-03T03:06:53.576 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-04-03T03:06:53.624 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-oauth2client. 2021-04-03T03:06:53.631 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2021-04-03T03:06:53.673 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-04-03T03:06:53.801 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-setuptools. 2021-04-03T03:06:53.807 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2021-04-03T03:06:53.850 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-04-03T03:06:53.862 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package socat. 2021-04-03T03:06:53.866 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-httplib2. 2021-04-03T03:06:53.879 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-04-03T03:06:53.883 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-04-03T03:06:53.886 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-04-03T03:06:53.901 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:53.920 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:06:53.932 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:06:53.937 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:53.993 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-setuptools. 2021-04-03T03:06:54.014 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2021-04-03T03:06:54.059 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-04-03T03:06:54.211 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-websocket. 2021-04-03T03:06:54.232 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-04-03T03:06:54.252 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pyasn1. 2021-04-03T03:06:54.260 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2021-04-03T03:06:54.270 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:06:54.274 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-json-pointer. 2021-04-03T03:06:54.282 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2021-04-03T03:06:54.298 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-04-03T03:06:54.301 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:06:54.306 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-04-03T03:06:54.348 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:06:54.400 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-04-03T03:06:54.404 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-websocket. 2021-04-03T03:06:54.412 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-04-03T03:06:54.454 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:06:54.590 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-kubernetes. 2021-04-03T03:06:54.597 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-04-03T03:06:54.609 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package xmlstarlet. 2021-04-03T03:06:54.613 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-04-03T03:06:54.630 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2021-04-03T03:06:54.634 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-04-03T03:06:54.639 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:06:54.667 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-04-03T03:06:54.679 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:06:54.803 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-jsonpatch. 2021-04-03T03:06:54.811 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-04-03T03:06:54.816 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-kubernetes. 2021-04-03T03:06:54.836 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-04-03T03:06:54.854 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:06:54.882 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:06:54.993 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-04-03T03:06:54.999 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-rsa. 2021-04-03T03:06:55.005 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package ceph-test. 2021-04-03T03:06:55.006 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2021-04-03T03:06:55.014 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:55.025 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../089-ceph-test_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:55.048 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-04-03T03:06:55.059 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:55.071 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:55.181 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-04-03T03:06:55.201 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:55.240 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:55.261 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-04-03T03:06:55.280 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:55.319 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:55.369 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-04-03T03:06:55.389 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-04-03T03:06:55.395 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-json-pointer. 2021-04-03T03:06:55.403 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2021-04-03T03:06:55.427 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:06:55.522 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-04-03T03:06:55.576 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libonig4:amd64. 2021-04-03T03:06:55.583 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2021-04-03T03:06:55.625 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:06:55.663 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-json-pointer. 2021-04-03T03:06:55.671 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2021-04-03T03:06:55.790 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-04-03T03:06:55.831 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-oauth2client. 2021-04-03T03:06:55.838 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2021-04-03T03:06:55.871 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-04-03T03:06:55.908 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-jsonpatch. 2021-04-03T03:06:55.915 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-04-03T03:06:55.959 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:06:55.978 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libjq1:amd64. 2021-04-03T03:06:55.999 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:56.045 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:06:56.176 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-jsonpatch. 2021-04-03T03:06:56.183 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-setuptools. 2021-04-03T03:06:56.183 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-04-03T03:06:56.203 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2021-04-03T03:06:56.241 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-04-03T03:06:56.244 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:06:56.277 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-04-03T03:06:56.298 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:56.344 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:56.355 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package jq. 2021-04-03T03:06:56.363 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:56.405 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-04-03T03:06:56.571 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-04-03T03:06:56.591 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:56.620 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-websocket. 2021-04-03T03:06:56.627 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-04-03T03:06:56.629 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:56.659 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package socat. 2021-04-03T03:06:56.669 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:06:56.671 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libonig4:amd64. 2021-04-03T03:06:56.678 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2021-04-03T03:06:56.679 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-04-03T03:06:56.712 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:06:56.717 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:06:56.966 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libonig4:amd64. 2021-04-03T03:06:56.973 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2021-04-03T03:06:57.015 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:06:57.058 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libjq1:amd64. 2021-04-03T03:06:57.079 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:57.095 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-04-03T03:06:57.102 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-04-03T03:06:57.116 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:06:57.144 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:06:57.347 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-kubernetes. 2021-04-03T03:06:57.350 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libjq1:amd64. 2021-04-03T03:06:57.355 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:57.368 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-04-03T03:06:57.385 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package jq. 2021-04-03T03:06:57.389 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package xmlstarlet. 2021-04-03T03:06:57.400 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:06:57.405 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:06:57.406 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:57.410 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2021-04-03T03:06:57.451 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-04-03T03:06:57.455 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-04-03T03:06:57.592 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-colorama. 2021-04-03T03:06:57.599 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2021-04-03T03:06:57.636 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package jq. 2021-04-03T03:06:57.640 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:57.641 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-colorama (0.3.7-1) ... 2021-04-03T03:06:57.678 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-04-03T03:06:57.729 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package socat. 2021-04-03T03:06:57.736 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-04-03T03:06:57.759 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-04-03T03:06:57.778 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:06:57.780 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:57.825 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:57.826 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package ceph-test. 2021-04-03T03:06:57.847 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../089-ceph-test_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:57.893 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:57.910 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package cephfs-shell. 2021-04-03T03:06:57.913 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package socat. 2021-04-03T03:06:57.918 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-04-03T03:06:57.929 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../091-cephfs-shell_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:57.977 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:57.980 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:06:58.132 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-04-03T03:06:58.153 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-04-03T03:06:58.170 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-json-pointer. 2021-04-03T03:06:58.177 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2021-04-03T03:06:58.190 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:06:58.238 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libcephfs-dev. 2021-04-03T03:06:58.246 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:58.288 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:58.288 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-04-03T03:06:58.308 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-04-03T03:06:58.313 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-04-03T03:06:58.359 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:06:58.476 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package xmlstarlet. 2021-04-03T03:06:58.497 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2021-04-03T03:06:58.534 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-04-03T03:06:58.582 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package nvme-cli. 2021-04-03T03:06:58.603 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-04-03T03:06:58.620 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package xmlstarlet. 2021-04-03T03:06:58.627 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2021-04-03T03:06:58.632 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-jsonpatch. 2021-04-03T03:06:58.648 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:06:58.653 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-04-03T03:06:58.670 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-04-03T03:06:58.694 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:06:58.855 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package ceph-test. 2021-04-03T03:06:58.875 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../089-ceph-test_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:58.912 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:58.983 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package ceph-test. 2021-04-03T03:06:59.002 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../089-ceph-test_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:06:59.018 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-04-03T03:06:59.018 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python-attr. 2021-04-03T03:06:59.027 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2021-04-03T03:06:59.039 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:06:59.041 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:59.068 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python-attr (17.4.0-2) ... 2021-04-03T03:06:59.084 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:06:59.372 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python-funcsigs. 2021-04-03T03:06:59.392 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2021-04-03T03:06:59.429 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libonig4:amd64. 2021-04-03T03:06:59.436 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2021-04-03T03:06:59.438 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-04-03T03:06:59.479 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:06:59.774 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python-pluggy. 2021-04-03T03:06:59.782 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:06:59.790 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libjq1:amd64. 2021-04-03T03:06:59.811 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:06:59.823 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-04-03T03:06:59.865 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:07:00.143 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python-py. 2021-04-03T03:07:00.159 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package jq. 2021-04-03T03:07:00.164 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2021-04-03T03:07:00.167 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2021-04-03T03:07:00.201 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python-py (1.5.2-1) ... 2021-04-03T03:07:00.208 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-04-03T03:07:00.413 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-colorama. 2021-04-03T03:07:00.432 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2021-04-03T03:07:00.471 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-colorama (0.3.7-1) ... 2021-04-03T03:07:00.478 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package socat. 2021-04-03T03:07:00.499 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-04-03T03:07:00.544 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:07:00.546 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python-pytest. 2021-04-03T03:07:00.554 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:00.596 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-04-03T03:07:00.740 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package cephfs-shell. 2021-04-03T03:07:00.759 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../091-cephfs-shell_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:07:00.798 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:00.889 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-04-03T03:07:00.895 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-04-03T03:07:00.907 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-attr. 2021-04-03T03:07:00.928 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:00.939 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:07:00.965 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-04-03T03:07:01.119 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libcephfs-dev. 2021-04-03T03:07:01.188 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:01.192 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package xmlstarlet. 2021-04-03T03:07:01.213 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2021-04-03T03:07:01.250 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-04-03T03:07:01.293 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-cycler. 2021-04-03T03:07:01.300 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2021-04-03T03:07:01.342 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-04-03T03:07:01.400 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-colorama. 2021-04-03T03:07:01.419 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2021-04-03T03:07:01.451 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:01.458 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-colorama (0.3.7-1) ... 2021-04-03T03:07:01.569 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-colorama. 2021-04-03T03:07:01.577 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2021-04-03T03:07:01.587 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package ceph-test. 2021-04-03T03:07:01.594 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../089-ceph-test_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:01.619 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-colorama (0.3.7-1) ... 2021-04-03T03:07:01.620 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-webencodings. 2021-04-03T03:07:01.636 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:01.641 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2021-04-03T03:07:01.678 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-04-03T03:07:01.727 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package cephfs-shell. 2021-04-03T03:07:01.734 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../091-cephfs-shell_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:07:01.776 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:01.779 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package nvme-cli. 2021-04-03T03:07:01.799 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-04-03T03:07:01.845 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:01.863 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package cephfs-shell. 2021-04-03T03:07:01.881 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../091-cephfs-shell_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:07:01.930 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:01.990 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-html5lib. 2021-04-03T03:07:01.998 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2021-04-03T03:07:02.039 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:02.046 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libcephfs-dev. 2021-04-03T03:07:02.064 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:02.112 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:02.191 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python-attr. 2021-04-03T03:07:02.198 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:02.207 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libcephfs-dev. 2021-04-03T03:07:02.215 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:02.240 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python-attr (17.4.0-2) ... 2021-04-03T03:07:02.248 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:02.376 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-04-03T03:07:02.398 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.4_amd64.deb ... 2021-04-03T03:07:02.434 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:02.543 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package nvme-cli. 2021-04-03T03:07:02.544 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python-funcsigs. 2021-04-03T03:07:02.562 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-04-03T03:07:02.565 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2021-04-03T03:07:02.602 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:02.602 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:02.693 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package nvme-cli. 2021-04-03T03:07:02.701 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-04-03T03:07:02.855 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-tz. 2021-04-03T03:07:02.856 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:02.862 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2021-04-03T03:07:02.904 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-tz (2018.3-2) ... 2021-04-03T03:07:02.929 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python-pluggy. 2021-04-03T03:07:02.930 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python-attr. 2021-04-03T03:07:02.937 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:02.937 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:02.971 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python-attr (17.4.0-2) ... 2021-04-03T03:07:02.979 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-04-03T03:07:03.174 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-matplotlib. 2021-04-03T03:07:03.195 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-04-03T03:07:03.202 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python-attr. 2021-04-03T03:07:03.222 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:03.232 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:03.258 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python-funcsigs. 2021-04-03T03:07:03.260 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python-attr (17.4.0-2) ... 2021-04-03T03:07:03.278 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2021-04-03T03:07:03.307 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python-py. 2021-04-03T03:07:03.316 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:03.328 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2021-04-03T03:07:03.373 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python-py (1.5.2-1) ... 2021-04-03T03:07:03.578 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python-pluggy. 2021-04-03T03:07:03.596 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python-funcsigs. 2021-04-03T03:07:03.599 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:03.604 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2021-04-03T03:07:03.636 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-04-03T03:07:03.645 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:03.701 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python-pytest. 2021-04-03T03:07:03.709 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:03.750 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-04-03T03:07:03.779 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-olefile. 2021-04-03T03:07:03.785 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2021-04-03T03:07:03.828 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-04-03T03:07:03.955 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python-py. 2021-04-03T03:07:03.957 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python-pluggy. 2021-04-03T03:07:03.961 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2021-04-03T03:07:03.978 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:04.004 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python-py (1.5.2-1) ... 2021-04-03T03:07:04.023 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-04-03T03:07:04.055 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-attr. 2021-04-03T03:07:04.076 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:04.121 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-04-03T03:07:04.141 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-colorama. 2021-04-03T03:07:04.149 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2021-04-03T03:07:04.173 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pastescript. 2021-04-03T03:07:04.190 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-colorama (0.3.7-1) ... 2021-04-03T03:07:04.194 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2021-04-03T03:07:04.231 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:04.316 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python-pytest. 2021-04-03T03:07:04.334 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python-py. 2021-04-03T03:07:04.337 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:04.341 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2021-04-03T03:07:04.374 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-04-03T03:07:04.376 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python-py (1.5.2-1) ... 2021-04-03T03:07:04.435 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package cephfs-shell. 2021-04-03T03:07:04.440 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-cycler. 2021-04-03T03:07:04.447 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2021-04-03T03:07:04.452 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../091-cephfs-shell_15.2.10-137-g5feb67b5-1bionic_all.deb ... 2021-04-03T03:07:04.493 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:04.508 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-04-03T03:07:04.551 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pil:amd64. 2021-04-03T03:07:04.559 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.5_amd64.deb ... 2021-04-03T03:07:04.601 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:04.670 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python-pytest. 2021-04-03T03:07:04.691 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:04.703 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-attr. 2021-04-03T03:07:04.710 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:04.729 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-04-03T03:07:04.762 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libcephfs-dev. 2021-04-03T03:07:04.777 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-04-03T03:07:04.779 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:04.820 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:04.842 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-webencodings. 2021-04-03T03:07:04.863 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2021-04-03T03:07:04.909 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-04-03T03:07:04.946 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pluggy. 2021-04-03T03:07:04.967 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:05.012 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:05.072 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-cycler. 2021-04-03T03:07:05.090 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-attr. 2021-04-03T03:07:05.093 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2021-04-03T03:07:05.097 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:05.106 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package nvme-cli. 2021-04-03T03:07:05.114 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-04-03T03:07:05.138 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-04-03T03:07:05.139 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-04-03T03:07:05.156 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:05.237 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-html5lib. 2021-04-03T03:07:05.245 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2021-04-03T03:07:05.287 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:05.331 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-py. 2021-04-03T03:07:05.338 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2021-04-03T03:07:05.380 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-py (1.5.2-1) ... 2021-04-03T03:07:05.416 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-webencodings. 2021-04-03T03:07:05.437 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2021-04-03T03:07:05.467 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-cycler. 2021-04-03T03:07:05.474 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-04-03T03:07:05.488 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2021-04-03T03:07:05.510 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python-attr. 2021-04-03T03:07:05.529 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:05.533 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-04-03T03:07:05.568 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python-attr (17.4.0-2) ... 2021-04-03T03:07:05.615 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-04-03T03:07:05.636 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.4_amd64.deb ... 2021-04-03T03:07:05.681 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:05.735 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pyinotify. 2021-04-03T03:07:05.756 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2021-04-03T03:07:05.793 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:05.793 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-html5lib. 2021-04-03T03:07:05.801 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2021-04-03T03:07:05.843 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:05.845 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-webencodings. 2021-04-03T03:07:05.853 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2021-04-03T03:07:05.887 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python-funcsigs. 2021-04-03T03:07:05.894 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2021-04-03T03:07:05.895 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-04-03T03:07:05.936 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:06.101 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-tz. 2021-04-03T03:07:06.109 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2021-04-03T03:07:06.112 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-pytest. 2021-04-03T03:07:06.120 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:06.150 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-tz (2018.3-2) ... 2021-04-03T03:07:06.161 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-04-03T03:07:06.172 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-04-03T03:07:06.193 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.4_amd64.deb ... 2021-04-03T03:07:06.198 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python-pluggy. 2021-04-03T03:07:06.214 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-html5lib. 2021-04-03T03:07:06.217 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:06.235 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2021-04-03T03:07:06.238 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:06.256 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-04-03T03:07:06.272 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:06.454 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-matplotlib. 2021-04-03T03:07:06.473 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-simplejson. 2021-04-03T03:07:06.475 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-04-03T03:07:06.494 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2021-04-03T03:07:06.517 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python-py. 2021-04-03T03:07:06.520 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:06.536 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2021-04-03T03:07:06.539 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:06.575 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python-py (1.5.2-1) ... 2021-04-03T03:07:06.641 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-tz. 2021-04-03T03:07:06.649 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2021-04-03T03:07:06.658 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-04-03T03:07:06.666 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.4_amd64.deb ... 2021-04-03T03:07:06.690 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-tz (2018.3-2) ... 2021-04-03T03:07:06.707 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:06.859 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package python3-tk:amd64. 2021-04-03T03:07:06.878 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python-pytest. 2021-04-03T03:07:06.880 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-04-03T03:07:06.886 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:06.926 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:06.927 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-04-03T03:07:07.019 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-matplotlib. 2021-04-03T03:07:07.040 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-04-03T03:07:07.051 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-olefile. 2021-04-03T03:07:07.058 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2021-04-03T03:07:07.076 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:07.100 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-04-03T03:07:07.103 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-tz. 2021-04-03T03:07:07.124 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2021-04-03T03:07:07.170 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-tz (2018.3-2) ... 2021-04-03T03:07:07.198 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-attr. 2021-04-03T03:07:07.217 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2021-04-03T03:07:07.256 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-04-03T03:07:07.379 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package radosgw. 2021-04-03T03:07:07.387 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../115-radosgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:07.428 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:07.461 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pastescript. 2021-04-03T03:07:07.483 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2021-04-03T03:07:07.489 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-matplotlib. 2021-04-03T03:07:07.496 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-04-03T03:07:07.519 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:07.538 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:07.559 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-cycler. 2021-04-03T03:07:07.580 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2021-04-03T03:07:07.617 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-04-03T03:07:07.667 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-olefile. 2021-04-03T03:07:07.672 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2021-04-03T03:07:07.716 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-04-03T03:07:07.849 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pil:amd64. 2021-04-03T03:07:07.855 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.5_amd64.deb ... 2021-04-03T03:07:07.898 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:07.944 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-webencodings. 2021-04-03T03:07:07.951 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2021-04-03T03:07:07.994 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-04-03T03:07:08.036 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pastescript. 2021-04-03T03:07:08.057 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2021-04-03T03:07:08.085 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-olefile. 2021-04-03T03:07:08.093 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2021-04-03T03:07:08.094 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:08.134 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-04-03T03:07:08.226 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pluggy. 2021-04-03T03:07:08.247 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:08.255 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-html5lib. 2021-04-03T03:07:08.275 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2021-04-03T03:07:08.281 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package rbd-fuse. 2021-04-03T03:07:08.284 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:08.303 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../116-rbd-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:08.313 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:08.339 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:08.440 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pil:amd64. 2021-04-03T03:07:08.447 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.5_amd64.deb ... 2021-04-03T03:07:08.480 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pastescript. 2021-04-03T03:07:08.489 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:08.501 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2021-04-03T03:07:08.546 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:08.595 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-py. 2021-04-03T03:07:08.616 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2021-04-03T03:07:08.622 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-04-03T03:07:08.657 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-04-03T03:07:08.665 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.4_amd64.deb ... 2021-04-03T03:07:08.678 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-py (1.5.2-1) ... 2021-04-03T03:07:08.708 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:08.811 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:07:08.834 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pluggy. 2021-04-03T03:07:08.849 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pil:amd64. 2021-04-03T03:07:08.855 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:08.870 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.5_amd64.deb ... 2021-04-03T03:07:08.900 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:08.907 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:08.999 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pyinotify. 2021-04-03T03:07:09.006 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2021-04-03T03:07:09.048 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:09.104 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:07:09.128 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-tz. 2021-04-03T03:07:09.149 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2021-04-03T03:07:09.186 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-tz (2018.3-2) ... 2021-04-03T03:07:09.211 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-py. 2021-04-03T03:07:09.220 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2021-04-03T03:07:09.247 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:07:09.261 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-py (1.5.2-1) ... 2021-04-03T03:07:09.269 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pluggy. 2021-04-03T03:07:09.276 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:09.318 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:09.359 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-pytest. 2021-04-03T03:07:09.380 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:09.425 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-04-03T03:07:09.505 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-matplotlib. 2021-04-03T03:07:09.511 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-04-03T03:07:09.554 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:09.590 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pyinotify. 2021-04-03T03:07:09.611 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2021-04-03T03:07:09.654 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-py. 2021-04-03T03:07:09.656 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:09.673 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:09.675 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2021-04-03T03:07:09.722 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-py (1.5.2-1) ... 2021-04-03T03:07:09.754 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-simplejson. 2021-04-03T03:07:09.762 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2021-04-03T03:07:09.803 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:09.950 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:07:09.984 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-pytest. 2021-04-03T03:07:09.991 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:10.033 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-04-03T03:07:10.057 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pyinotify. 2021-04-03T03:07:10.064 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2021-04-03T03:07:10.099 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:10.110 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-olefile. 2021-04-03T03:07:10.118 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2021-04-03T03:07:10.141 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package python3-tk:amd64. 2021-04-03T03:07:10.160 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-04-03T03:07:10.162 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-04-03T03:07:10.185 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-webencodings (0.5-2) ... 2021-04-03T03:07:10.199 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:10.345 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-simplejson. 2021-04-03T03:07:10.366 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2021-04-03T03:07:10.411 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:10.419 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python-attr (17.4.0-2) ... 2021-04-03T03:07:10.426 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-pytest. 2021-04-03T03:07:10.447 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:10.471 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pastescript. 2021-04-03T03:07:10.484 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-04-03T03:07:10.492 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2021-04-03T03:07:10.529 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:10.562 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:07:10.686 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package radosgw. 2021-04-03T03:07:10.688 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-py (1.5.2-1) ... 2021-04-03T03:07:10.694 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../115-radosgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:10.732 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package python3-tk:amd64. 2021-04-03T03:07:10.735 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:10.753 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-04-03T03:07:10.798 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:10.804 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-simplejson. 2021-04-03T03:07:10.813 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2021-04-03T03:07:10.845 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:10.857 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pil:amd64. 2021-04-03T03:07:10.863 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.5_amd64.deb ... 2021-04-03T03:07:10.898 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:10.922 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-04-03T03:07:11.133 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package python3-tk:amd64. 2021-04-03T03:07:11.154 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-04-03T03:07:11.191 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:11.227 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pluggy. 2021-04-03T03:07:11.248 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2021-04-03T03:07:11.260 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package radosgw. 2021-04-03T03:07:11.268 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../115-radosgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:11.293 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:11.318 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:11.366 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-04-03T03:07:11.558 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-04-03T03:07:11.588 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-py. 2021-04-03T03:07:11.596 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2021-04-03T03:07:11.614 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package rbd-fuse. 2021-04-03T03:07:11.621 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../116-rbd-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:11.637 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-py (1.5.2-1) ... 2021-04-03T03:07:11.663 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:11.678 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package radosgw. 2021-04-03T03:07:11.685 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../115-radosgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:11.727 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:11.954 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-04-03T03:07:11.973 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pyinotify. 2021-04-03T03:07:11.995 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2021-04-03T03:07:12.032 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:12.102 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:12.138 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package rbd-fuse. 2021-04-03T03:07:12.159 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../116-rbd-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:12.204 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:12.210 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:07:12.294 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-04-03T03:07:12.368 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-pytest. 2021-04-03T03:07:12.375 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2021-04-03T03:07:12.417 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-04-03T03:07:12.486 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:07:12.501 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-04-03T03:07:12.512 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:07:12.581 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package rbd-fuse. 2021-04-03T03:07:12.596 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:07:12.602 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../116-rbd-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:12.638 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:07:12.647 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:12.701 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:07:12.738 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-simplejson. 2021-04-03T03:07:12.759 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2021-04-03T03:07:12.804 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:12.921 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-04-03T03:07:12.994 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:07:13.056 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:13.120 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:07:13.140 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package python3-tk:amd64. 2021-04-03T03:07:13.148 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-04-03T03:07:13.168 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:07:13.189 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:13.290 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:07:13.325 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:07:13.461 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:07:13.516 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-04-03T03:07:13.547 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:13.576 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-webencodings (0.5-2) ... 2021-04-03T03:07:13.587 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:07:13.652 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package radosgw. 2021-04-03T03:07:13.659 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../115-radosgw_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:13.701 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:13.767 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python-py (1.5.2-1) ... 2021-04-03T03:07:13.785 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python-attr (17.4.0-2) ... 2021-04-03T03:07:13.790 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:07:13.969 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:13.986 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:07:14.006 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:14.016 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-webencodings (0.5-2) ... 2021-04-03T03:07:14.113 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-py (1.5.2-1) ... 2021-04-03T03:07:14.252 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python-attr (17.4.0-2) ... 2021-04-03T03:07:14.262 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-04-03T03:07:14.308 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:07:14.336 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-04-03T03:07:14.410 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:07:14.414 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-04-03T03:07:14.488 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:07:14.521 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package rbd-fuse. 2021-04-03T03:07:14.534 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-webencodings (0.5-2) ... 2021-04-03T03:07:14.536 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-py (1.5.2-1) ... 2021-04-03T03:07:14.541 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../116-rbd-fuse_15.2.10-137-g5feb67b5-1bionic_amd64.deb ... 2021-04-03T03:07:14.579 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:14.613 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-04-03T03:07:14.727 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python-attr (17.4.0-2) ... 2021-04-03T03:07:14.773 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-colorama (0.3.7-1) ... 2021-04-03T03:07:14.812 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-04-03T03:07:14.833 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-04-03T03:07:14.886 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:07:14.898 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-04-03T03:07:15.024 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:07:15.025 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-04-03T03:07:15.029 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-py (1.5.2-1) ... 2021-04-03T03:07:15.109 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-04-03T03:07:15.263 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-04-03T03:07:15.274 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:07:15.276 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:15.281 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-04-03T03:07:15.402 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-04-03T03:07:15.457 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-04-03T03:07:15.501 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-04-03T03:07:15.528 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-04-03T03:07:15.528 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-04-03T03:07:15.570 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-04-03T03:07:15.692 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:15.735 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:07:15.767 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-04-03T03:07:15.820 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:07:15.884 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-04-03T03:07:15.946 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:07:15.947 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.5) ... 2021-04-03T03:07:16.001 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:16.012 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-04-03T03:07:16.126 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:07:16.215 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-04-03T03:07:16.235 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-04-03T03:07:16.253 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:07:16.263 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:16.444 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:07:16.464 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:16.474 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-webencodings (0.5-2) ... 2021-04-03T03:07:16.571 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:07:16.607 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:07:16.725 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python-attr (17.4.0-2) ... 2021-04-03T03:07:16.834 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-04-03T03:07:16.893 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-04-03T03:07:17.019 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-py (1.5.2-1) ... 2021-04-03T03:07:17.022 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:07:17.068 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python-py (1.5.2-1) ... 2021-04-03T03:07:17.142 INFO:teuthology.orchestra.run.gibba015.stdout:Adding system user cephadm....done 2021-04-03T03:07:17.240 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-04-03T03:07:17.253 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:17.254 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-04-03T03:07:17.264 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:07:17.269 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-04-03T03:07:17.454 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:07:17.466 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python-py (1.5.2-1) ... 2021-04-03T03:07:17.490 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-04-03T03:07:17.537 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-04-03T03:07:17.555 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:17.606 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-04-03T03:07:17.629 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-04-03T03:07:17.667 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:17.690 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:07:17.726 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python-py (1.5.2-1) ... 2021-04-03T03:07:17.780 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:07:17.790 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-04-03T03:07:17.907 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-04-03T03:07:17.927 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-04-03T03:07:17.935 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:17.940 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-04-03T03:07:17.999 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-ceph-argparse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:18.018 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-04-03T03:07:18.099 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-colorama (0.3.7-1) ... 2021-04-03T03:07:18.161 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:07:18.191 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:07:18.208 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-04-03T03:07:18.211 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-04-03T03:07:18.287 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-04-03T03:07:18.309 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:07:18.317 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:07:18.320 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-04-03T03:07:18.418 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-04-03T03:07:18.504 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:07:18.505 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-colorama (0.3.7-1) ... 2021-04-03T03:07:18.544 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:18.544 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-04-03T03:07:18.569 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:07:18.631 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-04-03T03:07:18.756 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:07:18.778 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-monotonic (1.1-2) ... 2021-04-03T03:07:18.786 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-04-03T03:07:18.789 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-colorama (0.3.7-1) ... 2021-04-03T03:07:18.991 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:07:19.004 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:07:19.037 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:07:19.049 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:07:19.106 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:07:19.188 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-04-03T03:07:19.233 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-04-03T03:07:19.241 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:07:19.289 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-04-03T03:07:19.340 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:19.390 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-04-03T03:07:19.459 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:07:19.485 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-04-03T03:07:19.540 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:19.583 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:07:19.607 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python-py (1.5.2-1) ... 2021-04-03T03:07:19.694 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-04-03T03:07:19.699 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-04-03T03:07:19.711 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:07:19.727 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-04-03T03:07:19.799 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:19.800 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-04-03T03:07:19.901 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-04-03T03:07:19.971 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-04-03T03:07:19.971 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:20.085 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-04-03T03:07:20.160 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:07:20.175 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-04-03T03:07:20.196 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:20.197 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:20.327 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-04-03T03:07:20.403 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:07:20.423 INFO:teuthology.orchestra.run.gibba028.stdout:Adding system user cephadm....done 2021-04-03T03:07:20.448 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:20.453 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-04-03T03:07:20.589 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-04-03T03:07:20.612 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-attr (17.4.0-2) ... 2021-04-03T03:07:20.654 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-colorama (0.3.7-1) ... 2021-04-03T03:07:20.767 INFO:teuthology.orchestra.run.gibba011.stdout:Adding system user cephadm....done 2021-04-03T03:07:20.798 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:07:20.805 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:20.839 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-04-03T03:07:20.911 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-04-03T03:07:20.924 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:20.990 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:07:21.035 INFO:teuthology.orchestra.run.gibba026.stdout:Adding system user cephadm....done 2021-04-03T03:07:21.065 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-04-03T03:07:21.075 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:07:21.120 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:07:21.196 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-04-03T03:07:21.247 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:21.274 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:07:21.293 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-04-03T03:07:21.308 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-04-03T03:07:21.389 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:07:21.400 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-04-03T03:07:21.405 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:07:21.503 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:07:21.531 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:21.550 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-04-03T03:07:21.559 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up liboath0 (2.6.1-1) ... 2021-04-03T03:07:21.629 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:07:21.631 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-04-03T03:07:21.690 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:07:21.744 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:07:21.843 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-04-03T03:07:21.861 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:21.864 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:21.899 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:07:21.933 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-04-03T03:07:22.001 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:07:22.046 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:07:22.086 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-04-03T03:07:22.132 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-monotonic (1.1-2) ... 2021-04-03T03:07:22.142 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:07:22.146 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-04-03T03:07:22.211 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:22.252 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:07:22.295 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:22.383 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:07:22.398 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:07:22.412 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-monotonic (1.1-2) ... 2021-04-03T03:07:22.462 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:22.517 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:07:22.524 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-04-03T03:07:22.621 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:07:22.680 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-monotonic (1.1-2) ... 2021-04-03T03:07:22.734 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-tz (2018.3-2) ... 2021-04-03T03:07:22.735 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:22.747 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:07:22.847 INFO:teuthology.orchestra.run.gibba031.stdout:Adding system user cephadm....done 2021-04-03T03:07:22.922 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:07:22.943 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:07:22.976 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-04-03T03:07:22.990 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:23.011 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:07:23.049 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:07:23.104 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-04-03T03:07:23.131 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-04-03T03:07:23.177 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-04-03T03:07:23.202 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:07:23.232 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:07:23.299 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:23.303 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-04-03T03:07:23.305 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-04-03T03:07:23.307 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-04-03T03:07:23.331 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-04-03T03:07:23.437 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-04-03T03:07:23.461 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-joblib (0.11-1) ... 2021-04-03T03:07:23.468 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-04-03T03:07:23.556 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:07:23.592 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:07:23.680 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-04-03T03:07:23.684 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-04-03T03:07:23.696 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-routes (2.4.1-1) ... 2021-04-03T03:07:23.707 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-04-03T03:07:23.736 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:07:23.832 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:07:23.864 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-04-03T03:07:23.885 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-04-03T03:07:23.888 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:23.966 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-04-03T03:07:23.979 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:07:24.117 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-attr (17.4.0-2) ... 2021-04-03T03:07:24.120 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:07:24.123 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-04-03T03:07:24.175 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:24.213 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-attr (17.4.0-2) ... 2021-04-03T03:07:24.321 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:07:24.351 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:24.383 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2021-04-03T03:07:24.418 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-monotonic (1.1-2) ... 2021-04-03T03:07:24.422 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:24.535 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:07:24.582 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:07:24.597 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-attr (17.4.0-2) ... 2021-04-03T03:07:24.634 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:07:24.661 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-04-03T03:07:24.787 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-04-03T03:07:24.815 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:24.861 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:07:24.894 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-04-03T03:07:24.908 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:07:24.954 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-04-03T03:07:24.983 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-04-03T03:07:25.016 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:07:25.029 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-04-03T03:07:25.106 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up liboath0 (2.6.1-1) ... 2021-04-03T03:07:25.111 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up jq (1.5+dfsg-2) ... 2021-04-03T03:07:25.127 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up liboath0 (2.6.1-1) ... 2021-04-03T03:07:25.229 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:07:25.272 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-04-03T03:07:25.295 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:07:25.307 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:07:25.342 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:07:25.344 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-04-03T03:07:25.370 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-04-03T03:07:25.417 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-04-03T03:07:25.421 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:25.433 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:25.507 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-04-03T03:07:25.611 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up liboath0 (2.6.1-1) ... 2021-04-03T03:07:25.622 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:07:25.634 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:07:25.735 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-04-03T03:07:25.742 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-04-03T03:07:25.748 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-04-03T03:07:25.779 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:07:25.889 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:25.976 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-04-03T03:07:25.986 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:07:26.016 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:07:26.076 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:26.089 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:07:26.112 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-04-03T03:07:26.142 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-04-03T03:07:26.216 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-04-03T03:07:26.235 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-attr (17.4.0-2) ... 2021-04-03T03:07:26.302 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:07:26.321 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-tz (2018.3-2) ... 2021-04-03T03:07:26.326 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-tz (2018.3-2) ... 2021-04-03T03:07:26.483 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:07:26.486 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-04-03T03:07:26.536 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:07:26.561 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:07:26.595 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:07:26.643 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-04-03T03:07:26.646 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-04-03T03:07:26.679 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-04-03T03:07:26.753 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:07:26.807 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:07:26.835 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-tz (2018.3-2) ... 2021-04-03T03:07:26.921 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:07:26.930 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-04-03T03:07:26.963 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-joblib (0.11-1) ... 2021-04-03T03:07:27.013 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-04-03T03:07:27.028 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:07:27.067 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-joblib (0.11-1) ... 2021-04-03T03:07:27.156 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python-pytest (3.3.2-2) ... 2021-04-03T03:07:27.156 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up liboath0 (2.6.1-1) ... 2021-04-03T03:07:27.262 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:07:27.264 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-routes (2.4.1-1) ... 2021-04-03T03:07:27.317 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-04-03T03:07:27.318 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-routes (2.4.1-1) ... 2021-04-03T03:07:27.390 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:07:27.418 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-04-03T03:07:27.482 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:27.496 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-joblib (0.11-1) ... 2021-04-03T03:07:27.510 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-04-03T03:07:27.561 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:27.634 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up librados2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:27.741 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-04-03T03:07:27.760 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-04-03T03:07:27.798 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-routes (2.4.1-1) ... 2021-04-03T03:07:27.812 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-04-03T03:07:27.844 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-04-03T03:07:27.970 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-04-03T03:07:27.978 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:28.009 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2021-04-03T03:07:28.058 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:28.080 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2021-04-03T03:07:28.104 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:28.230 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:28.235 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:07:28.237 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-04-03T03:07:28.317 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-04-03T03:07:28.331 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:07:28.364 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:07:28.364 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-04-03T03:07:28.470 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-04-03T03:07:28.535 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2021-04-03T03:07:28.573 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-tz (2018.3-2) ... 2021-04-03T03:07:28.583 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-04-03T03:07:28.633 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:07:28.671 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up jq (1.5+dfsg-2) ... 2021-04-03T03:07:28.727 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:07:28.797 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:07:28.799 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-04-03T03:07:28.842 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up jq (1.5+dfsg-2) ... 2021-04-03T03:07:28.876 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-04-03T03:07:28.952 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:07:28.962 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-04-03T03:07:28.983 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-04-03T03:07:29.146 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:29.176 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-joblib (0.11-1) ... 2021-04-03T03:07:29.197 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up jq (1.5+dfsg-2) ... 2021-04-03T03:07:29.323 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:07:29.461 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-routes (2.4.1-1) ... 2021-04-03T03:07:29.668 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:29.721 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-04-03T03:07:29.724 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-04-03T03:07:29.799 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:29.870 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:07:29.971 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-04-03T03:07:30.025 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:07:30.122 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:07:30.161 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:30.239 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2021-04-03T03:07:30.247 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-04-03T03:07:30.301 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:07:30.303 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:07:30.404 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:07:30.419 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-04-03T03:07:30.448 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-04-03T03:07:30.521 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:30.532 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:07:30.639 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:30.653 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:07:30.655 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:07:30.724 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-04-03T03:07:30.781 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-04-03T03:07:30.791 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python-pytest (3.3.2-2) ... 2021-04-03T03:07:30.822 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:30.924 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:30.929 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python-pytest (3.3.2-2) ... 2021-04-03T03:07:30.950 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up jq (1.5+dfsg-2) ... 2021-04-03T03:07:31.034 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:07:31.041 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:07:31.052 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-04-03T03:07:31.059 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:31.059 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:31.130 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:07:31.143 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-04-03T03:07:31.244 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up librados2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.249 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up librbd1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.257 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-04-03T03:07:31.259 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python-pytest (3.3.2-2) ... 2021-04-03T03:07:31.345 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-04-03T03:07:31.350 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.383 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up librados2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.477 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-04-03T03:07:31.509 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-04-03T03:07:31.518 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:07:31.547 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.568 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-04-03T03:07:31.596 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-04-03T03:07:31.653 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-04-03T03:07:31.673 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.727 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.754 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up librados2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.798 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:31.804 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:07:31.855 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-04-03T03:07:31.856 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.4) ... 2021-04-03T03:07:31.924 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:07:32.055 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.173 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.174 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-04-03T03:07:32.235 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:07:32.256 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.290 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.311 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.381 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.384 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-04-03T03:07:32.391 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:32.428 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:07:32.464 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-04-03T03:07:32.485 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-04-03T03:07:32.507 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:07:32.600 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.689 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:07:32.691 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:32.710 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.761 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2021-04-03T03:07:32.836 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:32.919 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:07:32.923 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-04-03T03:07:32.962 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:33.021 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python-pytest (3.3.2-2) ... 2021-04-03T03:07:33.161 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-04-03T03:07:33.255 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-04-03T03:07:33.381 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-04-03T03:07:33.412 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:33.431 INFO:teuthology.orchestra.run.gibba015.stdout:Adding group ceph....done 2021-04-03T03:07:33.443 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-04-03T03:07:33.507 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up librados2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:33.574 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:33.616 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-04-03T03:07:33.741 INFO:teuthology.orchestra.run.gibba015.stdout:Adding system user ceph....done 2021-04-03T03:07:33.809 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libcephfs2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:33.910 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libcephfs-dev (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:33.917 INFO:teuthology.orchestra.run.gibba015.stdout:Setting system user ceph properties....done 2021-04-03T03:07:33.935 INFO:teuthology.orchestra.run.gibba015.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-04-03T03:07:34.002 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-04-03T03:07:34.007 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-04-03T03:07:34.010 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-rados (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.023 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:07:34.112 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-04-03T03:07:34.138 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-04-03T03:07:34.177 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-04-03T03:07:34.241 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.338 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.367 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.398 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-04-03T03:07:34.576 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.579 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:34.579 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:34.607 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-04-03T03:07:34.612 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:07:34.694 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.781 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:07:34.804 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.829 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:34.829 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:34.866 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-04-03T03:07:34.956 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.973 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:34.999 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.019 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up librbd1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.074 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.120 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.142 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.164 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.230 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-04-03T03:07:35.271 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-04-03T03:07:35.291 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.304 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-04-03T03:07:35.328 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-04-03T03:07:35.351 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.439 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-04-03T03:07:35.443 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:35.444 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:35.458 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.478 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.516 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:07:35.613 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:35.614 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:35.630 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:35.630 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:35.775 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up librbd1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.818 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.829 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up librbd1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.901 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.929 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.935 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:35.977 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.027 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-04-03T03:07:36.039 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-04-03T03:07:36.042 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-04-03T03:07:36.078 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:36.102 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-04-03T03:07:36.129 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-04-03T03:07:36.134 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-04-03T03:07:36.161 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-04-03T03:07:36.235 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libradosstriper1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.246 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:36.247 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:36.279 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.336 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-cephfs (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.338 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:07:36.349 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:07:36.380 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.446 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.481 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.570 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up librgw2 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.580 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.607 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.681 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.715 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.832 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:36.833 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-04-03T03:07:36.835 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.841 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.841 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:36.937 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:36.967 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:37.023 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:37.023 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:37.024 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up librbd1 (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.091 INFO:teuthology.orchestra.run.gibba011.stdout:Adding group ceph....done 2021-04-03T03:07:37.149 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up cephfs-shell (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.162 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.201 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.217 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.275 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-04-03T03:07:37.289 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.328 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.347 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-04-03T03:07:37.371 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-04-03T03:07:37.403 INFO:teuthology.orchestra.run.gibba011.stdout:Adding system user ceph....done 2021-04-03T03:07:37.409 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:37.410 INFO:teuthology.orchestra.run.gibba015.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:37.414 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.437 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.538 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.540 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.552 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-04-03T03:07:37.584 INFO:teuthology.orchestra.run.gibba011.stdout:Setting system user ceph properties....done 2021-04-03T03:07:37.585 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.596 INFO:teuthology.orchestra.run.gibba011.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-04-03T03:07:37.665 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-04-03T03:07:37.685 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.800 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-04-03T03:07:37.811 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.939 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2021-04-03T03:07:37.975 INFO:teuthology.orchestra.run.gibba026.stdout:Adding group ceph....done 2021-04-03T03:07:37.984 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:37.996 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr-modules-core (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:38.074 INFO:teuthology.orchestra.run.gibba028.stdout:Adding group ceph....done 2021-04-03T03:07:38.097 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:38.114 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-04-03T03:07:38.187 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:38.187 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:38.340 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-rgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:38.426 INFO:teuthology.orchestra.run.gibba028.stdout:Adding system user ceph....done 2021-04-03T03:07:38.467 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up rbd-fuse (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:38.469 INFO:teuthology.orchestra.run.gibba026.stdout:Adding system user ceph....done 2021-04-03T03:07:38.485 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:38.577 INFO:teuthology.orchestra.run.gibba028.stdout:Setting system user ceph properties....done 2021-04-03T03:07:38.590 INFO:teuthology.orchestra.run.gibba028.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-04-03T03:07:38.592 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up python3-rbd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:38.603 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:38.641 INFO:teuthology.orchestra.run.gibba026.stdout:Setting system user ceph properties....done 2021-04-03T03:07:38.657 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-04-03T03:07:38.661 INFO:teuthology.orchestra.run.gibba026.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-04-03T03:07:38.710 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-common (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:38.731 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-04-03T03:07:38.800 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-04-03T03:07:38.807 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-04-03T03:07:38.861 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-04-03T03:07:38.996 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.023 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.066 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.120 INFO:teuthology.orchestra.run.gibba031.stdout:Adding group ceph....done 2021-04-03T03:07:39.134 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:39.135 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:39.150 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-04-03T03:07:39.193 INFO:teuthology.orchestra.run.gibba015.stdout:ureadahead will be reprofiled on next reboot 2021-04-03T03:07:39.245 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:39.246 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:39.250 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:39.290 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:39.290 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:39.305 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.415 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.481 INFO:teuthology.orchestra.run.gibba031.stdout:Adding system user ceph....done 2021-04-03T03:07:39.516 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.576 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.593 INFO:teuthology.orchestra.run.gibba015.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:39.595 DEBUG:teuthology.orchestra.run.gibba015:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-04-03T03:07:39.634 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.677 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.686 INFO:teuthology.orchestra.run.gibba031.stdout:Setting system user ceph properties....done 2021-04-03T03:07:39.694 INFO:teuthology.orchestra.run.gibba015.stdout:Reading package lists... 2021-04-03T03:07:39.695 INFO:teuthology.orchestra.run.gibba031.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-04-03T03:07:39.741 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:39.741 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:39.765 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-04-03T03:07:39.831 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-04-03T03:07:39.867 INFO:teuthology.orchestra.run.gibba015.stdout:Building dependency tree... 2021-04-03T03:07:39.867 INFO:teuthology.orchestra.run.gibba015.stdout:Reading state information... 2021-04-03T03:07:39.876 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-04-03T03:07:39.882 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-04-03T03:07:39.926 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:39.972 INFO:teuthology.orchestra.run.gibba015.stdout:libtool-bin is already the newest version (2.4.6-2). 2021-04-03T03:07:39.972 INFO:teuthology.orchestra.run.gibba015.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2021-04-03T03:07:39.972 INFO:teuthology.orchestra.run.gibba015.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2021-04-03T03:07:39.972 INFO:teuthology.orchestra.run.gibba015.stdout:The following additional packages will be installed: 2021-04-03T03:07:39.973 INFO:teuthology.orchestra.run.gibba015.stdout: libbison-dev libfl-dev libfl2 libssl1.1 zlib1g-dev 2021-04-03T03:07:39.973 INFO:teuthology.orchestra.run.gibba015.stdout:Suggested packages: 2021-04-03T03:07:39.974 INFO:teuthology.orchestra.run.gibba015.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-04-03T03:07:39.986 INFO:teuthology.orchestra.run.gibba015.stdout:The following NEW packages will be installed: 2021-04-03T03:07:39.987 INFO:teuthology.orchestra.run.gibba015.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2021-04-03T03:07:39.987 INFO:teuthology.orchestra.run.gibba015.stdout: libssl-dev zlib1g-dev 2021-04-03T03:07:39.988 INFO:teuthology.orchestra.run.gibba015.stdout:The following packages will be upgraded: 2021-04-03T03:07:39.988 INFO:teuthology.orchestra.run.gibba015.stdout: libssl1.1 2021-04-03T03:07:40.010 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.052 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.077 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up radosgw (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.098 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.158 INFO:teuthology.orchestra.run.gibba015.stdout:1 upgraded, 11 newly installed, 0 to remove and 127 not upgraded. 2021-04-03T03:07:40.159 INFO:teuthology.orchestra.run.gibba015.stdout:Need to get 4,262 kB of archives. 2021-04-03T03:07:40.159 INFO:teuthology.orchestra.run.gibba015.stdout:After this operation, 12.2 MB of additional disk space will be used. 2021-04-03T03:07:40.159 INFO:teuthology.orchestra.run.gibba015.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2021-04-03T03:07:40.186 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.204 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:40.204 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:40.251 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:40.251 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:40.254 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:40.255 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-04-03T03:07:40.320 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.396 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.447 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.465 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:40.465 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:40.528 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-test (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.538 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.589 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.613 INFO:teuthology.orchestra.run.gibba015.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.9 [1,301 kB] 2021-04-03T03:07:40.629 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-base (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.646 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.785 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:40.785 INFO:teuthology.orchestra.run.gibba011.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:40.821 INFO:teuthology.orchestra.run.gibba015.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2021-04-03T03:07:40.825 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-04-03T03:07:40.847 INFO:teuthology.orchestra.run.gibba015.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2021-04-03T03:07:40.863 INFO:teuthology.orchestra.run.gibba015.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2021-04-03T03:07:40.870 INFO:teuthology.orchestra.run.gibba015.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2021-04-03T03:07:40.874 INFO:teuthology.orchestra.run.gibba015.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2021-04-03T03:07:40.875 INFO:teuthology.orchestra.run.gibba015.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2021-04-03T03:07:40.875 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:40.876 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:40.886 INFO:teuthology.orchestra.run.gibba015.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2021-04-03T03:07:40.890 INFO:teuthology.orchestra.run.gibba015.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2021-04-03T03:07:40.891 INFO:teuthology.orchestra.run.gibba015.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2021-04-03T03:07:40.902 INFO:teuthology.orchestra.run.gibba015.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.9 [1,566 kB] 2021-04-03T03:07:40.928 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:40.928 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:40.964 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:40.989 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.057 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.065 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.125 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.145 INFO:teuthology.orchestra.run.gibba015.stdout:Preconfiguring packages ... 2021-04-03T03:07:41.166 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.168 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:41.169 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-04-03T03:07:41.200 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.233 INFO:teuthology.orchestra.run.gibba015.stdout:Fetched 4,262 kB in 1s (4,446 kB/s) 2021-04-03T03:07:41.267 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.275 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.341 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr-diskprediction-local (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.342 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2021-04-03T03:07:41.358 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package flex. 2021-04-03T03:07:41.376 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.382 INFO:teuthology.orchestra.run.gibba015.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145471 files and directories currently installed.) 2021-04-03T03:07:41.384 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2021-04-03T03:07:41.394 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.422 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking flex (2.6.4-6) ... 2021-04-03T03:07:41.476 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-osd (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.486 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.494 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:41.534 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:41.534 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:41.630 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:41.631 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:41.736 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.804 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:41.804 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-04-03T03:07:41.811 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.883 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:41.883 INFO:teuthology.orchestra.run.gibba028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:41.963 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../01-libssl1.1_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:41.977 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr-cephadm (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:41.989 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:41.990 INFO:teuthology.orchestra.run.gibba026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:42.045 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.094 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.124 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) over (1.1.1-1ubuntu2.1~18.04.6) ... 2021-04-03T03:07:42.174 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.179 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.212 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr-rook (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.290 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.321 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mds (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.388 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-04-03T03:07:42.398 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.423 INFO:teuthology.orchestra.run.gibba011.stdout:ureadahead will be reprofiled on next reboot 2021-04-03T03:07:42.423 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.463 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:42.464 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-04-03T03:07:42.472 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:42.500 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2021-04-03T03:07:42.550 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2021-04-03T03:07:42.630 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mon (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:42.650 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:42.708 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:42.802 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:42.803 INFO:teuthology.orchestra.run.gibba031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-04-03T03:07:42.805 INFO:teuthology.orchestra.run.gibba011.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:42.806 DEBUG:teuthology.orchestra.run.gibba011:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-04-03T03:07:42.903 INFO:teuthology.orchestra.run.gibba011.stdout:Reading package lists... 2021-04-03T03:07:42.982 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr-k8sevents (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:43.011 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libbison-dev:amd64. 2021-04-03T03:07:43.019 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../02-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:43.061 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:43.075 INFO:teuthology.orchestra.run.gibba011.stdout:Building dependency tree... 2021-04-03T03:07:43.076 INFO:teuthology.orchestra.run.gibba011.stdout:Reading state information... 2021-04-03T03:07:43.117 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph-mgr-dashboard (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:43.179 INFO:teuthology.orchestra.run.gibba011.stdout:libtool-bin is already the newest version (2.4.6-2). 2021-04-03T03:07:43.179 INFO:teuthology.orchestra.run.gibba011.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2021-04-03T03:07:43.179 INFO:teuthology.orchestra.run.gibba011.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2021-04-03T03:07:43.179 INFO:teuthology.orchestra.run.gibba011.stdout:The following additional packages will be installed: 2021-04-03T03:07:43.180 INFO:teuthology.orchestra.run.gibba011.stdout: libbison-dev libfl-dev libfl2 libssl1.1 zlib1g-dev 2021-04-03T03:07:43.180 INFO:teuthology.orchestra.run.gibba011.stdout:Suggested packages: 2021-04-03T03:07:43.180 INFO:teuthology.orchestra.run.gibba011.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-04-03T03:07:43.193 INFO:teuthology.orchestra.run.gibba011.stdout:The following NEW packages will be installed: 2021-04-03T03:07:43.194 INFO:teuthology.orchestra.run.gibba011.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2021-04-03T03:07:43.194 INFO:teuthology.orchestra.run.gibba011.stdout: libssl-dev zlib1g-dev 2021-04-03T03:07:43.194 INFO:teuthology.orchestra.run.gibba011.stdout:The following packages will be upgraded: 2021-04-03T03:07:43.195 INFO:teuthology.orchestra.run.gibba011.stdout: libssl1.1 2021-04-03T03:07:43.251 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up ceph (15.2.10-137-g5feb67b5-1bionic) ... 2021-04-03T03:07:43.363 INFO:teuthology.orchestra.run.gibba011.stdout:1 upgraded, 11 newly installed, 0 to remove and 127 not upgraded. 2021-04-03T03:07:43.364 INFO:teuthology.orchestra.run.gibba011.stdout:Need to get 4,262 kB of archives. 2021-04-03T03:07:43.364 INFO:teuthology.orchestra.run.gibba011.stdout:After this operation, 12.2 MB of additional disk space will be used. 2021-04-03T03:07:43.364 INFO:teuthology.orchestra.run.gibba011.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2021-04-03T03:07:43.377 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2021-04-03T03:07:43.405 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package bison. 2021-04-03T03:07:43.426 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../03-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:43.477 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:43.536 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:43.587 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-04-03T03:07:43.631 INFO:teuthology.orchestra.run.gibba028.stdout:ureadahead will be reprofiled on next reboot 2021-04-03T03:07:43.689 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:43.808 INFO:teuthology.orchestra.run.gibba011.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.9 [1,301 kB] 2021-04-03T03:07:43.877 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-04-03T03:07:43.911 INFO:teuthology.orchestra.run.gibba026.stdout:ureadahead will be reprofiled on next reboot 2021-04-03T03:07:43.923 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package dump. 2021-04-03T03:07:43.931 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../04-dump_0.4b46-3_amd64.deb ... 2021-04-03T03:07:43.953 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:43.967 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking dump (0.4b46-3) ... 2021-04-03T03:07:44.009 INFO:teuthology.orchestra.run.gibba011.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2021-04-03T03:07:44.036 INFO:teuthology.orchestra.run.gibba011.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2021-04-03T03:07:44.054 INFO:teuthology.orchestra.run.gibba011.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2021-04-03T03:07:44.056 INFO:teuthology.orchestra.run.gibba028.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:44.058 DEBUG:teuthology.orchestra.run.gibba028:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-04-03T03:07:44.060 INFO:teuthology.orchestra.run.gibba011.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2021-04-03T03:07:44.063 INFO:teuthology.orchestra.run.gibba011.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2021-04-03T03:07:44.064 INFO:teuthology.orchestra.run.gibba011.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2021-04-03T03:07:44.071 INFO:teuthology.orchestra.run.gibba011.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2021-04-03T03:07:44.074 INFO:teuthology.orchestra.run.gibba011.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2021-04-03T03:07:44.074 INFO:teuthology.orchestra.run.gibba011.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2021-04-03T03:07:44.090 INFO:teuthology.orchestra.run.gibba011.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.9 [1,566 kB] 2021-04-03T03:07:44.148 INFO:teuthology.orchestra.run.gibba028.stdout:Reading package lists... 2021-04-03T03:07:44.245 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package indent. 2021-04-03T03:07:44.266 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../05-indent_2.2.11-5_amd64.deb ... 2021-04-03T03:07:44.290 INFO:teuthology.orchestra.run.gibba026.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:44.291 DEBUG:teuthology.orchestra.run.gibba026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-04-03T03:07:44.303 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking indent (2.2.11-5) ... 2021-04-03T03:07:44.321 INFO:teuthology.orchestra.run.gibba028.stdout:Building dependency tree... 2021-04-03T03:07:44.322 INFO:teuthology.orchestra.run.gibba028.stdout:Reading state information... 2021-04-03T03:07:44.324 INFO:teuthology.orchestra.run.gibba011.stdout:Preconfiguring packages ... 2021-04-03T03:07:44.377 INFO:teuthology.orchestra.run.gibba026.stdout:Reading package lists... 2021-04-03T03:07:44.415 INFO:teuthology.orchestra.run.gibba011.stdout:Fetched 4,262 kB in 1s (4,580 kB/s) 2021-04-03T03:07:44.424 INFO:teuthology.orchestra.run.gibba028.stdout:libtool-bin is already the newest version (2.4.6-2). 2021-04-03T03:07:44.425 INFO:teuthology.orchestra.run.gibba028.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2021-04-03T03:07:44.425 INFO:teuthology.orchestra.run.gibba028.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2021-04-03T03:07:44.425 INFO:teuthology.orchestra.run.gibba028.stdout:The following additional packages will be installed: 2021-04-03T03:07:44.426 INFO:teuthology.orchestra.run.gibba028.stdout: libbison-dev libfl-dev libfl2 libssl1.1 zlib1g-dev 2021-04-03T03:07:44.426 INFO:teuthology.orchestra.run.gibba028.stdout:Suggested packages: 2021-04-03T03:07:44.426 INFO:teuthology.orchestra.run.gibba028.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-04-03T03:07:44.439 INFO:teuthology.orchestra.run.gibba028.stdout:The following NEW packages will be installed: 2021-04-03T03:07:44.440 INFO:teuthology.orchestra.run.gibba028.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2021-04-03T03:07:44.440 INFO:teuthology.orchestra.run.gibba028.stdout: libssl-dev zlib1g-dev 2021-04-03T03:07:44.441 INFO:teuthology.orchestra.run.gibba028.stdout:The following packages will be upgraded: 2021-04-03T03:07:44.441 INFO:teuthology.orchestra.run.gibba028.stdout: libssl1.1 2021-04-03T03:07:44.455 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-04-03T03:07:44.489 INFO:teuthology.orchestra.run.gibba031.stdout:ureadahead will be reprofiled on next reboot 2021-04-03T03:07:44.536 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package flex. 2021-04-03T03:07:44.539 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:44.547 INFO:teuthology.orchestra.run.gibba026.stdout:Building dependency tree... 2021-04-03T03:07:44.547 INFO:teuthology.orchestra.run.gibba026.stdout:Reading state information... 2021-04-03T03:07:44.572 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-04-03T03:07:44.578 INFO:teuthology.orchestra.run.gibba011.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145471 files and directories currently installed.) 2021-04-03T03:07:44.582 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2021-04-03T03:07:44.593 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2021-04-03T03:07:44.610 INFO:teuthology.orchestra.run.gibba028.stdout:1 upgraded, 11 newly installed, 0 to remove and 127 not upgraded. 2021-04-03T03:07:44.610 INFO:teuthology.orchestra.run.gibba028.stdout:Need to get 4,262 kB of archives. 2021-04-03T03:07:44.610 INFO:teuthology.orchestra.run.gibba028.stdout:After this operation, 12.2 MB of additional disk space will be used. 2021-04-03T03:07:44.610 INFO:teuthology.orchestra.run.gibba028.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2021-04-03T03:07:44.619 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking flex (2.6.4-6) ... 2021-04-03T03:07:44.639 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:44.646 INFO:teuthology.orchestra.run.gibba026.stdout:libtool-bin is already the newest version (2.4.6-2). 2021-04-03T03:07:44.646 INFO:teuthology.orchestra.run.gibba026.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2021-04-03T03:07:44.647 INFO:teuthology.orchestra.run.gibba026.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2021-04-03T03:07:44.647 INFO:teuthology.orchestra.run.gibba026.stdout:The following additional packages will be installed: 2021-04-03T03:07:44.647 INFO:teuthology.orchestra.run.gibba026.stdout: libbison-dev libfl-dev libfl2 libssl1.1 zlib1g-dev 2021-04-03T03:07:44.648 INFO:teuthology.orchestra.run.gibba026.stdout:Suggested packages: 2021-04-03T03:07:44.648 INFO:teuthology.orchestra.run.gibba026.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-04-03T03:07:44.660 INFO:teuthology.orchestra.run.gibba026.stdout:The following NEW packages will be installed: 2021-04-03T03:07:44.661 INFO:teuthology.orchestra.run.gibba026.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2021-04-03T03:07:44.661 INFO:teuthology.orchestra.run.gibba026.stdout: libssl-dev zlib1g-dev 2021-04-03T03:07:44.662 INFO:teuthology.orchestra.run.gibba026.stdout:The following packages will be upgraded: 2021-04-03T03:07:44.662 INFO:teuthology.orchestra.run.gibba026.stdout: libssl1.1 2021-04-03T03:07:44.830 INFO:teuthology.orchestra.run.gibba026.stdout:1 upgraded, 11 newly installed, 0 to remove and 127 not upgraded. 2021-04-03T03:07:44.830 INFO:teuthology.orchestra.run.gibba026.stdout:Need to get 4,262 kB of archives. 2021-04-03T03:07:44.831 INFO:teuthology.orchestra.run.gibba026.stdout:After this operation, 12.2 MB of additional disk space will be used. 2021-04-03T03:07:44.831 INFO:teuthology.orchestra.run.gibba026.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2021-04-03T03:07:44.877 INFO:teuthology.orchestra.run.gibba031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:44.878 DEBUG:teuthology.orchestra.run.gibba031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-04-03T03:07:44.909 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2021-04-03T03:07:44.930 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2021-04-03T03:07:44.966 INFO:teuthology.orchestra.run.gibba031.stdout:Reading package lists... 2021-04-03T03:07:44.975 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:45.072 INFO:teuthology.orchestra.run.gibba028.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.9 [1,301 kB] 2021-04-03T03:07:45.133 INFO:teuthology.orchestra.run.gibba031.stdout:Building dependency tree... 2021-04-03T03:07:45.134 INFO:teuthology.orchestra.run.gibba031.stdout:Reading state information... 2021-04-03T03:07:45.153 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../01-libssl1.1_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:45.230 INFO:teuthology.orchestra.run.gibba031.stdout:libtool-bin is already the newest version (2.4.6-2). 2021-04-03T03:07:45.230 INFO:teuthology.orchestra.run.gibba031.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2021-04-03T03:07:45.231 INFO:teuthology.orchestra.run.gibba031.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2021-04-03T03:07:45.231 INFO:teuthology.orchestra.run.gibba031.stdout:The following additional packages will be installed: 2021-04-03T03:07:45.232 INFO:teuthology.orchestra.run.gibba031.stdout: libbison-dev libfl-dev libfl2 libssl1.1 zlib1g-dev 2021-04-03T03:07:45.232 INFO:teuthology.orchestra.run.gibba031.stdout:Suggested packages: 2021-04-03T03:07:45.232 INFO:teuthology.orchestra.run.gibba031.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-04-03T03:07:45.243 INFO:teuthology.orchestra.run.gibba031.stdout:The following NEW packages will be installed: 2021-04-03T03:07:45.244 INFO:teuthology.orchestra.run.gibba031.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2021-04-03T03:07:45.245 INFO:teuthology.orchestra.run.gibba031.stdout: libssl-dev zlib1g-dev 2021-04-03T03:07:45.245 INFO:teuthology.orchestra.run.gibba031.stdout:The following packages will be upgraded: 2021-04-03T03:07:45.245 INFO:teuthology.orchestra.run.gibba031.stdout: libssl1.1 2021-04-03T03:07:45.261 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-04-03T03:07:45.269 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2021-04-03T03:07:45.283 INFO:teuthology.orchestra.run.gibba026.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.9 [1,301 kB] 2021-04-03T03:07:45.297 INFO:teuthology.orchestra.run.gibba028.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2021-04-03T03:07:45.311 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:45.320 INFO:teuthology.orchestra.run.gibba028.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2021-04-03T03:07:45.321 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) over (1.1.1-1ubuntu2.1~18.04.6) ... 2021-04-03T03:07:45.334 INFO:teuthology.orchestra.run.gibba028.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2021-04-03T03:07:45.337 INFO:teuthology.orchestra.run.gibba028.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2021-04-03T03:07:45.351 INFO:teuthology.orchestra.run.gibba028.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2021-04-03T03:07:45.351 INFO:teuthology.orchestra.run.gibba028.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2021-04-03T03:07:45.359 INFO:teuthology.orchestra.run.gibba028.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2021-04-03T03:07:45.361 INFO:teuthology.orchestra.run.gibba028.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2021-04-03T03:07:45.361 INFO:teuthology.orchestra.run.gibba028.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2021-04-03T03:07:45.378 INFO:teuthology.orchestra.run.gibba028.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.9 [1,566 kB] 2021-04-03T03:07:45.413 INFO:teuthology.orchestra.run.gibba031.stdout:1 upgraded, 11 newly installed, 0 to remove and 127 not upgraded. 2021-04-03T03:07:45.414 INFO:teuthology.orchestra.run.gibba031.stdout:Need to get 4,262 kB of archives. 2021-04-03T03:07:45.414 INFO:teuthology.orchestra.run.gibba031.stdout:After this operation, 12.2 MB of additional disk space will be used. 2021-04-03T03:07:45.414 INFO:teuthology.orchestra.run.gibba031.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2021-04-03T03:07:45.491 INFO:teuthology.orchestra.run.gibba026.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2021-04-03T03:07:45.519 INFO:teuthology.orchestra.run.gibba026.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2021-04-03T03:07:45.536 INFO:teuthology.orchestra.run.gibba026.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2021-04-03T03:07:45.542 INFO:teuthology.orchestra.run.gibba026.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2021-04-03T03:07:45.545 INFO:teuthology.orchestra.run.gibba026.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2021-04-03T03:07:45.546 INFO:teuthology.orchestra.run.gibba026.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2021-04-03T03:07:45.553 INFO:teuthology.orchestra.run.gibba026.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2021-04-03T03:07:45.556 INFO:teuthology.orchestra.run.gibba026.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2021-04-03T03:07:45.556 INFO:teuthology.orchestra.run.gibba026.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2021-04-03T03:07:45.572 INFO:teuthology.orchestra.run.gibba026.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.9 [1,566 kB] 2021-04-03T03:07:45.628 INFO:teuthology.orchestra.run.gibba028.stdout:Preconfiguring packages ... 2021-04-03T03:07:45.683 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libfl2:amd64. 2021-04-03T03:07:45.704 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2021-04-03T03:07:45.747 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:45.749 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libbison-dev:amd64. 2021-04-03T03:07:45.757 INFO:teuthology.orchestra.run.gibba028.stdout:Fetched 4,262 kB in 1s (4,312 kB/s) 2021-04-03T03:07:45.770 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../02-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:45.808 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:45.816 INFO:teuthology.orchestra.run.gibba026.stdout:Preconfiguring packages ... 2021-04-03T03:07:45.844 INFO:teuthology.orchestra.run.gibba031.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.9 [1,301 kB] 2021-04-03T03:07:45.890 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package flex. 2021-04-03T03:07:45.898 INFO:teuthology.orchestra.run.gibba026.stdout:Fetched 4,262 kB in 1s (4,450 kB/s) 2021-04-03T03:07:45.915 INFO:teuthology.orchestra.run.gibba028.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145471 files and directories currently installed.) 2021-04-03T03:07:45.917 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2021-04-03T03:07:45.954 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking flex (2.6.4-6) ... 2021-04-03T03:07:45.998 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package flex. 2021-04-03T03:07:46.016 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-04-03T03:07:46.037 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2021-04-03T03:07:46.045 INFO:teuthology.orchestra.run.gibba031.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2021-04-03T03:07:46.060 INFO:teuthology.orchestra.run.gibba026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145471 files and directories currently installed.) 2021-04-03T03:07:46.062 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2021-04-03T03:07:46.073 INFO:teuthology.orchestra.run.gibba031.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2021-04-03T03:07:46.083 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:46.091 INFO:teuthology.orchestra.run.gibba031.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2021-04-03T03:07:46.093 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package bison. 2021-04-03T03:07:46.096 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking flex (2.6.4-6) ... 2021-04-03T03:07:46.097 INFO:teuthology.orchestra.run.gibba031.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2021-04-03T03:07:46.100 INFO:teuthology.orchestra.run.gibba031.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2021-04-03T03:07:46.101 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../03-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:46.101 INFO:teuthology.orchestra.run.gibba031.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2021-04-03T03:07:46.107 INFO:teuthology.orchestra.run.gibba031.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2021-04-03T03:07:46.110 INFO:teuthology.orchestra.run.gibba031.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2021-04-03T03:07:46.110 INFO:teuthology.orchestra.run.gibba031.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2021-04-03T03:07:46.125 INFO:teuthology.orchestra.run.gibba031.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.9 [1,566 kB] 2021-04-03T03:07:46.136 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:46.344 INFO:teuthology.orchestra.run.gibba015.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-04-03T03:07:46.359 INFO:teuthology.orchestra.run.gibba031.stdout:Preconfiguring packages ... 2021-04-03T03:07:46.365 INFO:teuthology.orchestra.run.gibba015.stdout:Preparing to unpack .../11-libssl-dev_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:46.402 INFO:teuthology.orchestra.run.gibba015.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:46.456 INFO:teuthology.orchestra.run.gibba031.stdout:Fetched 4,262 kB in 1s (4,648 kB/s) 2021-04-03T03:07:46.475 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../01-libssl1.1_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:46.514 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package dump. 2021-04-03T03:07:46.533 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../04-dump_0.4b46-3_amd64.deb ... 2021-04-03T03:07:46.572 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking dump (0.4b46-3) ... 2021-04-03T03:07:46.579 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package flex. 2021-04-03T03:07:46.592 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../01-libssl1.1_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:46.619 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) over (1.1.1-1ubuntu2.1~18.04.6) ... 2021-04-03T03:07:46.640 INFO:teuthology.orchestra.run.gibba031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145471 files and directories currently installed.) 2021-04-03T03:07:46.643 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2021-04-03T03:07:46.685 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking flex (2.6.4-6) ... 2021-04-03T03:07:46.794 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) over (1.1.1-1ubuntu2.1~18.04.6) ... 2021-04-03T03:07:46.822 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up flex (2.6.4-6) ... 2021-04-03T03:07:46.834 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package indent. 2021-04-03T03:07:46.854 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../05-indent_2.2.11-5_amd64.deb ... 2021-04-03T03:07:46.892 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking indent (2.2.11-5) ... 2021-04-03T03:07:46.948 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up dump (0.4b46-3) ... 2021-04-03T03:07:46.980 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libbison-dev:amd64. 2021-04-03T03:07:47.001 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../02-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:47.038 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:47.055 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-04-03T03:07:47.102 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-04-03T03:07:47.123 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../01-libssl1.1_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:47.123 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2021-04-03T03:07:47.124 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:47.161 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:47.180 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libbison-dev:amd64. 2021-04-03T03:07:47.199 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../02-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:47.230 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:47.277 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) over (1.1.1-1ubuntu2.1~18.04.6) ... 2021-04-03T03:07:47.435 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up indent (2.2.11-5) ... 2021-04-03T03:07:47.449 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package bison. 2021-04-03T03:07:47.455 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../03-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:47.516 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package bison. 2021-04-03T03:07:47.536 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../03-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:47.561 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:47.579 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:47.623 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:47.657 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libbison-dev:amd64. 2021-04-03T03:07:47.677 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../02-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:47.687 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:47.716 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:47.913 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:47.949 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2021-04-03T03:07:47.952 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package dump. 2021-04-03T03:07:47.957 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2021-04-03T03:07:47.962 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package dump. 2021-04-03T03:07:47.973 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../04-dump_0.4b46-3_amd64.deb ... 2021-04-03T03:07:47.983 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../04-dump_0.4b46-3_amd64.deb ... 2021-04-03T03:07:47.999 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:48.005 INFO:teuthology.orchestra.run.gibba015.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-04-03T03:07:48.010 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking dump (0.4b46-3) ... 2021-04-03T03:07:48.021 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking dump (0.4b46-3) ... 2021-04-03T03:07:48.060 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package bison. 2021-04-03T03:07:48.064 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:48.067 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../03-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2021-04-03T03:07:48.111 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:48.191 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:48.249 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package indent. 2021-04-03T03:07:48.260 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-04-03T03:07:48.269 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../05-indent_2.2.11-5_amd64.deb ... 2021-04-03T03:07:48.280 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2021-04-03T03:07:48.289 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package indent. 2021-04-03T03:07:48.307 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking indent (2.2.11-5) ... 2021-04-03T03:07:48.309 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../05-indent_2.2.11-5_amd64.deb ... 2021-04-03T03:07:48.316 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:48.318 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:48.347 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking indent (2.2.11-5) ... 2021-04-03T03:07:48.442 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:48.568 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-04-03T03:07:48.569 INFO:teuthology.orchestra.run.gibba015.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:48.575 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2021-04-03T03:07:48.605 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libfl2:amd64. 2021-04-03T03:07:48.618 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:48.625 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2021-04-03T03:07:48.663 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:48.696 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:48.764 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package dump. 2021-04-03T03:07:48.785 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../04-dump_0.4b46-3_amd64.deb ... 2021-04-03T03:07:48.800 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-04-03T03:07:48.808 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2021-04-03T03:07:48.822 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking dump (0.4b46-3) ... 2021-04-03T03:07:48.842 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:48.880 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2021-04-03T03:07:48.900 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2021-04-03T03:07:48.907 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-04-03T03:07:48.928 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2021-04-03T03:07:48.938 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:48.965 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:49.100 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package indent. 2021-04-03T03:07:49.104 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2021-04-03T03:07:49.120 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../05-indent_2.2.11-5_amd64.deb ... 2021-04-03T03:07:49.125 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2021-04-03T03:07:49.158 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking indent (2.2.11-5) ... 2021-04-03T03:07:49.162 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:49.199 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-04-03T03:07:49.219 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2021-04-03T03:07:49.251 INFO:teuthology.orchestra.run.gibba011.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-04-03T03:07:49.257 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:49.259 INFO:teuthology.orchestra.run.gibba011.stdout:Preparing to unpack .../11-libssl-dev_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:49.300 INFO:teuthology.orchestra.run.gibba011.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:49.427 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-04-03T03:07:49.448 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2021-04-03T03:07:49.465 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-04-03T03:07:49.486 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2021-04-03T03:07:49.494 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:49.531 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:49.594 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libfl2:amd64. 2021-04-03T03:07:49.601 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2021-04-03T03:07:49.643 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:49.756 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2021-04-03T03:07:49.777 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2021-04-03T03:07:49.782 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up flex (2.6.4-6) ... 2021-04-03T03:07:49.822 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:49.834 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libfl2:amd64. 2021-04-03T03:07:49.842 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2021-04-03T03:07:49.875 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:49.906 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up dump (0.4b46-3) ... 2021-04-03T03:07:49.912 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-04-03T03:07:49.933 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2021-04-03T03:07:49.948 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2021-04-03T03:07:49.970 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:50.013 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-04-03T03:07:50.082 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:50.111 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-04-03T03:07:50.116 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-04-03T03:07:50.116 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2021-04-03T03:07:50.122 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2021-04-03T03:07:50.153 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:50.158 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:50.165 INFO:teuthology.orchestra.run.gibba015.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:50.240 INFO:teuthology.orchestra.run.gibba028.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-04-03T03:07:50.260 INFO:teuthology.orchestra.run.gibba028.stdout:Preparing to unpack .../11-libssl-dev_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:50.298 INFO:teuthology.orchestra.run.gibba028.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:50.308 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up indent (2.2.11-5) ... 2021-04-03T03:07:50.397 INFO:teuthology.orchestra.run.gibba026.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-04-03T03:07:50.409 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:50.418 INFO:teuthology.orchestra.run.gibba026.stdout:Preparing to unpack .../11-libssl-dev_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:50.455 INFO:teuthology.orchestra.run.gibba026.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:50.510 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:50.511 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libfl2:amd64. 2021-04-03T03:07:50.531 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2021-04-03T03:07:50.557 INFO:teuthology.orchestra.run.gibba015.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:50.559 DEBUG:teuthology.parallel:result is None 2021-04-03T03:07:50.569 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:50.611 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:50.728 INFO:teuthology.orchestra.run.gibba011.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-04-03T03:07:50.743 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up flex (2.6.4-6) ... 2021-04-03T03:07:50.796 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:50.847 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-04-03T03:07:50.862 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up dump (0.4b46-3) ... 2021-04-03T03:07:50.868 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2021-04-03T03:07:50.914 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:50.922 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:50.933 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up flex (2.6.4-6) ... 2021-04-03T03:07:50.965 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-04-03T03:07:51.029 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:51.048 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:51.060 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up dump (0.4b46-3) ... 2021-04-03T03:07:51.168 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-04-03T03:07:51.174 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:51.199 INFO:teuthology.orchestra.run.gibba031.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-04-03T03:07:51.207 INFO:teuthology.orchestra.run.gibba031.stdout:Preparing to unpack .../11-libssl-dev_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... 2021-04-03T03:07:51.237 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:51.249 INFO:teuthology.orchestra.run.gibba031.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:51.284 INFO:teuthology.orchestra.run.gibba011.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:51.306 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up indent (2.2.11-5) ... 2021-04-03T03:07:51.385 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:51.432 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:51.471 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up indent (2.2.11-5) ... 2021-04-03T03:07:51.574 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:51.597 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:51.701 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:51.719 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up flex (2.6.4-6) ... 2021-04-03T03:07:51.742 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:51.792 INFO:teuthology.orchestra.run.gibba028.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-04-03T03:07:51.853 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:51.853 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up dump (0.4b46-3) ... 2021-04-03T03:07:51.866 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:51.957 INFO:teuthology.orchestra.run.gibba026.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-04-03T03:07:51.959 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-04-03T03:07:51.979 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:52.026 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:52.030 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:52.105 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:52.152 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:52.231 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:52.278 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:52.281 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up indent (2.2.11-5) ... 2021-04-03T03:07:52.320 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2021-04-03T03:07:52.356 INFO:teuthology.orchestra.run.gibba028.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:52.405 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:52.407 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:52.483 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:52.530 INFO:teuthology.orchestra.run.gibba026.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:52.533 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2021-04-03T03:07:52.607 INFO:teuthology.orchestra.run.gibba011.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:52.651 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2021-04-03T03:07:52.657 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:52.742 INFO:teuthology.orchestra.run.gibba031.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-04-03T03:07:52.802 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2021-04-03T03:07:52.928 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2021-04-03T03:07:52.959 INFO:teuthology.orchestra.run.gibba011.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:52.961 DEBUG:teuthology.parallel:result is None 2021-04-03T03:07:53.054 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.9) ... 2021-04-03T03:07:53.171 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2021-04-03T03:07:53.298 INFO:teuthology.orchestra.run.gibba031.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2021-04-03T03:07:53.525 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-04-03T03:07:53.527 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2021-04-03T03:07:53.759 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2021-04-03T03:07:53.863 INFO:teuthology.orchestra.run.gibba028.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:54.085 INFO:teuthology.orchestra.run.gibba026.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:54.218 INFO:teuthology.orchestra.run.gibba028.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:54.219 DEBUG:teuthology.parallel:result is None 2021-04-03T03:07:54.441 INFO:teuthology.orchestra.run.gibba026.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:54.443 DEBUG:teuthology.parallel:result is None 2021-04-03T03:07:54.801 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2021-04-03T03:07:55.054 INFO:teuthology.orchestra.run.gibba031.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2021-04-03T03:07:55.392 INFO:teuthology.orchestra.run.gibba031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-04-03T03:07:55.393 DEBUG:teuthology.parallel:result is None 2021-04-03T03:07:55.394 DEBUG:teuthology.orchestra.run.gibba011:> sudo lsb_release -is 2021-04-03T03:07:55.426 INFO:teuthology.orchestra.run.gibba011.stdout:Ubuntu 2021-04-03T03:07:55.430 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:07:55.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:07:55.573 DEBUG:teuthology.orchestra.run.gibba011:> dpkg-query -W -f '${Version}' ceph 2021-04-03T03:07:55.621 INFO:teuthology.orchestra.run.gibba011.stdout:15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:55.621 INFO:teuthology.packaging:The installed version of ceph is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:55.622 INFO:teuthology.task.install:The correct ceph version 15.2.10-137-g5feb67b5-1bionic is installed. 2021-04-03T03:07:55.622 DEBUG:teuthology.orchestra.run.gibba015:> sudo lsb_release -is 2021-04-03T03:07:55.653 INFO:teuthology.orchestra.run.gibba015.stdout:Ubuntu 2021-04-03T03:07:55.656 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:07:55.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:07:55.786 DEBUG:teuthology.orchestra.run.gibba015:> dpkg-query -W -f '${Version}' ceph 2021-04-03T03:07:55.834 INFO:teuthology.orchestra.run.gibba015.stdout:15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:55.834 INFO:teuthology.packaging:The installed version of ceph is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:55.834 INFO:teuthology.task.install:The correct ceph version 15.2.10-137-g5feb67b5-1bionic is installed. 2021-04-03T03:07:55.835 DEBUG:teuthology.orchestra.run.gibba026:> sudo lsb_release -is 2021-04-03T03:07:55.865 INFO:teuthology.orchestra.run.gibba026.stdout:Ubuntu 2021-04-03T03:07:55.868 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:07:55.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:07:55.996 DEBUG:teuthology.orchestra.run.gibba026:> dpkg-query -W -f '${Version}' ceph 2021-04-03T03:07:56.011 INFO:teuthology.orchestra.run.gibba026.stdout:15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:56.011 INFO:teuthology.packaging:The installed version of ceph is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:56.012 INFO:teuthology.task.install:The correct ceph version 15.2.10-137-g5feb67b5-1bionic is installed. 2021-04-03T03:07:56.012 DEBUG:teuthology.orchestra.run.gibba028:> sudo lsb_release -is 2021-04-03T03:07:56.042 INFO:teuthology.orchestra.run.gibba028.stdout:Ubuntu 2021-04-03T03:07:56.045 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:07:56.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:07:56.177 DEBUG:teuthology.orchestra.run.gibba028:> dpkg-query -W -f '${Version}' ceph 2021-04-03T03:07:56.193 INFO:teuthology.orchestra.run.gibba028.stdout:15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:56.193 INFO:teuthology.packaging:The installed version of ceph is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:56.194 INFO:teuthology.task.install:The correct ceph version 15.2.10-137-g5feb67b5-1bionic is installed. 2021-04-03T03:07:56.194 DEBUG:teuthology.orchestra.run.gibba031:> sudo lsb_release -is 2021-04-03T03:07:56.224 INFO:teuthology.orchestra.run.gibba031.stdout:Ubuntu 2021-04-03T03:07:56.227 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-04-03T03:07:56.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=5feb67b5bda898b8c739180363606432d4f39831 2021-04-03T03:07:56.356 DEBUG:teuthology.orchestra.run.gibba031:> dpkg-query -W -f '${Version}' ceph 2021-04-03T03:07:56.378 INFO:teuthology.orchestra.run.gibba031.stdout:15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:56.378 INFO:teuthology.packaging:The installed version of ceph is 15.2.10-137-g5feb67b5-1bionic 2021-04-03T03:07:56.378 INFO:teuthology.task.install:The correct ceph version 15.2.10-137-g5feb67b5-1bionic is installed. 2021-04-03T03:07:56.379 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-04-03T03:07:56.382 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:07:56.382 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-04-03T03:07:56.393 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:07:56.394 DEBUG:teuthology.orchestra.run.gibba015:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-04-03T03:07:56.405 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:07:56.405 DEBUG:teuthology.orchestra.run.gibba026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-04-03T03:07:56.416 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:07:56.416 DEBUG:teuthology.orchestra.run.gibba028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-04-03T03:07:56.436 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:07:56.437 DEBUG:teuthology.orchestra.run.gibba031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-04-03T03:07:56.455 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-04-03T03:07:56.455 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:07:56.456 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/usr/bin/daemon-helper 2021-04-03T03:07:56.464 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-04-03T03:07:56.514 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:07:56.514 DEBUG:teuthology.orchestra.run.gibba015:> sudo dd of=/usr/bin/daemon-helper 2021-04-03T03:07:56.524 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-04-03T03:07:56.573 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:07:56.574 DEBUG:teuthology.orchestra.run.gibba026:> sudo dd of=/usr/bin/daemon-helper 2021-04-03T03:07:56.584 DEBUG:teuthology.orchestra.run.gibba026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-04-03T03:07:56.632 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:07:56.632 DEBUG:teuthology.orchestra.run.gibba028:> sudo dd of=/usr/bin/daemon-helper 2021-04-03T03:07:56.643 DEBUG:teuthology.orchestra.run.gibba028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-04-03T03:07:56.692 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:07:56.692 DEBUG:teuthology.orchestra.run.gibba031:> sudo dd of=/usr/bin/daemon-helper 2021-04-03T03:07:56.704 DEBUG:teuthology.orchestra.run.gibba031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-04-03T03:07:56.751 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-04-03T03:07:56.751 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:07:56.752 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/usr/bin/adjust-ulimits 2021-04-03T03:07:56.763 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-04-03T03:07:56.810 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:07:56.810 DEBUG:teuthology.orchestra.run.gibba015:> sudo dd of=/usr/bin/adjust-ulimits 2021-04-03T03:07:56.821 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-04-03T03:07:56.869 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:07:56.870 DEBUG:teuthology.orchestra.run.gibba026:> sudo dd of=/usr/bin/adjust-ulimits 2021-04-03T03:07:56.881 DEBUG:teuthology.orchestra.run.gibba026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-04-03T03:07:56.932 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:07:56.932 DEBUG:teuthology.orchestra.run.gibba028:> sudo dd of=/usr/bin/adjust-ulimits 2021-04-03T03:07:56.943 DEBUG:teuthology.orchestra.run.gibba028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-04-03T03:07:56.992 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:07:56.992 DEBUG:teuthology.orchestra.run.gibba031:> sudo dd of=/usr/bin/adjust-ulimits 2021-04-03T03:07:57.003 DEBUG:teuthology.orchestra.run.gibba031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-04-03T03:07:57.051 INFO:teuthology.run_tasks:Running task ceph... 2021-04-03T03:07:57.123 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-04-03T03:07:57.124 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 777 /var/log/ceph 2021-04-03T03:07:57.127 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 777 /var/log/ceph 2021-04-03T03:07:57.130 DEBUG:teuthology.orchestra.run.gibba026:> sudo chmod 777 /var/log/ceph 2021-04-03T03:07:57.133 DEBUG:teuthology.orchestra.run.gibba028:> sudo chmod 777 /var/log/ceph 2021-04-03T03:07:57.136 DEBUG:teuthology.orchestra.run.gibba031:> sudo chmod 777 /var/log/ceph 2021-04-03T03:07:57.153 INFO:tasks.ceph:Disabling ceph logrotate... 2021-04-03T03:07:57.154 DEBUG:teuthology.orchestra.run.gibba011:> sudo rm -f -- /etc/logrotate.d/ceph 2021-04-03T03:07:57.177 DEBUG:teuthology.orchestra.run.gibba015:> sudo rm -f -- /etc/logrotate.d/ceph 2021-04-03T03:07:57.180 DEBUG:teuthology.orchestra.run.gibba026:> sudo rm -f -- /etc/logrotate.d/ceph 2021-04-03T03:07:57.187 DEBUG:teuthology.orchestra.run.gibba028:> sudo rm -f -- /etc/logrotate.d/ceph 2021-04-03T03:07:57.199 DEBUG:teuthology.orchestra.run.gibba031:> sudo rm -f -- /etc/logrotate.d/ceph 2021-04-03T03:07:57.208 INFO:tasks.ceph:Creating extra log directories... 2021-04-03T03:07:57.208 DEBUG:teuthology.orchestra.run.gibba011:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-04-03T03:07:57.225 DEBUG:teuthology.orchestra.run.gibba015:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-04-03T03:07:57.229 DEBUG:teuthology.orchestra.run.gibba026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-04-03T03:07:57.235 DEBUG:teuthology.orchestra.run.gibba028:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-04-03T03:07:57.251 DEBUG:teuthology.orchestra.run.gibba031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-04-03T03:07:57.262 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2021-04-03T03:07:57.262 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2021-04-03T03:07:57.263 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2021-04-03T03:07:57.264 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:07:57.264 DEBUG:teuthology.orchestra.run.gibba011:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-04-03T03:07:57.295 DEBUG:teuthology.orchestra.run.gibba011:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-04-03T03:07:57.357 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:07:57.358 DEBUG:teuthology.orchestra.run.gibba015:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-04-03T03:07:57.367 DEBUG:teuthology.orchestra.run.gibba015:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-04-03T03:07:57.426 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:07:57.426 DEBUG:teuthology.orchestra.run.gibba026:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-04-03T03:07:57.435 DEBUG:teuthology.orchestra.run.gibba026:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-04-03T03:07:57.493 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:07:57.494 DEBUG:teuthology.orchestra.run.gibba028:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-04-03T03:07:57.503 DEBUG:teuthology.orchestra.run.gibba028:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-04-03T03:07:57.564 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:07:57.564 DEBUG:teuthology.orchestra.run.gibba031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-04-03T03:07:57.573 DEBUG:teuthology.orchestra.run.gibba031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-04-03T03:07:57.630 INFO:tasks.ceph:Creating ceph cluster ceph... 2021-04-03T03:07:57.631 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-04-03T03:07:57.631 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396', 'branch': 'octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '6016396', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'gibba', 'name': 'teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '5feb67b5bda898b8c739180363606432d4f39831'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '5feb67b5bda898b8c739180363606432d4f39831'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'octopus', 'sha1': '5feb67b5bda898b8c739180363606432d4f39831'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1'], ['client.2']], 'sha1': '5feb67b5bda898b8c739180363606432d4f39831', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '5feb67b5bda898b8c739180363606432d4f39831', 'targets': {'gibba011.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6Y6vlWddm8prpu69iW8w+0Xj1ViCGO5eSrP31z2EPpVGK5c1Zw+qt/bed4Qa9Oaw+hJXm3zY5Uvx5h0Knf4YUrHcdr+XWK4FQ5Z/RxgFDYeZbPlOVZYXEfe9EPYYFKtwa+sPL2gKmmVu/qQRkcJ0+K/AO1ZCE09Nki29UVFkCay3Zrr0EkmUuwGIgNglWmwheQb3hbq8JQLtlBCSnev9smhbYAfXgjqmZgl4Wpme6MBYvzkrgE7nL+YdDv8HHRfoeuFrrs4DgDTGzfTYIgKvEYiPELEn2PYv15tvsJVjdVnasZ+905qlgObZP62sOve0Kap5eNM9mXN/fbs4ImpYf', 'gibba015.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPASR2jxoPMxMglFCeXssE+skMFNq2HJCFV8IIvSQOF6JnO3vxPyffX6xI02mBjoRdalu0dm7iOwXMXoRHswN4a9hSchKtMiV4stmRtC5TN49uB1xuBId8PruyPJCKaAbpoOA5Xd4Gppvf/xx1L42umabhWuhf1sw+asCFMr9NUdcZCN3Qs5uU5M9h2Fj0FX5DdKxi4rBJBfnXQFIinLzGpZKmqECTKZrh1pPtz2iiDsrV9ZJdq/lgsgCyqEc8Obi1k+kX5XLjenibTqLD7LTMbrJtZ2VxyoNarBZWfNV+mVUztMcnKvNHEN8p/1IrlbNDtmOM3d6b6W229ymFf151', 'gibba026.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC25XRVxi10grxW2V0+jAq9v8qB4FIZ6qm1DkQ3VcBl8MRJni2PZNP4foc/7sv5kTflBFdhTFkzAtIUcJ4F4BdaYaakV5+lKzoR3tq5s9JWlJg3YOEeHl1WOfkWcNmS4UD5W+ziVZwvBtUhWIbJmExPdWiHQthiMazPEZw07LGHisYvZUlhr1Ya69ON7LnL1BP8rAf0/B1voLKVLcBMZyj2B/f6EvQ/Fq4mtF08UsYBUeHKjnnHCfGuknoLviK8V0nOo2J5d8m8voaiJv+u4SBwz4VR8oqoSLIouyZDOo4jlkNIsaDR0oFZDDVIYTFX6o+xuGcX9wTtDSKSINOJa0ad', 'gibba028.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYcK4PoB7Wx9U2bM549upyWOVecrKGoclr+3W56PyhnCgosNR+vfeKOZBQq0pDpaLgyGlFRx/4rugnYMz/g4yEW4Usfy+SbN0DmkEOeSVJg9D0W8GjAu1rFtCERKK+UMP7Ti/2ZguzOOQB0Flu4aGMfnwRCFGyRROvWRC+lA4bIMbHdBlMGhmv/beD0uZXFN9IlDMuTvuh5dYNtZ/7oFMTFqZXxJSdUWZZpgZ2D4s7vbJQbTzJrZkF/6C11rrb3MmuZ6sgBdt3+BrtkWUM1wUP3YiZT8W7IgMgEHu2eMaKx7HKEXU+gQZL4VHb6Yyc5qvfxIs7LEfRFvwmznMkQU3/', 'gibba031.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDESPwZOhAoh6gZ2+uiCjzA/B55OF/cnBcW9c99UmEjVQxLlTBe5HqWVMSb1u18m7oRiFg3NJEAH59rWYH+/518gP4aDw7SW73ZAzdOFdvYVJEsr2DDFerb6fsjGKuvdjJsaqN+poqHUWDZA1hCkEWwuQA/hDFt91IzF69wjGCYKUk6uIQDyenjO/6xN3ZUzrsIt6ltyDOZYR4xfr4TFnuIK7GXsx/qjNlDOPo5avsaTmJq9F75w0fh18Hu9AjzazaupJg019RFwdh2m6vtT1R7S9MGuBGgqi21fF1VOWtks2txIn6h8cJSvBMgVI1jbyfhB0O6dwrQqMussNk6clbJ'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '5feb67b5bda898b8c739180363606432d4f39831'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'master', 'teuthology_sha1': '85d61eae4759f46ce21e9a37cd816a7a1a66c9d5', 'timestamp': '2021-04-02_04:10:03', 'tube': 'gibba', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.gibba.13276'} 2021-04-03T03:07:57.632 DEBUG:teuthology.orchestra.run.gibba011:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-04-03T03:07:57.635 DEBUG:teuthology.orchestra.run.gibba015:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-04-03T03:07:57.638 DEBUG:teuthology.orchestra.run.gibba026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-04-03T03:07:57.641 DEBUG:teuthology.orchestra.run.gibba028:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-04-03T03:07:57.643 DEBUG:teuthology.orchestra.run.gibba031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-04-03T03:07:57.680 DEBUG:teuthology.orchestra.run.gibba011:> sudo install -d -m0777 -- /var/run/ceph 2021-04-03T03:07:57.682 DEBUG:teuthology.orchestra.run.gibba015:> sudo install -d -m0777 -- /var/run/ceph 2021-04-03T03:07:57.688 DEBUG:teuthology.orchestra.run.gibba026:> sudo install -d -m0777 -- /var/run/ceph 2021-04-03T03:07:57.690 DEBUG:teuthology.orchestra.run.gibba028:> sudo install -d -m0777 -- /var/run/ceph 2021-04-03T03:07:57.694 DEBUG:teuthology.orchestra.run.gibba031:> sudo install -d -m0777 -- /var/run/ceph 2021-04-03T03:07:57.731 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:07:57.732 DEBUG:teuthology.orchestra.run.gibba011:> dd if=/scratch_devs of=/dev/stdout 2021-04-03T03:07:57.736 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-04-03T03:07:57.736 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-04-03T03:07:57.787 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-04-03T03:07:57.787 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:57.787 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 336 Links: 1 2021-04-03T03:07:57.787 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:57.788 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-04-03 03:04:33.941682376 +0000 2021-04-03T03:07:57.788 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-04-03 03:04:33.869718376 +0000 2021-04-03T03:07:57.788 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-04-03 03:04:33.869718376 +0000 2021-04-03T03:07:57.788 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-04-03T03:07:57.794 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-04-03T03:07:57.794 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-04-03T03:07:57.794 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.000263023 s, 1.9 MB/s 2021-04-03T03:07:57.796 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-04-03T03:07:57.843 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-04-03T03:07:57.843 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:57.843 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 343 Links: 1 2021-04-03T03:07:57.844 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:57.844 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-04-03 03:04:34.073616374 +0000 2021-04-03T03:07:57.844 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-04-03 03:04:34.017644375 +0000 2021-04-03T03:07:57.844 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-04-03 03:04:34.017644375 +0000 2021-04-03T03:07:57.844 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-04-03T03:07:57.847 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-04-03T03:07:57.847 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-04-03T03:07:57.847 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.000164734 s, 3.1 MB/s 2021-04-03T03:07:57.849 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-04-03T03:07:57.900 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-04-03T03:07:57.900 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:57.900 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 349 Links: 1 2021-04-03T03:07:57.900 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:57.901 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-04-03 03:04:34.209548373 +0000 2021-04-03T03:07:57.901 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-04-03 03:04:34.137584374 +0000 2021-04-03T03:07:57.901 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-04-03 03:04:34.137584374 +0000 2021-04-03T03:07:57.901 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-04-03T03:07:57.912 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-04-03T03:07:57.912 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-04-03T03:07:57.912 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.000468913 s, 1.1 MB/s 2021-04-03T03:07:57.918 DEBUG:teuthology.orchestra.run.gibba011:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-04-03T03:07:57.968 INFO:teuthology.orchestra.run.gibba011.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-04-03T03:07:57.968 INFO:teuthology.orchestra.run.gibba011.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:57.968 INFO:teuthology.orchestra.run.gibba011.stdout:Device: 6h/6d Inode: 355 Links: 1 2021-04-03T03:07:57.968 INFO:teuthology.orchestra.run.gibba011.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:57.969 INFO:teuthology.orchestra.run.gibba011.stdout:Access: 2021-04-03 03:04:34.349478372 +0000 2021-04-03T03:07:57.969 INFO:teuthology.orchestra.run.gibba011.stdout:Modify: 2021-04-03 03:04:34.289508373 +0000 2021-04-03T03:07:57.969 INFO:teuthology.orchestra.run.gibba011.stdout:Change: 2021-04-03 03:04:34.289508373 +0000 2021-04-03T03:07:57.969 INFO:teuthology.orchestra.run.gibba011.stdout: Birth: - 2021-04-03T03:07:57.975 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records in 2021-04-03T03:07:57.975 INFO:teuthology.orchestra.run.gibba011.stderr:1+0 records out 2021-04-03T03:07:57.975 INFO:teuthology.orchestra.run.gibba011.stderr:512 bytes copied, 0.000146503 s, 3.5 MB/s 2021-04-03T03:07:57.977 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-04-03T03:07:57.977 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:07:57.977 DEBUG:teuthology.orchestra.run.gibba015:> dd if=/scratch_devs of=/dev/stdout 2021-04-03T03:07:57.986 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-04-03T03:07:57.986 DEBUG:teuthology.orchestra.run.gibba015:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-04-03T03:07:58.030 INFO:teuthology.orchestra.run.gibba015.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-04-03T03:07:58.030 INFO:teuthology.orchestra.run.gibba015.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:58.030 INFO:teuthology.orchestra.run.gibba015.stdout:Device: 6h/6d Inode: 338 Links: 1 2021-04-03T03:07:58.030 INFO:teuthology.orchestra.run.gibba015.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:58.031 INFO:teuthology.orchestra.run.gibba015.stdout:Access: 2021-04-03 03:04:42.604967450 +0000 2021-04-03T03:07:58.031 INFO:teuthology.orchestra.run.gibba015.stdout:Modify: 2021-04-03 03:04:42.533003450 +0000 2021-04-03T03:07:58.031 INFO:teuthology.orchestra.run.gibba015.stdout:Change: 2021-04-03 03:04:42.533003450 +0000 2021-04-03T03:07:58.031 INFO:teuthology.orchestra.run.gibba015.stdout: Birth: - 2021-04-03T03:07:58.033 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records in 2021-04-03T03:07:58.033 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records out 2021-04-03T03:07:58.033 INFO:teuthology.orchestra.run.gibba015.stderr:512 bytes copied, 0.00024482 s, 2.1 MB/s 2021-04-03T03:07:58.034 DEBUG:teuthology.orchestra.run.gibba015:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-04-03T03:07:58.086 INFO:teuthology.orchestra.run.gibba015.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-04-03T03:07:58.086 INFO:teuthology.orchestra.run.gibba015.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:58.086 INFO:teuthology.orchestra.run.gibba015.stdout:Device: 6h/6d Inode: 345 Links: 1 2021-04-03T03:07:58.087 INFO:teuthology.orchestra.run.gibba015.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:58.087 INFO:teuthology.orchestra.run.gibba015.stdout:Access: 2021-04-03 03:04:42.776881449 +0000 2021-04-03T03:07:58.087 INFO:teuthology.orchestra.run.gibba015.stdout:Modify: 2021-04-03 03:04:42.684927450 +0000 2021-04-03T03:07:58.087 INFO:teuthology.orchestra.run.gibba015.stdout:Change: 2021-04-03 03:04:42.684927450 +0000 2021-04-03T03:07:58.087 INFO:teuthology.orchestra.run.gibba015.stdout: Birth: - 2021-04-03T03:07:58.093 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records in 2021-04-03T03:07:58.093 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records out 2021-04-03T03:07:58.094 INFO:teuthology.orchestra.run.gibba015.stderr:512 bytes copied, 0.00022641 s, 2.3 MB/s 2021-04-03T03:07:58.095 DEBUG:teuthology.orchestra.run.gibba015:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-04-03T03:07:58.142 INFO:teuthology.orchestra.run.gibba015.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-04-03T03:07:58.143 INFO:teuthology.orchestra.run.gibba015.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:58.143 INFO:teuthology.orchestra.run.gibba015.stdout:Device: 6h/6d Inode: 351 Links: 1 2021-04-03T03:07:58.143 INFO:teuthology.orchestra.run.gibba015.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:58.143 INFO:teuthology.orchestra.run.gibba015.stdout:Access: 2021-04-03 03:04:42.924807449 +0000 2021-04-03T03:07:58.144 INFO:teuthology.orchestra.run.gibba015.stdout:Modify: 2021-04-03 03:04:42.860839449 +0000 2021-04-03T03:07:58.144 INFO:teuthology.orchestra.run.gibba015.stdout:Change: 2021-04-03 03:04:42.860839449 +0000 2021-04-03T03:07:58.144 INFO:teuthology.orchestra.run.gibba015.stdout: Birth: - 2021-04-03T03:07:58.147 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records in 2021-04-03T03:07:58.147 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records out 2021-04-03T03:07:58.148 INFO:teuthology.orchestra.run.gibba015.stderr:512 bytes copied, 0.000297571 s, 1.7 MB/s 2021-04-03T03:07:58.150 DEBUG:teuthology.orchestra.run.gibba015:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-04-03T03:07:58.199 INFO:teuthology.orchestra.run.gibba015.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-04-03T03:07:58.200 INFO:teuthology.orchestra.run.gibba015.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-04-03T03:07:58.200 INFO:teuthology.orchestra.run.gibba015.stdout:Device: 6h/6d Inode: 357 Links: 1 2021-04-03T03:07:58.200 INFO:teuthology.orchestra.run.gibba015.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-04-03T03:07:58.200 INFO:teuthology.orchestra.run.gibba015.stdout:Access: 2021-04-03 03:04:43.068735449 +0000 2021-04-03T03:07:58.200 INFO:teuthology.orchestra.run.gibba015.stdout:Modify: 2021-04-03 03:04:43.004767449 +0000 2021-04-03T03:07:58.201 INFO:teuthology.orchestra.run.gibba015.stdout:Change: 2021-04-03 03:04:43.004767449 +0000 2021-04-03T03:07:58.201 INFO:teuthology.orchestra.run.gibba015.stdout: Birth: - 2021-04-03T03:07:58.206 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records in 2021-04-03T03:07:58.207 INFO:teuthology.orchestra.run.gibba015.stderr:1+0 records out 2021-04-03T03:07:58.207 INFO:teuthology.orchestra.run.gibba015.stderr:512 bytes copied, 0.000154531 s, 3.3 MB/s 2021-04-03T03:07:58.208 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-04-03T03:07:58.208 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@gibba011.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba015.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-04-03T03:07:58.209 INFO:tasks.ceph:Generating config... 2021-04-03T03:07:58.211 INFO:tasks.ceph:[client] client mount timeout = 600 2021-04-03T03:07:58.212 INFO:tasks.ceph:[client] debug client = 20 2021-04-03T03:07:58.212 INFO:tasks.ceph:[client] debug ms = 1 2021-04-03T03:07:58.212 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2021-04-03T03:07:58.212 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2021-04-03T03:07:58.213 INFO:tasks.ceph:[mds] debug mds = 20 2021-04-03T03:07:58.213 INFO:tasks.ceph:[mds] debug ms = 1 2021-04-03T03:07:58.213 INFO:tasks.ceph:[mds] mds bal frag = True 2021-04-03T03:07:58.214 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2021-04-03T03:07:58.214 INFO:tasks.ceph:[mds] mds bal merge size = 5 2021-04-03T03:07:58.214 INFO:tasks.ceph:[mds] mds bal split bits = 3 2021-04-03T03:07:58.214 INFO:tasks.ceph:[mds] mds bal split size = 100 2021-04-03T03:07:58.215 INFO:tasks.ceph:[mds] mds debug frag = True 2021-04-03T03:07:58.215 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2021-04-03T03:07:58.215 INFO:tasks.ceph:[mds] mds op complaint time = 180 2021-04-03T03:07:58.216 INFO:tasks.ceph:[mds] mds verify scatter = True 2021-04-03T03:07:58.216 INFO:tasks.ceph:[mds] osd op complaint time = 180 2021-04-03T03:07:58.216 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2021-04-03T03:07:58.216 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2021-04-03T03:07:58.217 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-04-03T03:07:58.217 INFO:tasks.ceph:[mgr] debug ms = 1 2021-04-03T03:07:58.217 INFO:tasks.ceph:[mon] debug mon = 20 2021-04-03T03:07:58.218 INFO:tasks.ceph:[mon] debug ms = 1 2021-04-03T03:07:58.218 INFO:tasks.ceph:[mon] debug paxos = 20 2021-04-03T03:07:58.218 INFO:tasks.ceph:[mon] mon op complaint time = 120 2021-04-03T03:07:58.218 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-04-03T03:07:58.219 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2021-04-03T03:07:58.219 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-04-03T03:07:58.219 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-04-03T03:07:58.220 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-04-03T03:07:58.220 INFO:tasks.ceph:[osd] debug ms = 1 2021-04-03T03:07:58.220 INFO:tasks.ceph:[osd] debug osd = 20 2021-04-03T03:07:58.221 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-04-03T03:07:58.221 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-04-03T03:07:58.221 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-04-03T03:07:58.221 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-04-03T03:07:58.222 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-04-03T03:07:58.222 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-04-03T03:07:58.222 INFO:tasks.ceph:[osd] osd op complaint time = 180 2021-04-03T03:07:58.223 INFO:tasks.ceph:Setting up mon.a... 2021-04-03T03:07:58.223 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2021-04-03T03:07:58.240 INFO:teuthology.orchestra.run.gibba011.stdout:creating /etc/ceph/ceph.keyring 2021-04-03T03:07:58.241 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2021-04-03T03:07:58.299 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-04-03T03:07:58.352 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.2.111'), ('mon.b', '172.21.2.115'), ('mon.c', '[v2:172.21.2.115:3301,v1:172.21.2.115:6790]')] 2021-04-03T03:07:58.352 DEBUG:teuthology.orchestra.run.gibba011:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.2.111 --add b 172.21.2.115 --addv c '[v2:172.21.2.115:3301,v1:172.21.2.115:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:07:58.421 INFO:teuthology.orchestra.run.gibba011.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:07:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:monmaptool: generated fsid 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:07:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:epoch 0 2021-04-03T03:07:58.422 INFO:teuthology.orchestra.run.gibba011.stdout:fsid 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:07:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:last_changed 2021-04-03T03:07:58.417047+0000 2021-04-03T03:07:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:created 2021-04-03T03:07:58.417047+0000 2021-04-03T03:07:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:min_mon_release 0 (unknown) 2021-04-03T03:07:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:0: [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] mon.a 2021-04-03T03:07:58.423 INFO:teuthology.orchestra.run.gibba011.stdout:1: [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] mon.b 2021-04-03T03:07:58.424 INFO:teuthology.orchestra.run.gibba011.stdout:2: [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] mon.c 2021-04-03T03:07:58.424 INFO:teuthology.orchestra.run.gibba011.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2021-04-03T03:07:58.425 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6fa64cfd-c000-43a2-869e-5811f8b4928c... 2021-04-03T03:07:58.426 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:07:58.469 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:07:58.473 DEBUG:teuthology.orchestra.run.gibba026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:07:58.476 DEBUG:teuthology.orchestra.run.gibba028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:07:58.480 DEBUG:teuthology.orchestra.run.gibba031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:07:58.484 INFO:teuthology.orchestra.run.gibba011.stdout:[global] 2021-04-03T03:07:58.485 INFO:teuthology.orchestra.run.gibba011.stdout: chdir = "" 2021-04-03T03:07:58.485 INFO:teuthology.orchestra.run.gibba011.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:07:58.486 INFO:teuthology.orchestra.run.gibba011.stdout: auth supported = cephx 2021-04-03T03:07:58.486 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.486 INFO:teuthology.orchestra.run.gibba011.stdout: filestore xattr use omap = true 2021-04-03T03:07:58.486 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.487 INFO:teuthology.orchestra.run.gibba011.stdout: mon clock drift allowed = 1.000 2021-04-03T03:07:58.487 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.487 INFO:teuthology.orchestra.run.gibba011.stdout: osd crush chooseleaf type = 0 2021-04-03T03:07:58.488 INFO:teuthology.orchestra.run.gibba011.stdout: auth debug = true 2021-04-03T03:07:58.488 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.488 INFO:teuthology.orchestra.run.gibba011.stdout: ms die on old message = true 2021-04-03T03:07:58.489 INFO:teuthology.orchestra.run.gibba011.stdout: ms die on bug = true 2021-04-03T03:07:58.489 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.489 INFO:teuthology.orchestra.run.gibba011.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:07:58.490 INFO:teuthology.orchestra.run.gibba011.stdout: mon pg warn max object skew = 0 2021-04-03T03:07:58.490 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.490 INFO:teuthology.orchestra.run.gibba011.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:07:58.490 INFO:teuthology.orchestra.run.gibba011.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:07:58.490 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.491 INFO:teuthology.orchestra.run.gibba011.stdout: osd pool default size = 2 2021-04-03T03:07:58.491 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.491 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd allow primary affinity = true 2021-04-03T03:07:58.491 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd allow pg remap = true 2021-04-03T03:07:58.492 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:07:58.492 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:07:58.492 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on no sortbitwise = false 2021-04-03T03:07:58.492 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:07:58.492 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on too few osds = false 2021-04-03T03:07:58.493 INFO:teuthology.orchestra.run.gibba011.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:07:58.493 INFO:teuthology.orchestra.run.gibba011.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:07:58.493 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.493 INFO:teuthology.orchestra.run.gibba011.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:07:58.493 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.494 INFO:teuthology.orchestra.run.gibba011.stdout: osd default data pool replay window = 5 2021-04-03T03:07:58.494 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.494 INFO:teuthology.orchestra.run.gibba011.stdout: mon allow pool delete = true 2021-04-03T03:07:58.494 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.494 INFO:teuthology.orchestra.run.gibba011.stdout: mon cluster log file level = debug 2021-04-03T03:07:58.495 INFO:teuthology.orchestra.run.gibba011.stdout: debug asserts on shutdown = true 2021-04-03T03:07:58.495 INFO:teuthology.orchestra.run.gibba011.stdout: mon health detail to clog = false 2021-04-03T03:07:58.495 INFO:teuthology.orchestra.run.gibba011.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:07:58.495 INFO:teuthology.orchestra.run.gibba011.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:07:58.496 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.496 INFO:teuthology.orchestra.run.gibba011.stdout:[osd] 2021-04-03T03:07:58.496 INFO:teuthology.orchestra.run.gibba011.stdout: osd journal size = 100 2021-04-03T03:07:58.496 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.496 INFO:teuthology.orchestra.run.gibba011.stdout: osd scrub load threshold = 5.0 2021-04-03T03:07:58.497 INFO:teuthology.orchestra.run.gibba011.stdout: osd scrub max interval = 600 2021-04-03T03:07:58.497 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.497 INFO:teuthology.orchestra.run.gibba011.stdout: osd recover clone overlap = true 2021-04-03T03:07:58.497 INFO:teuthology.orchestra.run.gibba011.stdout: osd recovery max chunk = 1048576 2021-04-03T03:07:58.497 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.498 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug shutdown = true 2021-04-03T03:07:58.498 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug op order = true 2021-04-03T03:07:58.498 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug verify stray on activate = true 2021-04-03T03:07:58.498 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.499 INFO:teuthology.orchestra.run.gibba011.stdout: osd open classes on start = true 2021-04-03T03:07:58.499 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug pg log writeout = true 2021-04-03T03:07:58.499 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.499 INFO:teuthology.orchestra.run.gibba011.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:07:58.499 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.500 INFO:teuthology.orchestra.run.gibba011.stdout: osd map max advance = 10 2021-04-03T03:07:58.500 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.500 INFO:teuthology.orchestra.run.gibba011.stdout: journal zero on create = true 2021-04-03T03:07:58.500 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.500 INFO:teuthology.orchestra.run.gibba011.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:07:58.501 INFO:teuthology.orchestra.run.gibba011.stdout: filestore apply finisher threads = 3 2021-04-03T03:07:58.501 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.501 INFO:teuthology.orchestra.run.gibba011.stdout: bdev debug aio = true 2021-04-03T03:07:58.501 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug misdirected ops = true 2021-04-03T03:07:58.501 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore block size = 96636764160 2021-04-03T03:07:58.502 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore compression mode = aggressive 2021-04-03T03:07:58.502 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore fsck on mount = True 2021-04-03T03:07:58.502 INFO:teuthology.orchestra.run.gibba011.stdout: debug bluefs = 20 2021-04-03T03:07:58.502 INFO:teuthology.orchestra.run.gibba011.stdout: debug bluestore = 20 2021-04-03T03:07:58.503 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:07:58.503 INFO:teuthology.orchestra.run.gibba011.stdout: debug osd = 20 2021-04-03T03:07:58.503 INFO:teuthology.orchestra.run.gibba011.stdout: debug rocksdb = 10 2021-04-03T03:07:58.503 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:07:58.503 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd full ratio = 0.9 2021-04-03T03:07:58.504 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:07:58.504 INFO:teuthology.orchestra.run.gibba011.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:07:58.504 INFO:teuthology.orchestra.run.gibba011.stdout: osd objectstore = bluestore 2021-04-03T03:07:58.504 INFO:teuthology.orchestra.run.gibba011.stdout: osd op complaint time = 180 2021-04-03T03:07:58.504 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.505 INFO:teuthology.orchestra.run.gibba011.stdout:[mgr] 2021-04-03T03:07:58.505 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:07:58.505 INFO:teuthology.orchestra.run.gibba011.stdout: debug mgr = 20 2021-04-03T03:07:58.505 INFO:teuthology.orchestra.run.gibba011.stdout: debug mon = 20 2021-04-03T03:07:58.506 INFO:teuthology.orchestra.run.gibba011.stdout: debug auth = 20 2021-04-03T03:07:58.506 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.506 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.506 INFO:teuthology.orchestra.run.gibba011.stdout: mgr/telemetry/nag = false 2021-04-03T03:07:58.506 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.507 INFO:teuthology.orchestra.run.gibba011.stdout:[mon] 2021-04-03T03:07:58.507 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:07:58.507 INFO:teuthology.orchestra.run.gibba011.stdout: debug mon = 20 2021-04-03T03:07:58.507 INFO:teuthology.orchestra.run.gibba011.stdout: debug paxos = 20 2021-04-03T03:07:58.507 INFO:teuthology.orchestra.run.gibba011.stdout: debug auth = 20 2021-04-03T03:07:58.508 INFO:teuthology.orchestra.run.gibba011.stdout: mon data avail warn = 5 2021-04-03T03:07:58.508 INFO:teuthology.orchestra.run.gibba011.stdout: mon mgr mkfs grace = 240 2021-04-03T03:07:58.508 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.508 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd reporter subtree level = osd 2021-04-03T03:07:58.508 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd prime pg temp = true 2021-04-03T03:07:58.509 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.509 INFO:teuthology.orchestra.run.gibba011.stdout: mon op complaint time = 120 2021-04-03T03:07:58.509 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:07:58.509 INFO:teuthology.orchestra.run.gibba011.stdout:[client] 2021-04-03T03:07:58.509 INFO:teuthology.orchestra.run.gibba011.stdout: rgw cache enabled = true 2021-04-03T03:07:58.510 INFO:teuthology.orchestra.run.gibba011.stdout: rgw enable ops log = true 2021-04-03T03:07:58.510 INFO:teuthology.orchestra.run.gibba011.stdout: rgw enable usage log = true 2021-04-03T03:07:58.510 INFO:teuthology.orchestra.run.gibba011.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:07:58.510 INFO:teuthology.orchestra.run.gibba011.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:07:58.510 INFO:teuthology.orchestra.run.gibba011.stdout: client mount timeout = 600 2021-04-03T03:07:58.511 INFO:teuthology.orchestra.run.gibba011.stdout: debug client = 20 2021-04-03T03:07:58.511 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:07:58.511 INFO:teuthology.orchestra.run.gibba011.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.511 INFO:teuthology.orchestra.run.gibba011.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.512 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.a] 2021-04-03T03:07:58.512 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.b] 2021-04-03T03:07:58.512 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.c] 2021-04-03T03:07:58.512 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.a] 2021-04-03T03:07:58.512 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.b] 2021-04-03T03:07:58.513 INFO:teuthology.orchestra.run.gibba011.stdout:[mds] 2021-04-03T03:07:58.513 INFO:teuthology.orchestra.run.gibba011.stdout: debug mds = 20 2021-04-03T03:07:58.513 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:07:58.513 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal frag = True 2021-04-03T03:07:58.513 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal fragment size max = 10000 2021-04-03T03:07:58.514 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal merge size = 5 2021-04-03T03:07:58.514 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal split bits = 3 2021-04-03T03:07:58.514 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal split size = 100 2021-04-03T03:07:58.514 INFO:teuthology.orchestra.run.gibba011.stdout: mds debug frag = True 2021-04-03T03:07:58.514 INFO:teuthology.orchestra.run.gibba011.stdout: mds debug scatterstat = True 2021-04-03T03:07:58.515 INFO:teuthology.orchestra.run.gibba011.stdout: mds op complaint time = 180 2021-04-03T03:07:58.515 INFO:teuthology.orchestra.run.gibba011.stdout: mds verify scatter = True 2021-04-03T03:07:58.515 INFO:teuthology.orchestra.run.gibba011.stdout: osd op complaint time = 180 2021-04-03T03:07:58.515 INFO:teuthology.orchestra.run.gibba011.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.516 INFO:teuthology.orchestra.run.gibba011.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.521 INFO:teuthology.orchestra.run.gibba028.stdout:[global] 2021-04-03T03:07:58.521 INFO:teuthology.orchestra.run.gibba028.stdout: chdir = "" 2021-04-03T03:07:58.521 INFO:teuthology.orchestra.run.gibba028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:07:58.522 INFO:teuthology.orchestra.run.gibba028.stdout: auth supported = cephx 2021-04-03T03:07:58.522 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.522 INFO:teuthology.orchestra.run.gibba028.stdout: filestore xattr use omap = true 2021-04-03T03:07:58.522 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.523 INFO:teuthology.orchestra.run.gibba028.stdout: mon clock drift allowed = 1.000 2021-04-03T03:07:58.523 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.523 INFO:teuthology.orchestra.run.gibba028.stdout: osd crush chooseleaf type = 0 2021-04-03T03:07:58.523 INFO:teuthology.orchestra.run.gibba028.stdout: auth debug = true 2021-04-03T03:07:58.523 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.524 INFO:teuthology.orchestra.run.gibba028.stdout: ms die on old message = true 2021-04-03T03:07:58.524 INFO:teuthology.orchestra.run.gibba028.stdout: ms die on bug = true 2021-04-03T03:07:58.524 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.524 INFO:teuthology.orchestra.run.gibba028.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:07:58.525 INFO:teuthology.orchestra.run.gibba028.stdout: mon pg warn max object skew = 0 2021-04-03T03:07:58.525 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.525 INFO:teuthology.orchestra.run.gibba028.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:07:58.525 INFO:teuthology.orchestra.run.gibba028.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:07:58.525 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.526 INFO:teuthology.orchestra.run.gibba028.stdout: osd pool default size = 2 2021-04-03T03:07:58.526 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.526 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd allow primary affinity = true 2021-04-03T03:07:58.526 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd allow pg remap = true 2021-04-03T03:07:58.526 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:07:58.527 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:07:58.527 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on no sortbitwise = false 2021-04-03T03:07:58.527 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:07:58.527 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on too few osds = false 2021-04-03T03:07:58.528 INFO:teuthology.orchestra.run.gibba028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:07:58.528 INFO:teuthology.orchestra.run.gibba028.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:07:58.528 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.528 INFO:teuthology.orchestra.run.gibba028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:07:58.529 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.529 INFO:teuthology.orchestra.run.gibba028.stdout: osd default data pool replay window = 5 2021-04-03T03:07:58.529 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.529 INFO:teuthology.orchestra.run.gibba028.stdout: mon allow pool delete = true 2021-04-03T03:07:58.529 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.530 INFO:teuthology.orchestra.run.gibba028.stdout: mon cluster log file level = debug 2021-04-03T03:07:58.530 INFO:teuthology.orchestra.run.gibba028.stdout: debug asserts on shutdown = true 2021-04-03T03:07:58.530 INFO:teuthology.orchestra.run.gibba028.stdout: mon health detail to clog = false 2021-04-03T03:07:58.530 INFO:teuthology.orchestra.run.gibba028.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:07:58.531 INFO:teuthology.orchestra.run.gibba028.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:07:58.531 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.531 INFO:teuthology.orchestra.run.gibba028.stdout:[osd] 2021-04-03T03:07:58.531 INFO:teuthology.orchestra.run.gibba028.stdout: osd journal size = 100 2021-04-03T03:07:58.531 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.532 INFO:teuthology.orchestra.run.gibba028.stdout: osd scrub load threshold = 5.0 2021-04-03T03:07:58.532 INFO:teuthology.orchestra.run.gibba028.stdout: osd scrub max interval = 600 2021-04-03T03:07:58.532 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.532 INFO:teuthology.orchestra.run.gibba028.stdout: osd recover clone overlap = true 2021-04-03T03:07:58.533 INFO:teuthology.orchestra.run.gibba028.stdout: osd recovery max chunk = 1048576 2021-04-03T03:07:58.533 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.533 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug shutdown = true 2021-04-03T03:07:58.533 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug op order = true 2021-04-03T03:07:58.533 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug verify stray on activate = true 2021-04-03T03:07:58.534 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.534 INFO:teuthology.orchestra.run.gibba028.stdout: osd open classes on start = true 2021-04-03T03:07:58.534 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug pg log writeout = true 2021-04-03T03:07:58.534 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.534 INFO:teuthology.orchestra.run.gibba028.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:07:58.535 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.535 INFO:teuthology.orchestra.run.gibba028.stdout: osd map max advance = 10 2021-04-03T03:07:58.535 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.535 INFO:teuthology.orchestra.run.gibba028.stdout: journal zero on create = true 2021-04-03T03:07:58.535 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.536 INFO:teuthology.orchestra.run.gibba028.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:07:58.536 INFO:teuthology.orchestra.run.gibba028.stdout: filestore apply finisher threads = 3 2021-04-03T03:07:58.536 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.536 INFO:teuthology.orchestra.run.gibba028.stdout: bdev debug aio = true 2021-04-03T03:07:58.537 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug misdirected ops = true 2021-04-03T03:07:58.537 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore block size = 96636764160 2021-04-03T03:07:58.537 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore compression mode = aggressive 2021-04-03T03:07:58.537 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore fsck on mount = True 2021-04-03T03:07:58.537 INFO:teuthology.orchestra.run.gibba028.stdout: debug bluefs = 20 2021-04-03T03:07:58.538 INFO:teuthology.orchestra.run.gibba028.stdout: debug bluestore = 20 2021-04-03T03:07:58.538 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:07:58.538 INFO:teuthology.orchestra.run.gibba028.stdout: debug osd = 20 2021-04-03T03:07:58.538 INFO:teuthology.orchestra.run.gibba028.stdout: debug rocksdb = 10 2021-04-03T03:07:58.538 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:07:58.539 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd full ratio = 0.9 2021-04-03T03:07:58.539 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:07:58.539 INFO:teuthology.orchestra.run.gibba028.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:07:58.539 INFO:teuthology.orchestra.run.gibba028.stdout: osd objectstore = bluestore 2021-04-03T03:07:58.539 INFO:teuthology.orchestra.run.gibba028.stdout: osd op complaint time = 180 2021-04-03T03:07:58.540 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.540 INFO:teuthology.orchestra.run.gibba028.stdout:[mgr] 2021-04-03T03:07:58.540 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:07:58.540 INFO:teuthology.orchestra.run.gibba028.stdout: debug mgr = 20 2021-04-03T03:07:58.541 INFO:teuthology.orchestra.run.gibba028.stdout: debug mon = 20 2021-04-03T03:07:58.541 INFO:teuthology.orchestra.run.gibba028.stdout: debug auth = 20 2021-04-03T03:07:58.541 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.541 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.541 INFO:teuthology.orchestra.run.gibba028.stdout: mgr/telemetry/nag = false 2021-04-03T03:07:58.542 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.542 INFO:teuthology.orchestra.run.gibba028.stdout:[mon] 2021-04-03T03:07:58.542 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:07:58.542 INFO:teuthology.orchestra.run.gibba028.stdout: debug mon = 20 2021-04-03T03:07:58.542 INFO:teuthology.orchestra.run.gibba028.stdout: debug paxos = 20 2021-04-03T03:07:58.543 INFO:teuthology.orchestra.run.gibba028.stdout: debug auth = 20 2021-04-03T03:07:58.543 INFO:teuthology.orchestra.run.gibba028.stdout: mon data avail warn = 5 2021-04-03T03:07:58.543 INFO:teuthology.orchestra.run.gibba028.stdout: mon mgr mkfs grace = 240 2021-04-03T03:07:58.543 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.543 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd reporter subtree level = osd 2021-04-03T03:07:58.544 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd prime pg temp = true 2021-04-03T03:07:58.544 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.544 INFO:teuthology.orchestra.run.gibba028.stdout: mon op complaint time = 120 2021-04-03T03:07:58.544 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:07:58.545 INFO:teuthology.orchestra.run.gibba028.stdout:[client] 2021-04-03T03:07:58.545 INFO:teuthology.orchestra.run.gibba028.stdout: rgw cache enabled = true 2021-04-03T03:07:58.545 INFO:teuthology.orchestra.run.gibba028.stdout: rgw enable ops log = true 2021-04-03T03:07:58.545 INFO:teuthology.orchestra.run.gibba028.stdout: rgw enable usage log = true 2021-04-03T03:07:58.545 INFO:teuthology.orchestra.run.gibba028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:07:58.546 INFO:teuthology.orchestra.run.gibba028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:07:58.546 INFO:teuthology.orchestra.run.gibba028.stdout: client mount timeout = 600 2021-04-03T03:07:58.546 INFO:teuthology.orchestra.run.gibba028.stdout: debug client = 20 2021-04-03T03:07:58.546 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:07:58.546 INFO:teuthology.orchestra.run.gibba028.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.547 INFO:teuthology.orchestra.run.gibba028.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.547 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.a] 2021-04-03T03:07:58.547 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.b] 2021-04-03T03:07:58.547 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.c] 2021-04-03T03:07:58.547 INFO:teuthology.orchestra.run.gibba028.stdout:[mds.a] 2021-04-03T03:07:58.548 INFO:teuthology.orchestra.run.gibba028.stdout:[mds.b] 2021-04-03T03:07:58.548 INFO:teuthology.orchestra.run.gibba028.stdout:[mds] 2021-04-03T03:07:58.548 INFO:teuthology.orchestra.run.gibba028.stdout: debug mds = 20 2021-04-03T03:07:58.548 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:07:58.549 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal frag = True 2021-04-03T03:07:58.549 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal fragment size max = 10000 2021-04-03T03:07:58.549 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal merge size = 5 2021-04-03T03:07:58.549 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal split bits = 3 2021-04-03T03:07:58.549 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal split size = 100 2021-04-03T03:07:58.550 INFO:teuthology.orchestra.run.gibba028.stdout: mds debug frag = True 2021-04-03T03:07:58.550 INFO:teuthology.orchestra.run.gibba028.stdout: mds debug scatterstat = True 2021-04-03T03:07:58.550 INFO:teuthology.orchestra.run.gibba028.stdout: mds op complaint time = 180 2021-04-03T03:07:58.550 INFO:teuthology.orchestra.run.gibba028.stdout: mds verify scatter = True 2021-04-03T03:07:58.550 INFO:teuthology.orchestra.run.gibba028.stdout: osd op complaint time = 180 2021-04-03T03:07:58.551 INFO:teuthology.orchestra.run.gibba028.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.551 INFO:teuthology.orchestra.run.gibba028.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.551 INFO:teuthology.orchestra.run.gibba026.stdout:[global] 2021-04-03T03:07:58.551 INFO:teuthology.orchestra.run.gibba026.stdout: chdir = "" 2021-04-03T03:07:58.552 INFO:teuthology.orchestra.run.gibba026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:07:58.552 INFO:teuthology.orchestra.run.gibba026.stdout: auth supported = cephx 2021-04-03T03:07:58.552 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.552 INFO:teuthology.orchestra.run.gibba026.stdout: filestore xattr use omap = true 2021-04-03T03:07:58.552 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.553 INFO:teuthology.orchestra.run.gibba026.stdout: mon clock drift allowed = 1.000 2021-04-03T03:07:58.553 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.553 INFO:teuthology.orchestra.run.gibba026.stdout: osd crush chooseleaf type = 0 2021-04-03T03:07:58.553 INFO:teuthology.orchestra.run.gibba026.stdout: auth debug = true 2021-04-03T03:07:58.553 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.554 INFO:teuthology.orchestra.run.gibba026.stdout: ms die on old message = true 2021-04-03T03:07:58.554 INFO:teuthology.orchestra.run.gibba026.stdout: ms die on bug = true 2021-04-03T03:07:58.554 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.554 INFO:teuthology.orchestra.run.gibba026.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:07:58.554 INFO:teuthology.orchestra.run.gibba026.stdout: mon pg warn max object skew = 0 2021-04-03T03:07:58.555 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.555 INFO:teuthology.orchestra.run.gibba026.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:07:58.555 INFO:teuthology.orchestra.run.gibba026.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:07:58.555 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.556 INFO:teuthology.orchestra.run.gibba026.stdout: osd pool default size = 2 2021-04-03T03:07:58.556 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.556 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd allow primary affinity = true 2021-04-03T03:07:58.556 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd allow pg remap = true 2021-04-03T03:07:58.556 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:07:58.557 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:07:58.557 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on no sortbitwise = false 2021-04-03T03:07:58.557 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:07:58.557 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on too few osds = false 2021-04-03T03:07:58.558 INFO:teuthology.orchestra.run.gibba026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:07:58.558 INFO:teuthology.orchestra.run.gibba026.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:07:58.558 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.558 INFO:teuthology.orchestra.run.gibba026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:07:58.559 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.559 INFO:teuthology.orchestra.run.gibba026.stdout: osd default data pool replay window = 5 2021-04-03T03:07:58.559 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.559 INFO:teuthology.orchestra.run.gibba026.stdout: mon allow pool delete = true 2021-04-03T03:07:58.559 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.560 INFO:teuthology.orchestra.run.gibba026.stdout: mon cluster log file level = debug 2021-04-03T03:07:58.560 INFO:teuthology.orchestra.run.gibba026.stdout: debug asserts on shutdown = true 2021-04-03T03:07:58.560 INFO:teuthology.orchestra.run.gibba026.stdout: mon health detail to clog = false 2021-04-03T03:07:58.560 INFO:teuthology.orchestra.run.gibba026.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:07:58.560 INFO:teuthology.orchestra.run.gibba026.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:07:58.561 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.561 INFO:teuthology.orchestra.run.gibba026.stdout:[osd] 2021-04-03T03:07:58.561 INFO:teuthology.orchestra.run.gibba026.stdout: osd journal size = 100 2021-04-03T03:07:58.561 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.561 INFO:teuthology.orchestra.run.gibba026.stdout: osd scrub load threshold = 5.0 2021-04-03T03:07:58.562 INFO:teuthology.orchestra.run.gibba026.stdout: osd scrub max interval = 600 2021-04-03T03:07:58.562 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.562 INFO:teuthology.orchestra.run.gibba026.stdout: osd recover clone overlap = true 2021-04-03T03:07:58.562 INFO:teuthology.orchestra.run.gibba026.stdout: osd recovery max chunk = 1048576 2021-04-03T03:07:58.563 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.563 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug shutdown = true 2021-04-03T03:07:58.563 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug op order = true 2021-04-03T03:07:58.563 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug verify stray on activate = true 2021-04-03T03:07:58.563 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.564 INFO:teuthology.orchestra.run.gibba026.stdout: osd open classes on start = true 2021-04-03T03:07:58.564 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug pg log writeout = true 2021-04-03T03:07:58.564 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.564 INFO:teuthology.orchestra.run.gibba026.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:07:58.565 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.565 INFO:teuthology.orchestra.run.gibba026.stdout: osd map max advance = 10 2021-04-03T03:07:58.565 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.565 INFO:teuthology.orchestra.run.gibba026.stdout: journal zero on create = true 2021-04-03T03:07:58.565 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.566 INFO:teuthology.orchestra.run.gibba026.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:07:58.566 INFO:teuthology.orchestra.run.gibba026.stdout: filestore apply finisher threads = 3 2021-04-03T03:07:58.566 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.566 INFO:teuthology.orchestra.run.gibba026.stdout: bdev debug aio = true 2021-04-03T03:07:58.566 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug misdirected ops = true 2021-04-03T03:07:58.567 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore block size = 96636764160 2021-04-03T03:07:58.567 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore compression mode = aggressive 2021-04-03T03:07:58.567 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore fsck on mount = True 2021-04-03T03:07:58.567 INFO:teuthology.orchestra.run.gibba026.stdout: debug bluefs = 20 2021-04-03T03:07:58.567 INFO:teuthology.orchestra.run.gibba026.stdout: debug bluestore = 20 2021-04-03T03:07:58.568 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:07:58.568 INFO:teuthology.orchestra.run.gibba026.stdout: debug osd = 20 2021-04-03T03:07:58.568 INFO:teuthology.orchestra.run.gibba026.stdout: debug rocksdb = 10 2021-04-03T03:07:58.568 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:07:58.568 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd full ratio = 0.9 2021-04-03T03:07:58.569 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:07:58.569 INFO:teuthology.orchestra.run.gibba026.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:07:58.569 INFO:teuthology.orchestra.run.gibba026.stdout: osd objectstore = bluestore 2021-04-03T03:07:58.569 INFO:teuthology.orchestra.run.gibba026.stdout: osd op complaint time = 180 2021-04-03T03:07:58.569 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.570 INFO:teuthology.orchestra.run.gibba026.stdout:[mgr] 2021-04-03T03:07:58.570 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:07:58.570 INFO:teuthology.orchestra.run.gibba026.stdout: debug mgr = 20 2021-04-03T03:07:58.570 INFO:teuthology.orchestra.run.gibba026.stdout: debug mon = 20 2021-04-03T03:07:58.570 INFO:teuthology.orchestra.run.gibba026.stdout: debug auth = 20 2021-04-03T03:07:58.571 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.571 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.571 INFO:teuthology.orchestra.run.gibba026.stdout: mgr/telemetry/nag = false 2021-04-03T03:07:58.571 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.572 INFO:teuthology.orchestra.run.gibba026.stdout:[mon] 2021-04-03T03:07:58.572 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:07:58.572 INFO:teuthology.orchestra.run.gibba026.stdout: debug mon = 20 2021-04-03T03:07:58.572 INFO:teuthology.orchestra.run.gibba026.stdout: debug paxos = 20 2021-04-03T03:07:58.572 INFO:teuthology.orchestra.run.gibba026.stdout: debug auth = 20 2021-04-03T03:07:58.573 INFO:teuthology.orchestra.run.gibba026.stdout: mon data avail warn = 5 2021-04-03T03:07:58.573 INFO:teuthology.orchestra.run.gibba026.stdout: mon mgr mkfs grace = 240 2021-04-03T03:07:58.573 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.573 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd reporter subtree level = osd 2021-04-03T03:07:58.573 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd prime pg temp = true 2021-04-03T03:07:58.574 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.574 INFO:teuthology.orchestra.run.gibba026.stdout: mon op complaint time = 120 2021-04-03T03:07:58.574 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:07:58.574 INFO:teuthology.orchestra.run.gibba026.stdout:[client] 2021-04-03T03:07:58.574 INFO:teuthology.orchestra.run.gibba026.stdout: rgw cache enabled = true 2021-04-03T03:07:58.575 INFO:teuthology.orchestra.run.gibba026.stdout: rgw enable ops log = true 2021-04-03T03:07:58.575 INFO:teuthology.orchestra.run.gibba026.stdout: rgw enable usage log = true 2021-04-03T03:07:58.575 INFO:teuthology.orchestra.run.gibba026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:07:58.575 INFO:teuthology.orchestra.run.gibba026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:07:58.576 INFO:teuthology.orchestra.run.gibba026.stdout: client mount timeout = 600 2021-04-03T03:07:58.576 INFO:teuthology.orchestra.run.gibba026.stdout: debug client = 20 2021-04-03T03:07:58.576 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:07:58.576 INFO:teuthology.orchestra.run.gibba026.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.576 INFO:teuthology.orchestra.run.gibba026.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.577 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.a] 2021-04-03T03:07:58.577 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.b] 2021-04-03T03:07:58.577 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.c] 2021-04-03T03:07:58.577 INFO:teuthology.orchestra.run.gibba026.stdout:[mds.a] 2021-04-03T03:07:58.577 INFO:teuthology.orchestra.run.gibba026.stdout:[mds.b] 2021-04-03T03:07:58.578 INFO:teuthology.orchestra.run.gibba026.stdout:[mds] 2021-04-03T03:07:58.578 INFO:teuthology.orchestra.run.gibba026.stdout: debug mds = 20 2021-04-03T03:07:58.578 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:07:58.578 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal frag = True 2021-04-03T03:07:58.578 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal fragment size max = 10000 2021-04-03T03:07:58.579 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal merge size = 5 2021-04-03T03:07:58.579 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal split bits = 3 2021-04-03T03:07:58.579 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal split size = 100 2021-04-03T03:07:58.579 INFO:teuthology.orchestra.run.gibba026.stdout: mds debug frag = True 2021-04-03T03:07:58.579 INFO:teuthology.orchestra.run.gibba026.stdout: mds debug scatterstat = True 2021-04-03T03:07:58.580 INFO:teuthology.orchestra.run.gibba026.stdout: mds op complaint time = 180 2021-04-03T03:07:58.580 INFO:teuthology.orchestra.run.gibba026.stdout: mds verify scatter = True 2021-04-03T03:07:58.580 INFO:teuthology.orchestra.run.gibba026.stdout: osd op complaint time = 180 2021-04-03T03:07:58.580 INFO:teuthology.orchestra.run.gibba026.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.580 INFO:teuthology.orchestra.run.gibba026.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.581 INFO:teuthology.orchestra.run.gibba015.stdout:[global] 2021-04-03T03:07:58.581 INFO:teuthology.orchestra.run.gibba015.stdout: chdir = "" 2021-04-03T03:07:58.581 INFO:teuthology.orchestra.run.gibba015.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:07:58.581 INFO:teuthology.orchestra.run.gibba015.stdout: auth supported = cephx 2021-04-03T03:07:58.582 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.582 INFO:teuthology.orchestra.run.gibba015.stdout: filestore xattr use omap = true 2021-04-03T03:07:58.582 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.582 INFO:teuthology.orchestra.run.gibba015.stdout: mon clock drift allowed = 1.000 2021-04-03T03:07:58.582 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.583 INFO:teuthology.orchestra.run.gibba015.stdout: osd crush chooseleaf type = 0 2021-04-03T03:07:58.583 INFO:teuthology.orchestra.run.gibba015.stdout: auth debug = true 2021-04-03T03:07:58.583 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.583 INFO:teuthology.orchestra.run.gibba015.stdout: ms die on old message = true 2021-04-03T03:07:58.584 INFO:teuthology.orchestra.run.gibba015.stdout: ms die on bug = true 2021-04-03T03:07:58.584 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.584 INFO:teuthology.orchestra.run.gibba015.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:07:58.584 INFO:teuthology.orchestra.run.gibba015.stdout: mon pg warn max object skew = 0 2021-04-03T03:07:58.584 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.585 INFO:teuthology.orchestra.run.gibba015.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:07:58.585 INFO:teuthology.orchestra.run.gibba015.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:07:58.585 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.585 INFO:teuthology.orchestra.run.gibba015.stdout: osd pool default size = 2 2021-04-03T03:07:58.585 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.586 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd allow primary affinity = true 2021-04-03T03:07:58.586 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd allow pg remap = true 2021-04-03T03:07:58.586 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:07:58.586 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:07:58.586 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on no sortbitwise = false 2021-04-03T03:07:58.587 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:07:58.587 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on too few osds = false 2021-04-03T03:07:58.587 INFO:teuthology.orchestra.run.gibba015.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:07:58.587 INFO:teuthology.orchestra.run.gibba015.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:07:58.587 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.588 INFO:teuthology.orchestra.run.gibba015.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:07:58.588 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.588 INFO:teuthology.orchestra.run.gibba015.stdout: osd default data pool replay window = 5 2021-04-03T03:07:58.588 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.589 INFO:teuthology.orchestra.run.gibba015.stdout: mon allow pool delete = true 2021-04-03T03:07:58.589 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.589 INFO:teuthology.orchestra.run.gibba015.stdout: mon cluster log file level = debug 2021-04-03T03:07:58.589 INFO:teuthology.orchestra.run.gibba015.stdout: debug asserts on shutdown = true 2021-04-03T03:07:58.589 INFO:teuthology.orchestra.run.gibba015.stdout: mon health detail to clog = false 2021-04-03T03:07:58.590 INFO:teuthology.orchestra.run.gibba015.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:07:58.590 INFO:teuthology.orchestra.run.gibba015.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:07:58.590 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.590 INFO:teuthology.orchestra.run.gibba015.stdout:[osd] 2021-04-03T03:07:58.590 INFO:teuthology.orchestra.run.gibba015.stdout: osd journal size = 100 2021-04-03T03:07:58.591 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.591 INFO:teuthology.orchestra.run.gibba015.stdout: osd scrub load threshold = 5.0 2021-04-03T03:07:58.591 INFO:teuthology.orchestra.run.gibba015.stdout: osd scrub max interval = 600 2021-04-03T03:07:58.591 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.591 INFO:teuthology.orchestra.run.gibba015.stdout: osd recover clone overlap = true 2021-04-03T03:07:58.592 INFO:teuthology.orchestra.run.gibba015.stdout: osd recovery max chunk = 1048576 2021-04-03T03:07:58.592 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.592 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug shutdown = true 2021-04-03T03:07:58.592 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug op order = true 2021-04-03T03:07:58.592 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug verify stray on activate = true 2021-04-03T03:07:58.593 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.593 INFO:teuthology.orchestra.run.gibba015.stdout: osd open classes on start = true 2021-04-03T03:07:58.593 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug pg log writeout = true 2021-04-03T03:07:58.593 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.594 INFO:teuthology.orchestra.run.gibba015.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:07:58.594 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.594 INFO:teuthology.orchestra.run.gibba015.stdout: osd map max advance = 10 2021-04-03T03:07:58.594 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.594 INFO:teuthology.orchestra.run.gibba015.stdout: journal zero on create = true 2021-04-03T03:07:58.595 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.595 INFO:teuthology.orchestra.run.gibba015.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:07:58.595 INFO:teuthology.orchestra.run.gibba015.stdout: filestore apply finisher threads = 3 2021-04-03T03:07:58.595 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.595 INFO:teuthology.orchestra.run.gibba015.stdout: bdev debug aio = true 2021-04-03T03:07:58.596 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug misdirected ops = true 2021-04-03T03:07:58.596 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore block size = 96636764160 2021-04-03T03:07:58.596 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore compression mode = aggressive 2021-04-03T03:07:58.596 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore fsck on mount = True 2021-04-03T03:07:58.596 INFO:teuthology.orchestra.run.gibba015.stdout: debug bluefs = 20 2021-04-03T03:07:58.597 INFO:teuthology.orchestra.run.gibba015.stdout: debug bluestore = 20 2021-04-03T03:07:58.597 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:07:58.597 INFO:teuthology.orchestra.run.gibba015.stdout: debug osd = 20 2021-04-03T03:07:58.597 INFO:teuthology.orchestra.run.gibba015.stdout: debug rocksdb = 10 2021-04-03T03:07:58.598 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:07:58.598 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd full ratio = 0.9 2021-04-03T03:07:58.598 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:07:58.598 INFO:teuthology.orchestra.run.gibba015.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:07:58.599 INFO:teuthology.orchestra.run.gibba015.stdout: osd objectstore = bluestore 2021-04-03T03:07:58.599 INFO:teuthology.orchestra.run.gibba015.stdout: osd op complaint time = 180 2021-04-03T03:07:58.599 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.599 INFO:teuthology.orchestra.run.gibba015.stdout:[mgr] 2021-04-03T03:07:58.599 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:07:58.600 INFO:teuthology.orchestra.run.gibba015.stdout: debug mgr = 20 2021-04-03T03:07:58.600 INFO:teuthology.orchestra.run.gibba015.stdout: debug mon = 20 2021-04-03T03:07:58.600 INFO:teuthology.orchestra.run.gibba015.stdout: debug auth = 20 2021-04-03T03:07:58.600 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.600 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.601 INFO:teuthology.orchestra.run.gibba015.stdout: mgr/telemetry/nag = false 2021-04-03T03:07:58.601 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.601 INFO:teuthology.orchestra.run.gibba015.stdout:[mon] 2021-04-03T03:07:58.601 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:07:58.601 INFO:teuthology.orchestra.run.gibba015.stdout: debug mon = 20 2021-04-03T03:07:58.602 INFO:teuthology.orchestra.run.gibba015.stdout: debug paxos = 20 2021-04-03T03:07:58.602 INFO:teuthology.orchestra.run.gibba015.stdout: debug auth = 20 2021-04-03T03:07:58.602 INFO:teuthology.orchestra.run.gibba015.stdout: mon data avail warn = 5 2021-04-03T03:07:58.602 INFO:teuthology.orchestra.run.gibba015.stdout: mon mgr mkfs grace = 240 2021-04-03T03:07:58.603 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.603 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd reporter subtree level = osd 2021-04-03T03:07:58.603 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd prime pg temp = true 2021-04-03T03:07:58.603 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.603 INFO:teuthology.orchestra.run.gibba015.stdout: mon op complaint time = 120 2021-04-03T03:07:58.604 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:07:58.604 INFO:teuthology.orchestra.run.gibba015.stdout:[client] 2021-04-03T03:07:58.604 INFO:teuthology.orchestra.run.gibba015.stdout: rgw cache enabled = true 2021-04-03T03:07:58.604 INFO:teuthology.orchestra.run.gibba015.stdout: rgw enable ops log = true 2021-04-03T03:07:58.604 INFO:teuthology.orchestra.run.gibba015.stdout: rgw enable usage log = true 2021-04-03T03:07:58.605 INFO:teuthology.orchestra.run.gibba015.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:07:58.605 INFO:teuthology.orchestra.run.gibba015.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:07:58.605 INFO:teuthology.orchestra.run.gibba015.stdout: client mount timeout = 600 2021-04-03T03:07:58.605 INFO:teuthology.orchestra.run.gibba015.stdout: debug client = 20 2021-04-03T03:07:58.606 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:07:58.606 INFO:teuthology.orchestra.run.gibba015.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.606 INFO:teuthology.orchestra.run.gibba015.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.606 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.a] 2021-04-03T03:07:58.606 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.b] 2021-04-03T03:07:58.607 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.c] 2021-04-03T03:07:58.607 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.a] 2021-04-03T03:07:58.607 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.b] 2021-04-03T03:07:58.607 INFO:teuthology.orchestra.run.gibba015.stdout:[mds] 2021-04-03T03:07:58.608 INFO:teuthology.orchestra.run.gibba015.stdout: debug mds = 20 2021-04-03T03:07:58.608 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:07:58.608 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal frag = True 2021-04-03T03:07:58.608 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal fragment size max = 10000 2021-04-03T03:07:58.608 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal merge size = 5 2021-04-03T03:07:58.609 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal split bits = 3 2021-04-03T03:07:58.609 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal split size = 100 2021-04-03T03:07:58.609 INFO:teuthology.orchestra.run.gibba015.stdout: mds debug frag = True 2021-04-03T03:07:58.609 INFO:teuthology.orchestra.run.gibba015.stdout: mds debug scatterstat = True 2021-04-03T03:07:58.609 INFO:teuthology.orchestra.run.gibba015.stdout: mds op complaint time = 180 2021-04-03T03:07:58.610 INFO:teuthology.orchestra.run.gibba015.stdout: mds verify scatter = True 2021-04-03T03:07:58.610 INFO:teuthology.orchestra.run.gibba015.stdout: osd op complaint time = 180 2021-04-03T03:07:58.610 INFO:teuthology.orchestra.run.gibba015.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.610 INFO:teuthology.orchestra.run.gibba015.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.611 INFO:teuthology.orchestra.run.gibba031.stdout:[global] 2021-04-03T03:07:58.612 INFO:teuthology.orchestra.run.gibba031.stdout: chdir = "" 2021-04-03T03:07:58.612 INFO:teuthology.orchestra.run.gibba031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:07:58.612 INFO:teuthology.orchestra.run.gibba031.stdout: auth supported = cephx 2021-04-03T03:07:58.612 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.613 INFO:teuthology.orchestra.run.gibba031.stdout: filestore xattr use omap = true 2021-04-03T03:07:58.613 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.613 INFO:teuthology.orchestra.run.gibba031.stdout: mon clock drift allowed = 1.000 2021-04-03T03:07:58.613 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.614 INFO:teuthology.orchestra.run.gibba031.stdout: osd crush chooseleaf type = 0 2021-04-03T03:07:58.614 INFO:teuthology.orchestra.run.gibba031.stdout: auth debug = true 2021-04-03T03:07:58.614 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.614 INFO:teuthology.orchestra.run.gibba031.stdout: ms die on old message = true 2021-04-03T03:07:58.615 INFO:teuthology.orchestra.run.gibba031.stdout: ms die on bug = true 2021-04-03T03:07:58.615 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.615 INFO:teuthology.orchestra.run.gibba031.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:07:58.615 INFO:teuthology.orchestra.run.gibba031.stdout: mon pg warn max object skew = 0 2021-04-03T03:07:58.615 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.616 INFO:teuthology.orchestra.run.gibba031.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:07:58.616 INFO:teuthology.orchestra.run.gibba031.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:07:58.616 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.616 INFO:teuthology.orchestra.run.gibba031.stdout: osd pool default size = 2 2021-04-03T03:07:58.616 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.617 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd allow primary affinity = true 2021-04-03T03:07:58.617 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd allow pg remap = true 2021-04-03T03:07:58.617 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:07:58.617 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:07:58.617 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on no sortbitwise = false 2021-04-03T03:07:58.618 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:07:58.618 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on too few osds = false 2021-04-03T03:07:58.618 INFO:teuthology.orchestra.run.gibba031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:07:58.618 INFO:teuthology.orchestra.run.gibba031.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:07:58.619 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.619 INFO:teuthology.orchestra.run.gibba031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:07:58.619 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.619 INFO:teuthology.orchestra.run.gibba031.stdout: osd default data pool replay window = 5 2021-04-03T03:07:58.619 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.620 INFO:teuthology.orchestra.run.gibba031.stdout: mon allow pool delete = true 2021-04-03T03:07:58.620 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.620 INFO:teuthology.orchestra.run.gibba031.stdout: mon cluster log file level = debug 2021-04-03T03:07:58.620 INFO:teuthology.orchestra.run.gibba031.stdout: debug asserts on shutdown = true 2021-04-03T03:07:58.620 INFO:teuthology.orchestra.run.gibba031.stdout: mon health detail to clog = false 2021-04-03T03:07:58.621 INFO:teuthology.orchestra.run.gibba031.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:07:58.621 INFO:teuthology.orchestra.run.gibba031.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:07:58.621 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.621 INFO:teuthology.orchestra.run.gibba031.stdout:[osd] 2021-04-03T03:07:58.621 INFO:teuthology.orchestra.run.gibba031.stdout: osd journal size = 100 2021-04-03T03:07:58.622 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.622 INFO:teuthology.orchestra.run.gibba031.stdout: osd scrub load threshold = 5.0 2021-04-03T03:07:58.622 INFO:teuthology.orchestra.run.gibba031.stdout: osd scrub max interval = 600 2021-04-03T03:07:58.622 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.622 INFO:teuthology.orchestra.run.gibba031.stdout: osd recover clone overlap = true 2021-04-03T03:07:58.623 INFO:teuthology.orchestra.run.gibba031.stdout: osd recovery max chunk = 1048576 2021-04-03T03:07:58.623 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.623 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug shutdown = true 2021-04-03T03:07:58.623 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug op order = true 2021-04-03T03:07:58.624 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug verify stray on activate = true 2021-04-03T03:07:58.624 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.624 INFO:teuthology.orchestra.run.gibba031.stdout: osd open classes on start = true 2021-04-03T03:07:58.624 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug pg log writeout = true 2021-04-03T03:07:58.624 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.625 INFO:teuthology.orchestra.run.gibba031.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:07:58.625 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.625 INFO:teuthology.orchestra.run.gibba031.stdout: osd map max advance = 10 2021-04-03T03:07:58.625 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.625 INFO:teuthology.orchestra.run.gibba031.stdout: journal zero on create = true 2021-04-03T03:07:58.626 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.626 INFO:teuthology.orchestra.run.gibba031.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:07:58.626 INFO:teuthology.orchestra.run.gibba031.stdout: filestore apply finisher threads = 3 2021-04-03T03:07:58.626 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.626 INFO:teuthology.orchestra.run.gibba031.stdout: bdev debug aio = true 2021-04-03T03:07:58.627 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug misdirected ops = true 2021-04-03T03:07:58.627 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore block size = 96636764160 2021-04-03T03:07:58.627 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore compression mode = aggressive 2021-04-03T03:07:58.627 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore fsck on mount = True 2021-04-03T03:07:58.627 INFO:teuthology.orchestra.run.gibba031.stdout: debug bluefs = 20 2021-04-03T03:07:58.628 INFO:teuthology.orchestra.run.gibba031.stdout: debug bluestore = 20 2021-04-03T03:07:58.628 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:07:58.628 INFO:teuthology.orchestra.run.gibba031.stdout: debug osd = 20 2021-04-03T03:07:58.628 INFO:teuthology.orchestra.run.gibba031.stdout: debug rocksdb = 10 2021-04-03T03:07:58.628 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:07:58.629 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd full ratio = 0.9 2021-04-03T03:07:58.629 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:07:58.629 INFO:teuthology.orchestra.run.gibba031.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:07:58.629 INFO:teuthology.orchestra.run.gibba031.stdout: osd objectstore = bluestore 2021-04-03T03:07:58.630 INFO:teuthology.orchestra.run.gibba031.stdout: osd op complaint time = 180 2021-04-03T03:07:58.630 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.630 INFO:teuthology.orchestra.run.gibba031.stdout:[mgr] 2021-04-03T03:07:58.630 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:07:58.630 INFO:teuthology.orchestra.run.gibba031.stdout: debug mgr = 20 2021-04-03T03:07:58.631 INFO:teuthology.orchestra.run.gibba031.stdout: debug mon = 20 2021-04-03T03:07:58.631 INFO:teuthology.orchestra.run.gibba031.stdout: debug auth = 20 2021-04-03T03:07:58.631 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.631 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.631 INFO:teuthology.orchestra.run.gibba031.stdout: mgr/telemetry/nag = false 2021-04-03T03:07:58.632 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.632 INFO:teuthology.orchestra.run.gibba031.stdout:[mon] 2021-04-03T03:07:58.632 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:07:58.632 INFO:teuthology.orchestra.run.gibba031.stdout: debug mon = 20 2021-04-03T03:07:58.632 INFO:teuthology.orchestra.run.gibba031.stdout: debug paxos = 20 2021-04-03T03:07:58.633 INFO:teuthology.orchestra.run.gibba031.stdout: debug auth = 20 2021-04-03T03:07:58.633 INFO:teuthology.orchestra.run.gibba031.stdout: mon data avail warn = 5 2021-04-03T03:07:58.633 INFO:teuthology.orchestra.run.gibba031.stdout: mon mgr mkfs grace = 240 2021-04-03T03:07:58.633 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:07:58.634 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd reporter subtree level = osd 2021-04-03T03:07:58.634 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd prime pg temp = true 2021-04-03T03:07:58.634 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:07:58.634 INFO:teuthology.orchestra.run.gibba031.stdout: mon op complaint time = 120 2021-04-03T03:07:58.634 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:07:58.635 INFO:teuthology.orchestra.run.gibba031.stdout:[client] 2021-04-03T03:07:58.635 INFO:teuthology.orchestra.run.gibba031.stdout: rgw cache enabled = true 2021-04-03T03:07:58.635 INFO:teuthology.orchestra.run.gibba031.stdout: rgw enable ops log = true 2021-04-03T03:07:58.635 INFO:teuthology.orchestra.run.gibba031.stdout: rgw enable usage log = true 2021-04-03T03:07:58.635 INFO:teuthology.orchestra.run.gibba031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:07:58.636 INFO:teuthology.orchestra.run.gibba031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:07:58.636 INFO:teuthology.orchestra.run.gibba031.stdout: client mount timeout = 600 2021-04-03T03:07:58.636 INFO:teuthology.orchestra.run.gibba031.stdout: debug client = 20 2021-04-03T03:07:58.636 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:07:58.636 INFO:teuthology.orchestra.run.gibba031.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.637 INFO:teuthology.orchestra.run.gibba031.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.637 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.a] 2021-04-03T03:07:58.637 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.b] 2021-04-03T03:07:58.637 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.c] 2021-04-03T03:07:58.637 INFO:teuthology.orchestra.run.gibba031.stdout:[mds.a] 2021-04-03T03:07:58.638 INFO:teuthology.orchestra.run.gibba031.stdout:[mds.b] 2021-04-03T03:07:58.638 INFO:teuthology.orchestra.run.gibba031.stdout:[mds] 2021-04-03T03:07:58.638 INFO:teuthology.orchestra.run.gibba031.stdout: debug mds = 20 2021-04-03T03:07:58.638 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:07:58.638 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal frag = True 2021-04-03T03:07:58.639 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal fragment size max = 10000 2021-04-03T03:07:58.639 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal merge size = 5 2021-04-03T03:07:58.639 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal split bits = 3 2021-04-03T03:07:58.639 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal split size = 100 2021-04-03T03:07:58.639 INFO:teuthology.orchestra.run.gibba031.stdout: mds debug frag = True 2021-04-03T03:07:58.640 INFO:teuthology.orchestra.run.gibba031.stdout: mds debug scatterstat = True 2021-04-03T03:07:58.640 INFO:teuthology.orchestra.run.gibba031.stdout: mds op complaint time = 180 2021-04-03T03:07:58.640 INFO:teuthology.orchestra.run.gibba031.stdout: mds verify scatter = True 2021-04-03T03:07:58.640 INFO:teuthology.orchestra.run.gibba031.stdout: osd op complaint time = 180 2021-04-03T03:07:58.641 INFO:teuthology.orchestra.run.gibba031.stdout: rados mon op timeout = 15m 2021-04-03T03:07:58.641 INFO:teuthology.orchestra.run.gibba031.stdout: rados osd op timeout = 15m 2021-04-03T03:07:58.641 INFO:tasks.ceph:Creating admin key on mon.a... 2021-04-03T03:07:58.641 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2021-04-03T03:07:58.667 INFO:tasks.ceph:Copying monmap to all nodes... 2021-04-03T03:07:58.712 DEBUG:teuthology.orchestra.remote:gibba011:/etc/ceph/ceph.keyring is 206B 2021-04-03T03:07:58.772 DEBUG:teuthology.orchestra.remote:gibba011:/home/ubuntu/cephtest/ceph.monmap is 379B 2021-04-03T03:07:58.786 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T03:07:58.786 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:07:58.787 DEBUG:teuthology.orchestra.run.gibba011:> sudo dd of=/etc/ceph/ceph.keyring 2021-04-03T03:07:58.787 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-04-03T03:07:58.819 DEBUG:teuthology.orchestra.run.gibba011:> set -ex 2021-04-03T03:07:58.819 DEBUG:teuthology.orchestra.run.gibba011:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:07:58.867 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T03:07:58.867 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:07:58.868 DEBUG:teuthology.orchestra.run.gibba015:> sudo dd of=/etc/ceph/ceph.keyring 2021-04-03T03:07:58.868 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-04-03T03:07:58.884 DEBUG:teuthology.orchestra.run.gibba015:> set -ex 2021-04-03T03:07:58.885 DEBUG:teuthology.orchestra.run.gibba015:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:07:58.930 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba026.front.sepia.ceph.com 2021-04-03T03:07:58.931 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:07:58.931 DEBUG:teuthology.orchestra.run.gibba026:> sudo dd of=/etc/ceph/ceph.keyring 2021-04-03T03:07:58.931 DEBUG:teuthology.orchestra.run.gibba026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-04-03T03:07:58.948 DEBUG:teuthology.orchestra.run.gibba026:> set -ex 2021-04-03T03:07:58.948 DEBUG:teuthology.orchestra.run.gibba026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:07:58.992 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba028.front.sepia.ceph.com 2021-04-03T03:07:58.993 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:07:58.993 DEBUG:teuthology.orchestra.run.gibba028:> sudo dd of=/etc/ceph/ceph.keyring 2021-04-03T03:07:58.993 DEBUG:teuthology.orchestra.run.gibba028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-04-03T03:07:59.010 DEBUG:teuthology.orchestra.run.gibba028:> set -ex 2021-04-03T03:07:59.011 DEBUG:teuthology.orchestra.run.gibba028:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:07:59.057 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba031.front.sepia.ceph.com 2021-04-03T03:07:59.057 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:07:59.058 DEBUG:teuthology.orchestra.run.gibba031:> sudo dd of=/etc/ceph/ceph.keyring 2021-04-03T03:07:59.058 DEBUG:teuthology.orchestra.run.gibba031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-04-03T03:07:59.075 DEBUG:teuthology.orchestra.run.gibba031:> set -ex 2021-04-03T03:07:59.075 DEBUG:teuthology.orchestra.run.gibba031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:07:59.119 INFO:tasks.ceph:Setting up mon nodes... 2021-04-03T03:07:59.120 INFO:tasks.ceph:Setting up mgr nodes... 2021-04-03T03:07:59.120 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2021-04-03T03:07:59.143 INFO:teuthology.orchestra.run.gibba011.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2021-04-03T03:07:59.145 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2021-04-03T03:07:59.168 INFO:teuthology.orchestra.run.gibba015.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2021-04-03T03:07:59.170 INFO:tasks.ceph:Setting up mds nodes... 2021-04-03T03:07:59.170 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2021-04-03T03:07:59.209 INFO:teuthology.orchestra.run.gibba011.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2021-04-03T03:07:59.210 DEBUG:teuthology.orchestra.run.gibba011:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2021-04-03T03:07:59.258 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2021-04-03T03:07:59.298 INFO:teuthology.orchestra.run.gibba015.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2021-04-03T03:07:59.300 DEBUG:teuthology.orchestra.run.gibba015:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2021-04-03T03:07:59.349 INFO:tasks.ceph_client:Setting up client nodes... 2021-04-03T03:07:59.350 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-04-03T03:07:59.371 INFO:teuthology.orchestra.run.gibba026.stdout:creating /etc/ceph/ceph.client.0.keyring 2021-04-03T03:07:59.375 DEBUG:teuthology.orchestra.run.gibba028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2021-04-03T03:07:59.396 INFO:teuthology.orchestra.run.gibba028.stdout:creating /etc/ceph/ceph.client.1.keyring 2021-04-03T03:07:59.399 DEBUG:teuthology.orchestra.run.gibba031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2021-04-03T03:07:59.421 INFO:teuthology.orchestra.run.gibba031.stdout:creating /etc/ceph/ceph.client.2.keyring 2021-04-03T03:07:59.424 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-04-03T03:07:59.425 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@gibba011.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba015.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-04-03T03:07:59.425 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2021-04-03T03:07:59.436 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-04-03T03:07:59.437 INFO:tasks.ceph:role: osd.0 2021-04-03T03:07:59.437 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T03:07:59.437 DEBUG:teuthology.orchestra.run.gibba011:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-04-03T03:07:59.520 INFO:teuthology.orchestra.run.gibba011.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:07:59.520 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:07:59.520 INFO:teuthology.orchestra.run.gibba011.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:07:59.520 INFO:teuthology.orchestra.run.gibba011.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:07:59.521 INFO:teuthology.orchestra.run.gibba011.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:07:59.521 INFO:teuthology.orchestra.run.gibba011.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:07:59.521 INFO:teuthology.orchestra.run.gibba011.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:07:59.521 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:07:59.521 INFO:teuthology.orchestra.run.gibba011.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:07:59.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba011.front.sepia.ceph.com -o noatime 2021-04-03T03:07:59.523 DEBUG:teuthology.orchestra.run.gibba011:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2021-04-03T03:07:59.579 DEBUG:teuthology.orchestra.run.gibba011:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2021-04-03T03:07:59.635 INFO:teuthology.orchestra.run.gibba011.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:07:59.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:07:59.637 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2021-04-03T03:07:59.696 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-04-03T03:07:59.696 INFO:tasks.ceph:role: osd.1 2021-04-03T03:07:59.697 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T03:07:59.697 DEBUG:teuthology.orchestra.run.gibba011:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-04-03T03:07:59.778 INFO:teuthology.orchestra.run.gibba011.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:07:59.779 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:07:59.779 INFO:teuthology.orchestra.run.gibba011.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:07:59.779 INFO:teuthology.orchestra.run.gibba011.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:07:59.779 INFO:teuthology.orchestra.run.gibba011.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:07:59.780 INFO:teuthology.orchestra.run.gibba011.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:07:59.780 INFO:teuthology.orchestra.run.gibba011.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:07:59.780 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:07:59.780 INFO:teuthology.orchestra.run.gibba011.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:07:59.781 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba011.front.sepia.ceph.com -o noatime 2021-04-03T03:07:59.782 DEBUG:teuthology.orchestra.run.gibba011:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2021-04-03T03:07:59.855 DEBUG:teuthology.orchestra.run.gibba011:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2021-04-03T03:07:59.913 INFO:teuthology.orchestra.run.gibba011.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:07:59.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:07:59.915 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2021-04-03T03:07:59.975 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-04-03T03:07:59.975 INFO:tasks.ceph:role: osd.2 2021-04-03T03:07:59.975 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T03:07:59.976 DEBUG:teuthology.orchestra.run.gibba011:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-04-03T03:08:00.048 INFO:teuthology.orchestra.run.gibba011.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:08:00.048 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:08:00.048 INFO:teuthology.orchestra.run.gibba011.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:08:00.048 INFO:teuthology.orchestra.run.gibba011.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:08:00.049 INFO:teuthology.orchestra.run.gibba011.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:08:00.049 INFO:teuthology.orchestra.run.gibba011.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:08:00.049 INFO:teuthology.orchestra.run.gibba011.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:08:00.049 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:08:00.050 INFO:teuthology.orchestra.run.gibba011.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:08:00.051 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba011.front.sepia.ceph.com -o noatime 2021-04-03T03:08:00.051 DEBUG:teuthology.orchestra.run.gibba011:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2021-04-03T03:08:00.122 DEBUG:teuthology.orchestra.run.gibba011:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2021-04-03T03:08:00.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:08:00.173 INFO:teuthology.orchestra.run.gibba011.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:08:00.173 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2021-04-03T03:08:00.232 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-04-03T03:08:00.232 INFO:tasks.ceph:role: osd.3 2021-04-03T03:08:00.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T03:08:00.233 DEBUG:teuthology.orchestra.run.gibba011:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-04-03T03:08:00.327 INFO:teuthology.orchestra.run.gibba011.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:08:00.327 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:08:00.327 INFO:teuthology.orchestra.run.gibba011.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:08:00.328 INFO:teuthology.orchestra.run.gibba011.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:08:00.328 INFO:teuthology.orchestra.run.gibba011.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:08:00.328 INFO:teuthology.orchestra.run.gibba011.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:08:00.328 INFO:teuthology.orchestra.run.gibba011.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:08:00.328 INFO:teuthology.orchestra.run.gibba011.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:08:00.329 INFO:teuthology.orchestra.run.gibba011.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:08:00.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba011.front.sepia.ceph.com -o noatime 2021-04-03T03:08:00.330 DEBUG:teuthology.orchestra.run.gibba011:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2021-04-03T03:08:00.346 DEBUG:teuthology.orchestra.run.gibba011:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2021-04-03T03:08:00.400 INFO:teuthology.orchestra.run.gibba011.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:08:00.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:08:00.401 DEBUG:teuthology.orchestra.run.gibba011:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:00.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:00.475+0000 7f6720f43d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2021-04-03T03:08:00.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:00.475+0000 7f6720f43d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2021-04-03T03:08:00.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:00.475+0000 7f6720f43d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-04-03T03:08:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:00.475+0000 7f6720f43d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-04-03T03:08:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:00.475+0000 7f6720f43d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-04-03T03:08:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:00.475+0000 7f6720f43d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2021-04-03T03:08:00.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:00.483+0000 7f6720f43d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:02.928 DEBUG:teuthology.orchestra.run.gibba011:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-04-03T03:08:02.978 DEBUG:teuthology.orchestra.run.gibba011:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:03.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:03.027+0000 7f6d7252cd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2021-04-03T03:08:03.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:03.027+0000 7f6d7252cd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2021-04-03T03:08:03.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:03.027+0000 7f6d7252cd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-04-03T03:08:03.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:03.027+0000 7f6d7252cd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-04-03T03:08:03.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:03.027+0000 7f6d7252cd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-04-03T03:08:03.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:03.031+0000 7f6d7252cd80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2021-04-03T03:08:03.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:03.039+0000 7f6d7252cd80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:05.494 DEBUG:teuthology.orchestra.run.gibba011:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-04-03T03:08:05.546 DEBUG:teuthology.orchestra.run.gibba011:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:05.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:05.595+0000 7fbeb9318d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2021-04-03T03:08:05.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:05.595+0000 7fbeb9318d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2021-04-03T03:08:05.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:05.595+0000 7fbeb9318d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-04-03T03:08:05.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:05.595+0000 7fbeb9318d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-04-03T03:08:05.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:05.595+0000 7fbeb9318d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-04-03T03:08:05.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:05.595+0000 7fbeb9318d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2021-04-03T03:08:05.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:05.603+0000 7fbeb9318d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:08.090 DEBUG:teuthology.orchestra.run.gibba011:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-04-03T03:08:08.142 DEBUG:teuthology.orchestra.run.gibba011:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:08.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:08.191+0000 7f4697f35d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2021-04-03T03:08:08.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:08.191+0000 7f4697f35d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2021-04-03T03:08:08.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:08.191+0000 7f4697f35d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-04-03T03:08:08.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:08.191+0000 7f4697f35d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-04-03T03:08:08.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:08.191+0000 7f4697f35d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-04-03T03:08:08.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:08.191+0000 7f4697f35d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2021-04-03T03:08:08.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:08.199+0000 7f4697f35d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:10.640 DEBUG:teuthology.orchestra.run.gibba011:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-04-03T03:08:10.691 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2021-04-03T03:08:10.702 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-04-03T03:08:10.702 INFO:tasks.ceph:role: osd.4 2021-04-03T03:08:10.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T03:08:10.703 DEBUG:teuthology.orchestra.run.gibba015:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-04-03T03:08:10.783 INFO:teuthology.orchestra.run.gibba015.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:08:10.783 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:08:10.783 INFO:teuthology.orchestra.run.gibba015.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:08:10.783 INFO:teuthology.orchestra.run.gibba015.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:08:10.784 INFO:teuthology.orchestra.run.gibba015.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:08:10.784 INFO:teuthology.orchestra.run.gibba015.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:08:10.784 INFO:teuthology.orchestra.run.gibba015.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:08:10.785 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:08:10.785 INFO:teuthology.orchestra.run.gibba015.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:08:10.786 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba015.front.sepia.ceph.com -o noatime 2021-04-03T03:08:10.786 DEBUG:teuthology.orchestra.run.gibba015:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2021-04-03T03:08:10.858 DEBUG:teuthology.orchestra.run.gibba015:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2021-04-03T03:08:10.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:08:10.908 INFO:teuthology.orchestra.run.gibba015.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:08:10.909 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2021-04-03T03:08:10.969 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-04-03T03:08:10.970 INFO:tasks.ceph:role: osd.5 2021-04-03T03:08:10.970 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T03:08:10.970 DEBUG:teuthology.orchestra.run.gibba015:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-04-03T03:08:11.051 INFO:teuthology.orchestra.run.gibba015.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:08:11.052 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:08:11.052 INFO:teuthology.orchestra.run.gibba015.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:08:11.052 INFO:teuthology.orchestra.run.gibba015.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:08:11.053 INFO:teuthology.orchestra.run.gibba015.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:08:11.053 INFO:teuthology.orchestra.run.gibba015.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:08:11.053 INFO:teuthology.orchestra.run.gibba015.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:08:11.054 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:08:11.054 INFO:teuthology.orchestra.run.gibba015.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:08:11.055 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba015.front.sepia.ceph.com -o noatime 2021-04-03T03:08:11.056 DEBUG:teuthology.orchestra.run.gibba015:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2021-04-03T03:08:11.131 DEBUG:teuthology.orchestra.run.gibba015:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2021-04-03T03:08:11.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:08:11.184 INFO:teuthology.orchestra.run.gibba015.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:08:11.185 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2021-04-03T03:08:11.246 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-04-03T03:08:11.246 INFO:tasks.ceph:role: osd.6 2021-04-03T03:08:11.246 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T03:08:11.247 DEBUG:teuthology.orchestra.run.gibba015:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-04-03T03:08:11.327 INFO:teuthology.orchestra.run.gibba015.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:08:11.327 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:08:11.328 INFO:teuthology.orchestra.run.gibba015.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:08:11.328 INFO:teuthology.orchestra.run.gibba015.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:08:11.328 INFO:teuthology.orchestra.run.gibba015.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:08:11.328 INFO:teuthology.orchestra.run.gibba015.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:08:11.329 INFO:teuthology.orchestra.run.gibba015.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:08:11.329 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:08:11.329 INFO:teuthology.orchestra.run.gibba015.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:08:11.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba015.front.sepia.ceph.com -o noatime 2021-04-03T03:08:11.330 DEBUG:teuthology.orchestra.run.gibba015:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2021-04-03T03:08:11.401 DEBUG:teuthology.orchestra.run.gibba015:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2021-04-03T03:08:11.456 INFO:teuthology.orchestra.run.gibba015.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:08:11.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:08:11.458 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2021-04-03T03:08:11.518 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-04-03T03:08:11.518 INFO:tasks.ceph:role: osd.7 2021-04-03T03:08:11.518 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T03:08:11.519 DEBUG:teuthology.orchestra.run.gibba015:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-04-03T03:08:11.600 INFO:teuthology.orchestra.run.gibba015.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-04-03T03:08:11.601 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 attr=2, projid32bit=1 2021-04-03T03:08:11.601 INFO:teuthology.orchestra.run.gibba015.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-04-03T03:08:11.601 INFO:teuthology.orchestra.run.gibba015.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-04-03T03:08:11.601 INFO:teuthology.orchestra.run.gibba015.stdout: = sunit=0 swidth=0 blks 2021-04-03T03:08:11.601 INFO:teuthology.orchestra.run.gibba015.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-04-03T03:08:11.602 INFO:teuthology.orchestra.run.gibba015.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-04-03T03:08:11.602 INFO:teuthology.orchestra.run.gibba015.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-04-03T03:08:11.602 INFO:teuthology.orchestra.run.gibba015.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-04-03T03:08:11.603 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba015.front.sepia.ceph.com -o noatime 2021-04-03T03:08:11.604 DEBUG:teuthology.orchestra.run.gibba015:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2021-04-03T03:08:11.679 DEBUG:teuthology.orchestra.run.gibba015:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2021-04-03T03:08:11.737 INFO:teuthology.orchestra.run.gibba015.stderr:sudo: /sbin/restorecon: command not found 2021-04-03T03:08:11.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:08:11.738 DEBUG:teuthology.orchestra.run.gibba015:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:11.829 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:11.820+0000 7f79d2590d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2021-04-03T03:08:11.829 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:11.820+0000 7f79d2590d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2021-04-03T03:08:11.830 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:11.820+0000 7f79d2590d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-04-03T03:08:11.830 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:11.820+0000 7f79d2590d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-04-03T03:08:11.830 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:11.820+0000 7f79d2590d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-04-03T03:08:11.831 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:11.820+0000 7f79d2590d80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2021-04-03T03:08:11.837 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:11.828+0000 7f79d2590d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:14.301 DEBUG:teuthology.orchestra.run.gibba015:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-04-03T03:08:14.309 DEBUG:teuthology.orchestra.run.gibba015:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:14.371 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:14.360+0000 7f955e05ad80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2021-04-03T03:08:14.371 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:14.360+0000 7f955e05ad80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2021-04-03T03:08:14.371 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:14.360+0000 7f955e05ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-04-03T03:08:14.372 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:14.360+0000 7f955e05ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-04-03T03:08:14.372 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:14.360+0000 7f955e05ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-04-03T03:08:14.372 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:14.364+0000 7f955e05ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2021-04-03T03:08:14.379 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:14.372+0000 7f955e05ad80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:16.826 DEBUG:teuthology.orchestra.run.gibba015:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-04-03T03:08:16.877 DEBUG:teuthology.orchestra.run.gibba015:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:16.937 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:16.928+0000 7f6112b06d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2021-04-03T03:08:16.938 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:16.928+0000 7f6112b06d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2021-04-03T03:08:16.938 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:16.928+0000 7f6112b06d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-04-03T03:08:16.938 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:16.928+0000 7f6112b06d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-04-03T03:08:16.938 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:16.928+0000 7f6112b06d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-04-03T03:08:16.939 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:16.928+0000 7f6112b06d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2021-04-03T03:08:16.946 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:16.936+0000 7f6112b06d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:19.376 DEBUG:teuthology.orchestra.run.gibba015:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-04-03T03:08:19.426 DEBUG:teuthology.orchestra.run.gibba015:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:19.486 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:19.476+0000 7f5782510d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2021-04-03T03:08:19.486 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:19.476+0000 7f5782510d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2021-04-03T03:08:19.486 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:19.476+0000 7f5782510d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-04-03T03:08:19.486 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:19.476+0000 7f5782510d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-04-03T03:08:19.487 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:19.476+0000 7f5782510d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-04-03T03:08:19.487 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:19.476+0000 7f5782510d80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2021-04-03T03:08:19.494 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:19.484+0000 7f5782510d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-04-03T03:08:21.937 DEBUG:teuthology.orchestra.run.gibba015:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-04-03T03:08:21.989 INFO:tasks.ceph:Reading keys from all nodes... 2021-04-03T03:08:21.990 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:22.000 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.U7EFxf8AZi 2021-04-03T03:08:22.000 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.U7EFxf8AZi 2021-04-03T03:08:22.054 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.U7EFxf8AZi 2021-04-03T03:08:22.154 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.U7EFxf8AZi is 56B 2021-04-03T03:08:22.169 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.U7EFxf8AZi 2021-04-03T03:08:22.178 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:22.227 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.bVyVi6ISwM 2021-04-03T03:08:22.228 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.bVyVi6ISwM 2021-04-03T03:08:22.278 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.bVyVi6ISwM 2021-04-03T03:08:22.378 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.bVyVi6ISwM is 56B 2021-04-03T03:08:22.392 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.bVyVi6ISwM 2021-04-03T03:08:22.400 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:22.447 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.AuVj7sHvnR 2021-04-03T03:08:22.448 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.AuVj7sHvnR 2021-04-03T03:08:22.508 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.AuVj7sHvnR 2021-04-03T03:08:22.622 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.AuVj7sHvnR is 56B 2021-04-03T03:08:22.674 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.AuVj7sHvnR 2021-04-03T03:08:22.679 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:22.731 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.e0xpZVvDBo 2021-04-03T03:08:22.732 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.e0xpZVvDBo 2021-04-03T03:08:22.782 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.e0xpZVvDBo 2021-04-03T03:08:22.882 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.e0xpZVvDBo is 56B 2021-04-03T03:08:22.896 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.e0xpZVvDBo 2021-04-03T03:08:22.899 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:22.944 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.tlTX6xCRaw 2021-04-03T03:08:22.944 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.tlTX6xCRaw 2021-04-03T03:08:23.008 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.tlTX6xCRaw 2021-04-03T03:08:23.109 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.tlTX6xCRaw is 56B 2021-04-03T03:08:23.162 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.tlTX6xCRaw 2021-04-03T03:08:23.166 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:23.213 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.1rxbUy33it 2021-04-03T03:08:23.213 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.1rxbUy33it 2021-04-03T03:08:23.276 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.1rxbUy33it 2021-04-03T03:08:23.374 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.1rxbUy33it is 56B 2021-04-03T03:08:23.426 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.1rxbUy33it 2021-04-03T03:08:23.435 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:23.443 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.6WIatmVQqP 2021-04-03T03:08:23.444 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.6WIatmVQqP 2021-04-03T03:08:23.494 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.6WIatmVQqP 2021-04-03T03:08:23.605 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.6WIatmVQqP is 56B 2021-04-03T03:08:23.657 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.6WIatmVQqP 2021-04-03T03:08:23.662 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:23.711 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.BshTY9fcCw 2021-04-03T03:08:23.711 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.BshTY9fcCw 2021-04-03T03:08:23.761 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.BshTY9fcCw 2021-04-03T03:08:23.861 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.BshTY9fcCw is 56B 2021-04-03T03:08:23.871 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.BshTY9fcCw 2021-04-03T03:08:23.891 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:23.943 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.JpJYFNDe58 2021-04-03T03:08:23.943 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.JpJYFNDe58 2021-04-03T03:08:24.003 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.JpJYFNDe58 2021-04-03T03:08:24.105 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.JpJYFNDe58 is 56B 2021-04-03T03:08:24.114 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.JpJYFNDe58 2021-04-03T03:08:24.119 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:24.166 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.r3dZVPa6Y9 2021-04-03T03:08:24.167 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.r3dZVPa6Y9 2021-04-03T03:08:24.227 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.r3dZVPa6Y9 2021-04-03T03:08:24.329 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.r3dZVPa6Y9 is 56B 2021-04-03T03:08:24.382 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.r3dZVPa6Y9 2021-04-03T03:08:24.390 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:24.435 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.D6bAxjUnEn 2021-04-03T03:08:24.435 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.D6bAxjUnEn 2021-04-03T03:08:24.489 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.D6bAxjUnEn 2021-04-03T03:08:24.590 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.D6bAxjUnEn is 56B 2021-04-03T03:08:24.603 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.D6bAxjUnEn 2021-04-03T03:08:24.612 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:24.663 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.Bd7bfWC2RJ 2021-04-03T03:08:24.663 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.Bd7bfWC2RJ 2021-04-03T03:08:24.713 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.Bd7bfWC2RJ 2021-04-03T03:08:24.814 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.Bd7bfWC2RJ is 56B 2021-04-03T03:08:24.827 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.Bd7bfWC2RJ 2021-04-03T03:08:24.841 DEBUG:teuthology.orchestra.remote:gibba026:/etc/ceph/ceph.client.0.keyring is 59B 2021-04-03T03:08:24.867 DEBUG:teuthology.orchestra.remote:gibba028:/etc/ceph/ceph.client.1.keyring is 59B 2021-04-03T03:08:24.927 DEBUG:teuthology.orchestra.remote:gibba031:/etc/ceph/ceph.client.2.keyring is 59B 2021-04-03T03:08:24.978 INFO:tasks.ceph:Adding keys to all mons... 2021-04-03T03:08:24.978 DEBUG:teuthology.orchestra.run.gibba011:> sudo tee -a /etc/ceph/ceph.keyring 2021-04-03T03:08:24.981 DEBUG:teuthology.orchestra.run.gibba015:> sudo tee -a /etc/ceph/ceph.keyring 2021-04-03T03:08:24.988 INFO:teuthology.orchestra.run.gibba011.stdout:[mgr.y] 2021-04-03T03:08:24.989 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA== 2021-04-03T03:08:24.989 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.a] 2021-04-03T03:08:24.989 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw== 2021-04-03T03:08:24.989 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.0] 2021-04-03T03:08:24.989 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ== 2021-04-03T03:08:24.990 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.1] 2021-04-03T03:08:24.990 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q== 2021-04-03T03:08:24.990 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.2] 2021-04-03T03:08:24.990 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg== 2021-04-03T03:08:24.991 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.3] 2021-04-03T03:08:24.991 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ== 2021-04-03T03:08:24.991 INFO:teuthology.orchestra.run.gibba011.stdout:[mgr.x] 2021-04-03T03:08:24.991 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg== 2021-04-03T03:08:24.992 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.b] 2021-04-03T03:08:24.992 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA== 2021-04-03T03:08:24.992 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.4] 2021-04-03T03:08:24.992 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q== 2021-04-03T03:08:24.992 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.5] 2021-04-03T03:08:24.993 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ== 2021-04-03T03:08:24.993 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.6] 2021-04-03T03:08:24.993 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA== 2021-04-03T03:08:24.993 INFO:teuthology.orchestra.run.gibba011.stdout:[osd.7] 2021-04-03T03:08:24.993 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA== 2021-04-03T03:08:24.994 INFO:teuthology.orchestra.run.gibba011.stdout:[client.0] 2021-04-03T03:08:24.994 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg== 2021-04-03T03:08:24.994 INFO:teuthology.orchestra.run.gibba011.stdout:[client.1] 2021-04-03T03:08:24.994 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ== 2021-04-03T03:08:24.995 INFO:teuthology.orchestra.run.gibba011.stdout:[client.2] 2021-04-03T03:08:24.995 INFO:teuthology.orchestra.run.gibba011.stdout: key = AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA== 2021-04-03T03:08:24.999 INFO:teuthology.orchestra.run.gibba015.stdout:[mgr.y] 2021-04-03T03:08:24.999 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA== 2021-04-03T03:08:25.000 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.a] 2021-04-03T03:08:25.000 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw== 2021-04-03T03:08:25.000 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.0] 2021-04-03T03:08:25.000 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ== 2021-04-03T03:08:25.000 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.1] 2021-04-03T03:08:25.001 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q== 2021-04-03T03:08:25.001 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.2] 2021-04-03T03:08:25.001 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg== 2021-04-03T03:08:25.001 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.3] 2021-04-03T03:08:25.002 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ== 2021-04-03T03:08:25.002 INFO:teuthology.orchestra.run.gibba015.stdout:[mgr.x] 2021-04-03T03:08:25.002 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg== 2021-04-03T03:08:25.002 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.b] 2021-04-03T03:08:25.003 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA== 2021-04-03T03:08:25.003 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.4] 2021-04-03T03:08:25.003 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q== 2021-04-03T03:08:25.003 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.5] 2021-04-03T03:08:25.003 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ== 2021-04-03T03:08:25.004 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.6] 2021-04-03T03:08:25.004 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA== 2021-04-03T03:08:25.004 INFO:teuthology.orchestra.run.gibba015.stdout:[osd.7] 2021-04-03T03:08:25.004 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA== 2021-04-03T03:08:25.005 INFO:teuthology.orchestra.run.gibba015.stdout:[client.0] 2021-04-03T03:08:25.005 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg== 2021-04-03T03:08:25.005 INFO:teuthology.orchestra.run.gibba015.stdout:[client.1] 2021-04-03T03:08:25.005 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ== 2021-04-03T03:08:25.005 INFO:teuthology.orchestra.run.gibba015.stdout:[client.2] 2021-04-03T03:08:25.006 INFO:teuthology.orchestra.run.gibba015.stdout: key = AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA== 2021-04-03T03:08:25.007 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-04-03T03:08:25.042 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-04-03T03:08:25.069 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-04-03T03:08:25.109 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-04-03T03:08:25.147 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.190 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.255 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.258 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.283 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.321 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.355 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.394 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.462 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-04-03T03:08:25.465 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-04-03T03:08:25.503 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-04-03T03:08:25.547 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-04-03T03:08:25.572 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.614 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.666 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.669 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.705 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.738 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.784 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.810 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-04-03T03:08:25.895 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-04-03T03:08:25.898 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-04-03T03:08:25.953 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-04-03T03:08:25.956 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-04-03T03:08:25.989 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-04-03T03:08:26.026 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-04-03T03:08:26.063 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-04-03T03:08:26.063 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2021-04-03T03:08:26.095 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-04-03T03:08:26.212 DEBUG:teuthology.orchestra.run.gibba011:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2021-04-03T03:08:26.222 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2021-04-03T03:08:26.228 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-04-03T03:08:26.331 DEBUG:teuthology.orchestra.run.gibba015:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2021-04-03T03:08:26.342 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2021-04-03T03:08:26.390 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-04-03T03:08:26.476 DEBUG:teuthology.orchestra.run.gibba015:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2021-04-03T03:08:26.528 DEBUG:teuthology.orchestra.run.gibba011:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:26.531 DEBUG:teuthology.orchestra.run.gibba015:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-04-03T03:08:26.578 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2021-04-03T03:08:26.579 INFO:tasks.ceph.mon.a:Restarting daemon 2021-04-03T03:08:26.580 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2021-04-03T03:08:26.583 INFO:tasks.ceph.mon.a:Started 2021-04-03T03:08:26.583 INFO:tasks.ceph.mon.b:Restarting daemon 2021-04-03T03:08:26.584 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2021-04-03T03:08:26.622 INFO:tasks.ceph.mon.b:Started 2021-04-03T03:08:26.622 INFO:tasks.ceph.mon.c:Restarting daemon 2021-04-03T03:08:26.623 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2021-04-03T03:08:26.625 INFO:tasks.ceph.mon.c:Started 2021-04-03T03:08:26.625 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2021-04-03T03:08:26.626 INFO:tasks.ceph.mgr.y:Restarting daemon 2021-04-03T03:08:26.626 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2021-04-03T03:08:26.628 INFO:tasks.ceph.mgr.y:Started 2021-04-03T03:08:26.629 INFO:tasks.ceph.mgr.x:Restarting daemon 2021-04-03T03:08:26.629 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2021-04-03T03:08:26.631 INFO:tasks.ceph.mgr.x:Started 2021-04-03T03:08:26.631 INFO:tasks.ceph:Setting crush tunables to default 2021-04-03T03:08:26.632 DEBUG:teuthology.orchestra.run.gibba011:> sudo ceph --cluster ceph osd crush tunables default 2021-04-03T03:08:26.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.887+0000 7f15917b8700 1 Processor -- start 2021-04-03T03:08:26.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.887+0000 7f15917b8700 1 -- start start 2021-04-03T03:08:26.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.887+0000 7f15917b8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 0x7f1588046870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.887+0000 7f15917b8700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 0x7f158803e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.887+0000 7f15917b8700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158803fa00 con 0x7f1588041d40 2021-04-03T03:08:26.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.887+0000 7f15917b8700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880f1550 con 0x7f15880f1170 2021-04-03T03:08:26.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.887+0000 7f15917b8700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158803fc50 con 0x7f1588046d60 2021-04-03T03:08:26.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1593433700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 0x7f1588046870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:26.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1593433700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 0x7f1588046870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:48662/0 (socket says 172.21.2.111:48662) 2021-04-03T03:08:26.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1593433700 1 -- 172.21.2.111:0/1699874933 learned_addr learned my addr 172.21.2.111:0/1699874933 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:26.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1593433700 1 -- 172.21.2.111:0/1699874933 >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 msgr2=0x7f1588046870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2021-04-03T03:08:26.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1593433700 1 -- 172.21.2.111:0/1699874933 >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 msgr2=0x7f1588046870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-04-03T03:08:26.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1593433700 1 --2- 172.21.2.111:0/1699874933 >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 0x7f1588046870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-04-03T03:08:26.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1593433700 1 --2- 172.21.2.111:0/1699874933 >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 0x7f1588046870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-04-03T03:08:26.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1590fb7700 1 --2- 172.21.2.111:0/1699874933 >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 0x7f158803e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:26.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1590fb7700 1 -- 172.21.2.111:0/1699874933 >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 msgr2=0x7f158803e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2021-04-03T03:08:26.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1590fb7700 1 -- 172.21.2.111:0/1699874933 >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 msgr2=0x7f158803e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-04-03T03:08:26.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1590fb7700 1 --2- 172.21.2.111:0/1699874933 >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 0x7f158803e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-04-03T03:08:26.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.891+0000 7f1590fb7700 1 --2- 172.21.2.111:0/1699874933 >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 0x7f158803e9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-04-03T03:08:26.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.899+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58556199 0 0) 0x7f158803fa00 con 0x7f1588041d40 2021-04-03T03:08:26.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.899+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 --> v1:172.21.2.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1578003240 con 0x7f1588041d40 2021-04-03T03:08:26.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 <== mon.1 v1:172.21.2.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4003906957 0 0) 0x7f1578003240 con 0x7f1588041d40 2021-04-03T03:08:26.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 msgr2=0x7f158803e9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-04-03T03:08:26.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 --2- 172.21.2.111:0/1699874933 >> v2:172.21.2.115:3300/0 conn(0x7f1588046d60 0x7f158803e9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 msgr2=0x7f1588046870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-04-03T03:08:26.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 --2- 172.21.2.111:0/1699874933 >> v2:172.21.2.111:3300/0 conn(0x7f15880f1170 0x7f1588046870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 --> v1:172.21.2.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158803fef0 con 0x7f1588041d40 2021-04-03T03:08:26.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 <== mon.1 v1:172.21.2.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f158c0450f0 con 0x7f1588041d40 2021-04-03T03:08:26.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 >> v1:172.21.2.115:6789/0 conn(0x7f1588041d40 legacy=0x7f158803eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:26.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15780037b0 0x7f1578003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780044e0 0x7f1578004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f157804f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880f1550 con 0x7f1578004e50 2021-04-03T03:08:26.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158803fc50 con 0x7f15780037b0 2021-04-03T03:08:26.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157804fc30 con 0x7f15780044e0 2021-04-03T03:08:26.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f157804f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:26.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1590fb7700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780044e0 0x7f1578004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:26.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 -- 172.21.2.111:0/1699874933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780044e0 msgr2=0x7f1578004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:26.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780044e0 0x7f1578004960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 -- 172.21.2.111:0/1699874933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15780037b0 msgr2=0x7f1578003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:26.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15780037b0 0x7f1578003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:26.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 -- 172.21.2.111:0/1699874933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1578003240 con 0x7f1578004e50 2021-04-03T03:08:26.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 -- 172.21.2.111:0/1699874933 reap_dead start 2021-04-03T03:08:26.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1591fb9700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f157804f740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f158c069660 tx=0x7f1588042240).ready entity=mon.0 client_cookie=8c6456fbed867465 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:26.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158c05d450 con 0x7f1578004e50 2021-04-03T03:08:26.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c05d5b0 con 0x7f1578004e50 2021-04-03T03:08:26.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f1583fff700 1 -- 172.21.2.111:0/1699874933 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158c0685b0 con 0x7f1578004e50 2021-04-03T03:08:26.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f15917b8700 1 -- 172.21.2.111:0/1699874933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 msgr2=0x7f157804f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:26.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f15917b8700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f157804f740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f158c069660 tx=0x7f1588042240).stop 2021-04-03T03:08:26.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f15917b8700 1 -- 172.21.2.111:0/1699874933 shutdown_connections 2021-04-03T03:08:26.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f15917b8700 1 --2- 172.21.2.111:0/1699874933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f157804f740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.903+0000 7f15917b8700 1 -- 172.21.2.111:0/1699874933 >> 172.21.2.111:0/1699874933 conn(0x7f158800cf70 msgr2=0x7f1588046160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:26.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- 172.21.2.111:0/1699874933 shutdown_connections 2021-04-03T03:08:26.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- 172.21.2.111:0/1699874933 wait complete. 2021-04-03T03:08:26.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 Processor -- start 2021-04-03T03:08:26.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- start start 2021-04-03T03:08:26.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f1588043120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f158803c5b0 0x7f15880d7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15880d7f30 0x7f15880d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:26.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880d6f60 con 0x7f1578004e50 2021-04-03T03:08:26.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880d88a0 con 0x7f158803c5b0 2021-04-03T03:08:26.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15880d5d40 con 0x7f15880d7f30 2021-04-03T03:08:26.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f1588043120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:26.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f1588043120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:48690/0 (socket says 172.21.2.111:48690) 2021-04-03T03:08:26.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 -- 172.21.2.111:0/1423135085 learned_addr learned my addr 172.21.2.111:0/1423135085 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:26.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1590fb7700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f158803c5b0 0x7f15880d7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:26.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1591fb9700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15880d7f30 0x7f15880d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:26.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 -- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15880d7f30 msgr2=0x7f15880d83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:26.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15880d7f30 0x7f15880d83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 -- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f158803c5b0 msgr2=0x7f15880d7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:26.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f158803c5b0 0x7f15880d7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 -- 172.21.2.111:0/1423135085 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880d5fe0 con 0x7f1578004e50 2021-04-03T03:08:26.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1593433700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f1588043120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f15880d78e0 tx=0x7f1584009750).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:26.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1581ffb700 1 -- 172.21.2.111:0/1423135085 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1584010070 con 0x7f1578004e50 2021-04-03T03:08:26.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1581ffb700 1 -- 172.21.2.111:0/1423135085 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1584015420 con 0x7f1578004e50 2021-04-03T03:08:26.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1581ffb700 1 -- 172.21.2.111:0/1423135085 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1584015780 con 0x7f1578004e50 2021-04-03T03:08:26.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15880d6280 con 0x7f1578004e50 2021-04-03T03:08:26.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158803ca30 con 0x7f1578004e50 2021-04-03T03:08:26.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1581ffb700 1 -- 172.21.2.111:0/1423135085 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f1584015ab0 con 0x7f1578004e50 2021-04-03T03:08:26.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f1581ffb700 1 -- 172.21.2.111:0/1423135085 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f158401f850 con 0x7f1578004e50 2021-04-03T03:08:26.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.907+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880d6410 con 0x7f1578004e50 2021-04-03T03:08:26.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.911+0000 7f1581ffb700 1 -- 172.21.2.111:0/1423135085 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f158401e3e0 con 0x7f1578004e50 2021-04-03T03:08:26.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.963+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1588004be0 con 0x7f1578004e50 2021-04-03T03:08:26.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.967+0000 7f1581ffb700 1 -- 172.21.2.111:0/1423135085 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f158401fa60 con 0x7f1578004e50 2021-04-03T03:08:26.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.967+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 msgr2=0x7f1588043120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.967+0000 7f15917b8700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f1588043120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f15880d78e0 tx=0x7f1584009750).stop 2021-04-03T03:08:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.971+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 shutdown_connections 2021-04-03T03:08:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.971+0000 7f15917b8700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15880d7f30 0x7f15880d83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.971+0000 7f15917b8700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1578004e50 0x7f1588043120 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.971+0000 7f15917b8700 1 --2- 172.21.2.111:0/1423135085 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f158803c5b0 0x7f15880d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:26.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.971+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 >> 172.21.2.111:0/1423135085 conn(0x7f158800cf70 msgr2=0x7f1588005fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:26.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.971+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 shutdown_connections 2021-04-03T03:08:26.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:26.971+0000 7f15917b8700 1 -- 172.21.2.111:0/1423135085 wait complete. 2021-04-03T03:08:26.983 INFO:teuthology.orchestra.run.gibba011.stderr:adjusted tunables profile to default 2021-04-03T03:08:26.987 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2021-04-03T03:08:26.988 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:26.991 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.SvdWClrvUW 2021-04-03T03:08:26.991 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.SvdWClrvUW 2021-04-03T03:08:27.037 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.SvdWClrvUW 2021-04-03T03:08:27.132 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.SvdWClrvUW is 37B 2021-04-03T03:08:27.180 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.SvdWClrvUW 2021-04-03T03:08:27.184 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:27.226 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.xhFHtIZLlM 2021-04-03T03:08:27.227 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.xhFHtIZLlM 2021-04-03T03:08:27.273 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.xhFHtIZLlM 2021-04-03T03:08:27.368 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.xhFHtIZLlM is 37B 2021-04-03T03:08:27.416 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.xhFHtIZLlM 2021-04-03T03:08:27.419 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:27.467 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.nEzuvc7aM3 2021-04-03T03:08:27.468 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.nEzuvc7aM3 2021-04-03T03:08:27.513 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.nEzuvc7aM3 2021-04-03T03:08:27.610 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.nEzuvc7aM3 is 37B 2021-04-03T03:08:27.633 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:27.654 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:27.657 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:27.660 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:27.664 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:27.700 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.nEzuvc7aM3 2021-04-03T03:08:27.709 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T03:08:27.774 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.wFNL1SyjFP 2021-04-03T03:08:27.775 DEBUG:teuthology.orchestra.run.gibba011:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.wFNL1SyjFP 2021-04-03T03:08:27.825 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.wFNL1SyjFP 2021-04-03T03:08:27.925 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.wFNL1SyjFP is 37B 2021-04-03T03:08:27.938 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.wFNL1SyjFP 2021-04-03T03:08:27.946 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:27.949 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.65k57Y9fpf 2021-04-03T03:08:27.950 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.65k57Y9fpf 2021-04-03T03:08:27.996 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.65k57Y9fpf 2021-04-03T03:08:28.096 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.65k57Y9fpf is 37B 2021-04-03T03:08:28.144 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.65k57Y9fpf 2021-04-03T03:08:28.147 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:28.190 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.oFV3hdl9Tm 2021-04-03T03:08:28.190 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.oFV3hdl9Tm 2021-04-03T03:08:28.236 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.oFV3hdl9Tm 2021-04-03T03:08:28.337 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.oFV3hdl9Tm is 37B 2021-04-03T03:08:28.350 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.oFV3hdl9Tm 2021-04-03T03:08:28.358 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:28.404 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.1bXuzZI3kj 2021-04-03T03:08:28.405 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.1bXuzZI3kj 2021-04-03T03:08:28.453 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.1bXuzZI3kj 2021-04-03T03:08:28.547 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.1bXuzZI3kj is 37B 2021-04-03T03:08:28.595 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.1bXuzZI3kj 2021-04-03T03:08:28.598 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T03:08:28.642 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.SW3Enj4Xja 2021-04-03T03:08:28.642 DEBUG:teuthology.orchestra.run.gibba015:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.SW3Enj4Xja 2021-04-03T03:08:28.688 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.SW3Enj4Xja 2021-04-03T03:08:28.783 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.SW3Enj4Xja is 37B 2021-04-03T03:08:28.831 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.SW3Enj4Xja 2021-04-03T03:08:28.835 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new b944db97-2a22-405b-847f-9d436ceab0ad 0 2021-04-03T03:08:28.950 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 Processor -- start 2021-04-03T03:08:28.950 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 -- start start 2021-04-03T03:08:28.950 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f052c0f1170 0x7f052c046870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:28.950 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:28.951 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c03fa00 con 0x7f052c046d60 2021-04-03T03:08:28.951 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c0f1550 con 0x7f052c0f1170 2021-04-03T03:08:28.951 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c03fc50 con 0x7f052c041d40 2021-04-03T03:08:28.951 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f0541b1c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f052c0f1170 0x7f052c046870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:28.952 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:28.952 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f0541b1c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f052c0f1170 0x7f052c046870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:48980/0 (socket says 172.21.2.115:48980) 2021-04-03T03:08:28.952 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f0541b1c700 1 -- 172.21.2.115:0/1664917965 learned_addr learned my addr 172.21.2.115:0/1664917965 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:28.952 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053bfff700 1 -- 172.21.2.115:0/1664917965 >> v2:172.21.2.115:3300/0 conn(0x7f052c0f1170 msgr2=0x7f052c046870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:28.952 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053bfff700 1 --2- 172.21.2.115:0/1664917965 >> v2:172.21.2.115:3300/0 conn(0x7f052c0f1170 0x7f052c046870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:28.953 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053bfff700 1 -- 172.21.2.115:0/1664917965 >> v1:172.21.2.111:6789/0 conn(0x7f052c046d60 legacy=0x7f052c03e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:28.953 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053bfff700 1 -- 172.21.2.115:0/1664917965 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c03fef0 con 0x7f052c041d40 2021-04-03T03:08:28.953 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053bfff700 1 --2- 172.21.2.115:0/1664917965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03f0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f0534000bd0 tx=0x7f05340096a0).ready entity=mon.2 client_cookie=64d8b8780d0c6a60 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:28.953 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053a7fc700 1 -- 172.21.2.115:0/1664917965 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053400c620 con 0x7f052c041d40 2021-04-03T03:08:28.953 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053a7fc700 1 -- 172.21.2.115:0/1664917965 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053400c780 con 0x7f052c041d40 2021-04-03T03:08:28.954 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053a7fc700 1 -- 172.21.2.115:0/1664917965 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053400cae0 con 0x7f052c041d40 2021-04-03T03:08:28.954 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 -- 172.21.2.115:0/1664917965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 msgr2=0x7f052c03f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:28.954 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 --2- 172.21.2.115:0/1664917965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03f0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f0534000bd0 tx=0x7f05340096a0).stop 2021-04-03T03:08:28.954 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 -- 172.21.2.115:0/1664917965 shutdown_connections 2021-04-03T03:08:28.955 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 --2- 172.21.2.115:0/1664917965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03f0d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:28.955 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 --2- 172.21.2.115:0/1664917965 >> v2:172.21.2.115:3300/0 conn(0x7f052c0f1170 0x7f052c046870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:28.955 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.936+0000 7f053b7fe700 1 -- 172.21.2.115:0/1664917965 >> 172.21.2.115:0/1664917965 conn(0x7f052c00cf70 msgr2=0x7f052c046160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:28.955 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- 172.21.2.115:0/1664917965 shutdown_connections 2021-04-03T03:08:28.955 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- 172.21.2.115:0/1664917965 wait complete. 2021-04-03T03:08:28.956 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 Processor -- start 2021-04-03T03:08:28.956 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- start start 2021-04-03T03:08:28.956 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:28.956 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f052c046d60 0x7f052c0f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:28.957 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f052c0f1170 0x7f052c0f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:28.957 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0de170 con 0x7f052c046d60 2021-04-03T03:08:28.957 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0dfa50 con 0x7f052c0f1170 2021-04-03T03:08:28.957 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0df7b0 con 0x7f052c041d40 2021-04-03T03:08:28.957 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0541b1c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:28.958 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f052c0f1170 0x7f052c0f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:28.958 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0541b1c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.115:51240/0 (socket says 172.21.2.115:51240) 2021-04-03T03:08:28.958 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0541b1c700 1 -- 172.21.2.115:0/226147404 learned_addr learned my addr 172.21.2.115:0/226147404 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:28.958 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053affd700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f052c046d60 0x7f052c0f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:28.958 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053bfff700 1 -- 172.21.2.115:0/226147404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 msgr2=0x7f052c03c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:28.959 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053bfff700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:28.959 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053bfff700 1 -- 172.21.2.115:0/226147404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f052c046d60 msgr2=0x7f052c0f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:28.959 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053bfff700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f052c046d60 0x7f052c0f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:28.959 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053bfff700 1 -- 172.21.2.115:0/226147404 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c03aa00 con 0x7f052c0f1170 2021-04-03T03:08:28.959 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053affd700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f052c046d60 0x7f052c0f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:28.960 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053bfff700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f052c0f1170 0x7f052c0f0580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f0534000bd0 tx=0x7f0534016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:28.961 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0523fff700 1 -- 172.21.2.115:0/226147404 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0534018030 con 0x7f052c0f1170 2021-04-03T03:08:28.961 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0523fff700 1 -- 172.21.2.115:0/226147404 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534019040 con 0x7f052c0f1170 2021-04-03T03:08:28.962 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0523fff700 1 -- 172.21.2.115:0/226147404 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053400b420 con 0x7f052c0f1170 2021-04-03T03:08:28.962 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c03be50 con 0x7f052c0f1170 2021-04-03T03:08:28.962 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c03a530 con 0x7f052c0f1170 2021-04-03T03:08:28.962 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0523fff700 1 -- 172.21.2.115:0/226147404 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f0534008030 con 0x7f052c0f1170 2021-04-03T03:08:28.962 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f0523fff700 1 -- 172.21.2.115:0/226147404 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f0534027b50 con 0x7f052c0f1170 2021-04-03T03:08:28.963 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.940+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c004be0 con 0x7f052c0f1170 2021-04-03T03:08:28.963 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.944+0000 7f0523fff700 1 -- 172.21.2.115:0/226147404 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f0534027d60 con 0x7f052c0f1170 2021-04-03T03:08:29.004 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:28.996+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", "id": 0} v 0) v1 -- 0x7f052c0df5a0 con 0x7f052c0f1170 2021-04-03T03:08:29.008 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f0523fff700 1 -- 172.21.2.115:0/226147404 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0534031430 con 0x7f052c0f1170 2021-04-03T03:08:29.009 INFO:teuthology.orchestra.run.gibba015.stdout:0 2021-04-03T03:08:29.010 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f052c0f1170 msgr2=0x7f052c0f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.010 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f052c0f1170 0x7f052c0f0580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f0534000bd0 tx=0x7f0534016000).stop 2021-04-03T03:08:29.010 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 shutdown_connections 2021-04-03T03:08:29.011 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f052c0f1170 0x7f052c0f0580 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.011 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f052c041d40 0x7f052c03c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.011 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 --2- 172.21.2.115:0/226147404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f052c046d60 0x7f052c0f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.011 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 >> 172.21.2.115:0/226147404 conn(0x7f052c00cf70 msgr2=0x7f052c005430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.012 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 shutdown_connections 2021-04-03T03:08:29.012 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.000+0000 7f053b7fe700 1 -- 172.21.2.115:0/226147404 wait complete. 2021-04-03T03:08:29.016 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new a1b55cbd-c469-405c-92a4-a45166c2194a 1 2021-04-03T03:08:29.076 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 Processor -- start 2021-04-03T03:08:29.076 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- start start 2021-04-03T03:08:29.076 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e8080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.076 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe0e8025480 0x7fe0e80af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.077 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e80b3ca0 con 0x7fe0e80b38c0 2021-04-03T03:08:29.077 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e8025b90 con 0x7fe0e8025480 2021-04-03T03:08:29.077 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8025e30 con 0x7fe0e8153440 2021-04-03T03:08:29.077 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe0e8025480 0x7fe0e80af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.078 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe0e8025480 0x7fe0e80af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:48994/0 (socket says 172.21.2.115:48994) 2021-04-03T03:08:29.078 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 -- 172.21.2.115:0/3131706308 learned_addr learned my addr 172.21.2.115:0/3131706308 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.078 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0e7fff700 1 --2- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e8080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.078 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ecb21700 1 --1- 172.21.2.115:0/3131706308 >> v1:172.21.2.115:6789/0 conn(0x7fe0e80b38c0 0x7fe0e8080550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.115:55594/0 (socket says 172.21.2.115:55594) 2021-04-03T03:08:29.078 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 -- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 msgr2=0x7fe0e8080c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.079 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 --2- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e8080c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.079 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 -- 172.21.2.115:0/3131706308 >> v1:172.21.2.115:6789/0 conn(0x7fe0e80b38c0 legacy=0x7fe0e8080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.079 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 -- 172.21.2.115:0/3131706308 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80260d0 con 0x7fe0e8025480 2021-04-03T03:08:29.079 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0e7fff700 1 --2- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e8080c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.080 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ee79c700 1 --2- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e80af2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe0e00032b0 tx=0x7fe0e0009270).ready entity=mon.1 client_cookie=a1ea525435df7b8e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.080 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0e77fe700 1 -- 172.21.2.115:0/3131706308 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0e0009e60 con 0x7fe0e8025480 2021-04-03T03:08:29.080 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0e77fe700 1 -- 172.21.2.115:0/3131706308 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e000bc00 con 0x7fe0e8025480 2021-04-03T03:08:29.080 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0e77fe700 1 -- 172.21.2.115:0/3131706308 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0e000bd70 con 0x7fe0e8025480 2021-04-03T03:08:29.080 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 msgr2=0x7fe0e80af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.081 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 --2- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e80af2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe0e00032b0 tx=0x7fe0e0009270).stop 2021-04-03T03:08:29.081 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- 172.21.2.115:0/3131706308 shutdown_connections 2021-04-03T03:08:29.081 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 --2- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e80af2f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.081 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 --2- 172.21.2.115:0/3131706308 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e8080c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.081 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- 172.21.2.115:0/3131706308 >> 172.21.2.115:0/3131706308 conn(0x7fe0e8061610 msgr2=0x7fe0e8061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.082 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- 172.21.2.115:0/3131706308 shutdown_connections 2021-04-03T03:08:29.082 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.064+0000 7fe0ed322700 1 -- 172.21.2.115:0/3131706308 wait complete. 2021-04-03T03:08:29.084 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 Processor -- start 2021-04-03T03:08:29.085 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 -- start start 2021-04-03T03:08:29.085 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e812dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.085 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe0e80b38c0 0x7fe0e8080910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.085 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e80670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.086 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e809ef60 con 0x7fe0e80b38c0 2021-04-03T03:08:29.086 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80b4ee0 con 0x7fe0e8025480 2021-04-03T03:08:29.086 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80b5180 con 0x7fe0e8153440 2021-04-03T03:08:29.086 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e812dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.086 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ee79c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e80670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.087 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e812dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:48998/0 (socket says 172.21.2.115:48998) 2021-04-03T03:08:29.087 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 -- 172.21.2.115:0/1737945850 learned_addr learned my addr 172.21.2.115:0/1737945850 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.087 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0e7fff700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe0e80b38c0 0x7fe0e8080910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.087 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 -- 172.21.2.115:0/1737945850 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 msgr2=0x7fe0e80670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.087 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e80670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.088 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 -- 172.21.2.115:0/1737945850 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe0e80b38c0 msgr2=0x7fe0e8080910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.088 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe0e80b38c0 0x7fe0e8080910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.088 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 -- 172.21.2.115:0/1737945850 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80b5420 con 0x7fe0e8025480 2021-04-03T03:08:29.088 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0e7fff700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe0e80b38c0 0x7fe0e8080910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.089 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ecb21700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e812dd00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe0e80650a0 tx=0x7fe0dc007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.089 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0e57fa700 1 -- 172.21.2.115:0/1737945850 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0dc0073b0 con 0x7fe0e8025480 2021-04-03T03:08:29.089 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0e57fa700 1 -- 172.21.2.115:0/1737945850 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc005530 con 0x7fe0e8025480 2021-04-03T03:08:29.089 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e8080e00 con 0x7fe0e8025480 2021-04-03T03:08:29.089 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e80aa6c0 con 0x7fe0e8025480 2021-04-03T03:08:29.090 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0e57fa700 1 -- 172.21.2.115:0/1737945850 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0dc005890 con 0x7fe0e8025480 2021-04-03T03:08:29.090 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0e57fa700 1 -- 172.21.2.115:0/1737945850 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fe0dc01a060 con 0x7fe0e8025480 2021-04-03T03:08:29.090 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0e57fa700 1 -- 172.21.2.115:0/1737945850 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fe0dc013450 con 0x7fe0e8025480 2021-04-03T03:08:29.090 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.068+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e8066dc0 con 0x7fe0e8025480 2021-04-03T03:08:29.090 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.072+0000 7fe0e57fa700 1 -- 172.21.2.115:0/1737945850 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe0dc018070 con 0x7fe0e8025480 2021-04-03T03:08:29.131 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.120+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", "id": 1} v 0) v1 -- 0x7fe0e8057660 con 0x7fe0e8025480 2021-04-03T03:08:29.135 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.124+0000 7fe0e57fa700 1 -- 172.21.2.115:0/1737945850 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0dc013660 con 0x7fe0e8025480 2021-04-03T03:08:29.135 INFO:teuthology.orchestra.run.gibba015.stdout:1 2021-04-03T03:08:29.136 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 msgr2=0x7fe0e812dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.136 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e812dd00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe0e80650a0 tx=0x7fe0dc007c20).stop 2021-04-03T03:08:29.136 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 shutdown_connections 2021-04-03T03:08:29.137 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe0e8153440 0x7fe0e80670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.137 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0e8025480 0x7fe0e812dd00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.137 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 --2- 172.21.2.115:0/1737945850 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe0e80b38c0 0x7fe0e8080910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.137 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 >> 172.21.2.115:0/1737945850 conn(0x7fe0e8061610 msgr2=0x7fe0e8131900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.138 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 shutdown_connections 2021-04-03T03:08:29.138 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.128+0000 7fe0ed322700 1 -- 172.21.2.115:0/1737945850 wait complete. 2021-04-03T03:08:29.142 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new 0954b9bc-6c00-42c3-b666-fe5b20532ca1 2 2021-04-03T03:08:29.202 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 Processor -- start 2021-04-03T03:08:29.202 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 -- start start 2021-04-03T03:08:29.203 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff490153440 0x7ff490080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.203 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff490025480 0x7ff4900af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.203 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900b3ca0 con 0x7ff4900b38c0 2021-04-03T03:08:29.203 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff490025b90 con 0x7ff490153440 2021-04-03T03:08:29.204 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff490025e30 con 0x7ff490025480 2021-04-03T03:08:29.204 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff490025480 0x7ff4900af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.204 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff490025480 0x7ff4900af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49006/0 (socket says 172.21.2.115:49006) 2021-04-03T03:08:29.204 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 -- 172.21.2.115:0/3959938560 learned_addr learned my addr 172.21.2.115:0/3959938560 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.204 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff495bce700 1 --2- 172.21.2.115:0/3959938560 >> v2:172.21.2.111:3300/0 conn(0x7ff490153440 0x7ff490080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.205 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 -- 172.21.2.115:0/3959938560 >> v2:172.21.2.111:3300/0 conn(0x7ff490153440 msgr2=0x7ff490080c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.205 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 --2- 172.21.2.115:0/3959938560 >> v2:172.21.2.111:3300/0 conn(0x7ff490153440 0x7ff490080c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.205 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 -- 172.21.2.115:0/3959938560 >> v1:172.21.2.111:6789/0 conn(0x7ff4900b38c0 legacy=0x7ff490080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.205 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 -- 172.21.2.115:0/3959938560 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900260d0 con 0x7ff490025480 2021-04-03T03:08:29.206 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff49804a700 1 --2- 172.21.2.115:0/3959938560 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff4900af2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ff4900af0b0 tx=0x7ff488009270).ready entity=mon.1 client_cookie=6553dae8b343f0e3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.206 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff4953cd700 1 -- 172.21.2.115:0/3959938560 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff48800bbc0 con 0x7ff490025480 2021-04-03T03:08:29.206 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff4953cd700 1 -- 172.21.2.115:0/3959938560 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48800bd20 con 0x7ff490025480 2021-04-03T03:08:29.206 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff4953cd700 1 -- 172.21.2.115:0/3959938560 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff48800be90 con 0x7ff490025480 2021-04-03T03:08:29.206 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 -- 172.21.2.115:0/3959938560 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 msgr2=0x7ff4900af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.207 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.188+0000 7ff496bd0700 1 --2- 172.21.2.115:0/3959938560 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff4900af2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ff4900af0b0 tx=0x7ff488009270).stop 2021-04-03T03:08:29.207 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- 172.21.2.115:0/3959938560 shutdown_connections 2021-04-03T03:08:29.207 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 --2- 172.21.2.115:0/3959938560 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff4900af2f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.207 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 --2- 172.21.2.115:0/3959938560 >> v2:172.21.2.111:3300/0 conn(0x7ff490153440 0x7ff490080c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.207 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- 172.21.2.115:0/3959938560 >> 172.21.2.115:0/3959938560 conn(0x7ff490061610 msgr2=0x7ff490061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.208 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- 172.21.2.115:0/3959938560 shutdown_connections 2021-04-03T03:08:29.208 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- 172.21.2.115:0/3959938560 wait complete. 2021-04-03T03:08:29.208 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 Processor -- start 2021-04-03T03:08:29.208 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- start start 2021-04-03T03:08:29.209 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff49012f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.209 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4900b38c0 0x7ff490064f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.209 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff490153440 0x7ff490065460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.209 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490150d50 con 0x7ff4900b38c0 2021-04-03T03:08:29.209 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900aa560 con 0x7ff490025480 2021-04-03T03:08:29.210 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff490064a30 con 0x7ff490153440 2021-04-03T03:08:29.210 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff49012f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.210 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff49012f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49012/0 (socket says 172.21.2.115:49012) 2021-04-03T03:08:29.210 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 -- 172.21.2.115:0/2554812713 learned_addr learned my addr 172.21.2.115:0/2554812713 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.211 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff49804a700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff490153440 0x7ff490065460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.211 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff495bce700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4900b38c0 0x7ff490064f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.211 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 -- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff490153440 msgr2=0x7ff490065460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.211 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff490153440 0x7ff490065460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.211 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 -- 172.21.2.115:0/2554812713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4900b38c0 msgr2=0x7ff490064f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.212 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4900b38c0 0x7ff490064f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.212 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 -- 172.21.2.115:0/2554812713 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900640c0 con 0x7ff490025480 2021-04-03T03:08:29.212 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff49804a700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff490153440 0x7ff490065460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.212 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff4963cf700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff49012f660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff484001f60 tx=0x7ff484007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.212 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff482ffd700 1 -- 172.21.2.115:0/2554812713 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff48400e4a0 con 0x7ff490025480 2021-04-03T03:08:29.213 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff482ffd700 1 -- 172.21.2.115:0/2554812713 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48400e600 con 0x7ff490025480 2021-04-03T03:08:29.213 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49012fb50 con 0x7ff490025480 2021-04-03T03:08:29.213 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff482ffd700 1 -- 172.21.2.115:0/2554812713 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff48400e960 con 0x7ff490025480 2021-04-03T03:08:29.213 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff490067ce0 con 0x7ff490025480 2021-04-03T03:08:29.213 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff482ffd700 1 -- 172.21.2.115:0/2554812713 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7ff484010030 con 0x7ff490025480 2021-04-03T03:08:29.214 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff482ffd700 1 -- 172.21.2.115:0/2554812713 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7ff48400ec80 con 0x7ff490025480 2021-04-03T03:08:29.214 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.192+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490152f20 con 0x7ff490025480 2021-04-03T03:08:29.215 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.196+0000 7ff482ffd700 1 -- 172.21.2.115:0/2554812713 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7ff4840064f0 con 0x7ff490025480 2021-04-03T03:08:29.255 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.244+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", "id": 2} v 0) v1 -- 0x7ff490064c30 con 0x7ff490025480 2021-04-03T03:08:29.259 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.248+0000 7ff482ffd700 1 -- 172.21.2.115:0/2554812713 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff48400ee40 con 0x7ff490025480 2021-04-03T03:08:29.259 INFO:teuthology.orchestra.run.gibba015.stdout:2 2021-04-03T03:08:29.260 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.248+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 msgr2=0x7ff49012f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.260 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.248+0000 7ff496bd0700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff49012f660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff484001f60 tx=0x7ff484007c30).stop 2021-04-03T03:08:29.261 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.252+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 shutdown_connections 2021-04-03T03:08:29.261 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.252+0000 7ff496bd0700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff490153440 0x7ff490065460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.261 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.252+0000 7ff496bd0700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff490025480 0x7ff49012f660 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.261 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.252+0000 7ff496bd0700 1 --2- 172.21.2.115:0/2554812713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4900b38c0 0x7ff490064f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.262 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.252+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 >> 172.21.2.115:0/2554812713 conn(0x7ff490061610 msgr2=0x7ff4900b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.262 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.252+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 shutdown_connections 2021-04-03T03:08:29.262 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.252+0000 7ff496bd0700 1 -- 172.21.2.115:0/2554812713 wait complete. 2021-04-03T03:08:29.266 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new 69ab5309-0edd-4359-a5b8-56d9ceac0649 3 2021-04-03T03:08:29.325 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43ea344700 1 Processor -- start 2021-04-03T03:08:29.325 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43ea344700 1 -- start start 2021-04-03T03:08:29.326 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43ea344700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.326 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43ea344700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f43e4025480 0x7f43e40af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.326 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43ea344700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e40b3ca0 con 0x7f43e40b38c0 2021-04-03T03:08:29.326 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43ea344700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e4025b90 con 0x7f43e4025480 2021-04-03T03:08:29.326 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43ea344700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e4025e30 con 0x7f43e4153440 2021-04-03T03:08:29.327 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43e9342700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.327 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43eb7be700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f43e4025480 0x7f43e40af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.327 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43e9342700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4080c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.115:51270/0 (socket says 172.21.2.115:51270) 2021-04-03T03:08:29.327 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.312+0000 7f43e9342700 1 -- 172.21.2.115:0/1487023427 learned_addr learned my addr 172.21.2.115:0/1487023427 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.328 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 -- 172.21.2.115:0/1487023427 >> v2:172.21.2.115:3300/0 conn(0x7f43e4025480 msgr2=0x7f43e40af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.328 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 --2- 172.21.2.115:0/1487023427 >> v2:172.21.2.115:3300/0 conn(0x7f43e4025480 0x7f43e40af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.328 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 -- 172.21.2.115:0/1487023427 >> v1:172.21.2.111:6789/0 conn(0x7f43e40b38c0 legacy=0x7f43e4080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.328 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 -- 172.21.2.115:0/1487023427 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40260d0 con 0x7f43e4153440 2021-04-03T03:08:29.328 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 --2- 172.21.2.115:0/1487023427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4080c60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f43e0002f30 tx=0x7f43e0006e90).ready entity=mon.2 client_cookie=cc1396e803e5b8e2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.329 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e8b41700 1 -- 172.21.2.115:0/1487023427 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43e0009850 con 0x7f43e4153440 2021-04-03T03:08:29.329 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e8b41700 1 -- 172.21.2.115:0/1487023427 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e00099b0 con 0x7f43e4153440 2021-04-03T03:08:29.329 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e8b41700 1 -- 172.21.2.115:0/1487023427 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43e0009d10 con 0x7f43e4153440 2021-04-03T03:08:29.329 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- 172.21.2.115:0/1487023427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 msgr2=0x7f43e4080c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.329 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 --2- 172.21.2.115:0/1487023427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4080c60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f43e0002f30 tx=0x7f43e0006e90).stop 2021-04-03T03:08:29.330 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- 172.21.2.115:0/1487023427 shutdown_connections 2021-04-03T03:08:29.330 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 --2- 172.21.2.115:0/1487023427 >> v2:172.21.2.115:3300/0 conn(0x7f43e4025480 0x7f43e40af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.330 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 --2- 172.21.2.115:0/1487023427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4080c60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.330 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- 172.21.2.115:0/1487023427 >> 172.21.2.115:0/1487023427 conn(0x7f43e4061610 msgr2=0x7f43e4061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.331 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- 172.21.2.115:0/1487023427 shutdown_connections 2021-04-03T03:08:29.331 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- 172.21.2.115:0/1487023427 wait complete. 2021-04-03T03:08:29.333 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 Processor -- start 2021-04-03T03:08:29.333 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- start start 2021-04-03T03:08:29.334 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43e4025480 0x7f43e4150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.334 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43e40b38c0 0x7f43e41517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.334 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4065c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.334 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40642c0 con 0x7f43e4025480 2021-04-03T03:08:29.334 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e4064560 con 0x7f43e40b38c0 2021-04-03T03:08:29.335 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e4064800 con 0x7f43e4153440 2021-04-03T03:08:29.335 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43e40b38c0 0x7f43e41517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.335 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4065c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.335 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43e40b38c0 0x7f43e41517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49024/0 (socket says 172.21.2.115:49024) 2021-04-03T03:08:29.335 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4065c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.115:51278/0 (socket says 172.21.2.115:51278) 2021-04-03T03:08:29.336 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 -- 172.21.2.115:0/2391962327 learned_addr learned my addr 172.21.2.115:0/2391962327 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.336 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9b43700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43e4025480 0x7f43e4150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.336 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 -- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43e40b38c0 msgr2=0x7f43e41517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.336 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43e40b38c0 0x7f43e41517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.337 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 -- 172.21.2.115:0/2391962327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43e4025480 msgr2=0x7f43e4150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.337 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43e4025480 0x7f43e4150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.337 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 -- 172.21.2.115:0/2391962327 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4063f40 con 0x7f43e4153440 2021-04-03T03:08:29.337 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9b43700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43e4025480 0x7f43e4150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.337 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43e9342700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43e40b38c0 0x7f43e41517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.338 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43eb7be700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4065c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f43dc00c000 tx=0x7f43dc0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.338 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43d67fc700 1 -- 172.21.2.115:0/2391962327 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc00c8f0 con 0x7f43e4153440 2021-04-03T03:08:29.338 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43d67fc700 1 -- 172.21.2.115:0/2391962327 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc00ca50 con 0x7f43e4153440 2021-04-03T03:08:29.338 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43d67fc700 1 -- 172.21.2.115:0/2391962327 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc00cdb0 con 0x7f43e4153440 2021-04-03T03:08:29.338 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e40ad6b0 con 0x7f43e4153440 2021-04-03T03:08:29.339 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e4151e20 con 0x7f43e4153440 2021-04-03T03:08:29.339 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43d67fc700 1 -- 172.21.2.115:0/2391962327 <== mon.2 v2:172.21.2.115:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f43dc022070 con 0x7f43e4153440 2021-04-03T03:08:29.339 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.316+0000 7f43d67fc700 1 -- 172.21.2.115:0/2391962327 <== mon.2 v2:172.21.2.115:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f43dc0123f0 con 0x7f43e4153440 2021-04-03T03:08:29.339 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.320+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4057660 con 0x7f43e4153440 2021-04-03T03:08:29.340 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.320+0000 7f43d67fc700 1 -- 172.21.2.115:0/2391962327 <== mon.2 v2:172.21.2.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f43dc039870 con 0x7f43e4153440 2021-04-03T03:08:29.380 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.372+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "osd new", "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", "id": 3} v 0) v1 -- 0x7f43e40ac800 con 0x7f43e4153440 2021-04-03T03:08:29.384 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43d67fc700 1 -- 172.21.2.115:0/2391962327 <== mon.2 v2:172.21.2.115:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f43dc029070 con 0x7f43e4153440 2021-04-03T03:08:29.385 INFO:teuthology.orchestra.run.gibba015.stdout:3 2021-04-03T03:08:29.385 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 msgr2=0x7f43e4065c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.386 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4065c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f43dc00c000 tx=0x7f43dc0091d0).stop 2021-04-03T03:08:29.386 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 shutdown_connections 2021-04-03T03:08:29.386 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43e4153440 0x7f43e4065c60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.386 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43e4025480 0x7f43e4150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.386 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 --2- 172.21.2.115:0/2391962327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43e40b38c0 0x7f43e41517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.387 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 >> 172.21.2.115:0/2391962327 conn(0x7f43e4061610 msgr2=0x7f43e40b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.387 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 shutdown_connections 2021-04-03T03:08:29.387 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.376+0000 7f43ea344700 1 -- 172.21.2.115:0/2391962327 wait complete. 2021-04-03T03:08:29.392 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new 107b94d0-940a-4a28-b981-4eb105bea5fc 4 2021-04-03T03:08:29.452 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 Processor -- start 2021-04-03T03:08:29.452 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- start start 2021-04-03T03:08:29.452 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f2c2c0b38c0 0x7f2c2c025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.453 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2c2c080c60 0x7f2c2c0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.453 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c081090 con 0x7f2c2c153440 2021-04-03T03:08:29.453 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c025970 con 0x7f2c2c080c60 2021-04-03T03:08:29.453 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c0b3ca0 con 0x7f2c2c0b38c0 2021-04-03T03:08:29.454 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f2c2c0b38c0 0x7f2c2c025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.454 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f2c2c0b38c0 0x7f2c2c025480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49030/0 (socket says 172.21.2.115:49030) 2021-04-03T03:08:29.454 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 -- 172.21.2.115:0/4127710192 learned_addr learned my addr 172.21.2.115:0/4127710192 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.454 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c31ce0700 1 --2- 172.21.2.115:0/4127710192 >> v2:172.21.2.111:3300/0 conn(0x7f2c2c080c60 0x7f2c2c0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.454 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 -- 172.21.2.115:0/4127710192 >> v2:172.21.2.111:3300/0 conn(0x7f2c2c080c60 msgr2=0x7f2c2c0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.455 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 --2- 172.21.2.115:0/4127710192 >> v2:172.21.2.111:3300/0 conn(0x7f2c2c080c60 0x7f2c2c0af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.455 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 -- 172.21.2.115:0/4127710192 >> v1:172.21.2.111:6789/0 conn(0x7f2c2c153440 legacy=0x7f2c2c080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.455 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 -- 172.21.2.115:0/4127710192 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c025b90 con 0x7f2c2c0b38c0 2021-04-03T03:08:29.455 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2bfff700 1 --2- 172.21.2.115:0/4127710192 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c0b38c0 0x7f2c2c025480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f2c1c002f30 tx=0x7f2c1c006e90).ready entity=mon.1 client_cookie=9d874929c38f599 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.456 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2affd700 1 -- 172.21.2.115:0/4127710192 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1c009850 con 0x7f2c2c0b38c0 2021-04-03T03:08:29.456 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2affd700 1 -- 172.21.2.115:0/4127710192 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c0099b0 con 0x7f2c2c0b38c0 2021-04-03T03:08:29.456 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c2affd700 1 -- 172.21.2.115:0/4127710192 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1c009d10 con 0x7f2c2c0b38c0 2021-04-03T03:08:29.456 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- 172.21.2.115:0/4127710192 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c0b38c0 msgr2=0x7f2c2c025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.457 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 --2- 172.21.2.115:0/4127710192 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c0b38c0 0x7f2c2c025480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f2c1c002f30 tx=0x7f2c1c006e90).stop 2021-04-03T03:08:29.457 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- 172.21.2.115:0/4127710192 shutdown_connections 2021-04-03T03:08:29.457 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 --2- 172.21.2.115:0/4127710192 >> v2:172.21.2.111:3300/0 conn(0x7f2c2c080c60 0x7f2c2c0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.457 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 --2- 172.21.2.115:0/4127710192 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c0b38c0 0x7f2c2c025480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.457 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- 172.21.2.115:0/4127710192 >> 172.21.2.115:0/4127710192 conn(0x7f2c2c061610 msgr2=0x7f2c2c061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.458 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- 172.21.2.115:0/4127710192 shutdown_connections 2021-04-03T03:08:29.458 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.440+0000 7f2c30866700 1 -- 172.21.2.115:0/4127710192 wait complete. 2021-04-03T03:08:29.458 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 Processor -- start 2021-04-03T03:08:29.458 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 -- start start 2021-04-03T03:08:29.460 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c080c60 0x7f2c2c07a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.460 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c2c0b38c0 0x7f2c2c0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.460 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c2c153440 0x7f2c2c066720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.461 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c07ac60 con 0x7f2c2c153440 2021-04-03T03:08:29.461 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c064050 con 0x7f2c2c080c60 2021-04-03T03:08:29.461 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0642f0 con 0x7f2c2c0b38c0 2021-04-03T03:08:29.461 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c080c60 0x7f2c2c07a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.461 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c080c60 0x7f2c2c07a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49036/0 (socket says 172.21.2.115:49036) 2021-04-03T03:08:29.462 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 -- 172.21.2.115:0/1219248225 learned_addr learned my addr 172.21.2.115:0/1219248225 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.462 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2b7fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c2c0b38c0 0x7f2c2c0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.462 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c31ce0700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c2c153440 0x7f2c2c066720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.462 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 -- 172.21.2.115:0/1219248225 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c2c0b38c0 msgr2=0x7f2c2c0accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.463 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c2c0b38c0 0x7f2c2c0accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.463 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 -- 172.21.2.115:0/1219248225 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c2c153440 msgr2=0x7f2c2c066720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.463 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c2c153440 0x7f2c2c066720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.463 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 -- 172.21.2.115:0/1219248225 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c0ad1e0 con 0x7f2c2c080c60 2021-04-03T03:08:29.463 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c2bfff700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c080c60 0x7f2c2c07a6d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f2c1c002f30 tx=0x7f2c1c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.464 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c28ff9700 1 -- 172.21.2.115:0/1219248225 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1c011bd0 con 0x7f2c2c080c60 2021-04-03T03:08:29.464 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c28ff9700 1 -- 172.21.2.115:0/1219248225 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c011d30 con 0x7f2c2c080c60 2021-04-03T03:08:29.464 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c28ff9700 1 -- 172.21.2.115:0/1219248225 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1c011ea0 con 0x7f2c2c080c60 2021-04-03T03:08:29.464 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c065e30 con 0x7f2c2c080c60 2021-04-03T03:08:29.465 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c066e20 con 0x7f2c2c080c60 2021-04-03T03:08:29.465 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c28ff9700 1 -- 172.21.2.115:0/1219248225 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2c1c025030 con 0x7f2c2c080c60 2021-04-03T03:08:29.465 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c28ff9700 1 -- 172.21.2.115:0/1219248225 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f2c1c01d8f0 con 0x7f2c2c080c60 2021-04-03T03:08:29.465 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.444+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c065c00 con 0x7f2c2c080c60 2021-04-03T03:08:29.465 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.448+0000 7f2c28ff9700 1 -- 172.21.2.115:0/1219248225 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2c1c023070 con 0x7f2c2c080c60 2021-04-03T03:08:29.507 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.496+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", "id": 4} v 0) v1 -- 0x7f2c2c057660 con 0x7f2c2c080c60 2021-04-03T03:08:29.511 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c28ff9700 1 -- 172.21.2.115:0/1219248225 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2c1c01db00 con 0x7f2c2c080c60 2021-04-03T03:08:29.511 INFO:teuthology.orchestra.run.gibba015.stdout:4 2021-04-03T03:08:29.512 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c080c60 msgr2=0x7f2c2c07a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.512 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c080c60 0x7f2c2c07a6d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f2c1c002f30 tx=0x7f2c1c015000).stop 2021-04-03T03:08:29.512 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 shutdown_connections 2021-04-03T03:08:29.513 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c2c153440 0x7f2c2c066720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.513 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c2c080c60 0x7f2c2c07a6d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.513 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 --2- 172.21.2.115:0/1219248225 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c2c0b38c0 0x7f2c2c0accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.513 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 >> 172.21.2.115:0/1219248225 conn(0x7f2c2c061610 msgr2=0x7f2c2c07fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.513 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 shutdown_connections 2021-04-03T03:08:29.514 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.500+0000 7f2c30866700 1 -- 172.21.2.115:0/1219248225 wait complete. 2021-04-03T03:08:29.518 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new b020dc25-a06c-4de5-8896-b6af08322240 5 2021-04-03T03:08:29.578 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.564+0000 7f2dc5a4c700 1 Processor -- start 2021-04-03T03:08:29.578 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.564+0000 7f2dc5a4c700 1 -- start start 2021-04-03T03:08:29.578 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2db8046ce0 0x7f2db8041700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.579 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8040060 0x7f2db80f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.579 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db80f4ea0 con 0x7f2db803d350 2021-04-03T03:08:29.579 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db80470c0 con 0x7f2db8046ce0 2021-04-03T03:08:29.579 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db803f960 con 0x7f2db8040060 2021-04-03T03:08:29.579 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8040060 0x7f2db80f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.580 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8040060 0x7f2db80f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.115:51292/0 (socket says 172.21.2.115:51292) 2021-04-03T03:08:29.580 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 -- 172.21.2.115:0/481338519 learned_addr learned my addr 172.21.2.115:0/481338519 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.580 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc524b700 1 --2- 172.21.2.115:0/481338519 >> v2:172.21.2.111:3300/0 conn(0x7f2db8046ce0 0x7f2db8041700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.580 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 -- 172.21.2.115:0/481338519 >> v2:172.21.2.111:3300/0 conn(0x7f2db8046ce0 msgr2=0x7f2db8041700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.581 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 --2- 172.21.2.115:0/481338519 >> v2:172.21.2.111:3300/0 conn(0x7f2db8046ce0 0x7f2db8041700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.581 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 -- 172.21.2.115:0/481338519 >> v1:172.21.2.111:6789/0 conn(0x7f2db803d350 legacy=0x7f2db80f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.581 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 -- 172.21.2.115:0/481338519 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8040640 con 0x7f2db8040060 2021-04-03T03:08:29.581 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc524b700 1 --2- 172.21.2.115:0/481338519 >> v2:172.21.2.111:3300/0 conn(0x7f2db8046ce0 0x7f2db8041700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:08:29.581 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 --2- 172.21.2.115:0/481338519 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8040060 0x7f2db80f52d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f2db0002f30 tx=0x7f2db0006e90).ready entity=mon.2 client_cookie=dca3a1bf831532aa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.582 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2db7fff700 1 -- 172.21.2.115:0/481338519 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db0009970 con 0x7f2db8040060 2021-04-03T03:08:29.582 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2db7fff700 1 -- 172.21.2.115:0/481338519 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db0009ad0 con 0x7f2db8040060 2021-04-03T03:08:29.582 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2db7fff700 1 -- 172.21.2.115:0/481338519 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db0009e30 con 0x7f2db8040060 2021-04-03T03:08:29.582 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/481338519 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8040060 msgr2=0x7f2db80f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.583 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- 172.21.2.115:0/481338519 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8040060 0x7f2db80f52d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f2db0002f30 tx=0x7f2db0006e90).stop 2021-04-03T03:08:29.583 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/481338519 shutdown_connections 2021-04-03T03:08:29.583 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- 172.21.2.115:0/481338519 >> v2:172.21.2.111:3300/0 conn(0x7f2db8046ce0 0x7f2db8041700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.583 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- 172.21.2.115:0/481338519 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8040060 0x7f2db80f52d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.583 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/481338519 >> 172.21.2.115:0/481338519 conn(0x7f2db800d0c0 msgr2=0x7f2db80462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.584 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/481338519 shutdown_connections 2021-04-03T03:08:29.584 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/481338519 wait complete. 2021-04-03T03:08:29.584 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 Processor -- start 2021-04-03T03:08:29.584 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- start start 2021-04-03T03:08:29.586 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2db803d350 0x7f2db8041810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.587 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2db8040060 0x7f2db80c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.587 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8046ce0 0x7f2db80c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.587 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db803a270 con 0x7f2db8040060 2021-04-03T03:08:29.587 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db803a510 con 0x7f2db803d350 2021-04-03T03:08:29.588 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db803a7b0 con 0x7f2db8046ce0 2021-04-03T03:08:29.588 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc524b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2db803d350 0x7f2db8041810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.588 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2db8040060 0x7f2db80c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.588 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc524b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2db803d350 0x7f2db8041810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49046/0 (socket says 172.21.2.115:49046) 2021-04-03T03:08:29.588 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc524b700 1 -- 172.21.2.115:0/858553856 learned_addr learned my addr 172.21.2.115:0/858553856 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.589 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc6ec6700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8046ce0 0x7f2db80c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.589 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc6ec6700 1 -- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2db803d350 msgr2=0x7f2db8041810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.589 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc6ec6700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2db803d350 0x7f2db8041810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.590 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc6ec6700 1 -- 172.21.2.115:0/858553856 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2db8040060 msgr2=0x7f2db80c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.590 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc6ec6700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2db8040060 0x7f2db80c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.590 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc6ec6700 1 -- 172.21.2.115:0/858553856 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db80c9c40 con 0x7f2db8046ce0 2021-04-03T03:08:29.590 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc524b700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2db803d350 0x7f2db8041810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.590 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc4a4a700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2db8040060 0x7f2db80c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.591 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc6ec6700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8046ce0 0x7f2db80c4d10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f2db8046200 tx=0x7f2dc005ad70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.591 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2db5ffb700 1 -- 172.21.2.115:0/858553856 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dc00699f0 con 0x7f2db8046ce0 2021-04-03T03:08:29.591 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2db5ffb700 1 -- 172.21.2.115:0/858553856 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc006b5d0 con 0x7f2db8046ce0 2021-04-03T03:08:29.591 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db8039d80 con 0x7f2db8046ce0 2021-04-03T03:08:29.591 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2db5ffb700 1 -- 172.21.2.115:0/858553856 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dc006b740 con 0x7f2db8046ce0 2021-04-03T03:08:29.592 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.568+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db803c510 con 0x7f2db8046ce0 2021-04-03T03:08:29.592 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.572+0000 7f2db5ffb700 1 -- 172.21.2.115:0/858553856 <== mon.2 v2:172.21.2.115:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2dc007a020 con 0x7f2db8046ce0 2021-04-03T03:08:29.592 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.572+0000 7f2db5ffb700 1 -- 172.21.2.115:0/858553856 <== mon.2 v2:172.21.2.115:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f2dc005f680 con 0x7f2db8046ce0 2021-04-03T03:08:29.592 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.572+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db8004d30 con 0x7f2db8046ce0 2021-04-03T03:08:29.593 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.572+0000 7f2db5ffb700 1 -- 172.21.2.115:0/858553856 <== mon.2 v2:172.21.2.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2dc006a2d0 con 0x7f2db8046ce0 2021-04-03T03:08:29.633 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.624+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", "id": 5} v 0) v1 -- 0x7f2db80083c0 con 0x7f2db8046ce0 2021-04-03T03:08:29.637 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2db5ffb700 1 -- 172.21.2.115:0/858553856 <== mon.2 v2:172.21.2.115:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2dc006a470 con 0x7f2db8046ce0 2021-04-03T03:08:29.637 INFO:teuthology.orchestra.run.gibba015.stdout:5 2021-04-03T03:08:29.638 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8046ce0 msgr2=0x7f2db80c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.638 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8046ce0 0x7f2db80c4d10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f2db8046200 tx=0x7f2dc005ad70).stop 2021-04-03T03:08:29.639 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 shutdown_connections 2021-04-03T03:08:29.639 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2db8046ce0 0x7f2db80c4d10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.639 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2db803d350 0x7f2db8041810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.639 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 --2- 172.21.2.115:0/858553856 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2db8040060 0x7f2db80c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.640 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 >> 172.21.2.115:0/858553856 conn(0x7f2db800d0c0 msgr2=0x7f2db8005710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.640 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 shutdown_connections 2021-04-03T03:08:29.640 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.628+0000 7f2dc5a4c700 1 -- 172.21.2.115:0/858553856 wait complete. 2021-04-03T03:08:29.645 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new cc3194fc-248c-46e3-a1a6-c149abc74a9a 6 2021-04-03T03:08:29.704 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 Processor -- start 2021-04-03T03:08:29.704 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- start start 2021-04-03T03:08:29.704 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f75800b38c0 0x7f7580025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.704 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7580153440 0x7f7580080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.705 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75800b3ca0 con 0x7f7580080a40 2021-04-03T03:08:29.705 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75800b0300 con 0x7f7580153440 2021-04-03T03:08:29.705 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7580025970 con 0x7f75800b38c0 2021-04-03T03:08:29.705 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f75800b38c0 0x7f7580025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.706 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f75800b38c0 0x7f7580025480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49054/0 (socket says 172.21.2.115:49054) 2021-04-03T03:08:29.706 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 -- 172.21.2.115:0/3606969131 learned_addr learned my addr 172.21.2.115:0/3606969131 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.706 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 -- 172.21.2.115:0/3606969131 >> v2:172.21.2.111:3300/0 conn(0x7f7580153440 msgr2=0x7f7580080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.706 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 --2- 172.21.2.115:0/3606969131 >> v2:172.21.2.111:3300/0 conn(0x7f7580153440 0x7f7580080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:29.706 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 -- 172.21.2.115:0/3606969131 >> v1:172.21.2.111:6789/0 conn(0x7f7580080a40 legacy=0x7f758014feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.707 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 -- 172.21.2.115:0/3606969131 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580025b90 con 0x7f75800b38c0 2021-04-03T03:08:29.707 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757ed9d700 1 --2- 172.21.2.115:0/3606969131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75800b38c0 0x7f7580025480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f75800af0b0 tx=0x7f7570006e90).ready entity=mon.1 client_cookie=ff7f48f87d26198b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.707 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757dd9b700 1 -- 172.21.2.115:0/3606969131 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7570009930 con 0x7f75800b38c0 2021-04-03T03:08:29.707 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757dd9b700 1 -- 172.21.2.115:0/3606969131 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7570009a90 con 0x7f75800b38c0 2021-04-03T03:08:29.708 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757dd9b700 1 -- 172.21.2.115:0/3606969131 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7570009e30 con 0x7f75800b38c0 2021-04-03T03:08:29.708 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- 172.21.2.115:0/3606969131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75800b38c0 msgr2=0x7f7580025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.708 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 --2- 172.21.2.115:0/3606969131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75800b38c0 0x7f7580025480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f75800af0b0 tx=0x7f7570006e90).stop 2021-04-03T03:08:29.708 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- 172.21.2.115:0/3606969131 shutdown_connections 2021-04-03T03:08:29.708 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 --2- 172.21.2.115:0/3606969131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75800b38c0 0x7f7580025480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.709 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 --2- 172.21.2.115:0/3606969131 >> v2:172.21.2.111:3300/0 conn(0x7f7580153440 0x7f7580080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:29.709 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- 172.21.2.115:0/3606969131 >> 172.21.2.115:0/3606969131 conn(0x7f7580061610 msgr2=0x7f7580061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.709 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- 172.21.2.115:0/3606969131 shutdown_connections 2021-04-03T03:08:29.709 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.692+0000 7f757f59e700 1 -- 172.21.2.115:0/3606969131 wait complete. 2021-04-03T03:08:29.709 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 Processor -- start 2021-04-03T03:08:29.710 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 -- start start 2021-04-03T03:08:29.712 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7580080a40 0x7f7580129eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.712 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75800b38c0 0x7f7580151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.712 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7580153440 0x7f75801504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.712 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75801294d0 con 0x7f75800b38c0 2021-04-03T03:08:29.713 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580129770 con 0x7f7580080a40 2021-04-03T03:08:29.713 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7580129a10 con 0x7f7580153440 2021-04-03T03:08:29.713 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f7584dac700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7580153440 0x7f75801504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.713 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f7584dac700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7580153440 0x7f75801504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.115:51310/0 (socket says 172.21.2.115:51310) 2021-04-03T03:08:29.714 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757ed9d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7580080a40 0x7f7580129eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.714 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f7584dac700 1 -- 172.21.2.115:0/2383659708 learned_addr learned my addr 172.21.2.115:0/2383659708 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.714 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757e59c700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75800b38c0 0x7f7580151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.714 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757ed9d700 1 -- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7580153440 msgr2=0x7f75801504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.714 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757ed9d700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7580153440 0x7f75801504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.715 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757ed9d700 1 -- 172.21.2.115:0/2383659708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75800b38c0 msgr2=0x7f7580151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.715 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757ed9d700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75800b38c0 0x7f7580151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.715 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757ed9d700 1 -- 172.21.2.115:0/2383659708 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580128930 con 0x7f7580080a40 2021-04-03T03:08:29.715 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f7584dac700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7580153440 0x7f75801504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:29.716 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757e59c700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75800b38c0 0x7f7580151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:08:29.716 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757ed9d700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7580080a40 0x7f7580129eb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f75800af0b0 tx=0x7f7570015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.716 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f757001e460 con 0x7f7580080a40 2021-04-03T03:08:29.716 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757001e5c0 con 0x7f7580080a40 2021-04-03T03:08:29.716 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7580128b20 con 0x7f7580080a40 2021-04-03T03:08:29.717 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f757001e920 con 0x7f7580080a40 2021-04-03T03:08:29.717 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7580067c30 con 0x7f7580080a40 2021-04-03T03:08:29.717 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f757001a040 con 0x7f7580080a40 2021-04-03T03:08:29.717 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f7570027a70 con 0x7f7580080a40 2021-04-03T03:08:29.718 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.696+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7580057660 con 0x7f7580080a40 2021-04-03T03:08:29.718 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.700+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f7570027c80 con 0x7f7580080a40 2021-04-03T03:08:29.718 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.700+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mgrmap(e 2) v1 ==== 34034+0+0 (secure 0 0 0) 0x7f757000f070 con 0x7f7580080a40 2021-04-03T03:08:29.762 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.752+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", "id": 6} v 0) v1 -- 0x7f7580069270 con 0x7f7580080a40 2021-04-03T03:08:29.766 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f75777fe700 1 -- 172.21.2.115:0/2383659708 <== mon.1 v2:172.21.2.115:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f757001d3c0 con 0x7f7580080a40 2021-04-03T03:08:29.766 INFO:teuthology.orchestra.run.gibba015.stdout:6 2021-04-03T03:08:29.767 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7580080a40 msgr2=0x7f7580129eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.768 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7580080a40 0x7f7580129eb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f75800af0b0 tx=0x7f7570015000).stop 2021-04-03T03:08:29.768 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 shutdown_connections 2021-04-03T03:08:29.768 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7580153440 0x7f75801504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.768 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7580080a40 0x7f7580129eb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.769 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 --2- 172.21.2.115:0/2383659708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75800b38c0 0x7f7580151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.769 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 >> 172.21.2.115:0/2383659708 conn(0x7f7580061610 msgr2=0x7f7580159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.769 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 shutdown_connections 2021-04-03T03:08:29.769 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.756+0000 7f757f59e700 1 -- 172.21.2.115:0/2383659708 wait complete. 2021-04-03T03:08:29.774 DEBUG:teuthology.orchestra.run.gibba015:> sudo ceph --cluster ceph osd new f091dcf7-acf3-4d21-b0ad-c2ab84a82035 7 2021-04-03T03:08:29.837 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 Processor -- start 2021-04-03T03:08:29.837 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- start start 2021-04-03T03:08:29.837 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc0b38c0 0x7f9cbc025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.837 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9cbc153440 0x7f9cbc080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.838 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9cbc080a40 0x7f9cbc14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.838 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cbc025970 con 0x7f9cbc080a40 2021-04-03T03:08:29.838 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cbc0b3ca0 con 0x7f9cbc153440 2021-04-03T03:08:29.838 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc025b90 con 0x7f9cbc0b38c0 2021-04-03T03:08:29.839 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc0b38c0 0x7f9cbc025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.839 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc0b38c0 0x7f9cbc025480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.115:51318/0 (socket says 172.21.2.115:51318) 2021-04-03T03:08:29.839 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 -- 172.21.2.115:0/866739144 learned_addr learned my addr 172.21.2.115:0/866739144 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.839 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbb7fe700 1 --2- 172.21.2.115:0/866739144 >> v2:172.21.2.115:3300/0 conn(0x7f9cbc153440 0x7f9cbc080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.839 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 -- 172.21.2.115:0/866739144 >> v2:172.21.2.115:3300/0 conn(0x7f9cbc153440 msgr2=0x7f9cbc080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.840 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 --2- 172.21.2.115:0/866739144 >> v2:172.21.2.115:3300/0 conn(0x7f9cbc153440 0x7f9cbc080550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.840 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 -- 172.21.2.115:0/866739144 >> v2:172.21.2.111:3300/0 conn(0x7f9cbc080a40 msgr2=0x7f9cbc14feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:29.840 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 --2- 172.21.2.115:0/866739144 >> v2:172.21.2.111:3300/0 conn(0x7f9cbc080a40 0x7f9cbc14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:29.840 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 -- 172.21.2.115:0/866739144 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc025e30 con 0x7f9cbc0b38c0 2021-04-03T03:08:29.841 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 --2- 172.21.2.115:0/866739144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc0b38c0 0x7f9cbc025480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f9ca4002f30 tx=0x7f9ca4006e90).ready entity=mon.2 client_cookie=cce99c21466f6737 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.841 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbaffd700 1 -- 172.21.2.115:0/866739144 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ca4009850 con 0x7f9cbc0b38c0 2021-04-03T03:08:29.841 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbaffd700 1 -- 172.21.2.115:0/866739144 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca40099b0 con 0x7f9cbc0b38c0 2021-04-03T03:08:29.841 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbaffd700 1 -- 172.21.2.115:0/866739144 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ca4009d10 con 0x7f9cbc0b38c0 2021-04-03T03:08:29.841 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- 172.21.2.115:0/866739144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc0b38c0 msgr2=0x7f9cbc025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.842 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/866739144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc0b38c0 0x7f9cbc025480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f9ca4002f30 tx=0x7f9ca4006e90).stop 2021-04-03T03:08:29.842 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- 172.21.2.115:0/866739144 shutdown_connections 2021-04-03T03:08:29.842 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/866739144 >> v2:172.21.2.111:3300/0 conn(0x7f9cbc080a40 0x7f9cbc14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:29.842 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/866739144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc0b38c0 0x7f9cbc025480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.843 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/866739144 >> v2:172.21.2.115:3300/0 conn(0x7f9cbc153440 0x7f9cbc080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.843 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- 172.21.2.115:0/866739144 >> 172.21.2.115:0/866739144 conn(0x7f9cbc061610 msgr2=0x7f9cbc061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.843 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- 172.21.2.115:0/866739144 shutdown_connections 2021-04-03T03:08:29.843 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- 172.21.2.115:0/866739144 wait complete. 2021-04-03T03:08:29.843 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 Processor -- start 2021-04-03T03:08:29.844 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- start start 2021-04-03T03:08:29.844 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cbc080a40 0x7f9cbc067320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.844 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cbc0b38c0 0x7f9cbc0ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.844 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc153440 0x7f9cbc0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:29.845 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc067810 con 0x7f9cbc0b38c0 2021-04-03T03:08:29.845 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc067a00 con 0x7f9cbc080a40 2021-04-03T03:08:29.845 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc08bb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc065b40 con 0x7f9cbc153440 2021-04-03T03:08:29.845 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cbc080a40 0x7f9cbc067320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.846 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc1d35700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc153440 0x7f9cbc0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:29.846 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cbc080a40 0x7f9cbc067320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.115:49070/0 (socket says 172.21.2.115:49070) 2021-04-03T03:08:29.846 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cc1d35700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc153440 0x7f9cbc0ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.115:51326/0 (socket says 172.21.2.115:51326) 2021-04-03T03:08:29.846 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.824+0000 7f9cbbfff700 1 -- 172.21.2.115:0/3446956242 learned_addr learned my addr 172.21.2.115:0/3446956242 (peer_addr_for_me v2:172.21.2.115:0/0) 2021-04-03T03:08:29.846 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cbbfff700 1 -- 172.21.2.115:0/3446956242 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc153440 msgr2=0x7f9cbc0ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.847 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cbbfff700 1 --2- 172.21.2.115:0/3446956242 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc153440 0x7f9cbc0ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.847 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cbbfff700 1 -- 172.21.2.115:0/3446956242 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cbc0b38c0 msgr2=0x7f9cbc0ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.847 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cbbfff700 1 --2- 172.21.2.115:0/3446956242 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cbc0b38c0 0x7f9cbc0ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:29.847 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cbbfff700 1 -- 172.21.2.115:0/3446956242 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc0658a0 con 0x7f9cbc080a40 2021-04-03T03:08:29.848 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cbbfff700 1 --2- 172.21.2.115:0/3446956242 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cbc080a40 0x7f9cbc067320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9cbc0664d0 tx=0x7f9ca4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:29.848 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cb8ff9700 1 -- 172.21.2.115:0/3446956242 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ca4011b10 con 0x7f9cbc080a40 2021-04-03T03:08:29.848 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cb8ff9700 1 -- 172.21.2.115:0/3446956242 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4011c70 con 0x7f9cbc080a40 2021-04-03T03:08:29.848 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc0ab920 con 0x7f9cbc080a40 2021-04-03T03:08:29.848 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cb8ff9700 1 -- 172.21.2.115:0/3446956242 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ca401e560 con 0x7f9cbc080a40 2021-04-03T03:08:29.849 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc12e040 con 0x7f9cbc080a40 2021-04-03T03:08:29.849 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cb8ff9700 1 -- 172.21.2.115:0/3446956242 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 2) v1 ==== 34034+0+0 (secure 0 0 0) 0x7f9ca401a040 con 0x7f9cbc080a40 2021-04-03T03:08:29.849 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cb8ff9700 1 -- 172.21.2.115:0/3446956242 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1935+0+0 (secure 0 0 0) 0x7f9ca403ff90 con 0x7f9cbc080a40 2021-04-03T03:08:29.849 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc057660 con 0x7f9cbc080a40 2021-04-03T03:08:29.850 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.828+0000 7f9cb8ff9700 1 -- 172.21.2.115:0/3446956242 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f9ca401e6c0 con 0x7f9cbc080a40 2021-04-03T03:08:29.889 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.880+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", "id": 7} v 0) v1 -- 0x7f9cbc069270 con 0x7f9cbc080a40 2021-04-03T03:08:29.899 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.888+0000 7f9cb8ff9700 1 -- 172.21.2.115:0/3446956242 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ca40350b0 con 0x7f9cbc080a40 2021-04-03T03:08:29.900 INFO:teuthology.orchestra.run.gibba015.stdout:7 2021-04-03T03:08:29.900 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cbc080a40 msgr2=0x7f9cbc067320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:29.901 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/3446956242 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cbc080a40 0x7f9cbc067320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9cbc0664d0 tx=0x7f9ca4015000).stop 2021-04-03T03:08:29.901 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 shutdown_connections 2021-04-03T03:08:29.901 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/3446956242 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cbc153440 0x7f9cbc0ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.901 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/3446956242 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cbc080a40 0x7f9cbc067320 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:29.902 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 --2- 172.21.2.115:0/3446956242 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cbc0b38c0 0x7f9cbc0ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:29.902 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 >> 172.21.2.115:0/3446956242 conn(0x7f9cbc061610 msgr2=0x7f9cbc131850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:29.902 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 shutdown_connections 2021-04-03T03:08:29.902 INFO:teuthology.orchestra.run.gibba015.stderr:2021-04-03T03:08:29.892+0000 7f9cc08bb700 1 -- 172.21.2.115:0/3446956242 wait complete. 2021-04-03T03:08:29.907 INFO:tasks.ceph.osd.0:Restarting daemon 2021-04-03T03:08:29.908 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2021-04-03T03:08:29.911 INFO:tasks.ceph.osd.0:Started 2021-04-03T03:08:29.911 INFO:tasks.ceph.osd.1:Restarting daemon 2021-04-03T03:08:29.911 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2021-04-03T03:08:29.914 INFO:tasks.ceph.osd.1:Started 2021-04-03T03:08:29.914 INFO:tasks.ceph.osd.2:Restarting daemon 2021-04-03T03:08:29.914 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2021-04-03T03:08:29.916 INFO:tasks.ceph.osd.2:Started 2021-04-03T03:08:29.917 INFO:tasks.ceph.osd.3:Restarting daemon 2021-04-03T03:08:29.917 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2021-04-03T03:08:29.919 INFO:tasks.ceph.osd.3:Started 2021-04-03T03:08:29.919 INFO:tasks.ceph.osd.4:Restarting daemon 2021-04-03T03:08:29.919 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2021-04-03T03:08:29.921 INFO:tasks.ceph.osd.4:Started 2021-04-03T03:08:29.921 INFO:tasks.ceph.osd.5:Restarting daemon 2021-04-03T03:08:29.922 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2021-04-03T03:08:29.925 INFO:tasks.ceph.osd.5:Started 2021-04-03T03:08:29.925 INFO:tasks.ceph.osd.6:Restarting daemon 2021-04-03T03:08:29.926 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2021-04-03T03:08:29.927 INFO:tasks.ceph.osd.6:Started 2021-04-03T03:08:29.928 INFO:tasks.ceph.osd.7:Restarting daemon 2021-04-03T03:08:29.928 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2021-04-03T03:08:29.930 INFO:tasks.ceph.osd.7:Started 2021-04-03T03:08:29.930 INFO:tasks.ceph:Waiting for OSDs to come up 2021-04-03T03:08:30.483 INFO:tasks.ceph.osd.1.gibba011.stderr:2021-04-03T03:08:30.471+0000 7f2193bb4d80 -1 Falling back to public interface 2021-04-03T03:08:30.490 INFO:tasks.ceph.osd.4.gibba015.stderr:2021-04-03T03:08:30.480+0000 7fb7569a2d80 -1 Falling back to public interface 2021-04-03T03:08:30.500 INFO:tasks.ceph.osd.6.gibba015.stderr:2021-04-03T03:08:30.492+0000 7f466dd20d80 -1 Falling back to public interface 2021-04-03T03:08:30.505 INFO:tasks.ceph.osd.2.gibba011.stderr:2021-04-03T03:08:30.495+0000 7f269cabdd80 -1 Falling back to public interface 2021-04-03T03:08:30.505 INFO:tasks.ceph.osd.0.gibba011.stderr:2021-04-03T03:08:30.495+0000 7f0291e31d80 -1 Falling back to public interface 2021-04-03T03:08:30.517 INFO:tasks.ceph.osd.3.gibba011.stderr:2021-04-03T03:08:30.507+0000 7effd162fd80 -1 Falling back to public interface 2021-04-03T03:08:30.520 INFO:tasks.ceph.osd.5.gibba015.stderr:2021-04-03T03:08:30.512+0000 7f443f23ad80 -1 Falling back to public interface 2021-04-03T03:08:30.528 INFO:tasks.ceph.osd.7.gibba015.stderr:2021-04-03T03:08:30.520+0000 7f4e68d05d80 -1 Falling back to public interface 2021-04-03T03:08:30.737 DEBUG:teuthology.orchestra.run.gibba011:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-04-03T03:08:30.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 Processor -- start 2021-04-03T03:08:30.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- start start 2021-04-03T03:08:30.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd15403d350 0x7fd1540f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:30.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1540f4ea0 con 0x7fd154040060 2021-04-03T03:08:30.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1540470c0 con 0x7fd154046ce0 2021-04-03T03:08:30.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15403f960 con 0x7fd15403d350 2021-04-03T03:08:30.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15bfff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd154040060 0x7fd1540f1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49564/0 (socket says 172.21.2.111:49564) 2021-04-03T03:08:30.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15bfff700 1 -- 172.21.2.111:0/2183515816 learned_addr learned my addr 172.21.2.111:0/2183515816 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:30.841 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd162936700 1 --2- 172.21.2.111:0/2183515816 >> v2:172.21.2.115:3300/0 conn(0x7fd15403d350 0x7fd1540f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:30.841 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15b7fe700 1 -- 172.21.2.111:0/2183515816 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699328444 0 0) 0x7fd1540f4ea0 con 0x7fd154040060 2021-04-03T03:08:30.841 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15b7fe700 1 -- 172.21.2.111:0/2183515816 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd14c003240 con 0x7fd154040060 2021-04-03T03:08:30.841 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd162936700 1 -- 172.21.2.111:0/2183515816 >> v1:172.21.2.115:6789/0 conn(0x7fd154046ce0 legacy=0x7fd1540f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:30.842 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd162936700 1 -- 172.21.2.111:0/2183515816 >> v1:172.21.2.111:6789/0 conn(0x7fd154040060 legacy=0x7fd1540f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:30.842 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd162936700 1 -- 172.21.2.111:0/2183515816 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154040640 con 0x7fd15403d350 2021-04-03T03:08:30.842 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd162936700 1 --2- 172.21.2.111:0/2183515816 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd15403d350 0x7fd1540f59e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd15403d2c0 tx=0x7fd150006dd0).ready entity=mon.1 client_cookie=e278a07730ff01d6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:30.842 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15b7fe700 1 -- 172.21.2.111:0/2183515816 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1500097b0 con 0x7fd15403d350 2021-04-03T03:08:30.842 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15b7fe700 1 -- 172.21.2.111:0/2183515816 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150009910 con 0x7fd15403d350 2021-04-03T03:08:30.843 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15b7fe700 1 -- 172.21.2.111:0/2183515816 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd150009c70 con 0x7fd15403d350 2021-04-03T03:08:30.843 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- 172.21.2.111:0/2183515816 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd15403d350 msgr2=0x7fd1540f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:30.843 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2183515816 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd15403d350 0x7fd1540f59e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd15403d2c0 tx=0x7fd150006dd0).stop 2021-04-03T03:08:30.843 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- 172.21.2.111:0/2183515816 shutdown_connections 2021-04-03T03:08:30.843 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2183515816 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd15403d350 0x7fd1540f59e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:30.844 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- 172.21.2.111:0/2183515816 >> 172.21.2.111:0/2183515816 conn(0x7fd15400d0c0 msgr2=0x7fd1540462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:30.844 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- 172.21.2.111:0/2183515816 shutdown_connections 2021-04-03T03:08:30.844 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- 172.21.2.111:0/2183515816 wait complete. 2021-04-03T03:08:30.844 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 Processor -- start 2021-04-03T03:08:30.845 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- start start 2021-04-03T03:08:30.845 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd15403d350 0x7fd15403c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:30.845 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd154040060 0x7fd1540b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:30.845 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd154046ce0 0x7fd15403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:30.845 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540f3650 con 0x7fd15403d350 2021-04-03T03:08:30.846 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540f4930 con 0x7fd154040060 2021-04-03T03:08:30.846 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd1614bc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540f3840 con 0x7fd154046ce0 2021-04-03T03:08:30.846 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd160cbb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd15403d350 0x7fd15403c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:30.846 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd160cbb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd15403d350 0x7fd15403c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:48776/0 (socket says 172.21.2.111:48776) 2021-04-03T03:08:30.846 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd160cbb700 1 -- 172.21.2.111:0/2433096730 learned_addr learned my addr 172.21.2.111:0/2433096730 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:30.847 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd15bfff700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd154040060 0x7fd1540b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:30.847 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd162936700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd154046ce0 0x7fd15403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:30.847 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd160cbb700 1 -- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd154046ce0 msgr2=0x7fd15403b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:30.847 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.823+0000 7fd160cbb700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd154046ce0 0x7fd15403b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:30.848 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd160cbb700 1 -- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd154040060 msgr2=0x7fd1540b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:30.848 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd160cbb700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd154040060 0x7fd1540b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:30.848 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd160cbb700 1 -- 172.21.2.111:0/2433096730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540f4b60 con 0x7fd15403d350 2021-04-03T03:08:30.848 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd160cbb700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd15403d350 0x7fd15403c200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fd154043d60 tx=0x7fd148007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:30.848 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1597fa700 1 -- 172.21.2.111:0/2433096730 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd14800e4a0 con 0x7fd15403d350 2021-04-03T03:08:30.849 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1597fa700 1 -- 172.21.2.111:0/2433096730 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14800e600 con 0x7fd15403d350 2021-04-03T03:08:30.849 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15403c6f0 con 0x7fd15403d350 2021-04-03T03:08:30.849 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1597fa700 1 -- 172.21.2.111:0/2433096730 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd14800e960 con 0x7fd15403d350 2021-04-03T03:08:30.849 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1540b8540 con 0x7fd15403d350 2021-04-03T03:08:30.849 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1597fa700 1 -- 172.21.2.111:0/2433096730 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 3) v1 ==== 68196+0+0 (secure 0 0 0) 0x7fd148010070 con 0x7fd15403d350 2021-04-03T03:08:30.850 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1597fa700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd13c056ee0 0x7fd13c0593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:30.850 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1597fa700 1 -- 172.21.2.111:0/2433096730 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7fd14806e060 con 0x7fd15403d350 2021-04-03T03:08:30.850 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd15bfff700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd13c056ee0 0x7fd13c0593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:30.850 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1540b7e90 con 0x7fd15403d350 2021-04-03T03:08:30.851 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.827+0000 7fd15bfff700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd13c056ee0 0x7fd13c0593a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd15c066000 tx=0x7fd15c05ad70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:30.851 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.831+0000 7fd1597fa700 1 -- 172.21.2.111:0/2433096730 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd148017080 con 0x7fd15403d350 2021-04-03T03:08:30.935 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.923+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd154004d10 con 0x7fd15403d350 2021-04-03T03:08:30.936 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.923+0000 7fd1597fa700 1 -- 172.21.2.111:0/2433096730 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+8275 (secure 0 0 0) 0x7fd148047c40 con 0x7fd15403d350 2021-04-03T03:08:30.937 INFO:teuthology.misc.health.gibba011.stdout: 2021-04-03T03:08:30.937 INFO:teuthology.misc.health.gibba011.stdout:{"epoch":10,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:29.885214+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:30.938 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd13c056ee0 msgr2=0x7fd13c0593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:30.938 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd13c056ee0 0x7fd13c0593a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd15c066000 tx=0x7fd15c05ad70).stop 2021-04-03T03:08:30.938 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd15403d350 msgr2=0x7fd15403c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:30.939 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd15403d350 0x7fd15403c200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fd154043d60 tx=0x7fd148007c30).stop 2021-04-03T03:08:30.939 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 shutdown_connections 2021-04-03T03:08:30.939 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd154046ce0 0x7fd15403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:30.939 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd15403d350 0x7fd15403c200 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:30.939 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd13c056ee0 0x7fd13c0593a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:30.940 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 --2- 172.21.2.111:0/2433096730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd154040060 0x7fd1540b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:30.940 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 >> 172.21.2.111:0/2433096730 conn(0x7fd15400d0c0 msgr2=0x7fd1540414f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:30.940 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 shutdown_connections 2021-04-03T03:08:30.940 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:30.927+0000 7fd1614bc700 1 -- 172.21.2.111:0/2433096730 wait complete. 2021-04-03T03:08:30.944 DEBUG:teuthology.misc:0 of 8 OSDs are up 2021-04-03T03:08:31.466 INFO:tasks.ceph.osd.0.gibba011.stderr:2021-04-03T03:08:31.455+0000 7f0291e31d80 -1 osd.0 0 log_to_monitors {default=true} 2021-04-03T03:08:31.467 INFO:tasks.ceph.osd.3.gibba011.stderr:2021-04-03T03:08:31.455+0000 7effd162fd80 -1 osd.3 0 log_to_monitors {default=true} 2021-04-03T03:08:31.476 INFO:tasks.ceph.osd.7.gibba015.stderr:2021-04-03T03:08:31.468+0000 7f4e68d05d80 -1 osd.7 0 log_to_monitors {default=true} 2021-04-03T03:08:31.653 INFO:tasks.ceph.osd.1.gibba011.stderr:2021-04-03T03:08:31.643+0000 7f2193bb4d80 -1 osd.1 0 log_to_monitors {default=true} 2021-04-03T03:08:31.663 INFO:tasks.ceph.osd.6.gibba015.stderr:2021-04-03T03:08:31.652+0000 7f466dd20d80 -1 osd.6 0 log_to_monitors {default=true} 2021-04-03T03:08:31.663 INFO:tasks.ceph.osd.4.gibba015.stderr:2021-04-03T03:08:31.652+0000 7fb7569a2d80 -1 osd.4 0 log_to_monitors {default=true} 2021-04-03T03:08:31.704 INFO:tasks.ceph.osd.2.gibba011.stderr:2021-04-03T03:08:31.695+0000 7f269cabdd80 -1 osd.2 0 log_to_monitors {default=true} 2021-04-03T03:08:31.718 INFO:tasks.ceph.osd.5.gibba015.stderr:2021-04-03T03:08:31.708+0000 7f443f23ad80 -1 osd.5 0 log_to_monitors {default=true} 2021-04-03T03:08:33.092 INFO:tasks.ceph.osd.0.gibba011.stderr:2021-04-03T03:08:33.079+0000 7f0279303700 -1 osd.0 0 waiting for initial osdmap 2021-04-03T03:08:33.092 INFO:tasks.ceph.osd.3.gibba011.stderr:2021-04-03T03:08:33.079+0000 7effb8b01700 -1 osd.3 0 waiting for initial osdmap 2021-04-03T03:08:33.093 INFO:tasks.ceph.osd.2.gibba011.stderr:2021-04-03T03:08:33.079+0000 7f2683f8f700 -1 osd.2 0 waiting for initial osdmap 2021-04-03T03:08:33.093 INFO:tasks.ceph.osd.5.gibba015.stderr:2021-04-03T03:08:33.084+0000 7f442670c700 -1 osd.5 0 waiting for initial osdmap 2021-04-03T03:08:33.094 INFO:tasks.ceph.osd.1.gibba011.stderr:2021-04-03T03:08:33.083+0000 7f217b086700 -1 osd.1 0 waiting for initial osdmap 2021-04-03T03:08:33.094 INFO:tasks.ceph.osd.7.gibba015.stderr:2021-04-03T03:08:33.084+0000 7f4e501d7700 -1 osd.7 0 waiting for initial osdmap 2021-04-03T03:08:33.095 INFO:tasks.ceph.osd.6.gibba015.stderr:2021-04-03T03:08:33.084+0000 7f46551f2700 -1 osd.6 0 waiting for initial osdmap 2021-04-03T03:08:33.095 INFO:tasks.ceph.osd.4.gibba015.stderr:2021-04-03T03:08:33.084+0000 7fb73de74700 -1 osd.4 0 waiting for initial osdmap 2021-04-03T03:08:33.097 INFO:tasks.ceph.osd.1.gibba011.stderr:2021-04-03T03:08:33.087+0000 7f21883f6700 -1 osd.1 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:33.097 INFO:tasks.ceph.osd.2.gibba011.stderr:2021-04-03T03:08:33.087+0000 7f26912ff700 -1 osd.2 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:33.098 INFO:tasks.ceph.osd.3.gibba011.stderr:2021-04-03T03:08:33.087+0000 7effc5e71700 -1 osd.3 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:33.098 INFO:tasks.ceph.osd.0.gibba011.stderr:2021-04-03T03:08:33.087+0000 7f0286673700 -1 osd.0 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:33.099 INFO:tasks.ceph.osd.7.gibba015.stderr:2021-04-03T03:08:33.088+0000 7f4e5d547700 -1 osd.7 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:33.099 INFO:tasks.ceph.osd.4.gibba015.stderr:2021-04-03T03:08:33.088+0000 7fb74b1e4700 -1 osd.4 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:33.100 INFO:tasks.ceph.osd.5.gibba015.stderr:2021-04-03T03:08:33.088+0000 7f4433a7c700 -1 osd.5 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:33.100 INFO:tasks.ceph.osd.6.gibba015.stderr:2021-04-03T03:08:33.088+0000 7f4662562700 -1 osd.6 12 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-04-03T03:08:37.752 DEBUG:teuthology.orchestra.run.gibba011:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-04-03T03:08:37.821 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.803+0000 7fa9b359e700 1 Processor -- start 2021-04-03T03:08:37.822 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.803+0000 7fa9b359e700 1 -- start start 2021-04-03T03:08:37.822 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa9ac046ce0 0x7fa9ac041700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.822 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac03d350 0x7fa9ac0f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.822 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ac0f4ea0 con 0x7fa9ac040060 2021-04-03T03:08:37.823 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac0470c0 con 0x7fa9ac046ce0 2021-04-03T03:08:37.823 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac03f960 con 0x7fa9ac03d350 2021-04-03T03:08:37.823 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa9ac040060 0x7fa9ac0f52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49800/0 (socket says 172.21.2.111:49800) 2021-04-03T03:08:37.823 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 -- 172.21.2.111:0/4075129694 learned_addr learned my addr 172.21.2.111:0/4075129694 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:37.824 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860461810 0 0) 0x7fa9ac0f4ea0 con 0x7fa9ac040060 2021-04-03T03:08:37.824 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a0003240 con 0x7fa9ac040060 2021-04-03T03:08:37.824 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b8c37700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac03d350 0x7fa9ac0f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.824 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b2d9d700 1 --2- 172.21.2.111:0/4075129694 >> v2:172.21.2.115:3300/0 conn(0x7fa9ac046ce0 0x7fa9ac041700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.825 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 74116120 0 0) 0x7fa9a0003240 con 0x7fa9ac040060 2021-04-03T03:08:37.825 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac03d350 msgr2=0x7fa9ac0f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.825 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac03d350 0x7fa9ac0f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.825 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 >> v2:172.21.2.115:3300/0 conn(0x7fa9ac046ce0 msgr2=0x7fa9ac041700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.826 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 --2- 172.21.2.111:0/4075129694 >> v2:172.21.2.115:3300/0 conn(0x7fa9ac046ce0 0x7fa9ac041700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.826 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac040640 con 0x7fa9ac040060 2021-04-03T03:08:37.826 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa9a40016d0 con 0x7fa9ac040060 2021-04-03T03:08:37.826 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 >> v1:172.21.2.111:6789/0 conn(0x7fa9ac040060 legacy=0x7fa9ac0f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.827 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9a00037b0 0x7fa9a0003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.827 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9a00044e0 0x7fa9a0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.827 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac03d350 0x7fa9a0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.827 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0470c0 con 0x7fa9a00044e0 2021-04-03T03:08:37.827 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac03f960 con 0x7fa9a00037b0 2021-04-03T03:08:37.828 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a0005340 con 0x7fa9ac03d350 2021-04-03T03:08:37.828 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9a00044e0 0x7fa9a0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.828 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b2d9d700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9a00037b0 0x7fa9a0003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.828 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 -- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac03d350 msgr2=0x7fa9a0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.828 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac03d350 0x7fa9a0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:37.829 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 -- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9a00037b0 msgr2=0x7fa9a0003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.829 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9a00037b0 0x7fa9a0003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.829 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 -- 172.21.2.111:0/4075129694 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac040640 con 0x7fa9a00044e0 2021-04-03T03:08:37.829 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b259c700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9a00044e0 0x7fa9a0004960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa9a40013e0 tx=0x7fa9a40077d0).ready entity=mon.0 client_cookie=8ebe00097cf33d35 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:37.830 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9a400ce10 con 0x7fa9a00044e0 2021-04-03T03:08:37.830 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a40072e0 con 0x7fa9a00044e0 2021-04-03T03:08:37.830 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b1d9b700 1 -- 172.21.2.111:0/4075129694 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9a40057e0 con 0x7fa9a00044e0 2021-04-03T03:08:37.830 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- 172.21.2.111:0/4075129694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9a00044e0 msgr2=0x7fa9a0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.830 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 --2- 172.21.2.111:0/4075129694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9a00044e0 0x7fa9a0004960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa9a40013e0 tx=0x7fa9a40077d0).stop 2021-04-03T03:08:37.831 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b8c37700 1 -- 172.21.2.111:0/4075129694 reap_dead start 2021-04-03T03:08:37.831 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- 172.21.2.111:0/4075129694 shutdown_connections 2021-04-03T03:08:37.831 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- 172.21.2.111:0/4075129694 >> 172.21.2.111:0/4075129694 conn(0x7fa9ac00d0c0 msgr2=0x7fa9ac0462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:37.831 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- 172.21.2.111:0/4075129694 shutdown_connections 2021-04-03T03:08:37.832 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.807+0000 7fa9b359e700 1 -- 172.21.2.111:0/4075129694 wait complete. 2021-04-03T03:08:37.832 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 Processor -- start 2021-04-03T03:08:37.832 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 -- start start 2021-04-03T03:08:37.832 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac0f0a30 0x7fa9ac0d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.833 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9ac03c8d0 0x7fa9ac0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.833 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9ac0c1740 0x7fa9ac0c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.833 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0d1db0 con 0x7fa9ac0c1740 2021-04-03T03:08:37.833 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0d1680 con 0x7fa9ac03c8d0 2021-04-03T03:08:37.833 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0f0e10 con 0x7fa9ac0f0a30 2021-04-03T03:08:37.834 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9ac0c1740 0x7fa9ac0c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.834 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9ac0c1740 0x7fa9ac0c1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49020/0 (socket says 172.21.2.111:49020) 2021-04-03T03:08:37.834 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 -- 172.21.2.111:0/332003556 learned_addr learned my addr 172.21.2.111:0/332003556 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:37.836 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b2d9d700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac0f0a30 0x7fa9ac0d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.836 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b259c700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9ac03c8d0 0x7fa9ac0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.837 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 -- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac0f0a30 msgr2=0x7fa9ac0d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.837 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac0f0a30 0x7fa9ac0d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.837 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 -- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9ac03c8d0 msgr2=0x7fa9ac0d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.837 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9ac03c8d0 0x7fa9ac0d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.838 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 -- 172.21.2.111:0/332003556 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0be070 con 0x7fa9ac0c1740 2021-04-03T03:08:37.838 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b259c700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9ac03c8d0 0x7fa9ac0d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:37.838 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b8c37700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9ac0c1740 0x7fa9ac0c1bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa9b405aec0 tx=0x7fa9b405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:37.838 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9ab7fe700 1 -- 172.21.2.111:0/332003556 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b405df60 con 0x7fa9ac0c1740 2021-04-03T03:08:37.838 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9ab7fe700 1 -- 172.21.2.111:0/332003556 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b40699f0 con 0x7fa9ac0c1740 2021-04-03T03:08:37.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac0d1b10 con 0x7fa9ac0c1740 2021-04-03T03:08:37.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9ab7fe700 1 -- 172.21.2.111:0/332003556 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b405f4b0 con 0x7fa9ac0c1740 2021-04-03T03:08:37.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac03cd70 con 0x7fa9ac0c1740 2021-04-03T03:08:37.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac0d8fb0 con 0x7fa9ac0c1740 2021-04-03T03:08:37.839 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9ab7fe700 1 -- 172.21.2.111:0/332003556 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa9b405f610 con 0x7fa9ac0c1740 2021-04-03T03:08:37.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9ab7fe700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa998056f20 0x7fa9980593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9ab7fe700 1 -- 172.21.2.111:0/332003556 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4896+0+0 (secure 0 0 0) 0x7fa9b40cd8e0 con 0x7fa9ac0c1740 2021-04-03T03:08:37.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b2d9d700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa998056f20 0x7fa9980593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.840 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.811+0000 7fa9b2d9d700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa998056f20 0x7fa9980593e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa99c008000 tx=0x7fa99c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:37.841 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.815+0000 7fa9ab7fe700 1 -- 172.21.2.111:0/332003556 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa9b40a32c0 con 0x7fa9ac0c1740 2021-04-03T03:08:37.917 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9ac004d10 con 0x7fa9ac0c1740 2021-04-03T03:08:37.918 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9ab7fe700 1 -- 172.21.2.111:0/332003556 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11805 (secure 0 0 0) 0x7fa9b407a070 con 0x7fa9ac0c1740 2021-04-03T03:08:37.919 INFO:teuthology.misc.health.gibba011.stdout: 2021-04-03T03:08:37.919 INFO:teuthology.misc.health.gibba011.stdout:{"epoch":15,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:36.117963+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:37.920 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa998056f20 msgr2=0x7fa9980593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.920 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa998056f20 0x7fa9980593e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa99c008000 tx=0x7fa99c008040).stop 2021-04-03T03:08:37.920 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9ac0c1740 msgr2=0x7fa9ac0c1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.921 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9ac0c1740 0x7fa9ac0c1bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa9b405aec0 tx=0x7fa9b405ad70).stop 2021-04-03T03:08:37.921 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 shutdown_connections 2021-04-03T03:08:37.921 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9ac0c1740 0x7fa9ac0c1bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.921 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9ac0f0a30 0x7fa9ac0d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.922 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa998056f20 0x7fa9980593e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.922 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 --2- 172.21.2.111:0/332003556 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9ac03c8d0 0x7fa9ac0d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.922 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 >> 172.21.2.111:0/332003556 conn(0x7fa9ac00d0c0 msgr2=0x7fa9ac046290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:37.922 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 shutdown_connections 2021-04-03T03:08:37.922 INFO:teuthology.misc.health.gibba011.stderr:2021-04-03T03:08:37.907+0000 7fa9b359e700 1 -- 172.21.2.111:0/332003556 wait complete. 2021-04-03T03:08:37.926 DEBUG:teuthology.misc:8 of 8 OSDs are up 2021-04-03T03:08:37.926 INFO:tasks.ceph:Creating RBD pool 2021-04-03T03:08:37.926 DEBUG:teuthology.orchestra.run.gibba011:> sudo ceph --cluster ceph osd pool create rbd 8 2021-04-03T03:08:37.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 Processor -- start 2021-04-03T03:08:37.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 -- start start 2021-04-03T03:08:37.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa9080b38c0 0x7fa908025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa908081090 con 0x7fa908080c60 2021-04-03T03:08:37.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9080b3ca0 con 0x7fa908153440 2021-04-03T03:08:37.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa908025970 con 0x7fa9080b38c0 2021-04-03T03:08:37.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90cf82700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa908080c60 0x7fa9080af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49820/0 (socket says 172.21.2.111:49820) 2021-04-03T03:08:37.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90cf82700 1 -- 172.21.2.111:0/3762648484 learned_addr learned my addr 172.21.2.111:0/3762648484 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:37.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731890571 0 0) 0x7fa900001450 con 0x7fa908080c60 2021-04-03T03:08:37.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f4003240 con 0x7fa908080c60 2021-04-03T03:08:37.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1153933628 0 0) 0x7fa8f4003240 con 0x7fa908080c60 2021-04-03T03:08:37.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 >> v2:172.21.2.115:3300/0 conn(0x7fa9080b38c0 msgr2=0x7fa908025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa906d9d700 1 --2- 172.21.2.111:0/3762648484 >> v2:172.21.2.115:3300/0 conn(0x7fa9080b38c0 0x7fa908025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 --2- 172.21.2.111:0/3762648484 >> v2:172.21.2.115:3300/0 conn(0x7fa9080b38c0 0x7fa908025480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 >> v1:172.21.2.115:6789/0 conn(0x7fa908153440 legacy=0x7fa908080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908025b90 con 0x7fa908080c60 2021-04-03T03:08:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa900001be0 con 0x7fa908080c60 2021-04-03T03:08:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 >> v1:172.21.2.111:6789/0 conn(0x7fa908080c60 legacy=0x7fa9080af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8f40037b0 0x7fa8f4003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f40044e0 0x7fa8f4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f4004e50 0x7fa8f400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908025970 con 0x7fa8f40044e0 2021-04-03T03:08:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f4005300 con 0x7fa8f4004e50 2021-04-03T03:08:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f400bc20 con 0x7fa8f40037b0 2021-04-03T03:08:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa906d9d700 1 --2- 172.21.2.111:0/3762648484 >> v2:172.21.2.115:3300/0 conn(0x7fa9080b38c0 0x7fa908025480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90659c700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f40044e0 0x7fa8f4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90cf82700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f4004e50 0x7fa8f400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90659c700 1 -- 172.21.2.111:0/3762648484 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8f40037b0 msgr2=0x7fa8f4003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90659c700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8f40037b0 0x7fa8f4003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90659c700 1 -- 172.21.2.111:0/3762648484 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f4004e50 msgr2=0x7fa8f400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90659c700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f4004e50 0x7fa8f400b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90659c700 1 -- 172.21.2.111:0/3762648484 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908025b90 con 0x7fa8f40044e0 2021-04-03T03:08:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90cf82700 1 -- 172.21.2.111:0/3762648484 reap_dead start 2021-04-03T03:08:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90659c700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f40044e0 0x7fa8f4004960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa8f8001f60 tx=0x7fa8f8007b70).ready entity=mon.0 client_cookie=6644baeabb87426 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8f800c870 con 0x7fa8f40044e0 2021-04-03T03:08:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f800c9d0 con 0x7fa8f40044e0 2021-04-03T03:08:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa905d9b700 1 -- 172.21.2.111:0/3762648484 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8f800cd30 con 0x7fa8f40044e0 2021-04-03T03:08:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 -- 172.21.2.111:0/3762648484 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f40044e0 msgr2=0x7fa8f4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.971+0000 7fa90759e700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f40044e0 0x7fa8f4004960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa8f8001f60 tx=0x7fa8f8007b70).stop 2021-04-03T03:08:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- 172.21.2.111:0/3762648484 shutdown_connections 2021-04-03T03:08:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 --2- 172.21.2.111:0/3762648484 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f40044e0 0x7fa8f4004960 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- 172.21.2.111:0/3762648484 >> 172.21.2.111:0/3762648484 conn(0x7fa908061610 msgr2=0x7fa908061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- 172.21.2.111:0/3762648484 shutdown_connections 2021-04-03T03:08:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- 172.21.2.111:0/3762648484 wait complete. 2021-04-03T03:08:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 Processor -- start 2021-04-03T03:08:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- start start 2021-04-03T03:08:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8f40044e0 0x7fa9080acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9080ad150 0x7fa90809ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa90809f2d0 0x7fa90809f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908150cf0 con 0x7fa90809f2d0 2021-04-03T03:08:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908064d40 con 0x7fa9080ad150 2021-04-03T03:08:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa908065060 con 0x7fa8f40044e0 2021-04-03T03:08:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa90809f2d0 0x7fa90809f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa90809f2d0 0x7fa90809f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49038/0 (socket says 172.21.2.111:49038) 2021-04-03T03:08:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 -- 172.21.2.111:0/3705254319 learned_addr learned my addr 172.21.2.111:0/3705254319 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 -- 172.21.2.111:0/3705254319 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8f40044e0 msgr2=0x7fa9080acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8f40044e0 0x7fa9080acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:37.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 -- 172.21.2.111:0/3705254319 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9080ad150 msgr2=0x7fa90809ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:37.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9080ad150 0x7fa90809ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:37.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 -- 172.21.2.111:0/3705254319 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908064070 con 0x7fa90809f2d0 2021-04-03T03:08:37.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90cf82700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa90809f2d0 0x7fa90809f760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa908062480 tx=0x7fa900009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:38.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa8ff7fe700 1 -- 172.21.2.111:0/3705254319 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa900004a90 con 0x7fa90809f2d0 2021-04-03T03:08:38.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa8ff7fe700 1 -- 172.21.2.111:0/3705254319 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900004bf0 con 0x7fa90809f2d0 2021-04-03T03:08:38.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa8ff7fe700 1 -- 172.21.2.111:0/3705254319 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa900007170 con 0x7fa90809f2d0 2021-04-03T03:08:38.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90812dd40 con 0x7fa90809f2d0 2021-04-03T03:08:38.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908066df0 con 0x7fa90809f2d0 2021-04-03T03:08:38.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908065220 con 0x7fa90809f2d0 2021-04-03T03:08:38.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa8ff7fe700 1 -- 172.21.2.111:0/3705254319 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa90001e070 con 0x7fa90809f2d0 2021-04-03T03:08:38.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa8ff7fe700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8ec056fb0 0x7fa8ec059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:38.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa8ff7fe700 1 -- 172.21.2.111:0/3705254319 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4896+0+0 (secure 0 0 0) 0x7fa90006f120 con 0x7fa90809f2d0 2021-04-03T03:08:38.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa906d9d700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8ec056fb0 0x7fa8ec059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:38.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.975+0000 7fa906d9d700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8ec056fb0 0x7fa8ec059470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa8f0007000 tx=0x7fa8f0006bb0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:38.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:37.979+0000 7fa8ff7fe700 1 -- 172.21.2.111:0/3705254319 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa900044b00 con 0x7fa90809f2d0 2021-04-03T03:08:38.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.067+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa908081090 con 0x7fa90809f2d0 2021-04-03T03:08:38.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.147+0000 7fa8ff7fe700 1 -- 172.21.2.111:0/3705254319 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa908081090 con 0x7fa90809f2d0 2021-04-03T03:08:38.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.151+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8ec056fb0 msgr2=0x7fa8ec059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:38.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.151+0000 7fa90759e700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8ec056fb0 0x7fa8ec059470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa8f0007000 tx=0x7fa8f0006bb0).stop 2021-04-03T03:08:38.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.151+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa90809f2d0 msgr2=0x7fa90809f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:38.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.151+0000 7fa90759e700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa90809f2d0 0x7fa90809f760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa908062480 tx=0x7fa900009040).stop 2021-04-03T03:08:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 shutdown_connections 2021-04-03T03:08:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa90809f2d0 0x7fa90809f760 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8f40044e0 0x7fa9080acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8ec056fb0 0x7fa8ec059470 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 --2- 172.21.2.111:0/3705254319 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9080ad150 0x7fa90809ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 >> 172.21.2.111:0/3705254319 conn(0x7fa908061610 msgr2=0x7fa9080b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 shutdown_connections 2021-04-03T03:08:38.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.155+0000 7fa90759e700 1 -- 172.21.2.111:0/3705254319 wait complete. 2021-04-03T03:08:38.168 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'rbd' created 2021-04-03T03:08:38.192 DEBUG:teuthology.orchestra.run.gibba011:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2021-04-03T03:08:38.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 Processor -- start 2021-04-03T03:08:38.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- start start 2021-04-03T03:08:38.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0338153440 0x7f0338080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:38.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0338025480 0x7f03380af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:38.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380b3ca0 con 0x7f03380b38c0 2021-04-03T03:08:38.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0338025b90 con 0x7f0338025480 2021-04-03T03:08:38.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0338025e30 con 0x7f0338153440 2021-04-03T03:08:38.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0338025480 0x7f03380af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:38.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0338025480 0x7f03380af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49044/0 (socket says 172.21.2.111:49044) 2021-04-03T03:08:38.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 -- 172.21.2.111:0/4082282408 learned_addr learned my addr 172.21.2.111:0/4082282408 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:38.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 -- 172.21.2.111:0/4082282408 >> v2:172.21.2.115:3300/0 conn(0x7f0338153440 msgr2=0x7f0338080c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:38.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 --2- 172.21.2.111:0/4082282408 >> v2:172.21.2.115:3300/0 conn(0x7f0338153440 0x7f0338080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:38.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 -- 172.21.2.111:0/4082282408 >> v1:172.21.2.111:6789/0 conn(0x7f03380b38c0 legacy=0x7f0338080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:38.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 -- 172.21.2.111:0/4082282408 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380260d0 con 0x7f0338025480 2021-04-03T03:08:38.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033ed5d700 1 --2- 172.21.2.111:0/4082282408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338025480 0x7f03380af2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f03300032b0 tx=0x7f0330009270).ready entity=mon.0 client_cookie=2d81688c5fe8d017 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:38.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f032bfff700 1 -- 172.21.2.111:0/4082282408 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033000bc00 con 0x7f0338025480 2021-04-03T03:08:38.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f032bfff700 1 -- 172.21.2.111:0/4082282408 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033000bd60 con 0x7f0338025480 2021-04-03T03:08:38.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f032bfff700 1 -- 172.21.2.111:0/4082282408 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0330011400 con 0x7f0338025480 2021-04-03T03:08:38.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- 172.21.2.111:0/4082282408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338025480 msgr2=0x7f03380af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:38.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 --2- 172.21.2.111:0/4082282408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338025480 0x7f03380af2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f03300032b0 tx=0x7f0330009270).stop 2021-04-03T03:08:38.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- 172.21.2.111:0/4082282408 shutdown_connections 2021-04-03T03:08:38.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 --2- 172.21.2.111:0/4082282408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338025480 0x7f03380af2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:38.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 --2- 172.21.2.111:0/4082282408 >> v2:172.21.2.115:3300/0 conn(0x7f0338153440 0x7f0338080c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:38.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- 172.21.2.111:0/4082282408 >> 172.21.2.111:0/4082282408 conn(0x7f0338061610 msgr2=0x7f0338061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:38.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- 172.21.2.111:0/4082282408 shutdown_connections 2021-04-03T03:08:38.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.279+0000 7f033d8e3700 1 -- 172.21.2.111:0/4082282408 wait complete. 2021-04-03T03:08:38.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 Processor -- start 2021-04-03T03:08:38.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 -- start start 2021-04-03T03:08:38.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0338025480 0x7f03381566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:38.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f03380b38c0 0x7f0338080320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:38.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338153440 0x7f0338155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:38.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338067f30 con 0x7f0338153440 2021-04-03T03:08:38.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338066ac0 con 0x7f0338025480 2021-04-03T03:08:38.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0338154420 con 0x7f03380b38c0 2021-04-03T03:08:38.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338153440 0x7f0338155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:38.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338153440 0x7f0338155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49052/0 (socket says 172.21.2.111:49052) 2021-04-03T03:08:38.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 -- 172.21.2.111:0/3456335375 learned_addr learned my addr 172.21.2.111:0/3456335375 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:38.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 -- 172.21.2.111:0/3456335375 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f03380b38c0 msgr2=0x7f0338080320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:38.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f03380b38c0 0x7f0338080320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:38.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 -- 172.21.2.111:0/3456335375 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0338025480 msgr2=0x7f03381566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:38.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0338025480 0x7f03381566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:38.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 -- 172.21.2.111:0/3456335375 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03381546c0 con 0x7f0338153440 2021-04-03T03:08:38.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033ed5d700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338153440 0x7f0338155c10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f03300032b0 tx=0x7f0330016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:38.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f0329ffb700 1 -- 172.21.2.111:0/3456335375 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0330020620 con 0x7f0338153440 2021-04-03T03:08:38.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f0329ffb700 1 -- 172.21.2.111:0/3456335375 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0330020780 con 0x7f0338153440 2021-04-03T03:08:38.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f0329ffb700 1 -- 172.21.2.111:0/3456335375 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0330020ae0 con 0x7f0338153440 2021-04-03T03:08:38.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0338080810 con 0x7f0338153440 2021-04-03T03:08:38.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0338156310 con 0x7f0338153440 2021-04-03T03:08:38.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0338067d40 con 0x7f0338153440 2021-04-03T03:08:38.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f0329ffb700 1 -- 172.21.2.111:0/3456335375 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f033000f070 con 0x7f0338153440 2021-04-03T03:08:38.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f0329ffb700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0318057000 0x7f03180594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:38.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f0329ffb700 1 -- 172.21.2.111:0/3456335375 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5238+0+0 (secure 0 0 0) 0x7f0330071470 con 0x7f0338153440 2021-04-03T03:08:38.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d0e2700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0318057000 0x7f03180594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:38.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.283+0000 7f033d0e2700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0318057000 0x7f03180594c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f03380af0b0 tx=0x7f032c0077b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:38.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.287+0000 7f0329ffb700 1 -- 172.21.2.111:0/3456335375 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0330046be0 con 0x7f0338153440 2021-04-03T03:08:38.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:38.379+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0338057660 con 0x7f0338153440 2021-04-03T03:08:39.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f0329ffb700 1 -- 172.21.2.111:0/3456335375 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0330015020 con 0x7f0338153440 2021-04-03T03:08:39.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0318057000 msgr2=0x7f03180594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0318057000 0x7f03180594c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f03380af0b0 tx=0x7f032c0077b0).stop 2021-04-03T03:08:39.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338153440 msgr2=0x7f0338155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338153440 0x7f0338155c10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f03300032b0 tx=0x7f0330016000).stop 2021-04-03T03:08:39.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 shutdown_connections 2021-04-03T03:08:39.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0338153440 0x7f0338155c10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0318057000 0x7f03180594c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0338025480 0x7f03381566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 --2- 172.21.2.111:0/3456335375 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f03380b38c0 0x7f0338080320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 >> 172.21.2.111:0/3456335375 conn(0x7f0338061610 msgr2=0x7f03380b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 shutdown_connections 2021-04-03T03:08:39.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.155+0000 7f033d8e3700 1 -- 172.21.2.111:0/3456335375 wait complete. 2021-04-03T03:08:39.170 INFO:teuthology.orchestra.run.gibba011.stderr:enabled application 'rbd' on pool 'rbd' 2021-04-03T03:08:39.175 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2021-04-03T03:08:39.175 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:08:39.176 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:08:39.178 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:08:39.179 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:08:39.179 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:08:39.181 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:08:39.181 INFO:tasks.ceph:Setting up CephFS filesystem... 2021-04-03T03:08:39.182 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:39.212 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:08:39.217 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:08:39.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.227+0000 7f1fe5736700 1 Processor -- start 2021-04-03T03:08:39.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.227+0000 7f1fe5736700 1 -- start start 2021-04-03T03:08:39.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1fd80473f0 0x7f1fd8007590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80062e0 0x7f1fd80d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd8006e60 con 0x7f1fd8005ae0 2021-04-03T03:08:39.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd803e080 con 0x7f1fd80473f0 2021-04-03T03:08:39.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8007a80 con 0x7f1fd80062e0 2021-04-03T03:08:39.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe4f35700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1fd8005ae0 0x7f1fd80d7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49866/0 (socket says 172.21.2.111:49866) 2021-04-03T03:08:39.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe4f35700 1 -- 172.21.2.111:0/2651701793 learned_addr learned my addr 172.21.2.111:0/2651701793 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:39.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdffff700 1 --2- 172.21.2.111:0/2651701793 >> v2:172.21.2.111:3300/0 conn(0x7f1fd80473f0 0x7f1fd8007590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 --2- 172.21.2.111:0/2651701793 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80062e0 0x7f1fd80d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdffff700 1 -- 172.21.2.111:0/2651701793 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80062e0 msgr2=0x7f1fd80d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdffff700 1 --2- 172.21.2.111:0/2651701793 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80062e0 0x7f1fd80d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdffff700 1 -- 172.21.2.111:0/2651701793 >> v1:172.21.2.111:6789/0 conn(0x7f1fd8005ae0 legacy=0x7f1fd80d7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdffff700 1 -- 172.21.2.111:0/2651701793 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd803f7f0 con 0x7f1fd80473f0 2021-04-03T03:08:39.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdffff700 1 --2- 172.21.2.111:0/2651701793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80473f0 0x7f1fd8007590 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1fe0069660 tx=0x7f1fd0006dd0).ready entity=mon.0 client_cookie=2ac344a64893fc8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdeffd700 1 -- 172.21.2.111:0/2651701793 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd0009610 con 0x7f1fd80473f0 2021-04-03T03:08:39.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdeffd700 1 -- 172.21.2.111:0/2651701793 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0009770 con 0x7f1fd80473f0 2021-04-03T03:08:39.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdeffd700 1 -- 172.21.2.111:0/2651701793 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd0009ad0 con 0x7f1fd80473f0 2021-04-03T03:08:39.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- 172.21.2.111:0/2651701793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80473f0 msgr2=0x7f1fd8007590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- 172.21.2.111:0/2651701793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80473f0 0x7f1fd8007590 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1fe0069660 tx=0x7f1fd0006dd0).stop 2021-04-03T03:08:39.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- 172.21.2.111:0/2651701793 shutdown_connections 2021-04-03T03:08:39.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- 172.21.2.111:0/2651701793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80473f0 0x7f1fd8007590 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- 172.21.2.111:0/2651701793 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80062e0 0x7f1fd80d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- 172.21.2.111:0/2651701793 >> 172.21.2.111:0/2651701793 conn(0x7f1fd800d9e0 msgr2=0x7f1fd8046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- 172.21.2.111:0/2651701793 shutdown_connections 2021-04-03T03:08:39.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- 172.21.2.111:0/2651701793 wait complete. 2021-04-03T03:08:39.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 Processor -- start 2021-04-03T03:08:39.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- start start 2021-04-03T03:08:39.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1fd8005ae0 0x7f1fd80da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80062e0 0x7f1fd80d4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80473f0 0x7f1fd80d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd803afa0 con 0x7f1fd80062e0 2021-04-03T03:08:39.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80d85e0 con 0x7f1fd8005ae0 2021-04-03T03:08:39.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe5736700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80db380 con 0x7f1fd80473f0 2021-04-03T03:08:39.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80062e0 0x7f1fd80d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80062e0 0x7f1fd80d4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49078/0 (socket says 172.21.2.111:49078) 2021-04-03T03:08:39.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 -- 172.21.2.111:0/3479073256 learned_addr learned my addr 172.21.2.111:0/3479073256 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:39.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdffff700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1fd8005ae0 0x7f1fd80da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fe4f35700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80473f0 0x7f1fd80d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 -- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80473f0 msgr2=0x7f1fd80d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80473f0 0x7f1fd80d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 -- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1fd8005ae0 msgr2=0x7f1fd80da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1fd8005ae0 0x7f1fd80da3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 -- 172.21.2.111:0/3479073256 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80d8a50 con 0x7f1fd80062e0 2021-04-03T03:08:39.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.231+0000 7f1fdf7fe700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80062e0 0x7f1fd80d4030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1fd400c000 tx=0x7f1fd4006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdd7fa700 1 -- 172.21.2.111:0/3479073256 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd400ca10 con 0x7f1fd80062e0 2021-04-03T03:08:39.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdd7fa700 1 -- 172.21.2.111:0/3479073256 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd400cb70 con 0x7f1fd80062e0 2021-04-03T03:08:39.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd80d1f50 con 0x7f1fd80062e0 2021-04-03T03:08:39.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdd7fa700 1 -- 172.21.2.111:0/3479073256 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd40123f0 con 0x7f1fd80062e0 2021-04-03T03:08:39.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd80d23f0 con 0x7f1fd80062e0 2021-04-03T03:08:39.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd80d9360 con 0x7f1fd80062e0 2021-04-03T03:08:39.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdd7fa700 1 -- 172.21.2.111:0/3479073256 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1fd4012550 con 0x7f1fd80062e0 2021-04-03T03:08:39.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdd7fa700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1fc00570a0 0x7f1fc0059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdd7fa700 1 -- 172.21.2.111:0/3479073256 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1fd4070120 con 0x7f1fd80062e0 2021-04-03T03:08:39.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdffff700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1fc00570a0 0x7f1fc0059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdffff700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1fc00570a0 0x7f1fc0059560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f1fd001c000 tx=0x7f1fd0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.235+0000 7f1fdd7fa700 1 -- 172.21.2.111:0/3479073256 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1fd80d9360 con 0x7f1fd80062e0 2021-04-03T03:08:39.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fd80d9360 con 0x7f1fd80062e0 2021-04-03T03:08:39.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fdd7fa700 1 -- 172.21.2.111:0/3479073256 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f1fd80d9360 con 0x7f1fd80062e0 2021-04-03T03:08:39.343 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:39.344 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":17,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:39.148208+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:39.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1fc00570a0 msgr2=0x7f1fc0059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1fc00570a0 0x7f1fc0059560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f1fd001c000 tx=0x7f1fd0015000).stop 2021-04-03T03:08:39.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80062e0 msgr2=0x7f1fd80d4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80062e0 0x7f1fd80d4030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1fd400c000 tx=0x7f1fd4006dd0).stop 2021-04-03T03:08:39.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 shutdown_connections 2021-04-03T03:08:39.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1fd80473f0 0x7f1fd80d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1fc00570a0 0x7f1fc0059560 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1fd8005ae0 0x7f1fd80da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 --2- 172.21.2.111:0/3479073256 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1fd80062e0 0x7f1fd80d4030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 >> 172.21.2.111:0/3479073256 conn(0x7f1fd800d9e0 msgr2=0x7f1fd8046180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 shutdown_connections 2021-04-03T03:08:39.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.331+0000 7f1fe5736700 1 -- 172.21.2.111:0/3479073256 wait complete. 2021-04-03T03:08:39.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:08:39.352 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:08:39.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 Processor -- start 2021-04-03T03:08:39.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- start start 2021-04-03T03:08:39.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5d80603f0 0x7fb5d81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d80607d0 con 0x7fb5d80b17b0 2021-04-03T03:08:39.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d81451c0 con 0x7fb5d80603f0 2021-04-03T03:08:39.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d814b090 con 0x7fb5d80add50 2021-04-03T03:08:39.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5d80603f0 0x7fb5d81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5d80603f0 0x7fb5d81447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49084/0 (socket says 172.21.2.111:49084) 2021-04-03T03:08:39.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1547237281 learned_addr learned my addr 172.21.2.111:0/1547237281 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:39.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1547237281 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 msgr2=0x7fb5d8144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 --2- 172.21.2.111:0/1547237281 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1547237281 >> v1:172.21.2.115:6789/0 conn(0x7fb5d80b17b0 legacy=0x7fb5d8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1547237281 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d814b330 con 0x7fb5d80603f0 2021-04-03T03:08:39.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d7fff700 1 --2- 172.21.2.111:0/1547237281 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d81447e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fb5d80b5600 tx=0x7fb5c8006e90).ready entity=mon.0 client_cookie=25bd3e2bbf84e515 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d6ffd700 1 -- 172.21.2.111:0/1547237281 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c80098e0 con 0x7fb5d80603f0 2021-04-03T03:08:39.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d6ffd700 1 -- 172.21.2.111:0/1547237281 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8009a40 con 0x7fb5d80603f0 2021-04-03T03:08:39.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5d6ffd700 1 -- 172.21.2.111:0/1547237281 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c8009de0 con 0x7fb5d80603f0 2021-04-03T03:08:39.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- 172.21.2.111:0/1547237281 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 msgr2=0x7fb5d81447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1547237281 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d81447e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fb5d80b5600 tx=0x7fb5c8006e90).stop 2021-04-03T03:08:39.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- 172.21.2.111:0/1547237281 shutdown_connections 2021-04-03T03:08:39.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1547237281 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d81447e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1547237281 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- 172.21.2.111:0/1547237281 >> 172.21.2.111:0/1547237281 conn(0x7fb5d80b61c0 msgr2=0x7fb5d805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- 172.21.2.111:0/1547237281 shutdown_connections 2021-04-03T03:08:39.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.439+0000 7fb5dd499700 1 -- 172.21.2.111:0/1547237281 wait complete. 2021-04-03T03:08:39.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 Processor -- start 2021-04-03T03:08:39.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 -- start start 2021-04-03T03:08:39.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d80aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5d80b17b0 0x7fb5d815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8159f30 con 0x7fb5d80603f0 2021-04-03T03:08:39.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8148270 con 0x7fb5d80b17b0 2021-04-03T03:08:39.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8148510 con 0x7fb5d80add50 2021-04-03T03:08:39.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d80aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d80aa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49088/0 (socket says 172.21.2.111:49088) 2021-04-03T03:08:39.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1907163757 learned_addr learned my addr 172.21.2.111:0/1907163757 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:39.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d77fe700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 msgr2=0x7fb5d8149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5d80b17b0 msgr2=0x7fb5d815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5d80b17b0 0x7fb5d815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 -- 172.21.2.111:0/1907163757 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8149d10 con 0x7fb5d80603f0 2021-04-03T03:08:39.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d77fe700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:39.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d7fff700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d80aa1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb5d80b5600 tx=0x7fb5c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d57fa700 1 -- 172.21.2.111:0/1907163757 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c8011e30 con 0x7fb5d80603f0 2021-04-03T03:08:39.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d57fa700 1 -- 172.21.2.111:0/1907163757 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c801e460 con 0x7fb5d80603f0 2021-04-03T03:08:39.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d81479d0 con 0x7fb5d80603f0 2021-04-03T03:08:39.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d57fa700 1 -- 172.21.2.111:0/1907163757 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c801e7c0 con 0x7fb5d80603f0 2021-04-03T03:08:39.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d815a850 con 0x7fb5d80603f0 2021-04-03T03:08:39.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d815ae50 con 0x7fb5d80603f0 2021-04-03T03:08:39.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d57fa700 1 -- 172.21.2.111:0/1907163757 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb5c8025030 con 0x7fb5d80603f0 2021-04-03T03:08:39.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d57fa700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5bc056ec0 0x7fb5bc059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d57fa700 1 -- 172.21.2.111:0/1907163757 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb5c8071210 con 0x7fb5d80603f0 2021-04-03T03:08:39.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d77fe700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5bc056ec0 0x7fb5bc059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.443+0000 7fb5d77fe700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5bc056ec0 0x7fb5bc059380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb5c0007000 tx=0x7fb5c0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.447+0000 7fb5d57fa700 1 -- 172.21.2.111:0/1907163757 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb5d815ae50 con 0x7fb5d80603f0 2021-04-03T03:08:39.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.539+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb5d815ae50 con 0x7fb5d80603f0 2021-04-03T03:08:39.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.539+0000 7fb5d57fa700 1 -- 172.21.2.111:0/1907163757 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb5d815ae50 con 0x7fb5d80603f0 2021-04-03T03:08:39.554 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:08:39.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5bc056ec0 msgr2=0x7fb5bc059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5bc056ec0 0x7fb5bc059380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb5c0007000 tx=0x7fb5c0008040).stop 2021-04-03T03:08:39.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 msgr2=0x7fb5d80aa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d80aa1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb5d80b5600 tx=0x7fb5c8015000).stop 2021-04-03T03:08:39.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 shutdown_connections 2021-04-03T03:08:39.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5bc056ec0 0x7fb5bc059380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5d80b17b0 0x7fb5d815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5d80603f0 0x7fb5d80aa1a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 --2- 172.21.2.111:0/1907163757 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5d80add50 0x7fb5d8149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 >> 172.21.2.111:0/1907163757 conn(0x7fb5d80b61c0 msgr2=0x7fb5d814a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 shutdown_connections 2021-04-03T03:08:39.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.543+0000 7fb5dd499700 1 -- 172.21.2.111:0/1907163757 wait complete. 2021-04-03T03:08:39.563 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:08:39.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb50d5700 1 Processor -- start 2021-04-03T03:08:39.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb50d5700 1 -- start start 2021-04-03T03:08:39.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb50d5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 0x7feba80dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb50d5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba80dc560 con 0x7feba803d560 2021-04-03T03:08:39.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb50d5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba80dc800 con 0x7feba80dbba0 2021-04-03T03:08:39.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb50d5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba803de60 con 0x7feba8041060 2021-04-03T03:08:39.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb48d4700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7feba803d560 0x7feba8041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49892/0 (socket says 172.21.2.111:49892) 2021-04-03T03:08:39.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febb48d4700 1 -- 172.21.2.111:0/2814253883 learned_addr learned my addr 172.21.2.111:0/2814253883 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:39.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527545638 0 0) 0x7feba80dc560 con 0x7feba803d560 2021-04-03T03:08:39.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.619+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb9c003240 con 0x7feba803d560 2021-04-03T03:08:39.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaf7fe700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 0x7feba80dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1571553016 0 0) 0x7feb9c003240 con 0x7feba803d560 2021-04-03T03:08:39.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 msgr2=0x7feba80dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 0x7feba80dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 >> v1:172.21.2.115:6789/0 conn(0x7feba80dbba0 legacy=0x7feba803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba803da30 con 0x7feba803d560 2021-04-03T03:08:39.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7febb00450f0 con 0x7feba803d560 2021-04-03T03:08:39.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 >> v1:172.21.2.111:6789/0 conn(0x7feba803d560 legacy=0x7feba8041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb9c0037e0 0x7feb9c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 0x7feb9c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb9c0049d0 0x7feb9c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba803de60 con 0x7feb9c0037e0 2021-04-03T03:08:39.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c0041c0 con 0x7feb9c0049d0 2021-04-03T03:08:39.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c00b730 con 0x7feba8041060 2021-04-03T03:08:39.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb9c0037e0 0x7feb9c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaf7fe700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 0x7feb9c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 -- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 msgr2=0x7feb9c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba8041060 0x7feb9c0044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 -- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb9c0049d0 msgr2=0x7feb9c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb9c0049d0 0x7feb9c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 -- 172.21.2.111:0/2814253883 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c003240 con 0x7feb9c0037e0 2021-04-03T03:08:39.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb9c0037e0 0x7feb9c003be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7feb9c0054b0 tx=0x7feba0007b50).ready entity=mon.0 client_cookie=660f3021c606bf1d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba000c620 con 0x7feb9c0037e0 2021-04-03T03:08:39.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba000c780 con 0x7feb9c0037e0 2021-04-03T03:08:39.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaeffd700 1 -- 172.21.2.111:0/2814253883 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba000cae0 con 0x7feb9c0037e0 2021-04-03T03:08:39.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/2814253883 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb9c0037e0 msgr2=0x7feb9c003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 --2- 172.21.2.111:0/2814253883 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb9c0037e0 0x7feb9c003be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7feb9c0054b0 tx=0x7feba0007b50).stop 2021-04-03T03:08:39.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb48d4700 1 -- 172.21.2.111:0/2814253883 reap_dead start 2021-04-03T03:08:39.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/2814253883 shutdown_connections 2021-04-03T03:08:39.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/2814253883 >> 172.21.2.111:0/2814253883 conn(0x7feba80462b0 msgr2=0x7feba80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/2814253883 shutdown_connections 2021-04-03T03:08:39.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/2814253883 wait complete. 2021-04-03T03:08:39.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 Processor -- start 2021-04-03T03:08:39.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- start start 2021-04-03T03:08:39.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feba8039b60 0x7feba80a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feba809f4a0 0x7feba803b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba803b790 0x7feba80d5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba80a36c0 con 0x7feba8039b60 2021-04-03T03:08:39.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba803abd0 con 0x7feba809f4a0 2021-04-03T03:08:39.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba8039f40 con 0x7feba803b790 2021-04-03T03:08:39.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feba8039b60 0x7feba80a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feba8039b60 0x7feba80a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49108/0 (socket says 172.21.2.111:49108) 2021-04-03T03:08:39.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 -- 172.21.2.111:0/3783738389 learned_addr learned my addr 172.21.2.111:0/3783738389 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:39.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 -- 172.21.2.111:0/3783738389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba803b790 msgr2=0x7feba80d5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba803b790 0x7feba80d5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 -- 172.21.2.111:0/3783738389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feba809f4a0 msgr2=0x7feba803b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feba809f4a0 0x7feba803b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 -- 172.21.2.111:0/3783738389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80ac0c0 con 0x7feba8039b60 2021-04-03T03:08:39.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febaffff700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feba8039b60 0x7feba80a0460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7feb9c0054b0 tx=0x7feba0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febad7fa700 1 -- 172.21.2.111:0/3783738389 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba0013860 con 0x7feba8039b60 2021-04-03T03:08:39.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febad7fa700 1 -- 172.21.2.111:0/3783738389 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba00139c0 con 0x7feba8039b60 2021-04-03T03:08:39.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba803ae00 con 0x7feba8039b60 2021-04-03T03:08:39.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba803c460 con 0x7feba8039b60 2021-04-03T03:08:39.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febad7fa700 1 -- 172.21.2.111:0/3783738389 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba0013d20 con 0x7feba8039b60 2021-04-03T03:08:39.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.623+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba803af90 con 0x7feba8039b60 2021-04-03T03:08:39.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.627+0000 7febad7fa700 1 -- 172.21.2.111:0/3783738389 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7feba0010070 con 0x7feba8039b60 2021-04-03T03:08:39.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.627+0000 7febad7fa700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb90057050 0x7feb90059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.627+0000 7febad7fa700 1 -- 172.21.2.111:0/3783738389 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7feba0071250 con 0x7feba8039b60 2021-04-03T03:08:39.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.627+0000 7febaf7fe700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb90057050 0x7feb90059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.627+0000 7febaf7fe700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb90057050 0x7feb90059510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7feba4008000 tx=0x7feba4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.627+0000 7febad7fa700 1 -- 172.21.2.111:0/3783738389 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7feba00469b0 con 0x7feba8039b60 2021-04-03T03:08:39.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feba8004db0 con 0x7feba8039b60 2021-04-03T03:08:39.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febad7fa700 1 -- 172.21.2.111:0/3783738389 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7feba0049fd0 con 0x7feba8039b60 2021-04-03T03:08:39.734 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:39.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb90057050 msgr2=0x7feb90059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb90057050 0x7feb90059510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7feba4008000 tx=0x7feba4008040).stop 2021-04-03T03:08:39.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feba8039b60 msgr2=0x7feba80a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feba8039b60 0x7feba80a0460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7feb9c0054b0 tx=0x7feba0015000).stop 2021-04-03T03:08:39.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 shutdown_connections 2021-04-03T03:08:39.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feba803b790 0x7feba80d5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feba8039b60 0x7feba80a0460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb90057050 0x7feb90059510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 --2- 172.21.2.111:0/3783738389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feba809f4a0 0x7feba803b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 >> 172.21.2.111:0/3783738389 conn(0x7feba80462b0 msgr2=0x7feba8005ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 shutdown_connections 2021-04-03T03:08:39.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.723+0000 7febb50d5700 1 -- 172.21.2.111:0/3783738389 wait complete. 2021-04-03T03:08:39.742 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:08:39.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 Processor -- start 2021-04-03T03:08:39.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 -- start start 2021-04-03T03:08:39.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc25414b700 0x7fc2540b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2540af400 con 0x7fc2540b0c90 2021-04-03T03:08:39.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2540af6a0 con 0x7fc2540603f0 2021-04-03T03:08:39.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc254060860 con 0x7fc25414b700 2021-04-03T03:08:39.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2596be700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc2540b0c90 0x7fc254147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49912/0 (socket says 172.21.2.111:49912) 2021-04-03T03:08:39.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2596be700 1 -- 172.21.2.111:0/1176320194 learned_addr learned my addr 172.21.2.111:0/1176320194 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:39.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 --2- 172.21.2.111:0/1176320194 >> v2:172.21.2.115:3300/0 conn(0x7fc25414b700 0x7fc2540b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774811988 0 0) 0x7fc2540af400 con 0x7fc2540b0c90 2021-04-03T03:08:39.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc234003240 con 0x7fc2540b0c90 2021-04-03T03:08:39.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 201263601 0 0) 0x7fc234003240 con 0x7fc2540b0c90 2021-04-03T03:08:39.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 >> v2:172.21.2.115:3300/0 conn(0x7fc25414b700 msgr2=0x7fc2540b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 --2- 172.21.2.111:0/1176320194 >> v2:172.21.2.115:3300/0 conn(0x7fc25414b700 0x7fc2540b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 >> v1:172.21.2.115:6789/0 conn(0x7fc2540603f0 legacy=0x7fc2540b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540afb40 con 0x7fc2540b0c90 2021-04-03T03:08:39.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fc24c001a50 con 0x7fc2540b0c90 2021-04-03T03:08:39.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 >> v1:172.21.2.111:6789/0 conn(0x7fc2540b0c90 legacy=0x7fc254147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc234003730 0x7fc234003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc234004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc234004e50 0x7fc23400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc254060860 con 0x7fc2340044e0 2021-04-03T03:08:39.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc234004260 con 0x7fc234003730 2021-04-03T03:08:39.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400bc20 con 0x7fc234004e50 2021-04-03T03:08:39.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc234004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2596be700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc234004e50 0x7fc23400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc258ebd700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc234003730 0x7fc234003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 -- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc234004e50 msgr2=0x7fc23400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc234004e50 0x7fc23400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 -- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc234003730 msgr2=0x7fc234003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc234003730 0x7fc234003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 -- 172.21.2.111:0/1176320194 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540afb40 con 0x7fc2340044e0 2021-04-03T03:08:39.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2596be700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc234004e50 0x7fc23400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:39.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2596be700 1 -- 172.21.2.111:0/1176320194 reap_dead start 2021-04-03T03:08:39.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc253fff700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc234004960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fc24400c000 tx=0x7fc244008040).ready entity=mon.0 client_cookie=5ee325c6a9ff2874 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc24400e9d0 con 0x7fc2340044e0 2021-04-03T03:08:39.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24400eb30 con 0x7fc2340044e0 2021-04-03T03:08:39.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc2537fe700 1 -- 172.21.2.111:0/1176320194 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc24400ee90 con 0x7fc2340044e0 2021-04-03T03:08:39.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 -- 172.21.2.111:0/1176320194 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 msgr2=0x7fc234004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.791+0000 7fc259ebf700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc234004960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fc24400c000 tx=0x7fc244008040).stop 2021-04-03T03:08:39.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- 172.21.2.111:0/1176320194 shutdown_connections 2021-04-03T03:08:39.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 --2- 172.21.2.111:0/1176320194 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc234004960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- 172.21.2.111:0/1176320194 >> 172.21.2.111:0/1176320194 conn(0x7fc2540b61c0 msgr2=0x7fc25405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- 172.21.2.111:0/1176320194 shutdown_connections 2021-04-03T03:08:39.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- 172.21.2.111:0/1176320194 wait complete. 2021-04-03T03:08:39.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 Processor -- start 2021-04-03T03:08:39.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- start start 2021-04-03T03:08:39.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc254159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2540aadd0 0x7fc2540a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc2540628d0 0x7fc254062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540adb50 con 0x7fc2340044e0 2021-04-03T03:08:39.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540ad410 con 0x7fc2540aadd0 2021-04-03T03:08:39.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540ad6b0 con 0x7fc2540628d0 2021-04-03T03:08:39.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc254159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc254159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49128/0 (socket says 172.21.2.111:49128) 2021-04-03T03:08:39.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 -- 172.21.2.111:0/4037774884 learned_addr learned my addr 172.21.2.111:0/4037774884 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:39.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 -- 172.21.2.111:0/4037774884 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc2540628d0 msgr2=0x7fc254062d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc2540628d0 0x7fc254062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 -- 172.21.2.111:0/4037774884 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2540aadd0 msgr2=0x7fc2540a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2540aadd0 0x7fc2540a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 -- 172.21.2.111:0/4037774884 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254064d90 con 0x7fc2340044e0 2021-04-03T03:08:39.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc258ebd700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc254159770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fc23c00b000 tx=0x7fc23c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc251ffb700 1 -- 172.21.2.111:0/4037774884 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc23c010520 con 0x7fc2340044e0 2021-04-03T03:08:39.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc251ffb700 1 -- 172.21.2.111:0/4037774884 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c010680 con 0x7fc2340044e0 2021-04-03T03:08:39.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2540a90c0 con 0x7fc2340044e0 2021-04-03T03:08:39.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc254146a70 con 0x7fc2340044e0 2021-04-03T03:08:39.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc251ffb700 1 -- 172.21.2.111:0/4037774884 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc23c0109e0 con 0x7fc2340044e0 2021-04-03T03:08:39.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2540ad870 con 0x7fc2340044e0 2021-04-03T03:08:39.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc251ffb700 1 -- 172.21.2.111:0/4037774884 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc23c010b40 con 0x7fc2340044e0 2021-04-03T03:08:39.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc251ffb700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc238056fb0 0x7fc238059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc251ffb700 1 -- 172.21.2.111:0/4037774884 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc23c06fa70 con 0x7fc2340044e0 2021-04-03T03:08:39.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc253fff700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc238056fb0 0x7fc238059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.795+0000 7fc253fff700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc238056fb0 0x7fc238059470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc24400c000 tx=0x7fc24401a000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.799+0000 7fc251ffb700 1 -- 172.21.2.111:0/4037774884 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc23c043c50 con 0x7fc2340044e0 2021-04-03T03:08:39.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.891+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc254064ba0 con 0x7fc2340044e0 2021-04-03T03:08:39.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc251ffb700 1 -- 172.21.2.111:0/4037774884 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc23c06fe60 con 0x7fc2340044e0 2021-04-03T03:08:39.905 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:39.905 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:08:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:08:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:08:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:08:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:08:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:08:39.907 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:08:39.907 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:08:39.907 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:08:39.907 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:08:39.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc238056fb0 msgr2=0x7fc238059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc238056fb0 0x7fc238059470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc24400c000 tx=0x7fc24401a000).stop 2021-04-03T03:08:39.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 msgr2=0x7fc254159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc254159770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fc23c00b000 tx=0x7fc23c009040).stop 2021-04-03T03:08:39.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 shutdown_connections 2021-04-03T03:08:39.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc2540628d0 0x7fc254062d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2340044e0 0x7fc254159770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc238056fb0 0x7fc238059470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 --2- 172.21.2.111:0/4037774884 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2540aadd0 0x7fc2540a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 >> 172.21.2.111:0/4037774884 conn(0x7fc2540b61c0 msgr2=0x7fc254147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 shutdown_connections 2021-04-03T03:08:39.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.895+0000 7fc259ebf700 1 -- 172.21.2.111:0/4037774884 wait complete. 2021-04-03T03:08:39.912 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:08:39.913 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:08:39.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9daca95700 1 Processor -- start 2021-04-03T03:08:39.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9daca95700 1 -- start start 2021-04-03T03:08:39.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9daca95700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9da80add50 0x7f9da8147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9daca95700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9da8144ef0 0x7f9da8147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9daca95700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da80607d0 con 0x7f9da80603f0 2021-04-03T03:08:39.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9daca95700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da814b090 con 0x7f9da80add50 2021-04-03T03:08:39.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9daca95700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da814b330 con 0x7f9da8144ef0 2021-04-03T03:08:39.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9da80add50 0x7f9da8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9da80add50 0x7f9da8147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49140/0 (socket says 172.21.2.111:49140) 2021-04-03T03:08:39.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 -- 172.21.2.111:0/594108835 learned_addr learned my addr 172.21.2.111:0/594108835 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:39.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 -- 172.21.2.111:0/594108835 >> v2:172.21.2.115:3300/0 conn(0x7f9da8144ef0 msgr2=0x7f9da8147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 --2- 172.21.2.111:0/594108835 >> v2:172.21.2.115:3300/0 conn(0x7f9da8144ef0 0x7f9da8147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 -- 172.21.2.111:0/594108835 >> v1:172.21.2.115:6789/0 conn(0x7f9da80603f0 legacy=0x7f9da81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:39.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 -- 172.21.2.111:0/594108835 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da814b5d0 con 0x7f9da80add50 2021-04-03T03:08:39.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da6ffd700 1 --2- 172.21.2.111:0/594108835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8147740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9da80ae520 tx=0x7f9d98006e90).ready entity=mon.0 client_cookie=e65ee5eda71abd97 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.959+0000 7f9da67fc700 1 -- 172.21.2.111:0/594108835 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d98009930 con 0x7f9da80add50 2021-04-03T03:08:39.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da67fc700 1 -- 172.21.2.111:0/594108835 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d98009a90 con 0x7f9da80add50 2021-04-03T03:08:39.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da67fc700 1 -- 172.21.2.111:0/594108835 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d98009e30 con 0x7f9da80add50 2021-04-03T03:08:39.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/594108835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 msgr2=0x7f9da8147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 --2- 172.21.2.111:0/594108835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8147740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9da80ae520 tx=0x7f9d98006e90).stop 2021-04-03T03:08:39.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/594108835 shutdown_connections 2021-04-03T03:08:39.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 --2- 172.21.2.111:0/594108835 >> v2:172.21.2.115:3300/0 conn(0x7f9da8144ef0 0x7f9da8147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 --2- 172.21.2.111:0/594108835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8147740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:39.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/594108835 >> 172.21.2.111:0/594108835 conn(0x7f9da80b61c0 msgr2=0x7f9da805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:39.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/594108835 shutdown_connections 2021-04-03T03:08:39.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/594108835 wait complete. 2021-04-03T03:08:39.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 Processor -- start 2021-04-03T03:08:39.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- start start 2021-04-03T03:08:39.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9da80603f0 0x7f9da815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9da8144ef0 0x7f9da8143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da8157e80 con 0x7f9da80add50 2021-04-03T03:08:39.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80a9050 con 0x7f9da8144ef0 2021-04-03T03:08:39.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da8143500 con 0x7f9da80603f0 2021-04-03T03:08:39.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8142b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49144/0 (socket says 172.21.2.111:49144) 2021-04-03T03:08:39.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 -- 172.21.2.111:0/381994799 learned_addr learned my addr 172.21.2.111:0/381994799 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:39.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 -- 172.21.2.111:0/381994799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9da80603f0 msgr2=0x7f9da815a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9da80603f0 0x7f9da815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 -- 172.21.2.111:0/381994799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9da8144ef0 msgr2=0x7f9da8143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:39.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9da8144ef0 0x7f9da8143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:39.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 -- 172.21.2.111:0/381994799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da8067cd0 con 0x7f9da80add50 2021-04-03T03:08:39.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da6ffd700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8142b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9da80a8ed0 tx=0x7f9d98015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da4ff9700 1 -- 172.21.2.111:0/381994799 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d9801e460 con 0x7f9da80add50 2021-04-03T03:08:39.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da4ff9700 1 -- 172.21.2.111:0/381994799 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9801e5c0 con 0x7f9da80add50 2021-04-03T03:08:39.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da815ab60 con 0x7f9da80add50 2021-04-03T03:08:39.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da8146e70 con 0x7f9da80add50 2021-04-03T03:08:39.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9da4ff9700 1 -- 172.21.2.111:0/381994799 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d9801e920 con 0x7f9da80add50 2021-04-03T03:08:39.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.963+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da8147b50 con 0x7f9da80add50 2021-04-03T03:08:39.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.967+0000 7f9da4ff9700 1 -- 172.21.2.111:0/381994799 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9d98025030 con 0x7f9da80add50 2021-04-03T03:08:39.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.967+0000 7f9da4ff9700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d8c056f70 0x7f9d8c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:39.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.967+0000 7f9da4ff9700 1 -- 172.21.2.111:0/381994799 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9d98071390 con 0x7f9da80add50 2021-04-03T03:08:39.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.967+0000 7f9da77fe700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d8c056f70 0x7f9d8c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:39.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.967+0000 7f9da77fe700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d8c056f70 0x7f9d8c059430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9d90002f50 tx=0x7f9d90006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:39.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:39.967+0000 7f9da4ff9700 1 -- 172.21.2.111:0/381994799 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9d98046af0 con 0x7f9da80add50 2021-04-03T03:08:40.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.059+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9da8057660 con 0x7f9da80add50 2021-04-03T03:08:40.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.175+0000 7f9da4ff9700 1 -- 172.21.2.111:0/381994799 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9d9804a110 con 0x7f9da80add50 2021-04-03T03:08:40.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d8c056f70 msgr2=0x7f9d8c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:40.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d8c056f70 0x7f9d8c059430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9d90002f50 tx=0x7f9d90006990).stop 2021-04-03T03:08:40.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 msgr2=0x7f9da8142b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:40.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8142b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9da80a8ed0 tx=0x7f9d98015000).stop 2021-04-03T03:08:40.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 shutdown_connections 2021-04-03T03:08:40.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d8c056f70 0x7f9d8c059430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:40.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9da8144ef0 0x7f9da8143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:40.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9da80603f0 0x7f9da815a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:40.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 --2- 172.21.2.111:0/381994799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9da80add50 0x7f9da8142b20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:40.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 >> 172.21.2.111:0/381994799 conn(0x7f9da80b61c0 msgr2=0x7f9da814e230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:40.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 shutdown_connections 2021-04-03T03:08:40.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.179+0000 7f9daca95700 1 -- 172.21.2.111:0/381994799 wait complete. 2021-04-03T03:08:40.192 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:08:40.195 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2021-04-03T03:08:40.196 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2021-04-03T03:08:40.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 Processor -- start 2021-04-03T03:08:40.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- start start 2021-04-03T03:08:40.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b7403fdf0 0x7f3b74040fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:40.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7403d600 0x7f3b7403c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:40.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b74041610 con 0x7f3b74041f00 2021-04-03T03:08:40.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b740422e0 con 0x7f3b7403fdf0 2021-04-03T03:08:40.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74042700 con 0x7f3b7403d600 2021-04-03T03:08:40.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b7403fdf0 0x7f3b74040fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:40.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b7403fdf0 0x7f3b74040fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49154/0 (socket says 172.21.2.111:49154) 2021-04-03T03:08:40.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/778748944 learned_addr learned my addr 172.21.2.111:0/778748944 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:40.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7b7fe700 1 --2- 172.21.2.111:0/778748944 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7403d600 0x7f3b7403c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:40.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/778748944 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7403d600 msgr2=0x7f3b7403c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:40.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 --2- 172.21.2.111:0/778748944 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7403d600 0x7f3b7403c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:40.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/778748944 >> v1:172.21.2.115:6789/0 conn(0x7f3b74041f00 legacy=0x7f3b740d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/778748944 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b74041870 con 0x7f3b7403fdf0 2021-04-03T03:08:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7a7fc700 1 --2- 172.21.2.111:0/778748944 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b74040fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f3b74040980 tx=0x7f3b6c006e90).ready entity=mon.0 client_cookie=48c22b3e6442cccc server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b79ffb700 1 -- 172.21.2.111:0/778748944 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b6c0098e0 con 0x7f3b7403fdf0 2021-04-03T03:08:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b79ffb700 1 -- 172.21.2.111:0/778748944 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c009a40 con 0x7f3b7403fdf0 2021-04-03T03:08:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b79ffb700 1 -- 172.21.2.111:0/778748944 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b6c009de0 con 0x7f3b7403fdf0 2021-04-03T03:08:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- 172.21.2.111:0/778748944 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 msgr2=0x7f3b74040fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/778748944 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b74040fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f3b74040980 tx=0x7f3b6c006e90).stop 2021-04-03T03:08:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- 172.21.2.111:0/778748944 shutdown_connections 2021-04-03T03:08:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/778748944 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7403d600 0x7f3b7403c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/778748944 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b74040fa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- 172.21.2.111:0/778748944 >> 172.21.2.111:0/778748944 conn(0x7f3b740462b0 msgr2=0x7f3b740466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- 172.21.2.111:0/778748944 shutdown_connections 2021-04-03T03:08:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- 172.21.2.111:0/778748944 wait complete. 2021-04-03T03:08:40.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 Processor -- start 2021-04-03T03:08:40.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.252+0000 7f3b7bfff700 1 -- start start 2021-04-03T03:08:40.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7403d600 0x7f3b740d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:40.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b740d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:40.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b74041f00 0x7f3b740d3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:40.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b740e7330 con 0x7f3b7403fdf0 2021-04-03T03:08:40.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7403c7b0 con 0x7f3b7403d600 2021-04-03T03:08:40.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7403c510 con 0x7f3b74041f00 2021-04-03T03:08:40.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b740d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:40.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b740d2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49158/0 (socket says 172.21.2.111:49158) 2021-04-03T03:08:40.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/4261802775 learned_addr learned my addr 172.21.2.111:0/4261802775 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:40.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/4261802775 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b74041f00 msgr2=0x7f3b740d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:40.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b74041f00 0x7f3b740d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:40.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/4261802775 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7403d600 msgr2=0x7f3b740d70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:40.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7403d600 0x7f3b740d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:40.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 -- 172.21.2.111:0/4261802775 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b740e6e90 con 0x7f3b7403fdf0 2021-04-03T03:08:40.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7a7fc700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b740d2f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f3b74040980 tx=0x7f3b6c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:40.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b73fff700 1 -- 172.21.2.111:0/4261802775 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b6c011ce0 con 0x7f3b7403fdf0 2021-04-03T03:08:40.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b73fff700 1 -- 172.21.2.111:0/4261802775 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c011e40 con 0x7f3b7403fdf0 2021-04-03T03:08:40.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7403b070 con 0x7f3b7403fdf0 2021-04-03T03:08:40.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b73fff700 1 -- 172.21.2.111:0/4261802775 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b6c01e650 con 0x7f3b7403fdf0 2021-04-03T03:08:40.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b74039580 con 0x7f3b7403fdf0 2021-04-03T03:08:40.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b740e7020 con 0x7f3b7403fdf0 2021-04-03T03:08:40.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b73fff700 1 -- 172.21.2.111:0/4261802775 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3b6c025030 con 0x7f3b7403fdf0 2021-04-03T03:08:40.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b73fff700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ec0 0x7f3b60059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:40.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b73fff700 1 -- 172.21.2.111:0/4261802775 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f3b6c071440 con 0x7f3b7403fdf0 2021-04-03T03:08:40.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7affd700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ec0 0x7f3b60059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:40.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.256+0000 7f3b7affd700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ec0 0x7f3b60059380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3b64002eb0 tx=0x7f3b640068f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:40.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.260+0000 7f3b73fff700 1 -- 172.21.2.111:0/4261802775 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3b6c046a40 con 0x7f3b7403fdf0 2021-04-03T03:08:40.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:40.364+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f3b74004d50 con 0x7f3b7403fdf0 2021-04-03T03:08:41.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.180+0000 7f3b73fff700 1 -- 172.21.2.111:0/4261802775 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v19) v1 ==== 154+0+0 (secure 0 0 0) 0x7f3b6c0230e0 con 0x7f3b7403fdf0 2021-04-03T03:08:41.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ec0 msgr2=0x7f3b60059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:41.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ec0 0x7f3b60059380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3b64002eb0 tx=0x7f3b640068f0).stop 2021-04-03T03:08:41.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 msgr2=0x7f3b740d2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:41.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b740d2f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f3b74040980 tx=0x7f3b6c015000).stop 2021-04-03T03:08:41.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 shutdown_connections 2021-04-03T03:08:41.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b74041f00 0x7f3b740d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:41.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ec0 0x7f3b60059380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:41.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7403d600 0x7f3b740d70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:41.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 --2- 172.21.2.111:0/4261802775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7403fdf0 0x7f3b740d2f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:41.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 >> 172.21.2.111:0/4261802775 conn(0x7f3b740462b0 msgr2=0x7f3b7403eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:41.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 shutdown_connections 2021-04-03T03:08:41.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.184+0000 7f3b7bfff700 1 -- 172.21.2.111:0/4261802775 wait complete. 2021-04-03T03:08:41.201 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 erasure cephfs_data 2021-04-03T03:08:41.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.248+0000 7fb431913700 1 Processor -- start 2021-04-03T03:08:41.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.248+0000 7fb431913700 1 -- start start 2021-04-03T03:08:41.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.248+0000 7fb431913700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb4240d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:41.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.248+0000 7fb431913700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb424045ab0 0x7fb4240db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:41.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.248+0000 7fb431913700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb424047a40 con 0x7fb424047660 2021-04-03T03:08:41.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.248+0000 7fb431913700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4240422b0 con 0x7fb424045ab0 2021-04-03T03:08:41.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.248+0000 7fb431913700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb424046430 con 0x7fb4240454a0 2021-04-03T03:08:41.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431112700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb424045ab0 0x7fb4240db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:41.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb430911700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fb424047660 0x7fb4240d8a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34264/0 (socket says 172.21.2.111:34264) 2021-04-03T03:08:41.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb4240d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:41.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb430911700 1 -- 172.21.2.111:0/2231749050 learned_addr learned my addr 172.21.2.111:0/2231749050 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:41.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42bfff700 1 -- 172.21.2.111:0/2231749050 >> v2:172.21.2.115:3300/0 conn(0x7fb424045ab0 msgr2=0x7fb4240db880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:41.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42bfff700 1 --2- 172.21.2.111:0/2231749050 >> v2:172.21.2.115:3300/0 conn(0x7fb424045ab0 0x7fb4240db880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:41.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42bfff700 1 -- 172.21.2.111:0/2231749050 >> v1:172.21.2.115:6789/0 conn(0x7fb424047660 legacy=0x7fb4240d8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:41.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42bfff700 1 -- 172.21.2.111:0/2231749050 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240466d0 con 0x7fb4240454a0 2021-04-03T03:08:41.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42bfff700 1 --2- 172.21.2.111:0/2231749050 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb4240d9140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb418002f30 tx=0x7fb418006dd0).ready entity=mon.2 client_cookie=51ccc92ef0268f17 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:41.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42b7fe700 1 -- 172.21.2.111:0/2231749050 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb418009760 con 0x7fb4240454a0 2021-04-03T03:08:41.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42b7fe700 1 -- 172.21.2.111:0/2231749050 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4180098c0 con 0x7fb4240454a0 2021-04-03T03:08:41.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb42b7fe700 1 -- 172.21.2.111:0/2231749050 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb418009c20 con 0x7fb4240454a0 2021-04-03T03:08:41.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 -- 172.21.2.111:0/2231749050 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 msgr2=0x7fb4240d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:41.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 --2- 172.21.2.111:0/2231749050 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb4240d9140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb418002f30 tx=0x7fb418006dd0).stop 2021-04-03T03:08:41.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 -- 172.21.2.111:0/2231749050 shutdown_connections 2021-04-03T03:08:41.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 --2- 172.21.2.111:0/2231749050 >> v2:172.21.2.115:3300/0 conn(0x7fb424045ab0 0x7fb4240db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:41.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 --2- 172.21.2.111:0/2231749050 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb4240d9140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:41.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 -- 172.21.2.111:0/2231749050 >> 172.21.2.111:0/2231749050 conn(0x7fb424010a50 msgr2=0x7fb424010e60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:41.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 -- 172.21.2.111:0/2231749050 shutdown_connections 2021-04-03T03:08:41.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 -- 172.21.2.111:0/2231749050 wait complete. 2021-04-03T03:08:41.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 Processor -- start 2021-04-03T03:08:41.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.252+0000 7fb431913700 1 -- start start 2021-04-03T03:08:41.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb42400c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:41.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb424045ab0 0x7fb4240d76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:41.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb424047660 0x7fb4240d7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:41.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42400ab20 con 0x7fb424045ab0 2021-04-03T03:08:41.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240e7180 con 0x7fb424047660 2021-04-03T03:08:41.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb424041040 con 0x7fb4240454a0 2021-04-03T03:08:41.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb424045ab0 0x7fb4240d76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:41.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb424045ab0 0x7fb4240d76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49174/0 (socket says 172.21.2.111:49174) 2021-04-03T03:08:41.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 -- 172.21.2.111:0/3687000410 learned_addr learned my addr 172.21.2.111:0/3687000410 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:41.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb430911700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb42400c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:41.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 -- 172.21.2.111:0/3687000410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 msgr2=0x7fb42400c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:41.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb42400c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:41.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 -- 172.21.2.111:0/3687000410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb424047660 msgr2=0x7fb4240d7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:41.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb424047660 0x7fb4240d7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:41.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 -- 172.21.2.111:0/3687000410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42400ce00 con 0x7fb424045ab0 2021-04-03T03:08:41.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb42bfff700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb424045ab0 0x7fb4240d76f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb418002f30 tx=0x7fb418015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:41.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb429ffb700 1 -- 172.21.2.111:0/3687000410 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb41801d450 con 0x7fb424045ab0 2021-04-03T03:08:41.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb429ffb700 1 -- 172.21.2.111:0/3687000410 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41801d5b0 con 0x7fb424045ab0 2021-04-03T03:08:41.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42403fea0 con 0x7fb424045ab0 2021-04-03T03:08:41.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb429ffb700 1 -- 172.21.2.111:0/3687000410 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb41801d910 con 0x7fb424045ab0 2021-04-03T03:08:41.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4240d8260 con 0x7fb424045ab0 2021-04-03T03:08:41.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb424040150 con 0x7fb424045ab0 2021-04-03T03:08:41.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb429ffb700 1 -- 172.21.2.111:0/3687000410 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb41800f070 con 0x7fb424045ab0 2021-04-03T03:08:41.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb429ffb700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb414057050 0x7fb414059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:41.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb429ffb700 1 -- 172.21.2.111:0/3687000410 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5859+0+0 (secure 0 0 0) 0x7fb418071e20 con 0x7fb424045ab0 2021-04-03T03:08:41.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb430911700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb414057050 0x7fb414059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:41.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.256+0000 7fb430911700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb414057050 0x7fb414059510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb420008000 tx=0x7fb420009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:41.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.260+0000 7fb429ffb700 1 -- 172.21.2.111:0/3687000410 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb418047430 con 0x7fb424045ab0 2021-04-03T03:08:41.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:41.352+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7fb42400d230 con 0x7fb424045ab0 2021-04-03T03:08:43.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.209+0000 7fb429ffb700 1 -- 172.21.2.111:0/3687000410 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v21) v1 ==== 188+0+0 (secure 0 0 0) 0x7fb41804aa50 con 0x7fb424045ab0 2021-04-03T03:08:43.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb414057050 msgr2=0x7fb414059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:43.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb414057050 0x7fb414059510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb420008000 tx=0x7fb420009040).stop 2021-04-03T03:08:43.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb424045ab0 msgr2=0x7fb4240d76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:43.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb424045ab0 0x7fb4240d76f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb418002f30 tx=0x7fb418015000).stop 2021-04-03T03:08:43.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 shutdown_connections 2021-04-03T03:08:43.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb414057050 0x7fb414059510 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:43.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb424047660 0x7fb4240d7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:43.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4240454a0 0x7fb42400c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:43.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 --2- 172.21.2.111:0/3687000410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb424045ab0 0x7fb4240d76f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:43.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 >> 172.21.2.111:0/3687000410 conn(0x7fb424010a50 msgr2=0x7fb424006340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:43.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 shutdown_connections 2021-04-03T03:08:43.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.213+0000 7fb431913700 1 -- 172.21.2.111:0/3687000410 wait complete. 2021-04-03T03:08:43.222 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:08:43.227 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool set cephfs_data allow_ec_overwrites true 2021-04-03T03:08:43.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 Processor -- start 2021-04-03T03:08:43.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 -- start start 2021-04-03T03:08:43.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00b24d0 0x7f12f00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f12f00ad660 0x7f12f0129ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f00b8730 con 0x7f12f00b8350 2021-04-03T03:08:43.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f0142440 con 0x7f12f00ad660 2021-04-03T03:08:43.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0141350 con 0x7f12f00b24d0 2021-04-03T03:08:43.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f553f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f12f00b8350 0x7f12f0141d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49972/0 (socket says 172.21.2.111:49972) 2021-04-03T03:08:43.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f553f700 1 -- 172.21.2.111:0/2669790623 learned_addr learned my addr 172.21.2.111:0/2669790623 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:43.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328463679 0 0) 0x7f12f00b8730 con 0x7f12f00b8350 2021-04-03T03:08:43.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 --2- 172.21.2.111:0/2669790623 >> v2:172.21.2.115:3300/0 conn(0x7f12f00ad660 0x7f12f0129ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:43.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d8003240 con 0x7f12f00b8350 2021-04-03T03:08:43.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f4d3e700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00b24d0 0x7f12f00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:43.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3206891283 0 0) 0x7f12d8003240 con 0x7f12f00b8350 2021-04-03T03:08:43.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00b24d0 msgr2=0x7f12f00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:43.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00b24d0 0x7f12f00ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:43.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 >> v2:172.21.2.115:3300/0 conn(0x7f12f00ad660 msgr2=0x7f12f0129ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:43.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 --2- 172.21.2.111:0/2669790623 >> v2:172.21.2.115:3300/0 conn(0x7f12f00ad660 0x7f12f0129ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:43.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f01415f0 con 0x7f12f00b8350 2021-04-03T03:08:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f12ec0016d0 con 0x7f12f00b8350 2021-04-03T03:08:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 >> v1:172.21.2.111:6789/0 conn(0x7f12f00b8350 legacy=0x7f12f0141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12d80037b0 0x7f12d8003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00b24d0 0x7f12d80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12d80049d0 0x7f12d8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0142440 con 0x7f12d80049d0 2021-04-03T03:08:43.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0141350 con 0x7f12d80037b0 2021-04-03T03:08:43.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80042a0 con 0x7f12f00b24d0 2021-04-03T03:08:43.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12d80049d0 0x7f12d8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:43.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 -- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00b24d0 msgr2=0x7f12d80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:43.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00b24d0 0x7f12d80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:43.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 -- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12d80037b0 msgr2=0x7f12d8003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:43.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12d80037b0 0x7f12d8003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:43.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 -- 172.21.2.111:0/2669790623 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d8003240 con 0x7f12d80049d0 2021-04-03T03:08:43.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12d80049d0 0x7f12d8004e50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f12e800a000 tx=0x7f12e8008040).ready entity=mon.0 client_cookie=f13aaee7c27b10a4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:43.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e800ddf0 con 0x7f12d80049d0 2021-04-03T03:08:43.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e8013480 con 0x7f12d80049d0 2021-04-03T03:08:43.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12e3fff700 1 -- 172.21.2.111:0/2669790623 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e80137e0 con 0x7f12d80049d0 2021-04-03T03:08:43.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 -- 172.21.2.111:0/2669790623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12d80049d0 msgr2=0x7f12d8004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:43.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f6541700 1 --2- 172.21.2.111:0/2669790623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12d80049d0 0x7f12d8004e50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f12e800a000 tx=0x7f12e8008040).stop 2021-04-03T03:08:43.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.277+0000 7f12f5d40700 1 -- 172.21.2.111:0/2669790623 reap_dead start 2021-04-03T03:08:43.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- 172.21.2.111:0/2669790623 shutdown_connections 2021-04-03T03:08:43.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- 172.21.2.111:0/2669790623 >> 172.21.2.111:0/2669790623 conn(0x7f12f0060490 msgr2=0x7f12f00b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:43.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- 172.21.2.111:0/2669790623 shutdown_connections 2021-04-03T03:08:43.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- 172.21.2.111:0/2669790623 wait complete. 2021-04-03T03:08:43.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 Processor -- start 2021-04-03T03:08:43.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- start start 2021-04-03T03:08:43.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12f00b5a70 0x7f12f01512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12f00672c0 0x7f12f012b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00aec90 0x7f12f012b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f012bc20 con 0x7f12f00672c0 2021-04-03T03:08:43.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f00b5e50 con 0x7f12f00b5a70 2021-04-03T03:08:43.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f00b0410 con 0x7f12f00aec90 2021-04-03T03:08:43.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12f00672c0 0x7f12f012b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:43.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12f00672c0 0x7f12f012b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49196/0 (socket says 172.21.2.111:49196) 2021-04-03T03:08:43.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 -- 172.21.2.111:0/344345517 learned_addr learned my addr 172.21.2.111:0/344345517 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:43.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f5d40700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00aec90 0x7f12f012b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:43.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 -- 172.21.2.111:0/344345517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00aec90 msgr2=0x7f12f012b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:43.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00aec90 0x7f12f012b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:43.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 -- 172.21.2.111:0/344345517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12f00b5a70 msgr2=0x7f12f01512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:43.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12f00b5a70 0x7f12f01512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:43.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 -- 172.21.2.111:0/344345517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f00b06b0 con 0x7f12f00672c0 2021-04-03T03:08:43.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f4d3e700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12f00672c0 0x7f12f012b240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f12e400c000 tx=0x7f12e4006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:43.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12e27fc700 1 -- 172.21.2.111:0/344345517 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e4016070 con 0x7f12f00672c0 2021-04-03T03:08:43.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12e27fc700 1 -- 172.21.2.111:0/344345517 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400ca10 con 0x7f12f00672c0 2021-04-03T03:08:43.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f00b0020 con 0x7f12f00672c0 2021-04-03T03:08:43.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f0065c80 con 0x7f12f00672c0 2021-04-03T03:08:43.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12e27fc700 1 -- 172.21.2.111:0/344345517 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e400cd70 con 0x7f12f00672c0 2021-04-03T03:08:43.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12e27fc700 1 -- 172.21.2.111:0/344345517 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f12e401b410 con 0x7f12f00672c0 2021-04-03T03:08:43.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f00b0840 con 0x7f12f00672c0 2021-04-03T03:08:43.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12e27fc700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12d0057000 0x7f12d00594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:43.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12e27fc700 1 -- 172.21.2.111:0/344345517 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f12e4070380 con 0x7f12f00672c0 2021-04-03T03:08:43.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f553f700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12d0057000 0x7f12d00594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:43.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.281+0000 7f12f553f700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12d0057000 0x7f12d00594c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f12ec0013e0 tx=0x7f12ec0012c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:43.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.285+0000 7f12e27fc700 1 -- 172.21.2.111:0/344345517 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f12e4045660 con 0x7f12f00672c0 2021-04-03T03:08:43.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:43.381+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f12f0057660 con 0x7f12f00672c0 2021-04-03T03:08:44.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.213+0000 7f12e27fc700 1 -- 172.21.2.111:0/344345517 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v22) v1 ==== 166+0+0 (secure 0 0 0) 0x7f12e4048c80 con 0x7f12f00672c0 2021-04-03T03:08:44.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12d0057000 msgr2=0x7f12d00594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12d0057000 0x7f12d00594c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f12ec0013e0 tx=0x7f12ec0012c0).stop 2021-04-03T03:08:44.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12f00672c0 msgr2=0x7f12f012b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12f00672c0 0x7f12f012b240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f12e400c000 tx=0x7f12e4006dd0).stop 2021-04-03T03:08:44.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 shutdown_connections 2021-04-03T03:08:44.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12f00aec90 0x7f12f012b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12d0057000 0x7f12d00594c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12f00b5a70 0x7f12f01512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 --2- 172.21.2.111:0/344345517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12f00672c0 0x7f12f012b240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 >> 172.21.2.111:0/344345517 conn(0x7f12f0060490 msgr2=0x7f12f0062f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 shutdown_connections 2021-04-03T03:08:44.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.217+0000 7f12f6541700 1 -- 172.21.2.111:0/344345517 wait complete. 2021-04-03T03:08:44.227 INFO:teuthology.orchestra.run.gibba011.stderr:set pool 4 allow_ec_overwrites to true 2021-04-03T03:08:44.232 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:08:44.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 Processor -- start 2021-04-03T03:08:44.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- start start 2021-04-03T03:08:44.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd7680dbba0 0x7fd7680dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd768041060 0x7fd76803fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7680dc560 con 0x7fd76803d560 2021-04-03T03:08:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76803de60 con 0x7fd768041060 2021-04-03T03:08:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680dc800 con 0x7fd7680dbba0 2021-04-03T03:08:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd76803d560 0x7fd768041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:49994/0 (socket says 172.21.2.111:49994) 2021-04-03T03:08:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd768041060 0x7fd76803fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76ffff700 1 -- 172.21.2.111:0/839496683 learned_addr learned my addr 172.21.2.111:0/839496683 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76effd700 1 -- 172.21.2.111:0/839496683 >> v2:172.21.2.115:3300/0 conn(0x7fd7680dbba0 msgr2=0x7fd7680dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76effd700 1 --2- 172.21.2.111:0/839496683 >> v2:172.21.2.115:3300/0 conn(0x7fd7680dbba0 0x7fd7680dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76effd700 1 -- 172.21.2.111:0/839496683 >> v1:172.21.2.111:6789/0 conn(0x7fd76803d560 legacy=0x7fd768041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76effd700 1 -- 172.21.2.111:0/839496683 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76803da30 con 0x7fd768041060 2021-04-03T03:08:44.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76effd700 1 --2- 172.21.2.111:0/839496683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd768041060 0x7fd76803fa80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd764002f30 tx=0x7fd764006dd0).ready entity=mon.0 client_cookie=6a00963f710fa1d5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76e7fc700 1 -- 172.21.2.111:0/839496683 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7640097f0 con 0x7fd768041060 2021-04-03T03:08:44.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76e7fc700 1 -- 172.21.2.111:0/839496683 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764009950 con 0x7fd768041060 2021-04-03T03:08:44.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd76e7fc700 1 -- 172.21.2.111:0/839496683 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd764009cb0 con 0x7fd768041060 2021-04-03T03:08:44.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- 172.21.2.111:0/839496683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd768041060 msgr2=0x7fd76803fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 --2- 172.21.2.111:0/839496683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd768041060 0x7fd76803fa80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd764002f30 tx=0x7fd764006dd0).stop 2021-04-03T03:08:44.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- 172.21.2.111:0/839496683 shutdown_connections 2021-04-03T03:08:44.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 --2- 172.21.2.111:0/839496683 >> v2:172.21.2.115:3300/0 conn(0x7fd7680dbba0 0x7fd7680dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 --2- 172.21.2.111:0/839496683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd768041060 0x7fd76803fa80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- 172.21.2.111:0/839496683 >> 172.21.2.111:0/839496683 conn(0x7fd7680462b0 msgr2=0x7fd7680466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- 172.21.2.111:0/839496683 shutdown_connections 2021-04-03T03:08:44.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- 172.21.2.111:0/839496683 wait complete. 2021-04-03T03:08:44.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 Processor -- start 2021-04-03T03:08:44.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.290+0000 7fd774831700 1 -- start start 2021-04-03T03:08:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd76803d560 0x7fd7680a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768041060 0x7fd7680d6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680dbba0 0x7fd7680d6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680df3c0 con 0x7fd7680dbba0 2021-04-03T03:08:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76803bb30 con 0x7fd768041060 2021-04-03T03:08:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680a0cb0 con 0x7fd76803d560 2021-04-03T03:08:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680dbba0 0x7fd7680d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680dbba0 0x7fd7680d6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49210/0 (socket says 172.21.2.111:49210) 2021-04-03T03:08:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 -- 172.21.2.111:0/2327493153 learned_addr learned my addr 172.21.2.111:0/2327493153 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 -- 172.21.2.111:0/2327493153 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd76803d560 msgr2=0x7fd7680a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd76803d560 0x7fd7680a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 -- 172.21.2.111:0/2327493153 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768041060 msgr2=0x7fd7680d6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768041060 0x7fd7680d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 -- 172.21.2.111:0/2327493153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7680a0a10 con 0x7fd7680dbba0 2021-04-03T03:08:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76ffff700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680dbba0 0x7fd7680d6850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd7680087d0 tx=0x7fd77005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76cff9700 1 -- 172.21.2.111:0/2327493153 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd77005df60 con 0x7fd7680dbba0 2021-04-03T03:08:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76cff9700 1 -- 172.21.2.111:0/2327493153 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7700699f0 con 0x7fd7680dbba0 2021-04-03T03:08:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76803b690 con 0x7fd7680dbba0 2021-04-03T03:08:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76cff9700 1 -- 172.21.2.111:0/2327493153 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd77006b5d0 con 0x7fd7680dbba0 2021-04-03T03:08:44.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd768039ca0 con 0x7fd7680dbba0 2021-04-03T03:08:44.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7680a0820 con 0x7fd7680dbba0 2021-04-03T03:08:44.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76cff9700 1 -- 172.21.2.111:0/2327493153 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd77008b020 con 0x7fd7680dbba0 2021-04-03T03:08:44.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76cff9700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750056ea0 0x7fd750059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76cff9700 1 -- 172.21.2.111:0/2327493153 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fd7700df1c0 con 0x7fd7680dbba0 2021-04-03T03:08:44.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76f7fe700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750056ea0 0x7fd750059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.294+0000 7fd76f7fe700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750056ea0 0x7fd750059360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd760002f30 tx=0x7fd7600069b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.298+0000 7fd76cff9700 1 -- 172.21.2.111:0/2327493153 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd7700b4500 con 0x7fd7680dbba0 2021-04-03T03:08:44.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.418+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd76803b820 con 0x7fd7680dbba0 2021-04-03T03:08:44.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd76cff9700 1 -- 172.21.2.111:0/2327493153 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7fd7700b7b20 con 0x7fd7680dbba0 2021-04-03T03:08:44.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750056ea0 msgr2=0x7fd750059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750056ea0 0x7fd750059360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd760002f30 tx=0x7fd7600069b0).stop 2021-04-03T03:08:44.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680dbba0 msgr2=0x7fd7680d6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680dbba0 0x7fd7680d6850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd7680087d0 tx=0x7fd77005ad70).stop 2021-04-03T03:08:44.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 shutdown_connections 2021-04-03T03:08:44.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680dbba0 0x7fd7680d6850 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd76803d560 0x7fd7680a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750056ea0 0x7fd750059360 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 --2- 172.21.2.111:0/2327493153 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768041060 0x7fd7680d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 >> 172.21.2.111:0/2327493153 conn(0x7fd7680462b0 msgr2=0x7fd768041660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 shutdown_connections 2021-04-03T03:08:44.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.426+0000 7fd774831700 1 -- 172.21.2.111:0/2327493153 wait complete. 2021-04-03T03:08:44.435 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 3 and data pool 4 2021-04-03T03:08:44.440 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:44.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 Processor -- start 2021-04-03T03:08:44.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- start start 2021-04-03T03:08:44.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd2d40603f0 0x7fd2d40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd2d414b700 0x7fd2d40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d40607d0 con 0x7fd2d40b12b0 2021-04-03T03:08:44.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d40af400 con 0x7fd2d414b700 2021-04-03T03:08:44.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d40af6a0 con 0x7fd2d40603f0 2021-04-03T03:08:44.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2da78f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd2d40b12b0 0x7fd2d41465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50018/0 (socket says 172.21.2.111:50018) 2021-04-03T03:08:44.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2da78f700 1 -- 172.21.2.111:0/559801326 learned_addr learned my addr 172.21.2.111:0/559801326 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:44.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d978d700 1 --2- 172.21.2.111:0/559801326 >> v2:172.21.2.111:3300/0 conn(0x7fd2d414b700 0x7fd2d40b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d9f8e700 1 --2- 172.21.2.111:0/559801326 >> v2:172.21.2.115:3300/0 conn(0x7fd2d40603f0 0x7fd2d40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d8f8c700 1 -- 172.21.2.111:0/559801326 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306324909 0 0) 0x7fd2d40607d0 con 0x7fd2d40b12b0 2021-04-03T03:08:44.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d8f8c700 1 -- 172.21.2.111:0/559801326 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2bc003240 con 0x7fd2d40b12b0 2021-04-03T03:08:44.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d978d700 1 -- 172.21.2.111:0/559801326 >> v2:172.21.2.115:3300/0 conn(0x7fd2d40603f0 msgr2=0x7fd2d40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d978d700 1 --2- 172.21.2.111:0/559801326 >> v2:172.21.2.115:3300/0 conn(0x7fd2d40603f0 0x7fd2d40b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d978d700 1 -- 172.21.2.111:0/559801326 >> v1:172.21.2.111:6789/0 conn(0x7fd2d40b12b0 legacy=0x7fd2d41465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d978d700 1 -- 172.21.2.111:0/559801326 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d40afb40 con 0x7fd2d414b700 2021-04-03T03:08:44.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d978d700 1 --2- 172.21.2.111:0/559801326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d414b700 0x7fd2d40b0580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd2c8002f30 tx=0x7fd2c8006dd0).ready entity=mon.0 client_cookie=5a911669ec6a1c23 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d8f8c700 1 -- 172.21.2.111:0/559801326 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c80097b0 con 0x7fd2d414b700 2021-04-03T03:08:44.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d8f8c700 1 -- 172.21.2.111:0/559801326 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8009910 con 0x7fd2d414b700 2021-04-03T03:08:44.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2d8f8c700 1 -- 172.21.2.111:0/559801326 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c8009c70 con 0x7fd2d414b700 2021-04-03T03:08:44.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- 172.21.2.111:0/559801326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d414b700 msgr2=0x7fd2d40b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 --2- 172.21.2.111:0/559801326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d414b700 0x7fd2d40b0580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd2c8002f30 tx=0x7fd2c8006dd0).stop 2021-04-03T03:08:44.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- 172.21.2.111:0/559801326 shutdown_connections 2021-04-03T03:08:44.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 --2- 172.21.2.111:0/559801326 >> v2:172.21.2.115:3300/0 conn(0x7fd2d40603f0 0x7fd2d40b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 --2- 172.21.2.111:0/559801326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d414b700 0x7fd2d40b0580 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- 172.21.2.111:0/559801326 >> 172.21.2.111:0/559801326 conn(0x7fd2d40b61c0 msgr2=0x7fd2d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- 172.21.2.111:0/559801326 shutdown_connections 2021-04-03T03:08:44.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 -- 172.21.2.111:0/559801326 wait complete. 2021-04-03T03:08:44.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.490+0000 7fd2daf90700 1 Processor -- start 2021-04-03T03:08:44.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 -- start start 2021-04-03T03:08:44.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d40603f0 0x7fd2d40abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d40b12b0 0x7fd2d4149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d414b700 0x7fd2d4159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d40aa0a0 con 0x7fd2d40603f0 2021-04-03T03:08:44.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4159930 con 0x7fd2d40b12b0 2021-04-03T03:08:44.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4062a50 con 0x7fd2d414b700 2021-04-03T03:08:44.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d40603f0 0x7fd2d40abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d40603f0 0x7fd2d40abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49234/0 (socket says 172.21.2.111:49234) 2021-04-03T03:08:44.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 -- 172.21.2.111:0/2068793410 learned_addr learned my addr 172.21.2.111:0/2068793410 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:44.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2da78f700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d414b700 0x7fd2d4159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d978d700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d40b12b0 0x7fd2d4149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 -- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d414b700 msgr2=0x7fd2d4159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d414b700 0x7fd2d4159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 -- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d40b12b0 msgr2=0x7fd2d4149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d40b12b0 0x7fd2d4149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 -- 172.21.2.111:0/2068793410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d4062cf0 con 0x7fd2d40603f0 2021-04-03T03:08:44.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d9f8e700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d40603f0 0x7fd2d40abad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd2d0003070 tx=0x7fd2d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2c6ffd700 1 -- 172.21.2.111:0/2068793410 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d0009980 con 0x7fd2d40603f0 2021-04-03T03:08:44.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2c6ffd700 1 -- 172.21.2.111:0/2068793410 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d0009ae0 con 0x7fd2d40603f0 2021-04-03T03:08:44.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2c6ffd700 1 -- 172.21.2.111:0/2068793410 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d0009e40 con 0x7fd2d40603f0 2021-04-03T03:08:44.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d4062f90 con 0x7fd2d40603f0 2021-04-03T03:08:44.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d40aa7c0 con 0x7fd2d40603f0 2021-04-03T03:08:44.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d4057660 con 0x7fd2d40603f0 2021-04-03T03:08:44.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2c6ffd700 1 -- 172.21.2.111:0/2068793410 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd2d0013480 con 0x7fd2d40603f0 2021-04-03T03:08:44.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2c6ffd700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2b4056ef0 0x7fd2b40593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d978d700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2b4056ef0 0x7fd2b40593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2c6ffd700 1 -- 172.21.2.111:0/2068793410 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fd2d0070af0 con 0x7fd2d40603f0 2021-04-03T03:08:44.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.494+0000 7fd2d978d700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2b4056ef0 0x7fd2b40593b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd2c8019000 tx=0x7fd2c8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.498+0000 7fd2c6ffd700 1 -- 172.21.2.111:0/2068793410 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd2d0045d60 con 0x7fd2d40603f0 2021-04-03T03:08:44.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2d40686d0 con 0x7fd2d40603f0 2021-04-03T03:08:44.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2c6ffd700 1 -- 172.21.2.111:0/2068793410 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16312 (secure 0 0 0) 0x7fd2d0049380 con 0x7fd2d40603f0 2021-04-03T03:08:44.596 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:44.596 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":23,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:44.422660+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:44.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2b4056ef0 msgr2=0x7fd2b40593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2b4056ef0 0x7fd2b40593b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd2c8019000 tx=0x7fd2c8015000).stop 2021-04-03T03:08:44.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d40603f0 msgr2=0x7fd2d40abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d40603f0 0x7fd2d40abad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd2d0003070 tx=0x7fd2d0008040).stop 2021-04-03T03:08:44.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 shutdown_connections 2021-04-03T03:08:44.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d414b700 0x7fd2d4159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d40603f0 0x7fd2d40abad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2b4056ef0 0x7fd2b40593b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 --2- 172.21.2.111:0/2068793410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d40b12b0 0x7fd2d4149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 >> 172.21.2.111:0/2068793410 conn(0x7fd2d40b61c0 msgr2=0x7fd2d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 shutdown_connections 2021-04-03T03:08:44.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.590+0000 7fd2daf90700 1 -- 172.21.2.111:0/2068793410 wait complete. 2021-04-03T03:08:44.604 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:44.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27bfff700 1 Processor -- start 2021-04-03T03:08:44.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27bfff700 1 -- start start 2021-04-03T03:08:44.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc274005dc0 0x7fc274042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2740dc6e0 con 0x7fc274006bc0 2021-04-03T03:08:44.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc274042010 con 0x7fc2740dbb90 2021-04-03T03:08:44.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740d9560 con 0x7fc274005dc0 2021-04-03T03:08:44.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27a7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc274006bc0 0x7fc2740056b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50030/0 (socket says 172.21.2.111:50030) 2021-04-03T03:08:44.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27a7fc700 1 -- 172.21.2.111:0/2168955862 learned_addr learned my addr 172.21.2.111:0/2168955862 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:44.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888269963 0 0) 0x7fc2740dc6e0 con 0x7fc274006bc0 2021-04-03T03:08:44.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc25c003240 con 0x7fc274006bc0 2021-04-03T03:08:44.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2490050609 0 0) 0x7fc25c003240 con 0x7fc274006bc0 2021-04-03T03:08:44.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27b7fe700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc274005dc0 0x7fc274042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc274005dc0 msgr2=0x7fc274042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc274005dc0 0x7fc274042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 >> v1:172.21.2.115:6789/0 conn(0x7fc2740dbb90 legacy=0x7fc2740064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740422b0 con 0x7fc274006bc0 2021-04-03T03:08:44.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fc26c0016d0 con 0x7fc274006bc0 2021-04-03T03:08:44.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 >> v1:172.21.2.111:6789/0 conn(0x7fc274006bc0 legacy=0x7fc2740056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc25c0037e0 0x7fc25c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc274005dc0 0x7fc25c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc25c0049d0 0x7fc25c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740d9560 con 0x7fc25c0049d0 2021-04-03T03:08:44.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0042b0 con 0x7fc25c0037e0 2021-04-03T03:08:44.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c00b730 con 0x7fc274005dc0 2021-04-03T03:08:44.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27b7fe700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc25c0049d0 0x7fc25c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27b7fe700 1 -- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc274005dc0 msgr2=0x7fc25c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27b7fe700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc274005dc0 0x7fc25c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27b7fe700 1 -- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc25c0037e0 msgr2=0x7fc25c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27b7fe700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc25c0037e0 0x7fc25c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.654+0000 7fc27b7fe700 1 -- 172.21.2.111:0/2168955862 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c003240 con 0x7fc25c0049d0 2021-04-03T03:08:44.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27b7fe700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc25c0049d0 0x7fc25c004e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc27c063000 tx=0x7fc27c064040).ready entity=mon.0 client_cookie=e03808b6fcaba061 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc27c05bda0 con 0x7fc25c0049d0 2021-04-03T03:08:44.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c05df60 con 0x7fc25c0049d0 2021-04-03T03:08:44.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc279ffb700 1 -- 172.21.2.111:0/2168955862 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc27c0699f0 con 0x7fc25c0049d0 2021-04-03T03:08:44.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/2168955862 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc25c0049d0 msgr2=0x7fc25c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 --2- 172.21.2.111:0/2168955862 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc25c0049d0 0x7fc25c004e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc27c063000 tx=0x7fc27c064040).stop 2021-04-03T03:08:44.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27b7fe700 1 -- 172.21.2.111:0/2168955862 reap_dead start 2021-04-03T03:08:44.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/2168955862 shutdown_connections 2021-04-03T03:08:44.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/2168955862 >> 172.21.2.111:0/2168955862 conn(0x7fc2740462c0 msgr2=0x7fc2740466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/2168955862 shutdown_connections 2021-04-03T03:08:44.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/2168955862 wait complete. 2021-04-03T03:08:44.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 Processor -- start 2021-04-03T03:08:44.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- start start 2021-04-03T03:08:44.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2740d8030 0x7fc2740ea390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc27403a170 0x7fc27409f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2740df3c0 0x7fc27409fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740ea880 con 0x7fc2740d8030 2021-04-03T03:08:44.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740d8410 con 0x7fc2740df3c0 2021-04-03T03:08:44.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc274039590 con 0x7fc27403a170 2021-04-03T03:08:44.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2740d8030 0x7fc2740ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2740d8030 0x7fc2740ea390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49250/0 (socket says 172.21.2.111:49250) 2021-04-03T03:08:44.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 -- 172.21.2.111:0/42650534 learned_addr learned my addr 172.21.2.111:0/42650534 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:44.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 -- 172.21.2.111:0/42650534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc27403a170 msgr2=0x7fc27409f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc27403a170 0x7fc27409f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 -- 172.21.2.111:0/42650534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2740df3c0 msgr2=0x7fc27409fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2740df3c0 0x7fc27409fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 -- 172.21.2.111:0/42650534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274039830 con 0x7fc2740d8030 2021-04-03T03:08:44.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27affd700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2740d8030 0x7fc2740ea390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fc264000f30 tx=0x7fc264007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc273fff700 1 -- 172.21.2.111:0/42650534 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc26400e4a0 con 0x7fc2740d8030 2021-04-03T03:08:44.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc273fff700 1 -- 172.21.2.111:0/42650534 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26400e600 con 0x7fc2740d8030 2021-04-03T03:08:44.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc273fff700 1 -- 172.21.2.111:0/42650534 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc26400e960 con 0x7fc2740d8030 2021-04-03T03:08:44.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27409f550 con 0x7fc2740d8030 2021-04-03T03:08:44.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2740d5980 con 0x7fc2740d8030 2021-04-03T03:08:44.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27409f6e0 con 0x7fc2740d8030 2021-04-03T03:08:44.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc273fff700 1 -- 172.21.2.111:0/42650534 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc264010070 con 0x7fc2740d8030 2021-04-03T03:08:44.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc273fff700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc260056ea0 0x7fc260059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc273fff700 1 -- 172.21.2.111:0/42650534 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fc26406fba0 con 0x7fc2740d8030 2021-04-03T03:08:44.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27a7fc700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc260056ea0 0x7fc260059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.658+0000 7fc27a7fc700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc260056ea0 0x7fc260059360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc2740087e0 tx=0x7fc27403e7f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.662+0000 7fc273fff700 1 -- 172.21.2.111:0/42650534 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc264044e70 con 0x7fc2740d8030 2021-04-03T03:08:44.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc274004da0 con 0x7fc2740d8030 2021-04-03T03:08:44.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc273fff700 1 -- 172.21.2.111:0/42650534 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16312 (secure 0 0 0) 0x7fc264048490 con 0x7fc2740d8030 2021-04-03T03:08:44.764 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:44.764 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":23,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:44.422660+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:44.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc260056ea0 msgr2=0x7fc260059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc260056ea0 0x7fc260059360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc2740087e0 tx=0x7fc27403e7f0).stop 2021-04-03T03:08:44.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2740d8030 msgr2=0x7fc2740ea390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2740d8030 0x7fc2740ea390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fc264000f30 tx=0x7fc264007c10).stop 2021-04-03T03:08:44.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 shutdown_connections 2021-04-03T03:08:44.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc260056ea0 0x7fc260059360 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc2740df3c0 0x7fc27409fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2740d8030 0x7fc2740ea390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 --2- 172.21.2.111:0/42650534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc27403a170 0x7fc27409f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 >> 172.21.2.111:0/42650534 conn(0x7fc2740462c0 msgr2=0x7fc27403d700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 shutdown_connections 2021-04-03T03:08:44.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.758+0000 7fc27bfff700 1 -- 172.21.2.111:0/42650534 wait complete. 2021-04-03T03:08:44.772 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:08:44.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7777175700 1 Processor -- start 2021-04-03T03:08:44.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7777175700 1 -- start start 2021-04-03T03:08:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7777175700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7768048d60 0x7f77680d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7777175700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77680d2050 con 0x7f77680d2790 2021-04-03T03:08:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7777175700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77680d22f0 con 0x7f776803f810 2021-04-03T03:08:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7777175700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77680d1d80 con 0x7f7768048d60 2021-04-03T03:08:44.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7776974700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f77680d2790 0x7f776803e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50052/0 (socket says 172.21.2.111:50052) 2021-04-03T03:08:44.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7776974700 1 -- 172.21.2.111:0/3037433570 learned_addr learned my addr 172.21.2.111:0/3037433570 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:44.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751832263 0 0) 0x7f77680d2050 con 0x7f77680d2790 2021-04-03T03:08:44.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7758003240 con 0x7f77680d2790 2021-04-03T03:08:44.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7776173700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7768048d60 0x7f77680d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3567007865 0 0) 0x7f7758003240 con 0x7f77680d2790 2021-04-03T03:08:44.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7768048d60 msgr2=0x7f77680d5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7768048d60 0x7f77680d5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 >> v1:172.21.2.115:6789/0 conn(0x7f776803f810 legacy=0x7f7768043a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7768049140 con 0x7f77680d2790 2021-04-03T03:08:44.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f77700450f0 con 0x7f77680d2790 2021-04-03T03:08:44.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 >> v1:172.21.2.111:6789/0 conn(0x7f77680d2790 legacy=0x7f776803e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7768048d60 0x7f77580037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7758003cd0 0x7f77580044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77580049d0 0x7f7758004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680d1d80 con 0x7f77580049d0 2021-04-03T03:08:44.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580041c0 con 0x7f7758003cd0 2021-04-03T03:08:44.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775800b730 con 0x7f7768048d60 2021-04-03T03:08:44.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.822+0000 7f7776974700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77580049d0 0x7f7758004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7768048d60 msgr2=0x7f77580037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7768048d60 0x7f77580037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7758003cd0 msgr2=0x7f77580044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:44.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7758003cd0 0x7f77580044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:44.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/3037433570 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758003240 con 0x7f77580049d0 2021-04-03T03:08:44.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77580049d0 0x7f7758004e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f7768042b50 tx=0x7f777005ad70).ready entity=mon.0 client_cookie=8418ca0ede953a9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f777005df60 con 0x7f77580049d0 2021-04-03T03:08:44.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77700699f0 con 0x7f77580049d0 2021-04-03T03:08:44.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7775171700 1 -- 172.21.2.111:0/3037433570 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7770061110 con 0x7f77580049d0 2021-04-03T03:08:44.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/3037433570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77580049d0 msgr2=0x7f7758004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 --2- 172.21.2.111:0/3037433570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77580049d0 0x7f7758004e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f7768042b50 tx=0x7f777005ad70).stop 2021-04-03T03:08:44.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/3037433570 reap_dead start 2021-04-03T03:08:44.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/3037433570 shutdown_connections 2021-04-03T03:08:44.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/3037433570 >> 172.21.2.111:0/3037433570 conn(0x7f776800f350 msgr2=0x7f77680484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:44.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/3037433570 shutdown_connections 2021-04-03T03:08:44.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/3037433570 wait complete. 2021-04-03T03:08:44.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 Processor -- start 2021-04-03T03:08:44.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- start start 2021-04-03T03:08:44.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77680e3c10 0x7f77680a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77680e3ff0 0x7f77680a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77680a2d20 0x7f77680a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680a31b0 con 0x7f77680a2d20 2021-04-03T03:08:44.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680e44f0 con 0x7f77680e3ff0 2021-04-03T03:08:44.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77680a5240 con 0x7f77680e3c10 2021-04-03T03:08:44.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77680a2d20 0x7f77680a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77680a2d20 0x7f77680a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49274/0 (socket says 172.21.2.111:49274) 2021-04-03T03:08:44.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/1658347185 learned_addr learned my addr 172.21.2.111:0/1658347185 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:44.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776173700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77680e3c10 0x7f77680a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7775972700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77680e3ff0 0x7f77680a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77680e3c10 msgr2=0x7f77680a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77680e3c10 0x7f77680a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77680e3ff0 msgr2=0x7f77680a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:44.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77680e3ff0 0x7f77680a2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:44.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 -- 172.21.2.111:0/1658347185 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77680443c0 con 0x7f77680a2d20 2021-04-03T03:08:44.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776173700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77680e3c10 0x7f77680a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:44.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7776974700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77680a2d20 0x7f77680a3700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f7768049520 tx=0x7f7770067000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f77637fe700 1 -- 172.21.2.111:0/1658347185 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f777008b070 con 0x7f77680a2d20 2021-04-03T03:08:44.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f77637fe700 1 -- 172.21.2.111:0/1658347185 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77700699f0 con 0x7f77680a2d20 2021-04-03T03:08:44.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f77637fe700 1 -- 172.21.2.111:0/1658347185 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f777005df60 con 0x7f77680a2d20 2021-04-03T03:08:44.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77680a4cc0 con 0x7f77680a2d20 2021-04-03T03:08:44.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77680a4a50 con 0x7f77680a2d20 2021-04-03T03:08:44.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.826+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77680a6ee0 con 0x7f77680a2d20 2021-04-03T03:08:44.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.830+0000 7f77637fe700 1 -- 172.21.2.111:0/1658347185 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f777006a2d0 con 0x7f77680a2d20 2021-04-03T03:08:44.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.830+0000 7f77637fe700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7750056fe0 0x7f77500594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:44.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.830+0000 7f77637fe700 1 -- 172.21.2.111:0/1658347185 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f77700df070 con 0x7f77680a2d20 2021-04-03T03:08:44.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.830+0000 7f7776173700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7750056fe0 0x7f77500594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:44.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.830+0000 7f7776173700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7750056fe0 0x7f77500594a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f776c008000 tx=0x7f776c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:44.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.830+0000 7f77637fe700 1 -- 172.21.2.111:0/1658347185 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f77700b43f0 con 0x7f77680a2d20 2021-04-03T03:08:44.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:44.954+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7768004db0 con 0x7f77680a2d20 2021-04-03T03:08:45.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f77637fe700 1 -- 172.21.2.111:0/1658347185 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f77700b7a10 con 0x7f77680a2d20 2021-04-03T03:08:45.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7750056fe0 msgr2=0x7f77500594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7750056fe0 0x7f77500594a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f776c008000 tx=0x7f776c008040).stop 2021-04-03T03:08:45.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77680a2d20 msgr2=0x7f77680a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77680a2d20 0x7f77680a3700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f7768049520 tx=0x7f7770067000).stop 2021-04-03T03:08:45.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 shutdown_connections 2021-04-03T03:08:45.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77680a2d20 0x7f77680a3700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77680e3c10 0x7f77680a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7750056fe0 0x7f77500594a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 --2- 172.21.2.111:0/1658347185 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77680e3ff0 0x7f77680a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 >> 172.21.2.111:0/1658347185 conn(0x7f776800f350 msgr2=0x7f7768043240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:45.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 shutdown_connections 2021-04-03T03:08:45.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.434+0000 7f7777175700 1 -- 172.21.2.111:0/1658347185 wait complete. 2021-04-03T03:08:45.448 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:08:45.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.506+0000 7fb9cf59e700 1 Processor -- start 2021-04-03T03:08:45.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.506+0000 7fb9cf59e700 1 -- start start 2021-04-03T03:08:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb9c80d75f0 0x7fb9c80d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb9c803cf00 0x7fb9c8041440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c803f930 con 0x7fb9c803d6a0 2021-04-03T03:08:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c80d8390 con 0x7fb9c803cf00 2021-04-03T03:08:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c80402d0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb9c803cf00 0x7fb9c8041440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb9c803cf00 0x7fb9c8041440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49280/0 (socket says 172.21.2.111:49280) 2021-04-03T03:08:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/1443846518 learned_addr learned my addr 172.21.2.111:0/1443846518 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/1443846518 >> v2:172.21.2.115:3300/0 conn(0x7fb9c80d75f0 msgr2=0x7fb9c80d7ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- 172.21.2.111:0/1443846518 >> v2:172.21.2.115:3300/0 conn(0x7fb9c80d75f0 0x7fb9c80d7ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/1443846518 >> v1:172.21.2.111:6789/0 conn(0x7fb9c803d6a0 legacy=0x7fb9c803c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/1443846518 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c803fb90 con 0x7fb9c803cf00 2021-04-03T03:08:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- 172.21.2.111:0/1443846518 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c803cf00 0x7fb9c8041440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb9d006b5d0 tx=0x7fb9d005ad70).ready entity=mon.0 client_cookie=4f510ba5e664a7f1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cd59a700 1 -- 172.21.2.111:0/1443846518 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d005bda0 con 0x7fb9c803cf00 2021-04-03T03:08:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cd59a700 1 -- 172.21.2.111:0/1443846518 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d005df60 con 0x7fb9c803cf00 2021-04-03T03:08:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cd59a700 1 -- 172.21.2.111:0/1443846518 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d00699f0 con 0x7fb9c803cf00 2021-04-03T03:08:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/1443846518 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c803cf00 msgr2=0x7fb9c8041440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/1443846518 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c803cf00 0x7fb9c8041440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb9d006b5d0 tx=0x7fb9d005ad70).stop 2021-04-03T03:08:45.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/1443846518 shutdown_connections 2021-04-03T03:08:45.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/1443846518 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c803cf00 0x7fb9c8041440 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/1443846518 >> v2:172.21.2.115:3300/0 conn(0x7fb9c80d75f0 0x7fb9c80d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/1443846518 >> 172.21.2.111:0/1443846518 conn(0x7fb9c80462c0 msgr2=0x7fb9c80466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:45.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/1443846518 shutdown_connections 2021-04-03T03:08:45.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/1443846518 wait complete. 2021-04-03T03:08:45.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 Processor -- start 2021-04-03T03:08:45.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- start start 2021-04-03T03:08:45.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9c803cf00 0x7fb9c80cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9c803d6a0 0x7fb9c80cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c80d75f0 0x7fb9c80e6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80d5c20 con 0x7fb9c80d75f0 2021-04-03T03:08:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80d54e0 con 0x7fb9c803cf00 2021-04-03T03:08:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80d5780 con 0x7fb9c803d6a0 2021-04-03T03:08:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c80d75f0 0x7fb9c80e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c80d75f0 0x7fb9c80e6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49288/0 (socket says 172.21.2.111:49288) 2021-04-03T03:08:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/3983780461 learned_addr learned my addr 172.21.2.111:0/3983780461 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/3983780461 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9c803d6a0 msgr2=0x7fb9c80cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9c803d6a0 0x7fb9c80cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/3983780461 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9c803cf00 msgr2=0x7fb9c80cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9c803cf00 0x7fb9c80cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 -- 172.21.2.111:0/3983780461 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80d03f0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9ced9d700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c80d75f0 0x7fb9c80e6720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb9d006b5d0 tx=0x7fb9d0065000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9c77fe700 1 -- 172.21.2.111:0/3983780461 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d005b040 con 0x7fb9c80d75f0 2021-04-03T03:08:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9c77fe700 1 -- 172.21.2.111:0/3983780461 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d005f2c0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c80cecc0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9c77fe700 1 -- 172.21.2.111:0/3983780461 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d005b1a0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c80ce1e0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c8004da0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.510+0000 7fb9c77fe700 1 -- 172.21.2.111:0/3983780461 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb9d008c020 con 0x7fb9c80d75f0 2021-04-03T03:08:45.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.514+0000 7fb9c77fe700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b4056fb0 0x7fb9b4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.514+0000 7fb9c77fe700 1 -- 172.21.2.111:0/3983780461 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb9d00e0ba0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.514+0000 7fb9ce59c700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b4056fb0 0x7fb9b4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.514+0000 7fb9ce59c700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b4056fb0 0x7fb9b4059470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb9b8002eb0 tx=0x7fb9b80068f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.514+0000 7fb9c77fe700 1 -- 172.21.2.111:0/3983780461 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb9d00b5f80 con 0x7fb9c80d75f0 2021-04-03T03:08:45.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9c80d58f0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9c77fe700 1 -- 172.21.2.111:0/3983780461 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fb9d00b95a0 con 0x7fb9c80d75f0 2021-04-03T03:08:45.648 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:45.649 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1066299897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1066299897},{"type":"v1","addr":"172.21.2.115:6833","nonce":1066299897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:08:44.422766+0000","modified":"2021-04-03T03:08:45.425728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2149096292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2149096292},{"type":"v1","addr":"172.21.2.111:6835","nonce":2149096292}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-04-03T03:08:45.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b4056fb0 msgr2=0x7fb9b4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b4056fb0 0x7fb9b4059470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb9b8002eb0 tx=0x7fb9b80068f0).stop 2021-04-03T03:08:45.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c80d75f0 msgr2=0x7fb9c80e6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c80d75f0 0x7fb9c80e6720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb9d006b5d0 tx=0x7fb9d0065000).stop 2021-04-03T03:08:45.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 shutdown_connections 2021-04-03T03:08:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9c80d75f0 0x7fb9c80e6720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b4056fb0 0x7fb9b4059470 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9c803cf00 0x7fb9c80cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 --2- 172.21.2.111:0/3983780461 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9c803d6a0 0x7fb9c80cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 >> 172.21.2.111:0/3983780461 conn(0x7fb9c80462c0 msgr2=0x7fb9c8044350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 shutdown_connections 2021-04-03T03:08:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.638+0000 7fb9cf59e700 1 -- 172.21.2.111:0/3983780461 wait complete. 2021-04-03T03:08:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 5 2021-04-03T03:08:45.656 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:45.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 Processor -- start 2021-04-03T03:08:45.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- start start 2021-04-03T03:08:45.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7faed40603f0 0x7faed40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed414b700 0x7faed40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed40607d0 con 0x7faed40b12b0 2021-04-03T03:08:45.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed40af400 con 0x7faed40603f0 2021-04-03T03:08:45.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40af6a0 con 0x7faed414b700 2021-04-03T03:08:45.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7faed40603f0 0x7faed40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7faed40603f0 0x7faed40b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49292/0 (socket says 172.21.2.111:49292) 2021-04-03T03:08:45.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 -- 172.21.2.111:0/3230578909 learned_addr learned my addr 172.21.2.111:0/3230578909 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:45.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 -- 172.21.2.111:0/3230578909 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed414b700 msgr2=0x7faed40b0580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:45.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 --2- 172.21.2.111:0/3230578909 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed414b700 0x7faed40b0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 -- 172.21.2.111:0/3230578909 >> v1:172.21.2.115:6789/0 conn(0x7faed40b12b0 legacy=0x7faed41465d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:45.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 -- 172.21.2.111:0/3230578909 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed40afb40 con 0x7faed40603f0 2021-04-03T03:08:45.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed2384700 1 --2- 172.21.2.111:0/3230578909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed40603f0 0x7faed40b0090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7faed40af200 tx=0x7faec4006e90).ready entity=mon.0 client_cookie=e82a29aa2f85c96b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed1382700 1 -- 172.21.2.111:0/3230578909 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec40098e0 con 0x7faed40603f0 2021-04-03T03:08:45.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed1382700 1 -- 172.21.2.111:0/3230578909 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec4009a40 con 0x7faed40603f0 2021-04-03T03:08:45.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed1382700 1 -- 172.21.2.111:0/3230578909 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec4009de0 con 0x7faed40603f0 2021-04-03T03:08:45.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- 172.21.2.111:0/3230578909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed40603f0 msgr2=0x7faed40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 --2- 172.21.2.111:0/3230578909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed40603f0 0x7faed40b0090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7faed40af200 tx=0x7faec4006e90).stop 2021-04-03T03:08:45.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- 172.21.2.111:0/3230578909 shutdown_connections 2021-04-03T03:08:45.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 --2- 172.21.2.111:0/3230578909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed40603f0 0x7faed40b0090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 --2- 172.21.2.111:0/3230578909 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed414b700 0x7faed40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- 172.21.2.111:0/3230578909 >> 172.21.2.111:0/3230578909 conn(0x7faed40b61c0 msgr2=0x7faed405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:45.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- 172.21.2.111:0/3230578909 shutdown_connections 2021-04-03T03:08:45.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- 172.21.2.111:0/3230578909 wait complete. 2021-04-03T03:08:45.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 Processor -- start 2021-04-03T03:08:45.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.702+0000 7faed3386700 1 -- start start 2021-04-03T03:08:45.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed40603f0 0x7faed4159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faed40b12b0 0x7faed4062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed414b700 0x7faed40644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40a7560 con 0x7faed414b700 2021-04-03T03:08:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40a7800 con 0x7faed40b12b0 2021-04-03T03:08:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40a71e0 con 0x7faed40603f0 2021-04-03T03:08:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed414b700 0x7faed40644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed414b700 0x7faed40644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49302/0 (socket says 172.21.2.111:49302) 2021-04-03T03:08:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 -- 172.21.2.111:0/4080676960 learned_addr learned my addr 172.21.2.111:0/4080676960 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 -- 172.21.2.111:0/4080676960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed40603f0 msgr2=0x7faed4159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed40603f0 0x7faed4159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 -- 172.21.2.111:0/4080676960 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faed40b12b0 msgr2=0x7faed4062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faed40b12b0 0x7faed4062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 -- 172.21.2.111:0/4080676960 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed40ab7d0 con 0x7faed414b700 2021-04-03T03:08:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2b85700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed414b700 0x7faed40644a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7faecc0032f0 tx=0x7faecc0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faecb7fe700 1 -- 172.21.2.111:0/4080676960 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faecc00c9c0 con 0x7faed414b700 2021-04-03T03:08:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faecb7fe700 1 -- 172.21.2.111:0/4080676960 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc00cb20 con 0x7faed414b700 2021-04-03T03:08:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed40aaeb0 con 0x7faed414b700 2021-04-03T03:08:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faecb7fe700 1 -- 172.21.2.111:0/4080676960 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faecc015420 con 0x7faed414b700 2021-04-03T03:08:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed4064990 con 0x7faed414b700 2021-04-03T03:08:45.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed40a73a0 con 0x7faed414b700 2021-04-03T03:08:45.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faecb7fe700 1 -- 172.21.2.111:0/4080676960 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7faecc01c070 con 0x7faed414b700 2021-04-03T03:08:45.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faecb7fe700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faeb8056f20 0x7faeb80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faecb7fe700 1 -- 172.21.2.111:0/4080676960 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7faecc06ff70 con 0x7faed414b700 2021-04-03T03:08:45.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2384700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faeb8056f20 0x7faeb80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.706+0000 7faed2384700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faeb8056f20 0x7faeb80593e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7faed40af200 tx=0x7faec4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.710+0000 7faecb7fe700 1 -- 172.21.2.111:0/4080676960 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7faecc0451c0 con 0x7faed414b700 2021-04-03T03:08:45.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.798+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faed4057660 con 0x7faed414b700 2021-04-03T03:08:45.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faecb7fe700 1 -- 172.21.2.111:0/4080676960 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7faecc0487e0 con 0x7faed414b700 2021-04-03T03:08:45.810 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:45.811 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:45.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faeb8056f20 msgr2=0x7faeb80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faeb8056f20 0x7faeb80593e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7faed40af200 tx=0x7faec4015000).stop 2021-04-03T03:08:45.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed414b700 msgr2=0x7faed40644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed414b700 0x7faed40644a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7faecc0032f0 tx=0x7faecc0091a0).stop 2021-04-03T03:08:45.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 shutdown_connections 2021-04-03T03:08:45.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faed414b700 0x7faed40644a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faed40603f0 0x7faed4159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faeb8056f20 0x7faeb80593e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 --2- 172.21.2.111:0/4080676960 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faed40b12b0 0x7faed4062c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 >> 172.21.2.111:0/4080676960 conn(0x7faed40b61c0 msgr2=0x7faed40b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:45.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 shutdown_connections 2021-04-03T03:08:45.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.802+0000 7faed3386700 1 -- 172.21.2.111:0/4080676960 wait complete. 2021-04-03T03:08:45.818 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:45.819 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-04-03T03:08:45.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 Processor -- start 2021-04-03T03:08:45.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 -- start start 2021-04-03T03:08:45.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc0603f0 0x7f07bc0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07bc14b700 0x7f07bc0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f07bc0b12b0 0x7f07bc0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0b0f60 con 0x7f07bc14b700 2021-04-03T03:08:45.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0607d0 con 0x7f07bc0b12b0 2021-04-03T03:08:45.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c25af700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc0af400 con 0x7f07bc0603f0 2021-04-03T03:08:45.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07bc14b700 0x7f07bc0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07bc14b700 0x7f07bc0b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49308/0 (socket says 172.21.2.111:49308) 2021-04-03T03:08:45.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 -- 172.21.2.111:0/1838998025 learned_addr learned my addr 172.21.2.111:0/1838998025 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:45.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 -- 172.21.2.111:0/1838998025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc0603f0 msgr2=0x7f07bc0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:45.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 --2- 172.21.2.111:0/1838998025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc0603f0 0x7f07bc0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 -- 172.21.2.111:0/1838998025 >> v2:172.21.2.115:3300/0 conn(0x7f07bc0b12b0 msgr2=0x7f07bc0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:45.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 --2- 172.21.2.111:0/1838998025 >> v2:172.21.2.115:3300/0 conn(0x7f07bc0b12b0 0x7f07bc0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.866+0000 7f07c0dac700 1 -- 172.21.2.111:0/1838998025 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc0af6a0 con 0x7f07bc14b700 2021-04-03T03:08:45.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c0dac700 1 --2- 172.21.2.111:0/1838998025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc14b700 0x7f07bc0b0580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f07bc062740 tx=0x7f07ac006e90).ready entity=mon.0 client_cookie=7b1562ac05ae5037 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07abfff700 1 -- 172.21.2.111:0/1838998025 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07ac0098e0 con 0x7f07bc14b700 2021-04-03T03:08:45.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07abfff700 1 -- 172.21.2.111:0/1838998025 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac009a40 con 0x7f07bc14b700 2021-04-03T03:08:45.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07abfff700 1 -- 172.21.2.111:0/1838998025 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07ac009da0 con 0x7f07bc14b700 2021-04-03T03:08:45.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/1838998025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc14b700 msgr2=0x7f07bc0b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 --2- 172.21.2.111:0/1838998025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc14b700 0x7f07bc0b0580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f07bc062740 tx=0x7f07ac006e90).stop 2021-04-03T03:08:45.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/1838998025 shutdown_connections 2021-04-03T03:08:45.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 --2- 172.21.2.111:0/1838998025 >> v2:172.21.2.115:3300/0 conn(0x7f07bc0b12b0 0x7f07bc0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 --2- 172.21.2.111:0/1838998025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc0603f0 0x7f07bc0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 --2- 172.21.2.111:0/1838998025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc14b700 0x7f07bc0b0580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/1838998025 >> 172.21.2.111:0/1838998025 conn(0x7f07bc0b61c0 msgr2=0x7f07bc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:45.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/1838998025 shutdown_connections 2021-04-03T03:08:45.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/1838998025 wait complete. 2021-04-03T03:08:45.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 Processor -- start 2021-04-03T03:08:45.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- start start 2021-04-03T03:08:45.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc0603f0 0x7f07bc159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07bc0b12b0 0x7f07bc148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc14b700 0x7f07bc1455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc064d20 con 0x7f07bc0603f0 2021-04-03T03:08:45.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc064fc0 con 0x7f07bc0b12b0 2021-04-03T03:08:45.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc065260 con 0x7f07bc14b700 2021-04-03T03:08:45.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc0603f0 0x7f07bc159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc0603f0 0x7f07bc159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49310/0 (socket says 172.21.2.111:49310) 2021-04-03T03:08:45.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 -- 172.21.2.111:0/3522091264 learned_addr learned my addr 172.21.2.111:0/3522091264 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:45.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 -- 172.21.2.111:0/3522091264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc14b700 msgr2=0x7f07bc1455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc14b700 0x7f07bc1455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 -- 172.21.2.111:0/3522091264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07bc0b12b0 msgr2=0x7f07bc148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07bc0b12b0 0x7f07bc148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 -- 172.21.2.111:0/3522091264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc063b60 con 0x7f07bc0603f0 2021-04-03T03:08:45.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c15ad700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc0603f0 0x7f07bc159920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f07b8002f30 tx=0x7f07b8006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07aa7fc700 1 -- 172.21.2.111:0/3522091264 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07b800d840 con 0x7f07bc0603f0 2021-04-03T03:08:45.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07aa7fc700 1 -- 172.21.2.111:0/3522091264 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b8000e90 con 0x7f07bc0603f0 2021-04-03T03:08:45.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc149420 con 0x7f07bc0603f0 2021-04-03T03:08:45.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07aa7fc700 1 -- 172.21.2.111:0/3522091264 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07b800db90 con 0x7f07bc0603f0 2021-04-03T03:08:45.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc067750 con 0x7f07bc0603f0 2021-04-03T03:08:45.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.870+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc064210 con 0x7f07bc0603f0 2021-04-03T03:08:45.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.874+0000 7f07aa7fc700 1 -- 172.21.2.111:0/3522091264 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f07b8017030 con 0x7f07bc0603f0 2021-04-03T03:08:45.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.874+0000 7f07aa7fc700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f079c056ec0 0x7f079c059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:45.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.874+0000 7f07aa7fc700 1 -- 172.21.2.111:0/3522091264 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f07b806f7b0 con 0x7f07bc0603f0 2021-04-03T03:08:45.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.874+0000 7f07c0dac700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f079c056ec0 0x7f079c059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:45.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.874+0000 7f07c0dac700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f079c056ec0 0x7f079c059380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f07bc062740 tx=0x7f07ac015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:45.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.874+0000 7f07aa7fc700 1 -- 172.21.2.111:0/3522091264 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f07b8044a80 con 0x7f07bc0603f0 2021-04-03T03:08:45.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.966+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07bc057660 con 0x7f07bc0603f0 2021-04-03T03:08:45.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07aa7fc700 1 -- 172.21.2.111:0/3522091264 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f07b80480a0 con 0x7f07bc0603f0 2021-04-03T03:08:45.978 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:45.978 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:45.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f079c056ec0 msgr2=0x7f079c059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f079c056ec0 0x7f079c059380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f07bc062740 tx=0x7f07ac015000).stop 2021-04-03T03:08:45.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc0603f0 msgr2=0x7f07bc159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:45.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc0603f0 0x7f07bc159920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f07b8002f30 tx=0x7f07b8006ea0).stop 2021-04-03T03:08:45.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 shutdown_connections 2021-04-03T03:08:45.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07bc14b700 0x7f07bc1455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07bc0603f0 0x7f07bc159920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f079c056ec0 0x7f079c059380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:45.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 --2- 172.21.2.111:0/3522091264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07bc0b12b0 0x7f07bc148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:45.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 >> 172.21.2.111:0/3522091264 conn(0x7f07bc0b61c0 msgr2=0x7f07bc067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:45.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 shutdown_connections 2021-04-03T03:08:45.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:45.970+0000 7f07c25af700 1 -- 172.21.2.111:0/3522091264 wait complete. 2021-04-03T03:08:45.986 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:08:45.986 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:08:46.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.034+0000 7f808759e700 1 Processor -- start 2021-04-03T03:08:46.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.034+0000 7f808759e700 1 -- start start 2021-04-03T03:08:46.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.034+0000 7f808759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.034+0000 7f808759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8080041060 0x7f808003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.034+0000 7f808759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80800dc560 con 0x7f808003d560 2021-04-03T03:08:46.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.034+0000 7f808759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808003de60 con 0x7f8080041060 2021-04-03T03:08:46.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.034+0000 7f808759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800dc800 con 0x7f80800dbba0 2021-04-03T03:08:46.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:60898/0 (socket says 172.21.2.111:60898) 2021-04-03T03:08:46.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 -- 172.21.2.111:0/847506740 learned_addr learned my addr 172.21.2.111:0/847506740 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f8085d9b700 1 --2- 172.21.2.111:0/847506740 >> v2:172.21.2.115:3300/0 conn(0x7f8080041060 0x7f808003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 -- 172.21.2.111:0/847506740 >> v2:172.21.2.115:3300/0 conn(0x7f8080041060 msgr2=0x7f808003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 --2- 172.21.2.111:0/847506740 >> v2:172.21.2.115:3300/0 conn(0x7f8080041060 0x7f808003fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 -- 172.21.2.111:0/847506740 >> v1:172.21.2.115:6789/0 conn(0x7f808003d560 legacy=0x7f8080041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 -- 172.21.2.111:0/847506740 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808003da30 con 0x7f80800dbba0 2021-04-03T03:08:46.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808659c700 1 --2- 172.21.2.111:0/847506740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800dbf80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f807c002f30 tx=0x7f807c006e90).ready entity=mon.2 client_cookie=77f7e3d4073f1a95 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808559a700 1 -- 172.21.2.111:0/847506740 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f807c009850 con 0x7f80800dbba0 2021-04-03T03:08:46.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808559a700 1 -- 172.21.2.111:0/847506740 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0099b0 con 0x7f80800dbba0 2021-04-03T03:08:46.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808559a700 1 -- 172.21.2.111:0/847506740 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f807c009d10 con 0x7f80800dbba0 2021-04-03T03:08:46.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- 172.21.2.111:0/847506740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 msgr2=0x7f80800dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 --2- 172.21.2.111:0/847506740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800dbf80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f807c002f30 tx=0x7f807c006e90).stop 2021-04-03T03:08:46.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- 172.21.2.111:0/847506740 shutdown_connections 2021-04-03T03:08:46.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 --2- 172.21.2.111:0/847506740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800dbf80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 --2- 172.21.2.111:0/847506740 >> v2:172.21.2.115:3300/0 conn(0x7f8080041060 0x7f808003fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- 172.21.2.111:0/847506740 >> 172.21.2.111:0/847506740 conn(0x7f80800462b0 msgr2=0x7f80800466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- 172.21.2.111:0/847506740 shutdown_connections 2021-04-03T03:08:46.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- 172.21.2.111:0/847506740 wait complete. 2021-04-03T03:08:46.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 Processor -- start 2021-04-03T03:08:46.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- start start 2021-04-03T03:08:46.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f808003d560 0x7f80800de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8080041060 0x7f80800eac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800eb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800a1bc0 con 0x7f8080041060 2021-04-03T03:08:46.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800a1810 con 0x7f808003d560 2021-04-03T03:08:46.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f808759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800a19d0 con 0x7f80800dbba0 2021-04-03T03:08:46.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f8085d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8080041060 0x7f80800eac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f8085d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8080041060 0x7f80800eac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49324/0 (socket says 172.21.2.111:49324) 2021-04-03T03:08:46.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.038+0000 7f8085d9b700 1 -- 172.21.2.111:0/3879494339 learned_addr learned my addr 172.21.2.111:0/3879494339 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f8085d9b700 1 -- 172.21.2.111:0/3879494339 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 msgr2=0x7f80800eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f8085d9b700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f8085d9b700 1 -- 172.21.2.111:0/3879494339 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f808003d560 msgr2=0x7f80800de6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f8085d9b700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f808003d560 0x7f80800de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f8085d9b700 1 -- 172.21.2.111:0/3879494339 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808009fcb0 con 0x7f8080041060 2021-04-03T03:08:46.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f8085d9b700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8080041060 0x7f80800eac60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f807800c000 tx=0x7f8078006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f80777fe700 1 -- 172.21.2.111:0/3879494339 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f807800ca80 con 0x7f8080041060 2021-04-03T03:08:46.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f80777fe700 1 -- 172.21.2.111:0/3879494339 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807800cbe0 con 0x7f8080041060 2021-04-03T03:08:46.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f80777fe700 1 -- 172.21.2.111:0/3879494339 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80780123f0 con 0x7f8080041060 2021-04-03T03:08:46.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80800eb640 con 0x7f8080041060 2021-04-03T03:08:46.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808009f6b0 con 0x7f8080041060 2021-04-03T03:08:46.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80800a1dc0 con 0x7f8080041060 2021-04-03T03:08:46.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f80777fe700 1 -- 172.21.2.111:0/3879494339 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8078012550 con 0x7f8080041060 2021-04-03T03:08:46.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f80777fe700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8064057050 0x7f8064059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f80777fe700 1 -- 172.21.2.111:0/3879494339 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f80780706f0 con 0x7f8080041060 2021-04-03T03:08:46.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f808659c700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8064057050 0x7f8064059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.042+0000 7f808659c700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8064057050 0x7f8064059510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f807c002f30 tx=0x7f807c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.046+0000 7f80777fe700 1 -- 172.21.2.111:0/3879494339 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f80800a1dc0 con 0x7f8080041060 2021-04-03T03:08:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f80800a1dc0 con 0x7f8080041060 2021-04-03T03:08:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f80777fe700 1 -- 172.21.2.111:0/3879494339 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f80800a1dc0 con 0x7f8080041060 2021-04-03T03:08:46.148 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:08:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8064057050 msgr2=0x7f8064059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8064057050 0x7f8064059510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f807c002f30 tx=0x7f807c015000).stop 2021-04-03T03:08:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8080041060 msgr2=0x7f80800eac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8080041060 0x7f80800eac60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f807800c000 tx=0x7f8078006dd0).stop 2021-04-03T03:08:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 shutdown_connections 2021-04-03T03:08:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f80800dbba0 0x7f80800eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8064057050 0x7f8064059510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f808003d560 0x7f80800de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 --2- 172.21.2.111:0/3879494339 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8080041060 0x7f80800eac60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 >> 172.21.2.111:0/3879494339 conn(0x7f80800462b0 msgr2=0x7f8080041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 shutdown_connections 2021-04-03T03:08:46.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.138+0000 7f808759e700 1 -- 172.21.2.111:0/3879494339 wait complete. 2021-04-03T03:08:46.155 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:08:46.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f737cec8700 1 Processor -- start 2021-04-03T03:08:46.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f737cec8700 1 -- start start 2021-04-03T03:08:46.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f737cec8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7370039dd0 0x7f73700d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f737cec8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f737003fc60 0x7f7370005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f737cec8700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73700d1c70 con 0x7f737003f880 2021-04-03T03:08:46.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f737cec8700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700d2f00 con 0x7f7370039dd0 2021-04-03T03:08:46.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f737cec8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73700400d0 con 0x7f737003fc60 2021-04-03T03:08:46.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f73777fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f737003f880 0x7f73700ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50124/0 (socket says 172.21.2.111:50124) 2021-04-03T03:08:46.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f73777fe700 1 -- 172.21.2.111:0/3655830310 learned_addr learned my addr 172.21.2.111:0/3655830310 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:46.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f7376ffd700 1 --2- 172.21.2.111:0/3655830310 >> v2:172.21.2.111:3300/0 conn(0x7f7370039dd0 0x7f73700d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f7377fff700 1 --2- 172.21.2.111:0/3655830310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f737003fc60 0x7f7370005680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f7376ffd700 1 -- 172.21.2.111:0/3655830310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f737003fc60 msgr2=0x7f7370005680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f7376ffd700 1 --2- 172.21.2.111:0/3655830310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f737003fc60 0x7f7370005680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f7376ffd700 1 -- 172.21.2.111:0/3655830310 >> v1:172.21.2.111:6789/0 conn(0x7f737003f880 legacy=0x7f73700ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f7376ffd700 1 -- 172.21.2.111:0/3655830310 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700d1ea0 con 0x7f7370039dd0 2021-04-03T03:08:46.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f7376ffd700 1 --2- 172.21.2.111:0/3655830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7370039dd0 0x7f73700d2880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f736c002f30 tx=0x7f736c006dd0).ready entity=mon.0 client_cookie=4068b68fb3b17ca5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.202+0000 7f73767fc700 1 -- 172.21.2.111:0/3655830310 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f736c0097f0 con 0x7f7370039dd0 2021-04-03T03:08:46.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f73767fc700 1 -- 172.21.2.111:0/3655830310 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c009950 con 0x7f7370039dd0 2021-04-03T03:08:46.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f73767fc700 1 -- 172.21.2.111:0/3655830310 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f736c009cb0 con 0x7f7370039dd0 2021-04-03T03:08:46.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/3655830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7370039dd0 msgr2=0x7f73700d2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 --2- 172.21.2.111:0/3655830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7370039dd0 0x7f73700d2880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f736c002f30 tx=0x7f736c006dd0).stop 2021-04-03T03:08:46.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/3655830310 shutdown_connections 2021-04-03T03:08:46.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 --2- 172.21.2.111:0/3655830310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f737003fc60 0x7f7370005680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 --2- 172.21.2.111:0/3655830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7370039dd0 0x7f73700d2880 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/3655830310 >> 172.21.2.111:0/3655830310 conn(0x7f73700462b0 msgr2=0x7f73700466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/3655830310 shutdown_connections 2021-04-03T03:08:46.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/3655830310 wait complete. 2021-04-03T03:08:46.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 Processor -- start 2021-04-03T03:08:46.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- start start 2021-04-03T03:08:46.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7370039dd0 0x7f73700cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f737003f880 0x7f737003e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f737003fc60 0x7f73700d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737003efc0 con 0x7f737003f880 2021-04-03T03:08:46.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737003dd20 con 0x7f737003fc60 2021-04-03T03:08:46.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737003f1b0 con 0x7f7370039dd0 2021-04-03T03:08:46.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f737003f880 0x7f737003e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f737003f880 0x7f737003e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49338/0 (socket says 172.21.2.111:49338) 2021-04-03T03:08:46.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 -- 172.21.2.111:0/452376745 learned_addr learned my addr 172.21.2.111:0/452376745 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 -- 172.21.2.111:0/452376745 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7370039dd0 msgr2=0x7f73700cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7370039dd0 0x7f73700cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 -- 172.21.2.111:0/452376745 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f737003fc60 msgr2=0x7f73700d59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f737003fc60 0x7f73700d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 -- 172.21.2.111:0/452376745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737003e9e0 con 0x7f737003f880 2021-04-03T03:08:46.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7376ffd700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f737003f880 0x7f737003e4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f737003deb0 tx=0x7f736c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7374ff9700 1 -- 172.21.2.111:0/452376745 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f736c011c60 con 0x7f737003f880 2021-04-03T03:08:46.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7374ff9700 1 -- 172.21.2.111:0/452376745 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c011dc0 con 0x7f737003f880 2021-04-03T03:08:46.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7374ff9700 1 -- 172.21.2.111:0/452376745 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f736c01e650 con 0x7f737003f880 2021-04-03T03:08:46.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73700d3180 con 0x7f737003f880 2021-04-03T03:08:46.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73700d6070 con 0x7f737003f880 2021-04-03T03:08:46.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7370004db0 con 0x7f737003f880 2021-04-03T03:08:46.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7374ff9700 1 -- 172.21.2.111:0/452376745 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f736c025030 con 0x7f737003f880 2021-04-03T03:08:46.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7374ff9700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7358056f70 0x7f7358059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f7374ff9700 1 -- 172.21.2.111:0/452376745 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f736c0717b0 con 0x7f737003f880 2021-04-03T03:08:46.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.206+0000 7f73777fe700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7358056f70 0x7f7358059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.210+0000 7f73777fe700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7358056f70 0x7f7358059430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f737000c350 tx=0x7f73680077b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.210+0000 7f7374ff9700 1 -- 172.21.2.111:0/452376745 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f736c046ac0 con 0x7f737003f880 2021-04-03T03:08:46.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7370041f20 con 0x7f737003f880 2021-04-03T03:08:46.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f7374ff9700 1 -- 172.21.2.111:0/452376745 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f736c04a0a0 con 0x7f737003f880 2021-04-03T03:08:46.312 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:46.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7358056f70 msgr2=0x7f7358059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7358056f70 0x7f7358059430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f737000c350 tx=0x7f73680077b0).stop 2021-04-03T03:08:46.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f737003f880 msgr2=0x7f737003e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f737003f880 0x7f737003e4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f737003deb0 tx=0x7f736c015000).stop 2021-04-03T03:08:46.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 shutdown_connections 2021-04-03T03:08:46.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7358056f70 0x7f7358059430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f737003fc60 0x7f73700d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7370039dd0 0x7f73700cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 --2- 172.21.2.111:0/452376745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f737003f880 0x7f737003e4f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 >> 172.21.2.111:0/452376745 conn(0x7f73700462b0 msgr2=0x7f73700b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 shutdown_connections 2021-04-03T03:08:46.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.303+0000 7f737cec8700 1 -- 172.21.2.111:0/452376745 wait complete. 2021-04-03T03:08:46.320 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:08:46.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 Processor -- start 2021-04-03T03:08:46.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 -- start start 2021-04-03T03:08:46.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3588041060 0x7f35880dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f358803d560 0x7f3588041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35880dc560 con 0x7f35880dbba0 2021-04-03T03:08:46.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358803de60 con 0x7f358803d560 2021-04-03T03:08:46.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880dc800 con 0x7f3588041060 2021-04-03T03:08:46.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3595529700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f35880dbba0 0x7f358803fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50138/0 (socket says 172.21.2.111:50138) 2021-04-03T03:08:46.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3595529700 1 -- 172.21.2.111:0/782733025 learned_addr learned my addr 172.21.2.111:0/782733025 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:46.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3588041060 0x7f35880dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3595d2a700 1 --2- 172.21.2.111:0/782733025 >> v2:172.21.2.115:3300/0 conn(0x7f358803d560 0x7f3588041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987074499 0 0) 0x7f35880dc560 con 0x7f35880dbba0 2021-04-03T03:08:46.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3578003240 con 0x7f35880dbba0 2021-04-03T03:08:46.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2669395921 0 0) 0x7f3578003240 con 0x7f35880dbba0 2021-04-03T03:08:46.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3588041060 msgr2=0x7f35880dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3588041060 0x7f35880dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 >> v2:172.21.2.115:3300/0 conn(0x7f358803d560 msgr2=0x7f3588041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 --2- 172.21.2.111:0/782733025 >> v2:172.21.2.115:3300/0 conn(0x7f358803d560 0x7f3588041650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358803da30 con 0x7f35880dbba0 2021-04-03T03:08:46.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f358c0016d0 con 0x7f35880dbba0 2021-04-03T03:08:46.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3595d2a700 1 --2- 172.21.2.111:0/782733025 >> v2:172.21.2.115:3300/0 conn(0x7f358803d560 0x7f3588041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:08:46.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 >> v1:172.21.2.111:6789/0 conn(0x7f35880dbba0 legacy=0x7f358803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3588041060 0x7f35780037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3578003d10 0x7f35780044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35780049d0 0x7f3578004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358803de60 con 0x7f3578003d10 2021-04-03T03:08:46.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880dc800 con 0x7f35780049d0 2021-04-03T03:08:46.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35780042d0 con 0x7f3588041060 2021-04-03T03:08:46.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3578003d10 0x7f35780044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3595d2a700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35780049d0 0x7f3578004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 -- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3588041060 msgr2=0x7f35780037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3588041060 0x7f35780037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 -- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35780049d0 msgr2=0x7f3578004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35780049d0 0x7f3578004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 -- 172.21.2.111:0/782733025 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578003240 con 0x7f3578003d10 2021-04-03T03:08:46.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3594d28700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3578003d10 0x7f35780044e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f3590082000 tx=0x7f359005ad70).ready entity=mon.0 client_cookie=355d975d51835391 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359005df60 con 0x7f3578003d10 2021-04-03T03:08:46.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35900699f0 con 0x7f3578003d10 2021-04-03T03:08:46.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3583fff700 1 -- 172.21.2.111:0/782733025 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359006b610 con 0x7f3578003d10 2021-04-03T03:08:46.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 -- 172.21.2.111:0/782733025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3578003d10 msgr2=0x7f35780044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f359652b700 1 --2- 172.21.2.111:0/782733025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3578003d10 0x7f35780044e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f3590082000 tx=0x7f359005ad70).stop 2021-04-03T03:08:46.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.371+0000 7f3595d2a700 1 -- 172.21.2.111:0/782733025 reap_dead start 2021-04-03T03:08:46.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- 172.21.2.111:0/782733025 shutdown_connections 2021-04-03T03:08:46.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- 172.21.2.111:0/782733025 >> 172.21.2.111:0/782733025 conn(0x7f35880462b0 msgr2=0x7f35880466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- 172.21.2.111:0/782733025 shutdown_connections 2021-04-03T03:08:46.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- 172.21.2.111:0/782733025 wait complete. 2021-04-03T03:08:46.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 Processor -- start 2021-04-03T03:08:46.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- start start 2021-04-03T03:08:46.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3588044630 0x7f35880def30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f358803acb0 0x7f358803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f358803b620 0x7f35880ea530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880e6fa0 con 0x7f3588044630 2021-04-03T03:08:46.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880de490 con 0x7f358803acb0 2021-04-03T03:08:46.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358803bab0 con 0x7f358803b620 2021-04-03T03:08:46.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3588044630 0x7f35880def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3588044630 0x7f35880def30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49358/0 (socket says 172.21.2.111:49358) 2021-04-03T03:08:46.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 -- 172.21.2.111:0/3848747420 learned_addr learned my addr 172.21.2.111:0/3848747420 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 -- 172.21.2.111:0/3848747420 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f358803b620 msgr2=0x7f35880ea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595d2a700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f358803b620 0x7f35880ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f358803b620 0x7f35880ea530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 -- 172.21.2.111:0/3848747420 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f358803acb0 msgr2=0x7f358803b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f358803acb0 0x7f358803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 -- 172.21.2.111:0/3848747420 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588044a10 con 0x7f3588044630 2021-04-03T03:08:46.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3595529700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3588044630 0x7f35880def30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f358c0013e0 tx=0x7f358c001a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f35827fc700 1 -- 172.21.2.111:0/3848747420 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f358c007170 con 0x7f3588044630 2021-04-03T03:08:46.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f35827fc700 1 -- 172.21.2.111:0/3848747420 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c0072d0 con 0x7f3588044630 2021-04-03T03:08:46.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f35827fc700 1 -- 172.21.2.111:0/3848747420 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f358c0049c0 con 0x7f3588044630 2021-04-03T03:08:46.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3588046040 con 0x7f3588044630 2021-04-03T03:08:46.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35880de960 con 0x7f3588044630 2021-04-03T03:08:46.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588004db0 con 0x7f3588044630 2021-04-03T03:08:46.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f35827fc700 1 -- 172.21.2.111:0/3848747420 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f358c010070 con 0x7f3588044630 2021-04-03T03:08:46.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f35827fc700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3570056ea0 0x7f3570059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f35827fc700 1 -- 172.21.2.111:0/3848747420 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f358c06fac0 con 0x7f3588044630 2021-04-03T03:08:46.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3594d28700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3570056ea0 0x7f3570059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.375+0000 7f3594d28700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3570056ea0 0x7f3570059360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f3590082000 tx=0x7f3590065000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.379+0000 7f35827fc700 1 -- 172.21.2.111:0/3848747420 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f358c044ea0 con 0x7f3588044630 2021-04-03T03:08:46.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f358803c380 con 0x7f3588044630 2021-04-03T03:08:46.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f35827fc700 1 -- 172.21.2.111:0/3848747420 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f358c0484c0 con 0x7f3588044630 2021-04-03T03:08:46.483 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:46.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3570056ea0 msgr2=0x7f3570059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3570056ea0 0x7f3570059360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f3590082000 tx=0x7f3590065000).stop 2021-04-03T03:08:46.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3588044630 msgr2=0x7f35880def30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3588044630 0x7f35880def30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f358c0013e0 tx=0x7f358c001a20).stop 2021-04-03T03:08:46.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 shutdown_connections 2021-04-03T03:08:46.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f358803b620 0x7f35880ea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3588044630 0x7f35880def30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3570056ea0 0x7f3570059360 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 --2- 172.21.2.111:0/3848747420 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f358803acb0 0x7f358803b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 >> 172.21.2.111:0/3848747420 conn(0x7f35880462b0 msgr2=0x7f3588041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 shutdown_connections 2021-04-03T03:08:46.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.475+0000 7f359652b700 1 -- 172.21.2.111:0/3848747420 wait complete. 2021-04-03T03:08:46.491 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:08:46.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.539+0000 7f15a202c700 1 Processor -- start 2021-04-03T03:08:46.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.539+0000 7f15a202c700 1 -- start start 2021-04-03T03:08:46.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f159c0603f0 0x7f159c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0add50 0x7f159c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159c0607d0 con 0x7f159c0b17b0 2021-04-03T03:08:46.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c1451c0 con 0x7f159c0603f0 2021-04-03T03:08:46.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c14b090 con 0x7f159c0add50 2021-04-03T03:08:46.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a102a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f159c0603f0 0x7f159c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a0829700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0add50 0x7f159c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a0829700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0add50 0x7f159c144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:60946/0 (socket says 172.21.2.111:60946) 2021-04-03T03:08:46.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a102a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f159c0603f0 0x7f159c1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:39074/0 (socket says 172.21.2.111:39074) 2021-04-03T03:08:46.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a0829700 1 -- 172.21.2.111:0/3585731408 learned_addr learned my addr 172.21.2.111:0/3585731408 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a182b700 1 --1- 172.21.2.111:0/3585731408 >> v1:172.21.2.115:6789/0 conn(0x7f159c0b17b0 0x7f159c147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34462/0 (socket says 172.21.2.111:34462) 2021-04-03T03:08:46.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a102a700 1 -- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0add50 msgr2=0x7f159c144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a102a700 1 --2- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0add50 0x7f159c144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a102a700 1 -- 172.21.2.111:0/3585731408 >> v1:172.21.2.115:6789/0 conn(0x7f159c0b17b0 legacy=0x7f159c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a102a700 1 -- 172.21.2.111:0/3585731408 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c14b330 con 0x7f159c0603f0 2021-04-03T03:08:46.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a0829700 1 --2- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0add50 0x7f159c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:46.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a102a700 1 --2- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0603f0 0x7f159c1447e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1594001450 tx=0x7f1590006dd0).ready entity=mon.1 client_cookie=32831d883203d86c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f158ffff700 1 -- 172.21.2.111:0/3585731408 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15900095d0 con 0x7f159c0603f0 2021-04-03T03:08:46.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f158ffff700 1 -- 172.21.2.111:0/3585731408 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590009730 con 0x7f159c0603f0 2021-04-03T03:08:46.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f158ffff700 1 -- 172.21.2.111:0/3585731408 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1590009a90 con 0x7f159c0603f0 2021-04-03T03:08:46.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 -- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0603f0 msgr2=0x7f159c1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 --2- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0603f0 0x7f159c1447e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1594001450 tx=0x7f1590006dd0).stop 2021-04-03T03:08:46.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 -- 172.21.2.111:0/3585731408 shutdown_connections 2021-04-03T03:08:46.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 --2- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0603f0 0x7f159c1447e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 --2- 172.21.2.111:0/3585731408 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0add50 0x7f159c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.543+0000 7f15a202c700 1 -- 172.21.2.111:0/3585731408 >> 172.21.2.111:0/3585731408 conn(0x7f159c0b61c0 msgr2=0x7f159c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- 172.21.2.111:0/3585731408 shutdown_connections 2021-04-03T03:08:46.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- 172.21.2.111:0/3585731408 wait complete. 2021-04-03T03:08:46.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 Processor -- start 2021-04-03T03:08:46.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- start start 2021-04-03T03:08:46.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f159c0603f0 0x7f159c144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0add50 0x7f159c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0b17b0 0x7f159c15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c157ad0 con 0x7f159c0603f0 2021-04-03T03:08:46.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c0aa510 con 0x7f159c0add50 2021-04-03T03:08:46.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c15acd0 con 0x7f159c0b17b0 2021-04-03T03:08:46.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f159c0603f0 0x7f159c144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f159c0603f0 0x7f159c144590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49368/0 (socket says 172.21.2.111:49368) 2021-04-03T03:08:46.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 -- 172.21.2.111:0/3392753965 learned_addr learned my addr 172.21.2.111:0/3392753965 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a0829700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0add50 0x7f159c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 -- 172.21.2.111:0/3392753965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0b17b0 msgr2=0x7f159c15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0b17b0 0x7f159c15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 -- 172.21.2.111:0/3392753965 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0add50 msgr2=0x7f159c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0add50 0x7f159c15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 -- 172.21.2.111:0/3392753965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c0a81b0 con 0x7f159c0603f0 2021-04-03T03:08:46.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a102a700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f159c0603f0 0x7f159c144590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f1594001450 tx=0x7f1590015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f158e7fc700 1 -- 172.21.2.111:0/3392753965 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f159001e460 con 0x7f159c0603f0 2021-04-03T03:08:46.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f158e7fc700 1 -- 172.21.2.111:0/3392753965 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159001e5c0 con 0x7f159c0603f0 2021-04-03T03:08:46.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f158e7fc700 1 -- 172.21.2.111:0/3392753965 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f159001e920 con 0x7f159c0603f0 2021-04-03T03:08:46.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c0aa740 con 0x7f159c0603f0 2021-04-03T03:08:46.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c159e20 con 0x7f159c0603f0 2021-04-03T03:08:46.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c057660 con 0x7f159c0603f0 2021-04-03T03:08:46.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f158e7fc700 1 -- 172.21.2.111:0/3392753965 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1590025030 con 0x7f159c0603f0 2021-04-03T03:08:46.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f158e7fc700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f157c057000 0x7f157c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f158e7fc700 1 -- 172.21.2.111:0/3392753965 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f15900716c0 con 0x7f159c0603f0 2021-04-03T03:08:46.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a0829700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f157c057000 0x7f157c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.547+0000 7f15a0829700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f157c057000 0x7f157c0594c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f159800b000 tx=0x7f1598008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.551+0000 7f158e7fc700 1 -- 172.21.2.111:0/3392753965 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1590046990 con 0x7f159c0603f0 2021-04-03T03:08:46.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.643+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f159c148850 con 0x7f159c0603f0 2021-04-03T03:08:46.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.643+0000 7f158e7fc700 1 -- 172.21.2.111:0/3392753965 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1590049fb0 con 0x7f159c0603f0 2021-04-03T03:08:46.653 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:46.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f157c057000 msgr2=0x7f157c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f157c057000 0x7f157c0594c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f159800b000 tx=0x7f1598008040).stop 2021-04-03T03:08:46.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f159c0603f0 msgr2=0x7f159c144590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f159c0603f0 0x7f159c144590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f1594001450 tx=0x7f1590015000).stop 2021-04-03T03:08:46.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 shutdown_connections 2021-04-03T03:08:46.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f159c0b17b0 0x7f159c15a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f159c0603f0 0x7f159c144590 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f157c057000 0x7f157c0594c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 --2- 172.21.2.111:0/3392753965 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f159c0add50 0x7f159c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 >> 172.21.2.111:0/3392753965 conn(0x7f159c0b61c0 msgr2=0x7f159c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 shutdown_connections 2021-04-03T03:08:46.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.647+0000 7f15a202c700 1 -- 172.21.2.111:0/3392753965 wait complete. 2021-04-03T03:08:46.661 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-04-03T03:08:46.661 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2021-04-03T03:08:46.662 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2021-04-03T03:08:46.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 Processor -- start 2021-04-03T03:08:46.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 -- start start 2021-04-03T03:08:46.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff5dc14b700 0x7ff5dc0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc0607d0 con 0x7ff5dc0b12b0 2021-04-03T03:08:46.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5dc0af400 con 0x7ff5dc14b700 2021-04-03T03:08:46.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0af6a0 con 0x7ff5dc0603f0 2021-04-03T03:08:46.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5da384700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5dab85700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7ff5dc0b12b0 0x7ff5dc1465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34476/0 (socket says 172.21.2.111:34476) 2021-04-03T03:08:46.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5dab85700 1 -- 172.21.2.111:0/1372451696 learned_addr learned my addr 172.21.2.111:0/1372451696 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:46.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9b83700 1 --2- 172.21.2.111:0/1372451696 >> v2:172.21.2.115:3300/0 conn(0x7ff5dc14b700 0x7ff5dc0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9b83700 1 -- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 msgr2=0x7ff5dc0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9b83700 1 --2- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9b83700 1 -- 172.21.2.111:0/1372451696 >> v1:172.21.2.115:6789/0 conn(0x7ff5dc0b12b0 legacy=0x7ff5dc1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9b83700 1 -- 172.21.2.111:0/1372451696 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0afb40 con 0x7ff5dc14b700 2021-04-03T03:08:46.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5da384700 1 --2- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:46.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9b83700 1 --2- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc14b700 0x7ff5dc0b0580 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff5d0002f30 tx=0x7ff5d0006dd0).ready entity=mon.1 client_cookie=f6929204471d65b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9382700 1 -- 172.21.2.111:0/1372451696 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5d0009760 con 0x7ff5dc14b700 2021-04-03T03:08:46.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9382700 1 -- 172.21.2.111:0/1372451696 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d00098c0 con 0x7ff5dc14b700 2021-04-03T03:08:46.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5d9382700 1 -- 172.21.2.111:0/1372451696 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5d0009c20 con 0x7ff5dc14b700 2021-04-03T03:08:46.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 -- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc14b700 msgr2=0x7ff5dc0b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 --2- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc14b700 0x7ff5dc0b0580 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff5d0002f30 tx=0x7ff5d0006dd0).stop 2021-04-03T03:08:46.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 -- 172.21.2.111:0/1372451696 shutdown_connections 2021-04-03T03:08:46.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 --2- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 --2- 172.21.2.111:0/1372451696 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc14b700 0x7ff5dc0b0580 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.711+0000 7ff5db386700 1 -- 172.21.2.111:0/1372451696 >> 172.21.2.111:0/1372451696 conn(0x7ff5dc0b61c0 msgr2=0x7ff5dc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- 172.21.2.111:0/1372451696 shutdown_connections 2021-04-03T03:08:46.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- 172.21.2.111:0/1372451696 wait complete. 2021-04-03T03:08:46.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 Processor -- start 2021-04-03T03:08:46.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- start start 2021-04-03T03:08:46.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc064bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc0b12b0 0x7ff5dc062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff5dc14b700 0x7ff5dc159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0641f0 con 0x7ff5dc14b700 2021-04-03T03:08:46.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc064490 con 0x7ff5dc0b12b0 2021-04-03T03:08:46.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc064730 con 0x7ff5dc0603f0 2021-04-03T03:08:46.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff5dc14b700 0x7ff5dc159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff5dc14b700 0x7ff5dc159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49386/0 (socket says 172.21.2.111:49386) 2021-04-03T03:08:46.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 -- 172.21.2.111:0/3621481698 learned_addr learned my addr 172.21.2.111:0/3621481698 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5da384700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc064bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5d9b83700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc0b12b0 0x7ff5dc062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 -- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 msgr2=0x7ff5dc064bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc064bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 -- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc0b12b0 msgr2=0x7ff5dc062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc0b12b0 0x7ff5dc062c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 -- 172.21.2.111:0/3621481698 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0a9160 con 0x7ff5dc14b700 2021-04-03T03:08:46.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5d9b83700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc0b12b0 0x7ff5dc062c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:46.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5dab85700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff5dc14b700 0x7ff5dc159910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ff5dc0628d0 tx=0x7ff5d4007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5cb7fe700 1 -- 172.21.2.111:0/3621481698 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5d4005b80 con 0x7ff5dc14b700 2021-04-03T03:08:46.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5cb7fe700 1 -- 172.21.2.111:0/3621481698 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d4005ce0 con 0x7ff5dc14b700 2021-04-03T03:08:46.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5cb7fe700 1 -- 172.21.2.111:0/3621481698 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5d4007300 con 0x7ff5dc14b700 2021-04-03T03:08:46.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc0a9cb0 con 0x7ff5dc14b700 2021-04-03T03:08:46.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc0aa2c0 con 0x7ff5dc14b700 2021-04-03T03:08:46.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5dc0648f0 con 0x7ff5dc14b700 2021-04-03T03:08:46.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5cb7fe700 1 -- 172.21.2.111:0/3621481698 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff5d4010030 con 0x7ff5dc14b700 2021-04-03T03:08:46.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5cb7fe700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff5b8056f20 0x7ff5b80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5cb7fe700 1 -- 172.21.2.111:0/3621481698 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff5d4070140 con 0x7ff5dc14b700 2021-04-03T03:08:46.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5da384700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff5b8056f20 0x7ff5b80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.715+0000 7ff5da384700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff5b8056f20 0x7ff5b80593e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff5cc00b000 tx=0x7ff5cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.719+0000 7ff5cb7fe700 1 -- 172.21.2.111:0/3621481698 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff5d4045520 con 0x7ff5dc14b700 2021-04-03T03:08:46.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.831+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff5dc057660 con 0x7ff5dc14b700 2021-04-03T03:08:46.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5cb7fe700 1 -- 172.21.2.111:0/3621481698 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+136588 (secure 0 0 0) 0x7ff5d4048b40 con 0x7ff5dc14b700 2021-04-03T03:08:46.845 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:46.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff5b8056f20 msgr2=0x7ff5b80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff5b8056f20 0x7ff5b80593e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff5cc00b000 tx=0x7ff5cc008040).stop 2021-04-03T03:08:46.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff5dc14b700 msgr2=0x7ff5dc159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff5dc14b700 0x7ff5dc159910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ff5dc0628d0 tx=0x7ff5d4007f20).stop 2021-04-03T03:08:46.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 shutdown_connections 2021-04-03T03:08:46.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff5dc14b700 0x7ff5dc159910 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff5dc0603f0 0x7ff5dc064bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff5b8056f20 0x7ff5b80593e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 --2- 172.21.2.111:0/3621481698 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff5dc0b12b0 0x7ff5dc062c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 >> 172.21.2.111:0/3621481698 conn(0x7ff5dc0b61c0 msgr2=0x7ff5dc0ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 shutdown_connections 2021-04-03T03:08:46.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.835+0000 7ff5db386700 1 -- 172.21.2.111:0/3621481698 wait complete. 2021-04-03T03:08:46.851 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":4,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6800","nonce":15722},{"type":"v1","addr":"172.21.2.111:6801","nonce":15722}]},"active_addr":"172.21.2.111:6801/15722","active_change":"2021-04-03T03:08:29.699748+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.2.111:0","nonce":3743792647}]},{"addrvec":[{"type":"v2","addr":"172.21.2.111:0","nonce":2431150648}]},{"addrvec":[{"type":"v2","addr":"172.21.2.111:0","nonce":2298262849}]}]}} 2021-04-03T03:08:46.855 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2021-04-03T03:08:46.855 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2021-04-03T03:08:46.856 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:46.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 Processor -- start 2021-04-03T03:08:46.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- start start 2021-04-03T03:08:46.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 0x7fd4380b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4380607d0 con 0x7fd4380603f0 2021-04-03T03:08:46.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4380af400 con 0x7fd4380b0c90 2021-04-03T03:08:46.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380af6a0 con 0x7fd43814b700 2021-04-03T03:08:46.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd4380603f0 0x7fd4380b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50182/0 (socket says 172.21.2.111:50182) 2021-04-03T03:08:46.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 -- 172.21.2.111:0/978864528 learned_addr learned my addr 172.21.2.111:0/978864528 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:46.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343334459 0 0) 0x7fd4380607d0 con 0x7fd4380603f0 2021-04-03T03:08:46.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd420003240 con 0x7fd4380603f0 2021-04-03T03:08:46.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43d5dd700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 0x7fd4380b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2133705219 0 0) 0x7fd420003240 con 0x7fd4380603f0 2021-04-03T03:08:46.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 msgr2=0x7fd4380b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 0x7fd4380b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 >> v1:172.21.2.115:6789/0 conn(0x7fd4380b0c90 legacy=0x7fd438147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380afb40 con 0x7fd4380603f0 2021-04-03T03:08:46.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fd42c0016d0 con 0x7fd4380603f0 2021-04-03T03:08:46.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 >> v1:172.21.2.111:6789/0 conn(0x7fd4380603f0 legacy=0x7fd4380b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4200037e0 0x7fd420003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4200044e0 0x7fd420004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 0x7fd420004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380af6a0 con 0x7fd4200037e0 2021-04-03T03:08:46.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420005340 con 0x7fd4200044e0 2021-04-03T03:08:46.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42000b730 con 0x7fd43814b700 2021-04-03T03:08:46.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4200037e0 0x7fd420003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 -- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 msgr2=0x7fd420004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43e5df700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 0x7fd420004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd43814b700 0x7fd420004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:46.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 -- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4200044e0 msgr2=0x7fd420004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:46.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4200044e0 0x7fd420004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 -- 172.21.2.111:0/978864528 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420003240 con 0x7fd4200037e0 2021-04-03T03:08:46.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ddde700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4200037e0 0x7fd420003be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd420004450 tx=0x7fd4380af590).ready entity=mon.0 client_cookie=8145e930a48b89aa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd42c00cea0 con 0x7fd4200037e0 2021-04-03T03:08:46.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c0070d0 con 0x7fd4200037e0 2021-04-03T03:08:46.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43cddc700 1 -- 172.21.2.111:0/978864528 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd42c007430 con 0x7fd4200037e0 2021-04-03T03:08:46.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- 172.21.2.111:0/978864528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4200037e0 msgr2=0x7fd420003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 --2- 172.21.2.111:0/978864528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4200037e0 0x7fd420003be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd420004450 tx=0x7fd4380af590).stop 2021-04-03T03:08:46.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43e5df700 1 -- 172.21.2.111:0/978864528 reap_dead start 2021-04-03T03:08:46.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- 172.21.2.111:0/978864528 shutdown_connections 2021-04-03T03:08:46.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- 172.21.2.111:0/978864528 >> 172.21.2.111:0/978864528 conn(0x7fd4380b61c0 msgr2=0x7fd43805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:46.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- 172.21.2.111:0/978864528 shutdown_connections 2021-04-03T03:08:46.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.907+0000 7fd43ede0700 1 -- 172.21.2.111:0/978864528 wait complete. 2021-04-03T03:08:46.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 Processor -- start 2021-04-03T03:08:46.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 -- start start 2021-04-03T03:08:46.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4380b4900 0x7fd4381596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4380ab140 0x7fd4380628b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd438064c30 0x7fd4380650b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd438063500 con 0x7fd4380b4900 2021-04-03T03:08:46.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd438062da0 con 0x7fd438064c30 2021-04-03T03:08:46.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380647b0 con 0x7fd4380ab140 2021-04-03T03:08:46.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4380b4900 0x7fd4381596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4380b4900 0x7fd4381596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49404/0 (socket says 172.21.2.111:49404) 2021-04-03T03:08:46.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 -- 172.21.2.111:0/1140522534 learned_addr learned my addr 172.21.2.111:0/1140522534 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:46.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 -- 172.21.2.111:0/1140522534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4380ab140 msgr2=0x7fd4380628b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4380ab140 0x7fd4380628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 -- 172.21.2.111:0/1140522534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd438064c30 msgr2=0x7fd4380650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:46.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd438064c30 0x7fd4380650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:46.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 -- 172.21.2.111:0/1140522534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380b4ce0 con 0x7fd4380b4900 2021-04-03T03:08:46.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ddde700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4380b4900 0x7fd4381596a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd438062ee0 tx=0x7fd42c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd42affd700 1 -- 172.21.2.111:0/1140522534 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd42c0063f0 con 0x7fd4380b4900 2021-04-03T03:08:46.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd42affd700 1 -- 172.21.2.111:0/1140522534 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c006990 con 0x7fd4380b4900 2021-04-03T03:08:46.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4380655a0 con 0x7fd4380b4900 2021-04-03T03:08:46.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd42affd700 1 -- 172.21.2.111:0/1140522534 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd42c006cf0 con 0x7fd4380b4900 2021-04-03T03:08:46.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd438063260 con 0x7fd4380b4900 2021-04-03T03:08:46.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd438064970 con 0x7fd4380b4900 2021-04-03T03:08:46.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd42affd700 1 -- 172.21.2.111:0/1140522534 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd42c025070 con 0x7fd4380b4900 2021-04-03T03:08:46.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd42affd700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd418056fe0 0x7fd4180594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:46.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd42affd700 1 -- 172.21.2.111:0/1140522534 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fd42c0707e0 con 0x7fd4380b4900 2021-04-03T03:08:46.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43d5dd700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd418056fe0 0x7fd4180594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:46.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.911+0000 7fd43d5dd700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd418056fe0 0x7fd4180594a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd434007000 tx=0x7fd434008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:46.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:46.915+0000 7fd42affd700 1 -- 172.21.2.111:0/1140522534 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd438064970 con 0x7fd4380b4900 2021-04-03T03:08:47.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd438057660 con 0x7fd4380b4900 2021-04-03T03:08:47.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd42affd700 1 -- 172.21.2.111:0/1140522534 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fd42c045bc0 con 0x7fd4380b4900 2021-04-03T03:08:47.015 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:47.015 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:47.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd418056fe0 msgr2=0x7fd4180594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd418056fe0 0x7fd4180594a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd434007000 tx=0x7fd434008040).stop 2021-04-03T03:08:47.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4380b4900 msgr2=0x7fd4381596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4380b4900 0x7fd4381596a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd438062ee0 tx=0x7fd42c013000).stop 2021-04-03T03:08:47.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 shutdown_connections 2021-04-03T03:08:47.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd418056fe0 0x7fd4180594a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd438064c30 0x7fd4380650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4380b4900 0x7fd4381596a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 --2- 172.21.2.111:0/1140522534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4380ab140 0x7fd4380628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 >> 172.21.2.111:0/1140522534 conn(0x7fd4380b61c0 msgr2=0x7fd43814db40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 shutdown_connections 2021-04-03T03:08:47.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.007+0000 7fd43ede0700 1 -- 172.21.2.111:0/1140522534 wait complete. 2021-04-03T03:08:47.022 INFO:tasks.ceph.ceph_manager.ceph:all up! 2021-04-03T03:08:47.023 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:47.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.071+0000 7f15120ef700 1 Processor -- start 2021-04-03T03:08:47.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.071+0000 7f15120ef700 1 -- start start 2021-04-03T03:08:47.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.071+0000 7f15120ef700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f15040dbb90 0x7f1504042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.071+0000 7f15120ef700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1504006bc0 0x7f15040dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.071+0000 7f15120ef700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15040dc6e0 con 0x7f150403d360 2021-04-03T03:08:47.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.071+0000 7f15120ef700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15040d9560 con 0x7f15040dbb90 2021-04-03T03:08:47.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.071+0000 7f15120ef700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150403dcb0 con 0x7f1504006bc0 2021-04-03T03:08:47.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1504006bc0 0x7f15040dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1504006bc0 0x7f15040dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:60994/0 (socket says 172.21.2.111:60994) 2021-04-03T03:08:47.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 -- 172.21.2.111:0/2051491811 learned_addr learned my addr 172.21.2.111:0/2051491811 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15110ed700 1 --2- 172.21.2.111:0/2051491811 >> v2:172.21.2.115:3300/0 conn(0x7f15040dbb90 0x7f1504042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 -- 172.21.2.111:0/2051491811 >> v2:172.21.2.115:3300/0 conn(0x7f15040dbb90 msgr2=0x7f1504042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 --2- 172.21.2.111:0/2051491811 >> v2:172.21.2.115:3300/0 conn(0x7f15040dbb90 0x7f1504042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 -- 172.21.2.111:0/2051491811 >> v1:172.21.2.115:6789/0 conn(0x7f150403d360 legacy=0x7f15040064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 -- 172.21.2.111:0/2051491811 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15040dc460 con 0x7f1504006bc0 2021-04-03T03:08:47.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15118ee700 1 --2- 172.21.2.111:0/2051491811 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1504006bc0 0x7f15040dbf70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f150c06b5d0 tx=0x7f150c05ad70).ready entity=mon.2 client_cookie=70ea51fc8bb1a48e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f14fbfff700 1 -- 172.21.2.111:0/2051491811 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150c05bda0 con 0x7f1504006bc0 2021-04-03T03:08:47.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f14fbfff700 1 -- 172.21.2.111:0/2051491811 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c05df60 con 0x7f1504006bc0 2021-04-03T03:08:47.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f14fbfff700 1 -- 172.21.2.111:0/2051491811 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150c0699f0 con 0x7f1504006bc0 2021-04-03T03:08:47.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 -- 172.21.2.111:0/2051491811 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1504006bc0 msgr2=0x7f15040dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 --2- 172.21.2.111:0/2051491811 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1504006bc0 0x7f15040dbf70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f150c06b5d0 tx=0x7f150c05ad70).stop 2021-04-03T03:08:47.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 -- 172.21.2.111:0/2051491811 shutdown_connections 2021-04-03T03:08:47.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 --2- 172.21.2.111:0/2051491811 >> v2:172.21.2.115:3300/0 conn(0x7f15040dbb90 0x7f1504042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 --2- 172.21.2.111:0/2051491811 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1504006bc0 0x7f15040dbf70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 -- 172.21.2.111:0/2051491811 >> 172.21.2.111:0/2051491811 conn(0x7f15040462c0 msgr2=0x7f15040466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 -- 172.21.2.111:0/2051491811 shutdown_connections 2021-04-03T03:08:47.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 -- 172.21.2.111:0/2051491811 wait complete. 2021-04-03T03:08:47.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 Processor -- start 2021-04-03T03:08:47.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.075+0000 7f15120ef700 1 -- start start 2021-04-03T03:08:47.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f150403d360 0x7f15040d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15040dbb90 0x7f150403a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f150403a690 0x7f150403ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15040d7420 con 0x7f15040dbb90 2021-04-03T03:08:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1504039a70 con 0x7f150403d360 2021-04-03T03:08:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15040dafb0 con 0x7f150403a690 2021-04-03T03:08:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15040dbb90 0x7f150403a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15040dbb90 0x7f150403a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49418/0 (socket says 172.21.2.111:49418) 2021-04-03T03:08:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 -- 172.21.2.111:0/114032684 learned_addr learned my addr 172.21.2.111:0/114032684 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15118ee700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f150403a690 0x7f150403ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 -- 172.21.2.111:0/114032684 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f150403a690 msgr2=0x7f150403ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f150403a690 0x7f150403ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 -- 172.21.2.111:0/114032684 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f150403d360 msgr2=0x7f15040d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f150403d360 0x7f15040d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 -- 172.21.2.111:0/114032684 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504039ca0 con 0x7f15040dbb90 2021-04-03T03:08:47.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15108ec700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15040dbb90 0x7f150403a1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f15040db280 tx=0x7f14fc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f14fa7fc700 1 -- 172.21.2.111:0/114032684 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14fc00e4a0 con 0x7f15040dbb90 2021-04-03T03:08:47.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f14fa7fc700 1 -- 172.21.2.111:0/114032684 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc00e600 con 0x7f15040dbb90 2021-04-03T03:08:47.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f14fa7fc700 1 -- 172.21.2.111:0/114032684 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14fc00e960 con 0x7f15040dbb90 2021-04-03T03:08:47.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1504039590 con 0x7f15040dbb90 2021-04-03T03:08:47.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150403cc10 con 0x7f15040dbb90 2021-04-03T03:08:47.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150403d100 con 0x7f15040dbb90 2021-04-03T03:08:47.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f14fa7fc700 1 -- 172.21.2.111:0/114032684 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f14fc019030 con 0x7f15040dbb90 2021-04-03T03:08:47.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f14fa7fc700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f14ec0570c0 0x7f14ec059580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f14fa7fc700 1 -- 172.21.2.111:0/114032684 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f14fc06f9d0 con 0x7f15040dbb90 2021-04-03T03:08:47.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15110ed700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f14ec0570c0 0x7f14ec059580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.079+0000 7f15110ed700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f14ec0570c0 0x7f14ec059580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f150800c000 tx=0x7f1508006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.083+0000 7f14fa7fc700 1 -- 172.21.2.111:0/114032684 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f14fc044ca0 con 0x7f15040dbb90 2021-04-03T03:08:47.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1504039fe0 con 0x7f15040dbb90 2021-04-03T03:08:47.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f14fa7fc700 1 -- 172.21.2.111:0/114032684 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f14fc0482c0 con 0x7f15040dbb90 2021-04-03T03:08:47.182 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:47.183 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:47.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f14ec0570c0 msgr2=0x7f14ec059580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f14ec0570c0 0x7f14ec059580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f150800c000 tx=0x7f1508006dd0).stop 2021-04-03T03:08:47.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15040dbb90 msgr2=0x7f150403a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15040dbb90 0x7f150403a1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f15040db280 tx=0x7f14fc007c30).stop 2021-04-03T03:08:47.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 shutdown_connections 2021-04-03T03:08:47.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f150403a690 0x7f150403ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f14ec0570c0 0x7f14ec059580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f150403d360 0x7f15040d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 --2- 172.21.2.111:0/114032684 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15040dbb90 0x7f150403a1a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 >> 172.21.2.111:0/114032684 conn(0x7f15040462c0 msgr2=0x7f15040d7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 shutdown_connections 2021-04-03T03:08:47.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.175+0000 7f15120ef700 1 -- 172.21.2.111:0/114032684 wait complete. 2021-04-03T03:08:47.190 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2021-04-03T03:08:47.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 Processor -- start 2021-04-03T03:08:47.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- start start 2021-04-03T03:08:47.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f168c0603f0 0x7f168c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f168c0b17b0 0x7f168c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168c0607d0 con 0x7f168c0add50 2021-04-03T03:08:47.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168c1451c0 con 0x7f168c0603f0 2021-04-03T03:08:47.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c14b090 con 0x7f168c0b17b0 2021-04-03T03:08:47.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f168c0603f0 0x7f168c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f168c0603f0 0x7f168c1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49424/0 (socket says 172.21.2.111:49424) 2021-04-03T03:08:47.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1690e54700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f168c0add50 0x7f168c147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50220/0 (socket says 172.21.2.111:50220) 2021-04-03T03:08:47.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 -- 172.21.2.111:0/2188919097 learned_addr learned my addr 172.21.2.111:0/2188919097 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 -- 172.21.2.111:0/2188919097 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f168c0b17b0 msgr2=0x7f168c144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 --2- 172.21.2.111:0/2188919097 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f168c0b17b0 0x7f168c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 -- 172.21.2.111:0/2188919097 >> v1:172.21.2.111:6789/0 conn(0x7f168c0add50 legacy=0x7f168c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 -- 172.21.2.111:0/2188919097 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c14b330 con 0x7f168c0603f0 2021-04-03T03:08:47.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1691655700 1 --2- 172.21.2.111:0/2188919097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0603f0 0x7f168c1447e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1688002f30 tx=0x7f1688006e90).ready entity=mon.0 client_cookie=992bbbcc27c05746 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f167bfff700 1 -- 172.21.2.111:0/2188919097 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1688009970 con 0x7f168c0603f0 2021-04-03T03:08:47.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f167bfff700 1 -- 172.21.2.111:0/2188919097 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688009ad0 con 0x7f168c0603f0 2021-04-03T03:08:47.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f167bfff700 1 -- 172.21.2.111:0/2188919097 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1688009e30 con 0x7f168c0603f0 2021-04-03T03:08:47.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- 172.21.2.111:0/2188919097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0603f0 msgr2=0x7f168c1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 --2- 172.21.2.111:0/2188919097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0603f0 0x7f168c1447e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1688002f30 tx=0x7f1688006e90).stop 2021-04-03T03:08:47.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- 172.21.2.111:0/2188919097 shutdown_connections 2021-04-03T03:08:47.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 --2- 172.21.2.111:0/2188919097 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f168c0b17b0 0x7f168c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 --2- 172.21.2.111:0/2188919097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0603f0 0x7f168c1447e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- 172.21.2.111:0/2188919097 >> 172.21.2.111:0/2188919097 conn(0x7f168c0b61c0 msgr2=0x7f168c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- 172.21.2.111:0/2188919097 shutdown_connections 2021-04-03T03:08:47.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.239+0000 7f1692657700 1 -- 172.21.2.111:0/2188919097 wait complete. 2021-04-03T03:08:47.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 Processor -- start 2021-04-03T03:08:47.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 -- start start 2021-04-03T03:08:47.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f168c0603f0 0x7f168c147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0add50 0x7f168c147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f168c0b17b0 0x7f168c0a8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c157960 con 0x7f168c0add50 2021-04-03T03:08:47.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c145c90 con 0x7f168c0b17b0 2021-04-03T03:08:47.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c157540 con 0x7f168c0603f0 2021-04-03T03:08:47.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0add50 0x7f168c147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0add50 0x7f168c147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49432/0 (socket says 172.21.2.111:49432) 2021-04-03T03:08:47.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 -- 172.21.2.111:0/1031398441 learned_addr learned my addr 172.21.2.111:0/1031398441 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 -- 172.21.2.111:0/1031398441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f168c0603f0 msgr2=0x7f168c147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f168c0603f0 0x7f168c147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 -- 172.21.2.111:0/1031398441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f168c0b17b0 msgr2=0x7f168c0a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f168c0b17b0 0x7f168c0a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 -- 172.21.2.111:0/1031398441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c157700 con 0x7f168c0add50 2021-04-03T03:08:47.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1690e54700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0add50 0x7f168c147e60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f168c0b5600 tx=0x7f167c007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167c0072e0 con 0x7f168c0add50 2021-04-03T03:08:47.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c005530 con 0x7f168c0add50 2021-04-03T03:08:47.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c146490 con 0x7f168c0add50 2021-04-03T03:08:47.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c0a8890 con 0x7f168c0add50 2021-04-03T03:08:47.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167c005890 con 0x7f168c0add50 2021-04-03T03:08:47.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1668003640 con 0x7f168c0add50 2021-04-03T03:08:47.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f167c010070 con 0x7f168c0add50 2021-04-03T03:08:47.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f166c056fb0 0x7f166c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f167c06f8c0 con 0x7f168c0add50 2021-04-03T03:08:47.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1691655700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f166c056fb0 0x7f166c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7f166c05eb30 0x7f166c060f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 --> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f166c0615b0 con 0x7f166c05eb30 2021-04-03T03:08:47.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f167c074240 con 0x7f168c0add50 2021-04-03T03:08:47.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1691e56700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7f166c05eb30 0x7f166c060f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1691655700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f166c056fb0 0x7f166c059470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1688002f30 tx=0x7f1688015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f1691e56700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7f166c05eb30 0x7f166c060f50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.243+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== osd.0 v2:172.21.2.111:6812/15909 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f166c0615b0 con 0x7f166c05eb30 2021-04-03T03:08:47.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 --> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16680027f0 con 0x7f166c05eb30 2021-04-03T03:08:47.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f167a7fc700 1 -- 172.21.2.111:0/1031398441 <== osd.0 v2:172.21.2.111:6812/15909 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f16680027f0 con 0x7f166c05eb30 2021-04-03T03:08:47.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7f166c05eb30 msgr2=0x7f166c060f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7f166c05eb30 0x7f166c060f50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f166c056fb0 msgr2=0x7f166c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f166c056fb0 0x7f166c059470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1688002f30 tx=0x7f1688015000).stop 2021-04-03T03:08:47.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0add50 msgr2=0x7f168c147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 --2- 172.21.2.111:0/1031398441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f168c0add50 0x7f168c147e60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f168c0b5600 tx=0x7f167c007bc0).stop 2021-04-03T03:08:47.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1691e56700 1 -- 172.21.2.111:0/1031398441 reap_dead start 2021-04-03T03:08:47.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 shutdown_connections 2021-04-03T03:08:47.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 >> 172.21.2.111:0/1031398441 conn(0x7f168c0b61c0 msgr2=0x7f168c0aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 shutdown_connections 2021-04-03T03:08:47.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.255+0000 7f1692657700 1 -- 172.21.2.111:0/1031398441 wait complete. 2021-04-03T03:08:47.270 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:47.271 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2021-04-03T03:08:47.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 Processor -- start 2021-04-03T03:08:47.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 -- start start 2021-04-03T03:08:47.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f19e40603f0 0x7f19e41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f19e40b17b0 0x7f19e4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e40607d0 con 0x7f19e40add50 2021-04-03T03:08:47.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e41451c0 con 0x7f19e40603f0 2021-04-03T03:08:47.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e414b090 con 0x7f19e40b17b0 2021-04-03T03:08:47.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e25e4700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f19e40add50 0x7f19e4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50236/0 (socket says 172.21.2.111:50236) 2021-04-03T03:08:47.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e25e4700 1 -- 172.21.2.111:0/196306270 learned_addr learned my addr 172.21.2.111:0/196306270 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:47.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e2de5700 1 --2- 172.21.2.111:0/196306270 >> v2:172.21.2.111:3300/0 conn(0x7f19e40603f0 0x7f19e41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e2de5700 1 -- 172.21.2.111:0/196306270 >> v2:172.21.2.115:3300/0 conn(0x7f19e40b17b0 msgr2=0x7f19e4144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e2de5700 1 --2- 172.21.2.111:0/196306270 >> v2:172.21.2.115:3300/0 conn(0x7f19e40b17b0 0x7f19e4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e2de5700 1 -- 172.21.2.111:0/196306270 >> v1:172.21.2.111:6789/0 conn(0x7f19e40add50 legacy=0x7f19e4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e2de5700 1 -- 172.21.2.111:0/196306270 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e414b330 con 0x7f19e40603f0 2021-04-03T03:08:47.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e2de5700 1 --2- 172.21.2.111:0/196306270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40603f0 0x7f19e41447e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f19d4002f30 tx=0x7f19d4006dd0).ready entity=mon.0 client_cookie=4af000087965af6c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e1de3700 1 -- 172.21.2.111:0/196306270 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d40097f0 con 0x7f19e40603f0 2021-04-03T03:08:47.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e1de3700 1 -- 172.21.2.111:0/196306270 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4009950 con 0x7f19e40603f0 2021-04-03T03:08:47.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e1de3700 1 -- 172.21.2.111:0/196306270 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d4009cb0 con 0x7f19e40603f0 2021-04-03T03:08:47.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 -- 172.21.2.111:0/196306270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40603f0 msgr2=0x7f19e41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.319+0000 7f19e3de7700 1 --2- 172.21.2.111:0/196306270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40603f0 0x7f19e41447e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f19d4002f30 tx=0x7f19d4006dd0).stop 2021-04-03T03:08:47.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- 172.21.2.111:0/196306270 shutdown_connections 2021-04-03T03:08:47.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 --2- 172.21.2.111:0/196306270 >> v2:172.21.2.115:3300/0 conn(0x7f19e40b17b0 0x7f19e4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 --2- 172.21.2.111:0/196306270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40603f0 0x7f19e41447e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- 172.21.2.111:0/196306270 >> 172.21.2.111:0/196306270 conn(0x7f19e40b61c0 msgr2=0x7f19e405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- 172.21.2.111:0/196306270 shutdown_connections 2021-04-03T03:08:47.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- 172.21.2.111:0/196306270 wait complete. 2021-04-03T03:08:47.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 Processor -- start 2021-04-03T03:08:47.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- start start 2021-04-03T03:08:47.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e40603f0 0x7f19e4147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19e40add50 0x7f19e40aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40b17b0 0x7f19e40ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40b4b80 con 0x7f19e40b17b0 2021-04-03T03:08:47.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40a8710 con 0x7f19e40add50 2021-04-03T03:08:47.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40a8940 con 0x7f19e40603f0 2021-04-03T03:08:47.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40b17b0 0x7f19e40ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40b17b0 0x7f19e40ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49446/0 (socket says 172.21.2.111:49446) 2021-04-03T03:08:47.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 -- 172.21.2.111:0/2057254907 learned_addr learned my addr 172.21.2.111:0/2057254907 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 -- 172.21.2.111:0/2057254907 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e40603f0 msgr2=0x7f19e4147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e40603f0 0x7f19e4147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 -- 172.21.2.111:0/2057254907 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19e40add50 msgr2=0x7f19e40aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19e40add50 0x7f19e40aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 -- 172.21.2.111:0/2057254907 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e40aa880 con 0x7f19e40b17b0 2021-04-03T03:08:47.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e35e6700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40b17b0 0x7f19e40ab350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f19dc0032b0 tx=0x7f19dc0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19dc00c960 con 0x7f19e40b17b0 2021-04-03T03:08:47.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00cac0 con 0x7f19e40b17b0 2021-04-03T03:08:47.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e40ab840 con 0x7f19e40b17b0 2021-04-03T03:08:47.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e40b0b30 con 0x7f19e40b17b0 2021-04-03T03:08:47.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19dc00ce20 con 0x7f19e40b17b0 2021-04-03T03:08:47.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19e40aaa10 con 0x7f19e40b17b0 2021-04-03T03:08:47.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f19dc014410 con 0x7f19e40b17b0 2021-04-03T03:08:47.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0056ef0 0x7f19c00593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f19dc0708b0 con 0x7f19e40b17b0 2021-04-03T03:08:47.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f19c005ea90 0x7f19c0060eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 --> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19c0061510 con 0x7f19c005ea90 2021-04-03T03:08:47.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19dc070c90 con 0x7f19e40b17b0 2021-04-03T03:08:47.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e2de5700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0056ef0 0x7f19c00593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e25e4700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f19c005ea90 0x7f19c0060eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e2de5700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0056ef0 0x7f19c00593b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f19d4002f30 tx=0x7f19d4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.323+0000 7f19e25e4700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f19c005ea90 0x7f19c0060eb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.327+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== osd.1 v2:172.21.2.111:6802/15910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f19c0061510 con 0x7f19c005ea90 2021-04-03T03:08:47.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 --> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19e40abcd0 con 0x7f19c005ea90 2021-04-03T03:08:47.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19d3fff700 1 -- 172.21.2.111:0/2057254907 <== osd.1 v2:172.21.2.111:6802/15910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f19e40abcd0 con 0x7f19c005ea90 2021-04-03T03:08:47.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f19c005ea90 msgr2=0x7f19c0060eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f19c005ea90 0x7f19c0060eb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0056ef0 msgr2=0x7f19c00593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0056ef0 0x7f19c00593b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f19d4002f30 tx=0x7f19d4015000).stop 2021-04-03T03:08:47.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40b17b0 msgr2=0x7f19e40ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 --2- 172.21.2.111:0/2057254907 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e40b17b0 0x7f19e40ab350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f19dc0032b0 tx=0x7f19dc0091a0).stop 2021-04-03T03:08:47.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e35e6700 1 -- 172.21.2.111:0/2057254907 reap_dead start 2021-04-03T03:08:47.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 shutdown_connections 2021-04-03T03:08:47.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 >> 172.21.2.111:0/2057254907 conn(0x7f19e40b61c0 msgr2=0x7f19e40b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 shutdown_connections 2021-04-03T03:08:47.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.335+0000 7f19e3de7700 1 -- 172.21.2.111:0/2057254907 wait complete. 2021-04-03T03:08:47.351 INFO:teuthology.orchestra.run.gibba011.stdout:55834574854 2021-04-03T03:08:47.351 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2021-04-03T03:08:47.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 Processor -- start 2021-04-03T03:08:47.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- start start 2021-04-03T03:08:47.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff1b000dee0 0x7ff1b00d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b000e2c0 con 0x7ff1b002b970 2021-04-03T03:08:47.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b0044c30 con 0x7ff1b0045060 2021-04-03T03:08:47.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b0045c50 con 0x7ff1b000dee0 2021-04-03T03:08:47.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd0ff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff1b002b970 0x7ff1b002bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50252/0 (socket says 172.21.2.111:50252) 2021-04-03T03:08:47.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd0ff700 1 -- 172.21.2.111:0/1408766358 learned_addr learned my addr 172.21.2.111:0/1408766358 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:47.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bc8fe700 1 --2- 172.21.2.111:0/1408766358 >> v2:172.21.2.115:3300/0 conn(0x7ff1b000dee0 0x7ff1b00d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558608121 0 0) 0x7ff1b000e2c0 con 0x7ff1b002b970 2021-04-03T03:08:47.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a4003240 con 0x7ff1b002b970 2021-04-03T03:08:47.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2717580610 0 0) 0x7ff1a4003240 con 0x7ff1b002b970 2021-04-03T03:08:47.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 >> v2:172.21.2.115:3300/0 conn(0x7ff1b000dee0 msgr2=0x7ff1b00d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/1408766358 >> v2:172.21.2.115:3300/0 conn(0x7ff1b000dee0 0x7ff1b00d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 >> v1:172.21.2.115:6789/0 conn(0x7ff1b0045060 legacy=0x7ff1b00d7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b0045ef0 con 0x7ff1b002b970 2021-04-03T03:08:47.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ff1b80450f0 con 0x7ff1b002b970 2021-04-03T03:08:47.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 >> v1:172.21.2.111:6789/0 conn(0x7ff1b002b970 legacy=0x7ff1b002bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1a40037b0 0x7ff1a4003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1a4004940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1a4004e30 0x7ff1a400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0045c50 con 0x7ff1a40044e0 2021-04-03T03:08:47.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4004220 con 0x7ff1a40037b0 2021-04-03T03:08:47.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a400bc20 con 0x7ff1a4004e30 2021-04-03T03:08:47.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1a4004940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b7fff700 1 -- 172.21.2.111:0/1408766358 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1a4004e30 msgr2=0x7ff1a400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1a4004e30 0x7ff1a400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b7fff700 1 -- 172.21.2.111:0/1408766358 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1a40037b0 msgr2=0x7ff1a4003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1a40037b0 0x7ff1a4003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b7fff700 1 -- 172.21.2.111:0/1408766358 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4003240 con 0x7ff1a40044e0 2021-04-03T03:08:47.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd0ff700 1 -- 172.21.2.111:0/1408766358 reap_dead start 2021-04-03T03:08:47.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1a4004940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff1b000e610 tx=0x7ff1a8007b70).ready entity=mon.0 client_cookie=51a659dd3a13ace1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a800c660 con 0x7ff1a40044e0 2021-04-03T03:08:47.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800c7c0 con 0x7ff1a40044e0 2021-04-03T03:08:47.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1b77fe700 1 -- 172.21.2.111:0/1408766358 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a800cb20 con 0x7ff1a40044e0 2021-04-03T03:08:47.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- 172.21.2.111:0/1408766358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 msgr2=0x7ff1a4004940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1a4004940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff1b000e610 tx=0x7ff1a8007b70).stop 2021-04-03T03:08:47.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- 172.21.2.111:0/1408766358 shutdown_connections 2021-04-03T03:08:47.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 --2- 172.21.2.111:0/1408766358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1a4004940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- 172.21.2.111:0/1408766358 >> 172.21.2.111:0/1408766358 conn(0x7ff1b004a6c0 msgr2=0x7ff1b004aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- 172.21.2.111:0/1408766358 shutdown_connections 2021-04-03T03:08:47.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- 172.21.2.111:0/1408766358 wait complete. 2021-04-03T03:08:47.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 Processor -- start 2021-04-03T03:08:47.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- start start 2021-04-03T03:08:47.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1b00e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b00416d0 0x7ff1b00d6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1b0042f30 0x7ff1b0020920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0043b10 con 0x7ff1a40044e0 2021-04-03T03:08:47.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0042dc0 con 0x7ff1b0042f30 2021-04-03T03:08:47.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bd900700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0041480 con 0x7ff1b00416d0 2021-04-03T03:08:47.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bc8fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1b00e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bc8fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1b00e2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49468/0 (socket says 172.21.2.111:49468) 2021-04-03T03:08:47.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.403+0000 7ff1bc8fe700 1 -- 172.21.2.111:0/1290519034 learned_addr learned my addr 172.21.2.111:0/1290519034 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bc8fe700 1 -- 172.21.2.111:0/1290519034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b00416d0 msgr2=0x7ff1b00d6890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:08:47.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bc8fe700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b00416d0 0x7ff1b00d6890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bc8fe700 1 -- 172.21.2.111:0/1290519034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1b0042f30 msgr2=0x7ff1b0020920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:08:47.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bc8fe700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1b0042f30 0x7ff1b0020920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bc8fe700 1 -- 172.21.2.111:0/1290519034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b0040f40 con 0x7ff1a40044e0 2021-04-03T03:08:47.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bc8fe700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1b00e2a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff1b0020680 tx=0x7ff1ac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ac009820 con 0x7ff1a40044e0 2021-04-03T03:08:47.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac009980 con 0x7ff1a40044e0 2021-04-03T03:08:47.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ac009ce0 con 0x7ff1a40044e0 2021-04-03T03:08:47.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b00411e0 con 0x7ff1a40044e0 2021-04-03T03:08:47.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b0021020 con 0x7ff1a40044e0 2021-04-03T03:08:47.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1b0004d30 con 0x7ff1a40044e0 2021-04-03T03:08:47.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff1ac013480 con 0x7ff1a40044e0 2021-04-03T03:08:47.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198057050 0x7ff198059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff1ac070280 con 0x7ff1a40044e0 2021-04-03T03:08:47.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7ff19805ea50 0x7ff198060e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 --> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff198061430 con 0x7ff19805ea50 2021-04-03T03:08:47.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1ac045600 con 0x7ff1a40044e0 2021-04-03T03:08:47.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198057050 0x7ff198059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bd0ff700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7ff19805ea50 0x7ff198060e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1bd0ff700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7ff19805ea50 0x7ff198060e70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198057050 0x7ff198059510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff1b000e610 tx=0x7ff1a8017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.407+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== osd.2 v2:172.21.2.111:6810/15911 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7ff198061430 con 0x7ff19805ea50 2021-04-03T03:08:47.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 --> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff1b00d35c0 con 0x7ff19805ea50 2021-04-03T03:08:47.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1b5ffb700 1 -- 172.21.2.111:0/1290519034 <== osd.2 v2:172.21.2.111:6810/15911 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff1b00d35c0 con 0x7ff19805ea50 2021-04-03T03:08:47.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7ff19805ea50 msgr2=0x7ff198060e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7ff19805ea50 0x7ff198060e70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198057050 msgr2=0x7ff198059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198057050 0x7ff198059510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff1b000e610 tx=0x7ff1a8017000).stop 2021-04-03T03:08:47.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 msgr2=0x7ff1b00e2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 --2- 172.21.2.111:0/1290519034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a40044e0 0x7ff1b00e2a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff1b0020680 tx=0x7ff1ac008040).stop 2021-04-03T03:08:47.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd0ff700 1 -- 172.21.2.111:0/1290519034 reap_dead start 2021-04-03T03:08:47.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 shutdown_connections 2021-04-03T03:08:47.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 >> 172.21.2.111:0/1290519034 conn(0x7ff1b004a6c0 msgr2=0x7ff1b0046fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 shutdown_connections 2021-04-03T03:08:47.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.419+0000 7ff1bd900700 1 -- 172.21.2.111:0/1290519034 wait complete. 2021-04-03T03:08:47.435 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:47.436 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2021-04-03T03:08:47.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 Processor -- start 2021-04-03T03:08:47.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- start start 2021-04-03T03:08:47.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8abc005640 0x7f8abc0d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8abc0d8e90 0x7f8abc0e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc03d840 con 0x7f8abc03f120 2021-04-03T03:08:47.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc03faa0 con 0x7f8abc005640 2021-04-03T03:08:47.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc03dca0 con 0x7f8abc0d8e90 2021-04-03T03:08:47.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8abc005640 0x7f8abc0d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8abc005640 0x7f8abc0d89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49480/0 (socket says 172.21.2.111:49480) 2021-04-03T03:08:47.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/2600376361 learned_addr learned my addr 172.21.2.111:0/2600376361 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac887d700 1 --2- 172.21.2.111:0/2600376361 >> v2:172.21.2.115:3300/0 conn(0x7f8abc0d8e90 0x7f8abc0e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/2600376361 >> v2:172.21.2.115:3300/0 conn(0x7f8abc0d8e90 msgr2=0x7f8abc0e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- 172.21.2.111:0/2600376361 >> v2:172.21.2.115:3300/0 conn(0x7f8abc0d8e90 0x7f8abc0e2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/2600376361 >> v1:172.21.2.111:6789/0 conn(0x7f8abc03f120 legacy=0x7f8abc041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac2ffd700 1 -- 172.21.2.111:0/2600376361 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351028098 0 0) 0x7f8abc03d840 con 0x7f8abc03f120 2021-04-03T03:08:47.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/2600376361 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc03f800 con 0x7f8abc005640 2021-04-03T03:08:47.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- 172.21.2.111:0/2600376361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc005640 0x7f8abc0d89a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8006e90).ready entity=mon.0 client_cookie=7228cae15f9dc707 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac2ffd700 1 -- 172.21.2.111:0/2600376361 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab8009850 con 0x7f8abc005640 2021-04-03T03:08:47.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac2ffd700 1 -- 172.21.2.111:0/2600376361 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab80099b0 con 0x7f8abc005640 2021-04-03T03:08:47.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac2ffd700 1 -- 172.21.2.111:0/2600376361 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab8009d10 con 0x7f8abc005640 2021-04-03T03:08:47.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- 172.21.2.111:0/2600376361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc005640 msgr2=0x7f8abc0d89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- 172.21.2.111:0/2600376361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc005640 0x7f8abc0d89a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8006e90).stop 2021-04-03T03:08:47.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- 172.21.2.111:0/2600376361 shutdown_connections 2021-04-03T03:08:47.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- 172.21.2.111:0/2600376361 >> v2:172.21.2.115:3300/0 conn(0x7f8abc0d8e90 0x7f8abc0e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- 172.21.2.111:0/2600376361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc005640 0x7f8abc0d89a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- 172.21.2.111:0/2600376361 >> 172.21.2.111:0/2600376361 conn(0x7f8abc0462b0 msgr2=0x7f8abc0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- 172.21.2.111:0/2600376361 shutdown_connections 2021-04-03T03:08:47.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- 172.21.2.111:0/2600376361 wait complete. 2021-04-03T03:08:47.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 Processor -- start 2021-04-03T03:08:47.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- start start 2021-04-03T03:08:47.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8abc005640 0x7f8abc09fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc03f120 0x7f8abc0d7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8abc0d8e90 0x7f8abc0d7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc03bf50 con 0x7f8abc03f120 2021-04-03T03:08:47.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0a2fa0 con 0x7f8abc005640 2021-04-03T03:08:47.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac907e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0a3240 con 0x7f8abc0d8e90 2021-04-03T03:08:47.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc03f120 0x7f8abc0d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc03f120 0x7f8abc0d7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49486/0 (socket says 172.21.2.111:49486) 2021-04-03T03:08:47.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/1605615990 learned_addr learned my addr 172.21.2.111:0/1605615990 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/1605615990 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8abc0d8e90 msgr2=0x7f8abc0d7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8abc0d8e90 0x7f8abc0d7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/1605615990 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8abc005640 msgr2=0x7f8abc09fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8abc005640 0x7f8abc09fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.487+0000 7f8ac37fe700 1 -- 172.21.2.111:0/1605615990 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc03b810 con 0x7f8abc03f120 2021-04-03T03:08:47.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac37fe700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc03f120 0x7f8abc0d7760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab800bec0 con 0x7f8abc03f120 2021-04-03T03:08:47.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8009cb0 con 0x7f8abc03f120 2021-04-03T03:08:47.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab801e650 con 0x7f8abc03f120 2021-04-03T03:08:47.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc03bab0 con 0x7f8abc03f120 2021-04-03T03:08:47.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc0a0160 con 0x7f8abc03f120 2021-04-03T03:08:47.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8abc03bc40 con 0x7f8abc03f120 2021-04-03T03:08:47.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8ab801a040 con 0x7f8abc03f120 2021-04-03T03:08:47.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8aa4056f20 0x7f8aa40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f8ab8071290 con 0x7f8abc03f120 2021-04-03T03:08:47.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f8aa405ea20 0x7f8aa4060e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 --> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8aa40614a0 con 0x7f8aa405ea20 2021-04-03T03:08:47.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab8073850 con 0x7f8abc03f120 2021-04-03T03:08:47.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac3fff700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8aa4056f20 0x7f8aa40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac887d700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f8aa405ea20 0x7f8aa4060e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac3fff700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8aa4056f20 0x7f8aa40593e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8ab4000e90 tx=0x7f8ab4007660).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac887d700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f8aa405ea20 0x7f8aa4060e40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.491+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== osd.3 v2:172.21.2.111:6826/15913 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f8aa40614a0 con 0x7f8aa405ea20 2021-04-03T03:08:47.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 --> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8abc03c3f0 con 0x7f8aa405ea20 2021-04-03T03:08:47.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac17fa700 1 -- 172.21.2.111:0/1605615990 <== osd.3 v2:172.21.2.111:6826/15913 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8abc03c3f0 con 0x7f8aa405ea20 2021-04-03T03:08:47.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f8aa405ea20 msgr2=0x7f8aa4060e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f8aa405ea20 0x7f8aa4060e40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8aa4056f20 msgr2=0x7f8aa40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8aa4056f20 0x7f8aa40593e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8ab4000e90 tx=0x7f8ab4007660).stop 2021-04-03T03:08:47.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc03f120 msgr2=0x7f8abc0d7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 --2- 172.21.2.111:0/1605615990 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8abc03f120 0x7f8abc0d7760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8015000).stop 2021-04-03T03:08:47.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac887d700 1 -- 172.21.2.111:0/1605615990 reap_dead start 2021-04-03T03:08:47.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 shutdown_connections 2021-04-03T03:08:47.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 >> 172.21.2.111:0/1605615990 conn(0x7f8abc0462b0 msgr2=0x7f8abc0db770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 shutdown_connections 2021-04-03T03:08:47.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.503+0000 7f8ac907e700 1 -- 172.21.2.111:0/1605615990 wait complete. 2021-04-03T03:08:47.517 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:47.517 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2021-04-03T03:08:47.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d3386700 1 Processor -- start 2021-04-03T03:08:47.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d3386700 1 -- start start 2021-04-03T03:08:47.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d3386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f00d40603f0 0x7f00d41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40add50 0x7f00d4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d3386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d40607d0 con 0x7f00d40b17b0 2021-04-03T03:08:47.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d3386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d41451c0 con 0x7f00d40603f0 2021-04-03T03:08:47.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d414b090 con 0x7f00d40add50 2021-04-03T03:08:47.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d2b85700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f00d40b17b0 0x7f00d4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34592/0 (socket says 172.21.2.111:34592) 2021-04-03T03:08:47.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d1b83700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40add50 0x7f00d4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d2384700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f00d40603f0 0x7f00d41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.567+0000 7f00d2b85700 1 -- 172.21.2.111:0/3857482672 learned_addr learned my addr 172.21.2.111:0/3857482672 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:47.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1b83700 1 -- 172.21.2.111:0/3857482672 >> v2:172.21.2.115:3300/0 conn(0x7f00d40603f0 msgr2=0x7f00d41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1b83700 1 --2- 172.21.2.111:0/3857482672 >> v2:172.21.2.115:3300/0 conn(0x7f00d40603f0 0x7f00d41447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1b83700 1 -- 172.21.2.111:0/3857482672 >> v1:172.21.2.115:6789/0 conn(0x7f00d40b17b0 legacy=0x7f00d4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1b83700 1 -- 172.21.2.111:0/3857482672 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d414b330 con 0x7f00d40add50 2021-04-03T03:08:47.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1b83700 1 --2- 172.21.2.111:0/3857482672 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40add50 0x7f00d4144cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f00bc002f30 tx=0x7f00bc006dd0).ready entity=mon.2 client_cookie=5ecd99419cdc7aec server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1382700 1 -- 172.21.2.111:0/3857482672 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00bc009760 con 0x7f00d40add50 2021-04-03T03:08:47.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1382700 1 -- 172.21.2.111:0/3857482672 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc0098c0 con 0x7f00d40add50 2021-04-03T03:08:47.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d1382700 1 -- 172.21.2.111:0/3857482672 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00bc009c20 con 0x7f00d40add50 2021-04-03T03:08:47.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- 172.21.2.111:0/3857482672 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40add50 msgr2=0x7f00d4144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 --2- 172.21.2.111:0/3857482672 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40add50 0x7f00d4144cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f00bc002f30 tx=0x7f00bc006dd0).stop 2021-04-03T03:08:47.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- 172.21.2.111:0/3857482672 shutdown_connections 2021-04-03T03:08:47.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 --2- 172.21.2.111:0/3857482672 >> v2:172.21.2.115:3300/0 conn(0x7f00d40603f0 0x7f00d41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 --2- 172.21.2.111:0/3857482672 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40add50 0x7f00d4144cd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- 172.21.2.111:0/3857482672 >> 172.21.2.111:0/3857482672 conn(0x7f00d40b61c0 msgr2=0x7f00d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- 172.21.2.111:0/3857482672 shutdown_connections 2021-04-03T03:08:47.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- 172.21.2.111:0/3857482672 wait complete. 2021-04-03T03:08:47.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 Processor -- start 2021-04-03T03:08:47.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- start start 2021-04-03T03:08:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40603f0 0x7f00d40b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f00d40add50 0x7f00d415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00d40b17b0 0x7f00d415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40a8160 con 0x7f00d40b17b0 2021-04-03T03:08:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40a83b0 con 0x7f00d40add50 2021-04-03T03:08:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40b0640 con 0x7f00d40603f0 2021-04-03T03:08:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00d40b17b0 0x7f00d415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00d40b17b0 0x7f00d415a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49502/0 (socket says 172.21.2.111:49502) 2021-04-03T03:08:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 -- 172.21.2.111:0/4233767689 learned_addr learned my addr 172.21.2.111:0/4233767689 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 -- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40603f0 msgr2=0x7f00d40b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00d40603f0 0x7f00d40b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 -- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f00d40add50 msgr2=0x7f00d415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f00d40add50 0x7f00d415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 -- 172.21.2.111:0/4233767689 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d415ac90 con 0x7f00d40b17b0 2021-04-03T03:08:47.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d2b85700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00d40b17b0 0x7f00d415a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f00d40b5600 tx=0x7f00cc007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00cc005b80 con 0x7f00d40b17b0 2021-04-03T03:08:47.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc005ce0 con 0x7f00d40b17b0 2021-04-03T03:08:47.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d40648d0 con 0x7f00d40b17b0 2021-04-03T03:08:47.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00cc007300 con 0x7f00d40b17b0 2021-04-03T03:08:47.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.571+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d4064ee0 con 0x7f00d40b17b0 2021-04-03T03:08:47.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00d40a8570 con 0x7f00d40b17b0 2021-04-03T03:08:47.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f00cc010030 con 0x7f00d40b17b0 2021-04-03T03:08:47.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00cb7fe700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f00b8056ea0 0x7f00b8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f00cc0701a0 con 0x7f00d40b17b0 2021-04-03T03:08:47.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00cb7fe700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f00b805e940 0x7f00b8060d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 --> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00b80613c0 con 0x7f00b805e940 2021-04-03T03:08:47.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00cc072810 con 0x7f00d40b17b0 2021-04-03T03:08:47.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00d2384700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f00b8056ea0 0x7f00b8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00d2384700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f00b8056ea0 0x7f00b8059360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f00c400c000 tx=0x7f00c4006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00d1b83700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f00b805e940 0x7f00b8060d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.575+0000 7f00d1b83700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f00b805e940 0x7f00b8060d60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.579+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== osd.4 v2:172.21.2.115:6800/16012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f00b80613c0 con 0x7f00b805e940 2021-04-03T03:08:47.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.587+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 --> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f00d4057660 con 0x7f00b805e940 2021-04-03T03:08:47.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00cb7fe700 1 -- 172.21.2.111:0/4233767689 <== osd.4 v2:172.21.2.115:6800/16012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f00d4057660 con 0x7f00b805e940 2021-04-03T03:08:47.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f00b805e940 msgr2=0x7f00b8060d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f00b805e940 0x7f00b8060d60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f00b8056ea0 msgr2=0x7f00b8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f00b8056ea0 0x7f00b8059360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f00c400c000 tx=0x7f00c4006dd0).stop 2021-04-03T03:08:47.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00d40b17b0 msgr2=0x7f00d415a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 --2- 172.21.2.111:0/4233767689 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00d40b17b0 0x7f00d415a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f00d40b5600 tx=0x7f00cc007f10).stop 2021-04-03T03:08:47.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d2b85700 1 -- 172.21.2.111:0/4233767689 reap_dead start 2021-04-03T03:08:47.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 shutdown_connections 2021-04-03T03:08:47.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 >> 172.21.2.111:0/4233767689 conn(0x7f00d40b61c0 msgr2=0x7f00d4143ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 shutdown_connections 2021-04-03T03:08:47.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.591+0000 7f00d3386700 1 -- 172.21.2.111:0/4233767689 wait complete. 2021-04-03T03:08:47.604 INFO:teuthology.orchestra.run.gibba011.stdout:55834574854 2021-04-03T03:08:47.605 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2021-04-03T03:08:47.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e44826700 1 Processor -- start 2021-04-03T03:08:47.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e44826700 1 -- start start 2021-04-03T03:08:47.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e44826700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6e400603f0 0x7f6e40067b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e44826700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 0x7f6e400acb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e44826700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e40140b60 con 0x7f6e400ad020 2021-04-03T03:08:47.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e44826700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e400af3f0 con 0x7f6e400603f0 2021-04-03T03:08:47.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e44826700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40062bf0 con 0x7f6e400b18b0 2021-04-03T03:08:47.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3ffff700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f6e400ad020 0x7f6e4013d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34608/0 (socket says 172.21.2.111:34608) 2021-04-03T03:08:47.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3ffff700 1 -- 172.21.2.111:0/3069596575 learned_addr learned my addr 172.21.2.111:0/3069596575 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:47.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3effd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 0x7f6e400acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3f7fe700 1 --2- 172.21.2.111:0/3069596575 >> v2:172.21.2.115:3300/0 conn(0x7f6e400603f0 0x7f6e40067b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3f7fe700 1 -- 172.21.2.111:0/3069596575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 msgr2=0x7f6e400acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3f7fe700 1 --2- 172.21.2.111:0/3069596575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 0x7f6e400acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3f7fe700 1 -- 172.21.2.111:0/3069596575 >> v1:172.21.2.115:6789/0 conn(0x7f6e400ad020 legacy=0x7f6e4013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3f7fe700 1 -- 172.21.2.111:0/3069596575 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40068000 con 0x7f6e400603f0 2021-04-03T03:08:47.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.655+0000 7f6e3f7fe700 1 --2- 172.21.2.111:0/3069596575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6e400603f0 0x7f6e40067b10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6e40062a50 tx=0x7f6e28006dd0).ready entity=mon.1 client_cookie=9a15590d76c00fff server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3e7fc700 1 -- 172.21.2.111:0/3069596575 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e280097b0 con 0x7f6e400603f0 2021-04-03T03:08:47.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3e7fc700 1 -- 172.21.2.111:0/3069596575 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e28009910 con 0x7f6e400603f0 2021-04-03T03:08:47.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3e7fc700 1 -- 172.21.2.111:0/3069596575 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e28009c70 con 0x7f6e400603f0 2021-04-03T03:08:47.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/3069596575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6e400603f0 msgr2=0x7f6e40067b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 --2- 172.21.2.111:0/3069596575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6e400603f0 0x7f6e40067b10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6e40062a50 tx=0x7f6e28006dd0).stop 2021-04-03T03:08:47.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/3069596575 shutdown_connections 2021-04-03T03:08:47.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 --2- 172.21.2.111:0/3069596575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6e400603f0 0x7f6e40067b10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 --2- 172.21.2.111:0/3069596575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 0x7f6e400acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/3069596575 >> 172.21.2.111:0/3069596575 conn(0x7f6e400b61c0 msgr2=0x7f6e4005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/3069596575 shutdown_connections 2021-04-03T03:08:47.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/3069596575 wait complete. 2021-04-03T03:08:47.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 Processor -- start 2021-04-03T03:08:47.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- start start 2021-04-03T03:08:47.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6e400603f0 0x7f6e4012f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6e400ad020 0x7f6e400ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 0x7f6e4013dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4013edf0 con 0x7f6e400ad020 2021-04-03T03:08:47.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40064040 con 0x7f6e400603f0 2021-04-03T03:08:47.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40065920 con 0x7f6e400b18b0 2021-04-03T03:08:47.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6e400ad020 0x7f6e400ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6e400ad020 0x7f6e400ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49520/0 (socket says 172.21.2.111:49520) 2021-04-03T03:08:47.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 -- 172.21.2.111:0/77316036 learned_addr learned my addr 172.21.2.111:0/77316036 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 -- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 msgr2=0x7f6e4013dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6e400b18b0 0x7f6e4013dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 -- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6e400603f0 msgr2=0x7f6e4012f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6e400603f0 0x7f6e4012f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 -- 172.21.2.111:0/77316036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40065680 con 0x7f6e400ad020 2021-04-03T03:08:47.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3effd700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6e400ad020 0x7f6e400ab8c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6e40065ab0 tx=0x7f6e30006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e3000ca10 con 0x7f6e400ad020 2021-04-03T03:08:47.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3000cb70 con 0x7f6e400ad020 2021-04-03T03:08:47.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e400abdb0 con 0x7f6e400ad020 2021-04-03T03:08:47.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e40066e70 con 0x7f6e400ad020 2021-04-03T03:08:47.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e300123f0 con 0x7f6e400ad020 2021-04-03T03:08:47.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e400643c0 con 0x7f6e400ad020 2021-04-03T03:08:47.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6e30012550 con 0x7f6e400ad020 2021-04-03T03:08:47.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e24057000 0x7f6e240594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f6e30071230 con 0x7f6e400ad020 2021-04-03T03:08:47.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f6e2405eb80 0x7f6e24060fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 --> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e24061560 con 0x7f6e2405eb80 2021-04-03T03:08:47.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e30071540 con 0x7f6e400ad020 2021-04-03T03:08:47.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.659+0000 7f6e3f7fe700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e24057000 0x7f6e240594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.663+0000 7f6e3f7fe700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e24057000 0x7f6e240594c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6e40062a50 tx=0x7f6e28015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.663+0000 7f6e3ffff700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f6e2405eb80 0x7f6e24060fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.663+0000 7f6e3ffff700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f6e2405eb80 0x7f6e24060fa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.663+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== osd.5 v2:172.21.2.115:6816/16014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f6e24061560 con 0x7f6e2405eb80 2021-04-03T03:08:47.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 --> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e40057660 con 0x7f6e2405eb80 2021-04-03T03:08:47.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e3cff9700 1 -- 172.21.2.111:0/77316036 <== osd.5 v2:172.21.2.115:6816/16014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6e40057660 con 0x7f6e2405eb80 2021-04-03T03:08:47.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f6e2405eb80 msgr2=0x7f6e24060fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f6e2405eb80 0x7f6e24060fa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e24057000 msgr2=0x7f6e240594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e24057000 0x7f6e240594c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6e40062a50 tx=0x7f6e28015000).stop 2021-04-03T03:08:47.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6e400ad020 msgr2=0x7f6e400ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 --2- 172.21.2.111:0/77316036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6e400ad020 0x7f6e400ab8c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6e40065ab0 tx=0x7f6e30006dd0).stop 2021-04-03T03:08:47.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e3ffff700 1 -- 172.21.2.111:0/77316036 reap_dead start 2021-04-03T03:08:47.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 shutdown_connections 2021-04-03T03:08:47.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 >> 172.21.2.111:0/77316036 conn(0x7f6e400b61c0 msgr2=0x7f6e400ad490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 shutdown_connections 2021-04-03T03:08:47.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.675+0000 7f6e44826700 1 -- 172.21.2.111:0/77316036 wait complete. 2021-04-03T03:08:47.690 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:47.690 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2021-04-03T03:08:47.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 Processor -- start 2021-04-03T03:08:47.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 -- start start 2021-04-03T03:08:47.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83ac0603f0 0x7f83ac1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ac0607d0 con 0x7f83ac0b17b0 2021-04-03T03:08:47.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ac14b090 con 0x7f83ac0add50 2021-04-03T03:08:47.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ac14b330 con 0x7f83ac0603f0 2021-04-03T03:08:47.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b1b31700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f83ac0b17b0 0x7f83ac147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50318/0 (socket says 172.21.2.111:50318) 2021-04-03T03:08:47.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b1b31700 1 -- 172.21.2.111:0/3195342748 learned_addr learned my addr 172.21.2.111:0/3195342748 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:47.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b1330700 1 --2- 172.21.2.111:0/3195342748 >> v2:172.21.2.111:3300/0 conn(0x7f83ac0603f0 0x7f83ac1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b1330700 1 -- 172.21.2.111:0/3195342748 >> v1:172.21.2.115:6789/0 conn(0x7f83ac0add50 legacy=0x7f83ac147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b1330700 1 -- 172.21.2.111:0/3195342748 >> v1:172.21.2.111:6789/0 conn(0x7f83ac0b17b0 legacy=0x7f83ac147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b1330700 1 -- 172.21.2.111:0/3195342748 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac14b5d0 con 0x7f83ac0603f0 2021-04-03T03:08:47.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b1330700 1 --2- 172.21.2.111:0/3195342748 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac1447e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f83ac0b5600 tx=0x7f839c006dd0).ready entity=mon.0 client_cookie=8325ef70527fd871 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f839bfff700 1 -- 172.21.2.111:0/3195342748 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c0097b0 con 0x7f83ac0603f0 2021-04-03T03:08:47.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f839bfff700 1 -- 172.21.2.111:0/3195342748 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c009910 con 0x7f83ac0603f0 2021-04-03T03:08:47.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f839bfff700 1 -- 172.21.2.111:0/3195342748 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c009c70 con 0x7f83ac0603f0 2021-04-03T03:08:47.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 -- 172.21.2.111:0/3195342748 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 msgr2=0x7f83ac1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.739+0000 7f83b2332700 1 --2- 172.21.2.111:0/3195342748 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac1447e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f83ac0b5600 tx=0x7f839c006dd0).stop 2021-04-03T03:08:47.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- 172.21.2.111:0/3195342748 shutdown_connections 2021-04-03T03:08:47.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 --2- 172.21.2.111:0/3195342748 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac1447e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- 172.21.2.111:0/3195342748 >> 172.21.2.111:0/3195342748 conn(0x7f83ac0b61c0 msgr2=0x7f83ac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- 172.21.2.111:0/3195342748 shutdown_connections 2021-04-03T03:08:47.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- 172.21.2.111:0/3195342748 wait complete. 2021-04-03T03:08:47.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 Processor -- start 2021-04-03T03:08:47.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- start start 2021-04-03T03:08:47.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac0641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83ac0add50 0x7f83ac14d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83ac0b17b0 0x7f83ac1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac0a9cb0 con 0x7f83ac0603f0 2021-04-03T03:08:47.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac0a9360 con 0x7f83ac0b17b0 2021-04-03T03:08:47.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac0a9600 con 0x7f83ac0add50 2021-04-03T03:08:47.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac0641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac0641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49534/0 (socket says 172.21.2.111:49534) 2021-04-03T03:08:47.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 -- 172.21.2.111:0/675264036 learned_addr learned my addr 172.21.2.111:0/675264036 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 -- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83ac0add50 msgr2=0x7f83ac14d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83ac0add50 0x7f83ac14d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 -- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83ac0b17b0 msgr2=0x7f83ac1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83ac0b17b0 0x7f83ac1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 -- 172.21.2.111:0/675264036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac0a98a0 con 0x7f83ac0603f0 2021-04-03T03:08:47.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1330700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac0641a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f83ac0b5600 tx=0x7f839c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c011db0 con 0x7f83ac0603f0 2021-04-03T03:08:47.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c01e460 con 0x7f83ac0603f0 2021-04-03T03:08:47.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c01e7c0 con 0x7f83ac0603f0 2021-04-03T03:08:47.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac067e00 con 0x7f83ac0603f0 2021-04-03T03:08:47.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac156ed0 con 0x7f83ac0603f0 2021-04-03T03:08:47.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83ac0ae690 con 0x7f83ac0603f0 2021-04-03T03:08:47.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f839c025030 con 0x7f83ac0603f0 2021-04-03T03:08:47.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f838c056f70 0x7f838c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f839c0718f0 con 0x7f83ac0603f0 2021-04-03T03:08:47.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7f838c05eaf0 0x7f838c060f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 --> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f838c061570 con 0x7f838c05eaf0 2021-04-03T03:08:47.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b0b2f700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f838c056f70 0x7f838c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f839c076240 con 0x7f83ac0603f0 2021-04-03T03:08:47.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b0b2f700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f838c056f70 0x7f838c059430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f83a8002f50 tx=0x7f83a8006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.743+0000 7f83b1b31700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7f838c05eaf0 0x7f838c060f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.747+0000 7f83b1b31700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7f838c05eaf0 0x7f838c060f10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.747+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== osd.6 v2:172.21.2.115:6808/16015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f838c061570 con 0x7f838c05eaf0 2021-04-03T03:08:47.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 --> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83ac057660 con 0x7f838c05eaf0 2021-04-03T03:08:47.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f839a7fc700 1 -- 172.21.2.111:0/675264036 <== osd.6 v2:172.21.2.115:6808/16015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f83ac057660 con 0x7f838c05eaf0 2021-04-03T03:08:47.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7f838c05eaf0 msgr2=0x7f838c060f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7f838c05eaf0 0x7f838c060f10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f838c056f70 msgr2=0x7f838c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f838c056f70 0x7f838c059430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f83a8002f50 tx=0x7f83a8006990).stop 2021-04-03T03:08:47.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 msgr2=0x7f83ac0641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 --2- 172.21.2.111:0/675264036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83ac0603f0 0x7f83ac0641a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f83ac0b5600 tx=0x7f839c015000).stop 2021-04-03T03:08:47.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b1b31700 1 -- 172.21.2.111:0/675264036 reap_dead start 2021-04-03T03:08:47.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 shutdown_connections 2021-04-03T03:08:47.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 >> 172.21.2.111:0/675264036 conn(0x7f83ac0b61c0 msgr2=0x7f83ac1457d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 shutdown_connections 2021-04-03T03:08:47.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.759+0000 7f83b2332700 1 -- 172.21.2.111:0/675264036 wait complete. 2021-04-03T03:08:47.775 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:47.775 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2021-04-03T03:08:47.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 Processor -- start 2021-04-03T03:08:47.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- start start 2021-04-03T03:08:47.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb29005fce0 0x7fb2900ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb29013ef00 0x7fb2901429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29013f350 con 0x7fb2900b1990 2021-04-03T03:08:47.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2900b2430 con 0x7fb29013ef00 2021-04-03T03:08:47.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900600c0 con 0x7fb29005fce0 2021-04-03T03:08:47.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28dd9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fb2900b1990 0x7fb29013e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50336/0 (socket says 172.21.2.111:50336) 2021-04-03T03:08:47.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28dd9b700 1 -- 172.21.2.111:0/247316392 learned_addr learned my addr 172.21.2.111:0/247316392 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:47.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- 172.21.2.111:0/247316392 >> v2:172.21.2.111:3300/0 conn(0x7fb29013ef00 0x7fb2901429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 -- 172.21.2.111:0/247316392 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb29005fce0 msgr2=0x7fb2900ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- 172.21.2.111:0/247316392 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb29005fce0 0x7fb2900ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 -- 172.21.2.111:0/247316392 >> v1:172.21.2.111:6789/0 conn(0x7fb2900b1990 legacy=0x7fb29013e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 -- 172.21.2.111:0/247316392 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900afd90 con 0x7fb29013ef00 2021-04-03T03:08:47.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- 172.21.2.111:0/247316392 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb2901429f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb2880032b0 tx=0x7fb288009a60).ready entity=mon.0 client_cookie=c145a4eb81898550 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28d59a700 1 -- 172.21.2.111:0/247316392 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb288009780 con 0x7fb29013ef00 2021-04-03T03:08:47.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28d59a700 1 -- 172.21.2.111:0/247316392 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb288010410 con 0x7fb29013ef00 2021-04-03T03:08:47.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28d59a700 1 -- 172.21.2.111:0/247316392 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb288010770 con 0x7fb29013ef00 2021-04-03T03:08:47.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- 172.21.2.111:0/247316392 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 msgr2=0x7fb2901429f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- 172.21.2.111:0/247316392 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb2901429f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb2880032b0 tx=0x7fb288009a60).stop 2021-04-03T03:08:47.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- 172.21.2.111:0/247316392 shutdown_connections 2021-04-03T03:08:47.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- 172.21.2.111:0/247316392 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb2901429f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- 172.21.2.111:0/247316392 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb29005fce0 0x7fb2900ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- 172.21.2.111:0/247316392 >> 172.21.2.111:0/247316392 conn(0x7fb2900b5b50 msgr2=0x7fb2900b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- 172.21.2.111:0/247316392 shutdown_connections 2021-04-03T03:08:47.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- 172.21.2.111:0/247316392 wait complete. 2021-04-03T03:08:47.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 Processor -- start 2021-04-03T03:08:47.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- start start 2021-04-03T03:08:47.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb29005fce0 0x7fb290145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb2900b1990 0x7fb290067710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb290140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901422c0 con 0x7fb29013ef00 2021-04-03T03:08:47.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290146680 con 0x7fb2900b1990 2021-04-03T03:08:47.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb290142520 con 0x7fb29005fce0 2021-04-03T03:08:47.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb290140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb290140880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49550/0 (socket says 172.21.2.111:49550) 2021-04-03T03:08:47.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 -- 172.21.2.111:0/3280584514 learned_addr learned my addr 172.21.2.111:0/3280584514 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 -- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb29005fce0 msgr2=0x7fb290145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb29005fce0 0x7fb290145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 -- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb2900b1990 msgr2=0x7fb290067710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb2900b1990 0x7fb290067710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 -- 172.21.2.111:0/3280584514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290147d00 con 0x7fb29013ef00 2021-04-03T03:08:47.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28ed9d700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb290140880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb290146810 tx=0x7fb288017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28801a7f0 con 0x7fb29013ef00 2021-04-03T03:08:47.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28801a950 con 0x7fb29013ef00 2021-04-03T03:08:47.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb290140d70 con 0x7fb29013ef00 2021-04-03T03:08:47.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28801acb0 con 0x7fb29013ef00 2021-04-03T03:08:47.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.827+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb290144c80 con 0x7fb29013ef00 2021-04-03T03:08:47.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb290147e90 con 0x7fb29013ef00 2021-04-03T03:08:47.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb28800e040 con 0x7fb29013ef00 2021-04-03T03:08:47.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb2877fe700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb274056fb0 0x7fb274059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb288071670 con 0x7fb29013ef00 2021-04-03T03:08:47.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb2877fe700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7fb27405eab0 0x7fb274060ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 --> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb274061490 con 0x7fb27405eab0 2021-04-03T03:08:47.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb288073c30 con 0x7fb29013ef00 2021-04-03T03:08:47.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb28e59c700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb274056fb0 0x7fb274059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb28e59c700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb274056fb0 0x7fb274059470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fb280002f30 tx=0x7fb280006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb28dd9b700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7fb27405eab0 0x7fb274060ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.831+0000 7fb28dd9b700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7fb27405eab0 0x7fb274060ed0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.835+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== osd.7 v2:172.21.2.115:6824/16058 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fb274061490 con 0x7fb27405eab0 2021-04-03T03:08:47.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.843+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 --> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb290148760 con 0x7fb27405eab0 2021-04-03T03:08:47.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb2877fe700 1 -- 172.21.2.111:0/3280584514 <== osd.7 v2:172.21.2.115:6824/16058 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb290148760 con 0x7fb27405eab0 2021-04-03T03:08:47.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7fb27405eab0 msgr2=0x7fb274060ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7fb27405eab0 0x7fb274060ed0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb274056fb0 msgr2=0x7fb274059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb274056fb0 0x7fb274059470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fb280002f30 tx=0x7fb280006a60).stop 2021-04-03T03:08:47.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 msgr2=0x7fb290140880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 --2- 172.21.2.111:0/3280584514 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb29013ef00 0x7fb290140880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb290146810 tx=0x7fb288017000).stop 2021-04-03T03:08:47.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28ed9d700 1 -- 172.21.2.111:0/3280584514 reap_dead start 2021-04-03T03:08:47.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 shutdown_connections 2021-04-03T03:08:47.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 >> 172.21.2.111:0/3280584514 conn(0x7fb2900b5b50 msgr2=0x7fb2900b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 shutdown_connections 2021-04-03T03:08:47.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.847+0000 7fb28f59e700 1 -- 172.21.2.111:0/3280584514 wait complete. 2021-04-03T03:08:47.860 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:47.860 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2021-04-03T03:08:47.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 Processor -- start 2021-04-03T03:08:47.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- start start 2021-04-03T03:08:47.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0d42e0 0x7faf5c042120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7faf5c03ddf0 0x7faf5c042610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c0d4780 con 0x7faf5c047570 2021-04-03T03:08:47.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c047950 con 0x7faf5c03ddf0 2021-04-03T03:08:47.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0d0490 con 0x7faf5c0d42e0 2021-04-03T03:08:47.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7faf5c047570 0x7faf5c0d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50348/0 (socket says 172.21.2.111:50348) 2021-04-03T03:08:47.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 -- 172.21.2.111:0/1489893697 learned_addr learned my addr 172.21.2.111:0/1489893697 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:47.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151110170 0 0) 0x7faf5c0d4780 con 0x7faf5c047570 2021-04-03T03:08:47.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf44003240 con 0x7faf5c047570 2021-04-03T03:08:47.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf627fc700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0d42e0 0x7faf5c042120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf637fe700 1 --2- 172.21.2.111:0/1489893697 >> v2:172.21.2.115:3300/0 conn(0x7faf5c03ddf0 0x7faf5c042610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4169267889 0 0) 0x7faf44003240 con 0x7faf5c047570 2021-04-03T03:08:47.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0d42e0 msgr2=0x7faf5c042120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0d42e0 0x7faf5c042120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 >> v2:172.21.2.115:3300/0 conn(0x7faf5c03ddf0 msgr2=0x7faf5c042610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 --2- 172.21.2.111:0/1489893697 >> v2:172.21.2.115:3300/0 conn(0x7faf5c03ddf0 0x7faf5c042610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:47.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0d0730 con 0x7faf5c047570 2021-04-03T03:08:47.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7faf4c0016d0 con 0x7faf5c047570 2021-04-03T03:08:47.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 >> v1:172.21.2.111:6789/0 conn(0x7faf5c047570 legacy=0x7faf5c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf440037b0 0x7faf44003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf440044e0 0x7faf44004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0d42e0 0x7faf44004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c047950 con 0x7faf440037b0 2021-04-03T03:08:47.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0d0490 con 0x7faf440044e0 2021-04-03T03:08:47.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf44005340 con 0x7faf5c0d42e0 2021-04-03T03:08:47.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf440037b0 0x7faf44003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 -- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0d42e0 msgr2=0x7faf44004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0d42e0 0x7faf44004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 -- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf440044e0 msgr2=0x7faf44004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:47.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf440044e0 0x7faf44004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 -- 172.21.2.111:0/1489893697 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf44003240 con 0x7faf440037b0 2021-04-03T03:08:47.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf62ffd700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf440037b0 0x7faf44003bd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7faf4c0013e0 tx=0x7faf5c0d08c0).ready entity=mon.0 client_cookie=b64f2015b04d508e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf4c0070d0 con 0x7faf440037b0 2021-04-03T03:08:47.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c007230 con 0x7faf440037b0 2021-04-03T03:08:47.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf61ffb700 1 -- 172.21.2.111:0/1489893697 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf4c0055f0 con 0x7faf440037b0 2021-04-03T03:08:47.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- 172.21.2.111:0/1489893697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf440037b0 msgr2=0x7faf44003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 --2- 172.21.2.111:0/1489893697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf440037b0 0x7faf44003bd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7faf4c0013e0 tx=0x7faf5c0d08c0).stop 2021-04-03T03:08:47.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf637fe700 1 -- 172.21.2.111:0/1489893697 reap_dead start 2021-04-03T03:08:47.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- 172.21.2.111:0/1489893697 shutdown_connections 2021-04-03T03:08:47.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- 172.21.2.111:0/1489893697 >> 172.21.2.111:0/1489893697 conn(0x7faf5c00d990 msgr2=0x7faf5c046b40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:47.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- 172.21.2.111:0/1489893697 shutdown_connections 2021-04-03T03:08:47.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.911+0000 7faf63fff700 1 -- 172.21.2.111:0/1489893697 wait complete. 2021-04-03T03:08:47.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 Processor -- start 2021-04-03T03:08:47.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 -- start start 2021-04-03T03:08:47.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf5c0a29c0 0x7faf5c0dba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0df690 0x7faf5c0dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf5c0a2da0 0x7faf5c0a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0a3500 con 0x7faf5c0a2da0 2021-04-03T03:08:47.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c03ba50 con 0x7faf5c0a29c0 2021-04-03T03:08:47.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c03cab0 con 0x7faf5c0df690 2021-04-03T03:08:47.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf5c0a2da0 0x7faf5c0a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf5c0a2da0 0x7faf5c0a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49572/0 (socket says 172.21.2.111:49572) 2021-04-03T03:08:47.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 -- 172.21.2.111:0/2962581908 learned_addr learned my addr 172.21.2.111:0/2962581908 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:47.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 -- 172.21.2.111:0/2962581908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0df690 msgr2=0x7faf5c0dbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0df690 0x7faf5c0dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 -- 172.21.2.111:0/2962581908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf5c0a29c0 msgr2=0x7faf5c0dba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:47.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf5c0a29c0 0x7faf5c0dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:47.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 -- 172.21.2.111:0/2962581908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c03c810 con 0x7faf5c0a2da0 2021-04-03T03:08:47.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf637fe700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf5c0a2da0 0x7faf5c0a03a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7faf5c03bd20 tx=0x7faf64064040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf5bfff700 1 -- 172.21.2.111:0/2962581908 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6405df60 con 0x7faf5c0a2da0 2021-04-03T03:08:47.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf5bfff700 1 -- 172.21.2.111:0/2962581908 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf640699f0 con 0x7faf5c0a2da0 2021-04-03T03:08:47.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c0a3260 con 0x7faf5c0a2da0 2021-04-03T03:08:47.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c03cd20 con 0x7faf5c0a2da0 2021-04-03T03:08:47.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf5bfff700 1 -- 172.21.2.111:0/2962581908 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6405f4b0 con 0x7faf5c0a2da0 2021-04-03T03:08:47.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c004d30 con 0x7faf5c0a2da0 2021-04-03T03:08:47.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf5bfff700 1 -- 172.21.2.111:0/2962581908 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7faf6405f610 con 0x7faf5c0a2da0 2021-04-03T03:08:47.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf5bfff700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf48057030 0x7faf480594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:47.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf5bfff700 1 -- 172.21.2.111:0/2962581908 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7faf640df2e0 con 0x7faf5c0a2da0 2021-04-03T03:08:47.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf62ffd700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf48057030 0x7faf480594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:47.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.915+0000 7faf62ffd700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf48057030 0x7faf480594f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7faf4c0013e0 tx=0x7faf4c013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:47.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:47.919+0000 7faf5bfff700 1 -- 172.21.2.111:0/2962581908 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7faf640b4660 con 0x7faf5c0a2da0 2021-04-03T03:08:48.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.011+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faf5c0d0160 con 0x7faf5c0a2da0 2021-04-03T03:08:48.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.011+0000 7faf5bfff700 1 -- 172.21.2.111:0/2962581908 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf640b7c80 con 0x7faf5c0a2da0 2021-04-03T03:08:48.020 INFO:teuthology.orchestra.run.gibba011.stdout:55834574852 2021-04-03T03:08:48.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.011+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf48057030 msgr2=0x7faf480594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:48.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.011+0000 7faf63fff700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf48057030 0x7faf480594f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7faf4c0013e0 tx=0x7faf4c013000).stop 2021-04-03T03:08:48.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.011+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf5c0a2da0 msgr2=0x7faf5c0a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:48.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.011+0000 7faf63fff700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf5c0a2da0 0x7faf5c0a03a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7faf5c03bd20 tx=0x7faf64064040).stop 2021-04-03T03:08:48.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 shutdown_connections 2021-04-03T03:08:48.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf5c0a2da0 0x7faf5c0a03a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:48.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf48057030 0x7faf480594f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:48.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf5c0a29c0 0x7faf5c0dba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:48.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 --2- 172.21.2.111:0/2962581908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf5c0df690 0x7faf5c0dbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:48.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 >> 172.21.2.111:0/2962581908 conn(0x7faf5c00d990 msgr2=0x7faf5c0cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:48.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 shutdown_connections 2021-04-03T03:08:48.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:48.015+0000 7faf63fff700 1 -- 172.21.2.111:0/2962581908 wait complete. 2021-04-03T03:08:48.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2021-04-03T03:08:49.029 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2021-04-03T03:08:49.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 Processor -- start 2021-04-03T03:08:49.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 -- start start 2021-04-03T03:08:49.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 0x7fbec80b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbec80b5620 0x7fbec8142310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec80607d0 con 0x7fbec80603f0 2021-04-03T03:08:49.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec81433c0 con 0x7fbec80b5620 2021-04-03T03:08:49.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80b3f80 con 0x7fbec80b44a0 2021-04-03T03:08:49.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd1c7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbec80b5620 0x7fbec8142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecc9c6700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fbec80603f0 0x7fbec80b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50368/0 (socket says 172.21.2.111:50368) 2021-04-03T03:08:49.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecc9c6700 1 -- 172.21.2.111:0/890125707 learned_addr learned my addr 172.21.2.111:0/890125707 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:49.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd1c7700 1 -- 172.21.2.111:0/890125707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 msgr2=0x7fbec80b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd1c7700 1 --2- 172.21.2.111:0/890125707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 0x7fbec80b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd1c7700 1 -- 172.21.2.111:0/890125707 >> v1:172.21.2.111:6789/0 conn(0x7fbec80603f0 legacy=0x7fbec80b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd1c7700 1 -- 172.21.2.111:0/890125707 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80b4c60 con 0x7fbec80b5620 2021-04-03T03:08:49.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd1c7700 1 --2- 172.21.2.111:0/890125707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80b5620 0x7fbec8142310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fbec00032b0 tx=0x7fbec00091d0).ready entity=mon.0 client_cookie=aefe68d6c535a46 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbec77fe700 1 -- 172.21.2.111:0/890125707 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec000ba50 con 0x7fbec80b5620 2021-04-03T03:08:49.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbec77fe700 1 -- 172.21.2.111:0/890125707 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec000bbb0 con 0x7fbec80b5620 2021-04-03T03:08:49.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbec77fe700 1 -- 172.21.2.111:0/890125707 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec000bd20 con 0x7fbec80b5620 2021-04-03T03:08:49.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 -- 172.21.2.111:0/890125707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80b5620 msgr2=0x7fbec8142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.083+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/890125707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80b5620 0x7fbec8142310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fbec00032b0 tx=0x7fbec00091d0).stop 2021-04-03T03:08:49.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- 172.21.2.111:0/890125707 shutdown_connections 2021-04-03T03:08:49.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/890125707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80b5620 0x7fbec8142310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/890125707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 0x7fbec80b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- 172.21.2.111:0/890125707 >> 172.21.2.111:0/890125707 conn(0x7fbec805c7d0 msgr2=0x7fbec805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- 172.21.2.111:0/890125707 shutdown_connections 2021-04-03T03:08:49.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- 172.21.2.111:0/890125707 wait complete. 2021-04-03T03:08:49.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 Processor -- start 2021-04-03T03:08:49.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- start start 2021-04-03T03:08:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80603f0 0x7fbec80ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 0x7fbec8145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbec80b5620 0x7fbec8155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8113a40 con 0x7fbec80603f0 2021-04-03T03:08:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec81558d0 con 0x7fbec80b5620 2021-04-03T03:08:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80aec90 con 0x7fbec80b44a0 2021-04-03T03:08:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80603f0 0x7fbec80ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80603f0 0x7fbec80ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49580/0 (socket says 172.21.2.111:49580) 2021-04-03T03:08:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 -- 172.21.2.111:0/2117838564 learned_addr learned my addr 172.21.2.111:0/2117838564 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 -- 172.21.2.111:0/2117838564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 msgr2=0x7fbec8145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 0x7fbec8145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 -- 172.21.2.111:0/2117838564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbec80b5620 msgr2=0x7fbec8155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbec80b5620 0x7fbec8155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 -- 172.21.2.111:0/2117838564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80ad4d0 con 0x7fbec80603f0 2021-04-03T03:08:49.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecc9c6700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80603f0 0x7fbec80ae7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fbec80b5500 tx=0x7fbeb8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbec5ffb700 1 -- 172.21.2.111:0/2117838564 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeb8007320 con 0x7fbec80603f0 2021-04-03T03:08:49.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbec5ffb700 1 -- 172.21.2.111:0/2117838564 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb8007480 con 0x7fbec80603f0 2021-04-03T03:08:49.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec80ad820 con 0x7fbec80603f0 2021-04-03T03:08:49.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec81458f0 con 0x7fbec80603f0 2021-04-03T03:08:49.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbec5ffb700 1 -- 172.21.2.111:0/2117838564 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeb80057e0 con 0x7fbec80603f0 2021-04-03T03:08:49.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec8155a90 con 0x7fbec80603f0 2021-04-03T03:08:49.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbec5ffb700 1 -- 172.21.2.111:0/2117838564 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbeb8010030 con 0x7fbec80603f0 2021-04-03T03:08:49.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbec5ffb700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbeac056ea0 0x7fbeac059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbec5ffb700 1 -- 172.21.2.111:0/2117838564 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fbeb806ff20 con 0x7fbec80603f0 2021-04-03T03:08:49.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.087+0000 7fbec7fff700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbeac056ea0 0x7fbeac059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.091+0000 7fbec7fff700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbeac056ea0 0x7fbeac059360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fbeb0002f30 tx=0x7fbeb0006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.091+0000 7fbec5ffb700 1 -- 172.21.2.111:0/2117838564 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbeb80451f0 con 0x7fbec80603f0 2021-04-03T03:08:49.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbec81556e0 con 0x7fbec80603f0 2021-04-03T03:08:49.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbec5ffb700 1 -- 172.21.2.111:0/2117838564 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbeb8048810 con 0x7fbec80603f0 2021-04-03T03:08:49.192 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:49.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbeac056ea0 msgr2=0x7fbeac059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbeac056ea0 0x7fbeac059360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fbeb0002f30 tx=0x7fbeb0006a60).stop 2021-04-03T03:08:49.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80603f0 msgr2=0x7fbec80ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80603f0 0x7fbec80ae7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fbec80b5500 tx=0x7fbeb8007c20).stop 2021-04-03T03:08:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 shutdown_connections 2021-04-03T03:08:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbeac056ea0 0x7fbeac059360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbec80b5620 0x7fbec8155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbec80603f0 0x7fbec80ae7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 --2- 172.21.2.111:0/2117838564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbec80b44a0 0x7fbec8145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 >> 172.21.2.111:0/2117838564 conn(0x7fbec805c7d0 msgr2=0x7fbec81420c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 shutdown_connections 2021-04-03T03:08:49.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.183+0000 7fbecd9c8700 1 -- 172.21.2.111:0/2117838564 wait complete. 2021-04-03T03:08:49.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2021-04-03T03:08:49.199 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2021-04-03T03:08:49.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.247+0000 7fb1e8eb0700 1 Processor -- start 2021-04-03T03:08:49.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.247+0000 7fb1e8eb0700 1 -- start start 2021-04-03T03:08:49.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb1dc03f120 0x7fb1dc005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 0x7fb1dc0d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1dc03d840 con 0x7fb1dc041fe0 2021-04-03T03:08:49.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc03faa0 con 0x7fb1dc03f120 2021-04-03T03:08:49.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc03dca0 con 0x7fb1dc042470 2021-04-03T03:08:49.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e2ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fb1dc041fe0 0x7fb1dc0d89a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50382/0 (socket says 172.21.2.111:50382) 2021-04-03T03:08:49.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e2ffd700 1 -- 172.21.2.111:0/2073584844 learned_addr learned my addr 172.21.2.111:0/2073584844 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:49.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473870540 0 0) 0x7fb1dc03d840 con 0x7fb1dc041fe0 2021-04-03T03:08:49.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c8003240 con 0x7fb1dc041fe0 2021-04-03T03:08:49.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 0x7fb1dc0d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 --2- 172.21.2.111:0/2073584844 >> v2:172.21.2.115:3300/0 conn(0x7fb1dc03f120 0x7fb1dc005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4084612865 0 0) 0x7fb1c8003240 con 0x7fb1dc041fe0 2021-04-03T03:08:49.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 msgr2=0x7fb1dc0d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 0x7fb1dc0d90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 >> v2:172.21.2.115:3300/0 conn(0x7fb1dc03f120 msgr2=0x7fb1dc005640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 --2- 172.21.2.111:0/2073584844 >> v2:172.21.2.115:3300/0 conn(0x7fb1dc03f120 0x7fb1dc005640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc005b30 con 0x7fb1dc041fe0 2021-04-03T03:08:49.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fb1d00016d0 con 0x7fb1dc041fe0 2021-04-03T03:08:49.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 >> v1:172.21.2.111:6789/0 conn(0x7fb1dc041fe0 legacy=0x7fb1dc0d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c80037b0 0x7fb1c8003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 0x7fb1c80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1c80049d0 0x7fb1c8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc03faa0 con 0x7fb1c80037b0 2021-04-03T03:08:49.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc03dca0 con 0x7fb1c80049d0 2021-04-03T03:08:49.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80042a0 con 0x7fb1dc042470 2021-04-03T03:08:49.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c80037b0 0x7fb1c8003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 -- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 msgr2=0x7fb1c80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e2ffd700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 0x7fb1c80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc042470 0x7fb1c80044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 -- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1c80049d0 msgr2=0x7fb1c8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1c80049d0 0x7fb1c8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 -- 172.21.2.111:0/2073584844 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c8003240 con 0x7fb1c80037b0 2021-04-03T03:08:49.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e37fe700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c80037b0 0x7fb1c8003bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fb1e4066000 tx=0x7fb1e405ad70).ready entity=mon.0 client_cookie=7deae168973facc2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e405df60 con 0x7fb1c80037b0 2021-04-03T03:08:49.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e40699f0 con 0x7fb1c80037b0 2021-04-03T03:08:49.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e27fc700 1 -- 172.21.2.111:0/2073584844 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e405b040 con 0x7fb1c80037b0 2021-04-03T03:08:49.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/2073584844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c80037b0 msgr2=0x7fb1c8003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 --2- 172.21.2.111:0/2073584844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c80037b0 0x7fb1c8003bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fb1e4066000 tx=0x7fb1e405ad70).stop 2021-04-03T03:08:49.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 -- 172.21.2.111:0/2073584844 reap_dead start 2021-04-03T03:08:49.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/2073584844 shutdown_connections 2021-04-03T03:08:49.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/2073584844 >> 172.21.2.111:0/2073584844 conn(0x7fb1dc0462b0 msgr2=0x7fb1dc0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/2073584844 shutdown_connections 2021-04-03T03:08:49.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/2073584844 wait complete. 2021-04-03T03:08:49.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 Processor -- start 2021-04-03T03:08:49.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- start start 2021-04-03T03:08:49.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1dc0d7ab0 0x7fb1dc0dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc09fd30 0x7fb1dc0e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1dc0e76e0 0x7fb1dc0eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc0dd590 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc0d7e90 con 0x7fb1dc0d7ab0 2021-04-03T03:08:49.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e8eb0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc0ea890 con 0x7fb1dc09fd30 2021-04-03T03:08:49.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1dc0e76e0 0x7fb1dc0eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1dc0e76e0 0x7fb1dc0eafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49604/0 (socket says 172.21.2.111:49604) 2021-04-03T03:08:49.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 -- 172.21.2.111:0/3946428927 learned_addr learned my addr 172.21.2.111:0/3946428927 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:49.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 -- 172.21.2.111:0/3946428927 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc09fd30 msgr2=0x7fb1dc0e71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc09fd30 0x7fb1dc0e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 -- 172.21.2.111:0/3946428927 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1dc0d7ab0 msgr2=0x7fb1dc0dddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1dc0d7ab0 0x7fb1dc0dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 -- 172.21.2.111:0/3946428927 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc0eab30 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.251+0000 7fb1e3fff700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1dc0e76e0 0x7fb1dc0eafd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb1d8000b20 tx=0x7fb1d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e0ff9700 1 -- 172.21.2.111:0/3946428927 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d80106e0 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e0ff9700 1 -- 172.21.2.111:0/3946428927 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8010840 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e0ff9700 1 -- 172.21.2.111:0/3946428927 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d8010ba0 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc0de3e0 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc0eb4c0 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc0eacc0 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e0ff9700 1 -- 172.21.2.111:0/3946428927 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb1d801a030 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e0ff9700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1cc056fb0 0x7fb1cc059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e0ff9700 1 -- 172.21.2.111:0/3946428927 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb1d806fb10 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e37fe700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1cc056fb0 0x7fb1cc059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e37fe700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1cc056fb0 0x7fb1cc059470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb1e4066000 tx=0x7fb1e4082000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.255+0000 7fb1e0ff9700 1 -- 172.21.2.111:0/3946428927 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb1d8044e90 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb1dc004d30 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e0ff9700 1 -- 172.21.2.111:0/3946428927 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1d80484b0 con 0x7fb1dc0e76e0 2021-04-03T03:08:49.359 INFO:teuthology.orchestra.run.gibba011.stdout:55834574854 2021-04-03T03:08:49.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1cc056fb0 msgr2=0x7fb1cc059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1cc056fb0 0x7fb1cc059470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb1e4066000 tx=0x7fb1e4082000).stop 2021-04-03T03:08:49.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1dc0e76e0 msgr2=0x7fb1dc0eafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1dc0e76e0 0x7fb1dc0eafd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb1d8000b20 tx=0x7fb1d8008040).stop 2021-04-03T03:08:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 shutdown_connections 2021-04-03T03:08:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1dc0e76e0 0x7fb1dc0eafd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1cc056fb0 0x7fb1cc059470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1dc0d7ab0 0x7fb1dc0dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 --2- 172.21.2.111:0/3946428927 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1dc09fd30 0x7fb1dc0e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 >> 172.21.2.111:0/3946428927 conn(0x7fb1dc0462b0 msgr2=0x7fb1dc041590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 shutdown_connections 2021-04-03T03:08:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.351+0000 7fb1e8eb0700 1 -- 172.21.2.111:0/3946428927 wait complete. 2021-04-03T03:08:49.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2021-04-03T03:08:49.367 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2021-04-03T03:08:49.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 Processor -- start 2021-04-03T03:08:49.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 -- start start 2021-04-03T03:08:49.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0e500add50 0x7f0e50147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e500607d0 con 0x7f0e50144ef0 2021-04-03T03:08:49.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5014b090 con 0x7f0e500603f0 2021-04-03T03:08:49.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5014b330 con 0x7f0e500add50 2021-04-03T03:08:49.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e54da8700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0e50144ef0 0x7f0e50147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50400/0 (socket says 172.21.2.111:50400) 2021-04-03T03:08:49.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e54da8700 1 -- 172.21.2.111:0/2748145069 learned_addr learned my addr 172.21.2.111:0/2748145069 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:49.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619164955 0 0) 0x7f0e500607d0 con 0x7f0e50144ef0 2021-04-03T03:08:49.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e30003240 con 0x7f0e50144ef0 2021-04-03T03:08:49.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4f7fe700 1 --2- 172.21.2.111:0/2748145069 >> v2:172.21.2.115:3300/0 conn(0x7f0e500add50 0x7f0e50147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3249063206 0 0) 0x7f0e30003240 con 0x7f0e50144ef0 2021-04-03T03:08:49.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 >> v2:172.21.2.115:3300/0 conn(0x7f0e500add50 msgr2=0x7f0e50147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 --2- 172.21.2.111:0/2748145069 >> v2:172.21.2.115:3300/0 conn(0x7f0e500add50 0x7f0e50147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 >> v1:172.21.2.115:6789/0 conn(0x7f0e500603f0 legacy=0x7f0e501447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5014b5d0 con 0x7f0e50144ef0 2021-04-03T03:08:49.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f0e48001a50 con 0x7f0e50144ef0 2021-04-03T03:08:49.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 >> v1:172.21.2.111:6789/0 conn(0x7f0e50144ef0 legacy=0x7f0e50147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e30003800 0x7f0e30003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e300044e0 0x7f0e30004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e30004e50 0x7f0e3000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5014b330 con 0x7f0e30003800 2021-04-03T03:08:49.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30005300 con 0x7f0e30004e50 2021-04-03T03:08:49.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3000bc20 con 0x7f0e300044e0 2021-04-03T03:08:49.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4ffff700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e30003800 0x7f0e30003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4ffff700 1 -- 172.21.2.111:0/2748145069 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e300044e0 msgr2=0x7f0e30004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4ffff700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e300044e0 0x7f0e30004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4ffff700 1 -- 172.21.2.111:0/2748145069 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e30004e50 msgr2=0x7f0e3000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4ffff700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e30004e50 0x7f0e3000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4ffff700 1 -- 172.21.2.111:0/2748145069 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30003240 con 0x7f0e30003800 2021-04-03T03:08:49.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e54da8700 1 -- 172.21.2.111:0/2748145069 reap_dead start 2021-04-03T03:08:49.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4ffff700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e30003800 0x7f0e30003c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0e38001f60 tx=0x7f0e38007b70).ready entity=mon.0 client_cookie=8ea70e7eb8b749ef server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e3800c870 con 0x7f0e30003800 2021-04-03T03:08:49.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3800c9d0 con 0x7f0e30003800 2021-04-03T03:08:49.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e4effd700 1 -- 172.21.2.111:0/2748145069 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e3800cd30 con 0x7f0e30003800 2021-04-03T03:08:49.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 -- 172.21.2.111:0/2748145069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e30003800 msgr2=0x7f0e30003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.415+0000 7f0e555a9700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e30003800 0x7f0e30003c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0e38001f60 tx=0x7f0e38007b70).stop 2021-04-03T03:08:49.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- 172.21.2.111:0/2748145069 shutdown_connections 2021-04-03T03:08:49.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 --2- 172.21.2.111:0/2748145069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e30003800 0x7f0e30003c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- 172.21.2.111:0/2748145069 >> 172.21.2.111:0/2748145069 conn(0x7f0e500b61c0 msgr2=0x7f0e5005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- 172.21.2.111:0/2748145069 shutdown_connections 2021-04-03T03:08:49.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- 172.21.2.111:0/2748145069 wait complete. 2021-04-03T03:08:49.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 Processor -- start 2021-04-03T03:08:49.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- start start 2021-04-03T03:08:49.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e30003800 0x7f0e500a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e50067cb0 0x7f0e501462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e50147ef0 0x7f0e501467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500aa800 con 0x7f0e50147ef0 2021-04-03T03:08:49.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500aaaa0 con 0x7f0e50067cb0 2021-04-03T03:08:49.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e50157120 con 0x7f0e30003800 2021-04-03T03:08:49.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e50147ef0 0x7f0e501467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e50147ef0 0x7f0e501467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49624/0 (socket says 172.21.2.111:49624) 2021-04-03T03:08:49.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 -- 172.21.2.111:0/759072538 learned_addr learned my addr 172.21.2.111:0/759072538 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:49.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 -- 172.21.2.111:0/759072538 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e30003800 msgr2=0x7f0e500a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e30003800 0x7f0e500a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 -- 172.21.2.111:0/759072538 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e50067cb0 msgr2=0x7f0e501462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e50067cb0 0x7f0e501462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 -- 172.21.2.111:0/759072538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500aa4a0 con 0x7f0e50147ef0 2021-04-03T03:08:49.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e54da8700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e50147ef0 0x7f0e501467d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0e500af1d0 tx=0x7f0e5014b220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4d7fa700 1 -- 172.21.2.111:0/759072538 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e48004a90 con 0x7f0e50147ef0 2021-04-03T03:08:49.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4d7fa700 1 -- 172.21.2.111:0/759072538 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48004bf0 con 0x7f0e50147ef0 2021-04-03T03:08:49.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4d7fa700 1 -- 172.21.2.111:0/759072538 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e48007030 con 0x7f0e50147ef0 2021-04-03T03:08:49.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e501575c0 con 0x7f0e50147ef0 2021-04-03T03:08:49.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e500aae60 con 0x7f0e50147ef0 2021-04-03T03:08:49.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e50156f70 con 0x7f0e50147ef0 2021-04-03T03:08:49.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4d7fa700 1 -- 172.21.2.111:0/759072538 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0e4801d070 con 0x7f0e50147ef0 2021-04-03T03:08:49.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4d7fa700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e34056ea0 0x7f0e34059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4d7fa700 1 -- 172.21.2.111:0/759072538 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0e4806f4b0 con 0x7f0e50147ef0 2021-04-03T03:08:49.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4ffff700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e34056ea0 0x7f0e34059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.419+0000 7f0e4ffff700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e34056ea0 0x7f0e34059360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f0e38001f60 tx=0x7f0e38016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.423+0000 7f0e4d7fa700 1 -- 172.21.2.111:0/759072538 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0e48044890 con 0x7f0e50147ef0 2021-04-03T03:08:49.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.515+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0e501572e0 con 0x7f0e50147ef0 2021-04-03T03:08:49.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.515+0000 7f0e4d7fa700 1 -- 172.21.2.111:0/759072538 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e48047eb0 con 0x7f0e50147ef0 2021-04-03T03:08:49.525 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:49.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e34056ea0 msgr2=0x7f0e34059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e34056ea0 0x7f0e34059360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f0e38001f60 tx=0x7f0e38016000).stop 2021-04-03T03:08:49.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e50147ef0 msgr2=0x7f0e501467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e50147ef0 0x7f0e501467d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0e500af1d0 tx=0x7f0e5014b220).stop 2021-04-03T03:08:49.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 shutdown_connections 2021-04-03T03:08:49.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e50147ef0 0x7f0e501467d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e30003800 0x7f0e500a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e34056ea0 0x7f0e34059360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 --2- 172.21.2.111:0/759072538 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e50067cb0 0x7f0e501462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 >> 172.21.2.111:0/759072538 conn(0x7f0e500b61c0 msgr2=0x7f0e500b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 shutdown_connections 2021-04-03T03:08:49.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.519+0000 7f0e555a9700 1 -- 172.21.2.111:0/759072538 wait complete. 2021-04-03T03:08:49.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2021-04-03T03:08:49.532 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2021-04-03T03:08:49.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.579+0000 7f1473fff700 1 Processor -- start 2021-04-03T03:08:49.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.579+0000 7f1473fff700 1 -- start start 2021-04-03T03:08:49.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f14740603f0 0x7f14741447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14740607d0 con 0x7f14740b17b0 2021-04-03T03:08:49.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14741451c0 con 0x7f14740603f0 2021-04-03T03:08:49.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147414b090 con 0x7f14740add50 2021-04-03T03:08:49.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:32976/0 (socket says 172.21.2.111:32976) 2021-04-03T03:08:49.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 -- 172.21.2.111:0/555938510 learned_addr learned my addr 172.21.2.111:0/555938510 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:49.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1472ffd700 1 --2- 172.21.2.111:0/555938510 >> v2:172.21.2.115:3300/0 conn(0x7f14740603f0 0x7f14741447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 -- 172.21.2.111:0/555938510 >> v2:172.21.2.115:3300/0 conn(0x7f14740603f0 msgr2=0x7f14741447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 --2- 172.21.2.111:0/555938510 >> v2:172.21.2.115:3300/0 conn(0x7f14740603f0 0x7f14741447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 -- 172.21.2.111:0/555938510 >> v1:172.21.2.115:6789/0 conn(0x7f14740b17b0 legacy=0x7f1474147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 -- 172.21.2.111:0/555938510 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147414b330 con 0x7f14740add50 2021-04-03T03:08:49.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f14727fc700 1 --2- 172.21.2.111:0/555938510 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474144cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1464002f30 tx=0x7f1464006e90).ready entity=mon.2 client_cookie=a3996a594bc6fdba server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1471ffb700 1 -- 172.21.2.111:0/555938510 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1464009850 con 0x7f14740add50 2021-04-03T03:08:49.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1471ffb700 1 -- 172.21.2.111:0/555938510 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14640099b0 con 0x7f14740add50 2021-04-03T03:08:49.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1471ffb700 1 -- 172.21.2.111:0/555938510 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1464009d10 con 0x7f14740add50 2021-04-03T03:08:49.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- 172.21.2.111:0/555938510 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 msgr2=0x7f1474144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 --2- 172.21.2.111:0/555938510 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474144cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1464002f30 tx=0x7f1464006e90).stop 2021-04-03T03:08:49.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- 172.21.2.111:0/555938510 shutdown_connections 2021-04-03T03:08:49.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 --2- 172.21.2.111:0/555938510 >> v2:172.21.2.115:3300/0 conn(0x7f14740603f0 0x7f14741447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 --2- 172.21.2.111:0/555938510 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474144cd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- 172.21.2.111:0/555938510 >> 172.21.2.111:0/555938510 conn(0x7f14740b61c0 msgr2=0x7f147405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- 172.21.2.111:0/555938510 shutdown_connections 2021-04-03T03:08:49.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 -- 172.21.2.111:0/555938510 wait complete. 2021-04-03T03:08:49.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.583+0000 7f1473fff700 1 Processor -- start 2021-04-03T03:08:49.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 -- start start 2021-04-03T03:08:49.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14740603f0 0x7f14740ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14740b17b0 0x7f14741571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1474156940 con 0x7f14740603f0 2021-04-03T03:08:49.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1474068010 con 0x7f14740b17b0 2021-04-03T03:08:49.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14740634e0 con 0x7f14740add50 2021-04-03T03:08:49.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14740603f0 0x7f14740ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14740603f0 0x7f14740ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49632/0 (socket says 172.21.2.111:49632) 2021-04-03T03:08:49.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 -- 172.21.2.111:0/3459532547 learned_addr learned my addr 172.21.2.111:0/3459532547 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:49.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f14737fe700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14740b17b0 0x7f14741571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f14727fc700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 -- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 msgr2=0x7f1474156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 -- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14740b17b0 msgr2=0x7f14741571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14740b17b0 0x7f14741571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 -- 172.21.2.111:0/3459532547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1474067d70 con 0x7f14740603f0 2021-04-03T03:08:49.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1472ffd700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14740603f0 0x7f14740ab3b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1474063670 tx=0x7f1468006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1463fff700 1 -- 172.21.2.111:0/3459532547 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146800cae0 con 0x7f14740603f0 2021-04-03T03:08:49.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1463fff700 1 -- 172.21.2.111:0/3459532547 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146800cc40 con 0x7f14740603f0 2021-04-03T03:08:49.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14740aa8a0 con 0x7f14740603f0 2021-04-03T03:08:49.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14740aa390 con 0x7f14740603f0 2021-04-03T03:08:49.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1463fff700 1 -- 172.21.2.111:0/3459532547 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14680123f0 con 0x7f14740603f0 2021-04-03T03:08:49.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1474057660 con 0x7f14740603f0 2021-04-03T03:08:49.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1463fff700 1 -- 172.21.2.111:0/3459532547 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1468012550 con 0x7f14740603f0 2021-04-03T03:08:49.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1463fff700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1450056f20 0x7f14500593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f1463fff700 1 -- 172.21.2.111:0/3459532547 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f1468070830 con 0x7f14740603f0 2021-04-03T03:08:49.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f14727fc700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1450056f20 0x7f14500593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.587+0000 7f14727fc700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1450056f20 0x7f14500593e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1464017000 tx=0x7f1464015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.591+0000 7f1463fff700 1 -- 172.21.2.111:0/3459532547 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1468045aa0 con 0x7f14740603f0 2021-04-03T03:08:49.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f14740aaa30 con 0x7f14740603f0 2021-04-03T03:08:49.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1463fff700 1 -- 172.21.2.111:0/3459532547 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14680490c0 con 0x7f14740603f0 2021-04-03T03:08:49.690 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:49.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1450056f20 msgr2=0x7f14500593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1450056f20 0x7f14500593e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1464017000 tx=0x7f1464015000).stop 2021-04-03T03:08:49.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14740603f0 msgr2=0x7f14740ab3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14740603f0 0x7f14740ab3b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1474063670 tx=0x7f1468006dd0).stop 2021-04-03T03:08:49.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 shutdown_connections 2021-04-03T03:08:49.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1450056f20 0x7f14500593e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14740b17b0 0x7f14741571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14740603f0 0x7f14740ab3b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 --2- 172.21.2.111:0/3459532547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14740add50 0x7f1474156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 >> 172.21.2.111:0/3459532547 conn(0x7f14740b61c0 msgr2=0x7f1474144980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 shutdown_connections 2021-04-03T03:08:49.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.683+0000 7f1473fff700 1 -- 172.21.2.111:0/3459532547 wait complete. 2021-04-03T03:08:49.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2021-04-03T03:08:49.698 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2021-04-03T03:08:49.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 Processor -- start 2021-04-03T03:08:49.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- start start 2021-04-03T03:08:49.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0add50 0x7f615c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c0607d0 con 0x7f615c144ef0 2021-04-03T03:08:49.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c14b090 con 0x7f615c0603f0 2021-04-03T03:08:49.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c14b330 con 0x7f615c0add50 2021-04-03T03:08:49.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f615c144ef0 0x7f615c147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50432/0 (socket says 172.21.2.111:50432) 2021-04-03T03:08:49.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 -- 172.21.2.111:0/2301703501 learned_addr learned my addr 172.21.2.111:0/2301703501 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:49.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507648375 0 0) 0x7f615c0607d0 con 0x7f615c144ef0 2021-04-03T03:08:49.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f613c003240 con 0x7f615c144ef0 2021-04-03T03:08:49.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1736012000 0 0) 0x7f613c003240 con 0x7f615c144ef0 2021-04-03T03:08:49.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0add50 msgr2=0x7f615c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0add50 0x7f615c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 >> v1:172.21.2.115:6789/0 conn(0x7f615c0603f0 legacy=0x7f615c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c14b5d0 con 0x7f615c144ef0 2021-04-03T03:08:49.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f6154001a50 con 0x7f615c144ef0 2021-04-03T03:08:49.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 >> v1:172.21.2.111:6789/0 conn(0x7f615c144ef0 legacy=0x7f615c147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f613c0037e0 0x7f613c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0add50 0x7f613c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f613c0049d0 0x7f613c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c14b330 con 0x7f613c0049d0 2021-04-03T03:08:49.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0042b0 con 0x7f613c0037e0 2021-04-03T03:08:49.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c00b730 con 0x7f615c0add50 2021-04-03T03:08:49.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f613c0049d0 0x7f613c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 -- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0add50 msgr2=0x7f613c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0add50 0x7f613c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 -- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f613c0037e0 msgr2=0x7f613c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f613c0037e0 0x7f613c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 -- 172.21.2.111:0/2301703501 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c003240 con 0x7f613c0049d0 2021-04-03T03:08:49.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f613c0049d0 0x7f613c004e50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f615c0b5640 tx=0x7f615c14b760).ready entity=mon.0 client_cookie=3940e459419c8c66 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6154005b80 con 0x7f613c0049d0 2021-04-03T03:08:49.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154005ce0 con 0x7f613c0049d0 2021-04-03T03:08:49.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615959a700 1 -- 172.21.2.111:0/2301703501 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61540072c0 con 0x7f613c0049d0 2021-04-03T03:08:49.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- 172.21.2.111:0/2301703501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f613c0049d0 msgr2=0x7f613c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 --2- 172.21.2.111:0/2301703501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f613c0049d0 0x7f613c004e50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f615c0b5640 tx=0x7f615c14b760).stop 2021-04-03T03:08:49.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615ad9d700 1 -- 172.21.2.111:0/2301703501 reap_dead start 2021-04-03T03:08:49.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- 172.21.2.111:0/2301703501 shutdown_connections 2021-04-03T03:08:49.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- 172.21.2.111:0/2301703501 >> 172.21.2.111:0/2301703501 conn(0x7f615c0b61c0 msgr2=0x7f615c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- 172.21.2.111:0/2301703501 shutdown_connections 2021-04-03T03:08:49.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.747+0000 7f615b59e700 1 -- 172.21.2.111:0/2301703501 wait complete. 2021-04-03T03:08:49.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 Processor -- start 2021-04-03T03:08:49.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 -- start start 2021-04-03T03:08:49.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0a7240 0x7f615c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f615c0abbf0 0x7f615c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f615c145310 0x7f615c159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c148100 con 0x7f615c0abbf0 2021-04-03T03:08:49.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c1572e0 con 0x7f615c145310 2021-04-03T03:08:49.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c0a7620 con 0x7f615c0a7240 2021-04-03T03:08:49.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f615c0abbf0 0x7f615c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f615c0abbf0 0x7f615c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49650/0 (socket says 172.21.2.111:49650) 2021-04-03T03:08:49.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 -- 172.21.2.111:0/4063181824 learned_addr learned my addr 172.21.2.111:0/4063181824 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:49.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 -- 172.21.2.111:0/4063181824 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0a7240 msgr2=0x7f615c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0a7240 0x7f615c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 -- 172.21.2.111:0/4063181824 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f615c145310 msgr2=0x7f615c159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f615c145310 0x7f615c159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 -- 172.21.2.111:0/4063181824 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c157780 con 0x7f615c0abbf0 2021-04-03T03:08:49.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f6159d9b700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f615c0abbf0 0x7f615c0b3a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f614c002f30 tx=0x7f614c006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f61537fe700 1 -- 172.21.2.111:0/4063181824 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f614c00d7c0 con 0x7f615c0abbf0 2021-04-03T03:08:49.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f61537fe700 1 -- 172.21.2.111:0/4063181824 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00d920 con 0x7f615c0abbf0 2021-04-03T03:08:49.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c147d80 con 0x7f615c0abbf0 2021-04-03T03:08:49.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f61537fe700 1 -- 172.21.2.111:0/4063181824 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f614c00dcc0 con 0x7f615c0abbf0 2021-04-03T03:08:49.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c067f90 con 0x7f615c0abbf0 2021-04-03T03:08:49.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c157110 con 0x7f615c0abbf0 2021-04-03T03:08:49.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f61537fe700 1 -- 172.21.2.111:0/4063181824 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f614c011460 con 0x7f615c0abbf0 2021-04-03T03:08:49.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f61537fe700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6140056ea0 0x7f6140059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f61537fe700 1 -- 172.21.2.111:0/4063181824 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f614c06fdc0 con 0x7f615c0abbf0 2021-04-03T03:08:49.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615a59c700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6140056ea0 0x7f6140059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.751+0000 7f615a59c700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6140056ea0 0x7f6140059360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6144000f30 tx=0x7f6144006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.755+0000 7f61537fe700 1 -- 172.21.2.111:0/4063181824 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f614c045140 con 0x7f615c0abbf0 2021-04-03T03:08:49.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f615c057660 con 0x7f615c0abbf0 2021-04-03T03:08:49.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f61537fe700 1 -- 172.21.2.111:0/4063181824 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f614c048760 con 0x7f615c0abbf0 2021-04-03T03:08:49.855 INFO:teuthology.orchestra.run.gibba011.stdout:55834574854 2021-04-03T03:08:49.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6140056ea0 msgr2=0x7f6140059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6140056ea0 0x7f6140059360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6144000f30 tx=0x7f6144006990).stop 2021-04-03T03:08:49.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f615c0abbf0 msgr2=0x7f615c0b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f615c0abbf0 0x7f615c0b3a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f614c002f30 tx=0x7f614c006e20).stop 2021-04-03T03:08:49.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 shutdown_connections 2021-04-03T03:08:49.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6140056ea0 0x7f6140059360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f615c145310 0x7f615c159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f615c0a7240 0x7f615c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 --2- 172.21.2.111:0/4063181824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f615c0abbf0 0x7f615c0b3a10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 >> 172.21.2.111:0/4063181824 conn(0x7f615c0b61c0 msgr2=0x7f615c143ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 shutdown_connections 2021-04-03T03:08:49.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.847+0000 7f615b59e700 1 -- 172.21.2.111:0/4063181824 wait complete. 2021-04-03T03:08:49.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2021-04-03T03:08:49.863 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2021-04-03T03:08:49.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd2995700 1 Processor -- start 2021-04-03T03:08:49.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd2995700 1 -- start start 2021-04-03T03:08:49.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd2995700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3ccc0add50 0x7f3ccc147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd2995700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0607d0 con 0x7f3ccc0603f0 2021-04-03T03:08:49.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd2995700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc14b090 con 0x7f3ccc144ef0 2021-04-03T03:08:49.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd2995700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc14b330 con 0x7f3ccc0add50 2021-04-03T03:08:49.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd1993700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3ccc0603f0 0x7f3ccc1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50450/0 (socket says 172.21.2.111:50450) 2021-04-03T03:08:49.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd1993700 1 -- 172.21.2.111:0/1223806668 learned_addr learned my addr 172.21.2.111:0/1223806668 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:49.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185408907 0 0) 0x7f3ccc0607d0 con 0x7f3ccc0603f0 2021-04-03T03:08:49.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb4003240 con 0x7f3ccc0603f0 2021-04-03T03:08:49.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 92989972 0 0) 0x7f3cb4003240 con 0x7f3ccc0603f0 2021-04-03T03:08:49.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 >> v2:172.21.2.115:3300/0 conn(0x7f3ccc0add50 msgr2=0x7f3ccc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 --2- 172.21.2.111:0/1223806668 >> v2:172.21.2.115:3300/0 conn(0x7f3ccc0add50 0x7f3ccc147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 >> v1:172.21.2.115:6789/0 conn(0x7f3ccc144ef0 legacy=0x7f3ccc147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc14b5d0 con 0x7f3ccc0603f0 2021-04-03T03:08:49.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f3cc80016d0 con 0x7f3ccc0603f0 2021-04-03T03:08:49.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 >> v1:172.21.2.111:6789/0 conn(0x7f3ccc0603f0 legacy=0x7f3ccc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3cb4003800 0x7f3cb4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3cb40044e0 0x7f3cb4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb4004e50 0x7f3cb400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc14b330 con 0x7f3cb40044e0 2021-04-03T03:08:49.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4005300 con 0x7f3cb4003800 2021-04-03T03:08:49.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb400bc20 con 0x7f3cb4004e50 2021-04-03T03:08:49.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.911+0000 7f3cd1192700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3cb40044e0 0x7f3cb4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 -- 172.21.2.111:0/1223806668 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb4004e50 msgr2=0x7f3cb400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb4004e50 0x7f3cb400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 -- 172.21.2.111:0/1223806668 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3cb4003800 msgr2=0x7f3cb4003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:49.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3cb4003800 0x7f3cb4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:49.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 -- 172.21.2.111:0/1223806668 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4003240 con 0x7f3cb40044e0 2021-04-03T03:08:49.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2194700 1 -- 172.21.2.111:0/1223806668 reap_dead start 2021-04-03T03:08:49.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3cb40044e0 0x7f3cb4004960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f3cc0002f30 tx=0x7f3cc0006e10).ready entity=mon.0 client_cookie=11ab4796c1254a6d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc00097f0 con 0x7f3cb40044e0 2021-04-03T03:08:49.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0009950 con 0x7f3cb40044e0 2021-04-03T03:08:49.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd0991700 1 -- 172.21.2.111:0/1223806668 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc0009cb0 con 0x7f3cb40044e0 2021-04-03T03:08:49.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- 172.21.2.111:0/1223806668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3cb40044e0 msgr2=0x7f3cb4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3cb40044e0 0x7f3cb4004960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f3cc0002f30 tx=0x7f3cc0006e10).stop 2021-04-03T03:08:49.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- 172.21.2.111:0/1223806668 shutdown_connections 2021-04-03T03:08:49.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 --2- 172.21.2.111:0/1223806668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3cb40044e0 0x7f3cb4004960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- 172.21.2.111:0/1223806668 >> 172.21.2.111:0/1223806668 conn(0x7f3ccc0b61c0 msgr2=0x7f3ccc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:49.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- 172.21.2.111:0/1223806668 shutdown_connections 2021-04-03T03:08:49.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- 172.21.2.111:0/1223806668 wait complete. 2021-04-03T03:08:49.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 Processor -- start 2021-04-03T03:08:49.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- start start 2021-04-03T03:08:49.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb40044e0 0x7f3ccc0a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ccc147d70 0x7f3ccc0a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ccc1566c0 0x7f3ccc156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0aa6b0 con 0x7f3ccc147d70 2021-04-03T03:08:49.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0aa950 con 0x7f3ccc1566c0 2021-04-03T03:08:49.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0a7aa0 con 0x7f3cb40044e0 2021-04-03T03:08:49.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ccc147d70 0x7f3ccc0a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ccc147d70 0x7f3ccc0a7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49674/0 (socket says 172.21.2.111:49674) 2021-04-03T03:08:49.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 -- 172.21.2.111:0/2362437217 learned_addr learned my addr 172.21.2.111:0/2362437217 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:49.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1993700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb40044e0 0x7f3ccc0a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2194700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ccc1566c0 0x7f3ccc156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 -- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb40044e0 msgr2=0x7f3ccc0a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb40044e0 0x7f3ccc0a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 -- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ccc1566c0 msgr2=0x7f3ccc156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:49.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ccc1566c0 0x7f3ccc156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:49.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 -- 172.21.2.111:0/2362437217 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc146e50 con 0x7f3ccc147d70 2021-04-03T03:08:49.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd1192700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ccc147d70 0x7f3ccc0a7f40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f3cc0002f30 tx=0x7f3cc0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cbeffd700 1 -- 172.21.2.111:0/2362437217 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc0011ce0 con 0x7f3ccc147d70 2021-04-03T03:08:49.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cbeffd700 1 -- 172.21.2.111:0/2362437217 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0011e40 con 0x7f3ccc147d70 2021-04-03T03:08:49.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc0a8430 con 0x7f3ccc147d70 2021-04-03T03:08:49.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc147b30 con 0x7f3ccc147d70 2021-04-03T03:08:49.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.915+0000 7f3cbeffd700 1 -- 172.21.2.111:0/2362437217 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc001e650 con 0x7f3ccc147d70 2021-04-03T03:08:49.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.919+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc057660 con 0x7f3ccc147d70 2021-04-03T03:08:49.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.919+0000 7f3cbeffd700 1 -- 172.21.2.111:0/2362437217 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3cc0025030 con 0x7f3ccc147d70 2021-04-03T03:08:49.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.919+0000 7f3cbeffd700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3cac056f70 0x7f3cac059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:49.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.919+0000 7f3cbeffd700 1 -- 172.21.2.111:0/2362437217 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f3cc0071760 con 0x7f3ccc147d70 2021-04-03T03:08:49.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.919+0000 7f3cd1993700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3cac056f70 0x7f3cac059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:49.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.919+0000 7f3cd1993700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3cac056f70 0x7f3cac059430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f3cc800b000 tx=0x7f3ccc14b220).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:49.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:49.919+0000 7f3cbeffd700 1 -- 172.21.2.111:0/2362437217 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3cc0046a30 con 0x7f3ccc147d70 2021-04-03T03:08:50.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3ccc148850 con 0x7f3ccc147d70 2021-04-03T03:08:50.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cbeffd700 1 -- 172.21.2.111:0/2362437217 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3cc004a050 con 0x7f3ccc147d70 2021-04-03T03:08:50.023 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:50.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3cac056f70 msgr2=0x7f3cac059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:50.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3cac056f70 0x7f3cac059430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f3cc800b000 tx=0x7f3ccc14b220).stop 2021-04-03T03:08:50.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ccc147d70 msgr2=0x7f3ccc0a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:50.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ccc147d70 0x7f3ccc0a7f40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f3cc0002f30 tx=0x7f3cc0015000).stop 2021-04-03T03:08:50.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 shutdown_connections 2021-04-03T03:08:50.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3cac056f70 0x7f3cac059430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ccc1566c0 0x7f3ccc156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3cb40044e0 0x7f3ccc0a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 --2- 172.21.2.111:0/2362437217 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ccc147d70 0x7f3ccc0a7f40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 >> 172.21.2.111:0/2362437217 conn(0x7f3ccc0b61c0 msgr2=0x7f3ccc0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:50.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 shutdown_connections 2021-04-03T03:08:50.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.015+0000 7f3cd2995700 1 -- 172.21.2.111:0/2362437217 wait complete. 2021-04-03T03:08:50.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2021-04-03T03:08:50.031 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2021-04-03T03:08:50.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 Processor -- start 2021-04-03T03:08:50.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 -- start start 2021-04-03T03:08:50.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900b8350 0x7f7990141d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:50.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79900b24d0 0x7f79900ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:50.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f79900ad660 0x7f7990129ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:50.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79900b8730 con 0x7f79900b24d0 2021-04-03T03:08:50.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7990141350 con 0x7f79900ad660 2021-04-03T03:08:50.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79901415f0 con 0x7f79900b8350 2021-04-03T03:08:50.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79900b24d0 0x7f79900ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:50.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79900b24d0 0x7f79900ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49678/0 (socket says 172.21.2.111:49678) 2021-04-03T03:08:50.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 -- 172.21.2.111:0/4062158645 learned_addr learned my addr 172.21.2.111:0/4062158645 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:50.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 -- 172.21.2.111:0/4062158645 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900b8350 msgr2=0x7f7990141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:50.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 --2- 172.21.2.111:0/4062158645 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900b8350 0x7f7990141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:50.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 -- 172.21.2.111:0/4062158645 >> v2:172.21.2.115:3300/0 conn(0x7f79900ad660 msgr2=0x7f7990129ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:50.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 --2- 172.21.2.111:0/4062158645 >> v2:172.21.2.115:3300/0 conn(0x7f79900ad660 0x7f7990129ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:50.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 -- 172.21.2.111:0/4062158645 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7990141890 con 0x7f79900b24d0 2021-04-03T03:08:50.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799519d700 1 --2- 172.21.2.111:0/4062158645 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b24d0 0x7f79900ad170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7984002f30 tx=0x7f7984006e90).ready entity=mon.0 client_cookie=8af79d07e0f0d667 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:50.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799499c700 1 -- 172.21.2.111:0/4062158645 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7984009920 con 0x7f79900b24d0 2021-04-03T03:08:50.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799499c700 1 -- 172.21.2.111:0/4062158645 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984009a80 con 0x7f79900b24d0 2021-04-03T03:08:50.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f799499c700 1 -- 172.21.2.111:0/4062158645 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7984009de0 con 0x7f79900b24d0 2021-04-03T03:08:50.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 -- 172.21.2.111:0/4062158645 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b24d0 msgr2=0x7f79900ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:50.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.079+0000 7f79969a0700 1 --2- 172.21.2.111:0/4062158645 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b24d0 0x7f79900ad170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7984002f30 tx=0x7f7984006e90).stop 2021-04-03T03:08:50.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- 172.21.2.111:0/4062158645 shutdown_connections 2021-04-03T03:08:50.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 --2- 172.21.2.111:0/4062158645 >> v2:172.21.2.115:3300/0 conn(0x7f79900ad660 0x7f7990129ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:50.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 --2- 172.21.2.111:0/4062158645 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900b8350 0x7f7990141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:50.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 --2- 172.21.2.111:0/4062158645 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b24d0 0x7f79900ad170 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- 172.21.2.111:0/4062158645 >> 172.21.2.111:0/4062158645 conn(0x7f7990060490 msgr2=0x7f79900b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:50.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- 172.21.2.111:0/4062158645 shutdown_connections 2021-04-03T03:08:50.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- 172.21.2.111:0/4062158645 wait complete. 2021-04-03T03:08:50.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 Processor -- start 2021-04-03T03:08:50.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- start start 2021-04-03T03:08:50.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900ad660 0x7f7990066960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:50.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79900b24d0 0x7f799013e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:50.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b8350 0x7f799013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:50.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799013c360 con 0x7f79900b8350 2021-04-03T03:08:50.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799013c600 con 0x7f79900b24d0 2021-04-03T03:08:50.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799013c8a0 con 0x7f79900ad660 2021-04-03T03:08:50.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b8350 0x7f799013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:50.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b8350 0x7f799013e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49688/0 (socket says 172.21.2.111:49688) 2021-04-03T03:08:50.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 -- 172.21.2.111:0/3640630606 learned_addr learned my addr 172.21.2.111:0/3640630606 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:50.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799599e700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900ad660 0x7f7990066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:50.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799519d700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79900b24d0 0x7f799013e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:50.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 -- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900ad660 msgr2=0x7f7990066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:50.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900ad660 0x7f7990066960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 -- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79900b24d0 msgr2=0x7f799013e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:50.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79900b24d0 0x7f799013e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 -- 172.21.2.111:0/3640630606 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799013b5f0 con 0x7f79900b8350 2021-04-03T03:08:50.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799619f700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b8350 0x7f799013e970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f79880032b0 tx=0x7f7988009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:50.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f7982ffd700 1 -- 172.21.2.111:0/3640630606 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f798800c9c0 con 0x7f79900b8350 2021-04-03T03:08:50.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f7982ffd700 1 -- 172.21.2.111:0/3640630606 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798800cb20 con 0x7f79900b8350 2021-04-03T03:08:50.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799013ee60 con 0x7f79900b8350 2021-04-03T03:08:50.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799013f320 con 0x7f79900b8350 2021-04-03T03:08:50.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f7982ffd700 1 -- 172.21.2.111:0/3640630606 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f798800ce80 con 0x7f79900b8350 2021-04-03T03:08:50.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799013b840 con 0x7f79900b8350 2021-04-03T03:08:50.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f7982ffd700 1 -- 172.21.2.111:0/3640630606 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f798801c070 con 0x7f79900b8350 2021-04-03T03:08:50.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f7982ffd700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7970056f70 0x7f7970059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:50.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f7982ffd700 1 -- 172.21.2.111:0/3640630606 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f798806fde0 con 0x7f79900b8350 2021-04-03T03:08:50.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799599e700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7970056f70 0x7f7970059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:50.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.083+0000 7f799599e700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7970056f70 0x7f7970059430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f798c00c000 tx=0x7f798c006cd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:50.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.087+0000 7f7982ffd700 1 -- 172.21.2.111:0/3640630606 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f79880450b0 con 0x7f79900b8350 2021-04-03T03:08:50.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7990057660 con 0x7f79900b8350 2021-04-03T03:08:50.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f7982ffd700 1 -- 172.21.2.111:0/3640630606 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79880486d0 con 0x7f79900b8350 2021-04-03T03:08:50.187 INFO:teuthology.orchestra.run.gibba011.stdout:55834574852 2021-04-03T03:08:50.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7970056f70 msgr2=0x7f7970059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:50.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7970056f70 0x7f7970059430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f798c00c000 tx=0x7f798c006cd0).stop 2021-04-03T03:08:50.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b8350 msgr2=0x7f799013e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:50.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b8350 0x7f799013e970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f79880032b0 tx=0x7f7988009220).stop 2021-04-03T03:08:50.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 shutdown_connections 2021-04-03T03:08:50.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79900b8350 0x7f799013e970 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79900ad660 0x7f7990066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7970056f70 0x7f7970059430 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 --2- 172.21.2.111:0/3640630606 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79900b24d0 0x7f799013e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:50.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 >> 172.21.2.111:0/3640630606 conn(0x7f7990060490 msgr2=0x7f79900627d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:50.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 shutdown_connections 2021-04-03T03:08:50.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:50.179+0000 7f79969a0700 1 -- 172.21.2.111:0/3640630606 wait complete. 2021-04-03T03:08:50.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2021-04-03T03:08:51.196 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2021-04-03T03:08:51.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 Processor -- start 2021-04-03T03:08:51.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- start start 2021-04-03T03:08:51.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5714047680 0x7f571403c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f57140420a0 0x7f5714042940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5714047a60 con 0x7f57140d4d70 2021-04-03T03:08:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57140d40f0 con 0x7f5714047680 2021-04-03T03:08:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57140d4390 con 0x7f57140420a0 2021-04-03T03:08:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57211e2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5714047680 0x7f571403c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57219e3700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f57140d4d70 0x7f5714042e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50486/0 (socket says 172.21.2.111:50486) 2021-04-03T03:08:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57219e3700 1 -- 172.21.2.111:0/3039850495 learned_addr learned my addr 172.21.2.111:0/3039850495 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57211e2700 1 -- 172.21.2.111:0/3039850495 >> v2:172.21.2.115:3300/0 conn(0x7f57140420a0 msgr2=0x7f5714042940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57211e2700 1 --2- 172.21.2.111:0/3039850495 >> v2:172.21.2.115:3300/0 conn(0x7f57140420a0 0x7f5714042940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57211e2700 1 -- 172.21.2.111:0/3039850495 >> v1:172.21.2.111:6789/0 conn(0x7f57140d4d70 legacy=0x7f5714042e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57211e2700 1 -- 172.21.2.111:0/3039850495 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140d4630 con 0x7f5714047680 2021-04-03T03:08:51.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57211e2700 1 --2- 172.21.2.111:0/3039850495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f571403c090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5718002f30 tx=0x7f5718006e60).ready entity=mon.0 client_cookie=b100a3567dca0f7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f570bfff700 1 -- 172.21.2.111:0/3039850495 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5718010470 con 0x7f5714047680 2021-04-03T03:08:51.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f570bfff700 1 -- 172.21.2.111:0/3039850495 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57180105d0 con 0x7f5714047680 2021-04-03T03:08:51.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f570bfff700 1 -- 172.21.2.111:0/3039850495 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5718010930 con 0x7f5714047680 2021-04-03T03:08:51.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- 172.21.2.111:0/3039850495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 msgr2=0x7f571403c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 --2- 172.21.2.111:0/3039850495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f571403c090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5718002f30 tx=0x7f5718006e60).stop 2021-04-03T03:08:51.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- 172.21.2.111:0/3039850495 shutdown_connections 2021-04-03T03:08:51.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 --2- 172.21.2.111:0/3039850495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f571403c090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 --2- 172.21.2.111:0/3039850495 >> v2:172.21.2.115:3300/0 conn(0x7f57140420a0 0x7f5714042940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- 172.21.2.111:0/3039850495 >> 172.21.2.111:0/3039850495 conn(0x7f571400a8f0 msgr2=0x7f571400d720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- 172.21.2.111:0/3039850495 shutdown_connections 2021-04-03T03:08:51.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- 172.21.2.111:0/3039850495 wait complete. 2021-04-03T03:08:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 Processor -- start 2021-04-03T03:08:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.251+0000 7f57221e4700 1 -- start start 2021-04-03T03:08:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57140420a0 0x7f571403ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f57140439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57140d4d70 0x7f571403fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140e3bb0 con 0x7f5714047680 2021-04-03T03:08:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140e6360 con 0x7f57140d4d70 2021-04-03T03:08:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57140e6600 con 0x7f57140420a0 2021-04-03T03:08:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f57140439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f57140439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49696/0 (socket says 172.21.2.111:49696) 2021-04-03T03:08:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 -- 172.21.2.111:0/3798605254 learned_addr learned my addr 172.21.2.111:0/3798605254 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 -- 172.21.2.111:0/3798605254 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57140420a0 msgr2=0x7f571403ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57140420a0 0x7f571403ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 -- 172.21.2.111:0/3798605254 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57140d4d70 msgr2=0x7f571403fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57140d4d70 0x7f571403fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 -- 172.21.2.111:0/3798605254 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140e37f0 con 0x7f5714047680 2021-04-03T03:08:51.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57209e1700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f57140439e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f570c002f30 tx=0x7f570c006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f570a7fc700 1 -- 172.21.2.111:0/3798605254 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f570c00d710 con 0x7f5714047680 2021-04-03T03:08:51.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f570a7fc700 1 -- 172.21.2.111:0/3798605254 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c000e90 con 0x7f5714047680 2021-04-03T03:08:51.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f570a7fc700 1 -- 172.21.2.111:0/3798605254 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f570c00da60 con 0x7f5714047680 2021-04-03T03:08:51.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571403e6b0 con 0x7f5714047680 2021-04-03T03:08:51.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57140e6aa0 con 0x7f5714047680 2021-04-03T03:08:51.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57140e67c0 con 0x7f5714047680 2021-04-03T03:08:51.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f570a7fc700 1 -- 172.21.2.111:0/3798605254 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f570c018020 con 0x7f5714047680 2021-04-03T03:08:51.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f570a7fc700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f56fc0570a0 0x7f56fc059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f570a7fc700 1 -- 172.21.2.111:0/3798605254 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f570c070210 con 0x7f5714047680 2021-04-03T03:08:51.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57211e2700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f56fc0570a0 0x7f56fc059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.255+0000 7f57211e2700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f56fc0570a0 0x7f56fc059560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5718002f30 tx=0x7f571400e550).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.259+0000 7f570a7fc700 1 -- 172.21.2.111:0/3798605254 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f570c0454e0 con 0x7f5714047680 2021-04-03T03:08:51.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f57140e39e0 con 0x7f5714047680 2021-04-03T03:08:51.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f570a7fc700 1 -- 172.21.2.111:0/3798605254 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f570c048b00 con 0x7f5714047680 2021-04-03T03:08:51.360 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:51.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f56fc0570a0 msgr2=0x7f56fc059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f56fc0570a0 0x7f56fc059560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5718002f30 tx=0x7f571400e550).stop 2021-04-03T03:08:51.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 msgr2=0x7f57140439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f57140439e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f570c002f30 tx=0x7f570c006e70).stop 2021-04-03T03:08:51.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 shutdown_connections 2021-04-03T03:08:51.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f56fc0570a0 0x7f56fc059560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57140d4d70 0x7f571403fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57140420a0 0x7f571403ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.351+0000 7f57221e4700 1 --2- 172.21.2.111:0/3798605254 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5714047680 0x7f57140439e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.355+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 >> 172.21.2.111:0/3798605254 conn(0x7f571400a8f0 msgr2=0x7f571403ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.355+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 shutdown_connections 2021-04-03T03:08:51.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.355+0000 7f57221e4700 1 -- 172.21.2.111:0/3798605254 wait complete. 2021-04-03T03:08:51.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2021-04-03T03:08:51.368 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2021-04-03T03:08:51.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029a04700 1 Processor -- start 2021-04-03T03:08:51.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029a04700 1 -- start start 2021-04-03T03:08:51.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029a04700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f50240603f0 0x7f50241447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029a04700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50240607d0 con 0x7f50240b17b0 2021-04-03T03:08:51.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029a04700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502414b090 con 0x7f50240add50 2021-04-03T03:08:51.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029a04700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502414b330 con 0x7f50240603f0 2021-04-03T03:08:51.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029203700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f50240b17b0 0x7f5024147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50500/0 (socket says 172.21.2.111:50500) 2021-04-03T03:08:51.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.415+0000 7f5029203700 1 -- 172.21.2.111:0/2328941690 learned_addr learned my addr 172.21.2.111:0/2328941690 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:51.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857042576 0 0) 0x7f50240607d0 con 0x7f50240b17b0 2021-04-03T03:08:51.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5004003240 con 0x7f50240b17b0 2021-04-03T03:08:51.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3710582389 0 0) 0x7f5004003240 con 0x7f50240b17b0 2021-04-03T03:08:51.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 >> v2:172.21.2.115:3300/0 conn(0x7f50240603f0 msgr2=0x7f50241447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 --2- 172.21.2.111:0/2328941690 >> v2:172.21.2.115:3300/0 conn(0x7f50240603f0 0x7f50241447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 >> v1:172.21.2.115:6789/0 conn(0x7f50240add50 legacy=0x7f5024147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502414b5d0 con 0x7f50240b17b0 2021-04-03T03:08:51.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f501c001a50 con 0x7f50240b17b0 2021-04-03T03:08:51.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 >> v1:172.21.2.111:6789/0 conn(0x7f50240b17b0 legacy=0x7f5024147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5004003730 0x7f5004003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f5004004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5004004e50 0x7f500400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502414b330 con 0x7f50040044e0 2021-04-03T03:08:51.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004004240 con 0x7f5004004e50 2021-04-03T03:08:51.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500400bc20 con 0x7f5004003730 2021-04-03T03:08:51.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5023fff700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f5004004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5023fff700 1 -- 172.21.2.111:0/2328941690 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5004003730 msgr2=0x7f5004003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5023fff700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5004003730 0x7f5004003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5023fff700 1 -- 172.21.2.111:0/2328941690 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5004004e50 msgr2=0x7f500400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5023fff700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5004004e50 0x7f500400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5023fff700 1 -- 172.21.2.111:0/2328941690 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004003240 con 0x7f50040044e0 2021-04-03T03:08:51.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029203700 1 -- 172.21.2.111:0/2328941690 reap_dead start 2021-04-03T03:08:51.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5023fff700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f5004004960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f50040054b0 tx=0x7f500c007b50).ready entity=mon.0 client_cookie=e044ac0af7e6d2b6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f500c00c620 con 0x7f50040044e0 2021-04-03T03:08:51.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c00c780 con 0x7f50040044e0 2021-04-03T03:08:51.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f50237fe700 1 -- 172.21.2.111:0/2328941690 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f500c00cae0 con 0x7f50040044e0 2021-04-03T03:08:51.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- 172.21.2.111:0/2328941690 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 msgr2=0x7f5004004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f5004004960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f50040054b0 tx=0x7f500c007b50).stop 2021-04-03T03:08:51.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- 172.21.2.111:0/2328941690 shutdown_connections 2021-04-03T03:08:51.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 --2- 172.21.2.111:0/2328941690 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f5004004960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- 172.21.2.111:0/2328941690 >> 172.21.2.111:0/2328941690 conn(0x7f50240b61c0 msgr2=0x7f502405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- 172.21.2.111:0/2328941690 shutdown_connections 2021-04-03T03:08:51.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- 172.21.2.111:0/2328941690 wait complete. 2021-04-03T03:08:51.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 Processor -- start 2021-04-03T03:08:51.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- start start 2021-04-03T03:08:51.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f50241469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50240a9650 0x7f5024149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5024067d00 0x7f5024156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5024063aa0 con 0x7f50040044e0 2021-04-03T03:08:51.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50241566f0 con 0x7f50240a9650 2021-04-03T03:08:51.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5029a04700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5024156920 con 0x7f5024067d00 2021-04-03T03:08:51.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f50241469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f50241469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49716/0 (socket says 172.21.2.111:49716) 2021-04-03T03:08:51.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 -- 172.21.2.111:0/3295709984 learned_addr learned my addr 172.21.2.111:0/3295709984 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:51.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 -- 172.21.2.111:0/3295709984 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5024067d00 msgr2=0x7f5024156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5024067d00 0x7f5024156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 -- 172.21.2.111:0/3295709984 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50240a9650 msgr2=0x7f5024149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50240a9650 0x7f5024149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 -- 172.21.2.111:0/3295709984 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024149d10 con 0x7f50040044e0 2021-04-03T03:08:51.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.419+0000 7f5028a02700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f50241469b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5014002f30 tx=0x7f5014006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5021ffb700 1 -- 172.21.2.111:0/3295709984 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f501400d840 con 0x7f50040044e0 2021-04-03T03:08:51.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5024063800 con 0x7f50040044e0 2021-04-03T03:08:51.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5021ffb700 1 -- 172.21.2.111:0/3295709984 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5014000e90 con 0x7f50040044e0 2021-04-03T03:08:51.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5021ffb700 1 -- 172.21.2.111:0/3295709984 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f501400db90 con 0x7f50040044e0 2021-04-03T03:08:51.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5024157270 con 0x7f50040044e0 2021-04-03T03:08:51.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5024057660 con 0x7f50040044e0 2021-04-03T03:08:51.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5021ffb700 1 -- 172.21.2.111:0/3295709984 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5014017030 con 0x7f50040044e0 2021-04-03T03:08:51.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5021ffb700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5008056f20 0x7f50080593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5021ffb700 1 -- 172.21.2.111:0/3295709984 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f501406f600 con 0x7f50040044e0 2021-04-03T03:08:51.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5023fff700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5008056f20 0x7f50080593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5023fff700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5008056f20 0x7f50080593e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f50040054b0 tx=0x7f500c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.423+0000 7f5021ffb700 1 -- 172.21.2.111:0/3295709984 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f50140449f0 con 0x7f50040044e0 2021-04-03T03:08:51.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5024148850 con 0x7f50040044e0 2021-04-03T03:08:51.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5021ffb700 1 -- 172.21.2.111:0/3295709984 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5014048010 con 0x7f50040044e0 2021-04-03T03:08:51.528 INFO:teuthology.orchestra.run.gibba011.stdout:55834574853 2021-04-03T03:08:51.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5008056f20 msgr2=0x7f50080593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5008056f20 0x7f50080593e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f50040054b0 tx=0x7f500c015000).stop 2021-04-03T03:08:51.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 msgr2=0x7f50241469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f50241469b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5014002f30 tx=0x7f5014006ea0).stop 2021-04-03T03:08:51.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 shutdown_connections 2021-04-03T03:08:51.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5024067d00 0x7f5024156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50040044e0 0x7f50241469b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5008056f20 0x7f50080593e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 --2- 172.21.2.111:0/3295709984 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50240a9650 0x7f5024149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 >> 172.21.2.111:0/3295709984 conn(0x7f50240b61c0 msgr2=0x7f50240ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 shutdown_connections 2021-04-03T03:08:51.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.519+0000 7f5029a04700 1 -- 172.21.2.111:0/3295709984 wait complete. 2021-04-03T03:08:51.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2021-04-03T03:08:51.535 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2021-04-03T03:08:51.536 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2021-04-03T03:08:51.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70dd4a700 1 Processor -- start 2021-04-03T03:08:51.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70dd4a700 1 -- start start 2021-04-03T03:08:51.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70dd4a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe70814b700 0x7fe7080b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70dd4a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080607d0 con 0x7fe7080b0c90 2021-04-03T03:08:51.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70dd4a700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080af400 con 0x7fe7080603f0 2021-04-03T03:08:51.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70dd4a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080af6a0 con 0x7fe70814b700 2021-04-03T03:08:51.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70d549700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe7080b0c90 0x7fe708147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50516/0 (socket says 172.21.2.111:50516) 2021-04-03T03:08:51.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe70d549700 1 -- 172.21.2.111:0/727337517 learned_addr learned my addr 172.21.2.111:0/727337517 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:51.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.583+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332568310 0 0) 0x7fe7080607d0 con 0x7fe7080b0c90 2021-04-03T03:08:51.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e8003240 con 0x7fe7080b0c90 2021-04-03T03:08:51.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2507263373 0 0) 0x7fe6e8003240 con 0x7fe7080b0c90 2021-04-03T03:08:51.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe70814b700 msgr2=0x7fe7080b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe70814b700 0x7fe7080b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 >> v1:172.21.2.115:6789/0 conn(0x7fe7080603f0 legacy=0x7fe7080b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080afb40 con 0x7fe7080b0c90 2021-04-03T03:08:51.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe700001a50 con 0x7fe7080b0c90 2021-04-03T03:08:51.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 >> v1:172.21.2.111:6789/0 conn(0x7fe7080b0c90 legacy=0x7fe708147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe70814b700 0x7fe6e80037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e8003d20 0x7fe6e80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6e80049d0 0x7fe6e8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080af6a0 con 0x7fe6e8003d20 2021-04-03T03:08:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80042b0 con 0x7fe6e80049d0 2021-04-03T03:08:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e800b730 con 0x7fe70814b700 2021-04-03T03:08:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e8003d20 0x7fe6e80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe70814b700 0x7fe6e80037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 -- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe70814b700 msgr2=0x7fe6e80037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe70814b700 0x7fe6e80037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 -- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6e80049d0 msgr2=0x7fe6e8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6e80049d0 0x7fe6e8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 -- 172.21.2.111:0/727337517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8003240 con 0x7fe6e8003d20 2021-04-03T03:08:51.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e8003d20 0x7fe6e80044e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe6f8002f30 tx=0x7fe6f8006e10).ready entity=mon.0 client_cookie=172c21ed481eb093 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f8009790 con 0x7fe6e8003d20 2021-04-03T03:08:51.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8000e90 con 0x7fe6e8003d20 2021-04-03T03:08:51.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe7077fe700 1 -- 172.21.2.111:0/727337517 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f8009ae0 con 0x7fe6e8003d20 2021-04-03T03:08:51.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- 172.21.2.111:0/727337517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e8003d20 msgr2=0x7fe6e80044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 --2- 172.21.2.111:0/727337517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e8003d20 0x7fe6e80044e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe6f8002f30 tx=0x7fe6f8006e10).stop 2021-04-03T03:08:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70d549700 1 -- 172.21.2.111:0/727337517 reap_dead start 2021-04-03T03:08:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- 172.21.2.111:0/727337517 shutdown_connections 2021-04-03T03:08:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- 172.21.2.111:0/727337517 >> 172.21.2.111:0/727337517 conn(0x7fe7080b61c0 msgr2=0x7fe70805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- 172.21.2.111:0/727337517 shutdown_connections 2021-04-03T03:08:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- 172.21.2.111:0/727337517 wait complete. 2021-04-03T03:08:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 Processor -- start 2021-04-03T03:08:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- start start 2021-04-03T03:08:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7080aa050 0x7fe708146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7080aaef0 0x7fe708146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe708156320 0x7fe7081567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080ab750 con 0x7fe7080aa050 2021-04-03T03:08:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7081474e0 con 0x7fe708156320 2021-04-03T03:08:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080aa430 con 0x7fe7080aaef0 2021-04-03T03:08:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7080aa050 0x7fe708146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7080aa050 0x7fe708146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49740/0 (socket says 172.21.2.111:49740) 2021-04-03T03:08:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 -- 172.21.2.111:0/2837399796 learned_addr learned my addr 172.21.2.111:0/2837399796 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7080aaef0 0x7fe708146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 -- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7080aaef0 msgr2=0x7fe708146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7080aaef0 0x7fe708146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 -- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe708156320 msgr2=0x7fe7081567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe708156320 0x7fe7081567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 -- 172.21.2.111:0/2837399796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708064d90 con 0x7fe7080aa050 2021-04-03T03:08:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe707fff700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7080aaef0 0x7fe708146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70cd48700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7080aa050 0x7fe708146b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fe708062ef0 tx=0x7fe6f0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe705ffb700 1 -- 172.21.2.111:0/2837399796 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f000f510 con 0x7fe7080aa050 2021-04-03T03:08:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe705ffb700 1 -- 172.21.2.111:0/2837399796 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f000f670 con 0x7fe7080aa050 2021-04-03T03:08:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7080628d0 con 0x7fe7080aa050 2021-04-03T03:08:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe705ffb700 1 -- 172.21.2.111:0/2837399796 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f000f9d0 con 0x7fe7080aa050 2021-04-03T03:08:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.587+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe708063010 con 0x7fe7080aa050 2021-04-03T03:08:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.591+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe708064be0 con 0x7fe7080aa050 2021-04-03T03:08:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.591+0000 7fe705ffb700 1 -- 172.21.2.111:0/2837399796 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe6f001a030 con 0x7fe7080aa050 2021-04-03T03:08:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.591+0000 7fe705ffb700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6ec056ea0 0x7fe6ec059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.591+0000 7fe705ffb700 1 -- 172.21.2.111:0/2837399796 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fe6f006fa50 con 0x7fe7080aa050 2021-04-03T03:08:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.591+0000 7fe707fff700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6ec056ea0 0x7fe6ec059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.591+0000 7fe707fff700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6ec056ea0 0x7fe6ec059360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe6f8016000 tx=0x7fe6f8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.591+0000 7fe705ffb700 1 -- 172.21.2.111:0/2837399796 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe6f0043c50 con 0x7fe7080aa050 2021-04-03T03:08:51.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.683+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe708062d70 con 0x7fe6ec056ea0 2021-04-03T03:08:51.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.683+0000 7fe705ffb700 1 -- 172.21.2.111:0/2837399796 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82569 (secure 0 0 0) 0x7fe6f800b5a0 con 0x7fe6ec056ea0 2021-04-03T03:08:51.693 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:51.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6ec056ea0 msgr2=0x7fe6ec059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6ec056ea0 0x7fe6ec059360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe6f8016000 tx=0x7fe6f8015000).stop 2021-04-03T03:08:51.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7080aa050 msgr2=0x7fe708146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7080aa050 0x7fe708146b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fe708062ef0 tx=0x7fe6f0009040).stop 2021-04-03T03:08:51.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 shutdown_connections 2021-04-03T03:08:51.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6ec056ea0 0x7fe6ec059360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe708156320 0x7fe7081567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7080aa050 0x7fe708146b00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 --2- 172.21.2.111:0/2837399796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7080aaef0 0x7fe708146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 >> 172.21.2.111:0/2837399796 conn(0x7fe7080b61c0 msgr2=0x7fe7080ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 shutdown_connections 2021-04-03T03:08:51.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.687+0000 7fe70dd4a700 1 -- 172.21.2.111:0/2837399796 wait complete. 2021-04-03T03:08:51.697 INFO:teuthology.orchestra.run.gibba011.stderr:dumped all 2021-04-03T03:08:51.700 INFO:teuthology.orchestra.run.gibba011.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2021-04-03T03:08:49.708853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":829032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001330"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.433561+0000","last_change":"2021-04-03T03:08:39.165326+0000","last_active":"2021-04-03T03:08:44.433561+0000","last_peered":"2021-04-03T03:08:44.433561+0000","last_clean":"2021-04-03T03:08:44.433561+0000","last_became_active":"2021-04-03T03:08:39.165110+0000","last_became_peered":"2021-04-03T03:08:39.165110+0000","last_unstale":"2021-04-03T03:08:44.433561+0000","last_undegraded":"2021-04-03T03:08:44.433561+0000","last_fullsized":"2021-04-03T03:08:44.433561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'3","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430184+0000","last_change":"2021-04-03T03:08:41.190527+0000","last_active":"2021-04-03T03:08:44.430184+0000","last_peered":"2021-04-03T03:08:44.430184+0000","last_clean":"2021-04-03T03:08:44.430184+0000","last_became_active":"2021-04-03T03:08:41.190366+0000","last_became_peered":"2021-04-03T03:08:41.190366+0000","last_unstale":"2021-04-03T03:08:44.430184+0000","last_undegraded":"2021-04-03T03:08:44.430184+0000","last_fullsized":"2021-04-03T03:08:44.430184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.433505+0000","last_change":"2021-04-03T03:08:44.433126+0000","last_active":"2021-04-03T03:08:44.433505+0000","last_peered":"2021-04-03T03:08:44.433505+0000","last_clean":"2021-04-03T03:08:44.433505+0000","last_became_active":"2021-04-03T03:08:44.432776+0000","last_became_peered":"2021-04-03T03:08:44.432776+0000","last_unstale":"2021-04-03T03:08:44.433505+0000","last_undegraded":"2021-04-03T03:08:44.433505+0000","last_fullsized":"2021-04-03T03:08:44.433505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430038+0000","last_change":"2021-04-03T03:08:39.164016+0000","last_active":"2021-04-03T03:08:44.430038+0000","last_peered":"2021-04-03T03:08:44.430038+0000","last_clean":"2021-04-03T03:08:44.430038+0000","last_became_active":"2021-04-03T03:08:39.163862+0000","last_became_peered":"2021-04-03T03:08:39.163862+0000","last_unstale":"2021-04-03T03:08:44.430038+0000","last_undegraded":"2021-04-03T03:08:44.430038+0000","last_fullsized":"2021-04-03T03:08:44.430038+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'7","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.433360+0000","last_change":"2021-04-03T03:08:41.568916+0000","last_active":"2021-04-03T03:08:44.433360+0000","last_peered":"2021-04-03T03:08:44.433360+0000","last_clean":"2021-04-03T03:08:44.433360+0000","last_became_active":"2021-04-03T03:08:41.567742+0000","last_became_peered":"2021-04-03T03:08:41.567742+0000","last_unstale":"2021-04-03T03:08:44.433360+0000","last_undegraded":"2021-04-03T03:08:44.433360+0000","last_fullsized":"2021-04-03T03:08:44.433360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435474+0000","last_change":"2021-04-03T03:08:44.434902+0000","last_active":"2021-04-03T03:08:44.435474+0000","last_peered":"2021-04-03T03:08:44.435474+0000","last_clean":"2021-04-03T03:08:44.435474+0000","last_became_active":"2021-04-03T03:08:44.434523+0000","last_became_peered":"2021-04-03T03:08:44.434523+0000","last_unstale":"2021-04-03T03:08:44.435474+0000","last_undegraded":"2021-04-03T03:08:44.435474+0000","last_fullsized":"2021-04-03T03:08:44.435474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431858+0000","last_change":"2021-04-03T03:08:39.484523+0000","last_active":"2021-04-03T03:08:44.431858+0000","last_peered":"2021-04-03T03:08:44.431858+0000","last_clean":"2021-04-03T03:08:44.431858+0000","last_became_active":"2021-04-03T03:08:39.483298+0000","last_became_peered":"2021-04-03T03:08:39.483298+0000","last_unstale":"2021-04-03T03:08:44.431858+0000","last_undegraded":"2021-04-03T03:08:44.431858+0000","last_fullsized":"2021-04-03T03:08:44.431858+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'8","reported_seq":"21","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435867+0000","last_change":"2021-04-03T03:08:41.662046+0000","last_active":"2021-04-03T03:08:44.435867+0000","last_peered":"2021-04-03T03:08:44.435867+0000","last_clean":"2021-04-03T03:08:44.435867+0000","last_became_active":"2021-04-03T03:08:41.661016+0000","last_became_peered":"2021-04-03T03:08:41.661016+0000","last_unstale":"2021-04-03T03:08:44.435867+0000","last_undegraded":"2021-04-03T03:08:44.435867+0000","last_fullsized":"2021-04-03T03:08:44.435867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435696+0000","last_change":"2021-04-03T03:08:44.434816+0000","last_active":"2021-04-03T03:08:44.435696+0000","last_peered":"2021-04-03T03:08:44.435696+0000","last_clean":"2021-04-03T03:08:44.435696+0000","last_became_active":"2021-04-03T03:08:44.434497+0000","last_became_peered":"2021-04-03T03:08:44.434497+0000","last_unstale":"2021-04-03T03:08:44.435696+0000","last_undegraded":"2021-04-03T03:08:44.435696+0000","last_fullsized":"2021-04-03T03:08:44.435696+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430029+0000","last_change":"2021-04-03T03:08:39.481713+0000","last_active":"2021-04-03T03:08:44.430029+0000","last_peered":"2021-04-03T03:08:44.430029+0000","last_clean":"2021-04-03T03:08:44.430029+0000","last_became_active":"2021-04-03T03:08:39.481308+0000","last_became_peered":"2021-04-03T03:08:39.481308+0000","last_unstale":"2021-04-03T03:08:44.430029+0000","last_undegraded":"2021-04-03T03:08:44.430029+0000","last_fullsized":"2021-04-03T03:08:44.430029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'6","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.434036+0000","last_change":"2021-04-03T03:08:41.189248+0000","last_active":"2021-04-03T03:08:44.434036+0000","last_peered":"2021-04-03T03:08:44.434036+0000","last_clean":"2021-04-03T03:08:44.434036+0000","last_became_active":"2021-04-03T03:08:41.189088+0000","last_became_peered":"2021-04-03T03:08:41.189088+0000","last_unstale":"2021-04-03T03:08:44.434036+0000","last_undegraded":"2021-04-03T03:08:44.434036+0000","last_fullsized":"2021-04-03T03:08:44.434036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.429984+0000","last_change":"2021-04-03T03:08:44.223857+0000","last_active":"2021-04-03T03:08:44.429984+0000","last_peered":"2021-04-03T03:08:44.429984+0000","last_clean":"2021-04-03T03:08:44.429984+0000","last_became_active":"2021-04-03T03:08:44.223575+0000","last_became_peered":"2021-04-03T03:08:44.223575+0000","last_unstale":"2021-04-03T03:08:44.429984+0000","last_undegraded":"2021-04-03T03:08:44.429984+0000","last_fullsized":"2021-04-03T03:08:44.429984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435721+0000","last_change":"2021-04-03T03:08:44.435044+0000","last_active":"2021-04-03T03:08:44.435721+0000","last_peered":"2021-04-03T03:08:44.435721+0000","last_clean":"2021-04-03T03:08:44.435721+0000","last_became_active":"2021-04-03T03:08:44.434762+0000","last_became_peered":"2021-04-03T03:08:44.434762+0000","last_unstale":"2021-04-03T03:08:44.435721+0000","last_undegraded":"2021-04-03T03:08:44.435721+0000","last_fullsized":"2021-04-03T03:08:44.435721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.428762+0000","last_change":"2021-04-03T03:08:41.189794+0000","last_active":"2021-04-03T03:08:44.428762+0000","last_peered":"2021-04-03T03:08:44.428762+0000","last_clean":"2021-04-03T03:08:44.428762+0000","last_became_active":"2021-04-03T03:08:41.189620+0000","last_became_peered":"2021-04-03T03:08:41.189620+0000","last_unstale":"2021-04-03T03:08:44.428762+0000","last_undegraded":"2021-04-03T03:08:44.428762+0000","last_fullsized":"2021-04-03T03:08:44.428762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431840+0000","last_change":"2021-04-03T03:08:39.163433+0000","last_active":"2021-04-03T03:08:44.431840+0000","last_peered":"2021-04-03T03:08:44.431840+0000","last_clean":"2021-04-03T03:08:44.431840+0000","last_became_active":"2021-04-03T03:08:39.163151+0000","last_became_peered":"2021-04-03T03:08:39.163151+0000","last_unstale":"2021-04-03T03:08:44.431840+0000","last_undegraded":"2021-04-03T03:08:44.431840+0000","last_fullsized":"2021-04-03T03:08:44.431840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431845+0000","last_change":"2021-04-03T03:08:35.464441+0000","last_active":"2021-04-03T03:08:44.431845+0000","last_peered":"2021-04-03T03:08:44.431845+0000","last_clean":"2021-04-03T03:08:44.431845+0000","last_became_active":"2021-04-03T03:08:35.463653+0000","last_became_peered":"2021-04-03T03:08:35.463653+0000","last_unstale":"2021-04-03T03:08:44.431845+0000","last_undegraded":"2021-04-03T03:08:44.431845+0000","last_fullsized":"2021-04-03T03:08:44.431845+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_clean_scrub_stamp":"2021-04-03T03:08:29.756831+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430720+0000","last_change":"2021-04-03T03:08:39.165073+0000","last_active":"2021-04-03T03:08:44.430720+0000","last_peered":"2021-04-03T03:08:44.430720+0000","last_clean":"2021-04-03T03:08:44.430720+0000","last_became_active":"2021-04-03T03:08:39.164901+0000","last_became_peered":"2021-04-03T03:08:39.164901+0000","last_unstale":"2021-04-03T03:08:44.430720+0000","last_undegraded":"2021-04-03T03:08:44.430720+0000","last_fullsized":"2021-04-03T03:08:44.430720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'1","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430395+0000","last_change":"2021-04-03T03:08:41.189862+0000","last_active":"2021-04-03T03:08:44.430395+0000","last_peered":"2021-04-03T03:08:44.430395+0000","last_clean":"2021-04-03T03:08:44.430395+0000","last_became_active":"2021-04-03T03:08:41.189709+0000","last_became_peered":"2021-04-03T03:08:41.189709+0000","last_unstale":"2021-04-03T03:08:44.430395+0000","last_undegraded":"2021-04-03T03:08:44.430395+0000","last_fullsized":"2021-04-03T03:08:44.430395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.434481+0000","last_change":"2021-04-03T03:08:44.434045+0000","last_active":"2021-04-03T03:08:44.434481+0000","last_peered":"2021-04-03T03:08:44.434481+0000","last_clean":"2021-04-03T03:08:44.434481+0000","last_became_active":"2021-04-03T03:08:44.433769+0000","last_became_peered":"2021-04-03T03:08:44.433769+0000","last_unstale":"2021-04-03T03:08:44.434481+0000","last_undegraded":"2021-04-03T03:08:44.434481+0000","last_fullsized":"2021-04-03T03:08:44.434481+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430086+0000","last_change":"2021-04-03T03:08:44.224483+0000","last_active":"2021-04-03T03:08:44.430086+0000","last_peered":"2021-04-03T03:08:44.430086+0000","last_clean":"2021-04-03T03:08:44.430086+0000","last_became_active":"2021-04-03T03:08:44.224229+0000","last_became_peered":"2021-04-03T03:08:44.224229+0000","last_unstale":"2021-04-03T03:08:44.430086+0000","last_undegraded":"2021-04-03T03:08:44.430086+0000","last_fullsized":"2021-04-03T03:08:44.430086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'8","reported_seq":"21","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.432870+0000","last_change":"2021-04-03T03:08:41.661681+0000","last_active":"2021-04-03T03:08:44.432870+0000","last_peered":"2021-04-03T03:08:44.432870+0000","last_clean":"2021-04-03T03:08:44.432870+0000","last_became_active":"2021-04-03T03:08:41.660835+0000","last_became_peered":"2021-04-03T03:08:41.660835+0000","last_unstale":"2021-04-03T03:08:44.432870+0000","last_undegraded":"2021-04-03T03:08:44.432870+0000","last_fullsized":"2021-04-03T03:08:44.432870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431320+0000","last_change":"2021-04-03T03:08:39.162544+0000","last_active":"2021-04-03T03:08:44.431320+0000","last_peered":"2021-04-03T03:08:44.431320+0000","last_clean":"2021-04-03T03:08:44.431320+0000","last_became_active":"2021-04-03T03:08:39.162345+0000","last_became_peered":"2021-04-03T03:08:39.162345+0000","last_unstale":"2021-04-03T03:08:44.431320+0000","last_undegraded":"2021-04-03T03:08:44.431320+0000","last_fullsized":"2021-04-03T03:08:44.431320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430693+0000","last_change":"2021-04-03T03:08:39.165106+0000","last_active":"2021-04-03T03:08:44.430693+0000","last_peered":"2021-04-03T03:08:44.430693+0000","last_clean":"2021-04-03T03:08:44.430693+0000","last_became_active":"2021-04-03T03:08:39.164923+0000","last_became_peered":"2021-04-03T03:08:39.164923+0000","last_unstale":"2021-04-03T03:08:44.430693+0000","last_undegraded":"2021-04-03T03:08:44.430693+0000","last_fullsized":"2021-04-03T03:08:44.430693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'5","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.432928+0000","last_change":"2021-04-03T03:08:41.189206+0000","last_active":"2021-04-03T03:08:44.432928+0000","last_peered":"2021-04-03T03:08:44.432928+0000","last_clean":"2021-04-03T03:08:44.432928+0000","last_became_active":"2021-04-03T03:08:41.189037+0000","last_became_peered":"2021-04-03T03:08:41.189037+0000","last_unstale":"2021-04-03T03:08:44.432928+0000","last_undegraded":"2021-04-03T03:08:44.432928+0000","last_fullsized":"2021-04-03T03:08:44.432928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.429992+0000","last_change":"2021-04-03T03:08:44.223808+0000","last_active":"2021-04-03T03:08:44.429992+0000","last_peered":"2021-04-03T03:08:44.429992+0000","last_clean":"2021-04-03T03:08:44.429992+0000","last_became_active":"2021-04-03T03:08:44.223468+0000","last_became_peered":"2021-04-03T03:08:44.223468+0000","last_unstale":"2021-04-03T03:08:44.429992+0000","last_undegraded":"2021-04-03T03:08:44.429992+0000","last_fullsized":"2021-04-03T03:08:44.429992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":103087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.183}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":103087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]}]}} 2021-04-03T03:08:51.702 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2021-04-03T03:08:51.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.759+0000 7fa22a43f700 1 Processor -- start 2021-04-03T03:08:51.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa22a43f700 1 -- start start 2021-04-03T03:08:51.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa22a43f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c047140 0x7fa21c0d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa22a43f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa21c042280 0x7fa21c03fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa22a43f700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c0d3520 con 0x7fa21c040230 2021-04-03T03:08:51.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa22a43f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c047520 con 0x7fa21c042280 2021-04-03T03:08:51.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa22a43f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0427d0 con 0x7fa21c047140 2021-04-03T03:08:51.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa22943d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c047140 0x7fa21c0d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa228c3c700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fa21c040230 0x7fa21c0d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34842/0 (socket says 172.21.2.111:34842) 2021-04-03T03:08:51.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa228c3c700 1 -- 172.21.2.111:0/2481872735 learned_addr learned my addr 172.21.2.111:0/2481872735 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:51.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa229c3e700 1 --2- 172.21.2.111:0/2481872735 >> v2:172.21.2.115:3300/0 conn(0x7fa21c042280 0x7fa21c03fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa229c3e700 1 -- 172.21.2.111:0/2481872735 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c047140 msgr2=0x7fa21c0d3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa229c3e700 1 --2- 172.21.2.111:0/2481872735 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c047140 0x7fa21c0d3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa229c3e700 1 -- 172.21.2.111:0/2481872735 >> v1:172.21.2.115:6789/0 conn(0x7fa21c040230 legacy=0x7fa21c0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa229c3e700 1 -- 172.21.2.111:0/2481872735 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c042a70 con 0x7fa21c042280 2021-04-03T03:08:51.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.763+0000 7fa229c3e700 1 --2- 172.21.2.111:0/2481872735 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c042280 0x7fa21c03fab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa22406b5d0 tx=0x7fa22405ad70).ready entity=mon.1 client_cookie=9ef6b27a30a6b583 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa213fff700 1 -- 172.21.2.111:0/2481872735 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2240699f0 con 0x7fa21c042280 2021-04-03T03:08:51.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa213fff700 1 -- 172.21.2.111:0/2481872735 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22406b610 con 0x7fa21c042280 2021-04-03T03:08:51.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa213fff700 1 -- 172.21.2.111:0/2481872735 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa22405d450 con 0x7fa21c042280 2021-04-03T03:08:51.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- 172.21.2.111:0/2481872735 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c042280 msgr2=0x7fa21c03fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 --2- 172.21.2.111:0/2481872735 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c042280 0x7fa21c03fab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa22406b5d0 tx=0x7fa22405ad70).stop 2021-04-03T03:08:51.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- 172.21.2.111:0/2481872735 shutdown_connections 2021-04-03T03:08:51.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 --2- 172.21.2.111:0/2481872735 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c042280 0x7fa21c03fab0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 --2- 172.21.2.111:0/2481872735 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c047140 0x7fa21c0d3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- 172.21.2.111:0/2481872735 >> 172.21.2.111:0/2481872735 conn(0x7fa21c00d520 msgr2=0x7fa21c046710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- 172.21.2.111:0/2481872735 shutdown_connections 2021-04-03T03:08:51.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- 172.21.2.111:0/2481872735 wait complete. 2021-04-03T03:08:51.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 Processor -- start 2021-04-03T03:08:51.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- start start 2021-04-03T03:08:51.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c040230 0x7fa21c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c042280 0x7fa21c03b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa21c047140 0x7fa21c0a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0a2130 con 0x7fa21c047140 2021-04-03T03:08:51.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c03b7e0 con 0x7fa21c040230 2021-04-03T03:08:51.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0a2ca0 con 0x7fa21c042280 2021-04-03T03:08:51.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa21c047140 0x7fa21c0a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa21c047140 0x7fa21c0a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49754/0 (socket says 172.21.2.111:49754) 2021-04-03T03:08:51.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 -- 172.21.2.111:0/3193433096 learned_addr learned my addr 172.21.2.111:0/3193433096 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:51.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 -- 172.21.2.111:0/3193433096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c042280 msgr2=0x7fa21c03b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c042280 0x7fa21c03b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 -- 172.21.2.111:0/3193433096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c040230 msgr2=0x7fa21c03ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c040230 0x7fa21c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 -- 172.21.2.111:0/3193433096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c09fc40 con 0x7fa21c047140 2021-04-03T03:08:51.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa229c3e700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa21c047140 0x7fa21c0a3100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fa22406b5d0 tx=0x7fa224067000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa2127fc700 1 -- 172.21.2.111:0/3193433096 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2240441b0 con 0x7fa21c047140 2021-04-03T03:08:51.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c09f9a0 con 0x7fa21c047140 2021-04-03T03:08:51.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c0a0590 con 0x7fa21c047140 2021-04-03T03:08:51.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa2127fc700 1 -- 172.21.2.111:0/3193433096 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240687c0 con 0x7fa21c047140 2021-04-03T03:08:51.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.767+0000 7fa2127fc700 1 -- 172.21.2.111:0/3193433096 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa22405f2c0 con 0x7fa21c047140 2021-04-03T03:08:51.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.771+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2000033f0 con 0x7fa21c047140 2021-04-03T03:08:51.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.771+0000 7fa2127fc700 1 -- 172.21.2.111:0/3193433096 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa22405f420 con 0x7fa21c047140 2021-04-03T03:08:51.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.771+0000 7fa2127fc700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa204056f20 0x7fa2040593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.771+0000 7fa2127fc700 1 -- 172.21.2.111:0/3193433096 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fa2240dfb60 con 0x7fa21c047140 2021-04-03T03:08:51.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.771+0000 7fa22943d700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa204056f20 0x7fa2040593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.771+0000 7fa22943d700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa204056f20 0x7fa2040593e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa21400c000 tx=0x7fa214006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.771+0000 7fa2127fc700 1 -- 172.21.2.111:0/3193433096 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa2240b4db0 con 0x7fa21c047140 2021-04-03T03:08:51.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.859+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa200002050 con 0x7fa204056f20 2021-04-03T03:08:51.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa2127fc700 1 -- 172.21.2.111:0/3193433096 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82465 (secure 0 0 0) 0x7fa200002050 con 0x7fa204056f20 2021-04-03T03:08:51.872 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:51.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa204056f20 msgr2=0x7fa2040593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa204056f20 0x7fa2040593e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa21400c000 tx=0x7fa214006dd0).stop 2021-04-03T03:08:51.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa21c047140 msgr2=0x7fa21c0a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa21c047140 0x7fa21c0a3100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fa22406b5d0 tx=0x7fa224067000).stop 2021-04-03T03:08:51.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 shutdown_connections 2021-04-03T03:08:51.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa21c047140 0x7fa21c0a3100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa204056f20 0x7fa2040593e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa21c040230 0x7fa21c03ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 --2- 172.21.2.111:0/3193433096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa21c042280 0x7fa21c03b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 >> 172.21.2.111:0/3193433096 conn(0x7fa21c00d520 msgr2=0x7fa21c046620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 shutdown_connections 2021-04-03T03:08:51.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.863+0000 7fa22a43f700 1 -- 172.21.2.111:0/3193433096 wait complete. 2021-04-03T03:08:51.876 INFO:teuthology.orchestra.run.gibba011.stderr:dumped all 2021-04-03T03:08:51.879 INFO:teuthology.orchestra.run.gibba011.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2021-04-03T03:08:51.709144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":829032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.495426"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.433561+0000","last_change":"2021-04-03T03:08:39.165326+0000","last_active":"2021-04-03T03:08:44.433561+0000","last_peered":"2021-04-03T03:08:44.433561+0000","last_clean":"2021-04-03T03:08:44.433561+0000","last_became_active":"2021-04-03T03:08:39.165110+0000","last_became_peered":"2021-04-03T03:08:39.165110+0000","last_unstale":"2021-04-03T03:08:44.433561+0000","last_undegraded":"2021-04-03T03:08:44.433561+0000","last_fullsized":"2021-04-03T03:08:44.433561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'4","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2021-04-03T03:08:45.565215+0000","last_change":"2021-04-03T03:08:41.190527+0000","last_active":"2021-04-03T03:08:45.565215+0000","last_peered":"2021-04-03T03:08:45.565215+0000","last_clean":"2021-04-03T03:08:45.565215+0000","last_became_active":"2021-04-03T03:08:41.190366+0000","last_became_peered":"2021-04-03T03:08:41.190366+0000","last_unstale":"2021-04-03T03:08:45.565215+0000","last_undegraded":"2021-04-03T03:08:45.565215+0000","last_fullsized":"2021-04-03T03:08:45.565215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.433505+0000","last_change":"2021-04-03T03:08:44.433126+0000","last_active":"2021-04-03T03:08:44.433505+0000","last_peered":"2021-04-03T03:08:44.433505+0000","last_clean":"2021-04-03T03:08:44.433505+0000","last_became_active":"2021-04-03T03:08:44.432776+0000","last_became_peered":"2021-04-03T03:08:44.432776+0000","last_unstale":"2021-04-03T03:08:44.433505+0000","last_undegraded":"2021-04-03T03:08:44.433505+0000","last_fullsized":"2021-04-03T03:08:44.433505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430038+0000","last_change":"2021-04-03T03:08:39.164016+0000","last_active":"2021-04-03T03:08:44.430038+0000","last_peered":"2021-04-03T03:08:44.430038+0000","last_clean":"2021-04-03T03:08:44.430038+0000","last_became_active":"2021-04-03T03:08:39.163862+0000","last_became_peered":"2021-04-03T03:08:39.163862+0000","last_unstale":"2021-04-03T03:08:44.430038+0000","last_undegraded":"2021-04-03T03:08:44.430038+0000","last_fullsized":"2021-04-03T03:08:44.430038+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'7","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.433360+0000","last_change":"2021-04-03T03:08:41.568916+0000","last_active":"2021-04-03T03:08:44.433360+0000","last_peered":"2021-04-03T03:08:44.433360+0000","last_clean":"2021-04-03T03:08:44.433360+0000","last_became_active":"2021-04-03T03:08:41.567742+0000","last_became_peered":"2021-04-03T03:08:41.567742+0000","last_unstale":"2021-04-03T03:08:44.433360+0000","last_undegraded":"2021-04-03T03:08:44.433360+0000","last_fullsized":"2021-04-03T03:08:44.433360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435474+0000","last_change":"2021-04-03T03:08:44.434902+0000","last_active":"2021-04-03T03:08:44.435474+0000","last_peered":"2021-04-03T03:08:44.435474+0000","last_clean":"2021-04-03T03:08:44.435474+0000","last_became_active":"2021-04-03T03:08:44.434523+0000","last_became_peered":"2021-04-03T03:08:44.434523+0000","last_unstale":"2021-04-03T03:08:44.435474+0000","last_undegraded":"2021-04-03T03:08:44.435474+0000","last_fullsized":"2021-04-03T03:08:44.435474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431858+0000","last_change":"2021-04-03T03:08:39.484523+0000","last_active":"2021-04-03T03:08:44.431858+0000","last_peered":"2021-04-03T03:08:44.431858+0000","last_clean":"2021-04-03T03:08:44.431858+0000","last_became_active":"2021-04-03T03:08:39.483298+0000","last_became_peered":"2021-04-03T03:08:39.483298+0000","last_unstale":"2021-04-03T03:08:44.431858+0000","last_undegraded":"2021-04-03T03:08:44.431858+0000","last_fullsized":"2021-04-03T03:08:44.431858+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'8","reported_seq":"21","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435867+0000","last_change":"2021-04-03T03:08:41.662046+0000","last_active":"2021-04-03T03:08:44.435867+0000","last_peered":"2021-04-03T03:08:44.435867+0000","last_clean":"2021-04-03T03:08:44.435867+0000","last_became_active":"2021-04-03T03:08:41.661016+0000","last_became_peered":"2021-04-03T03:08:41.661016+0000","last_unstale":"2021-04-03T03:08:44.435867+0000","last_undegraded":"2021-04-03T03:08:44.435867+0000","last_fullsized":"2021-04-03T03:08:44.435867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435696+0000","last_change":"2021-04-03T03:08:44.434816+0000","last_active":"2021-04-03T03:08:44.435696+0000","last_peered":"2021-04-03T03:08:44.435696+0000","last_clean":"2021-04-03T03:08:44.435696+0000","last_became_active":"2021-04-03T03:08:44.434497+0000","last_became_peered":"2021-04-03T03:08:44.434497+0000","last_unstale":"2021-04-03T03:08:44.435696+0000","last_undegraded":"2021-04-03T03:08:44.435696+0000","last_fullsized":"2021-04-03T03:08:44.435696+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430029+0000","last_change":"2021-04-03T03:08:39.481713+0000","last_active":"2021-04-03T03:08:44.430029+0000","last_peered":"2021-04-03T03:08:44.430029+0000","last_clean":"2021-04-03T03:08:44.430029+0000","last_became_active":"2021-04-03T03:08:39.481308+0000","last_became_peered":"2021-04-03T03:08:39.481308+0000","last_unstale":"2021-04-03T03:08:44.430029+0000","last_undegraded":"2021-04-03T03:08:44.430029+0000","last_fullsized":"2021-04-03T03:08:44.430029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'6","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.434036+0000","last_change":"2021-04-03T03:08:41.189248+0000","last_active":"2021-04-03T03:08:44.434036+0000","last_peered":"2021-04-03T03:08:44.434036+0000","last_clean":"2021-04-03T03:08:44.434036+0000","last_became_active":"2021-04-03T03:08:41.189088+0000","last_became_peered":"2021-04-03T03:08:41.189088+0000","last_unstale":"2021-04-03T03:08:44.434036+0000","last_undegraded":"2021-04-03T03:08:44.434036+0000","last_fullsized":"2021-04-03T03:08:44.434036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.429984+0000","last_change":"2021-04-03T03:08:44.223857+0000","last_active":"2021-04-03T03:08:44.429984+0000","last_peered":"2021-04-03T03:08:44.429984+0000","last_clean":"2021-04-03T03:08:44.429984+0000","last_became_active":"2021-04-03T03:08:44.223575+0000","last_became_peered":"2021-04-03T03:08:44.223575+0000","last_unstale":"2021-04-03T03:08:44.429984+0000","last_undegraded":"2021-04-03T03:08:44.429984+0000","last_fullsized":"2021-04-03T03:08:44.429984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.435721+0000","last_change":"2021-04-03T03:08:44.435044+0000","last_active":"2021-04-03T03:08:44.435721+0000","last_peered":"2021-04-03T03:08:44.435721+0000","last_clean":"2021-04-03T03:08:44.435721+0000","last_became_active":"2021-04-03T03:08:44.434762+0000","last_became_peered":"2021-04-03T03:08:44.434762+0000","last_unstale":"2021-04-03T03:08:44.435721+0000","last_undegraded":"2021-04-03T03:08:44.435721+0000","last_fullsized":"2021-04-03T03:08:44.435721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.428762+0000","last_change":"2021-04-03T03:08:41.189794+0000","last_active":"2021-04-03T03:08:44.428762+0000","last_peered":"2021-04-03T03:08:44.428762+0000","last_clean":"2021-04-03T03:08:44.428762+0000","last_became_active":"2021-04-03T03:08:41.189620+0000","last_became_peered":"2021-04-03T03:08:41.189620+0000","last_unstale":"2021-04-03T03:08:44.428762+0000","last_undegraded":"2021-04-03T03:08:44.428762+0000","last_fullsized":"2021-04-03T03:08:44.428762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431840+0000","last_change":"2021-04-03T03:08:39.163433+0000","last_active":"2021-04-03T03:08:44.431840+0000","last_peered":"2021-04-03T03:08:44.431840+0000","last_clean":"2021-04-03T03:08:44.431840+0000","last_became_active":"2021-04-03T03:08:39.163151+0000","last_became_peered":"2021-04-03T03:08:39.163151+0000","last_unstale":"2021-04-03T03:08:44.431840+0000","last_undegraded":"2021-04-03T03:08:44.431840+0000","last_fullsized":"2021-04-03T03:08:44.431840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431845+0000","last_change":"2021-04-03T03:08:35.464441+0000","last_active":"2021-04-03T03:08:44.431845+0000","last_peered":"2021-04-03T03:08:44.431845+0000","last_clean":"2021-04-03T03:08:44.431845+0000","last_became_active":"2021-04-03T03:08:35.463653+0000","last_became_peered":"2021-04-03T03:08:35.463653+0000","last_unstale":"2021-04-03T03:08:44.431845+0000","last_undegraded":"2021-04-03T03:08:44.431845+0000","last_fullsized":"2021-04-03T03:08:44.431845+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_clean_scrub_stamp":"2021-04-03T03:08:29.756831+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430720+0000","last_change":"2021-04-03T03:08:39.165073+0000","last_active":"2021-04-03T03:08:44.430720+0000","last_peered":"2021-04-03T03:08:44.430720+0000","last_clean":"2021-04-03T03:08:44.430720+0000","last_became_active":"2021-04-03T03:08:39.164901+0000","last_became_peered":"2021-04-03T03:08:39.164901+0000","last_unstale":"2021-04-03T03:08:44.430720+0000","last_undegraded":"2021-04-03T03:08:44.430720+0000","last_fullsized":"2021-04-03T03:08:44.430720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'1","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430395+0000","last_change":"2021-04-03T03:08:41.189862+0000","last_active":"2021-04-03T03:08:44.430395+0000","last_peered":"2021-04-03T03:08:44.430395+0000","last_clean":"2021-04-03T03:08:44.430395+0000","last_became_active":"2021-04-03T03:08:41.189709+0000","last_became_peered":"2021-04-03T03:08:41.189709+0000","last_unstale":"2021-04-03T03:08:44.430395+0000","last_undegraded":"2021-04-03T03:08:44.430395+0000","last_fullsized":"2021-04-03T03:08:44.430395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.434481+0000","last_change":"2021-04-03T03:08:44.434045+0000","last_active":"2021-04-03T03:08:44.434481+0000","last_peered":"2021-04-03T03:08:44.434481+0000","last_clean":"2021-04-03T03:08:44.434481+0000","last_became_active":"2021-04-03T03:08:44.433769+0000","last_became_peered":"2021-04-03T03:08:44.433769+0000","last_unstale":"2021-04-03T03:08:44.434481+0000","last_undegraded":"2021-04-03T03:08:44.434481+0000","last_fullsized":"2021-04-03T03:08:44.434481+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430086+0000","last_change":"2021-04-03T03:08:44.224483+0000","last_active":"2021-04-03T03:08:44.430086+0000","last_peered":"2021-04-03T03:08:44.430086+0000","last_clean":"2021-04-03T03:08:44.430086+0000","last_became_active":"2021-04-03T03:08:44.224229+0000","last_became_peered":"2021-04-03T03:08:44.224229+0000","last_unstale":"2021-04-03T03:08:44.430086+0000","last_undegraded":"2021-04-03T03:08:44.430086+0000","last_fullsized":"2021-04-03T03:08:44.430086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'8","reported_seq":"21","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.432870+0000","last_change":"2021-04-03T03:08:41.661681+0000","last_active":"2021-04-03T03:08:44.432870+0000","last_peered":"2021-04-03T03:08:44.432870+0000","last_clean":"2021-04-03T03:08:44.432870+0000","last_became_active":"2021-04-03T03:08:41.660835+0000","last_became_peered":"2021-04-03T03:08:41.660835+0000","last_unstale":"2021-04-03T03:08:44.432870+0000","last_undegraded":"2021-04-03T03:08:44.432870+0000","last_fullsized":"2021-04-03T03:08:44.432870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.431320+0000","last_change":"2021-04-03T03:08:39.162544+0000","last_active":"2021-04-03T03:08:44.431320+0000","last_peered":"2021-04-03T03:08:44.431320+0000","last_clean":"2021-04-03T03:08:44.431320+0000","last_became_active":"2021-04-03T03:08:39.162345+0000","last_became_peered":"2021-04-03T03:08:39.162345+0000","last_unstale":"2021-04-03T03:08:44.431320+0000","last_undegraded":"2021-04-03T03:08:44.431320+0000","last_fullsized":"2021-04-03T03:08:44.431320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.430693+0000","last_change":"2021-04-03T03:08:39.165106+0000","last_active":"2021-04-03T03:08:44.430693+0000","last_peered":"2021-04-03T03:08:44.430693+0000","last_clean":"2021-04-03T03:08:44.430693+0000","last_became_active":"2021-04-03T03:08:39.164923+0000","last_became_peered":"2021-04-03T03:08:39.164923+0000","last_unstale":"2021-04-03T03:08:44.430693+0000","last_undegraded":"2021-04-03T03:08:44.430693+0000","last_fullsized":"2021-04-03T03:08:44.430693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'5","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.432928+0000","last_change":"2021-04-03T03:08:41.189206+0000","last_active":"2021-04-03T03:08:44.432928+0000","last_peered":"2021-04-03T03:08:44.432928+0000","last_clean":"2021-04-03T03:08:44.432928+0000","last_became_active":"2021-04-03T03:08:41.189037+0000","last_became_peered":"2021-04-03T03:08:41.189037+0000","last_unstale":"2021-04-03T03:08:44.432928+0000","last_undegraded":"2021-04-03T03:08:44.432928+0000","last_fullsized":"2021-04-03T03:08:44.432928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:40.163130+0000","last_clean_scrub_stamp":"2021-04-03T03:08:40.163130+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2021-04-03T03:08:44.429992+0000","last_change":"2021-04-03T03:08:44.223808+0000","last_active":"2021-04-03T03:08:44.429992+0000","last_peered":"2021-04-03T03:08:44.429992+0000","last_clean":"2021-04-03T03:08:44.429992+0000","last_became_active":"2021-04-03T03:08:44.223468+0000","last_became_peered":"2021-04-03T03:08:44.223468+0000","last_unstale":"2021-04-03T03:08:44.429992+0000","last_undegraded":"2021-04-03T03:08:44.429992+0000","last_fullsized":"2021-04-03T03:08:44.429992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:43.202246+0000","last_clean_scrub_stamp":"2021-04-03T03:08:43.202246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":103087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.183}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":103087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]}]}} 2021-04-03T03:08:51.881 INFO:tasks.ceph.ceph_manager.ceph:clean! 2021-04-03T03:08:51.881 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-04-03T03:08:51.882 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2021-04-03T03:08:51.882 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2021-04-03T03:08:51.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.927+0000 7fadb2758700 1 Processor -- start 2021-04-03T03:08:51.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.927+0000 7fadb2758700 1 -- start start 2021-04-03T03:08:51.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 0x7fada4005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada403d840 con 0x7fada4041fe0 2021-04-03T03:08:51.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada4005b30 con 0x7fada4042440 2021-04-03T03:08:51.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada403faa0 con 0x7fada403f120 2021-04-03T03:08:51.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb0f55700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fada4041fe0 0x7fada40d89a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50550/0 (socket says 172.21.2.111:50550) 2021-04-03T03:08:51.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb0f55700 1 -- 172.21.2.111:0/1533692251 learned_addr learned my addr 172.21.2.111:0/1533692251 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:51.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 0x7fada4005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736734459 0 0) 0x7fada403d840 con 0x7fada4041fe0 2021-04-03T03:08:51.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad94003240 con 0x7fada4041fe0 2021-04-03T03:08:51.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 725884307 0 0) 0x7fad94003240 con 0x7fada4041fe0 2021-04-03T03:08:51.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 msgr2=0x7fada4005640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 0x7fada4005640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 >> v1:172.21.2.115:6789/0 conn(0x7fada4042440 legacy=0x7fada40e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada403f800 con 0x7fada4041fe0 2021-04-03T03:08:51.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fada80016d0 con 0x7fada4041fe0 2021-04-03T03:08:51.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 0x7fada4005640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:08:51.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 >> v1:172.21.2.111:6789/0 conn(0x7fada4041fe0 legacy=0x7fada40d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad94003760 0x7fad94003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad940044e0 0x7fad94004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 0x7fad94004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada403faa0 con 0x7fad94003760 2021-04-03T03:08:51.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94005390 con 0x7fad940044e0 2021-04-03T03:08:51.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad94004300 con 0x7fada403f120 2021-04-03T03:08:51.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad94003760 0x7fad94003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 -- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 msgr2=0x7fad94004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1f57700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 0x7fad94004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada403f120 0x7fad94004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:51.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 -- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad940044e0 msgr2=0x7fad94004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:51.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad940044e0 0x7fad94004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 -- 172.21.2.111:0/1533692251 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94003240 con 0x7fad94003760 2021-04-03T03:08:51.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1756700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad94003760 0x7fad94003b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fada000a000 tx=0x7fada0008040).ready entity=mon.0 client_cookie=b21310293acaac59 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fada000db30 con 0x7fad94003760 2021-04-03T03:08:51.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada000dc90 con 0x7fad94003760 2021-04-03T03:08:51.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fad9ffff700 1 -- 172.21.2.111:0/1533692251 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fada0012420 con 0x7fad94003760 2021-04-03T03:08:51.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- 172.21.2.111:0/1533692251 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad94003760 msgr2=0x7fad94003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 --2- 172.21.2.111:0/1533692251 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad94003760 0x7fad94003b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fada000a000 tx=0x7fada0008040).stop 2021-04-03T03:08:51.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1f57700 1 -- 172.21.2.111:0/1533692251 reap_dead start 2021-04-03T03:08:51.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- 172.21.2.111:0/1533692251 shutdown_connections 2021-04-03T03:08:51.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- 172.21.2.111:0/1533692251 >> 172.21.2.111:0/1533692251 conn(0x7fada40462b0 msgr2=0x7fada40466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:51.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- 172.21.2.111:0/1533692251 shutdown_connections 2021-04-03T03:08:51.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- 172.21.2.111:0/1533692251 wait complete. 2021-04-03T03:08:51.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 Processor -- start 2021-04-03T03:08:51.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- start start 2021-04-03T03:08:51.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fada4039ed0 0x7fada40a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada40d9760 0x7fada40a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fada409f9b0 0x7fada40ea690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada40e7ae0 con 0x7fada409f9b0 2021-04-03T03:08:51.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada40a32b0 con 0x7fada4039ed0 2021-04-03T03:08:51.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb2758700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada403a2b0 con 0x7fada40d9760 2021-04-03T03:08:51.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1f57700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fada409f9b0 0x7fada40ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1f57700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fada409f9b0 0x7fada40ea690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49772/0 (socket says 172.21.2.111:49772) 2021-04-03T03:08:51.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.931+0000 7fadb1f57700 1 -- 172.21.2.111:0/4099175958 learned_addr learned my addr 172.21.2.111:0/4099175958 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:51.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1f57700 1 -- 172.21.2.111:0/4099175958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada40d9760 msgr2=0x7fada40a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1f57700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada40d9760 0x7fada40a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1f57700 1 -- 172.21.2.111:0/4099175958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fada4039ed0 msgr2=0x7fada40a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:51.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1f57700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fada4039ed0 0x7fada40a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:51.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1f57700 1 -- 172.21.2.111:0/4099175958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada409f5b0 con 0x7fada409f9b0 2021-04-03T03:08:51.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1f57700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fada409f9b0 0x7fada40ea690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fadac05aec0 tx=0x7fadac064040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fad9e7fc700 1 -- 172.21.2.111:0/4099175958 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac05d7e0 con 0x7fada409f9b0 2021-04-03T03:08:51.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fad9e7fc700 1 -- 172.21.2.111:0/4099175958 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac05df60 con 0x7fada409f9b0 2021-04-03T03:08:51.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada40e7cc0 con 0x7fada409f9b0 2021-04-03T03:08:51.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fad9e7fc700 1 -- 172.21.2.111:0/4099175958 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac05f4b0 con 0x7fada409f9b0 2021-04-03T03:08:51.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada40eab80 con 0x7fada409f9b0 2021-04-03T03:08:51.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada40e7e50 con 0x7fada409f9b0 2021-04-03T03:08:51.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fad9e7fc700 1 -- 172.21.2.111:0/4099175958 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fadac05f610 con 0x7fada409f9b0 2021-04-03T03:08:51.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fad9e7fc700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad8c056f20 0x7fad8c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:51.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fad9e7fc700 1 -- 172.21.2.111:0/4099175958 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fadac0df210 con 0x7fada409f9b0 2021-04-03T03:08:51.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1756700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad8c056f20 0x7fad8c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:51.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.935+0000 7fadb1756700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad8c056f20 0x7fad8c0593e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fada000a000 tx=0x7fada0019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:51.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:51.939+0000 7fad9e7fc700 1 -- 172.21.2.111:0/4099175958 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fadac0b44e0 con 0x7fada409f9b0 2021-04-03T03:08:52.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.067+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fada4004d10 con 0x7fada409f9b0 2021-04-03T03:08:52.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.067+0000 7fad9e7fc700 1 -- 172.21.2.111:0/4099175958 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fadac0b7b00 con 0x7fada409f9b0 2021-04-03T03:08:52.076 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:52.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.067+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad8c056f20 msgr2=0x7fad8c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.067+0000 7fadb2758700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad8c056f20 0x7fad8c0593e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fada000a000 tx=0x7fada0019000).stop 2021-04-03T03:08:52.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.067+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fada409f9b0 msgr2=0x7fada40ea690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.067+0000 7fadb2758700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fada409f9b0 0x7fada40ea690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fadac05aec0 tx=0x7fadac064040).stop 2021-04-03T03:08:52.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 shutdown_connections 2021-04-03T03:08:52.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fada409f9b0 0x7fada40ea690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad8c056f20 0x7fad8c0593e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fada4039ed0 0x7fada40a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 --2- 172.21.2.111:0/4099175958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fada40d9760 0x7fada40a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 >> 172.21.2.111:0/4099175958 conn(0x7fada40462b0 msgr2=0x7fada40440e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 shutdown_connections 2021-04-03T03:08:52.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.071+0000 7fadb2758700 1 -- 172.21.2.111:0/4099175958 wait complete. 2021-04-03T03:08:52.084 INFO:teuthology.orchestra.run.gibba011.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-04-03T03:08:52.084 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2021-04-03T03:08:52.085 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:52.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.131+0000 7f52fdd60700 1 Processor -- start 2021-04-03T03:08:52.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.131+0000 7f52fdd60700 1 -- start start 2021-04-03T03:08:52.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.131+0000 7f52fdd60700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f52f00d4d10 0x7f52f0040ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.131+0000 7f52fdd60700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0040930 0x7f52f0006230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.131+0000 7f52fdd60700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f00481e0 con 0x7f52f0047e00 2021-04-03T03:08:52.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.131+0000 7f52fdd60700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f003f1d0 con 0x7f52f00d4d10 2021-04-03T03:08:52.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.131+0000 7f52fdd60700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f0042090 con 0x7f52f0040930 2021-04-03T03:08:52.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fd55f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0040930 0x7f52f0006230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fcd5e700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f52f0047e00 0x7f52f00d4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34874/0 (socket says 172.21.2.111:34874) 2021-04-03T03:08:52.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fcd5e700 1 -- 172.21.2.111:0/1569775914 learned_addr learned my addr 172.21.2.111:0/1569775914 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:52.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52f7fff700 1 --2- 172.21.2.111:0/1569775914 >> v2:172.21.2.115:3300/0 conn(0x7f52f00d4d10 0x7f52f0040ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fd55f700 1 -- 172.21.2.111:0/1569775914 >> v2:172.21.2.115:3300/0 conn(0x7f52f00d4d10 msgr2=0x7f52f0040ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:52.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fd55f700 1 --2- 172.21.2.111:0/1569775914 >> v2:172.21.2.115:3300/0 conn(0x7f52f00d4d10 0x7f52f0040ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fd55f700 1 -- 172.21.2.111:0/1569775914 >> v1:172.21.2.115:6789/0 conn(0x7f52f0047e00 legacy=0x7f52f00d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52f77fe700 1 -- 172.21.2.111:0/1569775914 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244983142 0 0) 0x7f52f00481e0 con 0x7f52f0047e00 2021-04-03T03:08:52.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fd55f700 1 -- 172.21.2.111:0/1569775914 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0042330 con 0x7f52f0040930 2021-04-03T03:08:52.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52f7fff700 1 --2- 172.21.2.111:0/1569775914 >> v2:172.21.2.115:3300/0 conn(0x7f52f00d4d10 0x7f52f0040ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:52.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fd55f700 1 --2- 172.21.2.111:0/1569775914 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0040930 0x7f52f0006230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f52f806b5d0 tx=0x7f52f805ad70).ready entity=mon.2 client_cookie=5232b4af351f227f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52f77fe700 1 -- 172.21.2.111:0/1569775914 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52f805bda0 con 0x7f52f0040930 2021-04-03T03:08:52.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52f77fe700 1 -- 172.21.2.111:0/1569775914 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f805df60 con 0x7f52f0040930 2021-04-03T03:08:52.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52f77fe700 1 -- 172.21.2.111:0/1569775914 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52f80699f0 con 0x7f52f0040930 2021-04-03T03:08:52.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fdd60700 1 -- 172.21.2.111:0/1569775914 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0040930 msgr2=0x7f52f0006230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.135+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1569775914 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0040930 0x7f52f0006230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f52f806b5d0 tx=0x7f52f805ad70).stop 2021-04-03T03:08:52.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- 172.21.2.111:0/1569775914 shutdown_connections 2021-04-03T03:08:52.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1569775914 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0040930 0x7f52f0006230 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1569775914 >> v2:172.21.2.115:3300/0 conn(0x7f52f00d4d10 0x7f52f0040ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- 172.21.2.111:0/1569775914 >> 172.21.2.111:0/1569775914 conn(0x7f52f000e220 msgr2=0x7f52f00473d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- 172.21.2.111:0/1569775914 shutdown_connections 2021-04-03T03:08:52.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- 172.21.2.111:0/1569775914 wait complete. 2021-04-03T03:08:52.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 Processor -- start 2021-04-03T03:08:52.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- start start 2021-04-03T03:08:52.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f0040930 0x7f52f0006530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0047e00 0x7f52f00eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f00d4d10 0x7f52f00eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00c5720 con 0x7f52f0040930 2021-04-03T03:08:52.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00cdee0 con 0x7f52f00d4d10 2021-04-03T03:08:52.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00ce110 con 0x7f52f0047e00 2021-04-03T03:08:52.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f0040930 0x7f52f0006530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f0040930 0x7f52f0006530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49786/0 (socket says 172.21.2.111:49786) 2021-04-03T03:08:52.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 -- 172.21.2.111:0/1585574885 learned_addr learned my addr 172.21.2.111:0/1585574885 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 -- 172.21.2.111:0/1585574885 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0047e00 msgr2=0x7f52f00eaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0047e00 0x7f52f00eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 -- 172.21.2.111:0/1585574885 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f00d4d10 msgr2=0x7f52f00eaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f00d4d10 0x7f52f00eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 -- 172.21.2.111:0/1585574885 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00ca990 con 0x7f52f0040930 2021-04-03T03:08:52.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fcd5e700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f0040930 0x7f52f0006530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f52f00c8890 tx=0x7f52ec007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52f5ffb700 1 -- 172.21.2.111:0/1585574885 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec007320 con 0x7f52f0040930 2021-04-03T03:08:52.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52f5ffb700 1 -- 172.21.2.111:0/1585574885 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec007480 con 0x7f52f0040930 2021-04-03T03:08:52.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f00cbbd0 con 0x7f52f0040930 2021-04-03T03:08:52.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52f5ffb700 1 -- 172.21.2.111:0/1585574885 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec0057e0 con 0x7f52f0040930 2021-04-03T03:08:52.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f0006bb0 con 0x7f52f0040930 2021-04-03T03:08:52.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.139+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f00cb690 con 0x7f52f0040930 2021-04-03T03:08:52.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.143+0000 7f52f5ffb700 1 -- 172.21.2.111:0/1585574885 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f52ec010030 con 0x7f52f0040930 2021-04-03T03:08:52.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.143+0000 7f52f5ffb700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52e0056fe0 0x7f52e00594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.143+0000 7f52f5ffb700 1 -- 172.21.2.111:0/1585574885 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f52ec06fe90 con 0x7f52f0040930 2021-04-03T03:08:52.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.143+0000 7f52f7fff700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52e0056fe0 0x7f52e00594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.143+0000 7f52f7fff700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52e0056fe0 0x7f52e00594a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f52e400c000 tx=0x7f52e4006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.143+0000 7f52f5ffb700 1 -- 172.21.2.111:0/1585574885 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f52f00cb690 con 0x7f52f0040930 2021-04-03T03:08:52.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.235+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f00cb660 con 0x7f52f0040930 2021-04-03T03:08:52.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52f5ffb700 1 -- 172.21.2.111:0/1585574885 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f52ec048ca0 con 0x7f52f0040930 2021-04-03T03:08:52.245 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:52.246 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:52.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52e0056fe0 msgr2=0x7f52e00594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52e0056fe0 0x7f52e00594a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f52e400c000 tx=0x7f52e4006dd0).stop 2021-04-03T03:08:52.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f0040930 msgr2=0x7f52f0006530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f0040930 0x7f52f0006530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f52f00c8890 tx=0x7f52ec007aa0).stop 2021-04-03T03:08:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 shutdown_connections 2021-04-03T03:08:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52e0056fe0 0x7f52e00594a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f00d4d10 0x7f52f00eaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f0040930 0x7f52f0006530 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 --2- 172.21.2.111:0/1585574885 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f0047e00 0x7f52f00eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 >> 172.21.2.111:0/1585574885 conn(0x7f52f000e220 msgr2=0x7f52f003fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 shutdown_connections 2021-04-03T03:08:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.239+0000 7f52fdd60700 1 -- 172.21.2.111:0/1585574885 wait complete. 2021-04-03T03:08:52.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:08:52.254 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:08:52.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d83fff700 1 Processor -- start 2021-04-03T03:08:52.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d83fff700 1 -- start start 2021-04-03T03:08:52.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d83fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6d840603f0 0x7f6d841447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d83fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6d840add50 0x7f6d84144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d83fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d840607d0 con 0x7f6d840b17b0 2021-04-03T03:08:52.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d83fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d841451c0 con 0x7f6d840add50 2021-04-03T03:08:52.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d83fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8414b090 con 0x7f6d840603f0 2021-04-03T03:08:52.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d837fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6d840b17b0 0x7f6d84147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50584/0 (socket says 172.21.2.111:50584) 2021-04-03T03:08:52.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d837fe700 1 -- 172.21.2.111:0/50314962 learned_addr learned my addr 172.21.2.111:0/50314962 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:52.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d827fc700 1 --2- 172.21.2.111:0/50314962 >> v2:172.21.2.111:3300/0 conn(0x7f6d840add50 0x7f6d84144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.303+0000 7f6d81ffb700 1 -- 172.21.2.111:0/50314962 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569172515 0 0) 0x7f6d840607d0 con 0x7f6d840b17b0 2021-04-03T03:08:52.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d81ffb700 1 -- 172.21.2.111:0/50314962 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d68003240 con 0x7f6d840b17b0 2021-04-03T03:08:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 -- 172.21.2.111:0/50314962 >> v2:172.21.2.115:3300/0 conn(0x7f6d840603f0 msgr2=0x7f6d841447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 --2- 172.21.2.111:0/50314962 >> v2:172.21.2.115:3300/0 conn(0x7f6d840603f0 0x7f6d841447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 -- 172.21.2.111:0/50314962 >> v1:172.21.2.111:6789/0 conn(0x7f6d840b17b0 legacy=0x7f6d84147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 -- 172.21.2.111:0/50314962 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8414b330 con 0x7f6d840add50 2021-04-03T03:08:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 --2- 172.21.2.111:0/50314962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84144cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6d78002f30 tx=0x7f6d78006dd0).ready entity=mon.0 client_cookie=3d6c9b1b54c4aac1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d81ffb700 1 -- 172.21.2.111:0/50314962 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d780097b0 con 0x7f6d840add50 2021-04-03T03:08:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d81ffb700 1 -- 172.21.2.111:0/50314962 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78009910 con 0x7f6d840add50 2021-04-03T03:08:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d81ffb700 1 -- 172.21.2.111:0/50314962 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d78009c70 con 0x7f6d840add50 2021-04-03T03:08:52.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- 172.21.2.111:0/50314962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 msgr2=0x7f6d84144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 --2- 172.21.2.111:0/50314962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84144cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6d78002f30 tx=0x7f6d78006dd0).stop 2021-04-03T03:08:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- 172.21.2.111:0/50314962 shutdown_connections 2021-04-03T03:08:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 --2- 172.21.2.111:0/50314962 >> v2:172.21.2.115:3300/0 conn(0x7f6d840603f0 0x7f6d841447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 --2- 172.21.2.111:0/50314962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84144cd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- 172.21.2.111:0/50314962 >> 172.21.2.111:0/50314962 conn(0x7f6d840b61c0 msgr2=0x7f6d8405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- 172.21.2.111:0/50314962 shutdown_connections 2021-04-03T03:08:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- 172.21.2.111:0/50314962 wait complete. 2021-04-03T03:08:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 Processor -- start 2021-04-03T03:08:52.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- start start 2021-04-03T03:08:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6d840603f0 0x7f6d8414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84063020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6d840b17b0 0x7f6d84159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840aa700 con 0x7f6d840add50 2021-04-03T03:08:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840aa930 con 0x7f6d840603f0 2021-04-03T03:08:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84062b80 con 0x7f6d840b17b0 2021-04-03T03:08:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84063020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84063020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49802/0 (socket says 172.21.2.111:49802) 2021-04-03T03:08:52.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 -- 172.21.2.111:0/633967772 learned_addr learned my addr 172.21.2.111:0/633967772 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 -- 172.21.2.111:0/633967772 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6d840b17b0 msgr2=0x7f6d84159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6d840b17b0 0x7f6d84159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 -- 172.21.2.111:0/633967772 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6d840603f0 msgr2=0x7f6d8414a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6d840603f0 0x7f6d8414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 -- 172.21.2.111:0/633967772 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840aa2f0 con 0x7f6d840add50 2021-04-03T03:08:52.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d827fc700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84063020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f6d78002f30 tx=0x7f6d78015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d73fff700 1 -- 172.21.2.111:0/633967772 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d78021070 con 0x7f6d840add50 2021-04-03T03:08:52.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d73fff700 1 -- 172.21.2.111:0/633967772 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78011e80 con 0x7f6d840add50 2021-04-03T03:08:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d73fff700 1 -- 172.21.2.111:0/633967772 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7801c7b0 con 0x7f6d840add50 2021-04-03T03:08:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d840abad0 con 0x7f6d840add50 2021-04-03T03:08:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.307+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d840aada0 con 0x7f6d840add50 2021-04-03T03:08:52.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.311+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d840aa480 con 0x7f6d840add50 2021-04-03T03:08:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.311+0000 7f6d73fff700 1 -- 172.21.2.111:0/633967772 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6d7800f070 con 0x7f6d840add50 2021-04-03T03:08:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.311+0000 7f6d73fff700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d60056ef0 0x7f6d600593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.311+0000 7f6d73fff700 1 -- 172.21.2.111:0/633967772 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f6d78071760 con 0x7f6d840add50 2021-04-03T03:08:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.311+0000 7f6d82ffd700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d60056ef0 0x7f6d600593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.311+0000 7f6d82ffd700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d60056ef0 0x7f6d600593b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f6d74002f30 tx=0x7f6d74006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.311+0000 7f6d73fff700 1 -- 172.21.2.111:0/633967772 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6d78046a30 con 0x7f6d840add50 2021-04-03T03:08:52.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6d84062d40 con 0x7f6d840add50 2021-04-03T03:08:52.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d73fff700 1 -- 172.21.2.111:0/633967772 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6d7804a050 con 0x7f6d840add50 2021-04-03T03:08:52.415 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:08:52.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d60056ef0 msgr2=0x7f6d600593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d60056ef0 0x7f6d600593b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f6d74002f30 tx=0x7f6d74006a60).stop 2021-04-03T03:08:52.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 msgr2=0x7f6d84063020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84063020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f6d78002f30 tx=0x7f6d78015000).stop 2021-04-03T03:08:52.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 shutdown_connections 2021-04-03T03:08:52.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6d840b17b0 0x7f6d84159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d60056ef0 0x7f6d600593b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6d840603f0 0x7f6d8414a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 --2- 172.21.2.111:0/633967772 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6d840add50 0x7f6d84063020 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 >> 172.21.2.111:0/633967772 conn(0x7f6d840b61c0 msgr2=0x7f6d8405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 shutdown_connections 2021-04-03T03:08:52.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.407+0000 7f6d83fff700 1 -- 172.21.2.111:0/633967772 wait complete. 2021-04-03T03:08:52.422 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:08:52.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.475+0000 7fbdb556e700 1 Processor -- start 2021-04-03T03:08:52.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.475+0000 7fbdb556e700 1 -- start start 2021-04-03T03:08:52.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.475+0000 7fbdb556e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbdb00603f0 0x7fbdb01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.475+0000 7fbdb556e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00add50 0x7fbdb0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.475+0000 7fbdb556e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb00607d0 con 0x7fbdb00b17b0 2021-04-03T03:08:52.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb01451c0 con 0x7fbdb00603f0 2021-04-03T03:08:52.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb014b090 con 0x7fbdb00add50 2021-04-03T03:08:52.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fbdb00b17b0 0x7fbdb0147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50602/0 (socket says 172.21.2.111:50602) 2021-04-03T03:08:52.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 -- 172.21.2.111:0/2805580613 learned_addr learned my addr 172.21.2.111:0/2805580613 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:52.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaf7fe700 1 --2- 172.21.2.111:0/2805580613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00add50 0x7fbdb0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaeffd700 1 -- 172.21.2.111:0/2805580613 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934535477 0 0) 0x7fbdb00607d0 con 0x7fbdb00b17b0 2021-04-03T03:08:52.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaeffd700 1 -- 172.21.2.111:0/2805580613 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd9c003240 con 0x7fbdb00b17b0 2021-04-03T03:08:52.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaffff700 1 --2- 172.21.2.111:0/2805580613 >> v2:172.21.2.115:3300/0 conn(0x7fbdb00603f0 0x7fbdb01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaeffd700 1 -- 172.21.2.111:0/2805580613 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 651927662 0 0) 0x7fbd9c003240 con 0x7fbdb00b17b0 2021-04-03T03:08:52.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaf7fe700 1 -- 172.21.2.111:0/2805580613 >> v2:172.21.2.115:3300/0 conn(0x7fbdb00603f0 msgr2=0x7fbdb01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:52.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaf7fe700 1 --2- 172.21.2.111:0/2805580613 >> v2:172.21.2.115:3300/0 conn(0x7fbdb00603f0 0x7fbdb01447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaf7fe700 1 -- 172.21.2.111:0/2805580613 >> v1:172.21.2.111:6789/0 conn(0x7fbdb00b17b0 legacy=0x7fbdb0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaf7fe700 1 -- 172.21.2.111:0/2805580613 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb014b330 con 0x7fbdb00add50 2021-04-03T03:08:52.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaf7fe700 1 --2- 172.21.2.111:0/2805580613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00add50 0x7fbdb0144cd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fbda0001870 tx=0x7fbda4006dd0).ready entity=mon.2 client_cookie=f953720c457f2eff server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaeffd700 1 -- 172.21.2.111:0/2805580613 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda00016d0 con 0x7fbdb00add50 2021-04-03T03:08:52.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaeffd700 1 -- 172.21.2.111:0/2805580613 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda400d570 con 0x7fbdb00add50 2021-04-03T03:08:52.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaeffd700 1 -- 172.21.2.111:0/2805580613 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda400d6e0 con 0x7fbdb00add50 2021-04-03T03:08:52.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- 172.21.2.111:0/2805580613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00add50 msgr2=0x7fbdb0144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 --2- 172.21.2.111:0/2805580613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00add50 0x7fbdb0144cd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fbda0001870 tx=0x7fbda4006dd0).stop 2021-04-03T03:08:52.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- 172.21.2.111:0/2805580613 shutdown_connections 2021-04-03T03:08:52.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 --2- 172.21.2.111:0/2805580613 >> v2:172.21.2.115:3300/0 conn(0x7fbdb00603f0 0x7fbdb01447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 --2- 172.21.2.111:0/2805580613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00add50 0x7fbdb0144cd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- 172.21.2.111:0/2805580613 >> 172.21.2.111:0/2805580613 conn(0x7fbdb00b61c0 msgr2=0x7fbdb005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- 172.21.2.111:0/2805580613 shutdown_connections 2021-04-03T03:08:52.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- 172.21.2.111:0/2805580613 wait complete. 2021-04-03T03:08:52.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 Processor -- start 2021-04-03T03:08:52.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- start start 2021-04-03T03:08:52.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00603f0 0x7fbdb01497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbdb00add50 0x7fbdb015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbdb00b17b0 0x7fbdb01435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00aaa50 con 0x7fbdb00b17b0 2021-04-03T03:08:52.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb0062a70 con 0x7fbdb00add50 2021-04-03T03:08:52.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb015abe0 con 0x7fbdb00603f0 2021-04-03T03:08:52.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbdb00b17b0 0x7fbdb01435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbdb00b17b0 0x7fbdb01435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49814/0 (socket says 172.21.2.111:49814) 2021-04-03T03:08:52.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 -- 172.21.2.111:0/3690118399 learned_addr learned my addr 172.21.2.111:0/3690118399 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaf7fe700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbdb00add50 0x7fbdb015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdaffff700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00603f0 0x7fbdb01497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 -- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00603f0 msgr2=0x7fbdb01497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00603f0 0x7fbdb01497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 -- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbdb00add50 msgr2=0x7fbdb015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbdb00add50 0x7fbdb015a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 -- 172.21.2.111:0/3690118399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0062ca0 con 0x7fbdb00b17b0 2021-04-03T03:08:52.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb4d6d700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbdb00b17b0 0x7fbdb01435a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fbda0000b80 tx=0x7fbda00016d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdad7fa700 1 -- 172.21.2.111:0/3690118399 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda0007170 con 0x7fbdb00b17b0 2021-04-03T03:08:52.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdad7fa700 1 -- 172.21.2.111:0/3690118399 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda00072d0 con 0x7fbdb00b17b0 2021-04-03T03:08:52.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdad7fa700 1 -- 172.21.2.111:0/3690118399 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda0004900 con 0x7fbdb00b17b0 2021-04-03T03:08:52.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb0062f40 con 0x7fbdb00b17b0 2021-04-03T03:08:52.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.479+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb00679e0 con 0x7fbdb00b17b0 2021-04-03T03:08:52.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.483+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb0057660 con 0x7fbdb00b17b0 2021-04-03T03:08:52.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.483+0000 7fbdad7fa700 1 -- 172.21.2.111:0/3690118399 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbda0004a60 con 0x7fbdb00b17b0 2021-04-03T03:08:52.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.483+0000 7fbdad7fa700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056ec0 0x7fbd90059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.483+0000 7fbdad7fa700 1 -- 172.21.2.111:0/3690118399 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fbda006fc90 con 0x7fbdb00b17b0 2021-04-03T03:08:52.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.483+0000 7fbdaffff700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056ec0 0x7fbd90059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.483+0000 7fbdaffff700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056ec0 0x7fbd90059380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fbda800b000 tx=0x7fbda8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.483+0000 7fbdad7fa700 1 -- 172.21.2.111:0/3690118399 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbda0044f60 con 0x7fbdb00b17b0 2021-04-03T03:08:52.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbdb00630d0 con 0x7fbdb00b17b0 2021-04-03T03:08:52.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdad7fa700 1 -- 172.21.2.111:0/3690118399 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbda0048580 con 0x7fbdb00b17b0 2021-04-03T03:08:52.586 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:52.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056ec0 msgr2=0x7fbd90059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056ec0 0x7fbd90059380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fbda800b000 tx=0x7fbda8008040).stop 2021-04-03T03:08:52.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbdb00b17b0 msgr2=0x7fbdb01435a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbdb00b17b0 0x7fbdb01435a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fbda0000b80 tx=0x7fbda00016d0).stop 2021-04-03T03:08:52.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 shutdown_connections 2021-04-03T03:08:52.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbdb00b17b0 0x7fbdb01435a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbdb00603f0 0x7fbdb01497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056ec0 0x7fbd90059380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 --2- 172.21.2.111:0/3690118399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbdb00add50 0x7fbdb015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 >> 172.21.2.111:0/3690118399 conn(0x7fbdb00b61c0 msgr2=0x7fbdb0147320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 shutdown_connections 2021-04-03T03:08:52.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.579+0000 7fbdb556e700 1 -- 172.21.2.111:0/3690118399 wait complete. 2021-04-03T03:08:52.594 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:08:52.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.643+0000 7f70f8e93700 1 Processor -- start 2021-04-03T03:08:52.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.643+0000 7f70f8e93700 1 -- start start 2021-04-03T03:08:52.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f70ec047580 0x7f70ec03bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec0d38b0 0x7f70ec006ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec043400 con 0x7f70ec0423a0 2021-04-03T03:08:52.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ec047960 con 0x7f70ec047580 2021-04-03T03:08:52.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec042ec0 con 0x7f70ec0d38b0 2021-04-03T03:08:52.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec0d38b0 0x7f70ec006ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec0d38b0 0x7f70ec006ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:33172/0 (socket says 172.21.2.111:33172) 2021-04-03T03:08:52.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 -- 172.21.2.111:0/2647584341 learned_addr learned my addr 172.21.2.111:0/2647584341 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 --2- 172.21.2.111:0/2647584341 >> v2:172.21.2.115:3300/0 conn(0x7f70ec047580 0x7f70ec03bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 -- 172.21.2.111:0/2647584341 >> v2:172.21.2.115:3300/0 conn(0x7f70ec047580 msgr2=0x7f70ec03bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:52.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 --2- 172.21.2.111:0/2647584341 >> v2:172.21.2.115:3300/0 conn(0x7f70ec047580 0x7f70ec03bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 -- 172.21.2.111:0/2647584341 >> v1:172.21.2.115:6789/0 conn(0x7f70ec0423a0 legacy=0x7f70ec03c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 -- 172.21.2.111:0/2647584341 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec043160 con 0x7f70ec0d38b0 2021-04-03T03:08:52.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 --2- 172.21.2.111:0/2647584341 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec0d38b0 0x7f70ec006ac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f70ec0d0310 tx=0x7f70f405ad70).ready entity=mon.2 client_cookie=80db5f2a011e4fdc server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f27fc700 1 -- 172.21.2.111:0/2647584341 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70f40450f0 con 0x7f70ec0d38b0 2021-04-03T03:08:52.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f27fc700 1 -- 172.21.2.111:0/2647584341 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f405bda0 con 0x7f70ec0d38b0 2021-04-03T03:08:52.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f27fc700 1 -- 172.21.2.111:0/2647584341 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70f405df60 con 0x7f70ec0d38b0 2021-04-03T03:08:52.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- 172.21.2.111:0/2647584341 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec0d38b0 msgr2=0x7f70ec006ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- 172.21.2.111:0/2647584341 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec0d38b0 0x7f70ec006ac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f70ec0d0310 tx=0x7f70f405ad70).stop 2021-04-03T03:08:52.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- 172.21.2.111:0/2647584341 shutdown_connections 2021-04-03T03:08:52.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- 172.21.2.111:0/2647584341 >> v2:172.21.2.115:3300/0 conn(0x7f70ec047580 0x7f70ec03bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- 172.21.2.111:0/2647584341 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec0d38b0 0x7f70ec006ac0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- 172.21.2.111:0/2647584341 >> 172.21.2.111:0/2647584341 conn(0x7f70ec00d9a0 msgr2=0x7f70ec046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- 172.21.2.111:0/2647584341 shutdown_connections 2021-04-03T03:08:52.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- 172.21.2.111:0/2647584341 wait complete. 2021-04-03T03:08:52.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 Processor -- start 2021-04-03T03:08:52.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- start start 2021-04-03T03:08:52.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f70ec0423a0 0x7f70ec0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec047580 0x7f70ec046180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f70ec0d38b0 0x7f70ec0e7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0e6f10 con 0x7f70ec0d38b0 2021-04-03T03:08:52.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0d2eb0 con 0x7f70ec0423a0 2021-04-03T03:08:52.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec03eb70 con 0x7f70ec047580 2021-04-03T03:08:52.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f70ec0d38b0 0x7f70ec0e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f70ec0d38b0 0x7f70ec0e7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49830/0 (socket says 172.21.2.111:49830) 2021-04-03T03:08:52.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f3fff700 1 -- 172.21.2.111:0/3764568371 learned_addr learned my addr 172.21.2.111:0/3764568371 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f70ec0423a0 0x7f70ec0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f2ffd700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec047580 0x7f70ec046180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 -- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec047580 msgr2=0x7f70ec046180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec047580 0x7f70ec046180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 -- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f70ec0d38b0 msgr2=0x7f70ec0e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f70ec0d38b0 0x7f70ec0e7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 -- 172.21.2.111:0/3764568371 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec046670 con 0x7f70ec0423a0 2021-04-03T03:08:52.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f37fe700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f70ec0423a0 0x7f70ec0d29c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f70ec03dd30 tx=0x7f70e0006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f0ff9700 1 -- 172.21.2.111:0/3764568371 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70e000ca10 con 0x7f70ec0423a0 2021-04-03T03:08:52.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f0ff9700 1 -- 172.21.2.111:0/3764568371 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e000cb70 con 0x7f70ec0423a0 2021-04-03T03:08:52.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f0ff9700 1 -- 172.21.2.111:0/3764568371 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70e000be80 con 0x7f70ec0423a0 2021-04-03T03:08:52.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec0d32d0 con 0x7f70ec0423a0 2021-04-03T03:08:52.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.647+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec0e77f0 con 0x7f70ec0423a0 2021-04-03T03:08:52.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.651+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec0d2ff0 con 0x7f70ec0423a0 2021-04-03T03:08:52.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.651+0000 7f70f0ff9700 1 -- 172.21.2.111:0/3764568371 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f70e001c460 con 0x7f70ec0423a0 2021-04-03T03:08:52.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.651+0000 7f70f0ff9700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f70dc056f20 0x7f70dc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.651+0000 7f70f0ff9700 1 -- 172.21.2.111:0/3764568371 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f70e0070a60 con 0x7f70ec0423a0 2021-04-03T03:08:52.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.651+0000 7f70f2ffd700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f70dc056f20 0x7f70dc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.651+0000 7f70f2ffd700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f70dc056f20 0x7f70dc0593e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f70e800d000 tx=0x7f70e8007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.651+0000 7f70f0ff9700 1 -- 172.21.2.111:0/3764568371 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f70e0045cd0 con 0x7f70ec0423a0 2021-04-03T03:08:52.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.747+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70ec004db0 con 0x7f70ec0423a0 2021-04-03T03:08:52.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.747+0000 7f70f0ff9700 1 -- 172.21.2.111:0/3764568371 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f70e00370a0 con 0x7f70ec0423a0 2021-04-03T03:08:52.756 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:52.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f70dc056f20 msgr2=0x7f70dc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f70dc056f20 0x7f70dc0593e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f70e800d000 tx=0x7f70e8007b70).stop 2021-04-03T03:08:52.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f70ec0423a0 msgr2=0x7f70ec0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f70ec0423a0 0x7f70ec0d29c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f70ec03dd30 tx=0x7f70e0006dd0).stop 2021-04-03T03:08:52.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 shutdown_connections 2021-04-03T03:08:52.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f70ec0d38b0 0x7f70ec0e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f70dc056f20 0x7f70dc0593e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f70ec0423a0 0x7f70ec0d29c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 --2- 172.21.2.111:0/3764568371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f70ec047580 0x7f70ec046180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 >> 172.21.2.111:0/3764568371 conn(0x7f70ec00d9a0 msgr2=0x7f70ec0cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 shutdown_connections 2021-04-03T03:08:52.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.751+0000 7f70f8e93700 1 -- 172.21.2.111:0/3764568371 wait complete. 2021-04-03T03:08:52.764 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:08:52.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 Processor -- start 2021-04-03T03:08:52.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- start start 2021-04-03T03:08:52.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f06ac0603f0 0x7f06ac0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f06ac0b5620 0x7f06ac142310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac0607d0 con 0x7f06ac0b44a0 2021-04-03T03:08:52.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac1433c0 con 0x7f06ac0603f0 2021-04-03T03:08:52.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac0b3f80 con 0x7f06ac0b5620 2021-04-03T03:08:52.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f06ac0603f0 0x7f06ac0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f06ac0603f0 0x7f06ac0b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49834/0 (socket says 172.21.2.111:49834) 2021-04-03T03:08:52.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 -- 172.21.2.111:0/1424019444 learned_addr learned my addr 172.21.2.111:0/1424019444 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 -- 172.21.2.111:0/1424019444 >> v2:172.21.2.115:3300/0 conn(0x7f06ac0b5620 msgr2=0x7f06ac142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:52.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b0f5e700 1 -- 172.21.2.111:0/1424019444 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119784060 0 0) 0x7f06ac0607d0 con 0x7f06ac0b44a0 2021-04-03T03:08:52.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 --2- 172.21.2.111:0/1424019444 >> v2:172.21.2.115:3300/0 conn(0x7f06ac0b5620 0x7f06ac142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 -- 172.21.2.111:0/1424019444 >> v1:172.21.2.111:6789/0 conn(0x7f06ac0b44a0 legacy=0x7f06ac0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 -- 172.21.2.111:0/1424019444 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0b4c60 con 0x7f06ac0603f0 2021-04-03T03:08:52.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b1f60700 1 --2- 172.21.2.111:0/1424019444 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0603f0 0x7f06ac0b2480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f06ac0b2320 tx=0x7f06a8006e90).ready entity=mon.0 client_cookie=4b02bb5794a94155 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b0f5e700 1 -- 172.21.2.111:0/1424019444 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a8009810 con 0x7f06ac0603f0 2021-04-03T03:08:52.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b0f5e700 1 -- 172.21.2.111:0/1424019444 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8009970 con 0x7f06ac0603f0 2021-04-03T03:08:52.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b0f5e700 1 -- 172.21.2.111:0/1424019444 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a8009cd0 con 0x7f06ac0603f0 2021-04-03T03:08:52.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- 172.21.2.111:0/1424019444 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0603f0 msgr2=0x7f06ac0b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1424019444 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0603f0 0x7f06ac0b2480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f06ac0b2320 tx=0x7f06a8006e90).stop 2021-04-03T03:08:52.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- 172.21.2.111:0/1424019444 shutdown_connections 2021-04-03T03:08:52.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1424019444 >> v2:172.21.2.115:3300/0 conn(0x7f06ac0b5620 0x7f06ac142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1424019444 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0603f0 0x7f06ac0b2480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- 172.21.2.111:0/1424019444 >> 172.21.2.111:0/1424019444 conn(0x7f06ac05c7d0 msgr2=0x7f06ac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- 172.21.2.111:0/1424019444 shutdown_connections 2021-04-03T03:08:52.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- 172.21.2.111:0/1424019444 wait complete. 2021-04-03T03:08:52.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 Processor -- start 2021-04-03T03:08:52.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- start start 2021-04-03T03:08:52.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06ac0603f0 0x7f06ac0b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06ac0b44a0 0x7f06ac0ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0b5620 0x7f06ac145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1139a0 con 0x7f06ac0b5620 2021-04-03T03:08:52.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac145880 con 0x7f06ac0b44a0 2021-04-03T03:08:52.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2f62700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac145b20 con 0x7f06ac0603f0 2021-04-03T03:08:52.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2761700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0b5620 0x7f06ac145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2761700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0b5620 0x7f06ac145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49844/0 (socket says 172.21.2.111:49844) 2021-04-03T03:08:52.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.815+0000 7f06b2761700 1 -- 172.21.2.111:0/1087431652 learned_addr learned my addr 172.21.2.111:0/1087431652 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2761700 1 -- 172.21.2.111:0/1087431652 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06ac0603f0 msgr2=0x7f06ac0b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2761700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06ac0603f0 0x7f06ac0b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2761700 1 -- 172.21.2.111:0/1087431652 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06ac0b44a0 msgr2=0x7f06ac0ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2761700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06ac0b44a0 0x7f06ac0ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2761700 1 -- 172.21.2.111:0/1087431652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac1136c0 con 0x7f06ac0b5620 2021-04-03T03:08:52.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2761700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0b5620 0x7f06ac145fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f06a40032b0 tx=0x7f06a40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f069effd700 1 -- 172.21.2.111:0/1087431652 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a400c980 con 0x7f06ac0b5620 2021-04-03T03:08:52.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f069effd700 1 -- 172.21.2.111:0/1087431652 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a400cae0 con 0x7f06ac0b5620 2021-04-03T03:08:52.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f069effd700 1 -- 172.21.2.111:0/1087431652 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a400ce40 con 0x7f06ac0b5620 2021-04-03T03:08:52.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac0ae500 con 0x7f06ac0b5620 2021-04-03T03:08:52.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac0ad300 con 0x7f06ac0b5620 2021-04-03T03:08:52.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac145ce0 con 0x7f06ac0b5620 2021-04-03T03:08:52.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f069effd700 1 -- 172.21.2.111:0/1087431652 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f06a401c070 con 0x7f06ac0b5620 2021-04-03T03:08:52.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f069effd700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f068c056ef0 0x7f068c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f069effd700 1 -- 172.21.2.111:0/1087431652 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f06a406fca0 con 0x7f06ac0b5620 2021-04-03T03:08:52.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b1f60700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f068c056ef0 0x7f068c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.819+0000 7f06b1f60700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f068c056ef0 0x7f068c0593b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f06ac0b2320 tx=0x7f06a8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.823+0000 7f069effd700 1 -- 172.21.2.111:0/1087431652 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f06a4045080 con 0x7f06ac0b5620 2021-04-03T03:08:52.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06ac069270 con 0x7f06ac0b5620 2021-04-03T03:08:52.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f069effd700 1 -- 172.21.2.111:0/1087431652 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f06a40486a0 con 0x7f06ac0b5620 2021-04-03T03:08:52.924 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:52.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f068c056ef0 msgr2=0x7f068c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f068c056ef0 0x7f068c0593b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f06ac0b2320 tx=0x7f06a8015000).stop 2021-04-03T03:08:52.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0b5620 msgr2=0x7f06ac145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0b5620 0x7f06ac145fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f06a40032b0 tx=0x7f06a40091a0).stop 2021-04-03T03:08:52.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 shutdown_connections 2021-04-03T03:08:52.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06ac0b5620 0x7f06ac145fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06ac0603f0 0x7f06ac0b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f068c056ef0 0x7f068c0593b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 --2- 172.21.2.111:0/1087431652 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06ac0b44a0 0x7f06ac0ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 >> 172.21.2.111:0/1087431652 conn(0x7f06ac05c7d0 msgr2=0x7f06ac0b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 shutdown_connections 2021-04-03T03:08:52.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.915+0000 7f06b2f62700 1 -- 172.21.2.111:0/1087431652 wait complete. 2021-04-03T03:08:52.931 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:08:52.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8293fff700 1 Processor -- start 2021-04-03T03:08:52.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8293fff700 1 -- start start 2021-04-03T03:08:52.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8293fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f82940603f0 0x7f82940b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8293fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 0x7f82940b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8293fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82940607d0 con 0x7f82940b12b0 2021-04-03T03:08:52.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8293fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82940af400 con 0x7f82940603f0 2021-04-03T03:08:52.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8293fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82940af6a0 con 0x7f829414b700 2021-04-03T03:08:52.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f82927fc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 0x7f82940b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f82937fe700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f82940b12b0 0x7f82941465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:34944/0 (socket says 172.21.2.111:34944) 2021-04-03T03:08:52.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f82937fe700 1 -- 172.21.2.111:0/1946321846 learned_addr learned my addr 172.21.2.111:0/1946321846 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:52.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.979+0000 7f8292ffd700 1 --2- 172.21.2.111:0/1946321846 >> v2:172.21.2.115:3300/0 conn(0x7f82940603f0 0x7f82940b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8292ffd700 1 -- 172.21.2.111:0/1946321846 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 msgr2=0x7f82940b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:52.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8292ffd700 1 --2- 172.21.2.111:0/1946321846 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 0x7f82940b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8292ffd700 1 -- 172.21.2.111:0/1946321846 >> v1:172.21.2.115:6789/0 conn(0x7f82940b12b0 legacy=0x7f82941465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8292ffd700 1 -- 172.21.2.111:0/1946321846 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940afb40 con 0x7f82940603f0 2021-04-03T03:08:52.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8292ffd700 1 --2- 172.21.2.111:0/1946321846 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 0x7f82940b0090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f827c002f30 tx=0x7f827c006dd0).ready entity=mon.1 client_cookie=f8ff010e77448b3c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8291ffb700 1 -- 172.21.2.111:0/1946321846 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827c009760 con 0x7f82940603f0 2021-04-03T03:08:52.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8291ffb700 1 -- 172.21.2.111:0/1946321846 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c0098c0 con 0x7f82940603f0 2021-04-03T03:08:52.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8291ffb700 1 -- 172.21.2.111:0/1946321846 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827c009c20 con 0x7f82940603f0 2021-04-03T03:08:52.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- 172.21.2.111:0/1946321846 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 msgr2=0x7f82940b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 --2- 172.21.2.111:0/1946321846 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 0x7f82940b0090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f827c002f30 tx=0x7f827c006dd0).stop 2021-04-03T03:08:52.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- 172.21.2.111:0/1946321846 shutdown_connections 2021-04-03T03:08:52.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 --2- 172.21.2.111:0/1946321846 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 0x7f82940b0090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 --2- 172.21.2.111:0/1946321846 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 0x7f82940b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:52.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- 172.21.2.111:0/1946321846 >> 172.21.2.111:0/1946321846 conn(0x7f82940b61c0 msgr2=0x7f829405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:52.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- 172.21.2.111:0/1946321846 shutdown_connections 2021-04-03T03:08:52.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- 172.21.2.111:0/1946321846 wait complete. 2021-04-03T03:08:52.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 Processor -- start 2021-04-03T03:08:52.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- start start 2021-04-03T03:08:52.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 0x7f829414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82940b12b0 0x7f82940635c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 0x7f8294063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:52.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940a9430 con 0x7f82940b12b0 2021-04-03T03:08:52.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940a9690 con 0x7f82940603f0 2021-04-03T03:08:52.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f8293fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82940a9930 con 0x7f829414b700 2021-04-03T03:08:52.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82940b12b0 0x7f82940635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:52.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82940b12b0 0x7f82940635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49858/0 (socket says 172.21.2.111:49858) 2021-04-03T03:08:52.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 -- 172.21.2.111:0/18103106 learned_addr learned my addr 172.21.2.111:0/18103106 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:52.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 -- 172.21.2.111:0/18103106 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 msgr2=0x7f8294063ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 0x7f8294063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 -- 172.21.2.111:0/18103106 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 msgr2=0x7f829414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:52.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 0x7f829414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:52.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 -- 172.21.2.111:0/18103106 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940a9030 con 0x7f82940b12b0 2021-04-03T03:08:52.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.983+0000 7f82927fc700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82940b12b0 0x7f82940635c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f828400c000 tx=0x7f8284006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:52.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f828bfff700 1 -- 172.21.2.111:0/18103106 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f828400ca10 con 0x7f82940b12b0 2021-04-03T03:08:52.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f828bfff700 1 -- 172.21.2.111:0/18103106 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828400cb70 con 0x7f82940b12b0 2021-04-03T03:08:53.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8294159ad0 con 0x7f82940b12b0 2021-04-03T03:08:53.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82940641b0 con 0x7f82940b12b0 2021-04-03T03:08:53.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f828bfff700 1 -- 172.21.2.111:0/18103106 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82840123f0 con 0x7f82940b12b0 2021-04-03T03:08:53.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f828bfff700 1 -- 172.21.2.111:0/18103106 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8284012550 con 0x7f82940b12b0 2021-04-03T03:08:53.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82940aa6d0 con 0x7f82940b12b0 2021-04-03T03:08:53.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f828bfff700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8278056f10 0x7f82780593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f828bfff700 1 -- 172.21.2.111:0/18103106 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f82840707f0 con 0x7f82940b12b0 2021-04-03T03:08:53.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f8292ffd700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8278056f10 0x7f82780593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f8292ffd700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8278056f10 0x7f82780593d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f827c002f30 tx=0x7f827c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:52.987+0000 7f828bfff700 1 -- 172.21.2.111:0/18103106 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8284045a60 con 0x7f82940b12b0 2021-04-03T03:08:53.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82940633c0 con 0x7f82940b12b0 2021-04-03T03:08:53.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f828bfff700 1 -- 172.21.2.111:0/18103106 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f8284049080 con 0x7f82940b12b0 2021-04-03T03:08:53.120 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:53.120 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1066299897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1066299897},{"type":"v1","addr":"172.21.2.115:6833","nonce":1066299897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:08:44.422766+0000","modified":"2021-04-03T03:08:45.425728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2149096292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2149096292},{"type":"v1","addr":"172.21.2.111:6835","nonce":2149096292}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-04-03T03:08:53.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8278056f10 msgr2=0x7f82780593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8278056f10 0x7f82780593d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f827c002f30 tx=0x7f827c015000).stop 2021-04-03T03:08:53.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82940b12b0 msgr2=0x7f82940635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82940b12b0 0x7f82940635c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f828400c000 tx=0x7f8284006dd0).stop 2021-04-03T03:08:53.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 shutdown_connections 2021-04-03T03:08:53.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f829414b700 0x7f8294063ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8278056f10 0x7f82780593d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82940603f0 0x7f829414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 --2- 172.21.2.111:0/18103106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82940b12b0 0x7f82940635c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 >> 172.21.2.111:0/18103106 conn(0x7f82940b61c0 msgr2=0x7f82940b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 shutdown_connections 2021-04-03T03:08:53.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.111+0000 7f8293fff700 1 -- 172.21.2.111:0/18103106 wait complete. 2021-04-03T03:08:53.124 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 5 2021-04-03T03:08:53.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:08:44.422766+0000', 'modified': '2021-04-03T03:08:45.425728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/2149096292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 2149096292}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 2149096292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:08:53.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:08:53.330 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:08:53.401 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":4239,"want_state":"up:active","state":"up:active","rank_uptime":8.9681428539999999,"mdsmap_epoch":5,"osdmap_epoch":24,"osdmap_epoch_barrier":23,"uptime":14.186328702999999} 2021-04-03T03:08:53.401 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 4239, "mdsmap_epoch": 5, "osdmap_epoch": 24, "osdmap_epoch_barrier": 23, "rank_uptime": 8.968142854, "state": "up:active", "uptime": 14.186328703, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:08:53.402 INFO:teuthology.run_tasks:Running task ceph-fuse... 2021-04-03T03:08:53.421 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2021-04-03T03:08:53.422 INFO:tasks.ceph_fuse:config is None 2021-04-03T03:08:53.422 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None} 2021-04-03T03:08:53.423 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2021-04-03T03:08:53.423 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:53.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597f59e700 1 Processor -- start 2021-04-03T03:08:53.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597f59e700 1 -- start start 2021-04-03T03:08:53.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f598014b700 0x7f59800b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800607d0 con 0x7f59800b0c90 2021-04-03T03:08:53.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800af400 con 0x7f59800603f0 2021-04-03T03:08:53.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800af6a0 con 0x7f598014b700 2021-04-03T03:08:53.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597ed9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f59800b0c90 0x7f5980147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50654/0 (socket says 172.21.2.111:50654) 2021-04-03T03:08:53.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597ed9d700 1 -- 172.21.2.111:0/742751829 learned_addr learned my addr 172.21.2.111:0/742751829 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:53.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.471+0000 7f597dd9b700 1 --2- 172.21.2.111:0/742751829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f598014b700 0x7f59800b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.475+0000 7f597dd9b700 1 -- 172.21.2.111:0/742751829 >> v1:172.21.2.115:6789/0 conn(0x7f59800603f0 legacy=0x7f59800b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.475+0000 7f597dd9b700 1 -- 172.21.2.111:0/742751829 >> v1:172.21.2.111:6789/0 conn(0x7f59800b0c90 legacy=0x7f5980147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.475+0000 7f597dd9b700 1 -- 172.21.2.111:0/742751829 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800afb40 con 0x7f598014b700 2021-04-03T03:08:53.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.475+0000 7f597dd9b700 1 --2- 172.21.2.111:0/742751829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f598014b700 0x7f59800b07a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5968002f30 tx=0x7f5968006dd0).ready entity=mon.2 client_cookie=cd1d906ceb76e965 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597d59a700 1 -- 172.21.2.111:0/742751829 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59680097f0 con 0x7f598014b700 2021-04-03T03:08:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597d59a700 1 -- 172.21.2.111:0/742751829 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968009950 con 0x7f598014b700 2021-04-03T03:08:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597d59a700 1 -- 172.21.2.111:0/742751829 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5968009cb0 con 0x7f598014b700 2021-04-03T03:08:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 -- 172.21.2.111:0/742751829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f598014b700 msgr2=0x7f59800b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 --2- 172.21.2.111:0/742751829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f598014b700 0x7f59800b07a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5968002f30 tx=0x7f5968006dd0).stop 2021-04-03T03:08:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 -- 172.21.2.111:0/742751829 shutdown_connections 2021-04-03T03:08:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 --2- 172.21.2.111:0/742751829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f598014b700 0x7f59800b07a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 -- 172.21.2.111:0/742751829 >> 172.21.2.111:0/742751829 conn(0x7f59800b61c0 msgr2=0x7f598005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 -- 172.21.2.111:0/742751829 shutdown_connections 2021-04-03T03:08:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 -- 172.21.2.111:0/742751829 wait complete. 2021-04-03T03:08:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 Processor -- start 2021-04-03T03:08:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 -- start start 2021-04-03T03:08:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59800603f0 0x7f5980156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59800b0c90 0x7f59800ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.479+0000 7f597f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f598014b700 0x7f5980156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980159a00 con 0x7f59800b0c90 2021-04-03T03:08:53.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980065720 con 0x7f598014b700 2021-04-03T03:08:53.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800659c0 con 0x7f59800603f0 2021-04-03T03:08:53.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59800b0c90 0x7f59800ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59800b0c90 0x7f59800ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49870/0 (socket says 172.21.2.111:49870) 2021-04-03T03:08:53.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 -- 172.21.2.111:0/1135725283 learned_addr learned my addr 172.21.2.111:0/1135725283 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:53.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 -- 172.21.2.111:0/1135725283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59800603f0 msgr2=0x7f5980156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59800603f0 0x7f5980156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 -- 172.21.2.111:0/1135725283 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f598014b700 msgr2=0x7f5980156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f598014b700 0x7f5980156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 -- 172.21.2.111:0/1135725283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5980065c60 con 0x7f59800b0c90 2021-04-03T03:08:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597dd9b700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59800b0c90 0x7f59800ace50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f5968002f30 tx=0x7f5968015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f59777fe700 1 -- 172.21.2.111:0/1135725283 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5968011e00 con 0x7f59800b0c90 2021-04-03T03:08:53.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f59777fe700 1 -- 172.21.2.111:0/1135725283 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596801e460 con 0x7f59800b0c90 2021-04-03T03:08:53.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59800a74e0 con 0x7f59800b0c90 2021-04-03T03:08:53.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59800a7130 con 0x7f59800b0c90 2021-04-03T03:08:53.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f59777fe700 1 -- 172.21.2.111:0/1135725283 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596801e7c0 con 0x7f59800b0c90 2021-04-03T03:08:53.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5980065df0 con 0x7f59800b0c90 2021-04-03T03:08:53.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f59777fe700 1 -- 172.21.2.111:0/1135725283 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5968025030 con 0x7f59800b0c90 2021-04-03T03:08:53.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f59777fe700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5964056ef0 0x7f59640593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f59777fe700 1 -- 172.21.2.111:0/1135725283 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5968071a10 con 0x7f59800b0c90 2021-04-03T03:08:53.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597e59c700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5964056ef0 0x7f59640593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.483+0000 7f597e59c700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5964056ef0 0x7f59640593b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f59800b5640 tx=0x7f59700077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.487+0000 7f59777fe700 1 -- 172.21.2.111:0/1135725283 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5968046ce0 con 0x7f59800b0c90 2021-04-03T03:08:53.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.575+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5980057660 con 0x7f59800b0c90 2021-04-03T03:08:53.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.575+0000 7f59777fe700 1 -- 172.21.2.111:0/1135725283 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f5968023090 con 0x7f59800b0c90 2021-04-03T03:08:53.584 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:53.585 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:53.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5964056ef0 msgr2=0x7f59640593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5964056ef0 0x7f59640593b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f59800b5640 tx=0x7f59700077f0).stop 2021-04-03T03:08:53.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59800b0c90 msgr2=0x7f59800ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59800b0c90 0x7f59800ace50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f5968002f30 tx=0x7f5968015000).stop 2021-04-03T03:08:53.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 shutdown_connections 2021-04-03T03:08:53.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5964056ef0 0x7f59640593b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f598014b700 0x7f5980156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59800603f0 0x7f5980156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 --2- 172.21.2.111:0/1135725283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59800b0c90 0x7f59800ace50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 >> 172.21.2.111:0/1135725283 conn(0x7f59800b61c0 msgr2=0x7f5980067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 shutdown_connections 2021-04-03T03:08:53.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.579+0000 7f597f59e700 1 -- 172.21.2.111:0/1135725283 wait complete. 2021-04-03T03:08:53.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:08:53.593 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:08:53.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 Processor -- start 2021-04-03T03:08:53.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- start start 2021-04-03T03:08:53.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f38a8060260 0x7f38a80ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8147750 0x7f38a80b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a8147bd0 con 0x7f38a80b3c90 2021-04-03T03:08:53.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a8141c50 con 0x7f38a8060260 2021-04-03T03:08:53.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80b28e0 con 0x7f38a8147750 2021-04-03T03:08:53.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f38a8060260 0x7f38a80ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f38a8060260 0x7f38a80ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49876/0 (socket says 172.21.2.111:49876) 2021-04-03T03:08:53.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 -- 172.21.2.111:0/1371020628 learned_addr learned my addr 172.21.2.111:0/1371020628 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:53.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 -- 172.21.2.111:0/1371020628 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8147750 msgr2=0x7f38a80b37a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:53.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 --2- 172.21.2.111:0/1371020628 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8147750 0x7f38a80b37a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 -- 172.21.2.111:0/1371020628 >> v1:172.21.2.115:6789/0 conn(0x7f38a80b3c90 legacy=0x7f38a814b1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:53.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 -- 172.21.2.111:0/1371020628 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80b56f0 con 0x7f38a8060260 2021-04-03T03:08:53.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38acad5700 1 --2- 172.21.2.111:0/1371020628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8060260 0x7f38a80ad170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f38a806b3e0 tx=0x7f3898006e90).ready entity=mon.0 client_cookie=fd5c51ddfa981a31 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38a77fe700 1 -- 172.21.2.111:0/1371020628 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38980098e0 con 0x7f38a8060260 2021-04-03T03:08:53.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38a77fe700 1 -- 172.21.2.111:0/1371020628 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898009a40 con 0x7f38a8060260 2021-04-03T03:08:53.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38a77fe700 1 -- 172.21.2.111:0/1371020628 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3898009de0 con 0x7f38a8060260 2021-04-03T03:08:53.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- 172.21.2.111:0/1371020628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8060260 msgr2=0x7f38a80ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- 172.21.2.111:0/1371020628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8060260 0x7f38a80ad170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f38a806b3e0 tx=0x7f3898006e90).stop 2021-04-03T03:08:53.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- 172.21.2.111:0/1371020628 shutdown_connections 2021-04-03T03:08:53.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- 172.21.2.111:0/1371020628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8060260 0x7f38a80ad170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- 172.21.2.111:0/1371020628 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8147750 0x7f38a80b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- 172.21.2.111:0/1371020628 >> 172.21.2.111:0/1371020628 conn(0x7f38a805c7d0 msgr2=0x7f38a805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- 172.21.2.111:0/1371020628 shutdown_connections 2021-04-03T03:08:53.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- 172.21.2.111:0/1371020628 wait complete. 2021-04-03T03:08:53.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 Processor -- start 2021-04-03T03:08:53.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- start start 2021-04-03T03:08:53.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8060260 0x7f38a80ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38a80b3c90 0x7f38a8065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8147750 0x7f38a80ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a8157930 con 0x7f38a8147750 2021-04-03T03:08:53.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80afda0 con 0x7f38a80b3c90 2021-04-03T03:08:53.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38adad7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80b0040 con 0x7f38a8060260 2021-04-03T03:08:53.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38ad2d6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8147750 0x7f38a80ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38ad2d6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8147750 0x7f38a80ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49884/0 (socket says 172.21.2.111:49884) 2021-04-03T03:08:53.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.643+0000 7f38ad2d6700 1 -- 172.21.2.111:0/1612376160 learned_addr learned my addr 172.21.2.111:0/1612376160 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:53.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38ad2d6700 1 -- 172.21.2.111:0/1612376160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8060260 msgr2=0x7f38a80ad4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:08:53.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38ad2d6700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8060260 0x7f38a80ad4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38ad2d6700 1 -- 172.21.2.111:0/1612376160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38a80b3c90 msgr2=0x7f38a8065ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:08:53.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38ad2d6700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38a80b3c90 0x7f38a8065ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38ad2d6700 1 -- 172.21.2.111:0/1612376160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80b02e0 con 0x7f38a8147750 2021-04-03T03:08:53.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38ad2d6700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8147750 0x7f38a80ae8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f38a00032f0 tx=0x7f38a00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38a5ffb700 1 -- 172.21.2.111:0/1612376160 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38a000c9c0 con 0x7f38a8147750 2021-04-03T03:08:53.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38a5ffb700 1 -- 172.21.2.111:0/1612376160 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a000cb20 con 0x7f38a8147750 2021-04-03T03:08:53.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a80ae470 con 0x7f38a8147750 2021-04-03T03:08:53.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a8147210 con 0x7f38a8147750 2021-04-03T03:08:53.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38a5ffb700 1 -- 172.21.2.111:0/1612376160 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38a0015420 con 0x7f38a8147750 2021-04-03T03:08:53.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a8065fb0 con 0x7f38a8147750 2021-04-03T03:08:53.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38a5ffb700 1 -- 172.21.2.111:0/1612376160 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f38a001c070 con 0x7f38a8147750 2021-04-03T03:08:53.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38a5ffb700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f388c056fb0 0x7f388c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38a5ffb700 1 -- 172.21.2.111:0/1612376160 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f38a006ff50 con 0x7f38a8147750 2021-04-03T03:08:53.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38acad5700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f388c056fb0 0x7f388c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.647+0000 7f38acad5700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f388c056fb0 0x7f388c059470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f38a806b3e0 tx=0x7f3898015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.651+0000 7f38a5ffb700 1 -- 172.21.2.111:0/1612376160 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f38a00451a0 con 0x7f38a8147750 2021-04-03T03:08:53.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.739+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f38a80b0470 con 0x7f38a8147750 2021-04-03T03:08:53.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.739+0000 7f38a5ffb700 1 -- 172.21.2.111:0/1612376160 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f38a00487c0 con 0x7f38a8147750 2021-04-03T03:08:53.749 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:08:53.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f388c056fb0 msgr2=0x7f388c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f388c056fb0 0x7f388c059470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f38a806b3e0 tx=0x7f3898015000).stop 2021-04-03T03:08:53.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8147750 msgr2=0x7f38a80ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8147750 0x7f38a80ae8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f38a00032f0 tx=0x7f38a00091a0).stop 2021-04-03T03:08:53.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 shutdown_connections 2021-04-03T03:08:53.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38a8147750 0x7f38a80ae8d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38a8060260 0x7f38a80ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f388c056fb0 0x7f388c059470 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 --2- 172.21.2.111:0/1612376160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38a80b3c90 0x7f38a8065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 >> 172.21.2.111:0/1612376160 conn(0x7f38a805c7d0 msgr2=0x7f38a814b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 shutdown_connections 2021-04-03T03:08:53.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.743+0000 7f38adad7700 1 -- 172.21.2.111:0/1612376160 wait complete. 2021-04-03T03:08:53.756 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:08:53.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124bfff700 1 Processor -- start 2021-04-03T03:08:53.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124bfff700 1 -- start start 2021-04-03T03:08:53.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0603f0 0x7f124c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f124c0add50 0x7f124c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0607d0 con 0x7f124c0b17b0 2021-04-03T03:08:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c1451c0 con 0x7f124c0add50 2021-04-03T03:08:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c14b090 con 0x7f124c0603f0 2021-04-03T03:08:53.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124affd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0603f0 0x7f124c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124a7fc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f124c0add50 0x7f124c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124affd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0603f0 0x7f124c1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:33238/0 (socket says 172.21.2.111:33238) 2021-04-03T03:08:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124affd700 1 -- 172.21.2.111:0/216069199 learned_addr learned my addr 172.21.2.111:0/216069199 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124a7fc700 1 -- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0603f0 msgr2=0x7f124c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124a7fc700 1 --2- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0603f0 0x7f124c1447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124a7fc700 1 -- 172.21.2.111:0/216069199 >> v1:172.21.2.115:6789/0 conn(0x7f124c0b17b0 legacy=0x7f124c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124a7fc700 1 -- 172.21.2.111:0/216069199 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c14b330 con 0x7f124c0add50 2021-04-03T03:08:53.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124affd700 1 --2- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0603f0 0x7f124c1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:53.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.807+0000 7f124a7fc700 1 --2- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0add50 0x7f124c144cd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1234002f30 tx=0x7f1234006dd0).ready entity=mon.1 client_cookie=7d0d1d0fdbab5f24 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f1249ffb700 1 -- 172.21.2.111:0/216069199 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1234009760 con 0x7f124c0add50 2021-04-03T03:08:53.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f1249ffb700 1 -- 172.21.2.111:0/216069199 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12340098c0 con 0x7f124c0add50 2021-04-03T03:08:53.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f1249ffb700 1 -- 172.21.2.111:0/216069199 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1234009c20 con 0x7f124c0add50 2021-04-03T03:08:53.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0add50 msgr2=0x7f124c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 --2- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0add50 0x7f124c144cd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1234002f30 tx=0x7f1234006dd0).stop 2021-04-03T03:08:53.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- 172.21.2.111:0/216069199 shutdown_connections 2021-04-03T03:08:53.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 --2- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0603f0 0x7f124c1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 --2- 172.21.2.111:0/216069199 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0add50 0x7f124c144cd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- 172.21.2.111:0/216069199 >> 172.21.2.111:0/216069199 conn(0x7f124c0b61c0 msgr2=0x7f124c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- 172.21.2.111:0/216069199 shutdown_connections 2021-04-03T03:08:53.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- 172.21.2.111:0/216069199 wait complete. 2021-04-03T03:08:53.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 Processor -- start 2021-04-03T03:08:53.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- start start 2021-04-03T03:08:53.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0603f0 0x7f124c157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f124c0add50 0x7f124c0a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0b17b0 0x7f124c0a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c062b20 con 0x7f124c0add50 2021-04-03T03:08:53.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0a8a40 con 0x7f124c0603f0 2021-04-03T03:08:53.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c157950 con 0x7f124c0b17b0 2021-04-03T03:08:53.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f124c0add50 0x7f124c0a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f124c0add50 0x7f124c0a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49896/0 (socket says 172.21.2.111:49896) 2021-04-03T03:08:53.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 -- 172.21.2.111:0/939696962 learned_addr learned my addr 172.21.2.111:0/939696962 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:53.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 -- 172.21.2.111:0/939696962 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0b17b0 msgr2=0x7f124c0a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0b17b0 0x7f124c0a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 -- 172.21.2.111:0/939696962 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0603f0 msgr2=0x7f124c157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0603f0 0x7f124c157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 -- 172.21.2.111:0/939696962 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c156a80 con 0x7f124c0add50 2021-04-03T03:08:53.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124a7fc700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f124c0add50 0x7f124c0a9760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f124c157ae0 tx=0x7f1234015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f1243fff700 1 -- 172.21.2.111:0/939696962 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123401d450 con 0x7f124c0add50 2021-04-03T03:08:53.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f1243fff700 1 -- 172.21.2.111:0/939696962 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123401d5b0 con 0x7f124c0add50 2021-04-03T03:08:53.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c156d20 con 0x7f124c0add50 2021-04-03T03:08:53.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f1243fff700 1 -- 172.21.2.111:0/939696962 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123401d910 con 0x7f124c0add50 2021-04-03T03:08:53.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.811+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c063870 con 0x7f124c0add50 2021-04-03T03:08:53.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.815+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c157150 con 0x7f124c0add50 2021-04-03T03:08:53.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.815+0000 7f1243fff700 1 -- 172.21.2.111:0/939696962 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f123400f070 con 0x7f124c0add50 2021-04-03T03:08:53.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.815+0000 7f1243fff700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1230056f70 0x7f1230059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.815+0000 7f1243fff700 1 -- 172.21.2.111:0/939696962 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f1234071e70 con 0x7f124c0add50 2021-04-03T03:08:53.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.815+0000 7f124affd700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1230056f70 0x7f1230059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.815+0000 7f124affd700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1230056f70 0x7f1230059430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f123c00c000 tx=0x7f123c006e90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.815+0000 7f1243fff700 1 -- 172.21.2.111:0/939696962 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1234047140 con 0x7f124c0add50 2021-04-03T03:08:53.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f124c057660 con 0x7f124c0add50 2021-04-03T03:08:53.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f1243fff700 1 -- 172.21.2.111:0/939696962 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f123404a760 con 0x7f124c0add50 2021-04-03T03:08:53.918 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:53.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1230056f70 msgr2=0x7f1230059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1230056f70 0x7f1230059430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f123c00c000 tx=0x7f123c006e90).stop 2021-04-03T03:08:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f124c0add50 msgr2=0x7f124c0a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f124c0add50 0x7f124c0a9760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f124c157ae0 tx=0x7f1234015000).stop 2021-04-03T03:08:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 shutdown_connections 2021-04-03T03:08:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f124c0b17b0 0x7f124c0a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1230056f70 0x7f1230059430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f124c0603f0 0x7f124c157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 --2- 172.21.2.111:0/939696962 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f124c0add50 0x7f124c0a9760 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 >> 172.21.2.111:0/939696962 conn(0x7f124c0b61c0 msgr2=0x7f124c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 shutdown_connections 2021-04-03T03:08:53.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.911+0000 7f124bfff700 1 -- 172.21.2.111:0/939696962 wait complete. 2021-04-03T03:08:53.926 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:08:53.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 Processor -- start 2021-04-03T03:08:53.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 -- start start 2021-04-03T03:08:53.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042080 0x7f79ac03ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79ac0cdcf0 0x7f79ac0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac039580 con 0x7f79ac042850 2021-04-03T03:08:53.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ac03da60 con 0x7f79ac0cdcf0 2021-04-03T03:08:53.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac042460 con 0x7f79ac042080 2021-04-03T03:08:53.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b2d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79ac0cdcf0 0x7f79ac0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b1d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f79ac042850 0x7f79ac03d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50694/0 (socket says 172.21.2.111:50694) 2021-04-03T03:08:53.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b1d9b700 1 -- 172.21.2.111:0/957310082 learned_addr learned my addr 172.21.2.111:0/957310082 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:53.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b2d9d700 1 -- 172.21.2.111:0/957310082 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042080 msgr2=0x7f79ac03ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:53.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b159a700 1 -- 172.21.2.111:0/957310082 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110205796 0 0) 0x7f79ac039580 con 0x7f79ac042850 2021-04-03T03:08:53.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b2d9d700 1 --2- 172.21.2.111:0/957310082 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042080 0x7f79ac03ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b2d9d700 1 -- 172.21.2.111:0/957310082 >> v1:172.21.2.111:6789/0 conn(0x7f79ac042850 legacy=0x7f79ac03d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b2d9d700 1 -- 172.21.2.111:0/957310082 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0d19c0 con 0x7f79ac0cdcf0 2021-04-03T03:08:53.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b2d9d700 1 --2- 172.21.2.111:0/957310082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac0cdcf0 0x7f79ac0ce180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f79a4002f30 tx=0x7f79a4006dd0).ready entity=mon.0 client_cookie=50ea853b65f94f4c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b159a700 1 -- 172.21.2.111:0/957310082 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79a40097f0 con 0x7f79ac0cdcf0 2021-04-03T03:08:53.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b159a700 1 -- 172.21.2.111:0/957310082 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4009950 con 0x7f79ac0cdcf0 2021-04-03T03:08:53.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b159a700 1 -- 172.21.2.111:0/957310082 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79a4009cb0 con 0x7f79ac0cdcf0 2021-04-03T03:08:53.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 -- 172.21.2.111:0/957310082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac0cdcf0 msgr2=0x7f79ac0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.975+0000 7f79b359e700 1 --2- 172.21.2.111:0/957310082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac0cdcf0 0x7f79ac0ce180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f79a4002f30 tx=0x7f79a4006dd0).stop 2021-04-03T03:08:53.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- 172.21.2.111:0/957310082 shutdown_connections 2021-04-03T03:08:53.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 --2- 172.21.2.111:0/957310082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac0cdcf0 0x7f79ac0ce180 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:53.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 --2- 172.21.2.111:0/957310082 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042080 0x7f79ac03ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- 172.21.2.111:0/957310082 >> 172.21.2.111:0/957310082 conn(0x7f79ac0462b0 msgr2=0x7f79ac0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:53.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- 172.21.2.111:0/957310082 shutdown_connections 2021-04-03T03:08:53.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- 172.21.2.111:0/957310082 wait complete. 2021-04-03T03:08:53.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 Processor -- start 2021-04-03T03:08:53.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- start start 2021-04-03T03:08:53.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac042080 0x7f79ac03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042850 0x7f79ac0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79ac0cdcf0 0x7f79ac0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0d25d0 con 0x7f79ac042080 2021-04-03T03:08:53.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0d2800 con 0x7f79ac0cdcf0 2021-04-03T03:08:53.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0b3a50 con 0x7f79ac042850 2021-04-03T03:08:53.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac042080 0x7f79ac03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac042080 0x7f79ac03aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49910/0 (socket says 172.21.2.111:49910) 2021-04-03T03:08:53.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 -- 172.21.2.111:0/417810902 learned_addr learned my addr 172.21.2.111:0/417810902 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:53.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 -- 172.21.2.111:0/417810902 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042850 msgr2=0x7f79ac0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042850 0x7f79ac0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 -- 172.21.2.111:0/417810902 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79ac0cdcf0 msgr2=0x7f79ac0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:53.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79ac0cdcf0 0x7f79ac0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:53.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 -- 172.21.2.111:0/417810902 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0cd7b0 con 0x7f79ac042080 2021-04-03T03:08:53.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b259c700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac042080 0x7f79ac03aa30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f79a8002f30 tx=0x7f79a8006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79a37fe700 1 -- 172.21.2.111:0/417810902 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79a800d840 con 0x7f79ac042080 2021-04-03T03:08:53.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79a37fe700 1 -- 172.21.2.111:0/417810902 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a8000e90 con 0x7f79ac042080 2021-04-03T03:08:53.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79a37fe700 1 -- 172.21.2.111:0/417810902 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79a800db90 con 0x7f79ac042080 2021-04-03T03:08:53.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac0cceb0 con 0x7f79ac042080 2021-04-03T03:08:53.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac0af980 con 0x7f79ac042080 2021-04-03T03:08:53.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac004db0 con 0x7f79ac042080 2021-04-03T03:08:53.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79a37fe700 1 -- 172.21.2.111:0/417810902 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f79a8017030 con 0x7f79ac042080 2021-04-03T03:08:53.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79a37fe700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7990056f20 0x7f79900593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:53.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79a37fe700 1 -- 172.21.2.111:0/417810902 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f79a806f640 con 0x7f79ac042080 2021-04-03T03:08:53.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.979+0000 7f79b1d9b700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7990056f20 0x7f79900593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:53.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.983+0000 7f79b1d9b700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7990056f20 0x7f79900593e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f79b4069660 tx=0x7f79b405ad70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:53.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:53.983+0000 7f79a37fe700 1 -- 172.21.2.111:0/417810902 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f79a80449c0 con 0x7f79ac042080 2021-04-03T03:08:54.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79ac03d110 con 0x7f79ac042080 2021-04-03T03:08:54.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79a37fe700 1 -- 172.21.2.111:0/417810902 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f79a8047fe0 con 0x7f79ac042080 2021-04-03T03:08:54.084 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:54.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7990056f20 msgr2=0x7f79900593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7990056f20 0x7f79900593e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f79b4069660 tx=0x7f79b405ad70).stop 2021-04-03T03:08:54.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac042080 msgr2=0x7f79ac03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac042080 0x7f79ac03aa30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f79a8002f30 tx=0x7f79a8006ea0).stop 2021-04-03T03:08:54.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 shutdown_connections 2021-04-03T03:08:54.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7990056f20 0x7f79900593e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79ac0cdcf0 0x7f79ac0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79ac042080 0x7f79ac03aa30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 --2- 172.21.2.111:0/417810902 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79ac042850 0x7f79ac0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 >> 172.21.2.111:0/417810902 conn(0x7f79ac0462b0 msgr2=0x7f79ac0d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:54.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 shutdown_connections 2021-04-03T03:08:54.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.075+0000 7f79b359e700 1 -- 172.21.2.111:0/417810902 wait complete. 2021-04-03T03:08:54.091 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:08:54.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a96b8d700 1 Processor -- start 2021-04-03T03:08:54.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a96b8d700 1 -- start start 2021-04-03T03:08:54.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a96b8d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4a900add50 0x7f4a90147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a96b8d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 0x7f4a90147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a96b8d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a900607d0 con 0x7f4a900603f0 2021-04-03T03:08:54.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a96b8d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9014b090 con 0x7f4a900add50 2021-04-03T03:08:54.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a96b8d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9014b330 con 0x7f4a90144ef0 2021-04-03T03:08:54.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.143+0000 7f4a9538a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4a900add50 0x7f4a90147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:54.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4a900add50 0x7f4a90147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49918/0 (socket says 172.21.2.111:49918) 2021-04-03T03:08:54.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1150141975 learned_addr learned my addr 172.21.2.111:0/1150141975 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:54.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1150141975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 msgr2=0x7f4a90147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:54.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- 172.21.2.111:0/1150141975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 0x7f4a90147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1150141975 >> v1:172.21.2.115:6789/0 conn(0x7f4a900603f0 legacy=0x7f4a901447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:54.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1150141975 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9014b5d0 con 0x7f4a900add50 2021-04-03T03:08:54.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- 172.21.2.111:0/1150141975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90147740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4a900b5600 tx=0x7f4a8c006e90).ready entity=mon.0 client_cookie=25d59d223f287bef server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:54.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a94b89700 1 -- 172.21.2.111:0/1150141975 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a8c0098e0 con 0x7f4a900add50 2021-04-03T03:08:54.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a94b89700 1 -- 172.21.2.111:0/1150141975 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c009a40 con 0x7f4a900add50 2021-04-03T03:08:54.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a94b89700 1 -- 172.21.2.111:0/1150141975 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a8c009de0 con 0x7f4a900add50 2021-04-03T03:08:54.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1150141975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 msgr2=0x7f4a90147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1150141975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90147740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4a900b5600 tx=0x7f4a8c006e90).stop 2021-04-03T03:08:54.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1150141975 shutdown_connections 2021-04-03T03:08:54.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1150141975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 0x7f4a90147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1150141975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90147740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1150141975 >> 172.21.2.111:0/1150141975 conn(0x7f4a900b61c0 msgr2=0x7f4a9005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:54.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1150141975 shutdown_connections 2021-04-03T03:08:54.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1150141975 wait complete. 2021-04-03T03:08:54.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 Processor -- start 2021-04-03T03:08:54.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- start start 2021-04-03T03:08:54.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4a900603f0 0x7f4a900a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 0x7f4a90063cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90157010 con 0x7f4a900add50 2021-04-03T03:08:54.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90157260 con 0x7f4a900603f0 2021-04-03T03:08:54.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90157500 con 0x7f4a90144ef0 2021-04-03T03:08:54.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:54.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49922/0 (socket says 172.21.2.111:49922) 2021-04-03T03:08:54.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1260555133 learned_addr learned my addr 172.21.2.111:0/1260555133 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:54.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1260555133 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 msgr2=0x7f4a90063cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 0x7f4a90063cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1260555133 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4a900603f0 msgr2=0x7f4a900a9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4a900603f0 0x7f4a900a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 -- 172.21.2.111:0/1260555133 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a901577a0 con 0x7f4a900add50 2021-04-03T03:08:54.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a9538a700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90156b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f4a900b5600 tx=0x7f4a8c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:54.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a7effd700 1 -- 172.21.2.111:0/1260555133 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a8c011e30 con 0x7f4a900add50 2021-04-03T03:08:54.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a7effd700 1 -- 172.21.2.111:0/1260555133 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c01e460 con 0x7f4a900add50 2021-04-03T03:08:54.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a900641c0 con 0x7f4a900add50 2021-04-03T03:08:54.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a90064680 con 0x7f4a900add50 2021-04-03T03:08:54.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.147+0000 7f4a7effd700 1 -- 172.21.2.111:0/1260555133 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a8c01e7c0 con 0x7f4a900add50 2021-04-03T03:08:54.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.151+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c0033f0 con 0x7f4a900add50 2021-04-03T03:08:54.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.151+0000 7f4a7effd700 1 -- 172.21.2.111:0/1260555133 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4a8c025030 con 0x7f4a900add50 2021-04-03T03:08:54.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.151+0000 7f4a7effd700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4a70056f70 0x7f4a70059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.151+0000 7f4a7effd700 1 -- 172.21.2.111:0/1260555133 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f4a8c071870 con 0x7f4a900add50 2021-04-03T03:08:54.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.151+0000 7f4a95b8b700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4a70056f70 0x7f4a70059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:54.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.151+0000 7f4a95b8b700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4a70056f70 0x7f4a70059430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4a80002f70 tx=0x7f4a80006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:54.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.151+0000 7f4a7effd700 1 -- 172.21.2.111:0/1260555133 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4a8c046b40 con 0x7f4a900add50 2021-04-03T03:08:54.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.243+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a6c003c20 con 0x7f4a900add50 2021-04-03T03:08:54.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.243+0000 7f4a7effd700 1 -- 172.21.2.111:0/1260555133 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4a8c0230a0 con 0x7f4a900add50 2021-04-03T03:08:54.253 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:54.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4a70056f70 msgr2=0x7f4a70059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4a70056f70 0x7f4a70059430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4a80002f70 tx=0x7f4a80006990).stop 2021-04-03T03:08:54.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 msgr2=0x7f4a90156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90156b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f4a900b5600 tx=0x7f4a8c015000).stop 2021-04-03T03:08:54.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 shutdown_connections 2021-04-03T03:08:54.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4a90144ef0 0x7f4a90063cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4a70056f70 0x7f4a70059430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4a900603f0 0x7f4a900a9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 --2- 172.21.2.111:0/1260555133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4a900add50 0x7f4a90156b20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 >> 172.21.2.111:0/1260555133 conn(0x7f4a900b61c0 msgr2=0x7f4a9005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:54.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 shutdown_connections 2021-04-03T03:08:54.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.247+0000 7f4a96b8d700 1 -- 172.21.2.111:0/1260555133 wait complete. 2021-04-03T03:08:54.260 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:08:54.261 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:08:54.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2198bb700 1 Processor -- start 2021-04-03T03:08:54.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2198bb700 1 -- start start 2021-04-03T03:08:54.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2198bb700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc2140603f0 0x7fc2140b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2198bb700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2140af400 con 0x7fc21414b700 2021-04-03T03:08:54.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2198bb700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2140af6a0 con 0x7fc2140b0c90 2021-04-03T03:08:54.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2198bb700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc214060860 con 0x7fc2140603f0 2021-04-03T03:08:54.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc213fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc21414b700 0x7fc2140b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50724/0 (socket says 172.21.2.111:50724) 2021-04-03T03:08:54.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc213fff700 1 -- 172.21.2.111:0/1519830310 learned_addr learned my addr 172.21.2.111:0/1519830310 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:54.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269819100 0 0) 0x7fc2140af400 con 0x7fc21414b700 2021-04-03T03:08:54.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f4003240 con 0x7fc21414b700 2021-04-03T03:08:54.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 956999067 0 0) 0x7fc1f4003240 con 0x7fc21414b700 2021-04-03T03:08:54.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 >> v2:172.21.2.115:3300/0 conn(0x7fc2140603f0 msgr2=0x7fc2140b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:54.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 --2- 172.21.2.111:0/1519830310 >> v2:172.21.2.115:3300/0 conn(0x7fc2140603f0 0x7fc2140b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 >> v1:172.21.2.115:6789/0 conn(0x7fc2140b0c90 legacy=0x7fc214147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:54.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140afb40 con 0x7fc21414b700 2021-04-03T03:08:54.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fc1fc0016d0 con 0x7fc21414b700 2021-04-03T03:08:54.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 >> v1:172.21.2.111:6789/0 conn(0x7fc21414b700 legacy=0x7fc2140b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4003730 0x7fc1f4003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1f40044e0 0x7fc1f4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1f4004e50 0x7fc1f400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214060860 con 0x7fc1f4004e50 2021-04-03T03:08:54.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f4004240 con 0x7fc1f4003730 2021-04-03T03:08:54.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.307+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f400bc20 con 0x7fc1f40044e0 2021-04-03T03:08:54.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1f4004e50 0x7fc1f400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:54.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1519830310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1f40044e0 msgr2=0x7fc1f4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:54.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1f40044e0 0x7fc1f4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2188b9700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4003730 0x7fc1f4003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:54.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1519830310 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4003730 msgr2=0x7fc1f4003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:54.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4003730 0x7fc1f4003b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1519830310 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f4003240 con 0x7fc1f4004e50 2021-04-03T03:08:54.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1519830310 reap_dead start 2021-04-03T03:08:54.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1f4004e50 0x7fc1f400b730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc20c0012b0 tx=0x7fc20c007ed0).ready entity=mon.0 client_cookie=e6685ae200afa9ce server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:54.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20c00ca70 con 0x7fc1f4004e50 2021-04-03T03:08:54.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00cbd0 con 0x7fc1f4004e50 2021-04-03T03:08:54.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2137fe700 1 -- 172.21.2.111:0/1519830310 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20c013410 con 0x7fc1f4004e50 2021-04-03T03:08:54.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1519830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1f4004e50 msgr2=0x7fc1f400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1f4004e50 0x7fc1f400b730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc20c0012b0 tx=0x7fc20c007ed0).stop 2021-04-03T03:08:54.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1519830310 shutdown_connections 2021-04-03T03:08:54.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1519830310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1f4004e50 0x7fc1f400b730 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1519830310 >> 172.21.2.111:0/1519830310 conn(0x7fc2140b61c0 msgr2=0x7fc21405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:54.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1519830310 shutdown_connections 2021-04-03T03:08:54.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1519830310 wait complete. 2021-04-03T03:08:54.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 Processor -- start 2021-04-03T03:08:54.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- start start 2021-04-03T03:08:54.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4004e50 0x7fc2140aa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc214064200 0x7fc214067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2140628d0 0x7fc214067950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140a99c0 con 0x7fc2140628d0 2021-04-03T03:08:54.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140646a0 con 0x7fc1f4004e50 2021-04-03T03:08:54.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc214064940 con 0x7fc214064200 2021-04-03T03:08:54.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2140628d0 0x7fc214067950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:54.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2140628d0 0x7fc214067950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49942/0 (socket says 172.21.2.111:49942) 2021-04-03T03:08:54.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1054098198 learned_addr learned my addr 172.21.2.111:0/1054098198 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:54.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1054098198 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc214064200 msgr2=0x7fc214067460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc214064200 0x7fc214067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1054098198 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4004e50 msgr2=0x7fc2140aa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4004e50 0x7fc2140aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 -- 172.21.2.111:0/1054098198 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140abdb0 con 0x7fc2140628d0 2021-04-03T03:08:54.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2190ba700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2140628d0 0x7fc214067950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fc20c0012b0 tx=0x7fc20c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:54.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc211ffb700 1 -- 172.21.2.111:0/1054098198 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20c0213b0 con 0x7fc2140628d0 2021-04-03T03:08:54.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc211ffb700 1 -- 172.21.2.111:0/1054098198 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c021510 con 0x7fc2140628d0 2021-04-03T03:08:54.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc214067e40 con 0x7fc2140628d0 2021-04-03T03:08:54.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc211ffb700 1 -- 172.21.2.111:0/1054098198 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20c021870 con 0x7fc2140628d0 2021-04-03T03:08:54.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc214159940 con 0x7fc2140628d0 2021-04-03T03:08:54.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc214057660 con 0x7fc2140628d0 2021-04-03T03:08:54.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc211ffb700 1 -- 172.21.2.111:0/1054098198 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc20c029070 con 0x7fc2140628d0 2021-04-03T03:08:54.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc211ffb700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1f8056ea0 0x7fc1f8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:54.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc211ffb700 1 -- 172.21.2.111:0/1054098198 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fc20c070f90 con 0x7fc2140628d0 2021-04-03T03:08:54.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2188b9700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1f8056ea0 0x7fc1f8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:54.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.311+0000 7fc2188b9700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1f8056ea0 0x7fc1f8059360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc204007000 tx=0x7fc204008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:54.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.315+0000 7fc211ffb700 1 -- 172.21.2.111:0/1054098198 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc20c0462f0 con 0x7fc2140628d0 2021-04-03T03:08:54.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2140686d0 con 0x7fc2140628d0 2021-04-03T03:08:54.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc211ffb700 1 -- 172.21.2.111:0/1054098198 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fc20c049910 con 0x7fc2140628d0 2021-04-03T03:08:54.447 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:54.448 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1066299897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1066299897},{"type":"v1","addr":"172.21.2.115:6833","nonce":1066299897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:08:44.422766+0000","modified":"2021-04-03T03:08:45.425728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2149096292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2149096292},{"type":"v1","addr":"172.21.2.111:6835","nonce":2149096292}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-04-03T03:08:54.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1f8056ea0 msgr2=0x7fc1f8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1f8056ea0 0x7fc1f8059360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc204007000 tx=0x7fc204008040).stop 2021-04-03T03:08:54.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2140628d0 msgr2=0x7fc214067950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:54.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2140628d0 0x7fc214067950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fc20c0012b0 tx=0x7fc20c016000).stop 2021-04-03T03:08:54.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 shutdown_connections 2021-04-03T03:08:54.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc2140628d0 0x7fc214067950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1f8056ea0 0x7fc1f8059360 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:54.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1f4004e50 0x7fc2140aa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 --2- 172.21.2.111:0/1054098198 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc214064200 0x7fc214067460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:54.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 >> 172.21.2.111:0/1054098198 conn(0x7fc2140b61c0 msgr2=0x7fc214147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:54.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 shutdown_connections 2021-04-03T03:08:54.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:54.439+0000 7fc2198bb700 1 -- 172.21.2.111:0/1054098198 wait complete. 2021-04-03T03:08:54.452 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 5 2021-04-03T03:08:54.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:08:44.422766+0000', 'modified': '2021-04-03T03:08:45.425728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/2149096292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 2149096292}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 2149096292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:08:54.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:08:54.657 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:08:54.728 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":4239,"want_state":"up:active","state":"up:active","rank_uptime":10.295219230000001,"mdsmap_epoch":5,"osdmap_epoch":24,"osdmap_epoch_barrier":23,"uptime":15.513407276000001} 2021-04-03T03:08:54.729 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 4239, "mdsmap_epoch": 5, "osdmap_epoch": 24, "osdmap_epoch_barrier": 23, "rank_uptime": 10.29521923, "state": "up:active", "uptime": 15.513407276, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:08:54.729 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:08:54.730 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:08:54.730 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:08:54.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:54.731 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:08:54.741 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:08:54.792 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:08:54.792 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:08:54.840 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:08:54.841 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:08:54.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:54.842 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:08:54.893 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:08:54.894 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:08:54.939 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:08:54.947 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:08:54.947 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:08:54.987 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:08:54.979+0000 7fa55e1700c0 -1 init, newargv = 0x564296549b50 newargc=9 2021-04-03T03:08:54.987 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[14866]: starting ceph client 2021-04-03T03:08:54.992 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:08:54.993 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:08:54.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:54.994 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:08:54.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[14866]: starting fuse 2021-04-03T03:08:55.000 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:08:55.001 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:08:55.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:55.002 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:08:55.002 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:08:55.002 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:08:55.003 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:08:55.003 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:08:55.003 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:08:55.004 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:08:55.004 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:08:55.004 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:08:55.004 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:08:55.005 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:08:55.005 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:08:55.005 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:08:55.006 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:08:55.006 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:08:55.006 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:08:55.007 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:08:55.007 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:08:55.007 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:08:55.008 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:08:55.008 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:08:55.008 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:08:55.009 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:08:55.065 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.14866.asok 2021-04-03T03:08:55.068 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14866.asok 2021-04-03T03:08:55.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:55.068 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14866.asok status 2021-04-03T03:08:55.484 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:08:55.484 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:08:55.485 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:08:55.485 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:08:55.485 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:08:55.485 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:08:55.485 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:08:55.486 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "14866", 2021-04-03T03:08:55.486 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:08:55.486 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:08:55.486 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:08:55.487 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:08:55.487 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4387, 2021-04-03T03:08:55.487 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:08:55.487 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:08:55.487 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:08:55.488 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4387 2021-04-03T03:08:55.488 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:08:55.488 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:08:55.488 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:08:55.488 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:08:55.489 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 549019164 2021-04-03T03:08:55.489 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:08:55.489 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:08:55.489 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:08:55.489 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:08:55.490 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:08:55.490 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 549019164 2021-04-03T03:08:55.490 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:08:55.490 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4387 172.21.2.126:0/549019164", 2021-04-03T03:08:55.490 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/549019164", 2021-04-03T03:08:55.491 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:08:55.491 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 5, 2021-04-03T03:08:55.491 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 24, 2021-04-03T03:08:55.491 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:08:55.492 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:08:55.492 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:08:55.492 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:08:55.497 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:55.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414ea93700 1 Processor -- start 2021-04-03T03:08:55.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414ea93700 1 -- start start 2021-04-03T03:08:55.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414ea93700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4140048d70 0x7f41400431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414ea93700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f41400436d0 0x7f4140043b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414ea93700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4140008df0 con 0x7f4140044020 2021-04-03T03:08:55.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414ea93700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4140009020 con 0x7f4140048d70 2021-04-03T03:08:55.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414ea93700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400092c0 con 0x7f41400436d0 2021-04-03T03:08:55.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414da91700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4140048d70 0x7f41400431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414d290700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f41400436d0 0x7f4140043b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414e292700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f4140044020 0x7f41400059a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:35048/0 (socket says 172.21.2.111:35048) 2021-04-03T03:08:55.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414e292700 1 -- 172.21.2.111:0/1720773487 learned_addr learned my addr 172.21.2.111:0/1720773487 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:55.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414d290700 1 -- 172.21.2.111:0/1720773487 >> v2:172.21.2.115:3300/0 conn(0x7f4140048d70 msgr2=0x7f41400431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:55.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414d290700 1 --2- 172.21.2.111:0/1720773487 >> v2:172.21.2.115:3300/0 conn(0x7f4140048d70 0x7f41400431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414d290700 1 -- 172.21.2.111:0/1720773487 >> v1:172.21.2.115:6789/0 conn(0x7f4140044020 legacy=0x7f41400059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.551+0000 7f414d290700 1 -- 172.21.2.111:0/1720773487 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414003f610 con 0x7f41400436d0 2021-04-03T03:08:55.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414d290700 1 --2- 172.21.2.111:0/1720773487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f41400436d0 0x7f4140043b30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f414003f3a0 tx=0x7f4138006dd0).ready entity=mon.2 client_cookie=81578f6d31b68cb9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ca8f700 1 -- 172.21.2.111:0/1720773487 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41380097b0 con 0x7f41400436d0 2021-04-03T03:08:55.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ca8f700 1 -- 172.21.2.111:0/1720773487 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4138009910 con 0x7f41400436d0 2021-04-03T03:08:55.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ca8f700 1 -- 172.21.2.111:0/1720773487 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4138009c70 con 0x7f41400436d0 2021-04-03T03:08:55.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- 172.21.2.111:0/1720773487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f41400436d0 msgr2=0x7f4140043b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 --2- 172.21.2.111:0/1720773487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f41400436d0 0x7f4140043b30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f414003f3a0 tx=0x7f4138006dd0).stop 2021-04-03T03:08:55.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- 172.21.2.111:0/1720773487 shutdown_connections 2021-04-03T03:08:55.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 --2- 172.21.2.111:0/1720773487 >> v2:172.21.2.115:3300/0 conn(0x7f4140048d70 0x7f41400431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 --2- 172.21.2.111:0/1720773487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f41400436d0 0x7f4140043b30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- 172.21.2.111:0/1720773487 >> 172.21.2.111:0/1720773487 conn(0x7f414000f2c0 msgr2=0x7f41400484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:55.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- 172.21.2.111:0/1720773487 shutdown_connections 2021-04-03T03:08:55.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- 172.21.2.111:0/1720773487 wait complete. 2021-04-03T03:08:55.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 Processor -- start 2021-04-03T03:08:55.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- start start 2021-04-03T03:08:55.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41400436d0 0x7f41400a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4140044020 0x7f41400a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4140048d70 0x7f41400a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400e7c10 con 0x7f41400436d0 2021-04-03T03:08:55.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400a27b0 con 0x7f4140048d70 2021-04-03T03:08:55.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400e7e70 con 0x7f4140044020 2021-04-03T03:08:55.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41400436d0 0x7f41400a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41400436d0 0x7f41400a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49954/0 (socket says 172.21.2.111:49954) 2021-04-03T03:08:55.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 -- 172.21.2.111:0/529218639 learned_addr learned my addr 172.21.2.111:0/529218639 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:55.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 -- 172.21.2.111:0/529218639 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4140044020 msgr2=0x7f41400a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4140044020 0x7f41400a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 -- 172.21.2.111:0/529218639 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4140048d70 msgr2=0x7f41400a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4140048d70 0x7f41400a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 -- 172.21.2.111:0/529218639 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41400a3500 con 0x7f41400436d0 2021-04-03T03:08:55.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414da91700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41400436d0 0x7f41400a22c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f414400c000 tx=0x7f4144008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f4136ffd700 1 -- 172.21.2.111:0/529218639 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414400c820 con 0x7f41400436d0 2021-04-03T03:08:55.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f4136ffd700 1 -- 172.21.2.111:0/529218639 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414400c980 con 0x7f41400436d0 2021-04-03T03:08:55.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41400a3180 con 0x7f41400436d0 2021-04-03T03:08:55.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f4136ffd700 1 -- 172.21.2.111:0/529218639 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414400cce0 con 0x7f41400436d0 2021-04-03T03:08:55.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.555+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414003cc10 con 0x7f41400436d0 2021-04-03T03:08:55.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.559+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4140004da0 con 0x7f41400436d0 2021-04-03T03:08:55.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.559+0000 7f4136ffd700 1 -- 172.21.2.111:0/529218639 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4144022070 con 0x7f41400436d0 2021-04-03T03:08:55.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.559+0000 7f4136ffd700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4128057050 0x7f4128059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.559+0000 7f4136ffd700 1 -- 172.21.2.111:0/529218639 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f4144070650 con 0x7f41400436d0 2021-04-03T03:08:55.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.559+0000 7f414d290700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4128057050 0x7f4128059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.559+0000 7f414d290700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4128057050 0x7f4128059510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f414003f3a0 tx=0x7f4138015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.559+0000 7f4136ffd700 1 -- 172.21.2.111:0/529218639 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4144045920 con 0x7f41400436d0 2021-04-03T03:08:55.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.651+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f414000a5e0 con 0x7f41400436d0 2021-04-03T03:08:55.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f4136ffd700 1 -- 172.21.2.111:0/529218639 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f4144048f40 con 0x7f41400436d0 2021-04-03T03:08:55.661 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:55.662 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:55.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4128057050 msgr2=0x7f4128059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4128057050 0x7f4128059510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f414003f3a0 tx=0x7f4138015000).stop 2021-04-03T03:08:55.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41400436d0 msgr2=0x7f41400a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41400436d0 0x7f41400a22c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f414400c000 tx=0x7f4144008040).stop 2021-04-03T03:08:55.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 shutdown_connections 2021-04-03T03:08:55.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4128057050 0x7f4128059510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4140048d70 0x7f41400a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41400436d0 0x7f41400a22c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 --2- 172.21.2.111:0/529218639 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4140044020 0x7f41400a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 >> 172.21.2.111:0/529218639 conn(0x7f414000f2c0 msgr2=0x7f41400d6940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:55.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 shutdown_connections 2021-04-03T03:08:55.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.655+0000 7f414ea93700 1 -- 172.21.2.111:0/529218639 wait complete. 2021-04-03T03:08:55.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:08:55.670 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:08:55.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181f59e700 1 Processor -- start 2021-04-03T03:08:55.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181f59e700 1 -- start start 2021-04-03T03:08:55.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1818041060 0x7f18180dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180dc560 con 0x7f181803d560 2021-04-03T03:08:55.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180dc800 con 0x7f18180dbba0 2021-04-03T03:08:55.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181803de60 con 0x7f1818041060 2021-04-03T03:08:55.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181ed9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f181803d560 0x7f1818041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50758/0 (socket says 172.21.2.111:50758) 2021-04-03T03:08:55.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181ed9d700 1 -- 172.21.2.111:0/1600678335 learned_addr learned my addr 172.21.2.111:0/1600678335 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:55.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181dd9b700 1 --2- 172.21.2.111:0/1600678335 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1818041060 0x7f18180dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181d59a700 1 -- 172.21.2.111:0/1600678335 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115073475 0 0) 0x7f18180dc560 con 0x7f181803d560 2021-04-03T03:08:55.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181d59a700 1 -- 172.21.2.111:0/1600678335 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1800003240 con 0x7f181803d560 2021-04-03T03:08:55.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181dd9b700 1 -- 172.21.2.111:0/1600678335 >> v1:172.21.2.115:6789/0 conn(0x7f18180dbba0 legacy=0x7f181803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181d59a700 1 -- 172.21.2.111:0/1600678335 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079657839 0 0) 0x7f18180dc800 con 0x7f18180dbba0 2021-04-03T03:08:55.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181dd9b700 1 -- 172.21.2.111:0/1600678335 >> v1:172.21.2.111:6789/0 conn(0x7f181803d560 legacy=0x7f1818041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181dd9b700 1 -- 172.21.2.111:0/1600678335 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181803da30 con 0x7f1818041060 2021-04-03T03:08:55.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.719+0000 7f181dd9b700 1 --2- 172.21.2.111:0/1600678335 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1818041060 0x7f18180dbf80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1810002f30 tx=0x7f1810006dd0).ready entity=mon.2 client_cookie=6f7455753f5d4d63 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181d59a700 1 -- 172.21.2.111:0/1600678335 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18100097f0 con 0x7f1818041060 2021-04-03T03:08:55.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181d59a700 1 -- 172.21.2.111:0/1600678335 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1810009950 con 0x7f1818041060 2021-04-03T03:08:55.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181d59a700 1 -- 172.21.2.111:0/1600678335 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1810009cb0 con 0x7f1818041060 2021-04-03T03:08:55.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1600678335 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1818041060 msgr2=0x7f18180dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 --2- 172.21.2.111:0/1600678335 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1818041060 0x7f18180dbf80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1810002f30 tx=0x7f1810006dd0).stop 2021-04-03T03:08:55.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1600678335 shutdown_connections 2021-04-03T03:08:55.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 --2- 172.21.2.111:0/1600678335 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1818041060 0x7f18180dbf80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1600678335 >> 172.21.2.111:0/1600678335 conn(0x7f18180462b0 msgr2=0x7f18180466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:55.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1600678335 shutdown_connections 2021-04-03T03:08:55.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1600678335 wait complete. 2021-04-03T03:08:55.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 Processor -- start 2021-04-03T03:08:55.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- start start 2021-04-03T03:08:55.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f181803d560 0x7f18180d7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1818041060 0x7f18180ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18180dbba0 0x7f18180eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180dcb80 con 0x7f1818041060 2021-04-03T03:08:55.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180dce20 con 0x7f181803d560 2021-04-03T03:08:55.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181803b8a0 con 0x7f18180dbba0 2021-04-03T03:08:55.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1818041060 0x7f18180ea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181ed9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18180dbba0 0x7f18180eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1818041060 0x7f18180ea5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49972/0 (socket says 172.21.2.111:49972) 2021-04-03T03:08:55.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 -- 172.21.2.111:0/1630041197 learned_addr learned my addr 172.21.2.111:0/1630041197 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:55.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181e59c700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f181803d560 0x7f18180d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 -- 172.21.2.111:0/1630041197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18180dbba0 msgr2=0x7f18180eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18180dbba0 0x7f18180eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 -- 172.21.2.111:0/1630041197 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f181803d560 msgr2=0x7f18180d7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f181803d560 0x7f18180d7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 -- 172.21.2.111:0/1630041197 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18180a0690 con 0x7f1818041060 2021-04-03T03:08:55.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181dd9b700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1818041060 0x7f18180ea5f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f1810002f30 tx=0x7f1810015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f18177fe700 1 -- 172.21.2.111:0/1630041197 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1810011c30 con 0x7f1818041060 2021-04-03T03:08:55.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f18177fe700 1 -- 172.21.2.111:0/1630041197 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1810011d90 con 0x7f1818041060 2021-04-03T03:08:55.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18180a0880 con 0x7f1818041060 2021-04-03T03:08:55.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f18177fe700 1 -- 172.21.2.111:0/1630041197 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f181001e650 con 0x7f1818041060 2021-04-03T03:08:55.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18180d6850 con 0x7f1818041060 2021-04-03T03:08:55.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18180dd1c0 con 0x7f1818041060 2021-04-03T03:08:55.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f18177fe700 1 -- 172.21.2.111:0/1630041197 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1810025030 con 0x7f1818041060 2021-04-03T03:08:55.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f18177fe700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1804057050 0x7f1804059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f18177fe700 1 -- 172.21.2.111:0/1630041197 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f1810071440 con 0x7f1818041060 2021-04-03T03:08:55.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.723+0000 7f181e59c700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1804057050 0x7f1804059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.727+0000 7f181e59c700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1804057050 0x7f1804059510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1808003040 tx=0x7f1808007b50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.727+0000 7f18177fe700 1 -- 172.21.2.111:0/1630041197 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f18180dd1c0 con 0x7f1818041060 2021-04-03T03:08:55.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.819+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f18180dd1c0 con 0x7f1818041060 2021-04-03T03:08:55.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.819+0000 7f18177fe700 1 -- 172.21.2.111:0/1630041197 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f18180dd1c0 con 0x7f1818041060 2021-04-03T03:08:55.828 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:08:55.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.819+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1804057050 msgr2=0x7f1804059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.819+0000 7f181f59e700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1804057050 0x7f1804059510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1808003040 tx=0x7f1808007b50).stop 2021-04-03T03:08:55.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.819+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1818041060 msgr2=0x7f18180ea5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.819+0000 7f181f59e700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1818041060 0x7f18180ea5f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f1810002f30 tx=0x7f1810015000).stop 2021-04-03T03:08:55.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 shutdown_connections 2021-04-03T03:08:55.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18180dbba0 0x7f18180eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1804057050 0x7f1804059510 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f181803d560 0x7f18180d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 --2- 172.21.2.111:0/1630041197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1818041060 0x7f18180ea5f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 >> 172.21.2.111:0/1630041197 conn(0x7f18180462b0 msgr2=0x7f1818041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:55.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 shutdown_connections 2021-04-03T03:08:55.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.823+0000 7f181f59e700 1 -- 172.21.2.111:0/1630041197 wait complete. 2021-04-03T03:08:55.836 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:08:55.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 Processor -- start 2021-04-03T03:08:55.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- start start 2021-04-03T03:08:55.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6050063c90 0x7f6050064070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 0x7f6050065490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60500b3ac0 con 0x7f6050144890 2021-04-03T03:08:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60500b11a0 con 0x7f6050063c90 2021-04-03T03:08:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6050065980 con 0x7f6050144d10 2021-04-03T03:08:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6050144890 0x7f6050064d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50770/0 (socket says 172.21.2.111:50770) 2021-04-03T03:08:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 -- 172.21.2.111:0/870066214 learned_addr learned my addr 172.21.2.111:0/870066214 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563500911 0 0) 0x7f60500b3ac0 con 0x7f6050144890 2021-04-03T03:08:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6030003240 con 0x7f6050144890 2021-04-03T03:08:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f605551a700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 0x7f6050065490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6054d19700 1 --2- 172.21.2.111:0/870066214 >> v2:172.21.2.115:3300/0 conn(0x7f6050063c90 0x7f6050064070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1561329314 0 0) 0x7f6030003240 con 0x7f6050144890 2021-04-03T03:08:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 msgr2=0x7f6050065490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 0x7f6050065490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 >> v2:172.21.2.115:3300/0 conn(0x7f6050063c90 msgr2=0x7f6050064070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 --2- 172.21.2.111:0/870066214 >> v2:172.21.2.115:3300/0 conn(0x7f6050063c90 0x7f6050064070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60500b3cf0 con 0x7f6050144890 2021-04-03T03:08:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f60380016d0 con 0x7f6050144890 2021-04-03T03:08:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 >> v1:172.21.2.111:6789/0 conn(0x7f6050144890 legacy=0x7f6050064d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 0x7f60300037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6030003cc0 0x7f60300044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60300049d0 0x7f6030004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60500b11a0 con 0x7f6030003cc0 2021-04-03T03:08:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6050065980 con 0x7f60300049d0 2021-04-03T03:08:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6030004280 con 0x7f6050144d10 2021-04-03T03:08:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6030003cc0 0x7f60300044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6054d19700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 0x7f60300037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f605551a700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60300049d0 0x7f6030004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 -- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 msgr2=0x7f60300037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050144d10 0x7f60300037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 -- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60300049d0 msgr2=0x7f6030004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60300049d0 0x7f6030004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 -- 172.21.2.111:0/870066214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60500b3cf0 con 0x7f6030003cc0 2021-04-03T03:08:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604ffff700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6030003cc0 0x7f60300044e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f60380013e0 tx=0x7f60380077d0).ready entity=mon.0 client_cookie=981403b833b2e45f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60380070d0 con 0x7f6030003cc0 2021-04-03T03:08:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6038007230 con 0x7f6030003cc0 2021-04-03T03:08:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f604f7fe700 1 -- 172.21.2.111:0/870066214 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60380055f0 con 0x7f6030003cc0 2021-04-03T03:08:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- 172.21.2.111:0/870066214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6030003cc0 msgr2=0x7f60300044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 --2- 172.21.2.111:0/870066214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6030003cc0 0x7f60300044e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f60380013e0 tx=0x7f60380077d0).stop 2021-04-03T03:08:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f605551a700 1 -- 172.21.2.111:0/870066214 reap_dead start 2021-04-03T03:08:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- 172.21.2.111:0/870066214 shutdown_connections 2021-04-03T03:08:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- 172.21.2.111:0/870066214 >> 172.21.2.111:0/870066214 conn(0x7f60500622a0 msgr2=0x7f6050060ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- 172.21.2.111:0/870066214 shutdown_connections 2021-04-03T03:08:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.887+0000 7f6055d1b700 1 -- 172.21.2.111:0/870066214 wait complete. 2021-04-03T03:08:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 Processor -- start 2021-04-03T03:08:55.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 -- start start 2021-04-03T03:08:55.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050066a60 0x7f6050146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6050111420 0x7f605015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60501562c0 0x7f6050156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6050066e40 con 0x7f60501562c0 2021-04-03T03:08:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6050159950 con 0x7f6050111420 2021-04-03T03:08:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6050159bf0 con 0x7f6050066a60 2021-04-03T03:08:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60501562c0 0x7f6050156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60501562c0 0x7f6050156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49992/0 (socket says 172.21.2.111:49992) 2021-04-03T03:08:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 -- 172.21.2.111:0/2967840056 learned_addr learned my addr 172.21.2.111:0/2967840056 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 -- 172.21.2.111:0/2967840056 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050066a60 msgr2=0x7f6050146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050066a60 0x7f6050146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 -- 172.21.2.111:0/2967840056 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6050111420 msgr2=0x7f605015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6050111420 0x7f605015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 -- 172.21.2.111:0/2967840056 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60501487a0 con 0x7f60501562c0 2021-04-03T03:08:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f605551a700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60501562c0 0x7f6050156770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f604800c000 tx=0x7f6048008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f604dffb700 1 -- 172.21.2.111:0/2967840056 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6048017070 con 0x7f60501562c0 2021-04-03T03:08:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f604dffb700 1 -- 172.21.2.111:0/2967840056 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048011790 con 0x7f60501562c0 2021-04-03T03:08:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605015a580 con 0x7f60501562c0 2021-04-03T03:08:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f604dffb700 1 -- 172.21.2.111:0/2967840056 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6048011af0 con 0x7f60501562c0 2021-04-03T03:08:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6050143d70 con 0x7f60501562c0 2021-04-03T03:08:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6050159db0 con 0x7f60501562c0 2021-04-03T03:08:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f604dffb700 1 -- 172.21.2.111:0/2967840056 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6048011c50 con 0x7f60501562c0 2021-04-03T03:08:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f604dffb700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6034056fb0 0x7f6034059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f604dffb700 1 -- 172.21.2.111:0/2967840056 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f6048070450 con 0x7f60501562c0 2021-04-03T03:08:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6054d19700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6034056fb0 0x7f6034059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:55.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.891+0000 7f6054d19700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6034056fb0 0x7f6034059470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f604000c000 tx=0x7f6040008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:55.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.895+0000 7f604dffb700 1 -- 172.21.2.111:0/2967840056 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6048045830 con 0x7f60501562c0 2021-04-03T03:08:55.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6050057660 con 0x7f60501562c0 2021-04-03T03:08:55.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f604dffb700 1 -- 172.21.2.111:0/2967840056 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6048048e50 con 0x7f60501562c0 2021-04-03T03:08:55.996 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:55.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6034056fb0 msgr2=0x7f6034059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6034056fb0 0x7f6034059470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f604000c000 tx=0x7f6040008040).stop 2021-04-03T03:08:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60501562c0 msgr2=0x7f6050156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60501562c0 0x7f6050156770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f604800c000 tx=0x7f6048008040).stop 2021-04-03T03:08:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 shutdown_connections 2021-04-03T03:08:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60501562c0 0x7f6050156770 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6050066a60 0x7f6050146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6034056fb0 0x7f6034059470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 --2- 172.21.2.111:0/2967840056 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6050111420 0x7f605015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 >> 172.21.2.111:0/2967840056 conn(0x7f60500622a0 msgr2=0x7f6050065af0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:55.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 shutdown_connections 2021-04-03T03:08:55.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:55.987+0000 7f6055d1b700 1 -- 172.21.2.111:0/2967840056 wait complete. 2021-04-03T03:08:56.003 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:08:56.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d8b9700 1 Processor -- start 2021-04-03T03:08:56.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d8b9700 1 -- start start 2021-04-03T03:08:56.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d8b9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880603f0 0x7f0d881447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d8b9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0d880b17b0 0x7f0d88144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d8b9700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d880607d0 con 0x7f0d880add50 2021-04-03T03:08:56.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d8b9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d881451c0 con 0x7f0d880b17b0 2021-04-03T03:08:56.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d8b9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8814b090 con 0x7f0d880603f0 2021-04-03T03:08:56.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d0b8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0d880b17b0 0x7f0d88144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d0b8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0d880b17b0 0x7f0d88144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:49996/0 (socket says 172.21.2.111:49996) 2021-04-03T03:08:56.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d0b8700 1 -- 172.21.2.111:0/1940897348 learned_addr learned my addr 172.21.2.111:0/1940897348 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:56.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d0b8700 1 -- 172.21.2.111:0/1940897348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880603f0 msgr2=0x7f0d881447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:56.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d0b8700 1 --2- 172.21.2.111:0/1940897348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880603f0 0x7f0d881447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d0b8700 1 -- 172.21.2.111:0/1940897348 >> v1:172.21.2.115:6789/0 conn(0x7f0d880add50 legacy=0x7f0d88147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:56.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.051+0000 7f0d8d0b8700 1 -- 172.21.2.111:0/1940897348 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8814b330 con 0x7f0d880b17b0 2021-04-03T03:08:56.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d0b8700 1 --2- 172.21.2.111:0/1940897348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880b17b0 0x7f0d88144cd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f0d880b5600 tx=0x7f0d80009270).ready entity=mon.0 client_cookie=e1b9983bb8902d5f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d877fe700 1 -- 172.21.2.111:0/1940897348 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d8000bbc0 con 0x7f0d880b17b0 2021-04-03T03:08:56.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d877fe700 1 -- 172.21.2.111:0/1940897348 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8000bd20 con 0x7f0d880b17b0 2021-04-03T03:08:56.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d877fe700 1 -- 172.21.2.111:0/1940897348 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d80011400 con 0x7f0d880b17b0 2021-04-03T03:08:56.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/1940897348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880b17b0 msgr2=0x7f0d88144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/1940897348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880b17b0 0x7f0d88144cd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f0d880b5600 tx=0x7f0d80009270).stop 2021-04-03T03:08:56.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/1940897348 shutdown_connections 2021-04-03T03:08:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/1940897348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880b17b0 0x7f0d88144cd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/1940897348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880603f0 0x7f0d881447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/1940897348 >> 172.21.2.111:0/1940897348 conn(0x7f0d880b61c0 msgr2=0x7f0d8805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/1940897348 shutdown_connections 2021-04-03T03:08:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/1940897348 wait complete. 2021-04-03T03:08:56.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 Processor -- start 2021-04-03T03:08:56.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- start start 2021-04-03T03:08:56.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d880603f0 0x7f0d880aa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880add50 0x7f0d880aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880b17b0 0x7f0d88062d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88065890 con 0x7f0d880add50 2021-04-03T03:08:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880655f0 con 0x7f0d880603f0 2021-04-03T03:08:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88065310 con 0x7f0d880b17b0 2021-04-03T03:08:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880add50 0x7f0d880aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880add50 0x7f0d880aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50004/0 (socket says 172.21.2.111:50004) 2021-04-03T03:08:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 -- 172.21.2.111:0/277092284 learned_addr learned my addr 172.21.2.111:0/277092284 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 -- 172.21.2.111:0/277092284 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880b17b0 msgr2=0x7f0d88062d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880b17b0 0x7f0d88062d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 -- 172.21.2.111:0/277092284 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d880603f0 msgr2=0x7f0d880aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d880603f0 0x7f0d880aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 -- 172.21.2.111:0/277092284 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d880aba30 con 0x7f0d880add50 2021-04-03T03:08:56.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d87fff700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880add50 0x7f0d880aad00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f0d78002f70 tx=0x7f0d78006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d85ffb700 1 -- 172.21.2.111:0/277092284 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d7800d570 con 0x7f0d880add50 2021-04-03T03:08:56.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d85ffb700 1 -- 172.21.2.111:0/277092284 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800d6d0 con 0x7f0d880add50 2021-04-03T03:08:56.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d880ab610 con 0x7f0d880add50 2021-04-03T03:08:56.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d85ffb700 1 -- 172.21.2.111:0/277092284 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d7800da30 con 0x7f0d880add50 2021-04-03T03:08:56.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88063400 con 0x7f0d880add50 2021-04-03T03:08:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d880abd10 con 0x7f0d880add50 2021-04-03T03:08:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d85ffb700 1 -- 172.21.2.111:0/277092284 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0d7801b030 con 0x7f0d880add50 2021-04-03T03:08:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d85ffb700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d6c056ec0 0x7f0d6c059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d85ffb700 1 -- 172.21.2.111:0/277092284 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0d7806ff00 con 0x7f0d880add50 2021-04-03T03:08:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.055+0000 7f0d8c8b7700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d6c056ec0 0x7f0d6c059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.059+0000 7f0d8c8b7700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d6c056ec0 0x7f0d6c059380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0d70002f70 tx=0x7f0d70006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.059+0000 7f0d85ffb700 1 -- 172.21.2.111:0/277092284 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0d780452e0 con 0x7f0d880add50 2021-04-03T03:08:56.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d88057660 con 0x7f0d880add50 2021-04-03T03:08:56.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d85ffb700 1 -- 172.21.2.111:0/277092284 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0d78048900 con 0x7f0d880add50 2021-04-03T03:08:56.160 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:56.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d6c056ec0 msgr2=0x7f0d6c059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d6c056ec0 0x7f0d6c059380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0d70002f70 tx=0x7f0d70006990).stop 2021-04-03T03:08:56.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880add50 msgr2=0x7f0d880aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880add50 0x7f0d880aad00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f0d78002f70 tx=0x7f0d78006de0).stop 2021-04-03T03:08:56.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 shutdown_connections 2021-04-03T03:08:56.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d880b17b0 0x7f0d88062d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d6c056ec0 0x7f0d6c059380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d880603f0 0x7f0d880aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.151+0000 7f0d8d8b9700 1 --2- 172.21.2.111:0/277092284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d880add50 0x7f0d880aad00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.155+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 >> 172.21.2.111:0/277092284 conn(0x7f0d880b61c0 msgr2=0x7f0d8805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:56.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.155+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 shutdown_connections 2021-04-03T03:08:56.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.155+0000 7f0d8d8b9700 1 -- 172.21.2.111:0/277092284 wait complete. 2021-04-03T03:08:56.168 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:08:56.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.219+0000 7f1e1bfff700 1 Processor -- start 2021-04-03T03:08:56.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.219+0000 7f1e1bfff700 1 -- start start 2021-04-03T03:08:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1e1c0603f0 0x7f1e1c067b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1e1c0b18b0 0x7f1e1c0acb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c140b60 con 0x7f1e1c0ad020 2021-04-03T03:08:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c0af3f0 con 0x7f1e1c0b18b0 2021-04-03T03:08:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c062bf0 con 0x7f1e1c0603f0 2021-04-03T03:08:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1e1c0b18b0 0x7f1e1c0acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1e1c0b18b0 0x7f1e1c0acb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50012/0 (socket says 172.21.2.111:50012) 2021-04-03T03:08:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/774610655 learned_addr learned my addr 172.21.2.111:0/774610655 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/774610655 >> v2:172.21.2.115:3300/0 conn(0x7f1e1c0603f0 msgr2=0x7f1e1c067b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:08:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- 172.21.2.111:0/774610655 >> v2:172.21.2.115:3300/0 conn(0x7f1e1c0603f0 0x7f1e1c067b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/774610655 >> v1:172.21.2.111:6789/0 conn(0x7f1e1c0ad020 legacy=0x7f1e1c13d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/774610655 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c068000 con 0x7f1e1c0b18b0 2021-04-03T03:08:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- 172.21.2.111:0/774610655 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0b18b0 0x7f1e1c0acb30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f1e04002f30 tx=0x7f1e04006e90).ready entity=mon.0 client_cookie=b17d4f366dc48c81 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e19ffb700 1 -- 172.21.2.111:0/774610655 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e040099c0 con 0x7f1e1c0b18b0 2021-04-03T03:08:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e19ffb700 1 -- 172.21.2.111:0/774610655 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e04009b20 con 0x7f1e1c0b18b0 2021-04-03T03:08:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e19ffb700 1 -- 172.21.2.111:0/774610655 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e04009e80 con 0x7f1e1c0b18b0 2021-04-03T03:08:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- 172.21.2.111:0/774610655 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0b18b0 msgr2=0x7f1e1c0acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/774610655 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0b18b0 0x7f1e1c0acb30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f1e04002f30 tx=0x7f1e04006e90).stop 2021-04-03T03:08:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- 172.21.2.111:0/774610655 shutdown_connections 2021-04-03T03:08:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/774610655 >> v2:172.21.2.115:3300/0 conn(0x7f1e1c0603f0 0x7f1e1c067b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/774610655 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0b18b0 0x7f1e1c0acb30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- 172.21.2.111:0/774610655 >> 172.21.2.111:0/774610655 conn(0x7f1e1c0b61c0 msgr2=0x7f1e1c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- 172.21.2.111:0/774610655 shutdown_connections 2021-04-03T03:08:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- 172.21.2.111:0/774610655 wait complete. 2021-04-03T03:08:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 Processor -- start 2021-04-03T03:08:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- start start 2021-04-03T03:08:56.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e1c0603f0 0x7f1e1c13c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0ad020 0x7f1e1c063230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e1c0b18b0 0x7f1e1c063720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c142320 con 0x7f1e1c0ad020 2021-04-03T03:08:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c141ef0 con 0x7f1e1c0b18b0 2021-04-03T03:08:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c13ee60 con 0x7f1e1c0603f0 2021-04-03T03:08:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0ad020 0x7f1e1c063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0ad020 0x7f1e1c063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50016/0 (socket says 172.21.2.111:50016) 2021-04-03T03:08:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/3702501336 learned_addr learned my addr 172.21.2.111:0/3702501336 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/3702501336 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e1c0603f0 msgr2=0x7f1e1c13c030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:08:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e1c0603f0 0x7f1e1c13c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/3702501336 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e1c0b18b0 msgr2=0x7f1e1c063720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:08:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e1c0b18b0 0x7f1e1c063720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 -- 172.21.2.111:0/3702501336 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c13f120 con 0x7f1e1c0ad020 2021-04-03T03:08:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1a7fc700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0ad020 0x7f1e1c063230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f1e04002f30 tx=0x7f1e04015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e13fff700 1 -- 172.21.2.111:0/3702501336 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e04009e20 con 0x7f1e1c0ad020 2021-04-03T03:08:56.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e13fff700 1 -- 172.21.2.111:0/3702501336 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0401e460 con 0x7f1e1c0ad020 2021-04-03T03:08:56.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c13eab0 con 0x7f1e1c0ad020 2021-04-03T03:08:56.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e13fff700 1 -- 172.21.2.111:0/3702501336 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0401e7c0 con 0x7f1e1c0ad020 2021-04-03T03:08:56.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.223+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c0669e0 con 0x7f1e1c0ad020 2021-04-03T03:08:56.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.227+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c143230 con 0x7f1e1c0ad020 2021-04-03T03:08:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.227+0000 7f1e13fff700 1 -- 172.21.2.111:0/3702501336 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1e04025030 con 0x7f1e1c0ad020 2021-04-03T03:08:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.227+0000 7f1e13fff700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e00057050 0x7f1e00059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.227+0000 7f1e13fff700 1 -- 172.21.2.111:0/3702501336 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f1e04071970 con 0x7f1e1c0ad020 2021-04-03T03:08:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.227+0000 7f1e1affd700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e00057050 0x7f1e00059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.227+0000 7f1e1affd700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e00057050 0x7f1e00059510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1e0c002f70 tx=0x7f1e0c006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.227+0000 7f1e13fff700 1 -- 172.21.2.111:0/3702501336 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1e04046c40 con 0x7f1e1c0ad020 2021-04-03T03:08:56.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e1c057660 con 0x7f1e1c0ad020 2021-04-03T03:08:56.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e13fff700 1 -- 172.21.2.111:0/3702501336 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1e040230a0 con 0x7f1e1c0ad020 2021-04-03T03:08:56.331 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:56.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e00057050 msgr2=0x7f1e00059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e00057050 0x7f1e00059510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1e0c002f70 tx=0x7f1e0c006910).stop 2021-04-03T03:08:56.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0ad020 msgr2=0x7f1e1c063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0ad020 0x7f1e1c063230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f1e04002f30 tx=0x7f1e04015000).stop 2021-04-03T03:08:56.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 shutdown_connections 2021-04-03T03:08:56.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e00057050 0x7f1e00059510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e1c0b18b0 0x7f1e1c063720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e1c0603f0 0x7f1e1c13c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:56.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 --2- 172.21.2.111:0/3702501336 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e1c0ad020 0x7f1e1c063230 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 >> 172.21.2.111:0/3702501336 conn(0x7f1e1c0b61c0 msgr2=0x7f1e1c0ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:56.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 shutdown_connections 2021-04-03T03:08:56.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.323+0000 7f1e1bfff700 1 -- 172.21.2.111:0/3702501336 wait complete. 2021-04-03T03:08:56.339 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:08:56.339 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:08:56.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb706517700 1 Processor -- start 2021-04-03T03:08:56.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb706517700 1 -- start start 2021-04-03T03:08:56.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb706517700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb7000603f0 0x7fb7000b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb706517700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb70014b700 0x7fb7000b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb706517700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7000607d0 con 0x7fb7000b12b0 2021-04-03T03:08:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb706517700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7000af400 con 0x7fb7000603f0 2021-04-03T03:08:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb706517700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7000af6a0 con 0x7fb70014b700 2021-04-03T03:08:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb7000603f0 0x7fb7000b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb7000603f0 0x7fb7000b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50024/0 (socket says 172.21.2.111:50024) 2021-04-03T03:08:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 -- 172.21.2.111:0/767648575 learned_addr learned my addr 172.21.2.111:0/767648575 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 -- 172.21.2.111:0/767648575 >> v2:172.21.2.115:3300/0 conn(0x7fb70014b700 msgr2=0x7fb7000b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb704d14700 1 --2- 172.21.2.111:0/767648575 >> v2:172.21.2.115:3300/0 conn(0x7fb70014b700 0x7fb7000b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 --2- 172.21.2.111:0/767648575 >> v2:172.21.2.115:3300/0 conn(0x7fb70014b700 0x7fb7000b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 -- 172.21.2.111:0/767648575 >> v1:172.21.2.115:6789/0 conn(0x7fb7000b12b0 legacy=0x7fb7001465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 -- 172.21.2.111:0/767648575 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000afb40 con 0x7fb7000603f0 2021-04-03T03:08:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb705515700 1 --2- 172.21.2.111:0/767648575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000603f0 0x7fb7000b0090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb70006a3b0 tx=0x7fb6fc006e90).ready entity=mon.0 client_cookie=df25f9c06fcb2887 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb6f3fff700 1 -- 172.21.2.111:0/767648575 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6fc009930 con 0x7fb7000603f0 2021-04-03T03:08:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb6f3fff700 1 -- 172.21.2.111:0/767648575 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc009a90 con 0x7fb7000603f0 2021-04-03T03:08:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.387+0000 7fb6f3fff700 1 -- 172.21.2.111:0/767648575 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6fc009e30 con 0x7fb7000603f0 2021-04-03T03:08:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/767648575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000603f0 msgr2=0x7fb7000b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 --2- 172.21.2.111:0/767648575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000603f0 0x7fb7000b0090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb70006a3b0 tx=0x7fb6fc006e90).stop 2021-04-03T03:08:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/767648575 shutdown_connections 2021-04-03T03:08:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 --2- 172.21.2.111:0/767648575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000603f0 0x7fb7000b0090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 --2- 172.21.2.111:0/767648575 >> v2:172.21.2.115:3300/0 conn(0x7fb70014b700 0x7fb7000b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/767648575 >> 172.21.2.111:0/767648575 conn(0x7fb7000b61c0 msgr2=0x7fb70005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:56.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/767648575 shutdown_connections 2021-04-03T03:08:56.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/767648575 wait complete. 2021-04-03T03:08:56.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 Processor -- start 2021-04-03T03:08:56.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- start start 2021-04-03T03:08:56.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb7000603f0 0x7fb7000ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000b12b0 0x7fb700064dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb70014b700 0x7fb7000652b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000649d0 con 0x7fb7000b12b0 2021-04-03T03:08:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb700146790 con 0x7fb70014b700 2021-04-03T03:08:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000a9cc0 con 0x7fb7000603f0 2021-04-03T03:08:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000b12b0 0x7fb700064dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000b12b0 0x7fb700064dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50030/0 (socket says 172.21.2.111:50030) 2021-04-03T03:08:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 -- 172.21.2.111:0/2768922275 learned_addr learned my addr 172.21.2.111:0/2768922275 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb705515700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb7000603f0 0x7fb7000ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb705d16700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb70014b700 0x7fb7000652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 -- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb7000603f0 msgr2=0x7fb7000ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb7000603f0 0x7fb7000ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 -- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb70014b700 msgr2=0x7fb7000652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb70014b700 0x7fb7000652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 -- 172.21.2.111:0/2768922275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000657a0 con 0x7fb7000b12b0 2021-04-03T03:08:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb705d16700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb70014b700 0x7fb7000652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb704d14700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000b12b0 0x7fb700064dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fb6f4007000 tx=0x7fb6f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb6f27fc700 1 -- 172.21.2.111:0/2768922275 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f400e5e0 con 0x7fb7000b12b0 2021-04-03T03:08:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb6f27fc700 1 -- 172.21.2.111:0/2768922275 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f400c030 con 0x7fb7000b12b0 2021-04-03T03:08:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7000aa6c0 con 0x7fb7000b12b0 2021-04-03T03:08:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb6f27fc700 1 -- 172.21.2.111:0/2768922275 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f400e930 con 0x7fb7000b12b0 2021-04-03T03:08:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7000abc50 con 0x7fb7000b12b0 2021-04-03T03:08:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7000aa850 con 0x7fb7000b12b0 2021-04-03T03:08:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.391+0000 7fb6f27fc700 1 -- 172.21.2.111:0/2768922275 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb6f4019030 con 0x7fb7000b12b0 2021-04-03T03:08:56.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.395+0000 7fb6f27fc700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb6e0056f20 0x7fb6e00593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:56.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.395+0000 7fb6f27fc700 1 -- 172.21.2.111:0/2768922275 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb6f406f9a0 con 0x7fb7000b12b0 2021-04-03T03:08:56.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.395+0000 7fb705515700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb6e0056f20 0x7fb6e00593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:56.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.395+0000 7fb705515700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb6e0056f20 0x7fb6e00593e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fb6fc01a000 tx=0x7fb6fc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:56.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.395+0000 7fb6f27fc700 1 -- 172.21.2.111:0/2768922275 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb6f4042c50 con 0x7fb7000b12b0 2021-04-03T03:08:56.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb700057660 con 0x7fb7000b12b0 2021-04-03T03:08:56.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb6f27fc700 1 -- 172.21.2.111:0/2768922275 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fb6f4044e30 con 0x7fb7000b12b0 2021-04-03T03:08:56.527 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:56.527 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1066299897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1066299897},{"type":"v1","addr":"172.21.2.115:6833","nonce":1066299897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:08:44.422766+0000","modified":"2021-04-03T03:08:45.425728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2149096292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2149096292},{"type":"v1","addr":"172.21.2.111:6835","nonce":2149096292}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-04-03T03:08:56.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb6e0056f20 msgr2=0x7fb6e00593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb6e0056f20 0x7fb6e00593e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fb6fc01a000 tx=0x7fb6fc015000).stop 2021-04-03T03:08:56.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000b12b0 msgr2=0x7fb700064dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:56.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000b12b0 0x7fb700064dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fb6f4007000 tx=0x7fb6f4008040).stop 2021-04-03T03:08:56.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 shutdown_connections 2021-04-03T03:08:56.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb6e0056f20 0x7fb6e00593e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb70014b700 0x7fb7000652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb7000603f0 0x7fb7000ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 --2- 172.21.2.111:0/2768922275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7000b12b0 0x7fb700064dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:56.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 >> 172.21.2.111:0/2768922275 conn(0x7fb7000b61c0 msgr2=0x7fb700147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:56.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 shutdown_connections 2021-04-03T03:08:56.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:56.519+0000 7fb706517700 1 -- 172.21.2.111:0/2768922275 wait complete. 2021-04-03T03:08:56.531 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 5 2021-04-03T03:08:56.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:08:44.422766+0000', 'modified': '2021-04-03T03:08:45.425728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/2149096292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 2149096292}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 2149096292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:08:56.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:08:56.737 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:08:56.809 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":4239,"want_state":"up:active","state":"up:active","rank_uptime":12.376398166,"mdsmap_epoch":5,"osdmap_epoch":24,"osdmap_epoch_barrier":23,"uptime":17.594583378999999} 2021-04-03T03:08:56.810 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 4239, "mdsmap_epoch": 5, "osdmap_epoch": 24, "osdmap_epoch_barrier": 23, "rank_uptime": 12.376398166, "state": "up:active", "uptime": 17.594583379, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:08:56.810 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:08:56.811 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:08:56.811 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:08:56.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:56.812 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:08:56.822 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:08:56.876 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:08:56.877 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:08:56.925 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:08:56.926 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:08:56.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:56.927 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:08:56.978 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:08:56.978 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:08:57.024 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:08:57.031 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:08:57.032 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:08:57.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:08:57.066+0000 7f3f8fe3f0c0 -1 init, newargv = 0x5606ccfca2b0 newargc=9 2021-04-03T03:08:57.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[14886]: starting ceph client 2021-04-03T03:08:57.079 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:08:57.079 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:08:57.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:57.081 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:08:57.084 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[14886]: starting fuse 2021-04-03T03:08:57.087 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:08:57.088 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:08:57.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:57.088 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:08:57.089 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:08:57.089 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:08:57.089 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:08:57.090 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:08:57.090 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:08:57.090 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:08:57.091 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:08:57.091 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:08:57.091 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:08:57.092 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:08:57.092 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:08:57.092 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:08:57.093 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:08:57.093 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:08:57.093 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:08:57.094 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:08:57.094 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:08:57.094 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:08:57.094 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:08:57.095 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:08:57.095 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:08:57.095 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:08:57.179 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.14886.asok 2021-04-03T03:08:57.183 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14886.asok 2021-04-03T03:08:57.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:57.184 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14886.asok status 2021-04-03T03:08:57.448 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:08:57.448 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:08:57.448 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:08:57.449 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:08:57.449 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:08:57.449 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:08:57.449 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:08:57.450 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "14886", 2021-04-03T03:08:57.450 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:08:57.450 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:08:57.450 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:08:57.450 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:08:57.451 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 4629, 2021-04-03T03:08:57.451 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:08:57.451 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:08:57.451 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:08:57.451 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 4629 2021-04-03T03:08:57.452 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:08:57.452 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:08:57.452 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:08:57.452 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:08:57.453 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 2481556921 2021-04-03T03:08:57.453 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:08:57.453 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:08:57.453 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:08:57.453 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:08:57.454 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:08:57.454 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 2481556921 2021-04-03T03:08:57.454 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:08:57.454 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.4629 172.21.2.128:0/2481556921", 2021-04-03T03:08:57.455 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/2481556921", 2021-04-03T03:08:57.455 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:08:57.455 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 5, 2021-04-03T03:08:57.455 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 24, 2021-04-03T03:08:57.455 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:08:57.456 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:08:57.456 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:08:57.456 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:08:57.459 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:57.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d3fff700 1 Processor -- start 2021-04-03T03:08:57.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d3fff700 1 -- start start 2021-04-03T03:08:57.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f48cc041e50 0x7f48cc0056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc005ba0 0x7f48cc0d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d3fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc041960 con 0x7f48cc041580 2021-04-03T03:08:57.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d3fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0d2c80 con 0x7f48cc041e50 2021-04-03T03:08:57.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0d1c10 con 0x7f48cc005ba0 2021-04-03T03:08:57.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d27fc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f48cc041e50 0x7f48cc0056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d37fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc005ba0 0x7f48cc0d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d37fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc005ba0 0x7f48cc0d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:33390/0 (socket says 172.21.2.111:33390) 2021-04-03T03:08:57.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.511+0000 7f48d37fe700 1 -- 172.21.2.111:0/92712367 learned_addr learned my addr 172.21.2.111:0/92712367 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:57.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d37fe700 1 -- 172.21.2.111:0/92712367 >> v2:172.21.2.115:3300/0 conn(0x7f48cc041e50 msgr2=0x7f48cc0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d37fe700 1 --2- 172.21.2.111:0/92712367 >> v2:172.21.2.115:3300/0 conn(0x7f48cc041e50 0x7f48cc0056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d37fe700 1 -- 172.21.2.111:0/92712367 >> v1:172.21.2.115:6789/0 conn(0x7f48cc041580 legacy=0x7f48cc0d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d37fe700 1 -- 172.21.2.111:0/92712367 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0d1e30 con 0x7f48cc005ba0 2021-04-03T03:08:57.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d27fc700 1 --2- 172.21.2.111:0/92712367 >> v2:172.21.2.115:3300/0 conn(0x7f48cc041e50 0x7f48cc0056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:57.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d37fe700 1 --2- 172.21.2.111:0/92712367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc005ba0 0x7f48cc0d0b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f48d406b5d0 tx=0x7f48d405ad70).ready entity=mon.2 client_cookie=f9510e1f0babc3d5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d1ffb700 1 -- 172.21.2.111:0/92712367 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d405bda0 con 0x7f48cc005ba0 2021-04-03T03:08:57.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d1ffb700 1 -- 172.21.2.111:0/92712367 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d405df60 con 0x7f48cc005ba0 2021-04-03T03:08:57.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d1ffb700 1 -- 172.21.2.111:0/92712367 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d40699f0 con 0x7f48cc005ba0 2021-04-03T03:08:57.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- 172.21.2.111:0/92712367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc005ba0 msgr2=0x7f48cc0d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 --2- 172.21.2.111:0/92712367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc005ba0 0x7f48cc0d0b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f48d406b5d0 tx=0x7f48d405ad70).stop 2021-04-03T03:08:57.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- 172.21.2.111:0/92712367 shutdown_connections 2021-04-03T03:08:57.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 --2- 172.21.2.111:0/92712367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc005ba0 0x7f48cc0d0b80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 --2- 172.21.2.111:0/92712367 >> v2:172.21.2.115:3300/0 conn(0x7f48cc041e50 0x7f48cc0056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- 172.21.2.111:0/92712367 >> 172.21.2.111:0/92712367 conn(0x7f48cc0462c0 msgr2=0x7f48cc0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:57.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- 172.21.2.111:0/92712367 shutdown_connections 2021-04-03T03:08:57.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- 172.21.2.111:0/92712367 wait complete. 2021-04-03T03:08:57.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 Processor -- start 2021-04-03T03:08:57.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- start start 2021-04-03T03:08:57.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc005ba0 0x7f48cc0d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc041580 0x7f48cc039c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc041e50 0x7f48cc03ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc03c3e0 con 0x7f48cc041e50 2021-04-03T03:08:57.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc03a330 con 0x7f48cc005ba0 2021-04-03T03:08:57.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.515+0000 7f48d3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc03c5c0 con 0x7f48cc041580 2021-04-03T03:08:57.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc041e50 0x7f48cc03ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc041e50 0x7f48cc03ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50048/0 (socket says 172.21.2.111:50048) 2021-04-03T03:08:57.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 -- 172.21.2.111:0/296462839 learned_addr learned my addr 172.21.2.111:0/296462839 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:57.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 -- 172.21.2.111:0/296462839 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc041580 msgr2=0x7f48cc039c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc041580 0x7f48cc039c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 -- 172.21.2.111:0/296462839 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc005ba0 msgr2=0x7f48cc0d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d2ffd700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc005ba0 0x7f48cc0d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc005ba0 0x7f48cc0d6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 -- 172.21.2.111:0/296462839 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc03cf50 con 0x7f48cc041e50 2021-04-03T03:08:57.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d37fe700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc041e50 0x7f48cc03ca60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f48d406b5d0 tx=0x7f48d4065000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48cbfff700 1 -- 172.21.2.111:0/296462839 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d405dfe0 con 0x7f48cc041e50 2021-04-03T03:08:57.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48cbfff700 1 -- 172.21.2.111:0/296462839 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d40699f0 con 0x7f48cc041e50 2021-04-03T03:08:57.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48cbfff700 1 -- 172.21.2.111:0/296462839 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d405f2c0 con 0x7f48cc041e50 2021-04-03T03:08:57.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc0e41f0 con 0x7f48cc041e50 2021-04-03T03:08:57.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc03b400 con 0x7f48cc041e50 2021-04-03T03:08:57.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc03a180 con 0x7f48cc041e50 2021-04-03T03:08:57.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48cbfff700 1 -- 172.21.2.111:0/296462839 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f48d408c020 con 0x7f48cc041e50 2021-04-03T03:08:57.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48cbfff700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48b8056fb0 0x7f48b8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d2ffd700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48b8056fb0 0x7f48b8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48cbfff700 1 -- 172.21.2.111:0/296462839 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f48d40e05b0 con 0x7f48cc041e50 2021-04-03T03:08:57.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.519+0000 7f48d2ffd700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48b8056fb0 0x7f48b8059470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f48bc008000 tx=0x7f48bc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.523+0000 7f48cbfff700 1 -- 172.21.2.111:0/296462839 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f48d40b5880 con 0x7f48cc041e50 2021-04-03T03:08:57.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.611+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48cc03a4f0 con 0x7f48cc041e50 2021-04-03T03:08:57.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48cbfff700 1 -- 172.21.2.111:0/296462839 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f48d40b8ea0 con 0x7f48cc041e50 2021-04-03T03:08:57.622 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:57.622 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:57.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48b8056fb0 msgr2=0x7f48b8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48b8056fb0 0x7f48b8059470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f48bc008000 tx=0x7f48bc009040).stop 2021-04-03T03:08:57.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc041e50 msgr2=0x7f48cc03ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc041e50 0x7f48cc03ca60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f48d406b5d0 tx=0x7f48d4065000).stop 2021-04-03T03:08:57.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 shutdown_connections 2021-04-03T03:08:57.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc041e50 0x7f48cc03ca60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48b8056fb0 0x7f48b8059470 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc005ba0 0x7f48cc0d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 --2- 172.21.2.111:0/296462839 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48cc041580 0x7f48cc039c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 >> 172.21.2.111:0/296462839 conn(0x7f48cc0462c0 msgr2=0x7f48cc004410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:57.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 shutdown_connections 2021-04-03T03:08:57.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.615+0000 7f48d3fff700 1 -- 172.21.2.111:0/296462839 wait complete. 2021-04-03T03:08:57.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:08:57.630 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:08:57.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c29964700 1 Processor -- start 2021-04-03T03:08:57.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c29964700 1 -- start start 2021-04-03T03:08:57.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c29964700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 0x7f7c24147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c29964700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c240607d0 con 0x7f7c240603f0 2021-04-03T03:08:57.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c29964700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2414b090 con 0x7f7c24144ef0 2021-04-03T03:08:57.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c29964700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2414b330 con 0x7f7c240add50 2021-04-03T03:08:57.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c28962700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7c240603f0 0x7f7c241447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50844/0 (socket says 172.21.2.111:50844) 2021-04-03T03:08:57.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c28962700 1 -- 172.21.2.111:0/4251525874 learned_addr learned my addr 172.21.2.111:0/4251525874 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:57.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286198906 0 0) 0x7f7c240607d0 con 0x7f7c240603f0 2021-04-03T03:08:57.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c10003240 con 0x7f7c240603f0 2021-04-03T03:08:57.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c23fff700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 0x7f7c24147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3954044819 0 0) 0x7f7c10003240 con 0x7f7c240603f0 2021-04-03T03:08:57.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 msgr2=0x7f7c24147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 0x7f7c24147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 >> v1:172.21.2.115:6789/0 conn(0x7f7c24144ef0 legacy=0x7f7c24147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2414b5d0 con 0x7f7c240603f0 2021-04-03T03:08:57.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f7c180016d0 con 0x7f7c240603f0 2021-04-03T03:08:57.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.679+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 >> v1:172.21.2.111:6789/0 conn(0x7f7c240603f0 legacy=0x7f7c241447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 0x7f7c10003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c10003c50 0x7f7c100044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c100049d0 0x7f7c10004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2414b330 con 0x7f7c100049d0 2021-04-03T03:08:57.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c100041e0 con 0x7f7c10003c50 2021-04-03T03:08:57.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1000b730 con 0x7f7c240add50 2021-04-03T03:08:57.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c100049d0 0x7f7c10004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c28962700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 0x7f7c10003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 msgr2=0x7f7c10003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c240add50 0x7f7c10003760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c10003c50 msgr2=0x7f7c100044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c10003c50 0x7f7c100044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/4251525874 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10003240 con 0x7f7c100049d0 2021-04-03T03:08:57.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c100049d0 0x7f7c10004e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f7c1c0022e0 tx=0x7f7c1c007ef0).ready entity=mon.0 client_cookie=2003a25e1d23ad4f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1c00c8e0 con 0x7f7c100049d0 2021-04-03T03:08:57.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00ca40 con 0x7f7c100049d0 2021-04-03T03:08:57.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c237fe700 1 -- 172.21.2.111:0/4251525874 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1c00cda0 con 0x7f7c100049d0 2021-04-03T03:08:57.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- 172.21.2.111:0/4251525874 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c100049d0 msgr2=0x7f7c10004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 --2- 172.21.2.111:0/4251525874 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c100049d0 0x7f7c10004e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f7c1c0022e0 tx=0x7f7c1c007ef0).stop 2021-04-03T03:08:57.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/4251525874 reap_dead start 2021-04-03T03:08:57.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- 172.21.2.111:0/4251525874 shutdown_connections 2021-04-03T03:08:57.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- 172.21.2.111:0/4251525874 >> 172.21.2.111:0/4251525874 conn(0x7f7c240b61c0 msgr2=0x7f7c2405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:57.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- 172.21.2.111:0/4251525874 shutdown_connections 2021-04-03T03:08:57.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- 172.21.2.111:0/4251525874 wait complete. 2021-04-03T03:08:57.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 Processor -- start 2021-04-03T03:08:57.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- start start 2021-04-03T03:08:57.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c240a9d90 0x7f7c240aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c24147ae0 0x7f7c24146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c24146810 0x7f7c24159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240aa660 con 0x7f7c24146810 2021-04-03T03:08:57.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c241574b0 con 0x7f7c240a9d90 2021-04-03T03:08:57.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c24147870 con 0x7f7c24147ae0 2021-04-03T03:08:57.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c24146810 0x7f7c24159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c24146810 0x7f7c24159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50064/0 (socket says 172.21.2.111:50064) 2021-04-03T03:08:57.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/1143817866 learned_addr learned my addr 172.21.2.111:0/1143817866 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:57.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/1143817866 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c24147ae0 msgr2=0x7f7c24146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c24147ae0 0x7f7c24146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/1143817866 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c240a9d90 msgr2=0x7f7c240aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c240a9d90 0x7f7c240aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 -- 172.21.2.111:0/1143817866 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24157160 con 0x7f7c24146810 2021-04-03T03:08:57.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29163700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c24146810 0x7f7c24159d60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7c18001930 tx=0x7f7c1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c21ffb700 1 -- 172.21.2.111:0/1143817866 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1c01f480 con 0x7f7c24146810 2021-04-03T03:08:57.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c21ffb700 1 -- 172.21.2.111:0/1143817866 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c01f5e0 con 0x7f7c24146810 2021-04-03T03:08:57.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c24146c90 con 0x7f7c24146810 2021-04-03T03:08:57.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c21ffb700 1 -- 172.21.2.111:0/1143817866 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1c01f940 con 0x7f7c24146810 2021-04-03T03:08:57.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.683+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2415a250 con 0x7f7c24146810 2021-04-03T03:08:57.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.687+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c241572f0 con 0x7f7c24146810 2021-04-03T03:08:57.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.687+0000 7f7c21ffb700 1 -- 172.21.2.111:0/1143817866 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7c1c010030 con 0x7f7c24146810 2021-04-03T03:08:57.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.687+0000 7f7c21ffb700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c04056ea0 0x7f7c04059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.687+0000 7f7c21ffb700 1 -- 172.21.2.111:0/1143817866 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f7c1c0719d0 con 0x7f7c24146810 2021-04-03T03:08:57.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.687+0000 7f7c28962700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c04056ea0 0x7f7c04059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.687+0000 7f7c28962700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c04056ea0 0x7f7c04059360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7c18008000 tx=0x7f7c18009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.687+0000 7f7c21ffb700 1 -- 172.21.2.111:0/1143817866 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7c1c046db0 con 0x7f7c24146810 2021-04-03T03:08:57.736 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:57.738 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:57.741 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:57.744 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:57.747 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:08:57.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7c24157670 con 0x7f7c24146810 2021-04-03T03:08:57.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c21ffb700 1 -- 172.21.2.111:0/1143817866 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7c1c04a3d0 con 0x7f7c24146810 2021-04-03T03:08:57.790 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:08:57.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c04056ea0 msgr2=0x7f7c04059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c04056ea0 0x7f7c04059360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7c18008000 tx=0x7f7c18009040).stop 2021-04-03T03:08:57.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c24146810 msgr2=0x7f7c24159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c24146810 0x7f7c24159d60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7c18001930 tx=0x7f7c1c016000).stop 2021-04-03T03:08:57.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 shutdown_connections 2021-04-03T03:08:57.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c24146810 0x7f7c24159d60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c04056ea0 0x7f7c04059360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c240a9d90 0x7f7c240aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 --2- 172.21.2.111:0/1143817866 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c24147ae0 0x7f7c24146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 >> 172.21.2.111:0/1143817866 conn(0x7f7c240b61c0 msgr2=0x7f7c24143510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:57.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 shutdown_connections 2021-04-03T03:08:57.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.783+0000 7f7c29964700 1 -- 172.21.2.111:0/1143817866 wait complete. 2021-04-03T03:08:57.798 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:08:57.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8beb59e700 1 Processor -- start 2021-04-03T03:08:57.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8beb59e700 1 -- start start 2021-04-03T03:08:57.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8beb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be4040150 0x7f8be40d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8beb59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be40405b0 con 0x7f8be4006230 2021-04-03T03:08:57.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8beb59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be4006610 con 0x7f8be40d09b0 2021-04-03T03:08:57.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8beb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be403db40 con 0x7f8be4040150 2021-04-03T03:08:57.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8be4006230 0x7f8be40cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50862/0 (socket says 172.21.2.111:50862) 2021-04-03T03:08:57.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 -- 172.21.2.111:0/1446034219 learned_addr learned my addr 172.21.2.111:0/1446034219 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:57.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440104987 0 0) 0x7f8be40405b0 con 0x7f8be4006230 2021-04-03T03:08:57.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bcc004350 con 0x7f8be4006230 2021-04-03T03:08:57.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1007126224 0 0) 0x7f8bcc004350 con 0x7f8be4006230 2021-04-03T03:08:57.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be4040150 msgr2=0x7f8be40d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be9d9b700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be4040150 0x7f8be40d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be4040150 0x7f8be40d04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 >> v1:172.21.2.115:6789/0 conn(0x7f8be40d09b0 legacy=0x7f8be40d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be403dde0 con 0x7f8be4006230 2021-04-03T03:08:57.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8bdc0016d0 con 0x7f8be4006230 2021-04-03T03:08:57.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 >> v1:172.21.2.111:6789/0 conn(0x7f8be4006230 legacy=0x7f8be40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bcc0048f0 0x7f8bcc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bcc0055f0 0x7f8bcc005a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bcc005f60 0x7f8bcc003240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be403db40 con 0x7f8bcc0048f0 2021-04-03T03:08:57.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0063e0 con 0x7f8bcc0055f0 2021-04-03T03:08:57.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc003730 con 0x7f8bcc005f60 2021-04-03T03:08:57.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8be9d9b700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be4040150 0x7f8be40d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:57.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bcc0048f0 0x7f8bcc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 -- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bcc005f60 msgr2=0x7f8bcc003240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bcc005f60 0x7f8bcc003240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 -- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bcc0055f0 msgr2=0x7f8bcc005a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:57.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bcc0055f0 0x7f8bcc005a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 -- 172.21.2.111:0/1446034219 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc004350 con 0x7f8bcc0048f0 2021-04-03T03:08:57.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.891+0000 7f8bea59c700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bcc0048f0 0x7f8bcc004cf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f8be400c350 tx=0x7f8bdc0076d0).ready entity=mon.0 client_cookie=1eda5e6f2972bc08 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bdc0070d0 con 0x7f8bcc0048f0 2021-04-03T03:08:57.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc007230 con 0x7f8bcc0048f0 2021-04-03T03:08:57.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be959a700 1 -- 172.21.2.111:0/1446034219 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bdc0055f0 con 0x7f8bcc0048f0 2021-04-03T03:08:57.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1446034219 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bcc0048f0 msgr2=0x7f8bcc004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 --2- 172.21.2.111:0/1446034219 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bcc0048f0 0x7f8bcc004cf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f8be400c350 tx=0x7f8bdc0076d0).stop 2021-04-03T03:08:57.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bead9d700 1 -- 172.21.2.111:0/1446034219 reap_dead start 2021-04-03T03:08:57.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1446034219 shutdown_connections 2021-04-03T03:08:57.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1446034219 >> 172.21.2.111:0/1446034219 conn(0x7f8be40462b0 msgr2=0x7f8be40466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:57.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1446034219 shutdown_connections 2021-04-03T03:08:57.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1446034219 wait complete. 2021-04-03T03:08:57.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 Processor -- start 2021-04-03T03:08:57.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- start start 2021-04-03T03:08:57.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be403b820 0x7f8be40e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be40a15a0 0x7f8be40a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be40ac390 0x7f8be40ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40a2b20 con 0x7f8be403b820 2021-04-03T03:08:57.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40e6f60 con 0x7f8be40a15a0 2021-04-03T03:08:57.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40e8170 con 0x7f8be40ac390 2021-04-03T03:08:57.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be403b820 0x7f8be40e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be403b820 0x7f8be40e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50082/0 (socket says 172.21.2.111:50082) 2021-04-03T03:08:57.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 -- 172.21.2.111:0/1376278673 learned_addr learned my addr 172.21.2.111:0/1376278673 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:57.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be9d9b700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be40a15a0 0x7f8be40a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 -- 172.21.2.111:0/1376278673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be40ac390 msgr2=0x7f8be40ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be40ac390 0x7f8be40ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:57.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 -- 172.21.2.111:0/1376278673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be40a15a0 msgr2=0x7f8be40a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:57.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be40a15a0 0x7f8be40a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:57.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 -- 172.21.2.111:0/1376278673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be40e7150 con 0x7f8be403b820 2021-04-03T03:08:57.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8bea59c700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be403b820 0x7f8be40e7b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f8be400c350 tx=0x7f8bdc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be37fe700 1 -- 172.21.2.111:0/1376278673 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bdc005cf0 con 0x7f8be403b820 2021-04-03T03:08:57.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be37fe700 1 -- 172.21.2.111:0/1376278673 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc005e50 con 0x7f8be403b820 2021-04-03T03:08:57.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be40e73f0 con 0x7f8be403b820 2021-04-03T03:08:57.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be40a1f10 con 0x7f8be403b820 2021-04-03T03:08:57.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be37fe700 1 -- 172.21.2.111:0/1376278673 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bdc020670 con 0x7f8be403b820 2021-04-03T03:08:57.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be40e7820 con 0x7f8be403b820 2021-04-03T03:08:57.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.895+0000 7f8be37fe700 1 -- 172.21.2.111:0/1376278673 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8bdc010070 con 0x7f8be403b820 2021-04-03T03:08:57.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.899+0000 7f8be37fe700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bd00570a0 0x7f8bd0059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:57.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.899+0000 7f8be37fe700 1 -- 172.21.2.111:0/1376278673 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f8bdc071480 con 0x7f8be403b820 2021-04-03T03:08:57.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.899+0000 7f8be9d9b700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bd00570a0 0x7f8bd0059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:57.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.899+0000 7f8be9d9b700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bd00570a0 0x7f8bd0059560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8bd4008000 tx=0x7f8bd4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:57.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.899+0000 7f8be37fe700 1 -- 172.21.2.111:0/1376278673 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8bdc046750 con 0x7f8be403b820 2021-04-03T03:08:58.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.991+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8be4004db0 con 0x7f8be403b820 2021-04-03T03:08:58.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.991+0000 7f8be37fe700 1 -- 172.21.2.111:0/1376278673 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8bdc049d70 con 0x7f8be403b820 2021-04-03T03:08:58.001 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:58.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bd00570a0 msgr2=0x7f8bd0059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bd00570a0 0x7f8bd0059560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8bd4008000 tx=0x7f8bd4008040).stop 2021-04-03T03:08:58.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be403b820 msgr2=0x7f8be40e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be403b820 0x7f8be40e7b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f8be400c350 tx=0x7f8bdc015000).stop 2021-04-03T03:08:58.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 shutdown_connections 2021-04-03T03:08:58.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be40ac390 0x7f8be40ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be403b820 0x7f8be40e7b30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bd00570a0 0x7f8bd0059560 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 --2- 172.21.2.111:0/1376278673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be40a15a0 0x7f8be40a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 >> 172.21.2.111:0/1376278673 conn(0x7f8be40462b0 msgr2=0x7f8be4045aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:58.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 shutdown_connections 2021-04-03T03:08:58.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:57.995+0000 7f8beb59e700 1 -- 172.21.2.111:0/1376278673 wait complete. 2021-04-03T03:08:58.008 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:08:58.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7819aae700 1 Processor -- start 2021-04-03T03:08:58.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7819aae700 1 -- start start 2021-04-03T03:08:58.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7819aae700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f78140603f0 0x7f78141447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7819aae700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140add50 0x7f7814144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7819aae700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78140607d0 con 0x7f78140b17b0 2021-04-03T03:08:58.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7819aae700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78141451c0 con 0x7f78140603f0 2021-04-03T03:08:58.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7819aae700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781414b090 con 0x7f78140add50 2021-04-03T03:08:58.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7813fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140add50 0x7f7814144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7813fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140add50 0x7f7814144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:33440/0 (socket says 172.21.2.111:33440) 2021-04-03T03:08:58.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7813fff700 1 -- 172.21.2.111:0/1110759845 learned_addr learned my addr 172.21.2.111:0/1110759845 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:58.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.059+0000 7f7818aac700 1 --2- 172.21.2.111:0/1110759845 >> v2:172.21.2.115:3300/0 conn(0x7f78140603f0 0x7f78141447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 -- 172.21.2.111:0/1110759845 >> v2:172.21.2.115:3300/0 conn(0x7f78140603f0 msgr2=0x7f78141447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:58.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 --2- 172.21.2.111:0/1110759845 >> v2:172.21.2.115:3300/0 conn(0x7f78140603f0 0x7f78141447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 -- 172.21.2.111:0/1110759845 >> v1:172.21.2.115:6789/0 conn(0x7f78140b17b0 legacy=0x7f7814147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 -- 172.21.2.111:0/1110759845 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781414b330 con 0x7f78140add50 2021-04-03T03:08:58.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 --2- 172.21.2.111:0/1110759845 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140add50 0x7f7814144cd0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f77fc002f30 tx=0x7f77fc006e90).ready entity=mon.2 client_cookie=fd54784c73fdc19a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f78137fe700 1 -- 172.21.2.111:0/1110759845 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77fc009850 con 0x7f78140add50 2021-04-03T03:08:58.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f78137fe700 1 -- 172.21.2.111:0/1110759845 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc0099b0 con 0x7f78140add50 2021-04-03T03:08:58.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f78137fe700 1 -- 172.21.2.111:0/1110759845 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77fc009d10 con 0x7f78140add50 2021-04-03T03:08:58.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- 172.21.2.111:0/1110759845 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140add50 msgr2=0x7f7814144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 --2- 172.21.2.111:0/1110759845 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140add50 0x7f7814144cd0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f77fc002f30 tx=0x7f77fc006e90).stop 2021-04-03T03:08:58.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- 172.21.2.111:0/1110759845 shutdown_connections 2021-04-03T03:08:58.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 --2- 172.21.2.111:0/1110759845 >> v2:172.21.2.115:3300/0 conn(0x7f78140603f0 0x7f78141447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 --2- 172.21.2.111:0/1110759845 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140add50 0x7f7814144cd0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- 172.21.2.111:0/1110759845 >> 172.21.2.111:0/1110759845 conn(0x7f78140b61c0 msgr2=0x7f781405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:58.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- 172.21.2.111:0/1110759845 shutdown_connections 2021-04-03T03:08:58.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- 172.21.2.111:0/1110759845 wait complete. 2021-04-03T03:08:58.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 Processor -- start 2021-04-03T03:08:58.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- start start 2021-04-03T03:08:58.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140603f0 0x7f7814062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78140add50 0x7f7814156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78140b17b0 0x7f7814156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78140abf90 con 0x7f78140add50 2021-04-03T03:08:58.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78140a7eb0 con 0x7f78140b17b0 2021-04-03T03:08:58.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78140aa9f0 con 0x7f78140603f0 2021-04-03T03:08:58.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78140add50 0x7f7814156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78140add50 0x7f7814156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50096/0 (socket says 172.21.2.111:50096) 2021-04-03T03:08:58.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 -- 172.21.2.111:0/490593358 learned_addr learned my addr 172.21.2.111:0/490593358 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:58.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 -- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140603f0 msgr2=0x7f7814062d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7818aac700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140603f0 0x7f7814062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140603f0 0x7f7814062d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 -- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78140b17b0 msgr2=0x7f7814156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f78192ad700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78140b17b0 0x7f7814156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78140b17b0 0x7f7814156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 -- 172.21.2.111:0/490593358 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78140aac90 con 0x7f78140add50 2021-04-03T03:08:58.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7813fff700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78140add50 0x7f7814156a70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f77fc002f30 tx=0x7f77fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7811ffb700 1 -- 172.21.2.111:0/490593358 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77fc011ca0 con 0x7f78140add50 2021-04-03T03:08:58.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7811ffb700 1 -- 172.21.2.111:0/490593358 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc011e00 con 0x7f78140add50 2021-04-03T03:08:58.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7811ffb700 1 -- 172.21.2.111:0/490593358 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77fc01e650 con 0x7f78140add50 2021-04-03T03:08:58.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78140abc40 con 0x7f78140add50 2021-04-03T03:08:58.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.063+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78140a9920 con 0x7f78140add50 2021-04-03T03:08:58.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.067+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78140aae20 con 0x7f78140add50 2021-04-03T03:08:58.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.067+0000 7f7811ffb700 1 -- 172.21.2.111:0/490593358 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f77fc025030 con 0x7f78140add50 2021-04-03T03:08:58.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.067+0000 7f7811ffb700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77f8056fb0 0x7f77f8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.067+0000 7f7811ffb700 1 -- 172.21.2.111:0/490593358 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f77fc0716d0 con 0x7f78140add50 2021-04-03T03:08:58.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.067+0000 7f7818aac700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77f8056fb0 0x7f77f8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.067+0000 7f7818aac700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77f8056fb0 0x7f77f8059470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f780400b000 tx=0x7f7804008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.067+0000 7f7811ffb700 1 -- 172.21.2.111:0/490593358 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f77fc0469a0 con 0x7f78140add50 2021-04-03T03:08:58.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7814057660 con 0x7f78140add50 2021-04-03T03:08:58.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7811ffb700 1 -- 172.21.2.111:0/490593358 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f77fc0230a0 con 0x7f78140add50 2021-04-03T03:08:58.170 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:58.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77f8056fb0 msgr2=0x7f77f8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77f8056fb0 0x7f77f8059470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f780400b000 tx=0x7f7804008040).stop 2021-04-03T03:08:58.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78140add50 msgr2=0x7f7814156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78140add50 0x7f7814156a70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f77fc002f30 tx=0x7f77fc015000).stop 2021-04-03T03:08:58.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 shutdown_connections 2021-04-03T03:08:58.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77f8056fb0 0x7f77f8059470 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78140b17b0 0x7f7814156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78140603f0 0x7f7814062d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 --2- 172.21.2.111:0/490593358 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78140add50 0x7f7814156a70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 >> 172.21.2.111:0/490593358 conn(0x7f78140b61c0 msgr2=0x7f7814144860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:58.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 shutdown_connections 2021-04-03T03:08:58.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.163+0000 7f7819aae700 1 -- 172.21.2.111:0/490593358 wait complete. 2021-04-03T03:08:58.177 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:08:58.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 Processor -- start 2021-04-03T03:08:58.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- start start 2021-04-03T03:08:58.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c0603f0 0x7f493c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f493c0b17b0 0x7f493c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c0607d0 con 0x7f493c0add50 2021-04-03T03:08:58.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c1451c0 con 0x7f493c0b17b0 2021-04-03T03:08:58.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c14b090 con 0x7f493c0603f0 2021-04-03T03:08:58.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493a7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f493c0add50 0x7f493c147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50898/0 (socket says 172.21.2.111:50898) 2021-04-03T03:08:58.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493a7fc700 1 -- 172.21.2.111:0/2331152389 learned_addr learned my addr 172.21.2.111:0/2331152389 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:58.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277177673 0 0) 0x7f493c0607d0 con 0x7f493c0add50 2021-04-03T03:08:58.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f491c003240 con 0x7f493c0add50 2021-04-03T03:08:58.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c0603f0 0x7f493c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493b7fe700 1 --2- 172.21.2.111:0/2331152389 >> v2:172.21.2.115:3300/0 conn(0x7f493c0b17b0 0x7f493c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3980346660 0 0) 0x7f491c003240 con 0x7f493c0add50 2021-04-03T03:08:58.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c0603f0 msgr2=0x7f493c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:58.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c0603f0 0x7f493c1447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 >> v2:172.21.2.115:3300/0 conn(0x7f493c0b17b0 msgr2=0x7f493c144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:58.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 --2- 172.21.2.111:0/2331152389 >> v2:172.21.2.115:3300/0 conn(0x7f493c0b17b0 0x7f493c144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c14b330 con 0x7f493c0add50 2021-04-03T03:08:58.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f49240016d0 con 0x7f493c0add50 2021-04-03T03:08:58.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 >> v1:172.21.2.111:6789/0 conn(0x7f493c0add50 legacy=0x7f493c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f491c0037b0 0x7f491c003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c0603f0 0x7f491c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f491c0049d0 0x7f491c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c14b090 con 0x7f491c0037b0 2021-04-03T03:08:58.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0042a0 con 0x7f491c0049d0 2021-04-03T03:08:58.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c00b730 con 0x7f493c0603f0 2021-04-03T03:08:58.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f491c0037b0 0x7f491c003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 -- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c0603f0 msgr2=0x7f491c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:58.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c0603f0 0x7f491c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 -- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f491c0049d0 msgr2=0x7f491c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:58.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f491c0049d0 0x7f491c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 -- 172.21.2.111:0/2331152389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c003240 con 0x7f491c0037b0 2021-04-03T03:08:58.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493affd700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f491c0037b0 0x7f491c003bd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f492c00a000 tx=0x7f492c008040).ready entity=mon.0 client_cookie=2a6cf69785af7d1c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c00da60 con 0x7f491c0037b0 2021-04-03T03:08:58.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00dbc0 con 0x7f491c0037b0 2021-04-03T03:08:58.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- 172.21.2.111:0/2331152389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f491c0037b0 msgr2=0x7f491c003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f4939ffb700 1 -- 172.21.2.111:0/2331152389 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c013480 con 0x7f491c0037b0 2021-04-03T03:08:58.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 --2- 172.21.2.111:0/2331152389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f491c0037b0 0x7f491c003bd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f492c00a000 tx=0x7f492c008040).stop 2021-04-03T03:08:58.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493b7fe700 1 -- 172.21.2.111:0/2331152389 reap_dead start 2021-04-03T03:08:58.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- 172.21.2.111:0/2331152389 shutdown_connections 2021-04-03T03:08:58.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- 172.21.2.111:0/2331152389 >> 172.21.2.111:0/2331152389 conn(0x7f493c0b61c0 msgr2=0x7f493c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:58.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- 172.21.2.111:0/2331152389 shutdown_connections 2021-04-03T03:08:58.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.227+0000 7f493bfff700 1 -- 172.21.2.111:0/2331152389 wait complete. 2021-04-03T03:08:58.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 Processor -- start 2021-04-03T03:08:58.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 -- start start 2021-04-03T03:08:58.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f493c0b0b20 0x7f493c063420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f493c065f90 0x7f493c066410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c066900 0x7f493c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c063910 con 0x7f493c0b0b20 2021-04-03T03:08:58.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c062a50 con 0x7f493c065f90 2021-04-03T03:08:58.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0b0f00 con 0x7f493c066900 2021-04-03T03:08:58.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f493c0b0b20 0x7f493c063420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f493c0b0b20 0x7f493c063420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50118/0 (socket says 172.21.2.111:50118) 2021-04-03T03:08:58.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 -- 172.21.2.111:0/4050972958 learned_addr learned my addr 172.21.2.111:0/4050972958 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:58.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 -- 172.21.2.111:0/4050972958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c066900 msgr2=0x7f493c155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c066900 0x7f493c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 -- 172.21.2.111:0/4050972958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f493c065f90 msgr2=0x7f493c066410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f493c065f90 0x7f493c066410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 -- 172.21.2.111:0/4050972958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c062cb0 con 0x7f493c0b0b20 2021-04-03T03:08:58.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493affd700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f493c0b0b20 0x7f493c063420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f492c00a000 tx=0x7f492c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f4933fff700 1 -- 172.21.2.111:0/4050972958 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c021070 con 0x7f493c0b0b20 2021-04-03T03:08:58.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f4933fff700 1 -- 172.21.2.111:0/4050972958 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c01d440 con 0x7f493c0b0b20 2021-04-03T03:08:58.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c062f50 con 0x7f493c0b0b20 2021-04-03T03:08:58.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f4933fff700 1 -- 172.21.2.111:0/4050972958 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c01d7a0 con 0x7f493c0b0b20 2021-04-03T03:08:58.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c157990 con 0x7f493c0b0b20 2021-04-03T03:08:58.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c065d90 con 0x7f493c0b0b20 2021-04-03T03:08:58.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f4933fff700 1 -- 172.21.2.111:0/4050972958 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f492c01d900 con 0x7f493c0b0b20 2021-04-03T03:08:58.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f4933fff700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4920056fb0 0x7f4920059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f4933fff700 1 -- 172.21.2.111:0/4050972958 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f492c071b00 con 0x7f493c0b0b20 2021-04-03T03:08:58.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493a7fc700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4920056fb0 0x7f4920059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.231+0000 7f493a7fc700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4920056fb0 0x7f4920059470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f49240013e0 tx=0x7f493c14b4c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.235+0000 7f4933fff700 1 -- 172.21.2.111:0/4050972958 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f492c046ee0 con 0x7f493c0b0b20 2021-04-03T03:08:58.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.327+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f493c057660 con 0x7f493c0b0b20 2021-04-03T03:08:58.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.327+0000 7f4933fff700 1 -- 172.21.2.111:0/4050972958 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f492c04a500 con 0x7f493c0b0b20 2021-04-03T03:08:58.337 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:08:58.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4920056fb0 msgr2=0x7f4920059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4920056fb0 0x7f4920059470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f49240013e0 tx=0x7f493c14b4c0).stop 2021-04-03T03:08:58.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f493c0b0b20 msgr2=0x7f493c063420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f493c0b0b20 0x7f493c063420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f492c00a000 tx=0x7f492c008040).stop 2021-04-03T03:08:58.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 shutdown_connections 2021-04-03T03:08:58.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f493c066900 0x7f493c155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f493c0b0b20 0x7f493c063420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4920056fb0 0x7f4920059470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 --2- 172.21.2.111:0/4050972958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f493c065f90 0x7f493c066410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 >> 172.21.2.111:0/4050972958 conn(0x7f493c0b61c0 msgr2=0x7f493c145e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:58.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 shutdown_connections 2021-04-03T03:08:58.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.331+0000 7f493bfff700 1 -- 172.21.2.111:0/4050972958 wait complete. 2021-04-03T03:08:58.345 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:08:58.345 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:08:58.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 Processor -- start 2021-04-03T03:08:58.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- start start 2021-04-03T03:08:58.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f38980065c0 0x7f38980074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38980079a0 0x7f38980d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389800c080 con 0x7f389800bca0 2021-04-03T03:08:58.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38980d8570 con 0x7f38980065c0 2021-04-03T03:08:58.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38980d7430 con 0x7f38980079a0 2021-04-03T03:08:58.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f38980065c0 0x7f38980074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f38980065c0 0x7f38980074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50124/0 (socket says 172.21.2.111:50124) 2021-04-03T03:08:58.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 -- 172.21.2.111:0/381248334 learned_addr learned my addr 172.21.2.111:0/381248334 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:58.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 -- 172.21.2.111:0/381248334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38980079a0 msgr2=0x7f38980d6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:58.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 --2- 172.21.2.111:0/381248334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38980079a0 0x7f38980d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 -- 172.21.2.111:0/381248334 >> v1:172.21.2.115:6789/0 conn(0x7f389800bca0 legacy=0x7f38980d7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:58.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 -- 172.21.2.111:0/381248334 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980d7720 con 0x7f38980065c0 2021-04-03T03:08:58.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389e5e4700 1 --2- 172.21.2.111:0/381248334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f38980074b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3898046070 tx=0x7f3890006e90).ready entity=mon.0 client_cookie=eb6a64517e1a77ac server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389dde3700 1 -- 172.21.2.111:0/381248334 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38900098e0 con 0x7f38980065c0 2021-04-03T03:08:58.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389dde3700 1 -- 172.21.2.111:0/381248334 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890009a40 con 0x7f38980065c0 2021-04-03T03:08:58.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389dde3700 1 -- 172.21.2.111:0/381248334 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3890009de0 con 0x7f38980065c0 2021-04-03T03:08:58.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- 172.21.2.111:0/381248334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 msgr2=0x7f38980074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- 172.21.2.111:0/381248334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f38980074b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3898046070 tx=0x7f3890006e90).stop 2021-04-03T03:08:58.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- 172.21.2.111:0/381248334 shutdown_connections 2021-04-03T03:08:58.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- 172.21.2.111:0/381248334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38980079a0 0x7f38980d6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- 172.21.2.111:0/381248334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f38980074b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- 172.21.2.111:0/381248334 >> 172.21.2.111:0/381248334 conn(0x7f38980484d0 msgr2=0x7f38980488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:58.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- 172.21.2.111:0/381248334 shutdown_connections 2021-04-03T03:08:58.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- 172.21.2.111:0/381248334 wait complete. 2021-04-03T03:08:58.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 Processor -- start 2021-04-03T03:08:58.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- start start 2021-04-03T03:08:58.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f389802d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38980079a0 0x7f38980e6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389800bca0 0x7f389802a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3898045610 con 0x7f38980065c0 2021-04-03T03:08:58.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3898045d80 con 0x7f38980079a0 2021-04-03T03:08:58.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.399+0000 7f389fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3898028960 con 0x7f389800bca0 2021-04-03T03:08:58.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f389802d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f389802d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50128/0 (socket says 172.21.2.111:50128) 2021-04-03T03:08:58.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 -- 172.21.2.111:0/2167532139 learned_addr learned my addr 172.21.2.111:0/2167532139 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:58.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 -- 172.21.2.111:0/2167532139 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389800bca0 msgr2=0x7f389802a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389800bca0 0x7f389802a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 -- 172.21.2.111:0/2167532139 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38980079a0 msgr2=0x7f38980e6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38980079a0 0x7f38980e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 -- 172.21.2.111:0/2167532139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898045840 con 0x7f38980065c0 2021-04-03T03:08:58.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389ede5700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f389802d550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f3898028af0 tx=0x7f3888006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f3897fff700 1 -- 172.21.2.111:0/2167532139 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f388800f510 con 0x7f38980065c0 2021-04-03T03:08:58.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f3897fff700 1 -- 172.21.2.111:0/2167532139 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800f670 con 0x7f38980065c0 2021-04-03T03:08:58.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3898045ae0 con 0x7f38980065c0 2021-04-03T03:08:58.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f3897fff700 1 -- 172.21.2.111:0/2167532139 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f388800f7e0 con 0x7f38980065c0 2021-04-03T03:08:58.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3898027780 con 0x7f38980065c0 2021-04-03T03:08:58.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3898029d40 con 0x7f38980065c0 2021-04-03T03:08:58.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f3897fff700 1 -- 172.21.2.111:0/2167532139 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f388800f070 con 0x7f38980065c0 2021-04-03T03:08:58.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f3897fff700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3884056f20 0x7f38840593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:58.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f3897fff700 1 -- 172.21.2.111:0/2167532139 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f388806fac0 con 0x7f38980065c0 2021-04-03T03:08:58.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389e5e4700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3884056f20 0x7f38840593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:58.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.403+0000 7f389e5e4700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3884056f20 0x7f38840593e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3898046070 tx=0x7f3890015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:58.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.407+0000 7f3897fff700 1 -- 172.21.2.111:0/2167532139 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3898029d40 con 0x7f38980065c0 2021-04-03T03:08:58.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.527+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3898029d40 con 0x7f38980065c0 2021-04-03T03:08:58.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f3897fff700 1 -- 172.21.2.111:0/2167532139 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f3898029d40 con 0x7f38980065c0 2021-04-03T03:08:58.539 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:58.539 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1066299897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1066299897},{"type":"v1","addr":"172.21.2.115:6833","nonce":1066299897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:08:44.422766+0000","modified":"2021-04-03T03:08:45.425728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2149096292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2149096292},{"type":"v1","addr":"172.21.2.111:6835","nonce":2149096292}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-04-03T03:08:58.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3884056f20 msgr2=0x7f38840593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3884056f20 0x7f38840593e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3898046070 tx=0x7f3890015000).stop 2021-04-03T03:08:58.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 msgr2=0x7f389802d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:58.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f389802d550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f3898028af0 tx=0x7f3888006d50).stop 2021-04-03T03:08:58.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 shutdown_connections 2021-04-03T03:08:58.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389800bca0 0x7f389802a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38980065c0 0x7f389802d550 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3884056f20 0x7f38840593e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:58.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 --2- 172.21.2.111:0/2167532139 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38980079a0 0x7f38980e6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:08:58.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 >> 172.21.2.111:0/2167532139 conn(0x7f38980484d0 msgr2=0x7f3898047110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:58.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 shutdown_connections 2021-04-03T03:08:58.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:58.531+0000 7f389fde7700 1 -- 172.21.2.111:0/2167532139 wait complete. 2021-04-03T03:08:58.543 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 5 2021-04-03T03:08:58.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:08:44.422766+0000', 'modified': '2021-04-03T03:08:45.425728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/2149096292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 2149096292}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 2149096292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:08:58.748 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:08:58.748 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:08:58.823 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":4239,"want_state":"up:active","state":"up:active","rank_uptime":14.389482618000001,"mdsmap_epoch":5,"osdmap_epoch":24,"osdmap_epoch_barrier":23,"uptime":19.607668044} 2021-04-03T03:08:58.823 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 4239, "mdsmap_epoch": 5, "osdmap_epoch": 24, "osdmap_epoch_barrier": 23, "rank_uptime": 14.389482618, "state": "up:active", "uptime": 19.607668044, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:08:58.824 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:08:58.824 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2021-04-03T03:08:58.824 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@gibba031.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2021-04-03T03:08:58.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:58.825 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:08:58.836 DEBUG:teuthology.orchestra.run.gibba031:> sudo modprobe fuse 2021-04-03T03:08:58.886 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:08:58.887 DEBUG:teuthology.orchestra.run.gibba031:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:08:58.934 INFO:teuthology.orchestra.run.gibba031.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:08:58.935 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:08:58.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:58.936 DEBUG:teuthology.orchestra.run.gibba031:> ls /sys/fs/fuse/connections 2021-04-03T03:08:58.988 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:08:58.988 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:08:59.034 DEBUG:teuthology.orchestra.run.gibba031:> sudo modprobe fuse 2021-04-03T03:08:59.042 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:08:59.042 DEBUG:teuthology.orchestra.run.gibba031:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:08:59.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.gibba031.stderr:2021-04-03T03:08:59.080+0000 7fb9bd0500c0 -1 init, newargv = 0x5569d8518b70 newargc=9 2021-04-03T03:08:59.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.gibba031.stderr:ceph-fuse[14749]: starting ceph client 2021-04-03T03:08:59.087 INFO:teuthology.orchestra.run.gibba031.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:08:59.088 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:08:59.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.089 DEBUG:teuthology.orchestra.run.gibba031:> ls /sys/fs/fuse/connections 2021-04-03T03:08:59.093 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.gibba031.stderr:ceph-fuse[14749]: starting fuse 2021-04-03T03:08:59.096 INFO:teuthology.orchestra.run.gibba031.stdout:49 2021-04-03T03:08:59.096 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:08:59.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.097 DEBUG:teuthology.orchestra.run.gibba031:> sudo python3 -c ' 2021-04-03T03:08:59.097 DEBUG:teuthology.orchestra.run.gibba031:> import glob 2021-04-03T03:08:59.098 DEBUG:teuthology.orchestra.run.gibba031:> import re 2021-04-03T03:08:59.098 DEBUG:teuthology.orchestra.run.gibba031:> import os 2021-04-03T03:08:59.098 DEBUG:teuthology.orchestra.run.gibba031:> import subprocess 2021-04-03T03:08:59.099 DEBUG:teuthology.orchestra.run.gibba031:> 2021-04-03T03:08:59.099 DEBUG:teuthology.orchestra.run.gibba031:> def find_socket(client_name): 2021-04-03T03:08:59.099 DEBUG:teuthology.orchestra.run.gibba031:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2021-04-03T03:08:59.100 DEBUG:teuthology.orchestra.run.gibba031:> files = glob.glob(asok_path) 2021-04-03T03:08:59.100 DEBUG:teuthology.orchestra.run.gibba031:> 2021-04-03T03:08:59.100 DEBUG:teuthology.orchestra.run.gibba031:> # Given a non-glob path, it better be there 2021-04-03T03:08:59.101 DEBUG:teuthology.orchestra.run.gibba031:> if "*" not in asok_path: 2021-04-03T03:08:59.101 DEBUG:teuthology.orchestra.run.gibba031:> assert(len(files) == 1) 2021-04-03T03:08:59.101 DEBUG:teuthology.orchestra.run.gibba031:> return files[0] 2021-04-03T03:08:59.102 DEBUG:teuthology.orchestra.run.gibba031:> 2021-04-03T03:08:59.102 DEBUG:teuthology.orchestra.run.gibba031:> for f in files: 2021-04-03T03:08:59.102 DEBUG:teuthology.orchestra.run.gibba031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:08:59.102 DEBUG:teuthology.orchestra.run.gibba031:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:08:59.103 DEBUG:teuthology.orchestra.run.gibba031:> return f 2021-04-03T03:08:59.103 DEBUG:teuthology.orchestra.run.gibba031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:08:59.103 DEBUG:teuthology.orchestra.run.gibba031:> 2021-04-03T03:08:59.104 DEBUG:teuthology.orchestra.run.gibba031:> print(find_socket("client.2")) 2021-04-03T03:08:59.104 DEBUG:teuthology.orchestra.run.gibba031:> ' 2021-04-03T03:08:59.159 INFO:teuthology.orchestra.run.gibba031.stdout:/var/run/ceph/ceph-client.2.14749.asok 2021-04-03T03:08:59.162 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.14749.asok 2021-04-03T03:08:59.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.163 DEBUG:teuthology.orchestra.run.gibba031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.14749.asok status 2021-04-03T03:08:59.397 INFO:teuthology.orchestra.run.gibba031.stdout:{ 2021-04-03T03:08:59.397 INFO:teuthology.orchestra.run.gibba031.stdout: "metadata": { 2021-04-03T03:08:59.398 INFO:teuthology.orchestra.run.gibba031.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:08:59.398 INFO:teuthology.orchestra.run.gibba031.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:08:59.398 INFO:teuthology.orchestra.run.gibba031.stdout: "entity_id": "2", 2021-04-03T03:08:59.398 INFO:teuthology.orchestra.run.gibba031.stdout: "hostname": "gibba031", 2021-04-03T03:08:59.399 INFO:teuthology.orchestra.run.gibba031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2021-04-03T03:08:59.399 INFO:teuthology.orchestra.run.gibba031.stdout: "pid": "14749", 2021-04-03T03:08:59.399 INFO:teuthology.orchestra.run.gibba031.stdout: "root": "/" 2021-04-03T03:08:59.399 INFO:teuthology.orchestra.run.gibba031.stdout: }, 2021-04-03T03:08:59.400 INFO:teuthology.orchestra.run.gibba031.stdout: "dentry_count": 0, 2021-04-03T03:08:59.400 INFO:teuthology.orchestra.run.gibba031.stdout: "dentry_pinned_count": 0, 2021-04-03T03:08:59.400 INFO:teuthology.orchestra.run.gibba031.stdout: "id": 4665, 2021-04-03T03:08:59.400 INFO:teuthology.orchestra.run.gibba031.stdout: "inst": { 2021-04-03T03:08:59.400 INFO:teuthology.orchestra.run.gibba031.stdout: "name": { 2021-04-03T03:08:59.401 INFO:teuthology.orchestra.run.gibba031.stdout: "type": "client", 2021-04-03T03:08:59.401 INFO:teuthology.orchestra.run.gibba031.stdout: "num": 4665 2021-04-03T03:08:59.401 INFO:teuthology.orchestra.run.gibba031.stdout: }, 2021-04-03T03:08:59.401 INFO:teuthology.orchestra.run.gibba031.stdout: "addr": { 2021-04-03T03:08:59.401 INFO:teuthology.orchestra.run.gibba031.stdout: "type": "v1", 2021-04-03T03:08:59.402 INFO:teuthology.orchestra.run.gibba031.stdout: "addr": "172.21.2.131:0", 2021-04-03T03:08:59.402 INFO:teuthology.orchestra.run.gibba031.stdout: "nonce": 2141835278 2021-04-03T03:08:59.402 INFO:teuthology.orchestra.run.gibba031.stdout: } 2021-04-03T03:08:59.402 INFO:teuthology.orchestra.run.gibba031.stdout: }, 2021-04-03T03:08:59.402 INFO:teuthology.orchestra.run.gibba031.stdout: "addr": { 2021-04-03T03:08:59.403 INFO:teuthology.orchestra.run.gibba031.stdout: "type": "v1", 2021-04-03T03:08:59.403 INFO:teuthology.orchestra.run.gibba031.stdout: "addr": "172.21.2.131:0", 2021-04-03T03:08:59.403 INFO:teuthology.orchestra.run.gibba031.stdout: "nonce": 2141835278 2021-04-03T03:08:59.403 INFO:teuthology.orchestra.run.gibba031.stdout: }, 2021-04-03T03:08:59.404 INFO:teuthology.orchestra.run.gibba031.stdout: "inst_str": "client.4665 172.21.2.131:0/2141835278", 2021-04-03T03:08:59.404 INFO:teuthology.orchestra.run.gibba031.stdout: "addr_str": "172.21.2.131:0/2141835278", 2021-04-03T03:08:59.404 INFO:teuthology.orchestra.run.gibba031.stdout: "inode_count": 1, 2021-04-03T03:08:59.404 INFO:teuthology.orchestra.run.gibba031.stdout: "mds_epoch": 5, 2021-04-03T03:08:59.404 INFO:teuthology.orchestra.run.gibba031.stdout: "osd_epoch": 24, 2021-04-03T03:08:59.404 INFO:teuthology.orchestra.run.gibba031.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:08:59.405 INFO:teuthology.orchestra.run.gibba031.stdout: "blacklisted": false, 2021-04-03T03:08:59.405 INFO:teuthology.orchestra.run.gibba031.stdout: "fs_name": "cephfs" 2021-04-03T03:08:59.405 INFO:teuthology.orchestra.run.gibba031.stdout:} 2021-04-03T03:08:59.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.407 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:08:59.407 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:08:59.418 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:08:59.419 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:08:59.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.420 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:08:59.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.479 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:08:59.479 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:08:59.490 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:08:59.491 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:08:59.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.492 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:08:59.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.560 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:08:59.560 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:08:59.566 INFO:teuthology.orchestra.run.gibba031.stdout:fuseblk 2021-04-03T03:08:59.566 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2021-04-03T03:08:59.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:08:59.567 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:08:59.629 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2021-04-03T03:08:59.656 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:08:59.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5950f4700 1 Processor -- start 2021-04-03T03:08:59.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5950f4700 1 -- start start 2021-04-03T03:08:59.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5950f4700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa588007e90 0x7fa588028df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5950f4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 0x7fa588043cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5950f4700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa588029cd0 con 0x7fa588041d70 2021-04-03T03:08:59.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5950f4700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa588008270 con 0x7fa588007e90 2021-04-03T03:08:59.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5950f4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880286b0 con 0x7fa588026960 2021-04-03T03:08:59.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5948f3700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa588041d70 0x7fa5880d4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50932/0 (socket says 172.21.2.111:50932) 2021-04-03T03:08:59.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa5948f3700 1 -- 172.21.2.111:0/3524031784 learned_addr learned my addr 172.21.2.111:0/3524031784 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:08:59.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937531756 0 0) 0x7fa588029cd0 con 0x7fa588041d70 2021-04-03T03:08:59.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa574003240 con 0x7fa588041d70 2021-04-03T03:08:59.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58f7fe700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 0x7fa588043cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1913805093 0 0) 0x7fa574003240 con 0x7fa588041d70 2021-04-03T03:08:59.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 msgr2=0x7fa588043cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:59.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 0x7fa588043cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 >> v2:172.21.2.115:3300/0 conn(0x7fa588007e90 msgr2=0x7fa588028df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:59.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58ffff700 1 --2- 172.21.2.111:0/3524031784 >> v2:172.21.2.115:3300/0 conn(0x7fa588007e90 0x7fa588028df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 --2- 172.21.2.111:0/3524031784 >> v2:172.21.2.115:3300/0 conn(0x7fa588007e90 0x7fa588028df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588028950 con 0x7fa588041d70 2021-04-03T03:08:59.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa5900450f0 con 0x7fa588041d70 2021-04-03T03:08:59.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 >> v1:172.21.2.111:6789/0 conn(0x7fa588041d70 legacy=0x7fa5880d4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5740037b0 0x7fa574003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 0x7fa5740044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5740049d0 0x7fa574004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa588008270 con 0x7fa5740037b0 2021-04-03T03:08:59.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880286b0 con 0x7fa5740049d0 2021-04-03T03:08:59.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.707+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740041b0 con 0x7fa588026960 2021-04-03T03:08:59.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 --2- 172.21.2.111:0/3524031784 >> v2:172.21.2.115:3300/0 conn(0x7fa588007e90 0x7fa588028df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:59.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5740037b0 0x7fa574003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58f7fe700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 0x7fa5740044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5740049d0 0x7fa574004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 -- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 msgr2=0x7fa5740044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:59.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa588026960 0x7fa5740044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 -- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5740049d0 msgr2=0x7fa574004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:59.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5740049d0 0x7fa574004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 -- 172.21.2.111:0/3524031784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa574003240 con 0x7fa5740037b0 2021-04-03T03:08:59.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5740037b0 0x7fa574003bd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fa57c007000 tx=0x7fa57c008040).ready entity=mon.0 client_cookie=ff3a369f10cb408b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:59.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa57c015070 con 0x7fa5740037b0 2021-04-03T03:08:59.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c00c8a0 con 0x7fa5740037b0 2021-04-03T03:08:59.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58effd700 1 -- 172.21.2.111:0/3524031784 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa57c00cc00 con 0x7fa5740037b0 2021-04-03T03:08:59.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/3524031784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5740037b0 msgr2=0x7fa574003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 --2- 172.21.2.111:0/3524031784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5740037b0 0x7fa574003bd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fa57c007000 tx=0x7fa57c008040).stop 2021-04-03T03:08:59.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 -- 172.21.2.111:0/3524031784 reap_dead start 2021-04-03T03:08:59.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/3524031784 shutdown_connections 2021-04-03T03:08:59.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/3524031784 >> 172.21.2.111:0/3524031784 conn(0x7fa58800e220 msgr2=0x7fa588007460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:59.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/3524031784 shutdown_connections 2021-04-03T03:08:59.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/3524031784 wait complete. 2021-04-03T03:08:59.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 Processor -- start 2021-04-03T03:08:59.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- start start 2021-04-03T03:08:59.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa58803f360 0x7fa588040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5880cb540 0x7fa5880cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5880cd870 0x7fa5880cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58803f740 con 0x7fa5880cd870 2021-04-03T03:08:59.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58803e6d0 con 0x7fa5880cb540 2021-04-03T03:08:59.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58803d570 con 0x7fa58803f360 2021-04-03T03:08:59.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5880cd870 0x7fa5880cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5880cd870 0x7fa5880cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50150/0 (socket says 172.21.2.111:50150) 2021-04-03T03:08:59.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 -- 172.21.2.111:0/2974238189 learned_addr learned my addr 172.21.2.111:0/2974238189 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:59.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58f7fe700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5880cb540 0x7fa5880cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 -- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa58803f360 msgr2=0x7fa588040510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58ffff700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa58803f360 0x7fa588040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa58803f360 0x7fa588040510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 -- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5880cb540 msgr2=0x7fa5880cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5880cb540 0x7fa5880cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 -- 172.21.2.111:0/2974238189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880436b0 con 0x7fa5880cd870 2021-04-03T03:08:59.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58f7fe700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5880cb540 0x7fa5880cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:59.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5948f3700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5880cd870 0x7fa5880cdcf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa59005aec0 tx=0x7fa588028ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:59.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58d7fa700 1 -- 172.21.2.111:0/2974238189 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5900699f0 con 0x7fa5880cd870 2021-04-03T03:08:59.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58d7fa700 1 -- 172.21.2.111:0/2974238189 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59006a730 con 0x7fa5880cd870 2021-04-03T03:08:59.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58803da10 con 0x7fa5880cd870 2021-04-03T03:08:59.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58d7fa700 1 -- 172.21.2.111:0/2974238189 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa59006a8a0 con 0x7fa5880cd870 2021-04-03T03:08:59.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5880cc040 con 0x7fa5880cd870 2021-04-03T03:08:59.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa588004da0 con 0x7fa5880cd870 2021-04-03T03:08:59.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.711+0000 7fa58d7fa700 1 -- 172.21.2.111:0/2974238189 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa59005f9c0 con 0x7fa5880cd870 2021-04-03T03:08:59.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.715+0000 7fa58d7fa700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578056ea0 0x7fa578059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.715+0000 7fa58d7fa700 1 -- 172.21.2.111:0/2974238189 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fa5900de6f0 con 0x7fa5880cd870 2021-04-03T03:08:59.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.715+0000 7fa58ffff700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578056ea0 0x7fa578059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.715+0000 7fa58ffff700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578056ea0 0x7fa578059360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fa57c002000 tx=0x7fa57c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:59.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.715+0000 7fa58d7fa700 1 -- 172.21.2.111:0/2974238189 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa5900b3a70 con 0x7fa5880cd870 2021-04-03T03:08:59.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.807+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa588009500 con 0x7fa5880cd870 2021-04-03T03:08:59.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa58d7fa700 1 -- 172.21.2.111:0/2974238189 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fa5900441b0 con 0x7fa5880cd870 2021-04-03T03:08:59.818 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:08:59.818 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:08:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578056ea0 msgr2=0x7fa578059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578056ea0 0x7fa578059360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fa57c002000 tx=0x7fa57c008040).stop 2021-04-03T03:08:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5880cd870 msgr2=0x7fa5880cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5880cd870 0x7fa5880cdcf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa59005aec0 tx=0x7fa588028ae0).stop 2021-04-03T03:08:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 shutdown_connections 2021-04-03T03:08:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5880cd870 0x7fa5880cdcf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa58803f360 0x7fa588040510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578056ea0 0x7fa578059360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 --2- 172.21.2.111:0/2974238189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5880cb540 0x7fa5880cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 >> 172.21.2.111:0/2974238189 conn(0x7fa58800e220 msgr2=0x7fa588046110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 shutdown_connections 2021-04-03T03:08:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.811+0000 7fa5950f4700 1 -- 172.21.2.111:0/2974238189 wait complete. 2021-04-03T03:08:59.825 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:08:59.826 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:08:59.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.875+0000 7f0fbe963700 1 Processor -- start 2021-04-03T03:08:59.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.875+0000 7f0fbe963700 1 -- start start 2021-04-03T03:08:59.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.875+0000 7f0fbe963700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb0041060 0x7f0fb00dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.875+0000 7f0fbe963700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0fb003d560 0x7f0fb0041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.875+0000 7f0fbe963700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb00dc560 con 0x7f0fb00dbba0 2021-04-03T03:08:59.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.875+0000 7f0fbe963700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb003de60 con 0x7f0fb003d560 2021-04-03T03:08:59.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.875+0000 7f0fbe963700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00dc800 con 0x7f0fb0041060 2021-04-03T03:08:59.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe162700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0fb003d560 0x7f0fb0041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbd160700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb0041060 0x7f0fb00dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbd160700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb0041060 0x7f0fb00dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:33508/0 (socket says 172.21.2.111:33508) 2021-04-03T03:08:59.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe162700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0fb003d560 0x7f0fb0041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:39866/0 (socket says 172.21.2.111:39866) 2021-04-03T03:08:59.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbd160700 1 -- 172.21.2.111:0/1047778387 learned_addr learned my addr 172.21.2.111:0/1047778387 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:59.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe162700 1 -- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb0041060 msgr2=0x7f0fb00dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:08:59.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe162700 1 --2- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb0041060 0x7f0fb00dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe162700 1 -- 172.21.2.111:0/1047778387 >> v1:172.21.2.115:6789/0 conn(0x7f0fb00dbba0 legacy=0x7f0fb003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe162700 1 -- 172.21.2.111:0/1047778387 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb003da30 con 0x7f0fb003d560 2021-04-03T03:08:59.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbd160700 1 --2- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb0041060 0x7f0fb00dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:08:59.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe162700 1 --2- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0041650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0fb40010d0 tx=0x7f0fb805ad70).ready entity=mon.1 client_cookie=f5e86022fe61817e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:59.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbc95f700 1 -- 172.21.2.111:0/1047778387 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb805bda0 con 0x7f0fb003d560 2021-04-03T03:08:59.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbc95f700 1 -- 172.21.2.111:0/1047778387 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb805df60 con 0x7f0fb003d560 2021-04-03T03:08:59.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbc95f700 1 -- 172.21.2.111:0/1047778387 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb80699f0 con 0x7f0fb003d560 2021-04-03T03:08:59.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 msgr2=0x7f0fb0041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 --2- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0041650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0fb40010d0 tx=0x7f0fb805ad70).stop 2021-04-03T03:08:59.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- 172.21.2.111:0/1047778387 shutdown_connections 2021-04-03T03:08:59.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 --2- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0041650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 --2- 172.21.2.111:0/1047778387 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb0041060 0x7f0fb00dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- 172.21.2.111:0/1047778387 >> 172.21.2.111:0/1047778387 conn(0x7f0fb00462b0 msgr2=0x7f0fb00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:59.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- 172.21.2.111:0/1047778387 shutdown_connections 2021-04-03T03:08:59.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- 172.21.2.111:0/1047778387 wait complete. 2021-04-03T03:08:59.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 Processor -- start 2021-04-03T03:08:59.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- start start 2021-04-03T03:08:59.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0045660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0fb0041060 0x7f0fb0039580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb00dbba0 0x7f0fb009fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0039a70 con 0x7f0fb0041060 2021-04-03T03:08:59.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0039d00 con 0x7f0fb003d560 2021-04-03T03:08:59.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbe963700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb003a0d0 con 0x7f0fb00dbba0 2021-04-03T03:08:59.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbd160700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0fb0041060 0x7f0fb0039580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbd160700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0fb0041060 0x7f0fb0039580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50162/0 (socket says 172.21.2.111:50162) 2021-04-03T03:08:59.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.879+0000 7f0fbd160700 1 -- 172.21.2.111:0/2500734575 learned_addr learned my addr 172.21.2.111:0/2500734575 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:08:59.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbe162700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb00dbba0 0x7f0fb009fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd961700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0045660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd160700 1 -- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb00dbba0 msgr2=0x7f0fb009fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd160700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb00dbba0 0x7f0fb009fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd160700 1 -- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 msgr2=0x7f0fb0045660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd160700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0045660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd160700 1 -- 172.21.2.111:0/2500734575 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb003a370 con 0x7f0fb0041060 2021-04-03T03:08:59.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbe162700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb00dbba0 0x7f0fb009fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:59.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd961700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0045660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:08:59.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd160700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0fb0041060 0x7f0fb0039580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f0fac00c000 tx=0x7f0fac006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:59.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0faaffd700 1 -- 172.21.2.111:0/2500734575 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fac00ca00 con 0x7f0fb0041060 2021-04-03T03:08:59.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb003a810 con 0x7f0fb0041060 2021-04-03T03:08:59.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0faaffd700 1 -- 172.21.2.111:0/2500734575 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac00cb60 con 0x7f0fb0041060 2021-04-03T03:08:59.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0faaffd700 1 -- 172.21.2.111:0/2500734575 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fac0123f0 con 0x7f0fb0041060 2021-04-03T03:08:59.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb00df8f0 con 0x7f0fb0041060 2021-04-03T03:08:59.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb003a500 con 0x7f0fb0041060 2021-04-03T03:08:59.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0faaffd700 1 -- 172.21.2.111:0/2500734575 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0fac012550 con 0x7f0fb0041060 2021-04-03T03:08:59.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0faaffd700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f98056fb0 0x7f0f98059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:08:59.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0faaffd700 1 -- 172.21.2.111:0/2500734575 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0fac070960 con 0x7f0fb0041060 2021-04-03T03:08:59.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd961700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f98056fb0 0x7f0f98059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:08:59.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.883+0000 7f0fbd961700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f98056fb0 0x7f0f98059470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0fb4008000 tx=0x7f0fb4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:08:59.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.887+0000 7f0faaffd700 1 -- 172.21.2.111:0/2500734575 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0fac045c40 con 0x7f0fb0041060 2021-04-03T03:08:59.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0fb0004db0 con 0x7f0fb0041060 2021-04-03T03:08:59.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0faaffd700 1 -- 172.21.2.111:0/2500734575 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0fac049260 con 0x7f0fb0041060 2021-04-03T03:08:59.988 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:08:59.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f98056fb0 msgr2=0x7f0f98059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f98056fb0 0x7f0f98059470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0fb4008000 tx=0x7f0fb4009040).stop 2021-04-03T03:08:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0fb0041060 msgr2=0x7f0fb0039580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:08:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0fb0041060 0x7f0fb0039580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f0fac00c000 tx=0x7f0fac006e90).stop 2021-04-03T03:08:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 shutdown_connections 2021-04-03T03:08:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0fb00dbba0 0x7f0fb009fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f98056fb0 0x7f0f98059470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0fb003d560 0x7f0fb0045660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 --2- 172.21.2.111:0/2500734575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0fb0041060 0x7f0fb0039580 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:08:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 >> 172.21.2.111:0/2500734575 conn(0x7f0fb00462b0 msgr2=0x7f0fb0006120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:08:59.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 shutdown_connections 2021-04-03T03:08:59.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:08:59.979+0000 7f0fbe963700 1 -- 172.21.2.111:0/2500734575 wait complete. 2021-04-03T03:08:59.995 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:09:00.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 Processor -- start 2021-04-03T03:09:00.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 -- start start 2021-04-03T03:09:00.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fea380603f0 0x7fea381447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380add50 0x7fea38144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fea380b17b0 0x7fea38147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea381451c0 con 0x7fea380b17b0 2021-04-03T03:09:00.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea380607d0 con 0x7fea380603f0 2021-04-03T03:09:00.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3814b090 con 0x7fea380add50 2021-04-03T03:09:00.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fea380b17b0 0x7fea38147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fea380b17b0 0x7fea38147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50174/0 (socket says 172.21.2.111:50174) 2021-04-03T03:09:00.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 -- 172.21.2.111:0/2753533876 learned_addr learned my addr 172.21.2.111:0/2753533876 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 -- 172.21.2.111:0/2753533876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380add50 msgr2=0x7fea38144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 --2- 172.21.2.111:0/2753533876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380add50 0x7fea38144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 -- 172.21.2.111:0/2753533876 >> v2:172.21.2.115:3300/0 conn(0x7fea380603f0 msgr2=0x7fea381447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea377fe700 1 --2- 172.21.2.111:0/2753533876 >> v2:172.21.2.115:3300/0 conn(0x7fea380603f0 0x7fea381447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 --2- 172.21.2.111:0/2753533876 >> v2:172.21.2.115:3300/0 conn(0x7fea380603f0 0x7fea381447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 -- 172.21.2.111:0/2753533876 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3814b330 con 0x7fea380b17b0 2021-04-03T03:09:00.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea377fe700 1 --2- 172.21.2.111:0/2753533876 >> v2:172.21.2.115:3300/0 conn(0x7fea380603f0 0x7fea381447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:00.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea37fff700 1 --2- 172.21.2.111:0/2753533876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380b17b0 0x7fea38147740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fea300032b0 tx=0x7fea30009270).ready entity=mon.0 client_cookie=eab3821eb37cfb32 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea367fc700 1 -- 172.21.2.111:0/2753533876 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea3000bc00 con 0x7fea380b17b0 2021-04-03T03:09:00.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea367fc700 1 -- 172.21.2.111:0/2753533876 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3000bd60 con 0x7fea380b17b0 2021-04-03T03:09:00.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea367fc700 1 -- 172.21.2.111:0/2753533876 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea30011400 con 0x7fea380b17b0 2021-04-03T03:09:00.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 -- 172.21.2.111:0/2753533876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380b17b0 msgr2=0x7fea38147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.043+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/2753533876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380b17b0 0x7fea38147740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fea300032b0 tx=0x7fea30009270).stop 2021-04-03T03:09:00.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- 172.21.2.111:0/2753533876 shutdown_connections 2021-04-03T03:09:00.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/2753533876 >> v2:172.21.2.115:3300/0 conn(0x7fea380603f0 0x7fea381447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/2753533876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380b17b0 0x7fea38147740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/2753533876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380add50 0x7fea38144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- 172.21.2.111:0/2753533876 >> 172.21.2.111:0/2753533876 conn(0x7fea380b61c0 msgr2=0x7fea3805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- 172.21.2.111:0/2753533876 shutdown_connections 2021-04-03T03:09:00.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- 172.21.2.111:0/2753533876 wait complete. 2021-04-03T03:09:00.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 Processor -- start 2021-04-03T03:09:00.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- start start 2021-04-03T03:09:00.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea380603f0 0x7fea380ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380add50 0x7fea380aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380b17b0 0x7fea38063390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38063880 con 0x7fea380add50 2021-04-03T03:09:00.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38064cd0 con 0x7fea380603f0 2021-04-03T03:09:00.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea38064f70 con 0x7fea380b17b0 2021-04-03T03:09:00.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380add50 0x7fea380aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380add50 0x7fea380aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50176/0 (socket says 172.21.2.111:50176) 2021-04-03T03:09:00.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 -- 172.21.2.111:0/1959612454 learned_addr learned my addr 172.21.2.111:0/1959612454 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 -- 172.21.2.111:0/1959612454 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380b17b0 msgr2=0x7fea38063390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380b17b0 0x7fea38063390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 -- 172.21.2.111:0/1959612454 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea380603f0 msgr2=0x7fea380ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea380603f0 0x7fea380ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 -- 172.21.2.111:0/1959612454 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380abf10 con 0x7fea380add50 2021-04-03T03:09:00.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea36ffd700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380add50 0x7fea380aaae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fea28002f30 tx=0x7fea28006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea34ff9700 1 -- 172.21.2.111:0/1959612454 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea2800d710 con 0x7fea380add50 2021-04-03T03:09:00.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea34ff9700 1 -- 172.21.2.111:0/1959612454 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28000e90 con 0x7fea380add50 2021-04-03T03:09:00.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea38065410 con 0x7fea380add50 2021-04-03T03:09:00.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea380658b0 con 0x7fea380add50 2021-04-03T03:09:00.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea34ff9700 1 -- 172.21.2.111:0/1959612454 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea2800da60 con 0x7fea380add50 2021-04-03T03:09:00.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea38065130 con 0x7fea380add50 2021-04-03T03:09:00.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea34ff9700 1 -- 172.21.2.111:0/1959612454 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fea2800dbc0 con 0x7fea380add50 2021-04-03T03:09:00.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea34ff9700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea18057000 0x7fea180594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea34ff9700 1 -- 172.21.2.111:0/1959612454 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fea2806fe30 con 0x7fea380add50 2021-04-03T03:09:00.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.047+0000 7fea377fe700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea18057000 0x7fea180594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.051+0000 7fea377fe700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea18057000 0x7fea180594c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fea2c007000 tx=0x7fea2c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.051+0000 7fea34ff9700 1 -- 172.21.2.111:0/1959612454 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fea28045210 con 0x7fea380add50 2021-04-03T03:09:00.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea38057660 con 0x7fea380add50 2021-04-03T03:09:00.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea34ff9700 1 -- 172.21.2.111:0/1959612454 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea28048830 con 0x7fea380add50 2021-04-03T03:09:00.155 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:00.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea18057000 msgr2=0x7fea180594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea18057000 0x7fea180594c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fea2c007000 tx=0x7fea2c008040).stop 2021-04-03T03:09:00.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380add50 msgr2=0x7fea380aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380add50 0x7fea380aaae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fea28002f30 tx=0x7fea28006e70).stop 2021-04-03T03:09:00.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 shutdown_connections 2021-04-03T03:09:00.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea380b17b0 0x7fea38063390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea18057000 0x7fea180594c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea380603f0 0x7fea380ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 --2- 172.21.2.111:0/1959612454 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea380add50 0x7fea380aaae0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 >> 172.21.2.111:0/1959612454 conn(0x7fea380b61c0 msgr2=0x7fea380b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 shutdown_connections 2021-04-03T03:09:00.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.147+0000 7fea3cfa7700 1 -- 172.21.2.111:0/1959612454 wait complete. 2021-04-03T03:09:00.163 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:09:00.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfed04700 1 Processor -- start 2021-04-03T03:09:00.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfed04700 1 -- start start 2021-04-03T03:09:00.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfed04700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 0x7f5bf8147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfed04700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5bf8144ef0 0x7f5bf8147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfed04700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf80607d0 con 0x7f5bf80603f0 2021-04-03T03:09:00.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfed04700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf814b090 con 0x7f5bf8144ef0 2021-04-03T03:09:00.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfed04700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf814b330 con 0x7f5bf80add50 2021-04-03T03:09:00.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfe503700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5bf8144ef0 0x7f5bf8147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfdd02700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5bf80603f0 0x7f5bf81447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50976/0 (socket says 172.21.2.111:50976) 2021-04-03T03:09:00.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.211+0000 7f5bfdd02700 1 -- 172.21.2.111:0/3976191746 learned_addr learned my addr 172.21.2.111:0/3976191746 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:00.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 -- 172.21.2.111:0/3976191746 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 msgr2=0x7f5bf8147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:09:00.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 --2- 172.21.2.111:0/3976191746 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 0x7f5bf8147740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 -- 172.21.2.111:0/3976191746 >> v1:172.21.2.111:6789/0 conn(0x7f5bf80603f0 legacy=0x7f5bf81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 -- 172.21.2.111:0/3976191746 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf814b5d0 con 0x7f5bf8144ef0 2021-04-03T03:09:00.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 --2- 172.21.2.111:0/3976191746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8147c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5bf00032b0 tx=0x7f5bf00091d0).ready entity=mon.0 client_cookie=a73f006f9571e779 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfcd00700 1 -- 172.21.2.111:0/3976191746 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bf000ba50 con 0x7f5bf8144ef0 2021-04-03T03:09:00.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfcd00700 1 -- 172.21.2.111:0/3976191746 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf000bbb0 con 0x7f5bf8144ef0 2021-04-03T03:09:00.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfcd00700 1 -- 172.21.2.111:0/3976191746 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bf000bd20 con 0x7f5bf8144ef0 2021-04-03T03:09:00.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/3976191746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 msgr2=0x7f5bf8147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 --2- 172.21.2.111:0/3976191746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8147c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5bf00032b0 tx=0x7f5bf00091d0).stop 2021-04-03T03:09:00.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/3976191746 shutdown_connections 2021-04-03T03:09:00.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 --2- 172.21.2.111:0/3976191746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8147c30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 --2- 172.21.2.111:0/3976191746 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 0x7f5bf8147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/3976191746 >> 172.21.2.111:0/3976191746 conn(0x7f5bf80b61c0 msgr2=0x7f5bf805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/3976191746 shutdown_connections 2021-04-03T03:09:00.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/3976191746 wait complete. 2021-04-03T03:09:00.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 Processor -- start 2021-04-03T03:09:00.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- start start 2021-04-03T03:09:00.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf80603f0 0x7f5bf8156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 0x7f5bf80a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8063cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8157030 con 0x7f5bf8144ef0 2021-04-03T03:09:00.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8157250 con 0x7f5bf80603f0 2021-04-03T03:09:00.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf81574f0 con 0x7f5bf80add50 2021-04-03T03:09:00.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8063cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8063cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50190/0 (socket says 172.21.2.111:50190) 2021-04-03T03:09:00.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 -- 172.21.2.111:0/2696974396 learned_addr learned my addr 172.21.2.111:0/2696974396 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 -- 172.21.2.111:0/2696974396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 msgr2=0x7f5bf80a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 0x7f5bf80a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 -- 172.21.2.111:0/2696974396 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf80603f0 msgr2=0x7f5bf8156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf80603f0 0x7f5bf8156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 -- 172.21.2.111:0/2696974396 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf8157790 con 0x7f5bf8144ef0 2021-04-03T03:09:00.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfe503700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8063cd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f5bf00032b0 tx=0x7f5bf0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5be6ffd700 1 -- 172.21.2.111:0/2696974396 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bf00205f0 con 0x7f5bf8144ef0 2021-04-03T03:09:00.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5be6ffd700 1 -- 172.21.2.111:0/2696974396 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf0020750 con 0x7f5bf8144ef0 2021-04-03T03:09:00.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf80a9d60 con 0x7f5bf8144ef0 2021-04-03T03:09:00.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5be6ffd700 1 -- 172.21.2.111:0/2696974396 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bf0020ab0 con 0x7f5bf8144ef0 2021-04-03T03:09:00.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf80643d0 con 0x7f5bf8144ef0 2021-04-03T03:09:00.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.215+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf8157920 con 0x7f5bf8144ef0 2021-04-03T03:09:00.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.219+0000 7f5be6ffd700 1 -- 172.21.2.111:0/2696974396 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5bf0027030 con 0x7f5bf8144ef0 2021-04-03T03:09:00.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.219+0000 7f5be6ffd700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056ef0 0x7f5bd80593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.219+0000 7f5be6ffd700 1 -- 172.21.2.111:0/2696974396 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5bf0071910 con 0x7f5bf8144ef0 2021-04-03T03:09:00.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.219+0000 7f5bfdd02700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056ef0 0x7f5bd80593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.219+0000 7f5bfdd02700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056ef0 0x7f5bd80593b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5bf80b5600 tx=0x7f5be80077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.219+0000 7f5be6ffd700 1 -- 172.21.2.111:0/2696974396 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5bf0046be0 con 0x7f5bf8144ef0 2021-04-03T03:09:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5bf8057660 con 0x7f5bf8144ef0 2021-04-03T03:09:00.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5be6ffd700 1 -- 172.21.2.111:0/2696974396 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5bf004a200 con 0x7f5bf8144ef0 2021-04-03T03:09:00.323 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:00.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056ef0 msgr2=0x7f5bd80593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056ef0 0x7f5bd80593b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5bf80b5600 tx=0x7f5be80077e0).stop 2021-04-03T03:09:00.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 msgr2=0x7f5bf8063cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8063cd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f5bf00032b0 tx=0x7f5bf0016000).stop 2021-04-03T03:09:00.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 shutdown_connections 2021-04-03T03:09:00.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf8144ef0 0x7f5bf8063cd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056ef0 0x7f5bd80593b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf80603f0 0x7f5bf8156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 --2- 172.21.2.111:0/2696974396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf80add50 0x7f5bf80a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 >> 172.21.2.111:0/2696974396 conn(0x7f5bf80b61c0 msgr2=0x7f5bf805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 shutdown_connections 2021-04-03T03:09:00.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.315+0000 7f5bfed04700 1 -- 172.21.2.111:0/2696974396 wait complete. 2021-04-03T03:09:00.331 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:09:00.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.379+0000 7f0481708700 1 Processor -- start 2021-04-03T03:09:00.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.379+0000 7f0481708700 1 -- start start 2021-04-03T03:09:00.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f047c144ef0 0x7f047c147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0607d0 con 0x7f047c0603f0 2021-04-03T03:09:00.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c14b090 con 0x7f047c0add50 2021-04-03T03:09:00.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047c14b330 con 0x7f047c144ef0 2021-04-03T03:09:00.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f047c0603f0 0x7f047c1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:50990/0 (socket says 172.21.2.111:50990) 2021-04-03T03:09:00.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 -- 172.21.2.111:0/1000427438 learned_addr learned my addr 172.21.2.111:0/1000427438 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:00.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0480f07700 1 --2- 172.21.2.111:0/1000427438 >> v2:172.21.2.111:3300/0 conn(0x7f047c144ef0 0x7f047c147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047affd700 1 -- 172.21.2.111:0/1000427438 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670147257 0 0) 0x7f047c0607d0 con 0x7f047c0603f0 2021-04-03T03:09:00.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047affd700 1 -- 172.21.2.111:0/1000427438 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f045c003240 con 0x7f047c0603f0 2021-04-03T03:09:00.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0480f07700 1 -- 172.21.2.111:0/1000427438 >> v1:172.21.2.115:6789/0 conn(0x7f047c0add50 legacy=0x7f047c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0480f07700 1 -- 172.21.2.111:0/1000427438 >> v1:172.21.2.111:6789/0 conn(0x7f047c0603f0 legacy=0x7f047c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0480f07700 1 -- 172.21.2.111:0/1000427438 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c14b5d0 con 0x7f047c144ef0 2021-04-03T03:09:00.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0480f07700 1 --2- 172.21.2.111:0/1000427438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c144ef0 0x7f047c147e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f047c0b5600 tx=0x7f04740091d0).ready entity=mon.0 client_cookie=8a87a1a272ed3a1d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047affd700 1 -- 172.21.2.111:0/1000427438 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047400b960 con 0x7f047c144ef0 2021-04-03T03:09:00.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047affd700 1 -- 172.21.2.111:0/1000427438 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047400bac0 con 0x7f047c144ef0 2021-04-03T03:09:00.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047affd700 1 -- 172.21.2.111:0/1000427438 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047400bc30 con 0x7f047c144ef0 2021-04-03T03:09:00.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- 172.21.2.111:0/1000427438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c144ef0 msgr2=0x7f047c147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 --2- 172.21.2.111:0/1000427438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c144ef0 0x7f047c147e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f047c0b5600 tx=0x7f04740091d0).stop 2021-04-03T03:09:00.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- 172.21.2.111:0/1000427438 shutdown_connections 2021-04-03T03:09:00.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 --2- 172.21.2.111:0/1000427438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c144ef0 0x7f047c147e50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- 172.21.2.111:0/1000427438 >> 172.21.2.111:0/1000427438 conn(0x7f047c0b61c0 msgr2=0x7f047c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- 172.21.2.111:0/1000427438 shutdown_connections 2021-04-03T03:09:00.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- 172.21.2.111:0/1000427438 wait complete. 2021-04-03T03:09:00.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 Processor -- start 2021-04-03T03:09:00.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- start start 2021-04-03T03:09:00.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c0603f0 0x7f047c15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f047c0add50 0x7f047c15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f047c144ef0 0x7f047c0a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0a9880 con 0x7f047c0603f0 2021-04-03T03:09:00.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0a9b20 con 0x7f047c144ef0 2021-04-03T03:09:00.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0a9dc0 con 0x7f047c0add50 2021-04-03T03:09:00.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c0603f0 0x7f047c15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c0603f0 0x7f047c15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50204/0 (socket says 172.21.2.111:50204) 2021-04-03T03:09:00.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 -- 172.21.2.111:0/2815529494 learned_addr learned my addr 172.21.2.111:0/2815529494 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 -- 172.21.2.111:0/2815529494 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f047c0add50 msgr2=0x7f047c15a780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:00.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f047c0add50 0x7f047c15a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 -- 172.21.2.111:0/2815529494 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f047c144ef0 msgr2=0x7f047c0a7b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:00.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f047c144ef0 0x7f047c0a7b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 -- 172.21.2.111:0/2815529494 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0a88b0 con 0x7f047c0603f0 2021-04-03T03:09:00.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f047bfff700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c0603f0 0x7f047c15a290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f04640010d0 tx=0x7f0464007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f04797fa700 1 -- 172.21.2.111:0/2815529494 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0464007320 con 0x7f047c0603f0 2021-04-03T03:09:00.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f04797fa700 1 -- 172.21.2.111:0/2815529494 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0464007480 con 0x7f047c0603f0 2021-04-03T03:09:00.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c0a9400 con 0x7f047c0603f0 2021-04-03T03:09:00.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f04797fa700 1 -- 172.21.2.111:0/2815529494 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04640057e0 con 0x7f047c0603f0 2021-04-03T03:09:00.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.383+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c0aa340 con 0x7f047c0603f0 2021-04-03T03:09:00.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.387+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c0a9590 con 0x7f047c0603f0 2021-04-03T03:09:00.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.387+0000 7f04797fa700 1 -- 172.21.2.111:0/2815529494 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0464010030 con 0x7f047c0603f0 2021-04-03T03:09:00.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.387+0000 7f04797fa700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0460056fb0 0x7f0460059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.387+0000 7f04797fa700 1 -- 172.21.2.111:0/2815529494 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0464070090 con 0x7f047c0603f0 2021-04-03T03:09:00.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.387+0000 7f047b7fe700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0460056fb0 0x7f0460059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.387+0000 7f047b7fe700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0460056fb0 0x7f0460059470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f046c002f50 tx=0x7f046c006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.387+0000 7f04797fa700 1 -- 172.21.2.111:0/2815529494 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0464045280 con 0x7f047c0603f0 2021-04-03T03:09:00.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.483+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f047c057660 con 0x7f047c0603f0 2021-04-03T03:09:00.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.483+0000 7f04797fa700 1 -- 172.21.2.111:0/2815529494 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f04640488a0 con 0x7f047c0603f0 2021-04-03T03:09:00.493 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:00.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0460056fb0 msgr2=0x7f0460059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0460056fb0 0x7f0460059470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f046c002f50 tx=0x7f046c006910).stop 2021-04-03T03:09:00.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c0603f0 msgr2=0x7f047c15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c0603f0 0x7f047c15a290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f04640010d0 tx=0x7f0464007ab0).stop 2021-04-03T03:09:00.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 shutdown_connections 2021-04-03T03:09:00.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0460056fb0 0x7f0460059470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f047c144ef0 0x7f047c0a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f047c0603f0 0x7f047c15a290 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 --2- 172.21.2.111:0/2815529494 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f047c0add50 0x7f047c15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 >> 172.21.2.111:0/2815529494 conn(0x7f047c0b61c0 msgr2=0x7f047c0b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 shutdown_connections 2021-04-03T03:09:00.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.487+0000 7f0481708700 1 -- 172.21.2.111:0/2815529494 wait complete. 2021-04-03T03:09:00.500 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:00.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 Processor -- start 2021-04-03T03:09:00.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 -- start start 2021-04-03T03:09:00.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb40c05f9c0 0x7fb40c05fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c1439c0 0x7fb40c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb40c0b1b70 0x7fb40c0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c0b04e0 con 0x7fb40c05f9c0 2021-04-03T03:09:00.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c143e60 con 0x7fb40c0b1b70 2021-04-03T03:09:00.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb412393700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0afda0 con 0x7fb40c1439c0 2021-04-03T03:09:00.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb411391700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb40c05f9c0 0x7fb40c05fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb411391700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb40c05f9c0 0x7fb40c05fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50212/0 (socket says 172.21.2.111:50212) 2021-04-03T03:09:00.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.547+0000 7fb411391700 1 -- 172.21.2.111:0/3546685441 learned_addr learned my addr 172.21.2.111:0/3546685441 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411391700 1 -- 172.21.2.111:0/3546685441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c1439c0 msgr2=0x7fb40c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411391700 1 --2- 172.21.2.111:0/3546685441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c1439c0 0x7fb40c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411391700 1 -- 172.21.2.111:0/3546685441 >> v2:172.21.2.115:3300/0 conn(0x7fb40c0b1b70 msgr2=0x7fb40c0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411391700 1 --2- 172.21.2.111:0/3546685441 >> v2:172.21.2.115:3300/0 conn(0x7fb40c0b1b70 0x7fb40c0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411391700 1 -- 172.21.2.111:0/3546685441 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0b0040 con 0x7fb40c05f9c0 2021-04-03T03:09:00.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411391700 1 --2- 172.21.2.111:0/3546685441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c05f9c0 0x7fb40c05fda0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb40c0afcc0 tx=0x7fb400006e90).ready entity=mon.0 client_cookie=c6b7548ec146b903 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fffff700 1 -- 172.21.2.111:0/3546685441 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4000098e0 con 0x7fb40c05f9c0 2021-04-03T03:09:00.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fffff700 1 -- 172.21.2.111:0/3546685441 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400009a40 con 0x7fb40c05f9c0 2021-04-03T03:09:00.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fffff700 1 -- 172.21.2.111:0/3546685441 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb400009da0 con 0x7fb40c05f9c0 2021-04-03T03:09:00.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3546685441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c05f9c0 msgr2=0x7fb40c05fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 --2- 172.21.2.111:0/3546685441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c05f9c0 0x7fb40c05fda0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb40c0afcc0 tx=0x7fb400006e90).stop 2021-04-03T03:09:00.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3546685441 shutdown_connections 2021-04-03T03:09:00.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 --2- 172.21.2.111:0/3546685441 >> v2:172.21.2.115:3300/0 conn(0x7fb40c0b1b70 0x7fb40c0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 --2- 172.21.2.111:0/3546685441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c05f9c0 0x7fb40c05fda0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 --2- 172.21.2.111:0/3546685441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c1439c0 0x7fb40c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3546685441 >> 172.21.2.111:0/3546685441 conn(0x7fb40c0b5fb0 msgr2=0x7fb40c0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3546685441 shutdown_connections 2021-04-03T03:09:00.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3546685441 wait complete. 2021-04-03T03:09:00.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 Processor -- start 2021-04-03T03:09:00.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- start start 2021-04-03T03:09:00.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c05f9c0 0x7fb40c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0b1b70 0x7fb40c0669e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c1439c0 0x7fb40c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0a1ea0 con 0x7fb40c1439c0 2021-04-03T03:09:00.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0a04b0 con 0x7fb40c05f9c0 2021-04-03T03:09:00.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0a1550 con 0x7fb40c0b1b70 2021-04-03T03:09:00.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c1439c0 0x7fb40c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c1439c0 0x7fb40c0a2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50222/0 (socket says 172.21.2.111:50222) 2021-04-03T03:09:00.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 -- 172.21.2.111:0/3946230083 learned_addr learned my addr 172.21.2.111:0/3946230083 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 -- 172.21.2.111:0/3946230083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0b1b70 msgr2=0x7fb40c0669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0b1b70 0x7fb40c0669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 -- 172.21.2.111:0/3946230083 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c05f9c0 msgr2=0x7fb40c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c05f9c0 0x7fb40c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 -- 172.21.2.111:0/3946230083 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c09f2f0 con 0x7fb40c1439c0 2021-04-03T03:09:00.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411b92700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c1439c0 0x7fb40c0a2cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb4040032b0 tx=0x7fb404009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fe7fc700 1 -- 172.21.2.111:0/3946230083 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40400ca60 con 0x7fb40c1439c0 2021-04-03T03:09:00.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fe7fc700 1 -- 172.21.2.111:0/3946230083 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40400cbc0 con 0x7fb40c1439c0 2021-04-03T03:09:00.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c09f820 con 0x7fb40c1439c0 2021-04-03T03:09:00.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fe7fc700 1 -- 172.21.2.111:0/3946230083 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb404015420 con 0x7fb40c1439c0 2021-04-03T03:09:00.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c09f0a0 con 0x7fb40c1439c0 2021-04-03T03:09:00.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c09f5d0 con 0x7fb40c1439c0 2021-04-03T03:09:00.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fe7fc700 1 -- 172.21.2.111:0/3946230083 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb40401c070 con 0x7fb40c1439c0 2021-04-03T03:09:00.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fe7fc700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec0570a0 0x7fb3ec059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb3fe7fc700 1 -- 172.21.2.111:0/3946230083 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb40406fd60 con 0x7fb40c1439c0 2021-04-03T03:09:00.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.551+0000 7fb411391700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec0570a0 0x7fb3ec059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.555+0000 7fb411391700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec0570a0 0x7fb3ec059560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fb40c0afcc0 tx=0x7fb400015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.555+0000 7fb3fe7fc700 1 -- 172.21.2.111:0/3946230083 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb404045140 con 0x7fb40c1439c0 2021-04-03T03:09:00.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.647+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb40c057660 con 0x7fb40c1439c0 2021-04-03T03:09:00.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.647+0000 7fb3fe7fc700 1 -- 172.21.2.111:0/3946230083 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fb404048760 con 0x7fb40c1439c0 2021-04-03T03:09:00.657 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:00.657 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec0570a0 msgr2=0x7fb3ec059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec0570a0 0x7fb3ec059560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fb40c0afcc0 tx=0x7fb400015000).stop 2021-04-03T03:09:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c1439c0 msgr2=0x7fb40c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c1439c0 0x7fb40c0a2cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb4040032b0 tx=0x7fb404009220).stop 2021-04-03T03:09:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 shutdown_connections 2021-04-03T03:09:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c1439c0 0x7fb40c0a2cf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec0570a0 0x7fb3ec059560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c05f9c0 0x7fb40c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 --2- 172.21.2.111:0/3946230083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0b1b70 0x7fb40c0669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 >> 172.21.2.111:0/3946230083 conn(0x7fb40c0b5fb0 msgr2=0x7fb40c0ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 shutdown_connections 2021-04-03T03:09:00.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.651+0000 7fb412393700 1 -- 172.21.2.111:0/3946230083 wait complete. 2021-04-03T03:09:00.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:09:00.665 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:09:00.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 Processor -- start 2021-04-03T03:09:00.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 -- start start 2021-04-03T03:09:00.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f4684147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4684144ef0 0x7f4684147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46840607d0 con 0x7f46840603f0 2021-04-03T03:09:00.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468414b090 con 0x7f4684144ef0 2021-04-03T03:09:00.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468414b330 con 0x7f46840add50 2021-04-03T03:09:00.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468a7e1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4684144ef0 0x7f4684147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f4684147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f4684147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:33574/0 (socket says 172.21.2.111:33574) 2021-04-03T03:09:00.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468a7e1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4684144ef0 0x7f4684147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:39934/0 (socket says 172.21.2.111:39934) 2021-04-03T03:09:00.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 -- 172.21.2.111:0/3948484256 learned_addr learned my addr 172.21.2.111:0/3948484256 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 -- 172.21.2.111:0/3948484256 >> v2:172.21.2.115:3300/0 conn(0x7f4684144ef0 msgr2=0x7f4684147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 --2- 172.21.2.111:0/3948484256 >> v2:172.21.2.115:3300/0 conn(0x7f4684144ef0 0x7f4684147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 -- 172.21.2.111:0/3948484256 >> v1:172.21.2.115:6789/0 conn(0x7f46840603f0 legacy=0x7f46841447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 -- 172.21.2.111:0/3948484256 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468414b5d0 con 0x7f46840add50 2021-04-03T03:09:00.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f46897df700 1 --2- 172.21.2.111:0/3948484256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f4684147740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4678002f30 tx=0x7f4678006e90).ready entity=mon.2 client_cookie=e6d64d7d639d1c0c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f4688fde700 1 -- 172.21.2.111:0/3948484256 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4678009850 con 0x7f46840add50 2021-04-03T03:09:00.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f4688fde700 1 -- 172.21.2.111:0/3948484256 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46780099b0 con 0x7f46840add50 2021-04-03T03:09:00.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f4688fde700 1 -- 172.21.2.111:0/3948484256 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4678009d10 con 0x7f46840add50 2021-04-03T03:09:00.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 -- 172.21.2.111:0/3948484256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 msgr2=0x7f4684147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 --2- 172.21.2.111:0/3948484256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f4684147740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4678002f30 tx=0x7f4678006e90).stop 2021-04-03T03:09:00.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 -- 172.21.2.111:0/3948484256 shutdown_connections 2021-04-03T03:09:00.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 --2- 172.21.2.111:0/3948484256 >> v2:172.21.2.115:3300/0 conn(0x7f4684144ef0 0x7f4684147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 --2- 172.21.2.111:0/3948484256 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f4684147740 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.715+0000 7f468afe2700 1 -- 172.21.2.111:0/3948484256 >> 172.21.2.111:0/3948484256 conn(0x7f46840b61c0 msgr2=0x7f468405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- 172.21.2.111:0/3948484256 shutdown_connections 2021-04-03T03:09:00.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- 172.21.2.111:0/3948484256 wait complete. 2021-04-03T03:09:00.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 Processor -- start 2021-04-03T03:09:00.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- start start 2021-04-03T03:09:00.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46840603f0 0x7f46840b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f46840674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4684144ef0 0x7f46840679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840aa5d0 con 0x7f4684144ef0 2021-04-03T03:09:00.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840ab320 con 0x7f46840603f0 2021-04-03T03:09:00.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46840aa330 con 0x7f46840add50 2021-04-03T03:09:00.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4684144ef0 0x7f46840679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4684144ef0 0x7f46840679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50232/0 (socket says 172.21.2.111:50232) 2021-04-03T03:09:00.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 -- 172.21.2.111:0/2464578519 learned_addr learned my addr 172.21.2.111:0/2464578519 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 -- 172.21.2.111:0/2464578519 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 msgr2=0x7f46840674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f46840674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 -- 172.21.2.111:0/2464578519 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46840603f0 msgr2=0x7f46840b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46840603f0 0x7f46840b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 -- 172.21.2.111:0/2464578519 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468415a420 con 0x7f4684144ef0 2021-04-03T03:09:00.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468a7e1700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4684144ef0 0x7f46840679b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f467c00c000 tx=0x7f467c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4676ffd700 1 -- 172.21.2.111:0/2464578519 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467c00ca00 con 0x7f4684144ef0 2021-04-03T03:09:00.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4676ffd700 1 -- 172.21.2.111:0/2464578519 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c00cb60 con 0x7f4684144ef0 2021-04-03T03:09:00.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46840a8320 con 0x7f4684144ef0 2021-04-03T03:09:00.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4676ffd700 1 -- 172.21.2.111:0/2464578519 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467c0123f0 con 0x7f4684144ef0 2021-04-03T03:09:00.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4684067ea0 con 0x7f4684144ef0 2021-04-03T03:09:00.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4684057660 con 0x7f4684144ef0 2021-04-03T03:09:00.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4676ffd700 1 -- 172.21.2.111:0/2464578519 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f467c022070 con 0x7f4684144ef0 2021-04-03T03:09:00.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4676ffd700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4664057050 0x7f4664059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4676ffd700 1 -- 172.21.2.111:0/2464578519 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f467c070880 con 0x7f4684144ef0 2021-04-03T03:09:00.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4689fe0700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4664057050 0x7f4664059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.719+0000 7f4689fe0700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4664057050 0x7f4664059510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f46840b5600 tx=0x7f46800077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.723+0000 7f4676ffd700 1 -- 172.21.2.111:0/2464578519 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f467c045c60 con 0x7f4684144ef0 2021-04-03T03:09:00.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.815+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f468406b370 con 0x7f4684144ef0 2021-04-03T03:09:00.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.815+0000 7f4676ffd700 1 -- 172.21.2.111:0/2464578519 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f467c016020 con 0x7f4684144ef0 2021-04-03T03:09:00.824 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:09:00.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.815+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4664057050 msgr2=0x7f4664059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.815+0000 7f468afe2700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4664057050 0x7f4664059510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f46840b5600 tx=0x7f46800077f0).stop 2021-04-03T03:09:00.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.815+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4684144ef0 msgr2=0x7f46840679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.815+0000 7f468afe2700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4684144ef0 0x7f46840679b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f467c00c000 tx=0x7f467c0091d0).stop 2021-04-03T03:09:00.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 shutdown_connections 2021-04-03T03:09:00.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4684144ef0 0x7f46840679b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4664057050 0x7f4664059510 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46840603f0 0x7f46840b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 --2- 172.21.2.111:0/2464578519 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46840add50 0x7f46840674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 >> 172.21.2.111:0/2464578519 conn(0x7f46840b61c0 msgr2=0x7f46840b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 shutdown_connections 2021-04-03T03:09:00.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.819+0000 7f468afe2700 1 -- 172.21.2.111:0/2464578519 wait complete. 2021-04-03T03:09:00.832 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:09:00.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 Processor -- start 2021-04-03T03:09:00.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- start start 2021-04-03T03:09:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0190041060 0x7f01900dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f019003d560 0x7f0190041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01900dc560 con 0x7f01900dbba0 2021-04-03T03:09:00.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019003de60 con 0x7f0190041060 2021-04-03T03:09:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900dc800 con 0x7f019003d560 2021-04-03T03:09:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0190041060 0x7f01900dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0190041060 0x7f01900dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50242/0 (socket says 172.21.2.111:50242) 2021-04-03T03:09:00.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 -- 172.21.2.111:0/235727876 learned_addr learned my addr 172.21.2.111:0/235727876 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 -- 172.21.2.111:0/235727876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f019003d560 msgr2=0x7f0190041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 --2- 172.21.2.111:0/235727876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f019003d560 0x7f0190041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 -- 172.21.2.111:0/235727876 >> v1:172.21.2.115:6789/0 conn(0x7f01900dbba0 legacy=0x7f019003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 -- 172.21.2.111:0/235727876 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019003da30 con 0x7f0190041060 2021-04-03T03:09:00.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01965e4700 1 --2- 172.21.2.111:0/235727876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0190041060 0x7f01900dbf80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f01900087d0 tx=0x7f0180006e90).ready entity=mon.0 client_cookie=456a5fa8ffb46fa7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0195de3700 1 -- 172.21.2.111:0/235727876 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01800098e0 con 0x7f0190041060 2021-04-03T03:09:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0195de3700 1 -- 172.21.2.111:0/235727876 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0180009a40 con 0x7f0190041060 2021-04-03T03:09:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0195de3700 1 -- 172.21.2.111:0/235727876 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0180009de0 con 0x7f0190041060 2021-04-03T03:09:00.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- 172.21.2.111:0/235727876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0190041060 msgr2=0x7f01900dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- 172.21.2.111:0/235727876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0190041060 0x7f01900dbf80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f01900087d0 tx=0x7f0180006e90).stop 2021-04-03T03:09:00.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- 172.21.2.111:0/235727876 shutdown_connections 2021-04-03T03:09:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- 172.21.2.111:0/235727876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f019003d560 0x7f0190041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- 172.21.2.111:0/235727876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0190041060 0x7f01900dbf80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- 172.21.2.111:0/235727876 >> 172.21.2.111:0/235727876 conn(0x7f01900462b0 msgr2=0x7f01900466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- 172.21.2.111:0/235727876 shutdown_connections 2021-04-03T03:09:00.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- 172.21.2.111:0/235727876 wait complete. 2021-04-03T03:09:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 Processor -- start 2021-04-03T03:09:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- start start 2021-04-03T03:09:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f019003d560 0x7f01900dca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0190041060 0x7f01900d6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01900dbba0 0x7f01900d6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900df710 con 0x7f01900dbba0 2021-04-03T03:09:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900d79c0 con 0x7f019003d560 2021-04-03T03:09:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900a0930 con 0x7f0190041060 2021-04-03T03:09:00.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01900dbba0 0x7f01900d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01900dbba0 0x7f01900d6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50248/0 (socket says 172.21.2.111:50248) 2021-04-03T03:09:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 -- 172.21.2.111:0/2542878767 learned_addr learned my addr 172.21.2.111:0/2542878767 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 -- 172.21.2.111:0/2542878767 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0190041060 msgr2=0x7f01900d6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0190041060 0x7f01900d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 -- 172.21.2.111:0/2542878767 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f019003d560 msgr2=0x7f01900dca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f019003d560 0x7f01900dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 -- 172.21.2.111:0/2542878767 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900a0bd0 con 0x7f01900dbba0 2021-04-03T03:09:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f01975e6700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01900dbba0 0x7f01900d6850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f01900df510 tx=0x7f019805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f018ffff700 1 -- 172.21.2.111:0/2542878767 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f019805f2c0 con 0x7f01900dbba0 2021-04-03T03:09:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f018ffff700 1 -- 172.21.2.111:0/2542878767 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019805f420 con 0x7f01900dbba0 2021-04-03T03:09:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01900d7640 con 0x7f01900dbba0 2021-04-03T03:09:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01900d7c30 con 0x7f01900dbba0 2021-04-03T03:09:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.883+0000 7f018ffff700 1 -- 172.21.2.111:0/2542878767 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f019805f780 con 0x7f01900dbba0 2021-04-03T03:09:00.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.887+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0190004db0 con 0x7f01900dbba0 2021-04-03T03:09:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.887+0000 7f018ffff700 1 -- 172.21.2.111:0/2542878767 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f019808b020 con 0x7f01900dbba0 2021-04-03T03:09:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.887+0000 7f018ffff700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f017c056ea0 0x7f017c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.887+0000 7f018ffff700 1 -- 172.21.2.111:0/2542878767 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f01980df2f0 con 0x7f01900dbba0 2021-04-03T03:09:00.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.887+0000 7f0196de5700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f017c056ea0 0x7f017c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.887+0000 7f0196de5700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f017c056ea0 0x7f017c059360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0188002ed0 tx=0x7f0188006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:00.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.887+0000 7f018ffff700 1 -- 172.21.2.111:0/2542878767 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f01980b45c0 con 0x7f01900dbba0 2021-04-03T03:09:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f01900083c0 con 0x7f01900dbba0 2021-04-03T03:09:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f018ffff700 1 -- 172.21.2.111:0/2542878767 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f01980b7be0 con 0x7f01900dbba0 2021-04-03T03:09:00.990 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:00.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f017c056ea0 msgr2=0x7f017c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f017c056ea0 0x7f017c059360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0188002ed0 tx=0x7f0188006910).stop 2021-04-03T03:09:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01900dbba0 msgr2=0x7f01900d6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01900dbba0 0x7f01900d6850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f01900df510 tx=0x7f019805ad70).stop 2021-04-03T03:09:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 shutdown_connections 2021-04-03T03:09:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01900dbba0 0x7f01900d6850 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f017c056ea0 0x7f017c059360 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f019003d560 0x7f01900dca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 --2- 172.21.2.111:0/2542878767 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0190041060 0x7f01900d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 >> 172.21.2.111:0/2542878767 conn(0x7f01900462b0 msgr2=0x7f0190041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 shutdown_connections 2021-04-03T03:09:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:00.983+0000 7f0197de7700 1 -- 172.21.2.111:0/2542878767 wait complete. 2021-04-03T03:09:00.998 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:09:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 Processor -- start 2021-04-03T03:09:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- start start 2021-04-03T03:09:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0bc80dbba0 0x7f0bc80dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc803d560 0x7f0bc8041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc80dc560 con 0x7f0bc8041060 2021-04-03T03:09:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc803de60 con 0x7f0bc80dbba0 2021-04-03T03:09:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc80dc800 con 0x7f0bc803d560 2021-04-03T03:09:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0bc80dbba0 0x7f0bc80dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0bc80dbba0 0x7f0bc80dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50252/0 (socket says 172.21.2.111:50252) 2021-04-03T03:09:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 -- 172.21.2.111:0/3370453052 learned_addr learned my addr 172.21.2.111:0/3370453052 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 -- 172.21.2.111:0/3370453052 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc803d560 msgr2=0x7f0bc8041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 --2- 172.21.2.111:0/3370453052 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc803d560 0x7f0bc8041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 -- 172.21.2.111:0/3370453052 >> v1:172.21.2.111:6789/0 conn(0x7f0bc8041060 legacy=0x7f0bc803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 -- 172.21.2.111:0/3370453052 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc803da30 con 0x7f0bc80dbba0 2021-04-03T03:09:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd49a0700 1 --2- 172.21.2.111:0/3370453052 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc80dbba0 0x7f0bc80dbf80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0bbc002f30 tx=0x7f0bbc006e90).ready entity=mon.0 client_cookie=8e40ea6256b6856f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bcf7fe700 1 -- 172.21.2.111:0/3370453052 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bbc009970 con 0x7f0bc80dbba0 2021-04-03T03:09:01.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bcf7fe700 1 -- 172.21.2.111:0/3370453052 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc009ad0 con 0x7f0bc80dbba0 2021-04-03T03:09:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bcf7fe700 1 -- 172.21.2.111:0/3370453052 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bbc009e30 con 0x7f0bc80dbba0 2021-04-03T03:09:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- 172.21.2.111:0/3370453052 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc80dbba0 msgr2=0x7f0bc80dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/3370453052 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc80dbba0 0x7f0bc80dbf80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0bbc002f30 tx=0x7f0bbc006e90).stop 2021-04-03T03:09:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- 172.21.2.111:0/3370453052 shutdown_connections 2021-04-03T03:09:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/3370453052 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc803d560 0x7f0bc8041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/3370453052 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc80dbba0 0x7f0bc80dbf80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- 172.21.2.111:0/3370453052 >> 172.21.2.111:0/3370453052 conn(0x7f0bc80462b0 msgr2=0x7f0bc80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- 172.21.2.111:0/3370453052 shutdown_connections 2021-04-03T03:09:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.047+0000 7f0bd59a2700 1 -- 172.21.2.111:0/3370453052 wait complete. 2021-04-03T03:09:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 Processor -- start 2021-04-03T03:09:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 -- start start 2021-04-03T03:09:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc803d560 0x7f0bc803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bc8041060 0x7f0bc80d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc80dbba0 0x7f0bc80d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc809f4a0 con 0x7f0bc803d560 2021-04-03T03:09:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc80d2280 con 0x7f0bc8041060 2021-04-03T03:09:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc80dde20 con 0x7f0bc80dbba0 2021-04-03T03:09:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc803d560 0x7f0bc803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc803d560 0x7f0bc803ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50260/0 (socket says 172.21.2.111:50260) 2021-04-03T03:09:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 -- 172.21.2.111:0/4221122311 learned_addr learned my addr 172.21.2.111:0/4221122311 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 -- 172.21.2.111:0/4221122311 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc80dbba0 msgr2=0x7f0bc80d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc80dbba0 0x7f0bc80d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 -- 172.21.2.111:0/4221122311 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bc8041060 msgr2=0x7f0bc80d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bc8041060 0x7f0bc80d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 -- 172.21.2.111:0/4221122311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc809f6d0 con 0x7f0bc803d560 2021-04-03T03:09:01.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd49a0700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc803d560 0x7f0bc803ad40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f0bbc002f30 tx=0x7f0bbc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcdffb700 1 -- 172.21.2.111:0/4221122311 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bbc011cc0 con 0x7f0bc803d560 2021-04-03T03:09:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcdffb700 1 -- 172.21.2.111:0/4221122311 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc011e20 con 0x7f0bc803d560 2021-04-03T03:09:01.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcdffb700 1 -- 172.21.2.111:0/4221122311 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bbc01e650 con 0x7f0bc803d560 2021-04-03T03:09:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc80df320 con 0x7f0bc803d560 2021-04-03T03:09:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc80a30d0 con 0x7f0bc803d560 2021-04-03T03:09:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc809fe90 con 0x7f0bc803d560 2021-04-03T03:09:01.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcdffb700 1 -- 172.21.2.111:0/4221122311 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0bbc025030 con 0x7f0bc803d560 2021-04-03T03:09:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcdffb700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bb80570a0 0x7f0bb8059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcdffb700 1 -- 172.21.2.111:0/4221122311 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0bbc071720 con 0x7f0bc803d560 2021-04-03T03:09:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcffff700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bb80570a0 0x7f0bb8059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.051+0000 7f0bcffff700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bb80570a0 0x7f0bb8059560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f0bc80087d0 tx=0x7f0bc40077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.055+0000 7f0bcdffb700 1 -- 172.21.2.111:0/4221122311 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0bc809fe90 con 0x7f0bc803d560 2021-04-03T03:09:01.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0bc809fe90 con 0x7f0bc803d560 2021-04-03T03:09:01.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bcdffb700 1 -- 172.21.2.111:0/4221122311 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0bc809fe90 con 0x7f0bc803d560 2021-04-03T03:09:01.156 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:01.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bb80570a0 msgr2=0x7f0bb8059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bb80570a0 0x7f0bb8059560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f0bc80087d0 tx=0x7f0bc40077f0).stop 2021-04-03T03:09:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc803d560 msgr2=0x7f0bc803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc803d560 0x7f0bc803ad40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f0bbc002f30 tx=0x7f0bbc015000).stop 2021-04-03T03:09:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 shutdown_connections 2021-04-03T03:09:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bc80dbba0 0x7f0bc80d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bc803d560 0x7f0bc803ad40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.147+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bb80570a0 0x7f0bb8059560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.151+0000 7f0bd59a2700 1 --2- 172.21.2.111:0/4221122311 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bc8041060 0x7f0bc80d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.151+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 >> 172.21.2.111:0/4221122311 conn(0x7f0bc80462b0 msgr2=0x7f0bc8041b70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.151+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 shutdown_connections 2021-04-03T03:09:01.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.151+0000 7f0bd59a2700 1 -- 172.21.2.111:0/4221122311 wait complete. 2021-04-03T03:09:01.164 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:09:01.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.211+0000 7f5f020e1700 1 Processor -- start 2021-04-03T03:09:01.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.211+0000 7f5f020e1700 1 -- start start 2021-04-03T03:09:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5efc0b1990 0x7f5efc0ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc060110 con 0x7f5efc05fce0 2021-04-03T03:09:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0b1350 con 0x7f5efc0685b0 2021-04-03T03:09:01.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc0b2430 con 0x7f5efc0b1990 2021-04-03T03:09:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5efc05fce0 0x7f5efc0b0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51060/0 (socket says 172.21.2.111:51060) 2021-04-03T03:09:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 -- 172.21.2.111:0/908214494 learned_addr learned my addr 172.21.2.111:0/908214494 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632314878 0 0) 0x7f5ef4001450 con 0x7f5efc05fce0 2021-04-03T03:09:01.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee4003240 con 0x7f5efc05fce0 2021-04-03T03:09:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f008de700 1 --2- 172.21.2.111:0/908214494 >> v2:172.21.2.115:3300/0 conn(0x7f5efc0b1990 0x7f5efc0ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2424287415 0 0) 0x7f5ee4003240 con 0x7f5efc05fce0 2021-04-03T03:09:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 >> v2:172.21.2.115:3300/0 conn(0x7f5efc0b1990 msgr2=0x7f5efc0ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 --2- 172.21.2.111:0/908214494 >> v2:172.21.2.115:3300/0 conn(0x7f5efc0b1990 0x7f5efc0ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 >> v1:172.21.2.115:6789/0 conn(0x7f5efc0685b0 legacy=0x7f5efc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0afd90 con 0x7f5efc05fce0 2021-04-03T03:09:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f5ef4001be0 con 0x7f5efc05fce0 2021-04-03T03:09:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 >> v1:172.21.2.111:6789/0 conn(0x7f5efc05fce0 legacy=0x7f5efc0b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee4003800 0x7f5ee4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee40044e0 0x7f5ee4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee4004e50 0x7f5ee400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0b2430 con 0x7f5ee4004e50 2021-04-03T03:09:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40053b0 con 0x7f5ee4003800 2021-04-03T03:09:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40100b0 con 0x7f5ee40044e0 2021-04-03T03:09:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee4004e50 0x7f5ee400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 -- 172.21.2.111:0/908214494 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee40044e0 msgr2=0x7f5ee4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee40044e0 0x7f5ee4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 -- 172.21.2.111:0/908214494 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee4003800 msgr2=0x7f5ee4003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee4003800 0x7f5ee4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 -- 172.21.2.111:0/908214494 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4003240 con 0x7f5ee4004e50 2021-04-03T03:09:01.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 -- 172.21.2.111:0/908214494 reap_dead start 2021-04-03T03:09:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f018e0700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee4004e50 0x7f5ee400b730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f5efc0b6a50 tx=0x7f5efc0afaf0).ready entity=mon.0 client_cookie=6859df25272c22fa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef4005c50 con 0x7f5ee4004e50 2021-04-03T03:09:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4005db0 con 0x7f5ee4004e50 2021-04-03T03:09:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5eeffff700 1 -- 172.21.2.111:0/908214494 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef4007300 con 0x7f5ee4004e50 2021-04-03T03:09:01.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- 172.21.2.111:0/908214494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee4004e50 msgr2=0x7f5ee400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee4004e50 0x7f5ee400b730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f5efc0b6a50 tx=0x7f5efc0afaf0).stop 2021-04-03T03:09:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- 172.21.2.111:0/908214494 shutdown_connections 2021-04-03T03:09:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 --2- 172.21.2.111:0/908214494 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee4004e50 0x7f5ee400b730 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- 172.21.2.111:0/908214494 >> 172.21.2.111:0/908214494 conn(0x7f5efc0b5b50 msgr2=0x7f5efc0b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- 172.21.2.111:0/908214494 shutdown_connections 2021-04-03T03:09:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- 172.21.2.111:0/908214494 wait complete. 2021-04-03T03:09:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 Processor -- start 2021-04-03T03:09:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.215+0000 7f5f020e1700 1 -- start start 2021-04-03T03:09:01.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee4004e50 0x7f5efc066e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc064700 0x7f5efc0651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc139c30 0x7f5efc13a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc13a5a0 con 0x7f5efc139c30 2021-04-03T03:09:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0674c0 con 0x7f5ee4004e50 2021-04-03T03:09:01.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc067760 con 0x7f5efc064700 2021-04-03T03:09:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc139c30 0x7f5efc13a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc139c30 0x7f5efc13a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50282/0 (socket says 172.21.2.111:50282) 2021-04-03T03:09:01.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 -- 172.21.2.111:0/1728758388 learned_addr learned my addr 172.21.2.111:0/1728758388 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 -- 172.21.2.111:0/1728758388 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc064700 msgr2=0x7f5efc0651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc064700 0x7f5efc0651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 -- 172.21.2.111:0/1728758388 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee4004e50 msgr2=0x7f5efc066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee4004e50 0x7f5efc066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 -- 172.21.2.111:0/1728758388 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc067a00 con 0x7f5efc139c30 2021-04-03T03:09:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f018e0700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc139c30 0x7f5efc13a0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f5efc0b6a50 tx=0x7f5ef4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5eee7fc700 1 -- 172.21.2.111:0/1728758388 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef4007900 con 0x7f5efc139c30 2021-04-03T03:09:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5eee7fc700 1 -- 172.21.2.111:0/1728758388 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef401f410 con 0x7f5efc139c30 2021-04-03T03:09:01.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5eee7fc700 1 -- 172.21.2.111:0/1728758388 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef401f770 con 0x7f5efc139c30 2021-04-03T03:09:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc159a20 con 0x7f5efc139c30 2021-04-03T03:09:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc159ec0 con 0x7f5efc139c30 2021-04-03T03:09:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc067b90 con 0x7f5efc139c30 2021-04-03T03:09:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5eee7fc700 1 -- 172.21.2.111:0/1728758388 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5ef4010030 con 0x7f5efc139c30 2021-04-03T03:09:01.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5eee7fc700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5edc056ea0 0x7f5edc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5eee7fc700 1 -- 172.21.2.111:0/1728758388 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5ef4018030 con 0x7f5efc139c30 2021-04-03T03:09:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f010df700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5edc056ea0 0x7f5edc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.219+0000 7f5f010df700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5edc056ea0 0x7f5edc059360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5ef8001f60 tx=0x7f5ef80077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.223+0000 7f5eee7fc700 1 -- 172.21.2.111:0/1728758388 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5ef4045ad0 con 0x7f5efc139c30 2021-04-03T03:09:01.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5efc060110 con 0x7f5efc139c30 2021-04-03T03:09:01.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5eee7fc700 1 -- 172.21.2.111:0/1728758388 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5efc060110 con 0x7f5efc139c30 2021-04-03T03:09:01.324 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5edc056ea0 msgr2=0x7f5edc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5edc056ea0 0x7f5edc059360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5ef8001f60 tx=0x7f5ef80077d0).stop 2021-04-03T03:09:01.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc139c30 msgr2=0x7f5efc13a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc139c30 0x7f5efc13a0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f5efc0b6a50 tx=0x7f5ef4014000).stop 2021-04-03T03:09:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 shutdown_connections 2021-04-03T03:09:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc139c30 0x7f5efc13a0b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5edc056ea0 0x7f5edc059360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee4004e50 0x7f5efc066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 --2- 172.21.2.111:0/1728758388 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc064700 0x7f5efc0651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 >> 172.21.2.111:0/1728758388 conn(0x7f5efc0b5b50 msgr2=0x7f5efc13fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 shutdown_connections 2021-04-03T03:09:01.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.315+0000 7f5f020e1700 1 -- 172.21.2.111:0/1728758388 wait complete. 2021-04-03T03:09:01.331 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:01.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 Processor -- start 2021-04-03T03:09:01.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 -- start start 2021-04-03T03:09:01.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd680603f0 0x7fbd680b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd680607d0 con 0x7fbd680b0c90 2021-04-03T03:09:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd680af400 con 0x7fbd6814b700 2021-04-03T03:09:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680af6a0 con 0x7fbd680603f0 2021-04-03T03:09:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd67fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fbd680b0c90 0x7fbd68147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51082/0 (socket says 172.21.2.111:51082) 2021-04-03T03:09:01.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd67fff700 1 -- 172.21.2.111:0/1162450908 learned_addr learned my addr 172.21.2.111:0/1162450908 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667923070 0 0) 0x7fbd680607d0 con 0x7fbd680b0c90 2021-04-03T03:09:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd48003240 con 0x7fbd680b0c90 2021-04-03T03:09:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd677fe700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd680603f0 0x7fbd680b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2560835277 0 0) 0x7fbd48003240 con 0x7fbd680b0c90 2021-04-03T03:09:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd680603f0 msgr2=0x7fbd680b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd680603f0 0x7fbd680b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 >> v1:172.21.2.115:6789/0 conn(0x7fbd6814b700 legacy=0x7fbd680b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd680afb40 con 0x7fbd680b0c90 2021-04-03T03:09:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fbd60001a50 con 0x7fbd680b0c90 2021-04-03T03:09:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 >> v1:172.21.2.111:6789/0 conn(0x7fbd680b0c90 legacy=0x7fbd68147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd680603f0 0x7fbd48003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd48003cc0 0x7fbd480044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbd480049d0 0x7fbd48004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680af6a0 con 0x7fbd48003cc0 2021-04-03T03:09:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48004250 con 0x7fbd480049d0 2021-04-03T03:09:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4800b730 con 0x7fbd680603f0 2021-04-03T03:09:01.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd66ffd700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd48003cc0 0x7fbd480044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd66ffd700 1 -- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd680603f0 msgr2=0x7fbd48003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd66ffd700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd680603f0 0x7fbd48003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd66ffd700 1 -- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbd480049d0 msgr2=0x7fbd48004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd66ffd700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbd480049d0 0x7fbd48004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd66ffd700 1 -- 172.21.2.111:0/1162450908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48003240 con 0x7fbd48003cc0 2021-04-03T03:09:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd66ffd700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd48003cc0 0x7fbd480044e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fbd680b5640 tx=0x7fbd58007b70).ready entity=mon.0 client_cookie=11347b83546051d0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd5800c600 con 0x7fbd48003cc0 2021-04-03T03:09:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5800c760 con 0x7fbd48003cc0 2021-04-03T03:09:01.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd667fc700 1 -- 172.21.2.111:0/1162450908 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd5800cac0 con 0x7fbd48003cc0 2021-04-03T03:09:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 -- 172.21.2.111:0/1162450908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd48003cc0 msgr2=0x7fbd480044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd6c869700 1 --2- 172.21.2.111:0/1162450908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd48003cc0 0x7fbd480044e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fbd680b5640 tx=0x7fbd58007b70).stop 2021-04-03T03:09:01.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.379+0000 7fbd67fff700 1 -- 172.21.2.111:0/1162450908 reap_dead start 2021-04-03T03:09:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- 172.21.2.111:0/1162450908 shutdown_connections 2021-04-03T03:09:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- 172.21.2.111:0/1162450908 >> 172.21.2.111:0/1162450908 conn(0x7fbd680b61c0 msgr2=0x7fbd6805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- 172.21.2.111:0/1162450908 shutdown_connections 2021-04-03T03:09:01.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- 172.21.2.111:0/1162450908 wait complete. 2021-04-03T03:09:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 Processor -- start 2021-04-03T03:09:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- start start 2021-04-03T03:09:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbd6814c720 0x7fbd681599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd68066210 0x7fbd680a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd680673d0 0x7fbd680a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680a7af0 con 0x7fbd680673d0 2021-04-03T03:09:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68066a30 con 0x7fbd6814c720 2021-04-03T03:09:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6814cb00 con 0x7fbd68066210 2021-04-03T03:09:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd680673d0 0x7fbd680a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd680673d0 0x7fbd680a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50300/0 (socket says 172.21.2.111:50300) 2021-04-03T03:09:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 -- 172.21.2.111:0/2404416349 learned_addr learned my addr 172.21.2.111:0/2404416349 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 -- 172.21.2.111:0/2404416349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd68066210 msgr2=0x7fbd680a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd68066210 0x7fbd680a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 -- 172.21.2.111:0/2404416349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbd6814c720 msgr2=0x7fbd681599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbd6814c720 0x7fbd681599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 -- 172.21.2.111:0/2404416349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68066790 con 0x7fbd680673d0 2021-04-03T03:09:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd67fff700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd680673d0 0x7fbd680a9ba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbd60001820 tx=0x7fbd680ae670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd64ff9700 1 -- 172.21.2.111:0/2404416349 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd60004b50 con 0x7fbd680673d0 2021-04-03T03:09:01.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd64ff9700 1 -- 172.21.2.111:0/2404416349 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60004cb0 con 0x7fbd680673d0 2021-04-03T03:09:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd680a9210 con 0x7fbd680673d0 2021-04-03T03:09:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd64ff9700 1 -- 172.21.2.111:0/2404416349 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd60007220 con 0x7fbd680673d0 2021-04-03T03:09:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd680ab350 con 0x7fbd680673d0 2021-04-03T03:09:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd680a93a0 con 0x7fbd680673d0 2021-04-03T03:09:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd64ff9700 1 -- 172.21.2.111:0/2404416349 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbd6001b060 con 0x7fbd680673d0 2021-04-03T03:09:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd64ff9700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd4c056ea0 0x7fbd4c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd64ff9700 1 -- 172.21.2.111:0/2404416349 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fbd6006fb50 con 0x7fbd680673d0 2021-04-03T03:09:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd677fe700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd4c056ea0 0x7fbd4c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.383+0000 7fbd677fe700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd4c056ea0 0x7fbd4c059360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbd5000a000 tx=0x7fbd50008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.387+0000 7fbd64ff9700 1 -- 172.21.2.111:0/2404416349 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbd60044f30 con 0x7fbd680673d0 2021-04-03T03:09:01.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd68057660 con 0x7fbd680673d0 2021-04-03T03:09:01.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd64ff9700 1 -- 172.21.2.111:0/2404416349 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fbd60048550 con 0x7fbd680673d0 2021-04-03T03:09:01.489 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:01.489 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:01.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd4c056ea0 msgr2=0x7fbd4c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd4c056ea0 0x7fbd4c059360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbd5000a000 tx=0x7fbd50008040).stop 2021-04-03T03:09:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd680673d0 msgr2=0x7fbd680a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd680673d0 0x7fbd680a9ba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbd60001820 tx=0x7fbd680ae670).stop 2021-04-03T03:09:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 shutdown_connections 2021-04-03T03:09:01.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbd680673d0 0x7fbd680a9ba0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd4c056ea0 0x7fbd4c059360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbd6814c720 0x7fbd681599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 --2- 172.21.2.111:0/2404416349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbd68066210 0x7fbd680a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 >> 172.21.2.111:0/2404416349 conn(0x7fbd680b61c0 msgr2=0x7fbd68148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 shutdown_connections 2021-04-03T03:09:01.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.479+0000 7fbd6c869700 1 -- 172.21.2.111:0/2404416349 wait complete. 2021-04-03T03:09:01.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:09:01.497 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:09:01.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 Processor -- start 2021-04-03T03:09:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- start start 2021-04-03T03:09:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4b380603f0 0x7f4b381447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4b380add50 0x7f4b38144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380b17b0 0x7f4b38147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b381451c0 con 0x7f4b380603f0 2021-04-03T03:09:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b380607d0 con 0x7f4b380add50 2021-04-03T03:09:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3814b090 con 0x7f4b380b17b0 2021-04-03T03:09:01.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4b380603f0 0x7f4b381447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4b380603f0 0x7f4b381447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50306/0 (socket says 172.21.2.111:50306) 2021-04-03T03:09:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 -- 172.21.2.111:0/1818870455 learned_addr learned my addr 172.21.2.111:0/1818870455 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 -- 172.21.2.111:0/1818870455 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380b17b0 msgr2=0x7f4b38147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 --2- 172.21.2.111:0/1818870455 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380b17b0 0x7f4b38147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 -- 172.21.2.111:0/1818870455 >> v2:172.21.2.115:3300/0 conn(0x7f4b380add50 msgr2=0x7f4b38144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 --2- 172.21.2.111:0/1818870455 >> v2:172.21.2.115:3300/0 conn(0x7f4b380add50 0x7f4b38144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 -- 172.21.2.111:0/1818870455 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3814b330 con 0x7f4b380603f0 2021-04-03T03:09:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3e29e700 1 --2- 172.21.2.111:0/1818870455 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380603f0 0x7f4b381447e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4b34002f30 tx=0x7f4b34006e90).ready entity=mon.0 client_cookie=27dd83f3608e6b14 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3d29c700 1 -- 172.21.2.111:0/1818870455 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b34009920 con 0x7f4b380603f0 2021-04-03T03:09:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3d29c700 1 -- 172.21.2.111:0/1818870455 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b34009a80 con 0x7f4b380603f0 2021-04-03T03:09:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3d29c700 1 -- 172.21.2.111:0/1818870455 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b34009de0 con 0x7f4b380603f0 2021-04-03T03:09:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1818870455 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380603f0 msgr2=0x7f4b381447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1818870455 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380603f0 0x7f4b381447e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4b34002f30 tx=0x7f4b34006e90).stop 2021-04-03T03:09:01.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1818870455 shutdown_connections 2021-04-03T03:09:01.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1818870455 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380b17b0 0x7f4b38147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1818870455 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380603f0 0x7f4b381447e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1818870455 >> v2:172.21.2.115:3300/0 conn(0x7f4b380add50 0x7f4b38144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1818870455 >> 172.21.2.111:0/1818870455 conn(0x7f4b380b61c0 msgr2=0x7f4b3805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1818870455 shutdown_connections 2021-04-03T03:09:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.547+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1818870455 wait complete. 2021-04-03T03:09:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 Processor -- start 2021-04-03T03:09:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 -- start start 2021-04-03T03:09:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380603f0 0x7f4b38142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b380add50 0x7f4b381568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380b17b0 0x7f4b3815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b380a8920 con 0x7f4b380b17b0 2021-04-03T03:09:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b38067fa0 con 0x7f4b380add50 2021-04-03T03:09:01.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b380a9c50 con 0x7f4b380603f0 2021-04-03T03:09:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380b17b0 0x7f4b3815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380b17b0 0x7f4b3815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50314/0 (socket says 172.21.2.111:50314) 2021-04-03T03:09:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 -- 172.21.2.111:0/1915693777 learned_addr learned my addr 172.21.2.111:0/1915693777 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 -- 172.21.2.111:0/1915693777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380603f0 msgr2=0x7f4b38142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380603f0 0x7f4b38142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 -- 172.21.2.111:0/1915693777 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b380add50 msgr2=0x7f4b381568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b380add50 0x7f4b381568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 -- 172.21.2.111:0/1915693777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b380a9eb0 con 0x7f4b380b17b0 2021-04-03T03:09:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3ea9f700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380b17b0 0x7f4b3815a630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f4b300032b0 tx=0x7f4b300090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b2b7fe700 1 -- 172.21.2.111:0/1915693777 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3000c900 con 0x7f4b380b17b0 2021-04-03T03:09:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b2b7fe700 1 -- 172.21.2.111:0/1915693777 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3000ca60 con 0x7f4b380b17b0 2021-04-03T03:09:01.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b2b7fe700 1 -- 172.21.2.111:0/1915693777 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3000cdc0 con 0x7f4b380b17b0 2021-04-03T03:09:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b38159ef0 con 0x7f4b380b17b0 2021-04-03T03:09:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b38147900 con 0x7f4b380b17b0 2021-04-03T03:09:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3815a320 con 0x7f4b380b17b0 2021-04-03T03:09:01.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b2b7fe700 1 -- 172.21.2.111:0/1915693777 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4b30015470 con 0x7f4b380b17b0 2021-04-03T03:09:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b2b7fe700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b180570a0 0x7f4b18059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b2b7fe700 1 -- 172.21.2.111:0/1915693777 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f4b30070090 con 0x7f4b380b17b0 2021-04-03T03:09:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3e29e700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b180570a0 0x7f4b18059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.551+0000 7f4b3e29e700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b180570a0 0x7f4b18059560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4b34002f30 tx=0x7f4b34015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.555+0000 7f4b2b7fe700 1 -- 172.21.2.111:0/1915693777 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4b30045410 con 0x7f4b380b17b0 2021-04-03T03:09:01.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4b38057660 con 0x7f4b380b17b0 2021-04-03T03:09:01.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b2b7fe700 1 -- 172.21.2.111:0/1915693777 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4b30048a30 con 0x7f4b380b17b0 2021-04-03T03:09:01.656 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:09:01.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b180570a0 msgr2=0x7f4b18059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b180570a0 0x7f4b18059560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4b34002f30 tx=0x7f4b34015000).stop 2021-04-03T03:09:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380b17b0 msgr2=0x7f4b3815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380b17b0 0x7f4b3815a630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f4b300032b0 tx=0x7f4b300090f0).stop 2021-04-03T03:09:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 shutdown_connections 2021-04-03T03:09:01.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b380b17b0 0x7f4b3815a630 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b380603f0 0x7f4b38142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b180570a0 0x7f4b18059560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 --2- 172.21.2.111:0/1915693777 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b380add50 0x7f4b381568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.647+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 >> 172.21.2.111:0/1915693777 conn(0x7f4b380b61c0 msgr2=0x7f4b380b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.651+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 shutdown_connections 2021-04-03T03:09:01.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.651+0000 7f4b3f2a0700 1 -- 172.21.2.111:0/1915693777 wait complete. 2021-04-03T03:09:01.664 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:09:01.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.711+0000 7fdf5da3e700 1 Processor -- start 2021-04-03T03:09:01.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.711+0000 7fdf5da3e700 1 -- start start 2021-04-03T03:09:01.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdf50040ee0 0x7fdf500d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdf5003d6c0 0x7fdf5003db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf500d0ae0 con 0x7fdf50047640 2021-04-03T03:09:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf500d24f0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf50047a20 con 0x7fdf50040ee0 2021-04-03T03:09:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdf5003d6c0 0x7fdf5003db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdf5003d6c0 0x7fdf5003db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50322/0 (socket says 172.21.2.111:50322) 2021-04-03T03:09:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 -- 172.21.2.111:0/2428201315 learned_addr learned my addr 172.21.2.111:0/2428201315 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 -- 172.21.2.111:0/2428201315 >> v2:172.21.2.115:3300/0 conn(0x7fdf50040ee0 msgr2=0x7fdf500d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 --2- 172.21.2.111:0/2428201315 >> v2:172.21.2.115:3300/0 conn(0x7fdf50040ee0 0x7fdf500d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 -- 172.21.2.111:0/2428201315 >> v1:172.21.2.115:6789/0 conn(0x7fdf50047640 legacy=0x7fdf500d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 -- 172.21.2.111:0/2428201315 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50040d10 con 0x7fdf5003d6c0 2021-04-03T03:09:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5d23d700 1 --2- 172.21.2.111:0/2428201315 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf5003db00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fdf5000cef0 tx=0x7fdf5805ad70).ready entity=mon.0 client_cookie=8a64de0a9ec9e809 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf577fe700 1 -- 172.21.2.111:0/2428201315 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf58088030 con 0x7fdf5003d6c0 2021-04-03T03:09:01.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf577fe700 1 -- 172.21.2.111:0/2428201315 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5805bda0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf577fe700 1 -- 172.21.2.111:0/2428201315 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf5805df60 con 0x7fdf5003d6c0 2021-04-03T03:09:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/2428201315 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 msgr2=0x7fdf5003db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/2428201315 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf5003db00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fdf5000cef0 tx=0x7fdf5805ad70).stop 2021-04-03T03:09:01.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/2428201315 shutdown_connections 2021-04-03T03:09:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/2428201315 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf5003db00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/2428201315 >> v2:172.21.2.115:3300/0 conn(0x7fdf50040ee0 0x7fdf500d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/2428201315 >> 172.21.2.111:0/2428201315 conn(0x7fdf5000d740 msgr2=0x7fdf5000db50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/2428201315 shutdown_connections 2021-04-03T03:09:01.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/2428201315 wait complete. 2021-04-03T03:09:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 Processor -- start 2021-04-03T03:09:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- start start 2021-04-03T03:09:01.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf500d4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdf50040ee0 0x7fdf500e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdf50047640 0x7fdf500d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500cf5a0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500d5260 con 0x7fdf50040ee0 2021-04-03T03:09:01.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500ce600 con 0x7fdf50047640 2021-04-03T03:09:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf500d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf500d4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50328/0 (socket says 172.21.2.111:50328) 2021-04-03T03:09:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 -- 172.21.2.111:0/3744730104 learned_addr learned my addr 172.21.2.111:0/3744730104 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 -- 172.21.2.111:0/3744730104 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdf50047640 msgr2=0x7fdf500d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdf50047640 0x7fdf500d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 -- 172.21.2.111:0/3744730104 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdf50040ee0 msgr2=0x7fdf500e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdf50040ee0 0x7fdf500e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 -- 172.21.2.111:0/3744730104 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf500ce360 con 0x7fdf5003d6c0 2021-04-03T03:09:01.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5ca3c700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf500d4850 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fdf500ce790 tx=0x7fdf48006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf55ffb700 1 -- 172.21.2.111:0/3744730104 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf4800f510 con 0x7fdf5003d6c0 2021-04-03T03:09:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf55ffb700 1 -- 172.21.2.111:0/3744730104 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4800f670 con 0x7fdf5003d6c0 2021-04-03T03:09:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5003d430 con 0x7fdf5003d6c0 2021-04-03T03:09:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf500e31a0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf55ffb700 1 -- 172.21.2.111:0/3744730104 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf4800f7e0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.715+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf500d5ed0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.719+0000 7fdf55ffb700 1 -- 172.21.2.111:0/3744730104 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdf4800f070 con 0x7fdf5003d6c0 2021-04-03T03:09:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.719+0000 7fdf55ffb700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdf38057050 0x7fdf38059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.719+0000 7fdf55ffb700 1 -- 172.21.2.111:0/3744730104 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fdf4806fbd0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.719+0000 7fdf57fff700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdf38057050 0x7fdf38059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.719+0000 7fdf57fff700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdf38057050 0x7fdf38059510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fdf4c002f30 tx=0x7fdf4c006a90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.719+0000 7fdf55ffb700 1 -- 172.21.2.111:0/3744730104 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdf48044ea0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf50004db0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf55ffb700 1 -- 172.21.2.111:0/3744730104 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf480484c0 con 0x7fdf5003d6c0 2021-04-03T03:09:01.822 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:01.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdf38057050 msgr2=0x7fdf38059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdf38057050 0x7fdf38059510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fdf4c002f30 tx=0x7fdf4c006a90).stop 2021-04-03T03:09:01.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 msgr2=0x7fdf500d4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf500d4850 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fdf500ce790 tx=0x7fdf48006d50).stop 2021-04-03T03:09:01.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 shutdown_connections 2021-04-03T03:09:01.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdf50047640 0x7fdf500d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdf5003d6c0 0x7fdf500d4850 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdf38057050 0x7fdf38059510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 --2- 172.21.2.111:0/3744730104 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdf50040ee0 0x7fdf500e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 >> 172.21.2.111:0/3744730104 conn(0x7fdf5000d740 msgr2=0x7fdf50042ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 shutdown_connections 2021-04-03T03:09:01.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.815+0000 7fdf5da3e700 1 -- 172.21.2.111:0/3744730104 wait complete. 2021-04-03T03:09:01.830 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:09:01.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 Processor -- start 2021-04-03T03:09:01.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- start start 2021-04-03T03:09:01.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fee98042080 0x7fee9803ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee98042850 0x7fee9803d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee98039580 con 0x7fee980cdcf0 2021-04-03T03:09:01.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee98042460 con 0x7fee98042080 2021-04-03T03:09:01.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980d19c0 con 0x7fee98042850 2021-04-03T03:09:01.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fee98042080 0x7fee9803ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fee98042080 0x7fee9803ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50334/0 (socket says 172.21.2.111:50334) 2021-04-03T03:09:01.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea556c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fee980cdcf0 0x7fee980d4190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51126/0 (socket says 172.21.2.111:51126) 2021-04-03T03:09:01.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 -- 172.21.2.111:0/401871904 learned_addr learned my addr 172.21.2.111:0/401871904 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 -- 172.21.2.111:0/401871904 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee98042850 msgr2=0x7fee9803d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:01.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 --2- 172.21.2.111:0/401871904 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee98042850 0x7fee9803d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 -- 172.21.2.111:0/401871904 >> v1:172.21.2.111:6789/0 conn(0x7fee980cdcf0 legacy=0x7fee980d4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 -- 172.21.2.111:0/401871904 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980397b0 con 0x7fee98042080 2021-04-03T03:09:01.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea4d6b700 1 --2- 172.21.2.111:0/401871904 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042080 0x7fee9803ffc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7feea0069660 tx=0x7fee94006e90).ready entity=mon.0 client_cookie=201ac19478cb5502 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7fee9f7fe700 1 -- 172.21.2.111:0/401871904 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee94009820 con 0x7fee98042080 2021-04-03T03:09:01.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7fee9f7fe700 1 -- 172.21.2.111:0/401871904 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94009980 con 0x7fee98042080 2021-04-03T03:09:01.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7fee9f7fe700 1 -- 172.21.2.111:0/401871904 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee94009ce0 con 0x7fee98042080 2021-04-03T03:09:01.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- 172.21.2.111:0/401871904 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042080 msgr2=0x7fee9803ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 --2- 172.21.2.111:0/401871904 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042080 0x7fee9803ffc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7feea0069660 tx=0x7fee94006e90).stop 2021-04-03T03:09:01.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- 172.21.2.111:0/401871904 shutdown_connections 2021-04-03T03:09:01.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 --2- 172.21.2.111:0/401871904 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042080 0x7fee9803ffc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 --2- 172.21.2.111:0/401871904 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee98042850 0x7fee9803d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- 172.21.2.111:0/401871904 >> 172.21.2.111:0/401871904 conn(0x7fee980462b0 msgr2=0x7fee980466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- 172.21.2.111:0/401871904 shutdown_connections 2021-04-03T03:09:01.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.879+0000 7feea5d6d700 1 -- 172.21.2.111:0/401871904 wait complete. 2021-04-03T03:09:01.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 Processor -- start 2021-04-03T03:09:01.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 -- start start 2021-04-03T03:09:01.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee98042080 0x7fee980d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042850 0x7fee980b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee980cdcf0 0x7fee980b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee98039f70 con 0x7fee98042850 2021-04-03T03:09:01.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980d32f0 con 0x7fee98042080 2021-04-03T03:09:01.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980d46c0 con 0x7fee980cdcf0 2021-04-03T03:09:01.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042850 0x7fee980b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042850 0x7fee980b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50342/0 (socket says 172.21.2.111:50342) 2021-04-03T03:09:01.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 -- 172.21.2.111:0/1508829671 learned_addr learned my addr 172.21.2.111:0/1508829671 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:01.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 -- 172.21.2.111:0/1508829671 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee980cdcf0 msgr2=0x7fee980b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee980cdcf0 0x7fee980b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 -- 172.21.2.111:0/1508829671 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee98042080 msgr2=0x7fee980d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee98042080 0x7fee980d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 -- 172.21.2.111:0/1508829671 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980d3520 con 0x7fee98042850 2021-04-03T03:09:01.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9ffff700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042850 0x7fee980b2570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fee90002f30 tx=0x7fee90006ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9dffb700 1 -- 172.21.2.111:0/1508829671 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee9000d790 con 0x7fee98042850 2021-04-03T03:09:01.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9dffb700 1 -- 172.21.2.111:0/1508829671 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9000d8f0 con 0x7fee98042850 2021-04-03T03:09:01.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9dffb700 1 -- 172.21.2.111:0/1508829671 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee9000dc50 con 0x7fee98042850 2021-04-03T03:09:01.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee980b9c70 con 0x7fee98042850 2021-04-03T03:09:01.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee980d4470 con 0x7fee98042850 2021-04-03T03:09:01.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee980d2b90 con 0x7fee98042850 2021-04-03T03:09:01.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9dffb700 1 -- 172.21.2.111:0/1508829671 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fee90019460 con 0x7fee98042850 2021-04-03T03:09:01.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9dffb700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee80057050 0x7fee80059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:01.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7fee9dffb700 1 -- 172.21.2.111:0/1508829671 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fee90070240 con 0x7fee98042850 2021-04-03T03:09:01.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea4d6b700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee80057050 0x7fee80059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:01.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.883+0000 7feea4d6b700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee80057050 0x7fee80059510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7feea0069660 tx=0x7fee94015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:01.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.887+0000 7fee9dffb700 1 -- 172.21.2.111:0/1508829671 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fee900455c0 con 0x7fee98042850 2021-04-03T03:09:01.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee980d36b0 con 0x7fee98042850 2021-04-03T03:09:01.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7fee9dffb700 1 -- 172.21.2.111:0/1508829671 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fee9000ddb0 con 0x7fee98042850 2021-04-03T03:09:01.988 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:01.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee80057050 msgr2=0x7fee80059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee80057050 0x7fee80059510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7feea0069660 tx=0x7fee94015000).stop 2021-04-03T03:09:01.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042850 msgr2=0x7fee980b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:01.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042850 0x7fee980b2570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fee90002f30 tx=0x7fee90006ed0).stop 2021-04-03T03:09:01.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 shutdown_connections 2021-04-03T03:09:01.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee980cdcf0 0x7fee980b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee80057050 0x7fee80059510 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee98042080 0x7fee980d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:01.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 --2- 172.21.2.111:0/1508829671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee98042850 0x7fee980b2570 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:01.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 >> 172.21.2.111:0/1508829671 conn(0x7fee980462b0 msgr2=0x7fee980056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:01.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 shutdown_connections 2021-04-03T03:09:01.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:01.979+0000 7feea5d6d700 1 -- 172.21.2.111:0/1508829671 wait complete. 2021-04-03T03:09:01.995 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:09:02.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48eb59e700 1 Processor -- start 2021-04-03T03:09:02.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48eb59e700 1 -- start start 2021-04-03T03:09:02.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48eb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec144ef0 0x7f48ec147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48eb59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0607d0 con 0x7f48ec0add50 2021-04-03T03:09:02.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48eb59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec14b090 con 0x7f48ec0603f0 2021-04-03T03:09:02.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48eb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec14b330 con 0x7f48ec144ef0 2021-04-03T03:09:02.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e9d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f48ec0add50 0x7f48ec147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51142/0 (socket says 172.21.2.111:51142) 2021-04-03T03:09:02.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e9d9b700 1 -- 172.21.2.111:0/2180420192 learned_addr learned my addr 172.21.2.111:0/2180420192 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:02.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976449480 0 0) 0x7f48ec0607d0 con 0x7f48ec0add50 2021-04-03T03:09:02.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48cc003240 con 0x7f48ec0add50 2021-04-03T03:09:02.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1529747348 0 0) 0x7f48cc003240 con 0x7f48ec0add50 2021-04-03T03:09:02.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec144ef0 msgr2=0x7f48ec147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e959a700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec144ef0 0x7f48ec147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 >> v1:172.21.2.115:6789/0 conn(0x7f48ec0603f0 legacy=0x7f48ec1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.043+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec14b5d0 con 0x7f48ec0add50 2021-04-03T03:09:02.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f48d40016d0 con 0x7f48ec0add50 2021-04-03T03:09:02.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 >> v1:172.21.2.111:6789/0 conn(0x7f48ec0add50 legacy=0x7f48ec147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc0037e0 0x7f48cc003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec144ef0 0x7f48cc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc0049d0 0x7f48cc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec14b330 con 0x7f48cc0037e0 2021-04-03T03:09:02.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0042b0 con 0x7f48cc0049d0 2021-04-03T03:09:02.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc00b730 con 0x7f48ec144ef0 2021-04-03T03:09:02.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ea59c700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc0037e0 0x7f48cc003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ea59c700 1 -- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec144ef0 msgr2=0x7f48cc0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ea59c700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec144ef0 0x7f48cc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ea59c700 1 -- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc0049d0 msgr2=0x7f48cc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ea59c700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48cc0049d0 0x7f48cc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ea59c700 1 -- 172.21.2.111:0/2180420192 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc003240 con 0x7f48cc0037e0 2021-04-03T03:09:02.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ea59c700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc0037e0 0x7f48cc003be0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f48cc0054b0 tx=0x7f48dc007b50).ready entity=mon.0 client_cookie=77eb7cd6aaf5de67 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48dc00c620 con 0x7f48cc0037e0 2021-04-03T03:09:02.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00c780 con 0x7f48cc0037e0 2021-04-03T03:09:02.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e959a700 1 -- 172.21.2.111:0/2180420192 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48dc00cae0 con 0x7f48cc0037e0 2021-04-03T03:09:02.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- 172.21.2.111:0/2180420192 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc0037e0 msgr2=0x7f48cc003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 --2- 172.21.2.111:0/2180420192 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48cc0037e0 0x7f48cc003be0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f48cc0054b0 tx=0x7f48dc007b50).stop 2021-04-03T03:09:02.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48ead9d700 1 -- 172.21.2.111:0/2180420192 reap_dead start 2021-04-03T03:09:02.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- 172.21.2.111:0/2180420192 shutdown_connections 2021-04-03T03:09:02.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- 172.21.2.111:0/2180420192 >> 172.21.2.111:0/2180420192 conn(0x7f48ec0b61c0 msgr2=0x7f48ec05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- 172.21.2.111:0/2180420192 shutdown_connections 2021-04-03T03:09:02.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- 172.21.2.111:0/2180420192 wait complete. 2021-04-03T03:09:02.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 Processor -- start 2021-04-03T03:09:02.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- start start 2021-04-03T03:09:02.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48ec147300 0x7f48ec1476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48ec0a8930 0x7f48ec1573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec0a6f20 0x7f48ec0a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0a8130 con 0x7f48ec0a8930 2021-04-03T03:09:02.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0a9000 con 0x7f48ec147300 2021-04-03T03:09:02.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec156c70 con 0x7f48ec0a6f20 2021-04-03T03:09:02.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48ec0a8930 0x7f48ec1573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48ec0a8930 0x7f48ec1573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50360/0 (socket says 172.21.2.111:50360) 2021-04-03T03:09:02.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 -- 172.21.2.111:0/2401405770 learned_addr learned my addr 172.21.2.111:0/2401405770 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 -- 172.21.2.111:0/2401405770 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec0a6f20 msgr2=0x7f48ec0a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec0a6f20 0x7f48ec0a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 -- 172.21.2.111:0/2401405770 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48ec147300 msgr2=0x7f48ec1476e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:02.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48ec147300 0x7f48ec1476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 -- 172.21.2.111:0/2401405770 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec156f10 con 0x7f48ec0a8930 2021-04-03T03:09:02.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e9d9b700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48ec0a8930 0x7f48ec1573b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f48ec0b5640 tx=0x7f48ec14b220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e37fe700 1 -- 172.21.2.111:0/2401405770 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d4007170 con 0x7f48ec0a8930 2021-04-03T03:09:02.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e37fe700 1 -- 172.21.2.111:0/2401405770 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d40072d0 con 0x7f48ec0a8930 2021-04-03T03:09:02.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec1578a0 con 0x7f48ec0a8930 2021-04-03T03:09:02.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec062c10 con 0x7f48ec0a8930 2021-04-03T03:09:02.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.047+0000 7f48e37fe700 1 -- 172.21.2.111:0/2401405770 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d400aaa0 con 0x7f48ec0a8930 2021-04-03T03:09:02.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.051+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec1570a0 con 0x7f48ec0a8930 2021-04-03T03:09:02.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.051+0000 7f48e37fe700 1 -- 172.21.2.111:0/2401405770 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f48d4010070 con 0x7f48ec0a8930 2021-04-03T03:09:02.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.051+0000 7f48e37fe700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d0057030 0x7f48d00594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.051+0000 7f48e37fe700 1 -- 172.21.2.111:0/2401405770 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f48d406fe90 con 0x7f48ec0a8930 2021-04-03T03:09:02.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.051+0000 7f48ea59c700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d0057030 0x7f48d00594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.051+0000 7f48ea59c700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d0057030 0x7f48d00594f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f48cc0054b0 tx=0x7f48dc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.051+0000 7f48e37fe700 1 -- 172.21.2.111:0/2401405770 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f48d4045160 con 0x7f48ec0a8930 2021-04-03T03:09:02.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.143+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48ec057660 con 0x7f48ec0a8930 2021-04-03T03:09:02.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.143+0000 7f48e37fe700 1 -- 172.21.2.111:0/2401405770 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f48d4048780 con 0x7f48ec0a8930 2021-04-03T03:09:02.153 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:02.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d0057030 msgr2=0x7f48d00594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d0057030 0x7f48d00594f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f48cc0054b0 tx=0x7f48dc015000).stop 2021-04-03T03:09:02.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48ec0a8930 msgr2=0x7f48ec1573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48ec0a8930 0x7f48ec1573b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f48ec0b5640 tx=0x7f48ec14b220).stop 2021-04-03T03:09:02.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 shutdown_connections 2021-04-03T03:09:02.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48ec0a6f20 0x7f48ec0a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d0057030 0x7f48d00594f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48ec147300 0x7f48ec1476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 --2- 172.21.2.111:0/2401405770 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48ec0a8930 0x7f48ec1573b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 >> 172.21.2.111:0/2401405770 conn(0x7f48ec0b61c0 msgr2=0x7f48ec142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 shutdown_connections 2021-04-03T03:09:02.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.147+0000 7f48eb59e700 1 -- 172.21.2.111:0/2401405770 wait complete. 2021-04-03T03:09:02.161 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:02.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 Processor -- start 2021-04-03T03:09:02.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 -- start start 2021-04-03T03:09:02.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f97800603f0 0x7f97800b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f978014b700 0x7f97800b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97800607d0 con 0x7f97800b12b0 2021-04-03T03:09:02.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97800af400 con 0x7f978014b700 2021-04-03T03:09:02.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97800af6a0 con 0x7f97800603f0 2021-04-03T03:09:02.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f978014b700 0x7f97800b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f978014b700 0x7f97800b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50370/0 (socket says 172.21.2.111:50370) 2021-04-03T03:09:02.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 -- 172.21.2.111:0/4163074308 learned_addr learned my addr 172.21.2.111:0/4163074308 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785e35700 1 --2- 172.21.2.111:0/4163074308 >> v2:172.21.2.115:3300/0 conn(0x7f97800603f0 0x7f97800b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 -- 172.21.2.111:0/4163074308 >> v2:172.21.2.115:3300/0 conn(0x7f97800603f0 msgr2=0x7f97800b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 --2- 172.21.2.111:0/4163074308 >> v2:172.21.2.115:3300/0 conn(0x7f97800603f0 0x7f97800b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 -- 172.21.2.111:0/4163074308 >> v1:172.21.2.111:6789/0 conn(0x7f97800b12b0 legacy=0x7f97801465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 -- 172.21.2.111:0/4163074308 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800afb40 con 0x7f978014b700 2021-04-03T03:09:02.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9785634700 1 --2- 172.21.2.111:0/4163074308 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f97800b0580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f9774002f30 tx=0x7f9774006e90).ready entity=mon.0 client_cookie=ce48927943f1a60f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9784e33700 1 -- 172.21.2.111:0/4163074308 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97740099c0 con 0x7f978014b700 2021-04-03T03:09:02.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9784e33700 1 -- 172.21.2.111:0/4163074308 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9774009b20 con 0x7f978014b700 2021-04-03T03:09:02.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9784e33700 1 -- 172.21.2.111:0/4163074308 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9774009e80 con 0x7f978014b700 2021-04-03T03:09:02.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 -- 172.21.2.111:0/4163074308 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 msgr2=0x7f97800b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.207+0000 7f9786e37700 1 --2- 172.21.2.111:0/4163074308 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f97800b0580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f9774002f30 tx=0x7f9774006e90).stop 2021-04-03T03:09:02.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- 172.21.2.111:0/4163074308 shutdown_connections 2021-04-03T03:09:02.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 --2- 172.21.2.111:0/4163074308 >> v2:172.21.2.115:3300/0 conn(0x7f97800603f0 0x7f97800b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 --2- 172.21.2.111:0/4163074308 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f97800b0580 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- 172.21.2.111:0/4163074308 >> 172.21.2.111:0/4163074308 conn(0x7f97800b61c0 msgr2=0x7f978005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- 172.21.2.111:0/4163074308 shutdown_connections 2021-04-03T03:09:02.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- 172.21.2.111:0/4163074308 wait complete. 2021-04-03T03:09:02.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 Processor -- start 2021-04-03T03:09:02.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- start start 2021-04-03T03:09:02.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97800603f0 0x7f9780159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97800b12b0 0x7f9780149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f9780064460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97800a9460 con 0x7f978014b700 2021-04-03T03:09:02.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780064070 con 0x7f97800b12b0 2021-04-03T03:09:02.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97800a9df0 con 0x7f97800603f0 2021-04-03T03:09:02.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f9780064460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f9780064460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50374/0 (socket says 172.21.2.111:50374) 2021-04-03T03:09:02.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 -- 172.21.2.111:0/1660783148 learned_addr learned my addr 172.21.2.111:0/1660783148 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 -- 172.21.2.111:0/1660783148 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97800603f0 msgr2=0x7f9780159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97800603f0 0x7f9780159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 -- 172.21.2.111:0/1660783148 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97800b12b0 msgr2=0x7f9780149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97800b12b0 0x7f9780149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 -- 172.21.2.111:0/1660783148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800aa090 con 0x7f978014b700 2021-04-03T03:09:02.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786636700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f9780064460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f978006a3b0 tx=0x7f9778007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9772ffd700 1 -- 172.21.2.111:0/1660783148 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97780030f0 con 0x7f978014b700 2021-04-03T03:09:02.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9772ffd700 1 -- 172.21.2.111:0/1660783148 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977800a7b0 con 0x7f978014b700 2021-04-03T03:09:02.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97800aa330 con 0x7f978014b700 2021-04-03T03:09:02.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97800aaf80 con 0x7f978014b700 2021-04-03T03:09:02.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9772ffd700 1 -- 172.21.2.111:0/1660783148 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f977800ab10 con 0x7f978014b700 2021-04-03T03:09:02.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97800641c0 con 0x7f978014b700 2021-04-03T03:09:02.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9772ffd700 1 -- 172.21.2.111:0/1660783148 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9778019030 con 0x7f978014b700 2021-04-03T03:09:02.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9772ffd700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9760057000 0x7f97600594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9772ffd700 1 -- 172.21.2.111:0/1660783148 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f977806fb50 con 0x7f978014b700 2021-04-03T03:09:02.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.211+0000 7f9785e35700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9760057000 0x7f97600594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.215+0000 7f9785e35700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9760057000 0x7f97600594c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f977c00a000 tx=0x7f977c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.215+0000 7f9772ffd700 1 -- 172.21.2.111:0/1660783148 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f97800641c0 con 0x7f978014b700 2021-04-03T03:09:02.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.307+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97800641c0 con 0x7f978014b700 2021-04-03T03:09:02.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.307+0000 7f9772ffd700 1 -- 172.21.2.111:0/1660783148 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f97800641c0 con 0x7f978014b700 2021-04-03T03:09:02.317 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:02.318 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":24,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:08:45.426626+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:08:40.074197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-04-03T03:08:42.209560+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:02.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9760057000 msgr2=0x7f97600594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9760057000 0x7f97600594c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f977c00a000 tx=0x7f977c008040).stop 2021-04-03T03:09:02.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 msgr2=0x7f9780064460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f9780064460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f978006a3b0 tx=0x7f9778007ee0).stop 2021-04-03T03:09:02.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 shutdown_connections 2021-04-03T03:09:02.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f978014b700 0x7f9780064460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97800603f0 0x7f9780159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9760057000 0x7f97600594c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 --2- 172.21.2.111:0/1660783148 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97800b12b0 0x7f9780149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 >> 172.21.2.111:0/1660783148 conn(0x7f97800b61c0 msgr2=0x7f9780147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 shutdown_connections 2021-04-03T03:09:02.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.311+0000 7f9786e37700 1 -- 172.21.2.111:0/1660783148 wait complete. 2021-04-03T03:09:02.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:08:40.074197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:08:42.209560+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:09:02.326 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:09:02.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff642e8d700 1 Processor -- start 2021-04-03T03:09:02.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff642e8d700 1 -- start start 2021-04-03T03:09:02.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff642e8d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff63c0add50 0x7ff63c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff642e8d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff63c144ef0 0x7ff63c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff642e8d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c0607d0 con 0x7ff63c0603f0 2021-04-03T03:09:02.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff642e8d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63c14b090 con 0x7ff63c144ef0 2021-04-03T03:09:02.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff642e8d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63c14b330 con 0x7ff63c0add50 2021-04-03T03:09:02.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff64268c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff63c144ef0 0x7ff63c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff64268c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff63c144ef0 0x7ff63c147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50386/0 (socket says 172.21.2.111:50386) 2021-04-03T03:09:02.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.375+0000 7ff64268c700 1 -- 172.21.2.111:0/370165865 learned_addr learned my addr 172.21.2.111:0/370165865 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 -- 172.21.2.111:0/370165865 >> v2:172.21.2.115:3300/0 conn(0x7ff63c0add50 msgr2=0x7ff63c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 --2- 172.21.2.111:0/370165865 >> v2:172.21.2.115:3300/0 conn(0x7ff63c0add50 0x7ff63c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 -- 172.21.2.111:0/370165865 >> v1:172.21.2.115:6789/0 conn(0x7ff63c0603f0 legacy=0x7ff63c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 -- 172.21.2.111:0/370165865 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c14b5d0 con 0x7ff63c144ef0 2021-04-03T03:09:02.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 --2- 172.21.2.111:0/370165865 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c147c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ff63c0ae520 tx=0x7ff634009270).ready entity=mon.0 client_cookie=e857ffd6315e3652 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff640e89700 1 -- 172.21.2.111:0/370165865 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63400bbc0 con 0x7ff63c144ef0 2021-04-03T03:09:02.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff640e89700 1 -- 172.21.2.111:0/370165865 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63400bd20 con 0x7ff63c144ef0 2021-04-03T03:09:02.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff640e89700 1 -- 172.21.2.111:0/370165865 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63400be90 con 0x7ff63c144ef0 2021-04-03T03:09:02.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- 172.21.2.111:0/370165865 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 msgr2=0x7ff63c147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 --2- 172.21.2.111:0/370165865 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c147c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ff63c0ae520 tx=0x7ff634009270).stop 2021-04-03T03:09:02.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- 172.21.2.111:0/370165865 shutdown_connections 2021-04-03T03:09:02.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 --2- 172.21.2.111:0/370165865 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c147c30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 --2- 172.21.2.111:0/370165865 >> v2:172.21.2.115:3300/0 conn(0x7ff63c0add50 0x7ff63c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- 172.21.2.111:0/370165865 >> 172.21.2.111:0/370165865 conn(0x7ff63c0b61c0 msgr2=0x7ff63c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- 172.21.2.111:0/370165865 shutdown_connections 2021-04-03T03:09:02.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- 172.21.2.111:0/370165865 wait complete. 2021-04-03T03:09:02.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 Processor -- start 2021-04-03T03:09:02.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- start start 2021-04-03T03:09:02.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff63c0603f0 0x7ff63c063dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff63c0add50 0x7ff63c0aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c0aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c062d50 con 0x7ff63c144ef0 2021-04-03T03:09:02.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c143760 con 0x7ff63c0603f0 2021-04-03T03:09:02.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c157b70 con 0x7ff63c0add50 2021-04-03T03:09:02.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c0aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c0aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50392/0 (socket says 172.21.2.111:50392) 2021-04-03T03:09:02.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 -- 172.21.2.111:0/2004295570 learned_addr learned my addr 172.21.2.111:0/2004295570 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 -- 172.21.2.111:0/2004295570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff63c0add50 msgr2=0x7ff63c0aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff63c0add50 0x7ff63c0aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 -- 172.21.2.111:0/2004295570 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff63c0603f0 msgr2=0x7ff63c063dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff63c0603f0 0x7ff63c063dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 -- 172.21.2.111:0/2004295570 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c15a760 con 0x7ff63c144ef0 2021-04-03T03:09:02.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff64268c700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c0aafe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7ff63c0ae520 tx=0x7ff634016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff62effd700 1 -- 172.21.2.111:0/2004295570 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63401f610 con 0x7ff63c144ef0 2021-04-03T03:09:02.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff62effd700 1 -- 172.21.2.111:0/2004295570 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63401f770 con 0x7ff63c144ef0 2021-04-03T03:09:02.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c15a9e0 con 0x7ff63c144ef0 2021-04-03T03:09:02.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff62effd700 1 -- 172.21.2.111:0/2004295570 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63401fad0 con 0x7ff63c144ef0 2021-04-03T03:09:02.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.379+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c0a97a0 con 0x7ff63c144ef0 2021-04-03T03:09:02.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.383+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c0645d0 con 0x7ff63c144ef0 2021-04-03T03:09:02.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.383+0000 7ff62effd700 1 -- 172.21.2.111:0/2004295570 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff634026030 con 0x7ff63c144ef0 2021-04-03T03:09:02.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.383+0000 7ff62effd700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff61c057000 0x7ff61c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.383+0000 7ff62effd700 1 -- 172.21.2.111:0/2004295570 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff6340712b0 con 0x7ff63c144ef0 2021-04-03T03:09:02.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.383+0000 7ff641e8b700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff61c057000 0x7ff61c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.383+0000 7ff641e8b700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff61c057000 0x7ff61c0594c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff630002f50 tx=0x7ff630006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.383+0000 7ff62effd700 1 -- 172.21.2.111:0/2004295570 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff634046580 con 0x7ff63c144ef0 2021-04-03T03:09:02.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff63c15ad70 con 0x7ff63c144ef0 2021-04-03T03:09:02.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff62effd700 1 -- 172.21.2.111:0/2004295570 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff6340240b0 con 0x7ff63c144ef0 2021-04-03T03:09:02.484 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:09:02.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff61c057000 msgr2=0x7ff61c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff61c057000 0x7ff61c0594c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff630002f50 tx=0x7ff630006990).stop 2021-04-03T03:09:02.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 msgr2=0x7ff63c0aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c0aafe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7ff63c0ae520 tx=0x7ff634016000).stop 2021-04-03T03:09:02.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 shutdown_connections 2021-04-03T03:09:02.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff63c144ef0 0x7ff63c0aafe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff61c057000 0x7ff61c0594c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff63c0603f0 0x7ff63c063dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 --2- 172.21.2.111:0/2004295570 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff63c0add50 0x7ff63c0aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 >> 172.21.2.111:0/2004295570 conn(0x7ff63c0b61c0 msgr2=0x7ff63c144120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 shutdown_connections 2021-04-03T03:09:02.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.475+0000 7ff642e8d700 1 -- 172.21.2.111:0/2004295570 wait complete. 2021-04-03T03:09:02.492 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:09:02.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.539+0000 7f6b3953d700 1 Processor -- start 2021-04-03T03:09:02.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.539+0000 7f6b3953d700 1 -- start start 2021-04-03T03:09:02.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b2c0dbba0 0x7f6b2c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6b2c041060 0x7f6b2c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0dc560 con 0x7f6b2c03d560 2021-04-03T03:09:02.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c03de60 con 0x7f6b2c0dbba0 2021-04-03T03:09:02.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0dc800 con 0x7f6b2c041060 2021-04-03T03:09:02.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b2c0dbba0 0x7f6b2c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b2c0dbba0 0x7f6b2c0dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50396/0 (socket says 172.21.2.111:50396) 2021-04-03T03:09:02.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/95179578 learned_addr learned my addr 172.21.2.111:0/95179578 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/95179578 >> v2:172.21.2.115:3300/0 conn(0x7f6b2c041060 msgr2=0x7f6b2c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- 172.21.2.111:0/95179578 >> v2:172.21.2.115:3300/0 conn(0x7f6b2c041060 0x7f6b2c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/95179578 >> v1:172.21.2.111:6789/0 conn(0x7f6b2c03d560 legacy=0x7f6b2c041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/95179578 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c03da30 con 0x7f6b2c0dbba0 2021-04-03T03:09:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- 172.21.2.111:0/95179578 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c0dbba0 0x7f6b2c0dbf80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f6b28002f30 tx=0x7f6b28006e90).ready entity=mon.0 client_cookie=482ff4e1682d7e54 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b32ffd700 1 -- 172.21.2.111:0/95179578 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b280099c0 con 0x7f6b2c0dbba0 2021-04-03T03:09:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b32ffd700 1 -- 172.21.2.111:0/95179578 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28009b20 con 0x7f6b2c0dbba0 2021-04-03T03:09:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b32ffd700 1 -- 172.21.2.111:0/95179578 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b28009e80 con 0x7f6b2c0dbba0 2021-04-03T03:09:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- 172.21.2.111:0/95179578 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c0dbba0 msgr2=0x7f6b2c0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- 172.21.2.111:0/95179578 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c0dbba0 0x7f6b2c0dbf80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f6b28002f30 tx=0x7f6b28006e90).stop 2021-04-03T03:09:02.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- 172.21.2.111:0/95179578 shutdown_connections 2021-04-03T03:09:02.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- 172.21.2.111:0/95179578 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c0dbba0 0x7f6b2c0dbf80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- 172.21.2.111:0/95179578 >> v2:172.21.2.115:3300/0 conn(0x7f6b2c041060 0x7f6b2c03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- 172.21.2.111:0/95179578 >> 172.21.2.111:0/95179578 conn(0x7f6b2c0462b0 msgr2=0x7f6b2c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- 172.21.2.111:0/95179578 shutdown_connections 2021-04-03T03:09:02.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- 172.21.2.111:0/95179578 wait complete. 2021-04-03T03:09:02.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 Processor -- start 2021-04-03T03:09:02.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- start start 2021-04-03T03:09:02.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c03d560 0x7f6b2c03bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b2c041060 0x7f6b2c09f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b2c0dbba0 0x7f6b2c0d7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c03c470 con 0x7f6b2c03d560 2021-04-03T03:09:02.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c039580 con 0x7f6b2c0dbba0 2021-04-03T03:09:02.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0de090 con 0x7f6b2c041060 2021-04-03T03:09:02.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c03d560 0x7f6b2c03bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c03d560 0x7f6b2c03bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50400/0 (socket says 172.21.2.111:50400) 2021-04-03T03:09:02.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/4033421208 learned_addr learned my addr 172.21.2.111:0/4033421208 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/4033421208 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b2c041060 msgr2=0x7f6b2c09f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b2c041060 0x7f6b2c09f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/4033421208 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b2c0dbba0 msgr2=0x7f6b2c0d7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b2c0dbba0 0x7f6b2c0d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 -- 172.21.2.111:0/4033421208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0de330 con 0x7f6b2c03d560 2021-04-03T03:09:02.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b33fff700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c03d560 0x7f6b2c03bf80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f6b28002f30 tx=0x7f6b28015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b317fa700 1 -- 172.21.2.111:0/4033421208 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b28009e20 con 0x7f6b2c03d560 2021-04-03T03:09:02.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b317fa700 1 -- 172.21.2.111:0/4033421208 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2801e460 con 0x7f6b2c03d560 2021-04-03T03:09:02.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c0de5d0 con 0x7f6b2c03d560 2021-04-03T03:09:02.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b317fa700 1 -- 172.21.2.111:0/4033421208 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2801e7c0 con 0x7f6b2c03d560 2021-04-03T03:09:02.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.543+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c03ac40 con 0x7f6b2c03d560 2021-04-03T03:09:02.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.547+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c004db0 con 0x7f6b2c03d560 2021-04-03T03:09:02.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.547+0000 7f6b317fa700 1 -- 172.21.2.111:0/4033421208 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6b28025030 con 0x7f6b2c03d560 2021-04-03T03:09:02.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.547+0000 7f6b317fa700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b140570c0 0x7f6b14059580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.547+0000 7f6b317fa700 1 -- 172.21.2.111:0/4033421208 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f6b280716a0 con 0x7f6b2c03d560 2021-04-03T03:09:02.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.547+0000 7f6b337fe700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b140570c0 0x7f6b14059580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.547+0000 7f6b337fe700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b140570c0 0x7f6b14059580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6b24002f30 tx=0x7f6b24006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.547+0000 7f6b317fa700 1 -- 172.21.2.111:0/4033421208 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6b28046a80 con 0x7f6b2c03d560 2021-04-03T03:09:02.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.639+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6b2c0083c0 con 0x7f6b2c03d560 2021-04-03T03:09:02.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.639+0000 7f6b317fa700 1 -- 172.21.2.111:0/4033421208 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6b2804a0a0 con 0x7f6b2c03d560 2021-04-03T03:09:02.650 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:02.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b140570c0 msgr2=0x7f6b14059580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b140570c0 0x7f6b14059580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6b24002f30 tx=0x7f6b24006a60).stop 2021-04-03T03:09:02.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c03d560 msgr2=0x7f6b2c03bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c03d560 0x7f6b2c03bf80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f6b28002f30 tx=0x7f6b28015000).stop 2021-04-03T03:09:02.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 shutdown_connections 2021-04-03T03:09:02.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b140570c0 0x7f6b14059580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b2c0dbba0 0x7f6b2c0d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b2c03d560 0x7f6b2c03bf80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 --2- 172.21.2.111:0/4033421208 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b2c041060 0x7f6b2c09f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 >> 172.21.2.111:0/4033421208 conn(0x7f6b2c0462b0 msgr2=0x7f6b2c0055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 shutdown_connections 2021-04-03T03:09:02.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.643+0000 7f6b3953d700 1 -- 172.21.2.111:0/4033421208 wait complete. 2021-04-03T03:09:02.657 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:09:02.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 Processor -- start 2021-04-03T03:09:02.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- start start 2021-04-03T03:09:02.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5f1c0dbba0 0x7f5f1c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c041060 0x7f5f1c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0dc560 con 0x7f5f1c03d560 2021-04-03T03:09:02.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c03de60 con 0x7f5f1c0dbba0 2021-04-03T03:09:02.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0dc800 con 0x7f5f1c041060 2021-04-03T03:09:02.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5f1c0dbba0 0x7f5f1c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5f1c0dbba0 0x7f5f1c0dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50408/0 (socket says 172.21.2.111:50408) 2021-04-03T03:09:02.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 -- 172.21.2.111:0/402744242 learned_addr learned my addr 172.21.2.111:0/402744242 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f237fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5f1c03d560 0x7f5f1c041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51200/0 (socket says 172.21.2.111:51200) 2021-04-03T03:09:02.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 -- 172.21.2.111:0/402744242 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c041060 msgr2=0x7f5f1c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 --2- 172.21.2.111:0/402744242 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c041060 0x7f5f1c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 -- 172.21.2.111:0/402744242 >> v1:172.21.2.111:6789/0 conn(0x7f5f1c03d560 legacy=0x7f5f1c041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 -- 172.21.2.111:0/402744242 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c03da30 con 0x7f5f1c0dbba0 2021-04-03T03:09:02.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f22ffd700 1 --2- 172.21.2.111:0/402744242 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c0dbba0 0x7f5f1c0dbf80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f5f14002f30 tx=0x7f5f14006e90).ready entity=mon.0 client_cookie=3a2f83dea237fa72 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f21ffb700 1 -- 172.21.2.111:0/402744242 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f14009970 con 0x7f5f1c0dbba0 2021-04-03T03:09:02.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f21ffb700 1 -- 172.21.2.111:0/402744242 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14009ad0 con 0x7f5f1c0dbba0 2021-04-03T03:09:02.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f21ffb700 1 -- 172.21.2.111:0/402744242 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f14009e30 con 0x7f5f1c0dbba0 2021-04-03T03:09:02.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- 172.21.2.111:0/402744242 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c0dbba0 msgr2=0x7f5f1c0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 --2- 172.21.2.111:0/402744242 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c0dbba0 0x7f5f1c0dbf80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f5f14002f30 tx=0x7f5f14006e90).stop 2021-04-03T03:09:02.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- 172.21.2.111:0/402744242 shutdown_connections 2021-04-03T03:09:02.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 --2- 172.21.2.111:0/402744242 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c0dbba0 0x7f5f1c0dbf80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 --2- 172.21.2.111:0/402744242 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c041060 0x7f5f1c03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- 172.21.2.111:0/402744242 >> 172.21.2.111:0/402744242 conn(0x7f5f1c0462b0 msgr2=0x7f5f1c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- 172.21.2.111:0/402744242 shutdown_connections 2021-04-03T03:09:02.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.707+0000 7f5f23fff700 1 -- 172.21.2.111:0/402744242 wait complete. 2021-04-03T03:09:02.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 Processor -- start 2021-04-03T03:09:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 -- start start 2021-04-03T03:09:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c03d560 0x7f5f1c0d8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c041060 0x7f5f1c03b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5f1c0dbba0 0x7f5f1c0ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c03ba20 con 0x7f5f1c041060 2021-04-03T03:09:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c03bc40 con 0x7f5f1c0dbba0 2021-04-03T03:09:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c03bee0 con 0x7f5f1c03d560 2021-04-03T03:09:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c041060 0x7f5f1c03b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c041060 0x7f5f1c03b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50416/0 (socket says 172.21.2.111:50416) 2021-04-03T03:09:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 -- 172.21.2.111:0/1966526465 learned_addr learned my addr 172.21.2.111:0/1966526465 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 -- 172.21.2.111:0/1966526465 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c03d560 msgr2=0x7f5f1c0d8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c03d560 0x7f5f1c0d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 -- 172.21.2.111:0/1966526465 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5f1c0dbba0 msgr2=0x7f5f1c0ea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5f1c0dbba0 0x7f5f1c0ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 -- 172.21.2.111:0/1966526465 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c03c180 con 0x7f5f1c041060 2021-04-03T03:09:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f227fc700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c041060 0x7f5f1c03b530 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f5f0c002f30 tx=0x7f5f0c006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f1bfff700 1 -- 172.21.2.111:0/1966526465 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f0c00d710 con 0x7f5f1c041060 2021-04-03T03:09:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f1bfff700 1 -- 172.21.2.111:0/1966526465 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c000e90 con 0x7f5f1c041060 2021-04-03T03:09:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f1bfff700 1 -- 172.21.2.111:0/1966526465 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f0c00da60 con 0x7f5f1c041060 2021-04-03T03:09:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c0e6fd0 con 0x7f5f1c041060 2021-04-03T03:09:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c03a760 con 0x7f5f1c041060 2021-04-03T03:09:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c004db0 con 0x7f5f1c041060 2021-04-03T03:09:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f1bfff700 1 -- 172.21.2.111:0/1966526465 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5f0c018020 con 0x7f5f1c041060 2021-04-03T03:09:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f1bfff700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5f08056f70 0x7f5f08059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f1bfff700 1 -- 172.21.2.111:0/1966526465 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5f0c070150 con 0x7f5f1c041060 2021-04-03T03:09:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f22ffd700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5f08056f70 0x7f5f08059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.711+0000 7f5f22ffd700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5f08056f70 0x7f5f08059430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5f14002f30 tx=0x7f5f14015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.715+0000 7f5f1bfff700 1 -- 172.21.2.111:0/1966526465 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5f0c045420 con 0x7f5f1c041060 2021-04-03T03:09:02.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.807+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f1c0083c0 con 0x7f5f1c041060 2021-04-03T03:09:02.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.807+0000 7f5f1bfff700 1 -- 172.21.2.111:0/1966526465 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5f0c048a40 con 0x7f5f1c041060 2021-04-03T03:09:02.817 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:02.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5f08056f70 msgr2=0x7f5f08059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5f08056f70 0x7f5f08059430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5f14002f30 tx=0x7f5f14015000).stop 2021-04-03T03:09:02.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c041060 msgr2=0x7f5f1c03b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c041060 0x7f5f1c03b530 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f5f0c002f30 tx=0x7f5f0c006e70).stop 2021-04-03T03:09:02.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 shutdown_connections 2021-04-03T03:09:02.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5f08056f70 0x7f5f08059430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5f1c0dbba0 0x7f5f1c0ea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5f1c03d560 0x7f5f1c0d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 --2- 172.21.2.111:0/1966526465 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5f1c041060 0x7f5f1c03b530 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 >> 172.21.2.111:0/1966526465 conn(0x7f5f1c0462b0 msgr2=0x7f5f1c0055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 shutdown_connections 2021-04-03T03:09:02.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.811+0000 7f5f23fff700 1 -- 172.21.2.111:0/1966526465 wait complete. 2021-04-03T03:09:02.825 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:09:02.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 Processor -- start 2021-04-03T03:09:02.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- start start 2021-04-03T03:09:02.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa5340603f0 0x7fa5341447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5340607d0 con 0x7fa5340b17b0 2021-04-03T03:09:02.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53414b090 con 0x7fa5340add50 2021-04-03T03:09:02.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53414b330 con 0x7fa5340603f0 2021-04-03T03:09:02.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa532ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa5340603f0 0x7fa5341447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa532ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa5340603f0 0x7fa5341447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50422/0 (socket says 172.21.2.111:50422) 2021-04-03T03:09:02.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa532ffd700 1 -- 172.21.2.111:0/2572463831 learned_addr learned my addr 172.21.2.111:0/2572463831 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa5337fe700 1 --1- 172.21.2.111:0/2572463831 >> v1:172.21.2.111:6789/0 conn(0x7fa5340b17b0 0x7fa534147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51214/0 (socket says 172.21.2.111:51214) 2021-04-03T03:09:02.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa532ffd700 1 -- 172.21.2.111:0/2572463831 >> v1:172.21.2.115:6789/0 conn(0x7fa5340add50 legacy=0x7fa534147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:02.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa532ffd700 1 -- 172.21.2.111:0/2572463831 >> v1:172.21.2.111:6789/0 conn(0x7fa5340b17b0 legacy=0x7fa534147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa532ffd700 1 -- 172.21.2.111:0/2572463831 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53414b5d0 con 0x7fa5340603f0 2021-04-03T03:09:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa532ffd700 1 --2- 172.21.2.111:0/2572463831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340603f0 0x7fa5341447e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fa5340b5600 tx=0x7fa524006e90).ready entity=mon.0 client_cookie=1119eee8b6c05be6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa531ffb700 1 -- 172.21.2.111:0/2572463831 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa524009980 con 0x7fa5340603f0 2021-04-03T03:09:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa531ffb700 1 -- 172.21.2.111:0/2572463831 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa524009ae0 con 0x7fa5340603f0 2021-04-03T03:09:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa531ffb700 1 -- 172.21.2.111:0/2572463831 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa524009e80 con 0x7fa5340603f0 2021-04-03T03:09:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- 172.21.2.111:0/2572463831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340603f0 msgr2=0x7fa5341447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 --2- 172.21.2.111:0/2572463831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340603f0 0x7fa5341447e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fa5340b5600 tx=0x7fa524006e90).stop 2021-04-03T03:09:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- 172.21.2.111:0/2572463831 shutdown_connections 2021-04-03T03:09:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 --2- 172.21.2.111:0/2572463831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340603f0 0x7fa5341447e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- 172.21.2.111:0/2572463831 >> 172.21.2.111:0/2572463831 conn(0x7fa5340b61c0 msgr2=0x7fa53405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- 172.21.2.111:0/2572463831 shutdown_connections 2021-04-03T03:09:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.875+0000 7fa533fff700 1 -- 172.21.2.111:0/2572463831 wait complete. 2021-04-03T03:09:02.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 Processor -- start 2021-04-03T03:09:02.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 -- start start 2021-04-03T03:09:02.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5340603f0 0x7fa5340aa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5340add50 0x7fa5340b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340b17b0 0x7fa534144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340a8d70 con 0x7fa5340b17b0 2021-04-03T03:09:02.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534156a80 con 0x7fa5340603f0 2021-04-03T03:09:02.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa534066a70 con 0x7fa5340add50 2021-04-03T03:09:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340b17b0 0x7fa534144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340b17b0 0x7fa534144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50432/0 (socket says 172.21.2.111:50432) 2021-04-03T03:09:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 -- 172.21.2.111:0/3893349049 learned_addr learned my addr 172.21.2.111:0/3893349049 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 -- 172.21.2.111:0/3893349049 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5340add50 msgr2=0x7fa5340b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5340add50 0x7fa5340b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 -- 172.21.2.111:0/3893349049 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5340603f0 msgr2=0x7fa5340aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5340603f0 0x7fa5340aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 -- 172.21.2.111:0/3893349049 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534066d10 con 0x7fa5340b17b0 2021-04-03T03:09:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa5337fe700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340b17b0 0x7fa534144910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fa5340ae520 tx=0x7fa52c007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa52bfff700 1 -- 172.21.2.111:0/3893349049 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa52c0058b0 con 0x7fa5340b17b0 2021-04-03T03:09:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa52bfff700 1 -- 172.21.2.111:0/3893349049 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c005a10 con 0x7fa5340b17b0 2021-04-03T03:09:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa534066fb0 con 0x7fa5340b17b0 2021-04-03T03:09:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa52bfff700 1 -- 172.21.2.111:0/3893349049 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa52c005d70 con 0x7fa5340b17b0 2021-04-03T03:09:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5340a6fa0 con 0x7fa5340b17b0 2021-04-03T03:09:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa534067140 con 0x7fa5340b17b0 2021-04-03T03:09:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa52bfff700 1 -- 172.21.2.111:0/3893349049 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa52c010030 con 0x7fa5340b17b0 2021-04-03T03:09:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa52bfff700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa518056ea0 0x7fa518059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa52bfff700 1 -- 172.21.2.111:0/3893349049 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fa52c070140 con 0x7fa5340b17b0 2021-04-03T03:09:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa532ffd700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa518056ea0 0x7fa518059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.879+0000 7fa532ffd700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa518056ea0 0x7fa518059360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa5340b5600 tx=0x7fa524015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.883+0000 7fa52bfff700 1 -- 172.21.2.111:0/3893349049 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa52c0454c0 con 0x7fa5340b17b0 2021-04-03T03:09:02.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.975+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa534057660 con 0x7fa5340b17b0 2021-04-03T03:09:02.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.975+0000 7fa52bfff700 1 -- 172.21.2.111:0/3893349049 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa52c048ae0 con 0x7fa5340b17b0 2021-04-03T03:09:02.984 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:02.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.975+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa518056ea0 msgr2=0x7fa518059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.975+0000 7fa533fff700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa518056ea0 0x7fa518059360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa5340b5600 tx=0x7fa524015000).stop 2021-04-03T03:09:02.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.975+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340b17b0 msgr2=0x7fa534144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:02.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.975+0000 7fa533fff700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340b17b0 0x7fa534144910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fa5340ae520 tx=0x7fa52c007f50).stop 2021-04-03T03:09:02.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 shutdown_connections 2021-04-03T03:09:02.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5340b17b0 0x7fa534144910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa518056ea0 0x7fa518059360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:02.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5340603f0 0x7fa5340aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 --2- 172.21.2.111:0/3893349049 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5340add50 0x7fa5340b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:02.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 >> 172.21.2.111:0/3893349049 conn(0x7fa5340b61c0 msgr2=0x7fa5341457d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:02.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 shutdown_connections 2021-04-03T03:09:02.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:02.979+0000 7fa533fff700 1 -- 172.21.2.111:0/3893349049 wait complete. 2021-04-03T03:09:02.998 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2021-04-03T03:09:02.999 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2021-04-03T03:09:03.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 Processor -- start 2021-04-03T03:09:03.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- start start 2021-04-03T03:09:03.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc0b12b0 0x7f6cbc1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0607d0 con 0x7f6cbc0603f0 2021-04-03T03:09:03.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0b0eb0 con 0x7f6cbc14b700 2021-04-03T03:09:03.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0af400 con 0x7f6cbc0b12b0 2021-04-03T03:09:03.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1c19700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6cbc0603f0 0x7f6cbc0b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51228/0 (socket says 172.21.2.111:51228) 2021-04-03T03:09:03.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1c19700 1 -- 172.21.2.111:0/227678195 learned_addr learned my addr 172.21.2.111:0/227678195 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:03.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678217042 0 0) 0x7f6cbc0607d0 con 0x7f6cbc0603f0 2021-04-03T03:09:03.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca4003240 con 0x7f6cbc0603f0 2021-04-03T03:09:03.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 691015005 0 0) 0x7f6ca4003240 con 0x7f6cbc0603f0 2021-04-03T03:09:03.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc0b12b0 msgr2=0x7f6cbc1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:03.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc0b12b0 0x7f6cbc1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 >> v1:172.21.2.115:6789/0 conn(0x7f6cbc14b700 legacy=0x7f6cbc0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:03.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0af6a0 con 0x7f6cbc0603f0 2021-04-03T03:09:03.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f6cb00016d0 con 0x7f6cbc0603f0 2021-04-03T03:09:03.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 >> v1:172.21.2.111:6789/0 conn(0x7f6cbc0603f0 legacy=0x7f6cbc0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc0b12b0 0x7f6ca40037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ca4003d20 0x7f6ca40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ca40049d0 0x7f6ca4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0af400 con 0x7f6ca4003d20 2021-04-03T03:09:03.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40042b0 con 0x7f6ca40049d0 2021-04-03T03:09:03.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca400b730 con 0x7f6cbc0b12b0 2021-04-03T03:09:03.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1418700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ca4003d20 0x7f6ca40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:03.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1418700 1 -- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc0b12b0 msgr2=0x7f6ca40037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:03.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1418700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc0b12b0 0x7f6ca40037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1418700 1 -- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ca40049d0 msgr2=0x7f6ca4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:03.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1418700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ca40049d0 0x7f6ca4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1418700 1 -- 172.21.2.111:0/227678195 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4003240 con 0x7f6ca4003d20 2021-04-03T03:09:03.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc1418700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ca4003d20 0x7f6ca40044e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f6cb8001f60 tx=0x7f6cb8007b70).ready entity=mon.0 client_cookie=628fee8ea4af8e09 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:03.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb800c810 con 0x7f6ca4003d20 2021-04-03T03:09:03.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb800c970 con 0x7f6ca4003d20 2021-04-03T03:09:03.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc0c17700 1 -- 172.21.2.111:0/227678195 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb800ccd0 con 0x7f6ca4003d20 2021-04-03T03:09:03.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/227678195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ca4003d20 msgr2=0x7f6ca40044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 --2- 172.21.2.111:0/227678195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ca4003d20 0x7f6ca40044e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f6cb8001f60 tx=0x7f6cb8007b70).stop 2021-04-03T03:09:03.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc241a700 1 -- 172.21.2.111:0/227678195 reap_dead start 2021-04-03T03:09:03.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/227678195 shutdown_connections 2021-04-03T03:09:03.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/227678195 >> 172.21.2.111:0/227678195 conn(0x7f6cbc0b61c0 msgr2=0x7f6cbc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:03.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/227678195 shutdown_connections 2021-04-03T03:09:03.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.055+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/227678195 wait complete. 2021-04-03T03:09:03.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 Processor -- start 2021-04-03T03:09:03.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 -- start start 2021-04-03T03:09:03.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc062bf0 0x7f6cbc1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6cbc0aa0b0 0x7f6cbc1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6cbc156ec0 0x7f6cbc159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0ab950 con 0x7f6cbc156ec0 2021-04-03T03:09:03.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc062fd0 con 0x7f6cbc0aa0b0 2021-04-03T03:09:03.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0640a0 con 0x7f6cbc062bf0 2021-04-03T03:09:03.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6cbc156ec0 0x7f6cbc159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:03.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6cbc156ec0 0x7f6cbc159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50452/0 (socket says 172.21.2.111:50452) 2021-04-03T03:09:03.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 -- 172.21.2.111:0/2291127585 learned_addr learned my addr 172.21.2.111:0/2291127585 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:03.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 -- 172.21.2.111:0/2291127585 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc062bf0 msgr2=0x7f6cbc1564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc062bf0 0x7f6cbc1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 -- 172.21.2.111:0/2291127585 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6cbc0aa0b0 msgr2=0x7f6cbc1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6cbc0aa0b0 0x7f6cbc1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 -- 172.21.2.111:0/2291127585 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc064340 con 0x7f6cbc156ec0 2021-04-03T03:09:03.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc241a700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6cbc156ec0 0x7f6cbc159a30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f6cb00010d0 tx=0x7f6cb400a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:03.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6caeffd700 1 -- 172.21.2.111:0/2291127585 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb4005c10 con 0x7f6cbc156ec0 2021-04-03T03:09:03.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6caeffd700 1 -- 172.21.2.111:0/2291127585 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4005d70 con 0x7f6cbc156ec0 2021-04-03T03:09:03.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc0645e0 con 0x7f6cbc156ec0 2021-04-03T03:09:03.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc064d20 con 0x7f6cbc156ec0 2021-04-03T03:09:03.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6caeffd700 1 -- 172.21.2.111:0/2291127585 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cb4007260 con 0x7f6cbc156ec0 2021-04-03T03:09:03.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc064a10 con 0x7f6cbc156ec0 2021-04-03T03:09:03.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6caeffd700 1 -- 172.21.2.111:0/2291127585 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6cb401b060 con 0x7f6cbc156ec0 2021-04-03T03:09:03.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6caeffd700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c9c056ea0 0x7f6c9c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6caeffd700 1 -- 172.21.2.111:0/2291127585 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f6cb406f940 con 0x7f6cbc156ec0 2021-04-03T03:09:03.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc1c19700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c9c056ea0 0x7f6c9c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:03.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.059+0000 7f6cc1c19700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c9c056ea0 0x7f6c9c059360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f6cbc0620f0 tx=0x7f6cb00014a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:03.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.063+0000 7f6caeffd700 1 -- 172.21.2.111:0/2291127585 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6cb4044d20 con 0x7f6cbc156ec0 2021-04-03T03:09:03.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.191+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f6cbc057660 con 0x7f6cbc156ec0 2021-04-03T03:09:03.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.791+0000 7f6caeffd700 1 -- 172.21.2.111:0/2291127585 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v37) v1 ==== 142+0+0 (secure 0 0 0) 0x7f6cb4048340 con 0x7f6cbc156ec0 2021-04-03T03:09:03.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c9c056ea0 msgr2=0x7f6c9c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c9c056ea0 0x7f6c9c059360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f6cbc0620f0 tx=0x7f6cb00014a0).stop 2021-04-03T03:09:03.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6cbc156ec0 msgr2=0x7f6cbc159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6cbc156ec0 0x7f6cbc159a30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f6cb00010d0 tx=0x7f6cb400a060).stop 2021-04-03T03:09:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 shutdown_connections 2021-04-03T03:09:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6cbc156ec0 0x7f6cbc159a30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6cbc062bf0 0x7f6cbc1564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c9c056ea0 0x7f6c9c059360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 --2- 172.21.2.111:0/2291127585 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6cbc0aa0b0 0x7f6cbc1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 >> 172.21.2.111:0/2291127585 conn(0x7f6cbc0b61c0 msgr2=0x7f6cbc0b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 shutdown_connections 2021-04-03T03:09:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.795+0000 7f6cc2c1b700 1 -- 172.21.2.111:0/2291127585 wait complete. 2021-04-03T03:09:03.810 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:09:03.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 Processor -- start 2021-04-03T03:09:03.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 -- start start 2021-04-03T03:09:03.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f47ec03d560 0x7f47ec0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec0dc560 con 0x7f47ec041060 2021-04-03T03:09:03.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec0dc800 con 0x7f47ec0dbba0 2021-04-03T03:09:03.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ec03de60 con 0x7f47ec03d560 2021-04-03T03:09:03.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f8f85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f47ec03d560 0x7f47ec0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:03.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f8f85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f47ec03d560 0x7f47ec0dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50458/0 (socket says 172.21.2.111:50458) 2021-04-03T03:09:03.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f8f85700 1 -- 172.21.2.111:0/677786109 learned_addr learned my addr 172.21.2.111:0/677786109 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:03.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f8f85700 1 -- 172.21.2.111:0/677786109 >> v1:172.21.2.115:6789/0 conn(0x7f47ec0dbba0 legacy=0x7f47ec03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:03.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f8f85700 1 -- 172.21.2.111:0/677786109 >> v1:172.21.2.111:6789/0 conn(0x7f47ec041060 legacy=0x7f47ec041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f8f85700 1 -- 172.21.2.111:0/677786109 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec03da30 con 0x7f47ec03d560 2021-04-03T03:09:03.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f8f85700 1 --2- 172.21.2.111:0/677786109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec03d560 0x7f47ec0dbf80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f47ec040a80 tx=0x7f47f405ad70).ready entity=mon.0 client_cookie=ae9e81ac962f81ab server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:03.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f2ffd700 1 -- 172.21.2.111:0/677786109 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f405bda0 con 0x7f47ec03d560 2021-04-03T03:09:03.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f2ffd700 1 -- 172.21.2.111:0/677786109 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f405df60 con 0x7f47ec03d560 2021-04-03T03:09:03.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f2ffd700 1 -- 172.21.2.111:0/677786109 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f40699f0 con 0x7f47ec03d560 2021-04-03T03:09:03.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 -- 172.21.2.111:0/677786109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec03d560 msgr2=0x7f47ec0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.859+0000 7f47f9786700 1 --2- 172.21.2.111:0/677786109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec03d560 0x7f47ec0dbf80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f47ec040a80 tx=0x7f47f405ad70).stop 2021-04-03T03:09:03.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- 172.21.2.111:0/677786109 shutdown_connections 2021-04-03T03:09:03.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 --2- 172.21.2.111:0/677786109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec03d560 0x7f47ec0dbf80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:03.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- 172.21.2.111:0/677786109 >> 172.21.2.111:0/677786109 conn(0x7f47ec0462b0 msgr2=0x7f47ec0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:03.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- 172.21.2.111:0/677786109 shutdown_connections 2021-04-03T03:09:03.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- 172.21.2.111:0/677786109 wait complete. 2021-04-03T03:09:03.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 Processor -- start 2021-04-03T03:09:03.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- start start 2021-04-03T03:09:03.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f47ec03d560 0x7f47ec03abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec041060 0x7f47ec03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f47ec0dbba0 0x7f47ec0e8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec03bd00 con 0x7f47ec041060 2021-04-03T03:09:03.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec03bfa0 con 0x7f47ec0dbba0 2021-04-03T03:09:03.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec039aa0 con 0x7f47ec03d560 2021-04-03T03:09:03.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec041060 0x7f47ec03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:03.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec041060 0x7f47ec03b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50462/0 (socket says 172.21.2.111:50462) 2021-04-03T03:09:03.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 -- 172.21.2.111:0/2058587215 learned_addr learned my addr 172.21.2.111:0/2058587215 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:03.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 -- 172.21.2.111:0/2058587215 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f47ec03d560 msgr2=0x7f47ec03abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f47ec03d560 0x7f47ec03abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 -- 172.21.2.111:0/2058587215 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f47ec0dbba0 msgr2=0x7f47ec0e8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f47ec0dbba0 0x7f47ec0e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 -- 172.21.2.111:0/2058587215 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec03a960 con 0x7f47ec041060 2021-04-03T03:09:03.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f37fe700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec041060 0x7f47ec03b690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f47ec0087d0 tx=0x7f47e4007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:03.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f17fa700 1 -- 172.21.2.111:0/2058587215 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47e400e4a0 con 0x7f47ec041060 2021-04-03T03:09:03.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f17fa700 1 -- 172.21.2.111:0/2058587215 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e400e600 con 0x7f47ec041060 2021-04-03T03:09:03.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f17fa700 1 -- 172.21.2.111:0/2058587215 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47e400e960 con 0x7f47ec041060 2021-04-03T03:09:03.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec0e8570 con 0x7f47ec041060 2021-04-03T03:09:03.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ec03a150 con 0x7f47ec041060 2021-04-03T03:09:03.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ec004db0 con 0x7f47ec041060 2021-04-03T03:09:03.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f17fa700 1 -- 172.21.2.111:0/2058587215 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f47e4019030 con 0x7f47ec041060 2021-04-03T03:09:03.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f17fa700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f47d40570a0 0x7f47d4059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:03.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f17fa700 1 -- 172.21.2.111:0/2058587215 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f47e406f970 con 0x7f47ec041060 2021-04-03T03:09:03.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.863+0000 7f47f3fff700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f47d40570a0 0x7f47d4059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:03.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.867+0000 7f47f3fff700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f47d40570a0 0x7f47d4059560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f47e8002eb0 tx=0x7f47e80068f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:03.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.867+0000 7f47f17fa700 1 -- 172.21.2.111:0/2058587215 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f47e4044c40 con 0x7f47ec041060 2021-04-03T03:09:03.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.963+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f47ec03b150 con 0x7f47ec041060 2021-04-03T03:09:03.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f17fa700 1 -- 172.21.2.111:0/2058587215 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f47e4048260 con 0x7f47ec041060 2021-04-03T03:09:03.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f47d40570a0 msgr2=0x7f47d4059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f47d40570a0 0x7f47d4059560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f47e8002eb0 tx=0x7f47e80068f0).stop 2021-04-03T03:09:03.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec041060 msgr2=0x7f47ec03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:03.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec041060 0x7f47ec03b690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f47ec0087d0 tx=0x7f47e4007c30).stop 2021-04-03T03:09:03.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 shutdown_connections 2021-04-03T03:09:03.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f47d40570a0 0x7f47d4059560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:03.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f47ec0dbba0 0x7f47ec0e8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f47ec03d560 0x7f47ec03abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:03.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 --2- 172.21.2.111:0/2058587215 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f47ec041060 0x7f47ec03b690 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:03.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 >> 172.21.2.111:0/2058587215 conn(0x7f47ec0462b0 msgr2=0x7f47ec041550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:03.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 shutdown_connections 2021-04-03T03:09:03.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:03.967+0000 7f47f9786700 1 -- 172.21.2.111:0/2058587215 wait complete. 2021-04-03T03:09:03.982 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:09:03.984 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:09:03.990 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:09:04.015 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:09:04.027 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:09:04.038 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:09:04.050 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:09:04.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.051 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.052 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:04.074 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:09:04.075 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:09:04.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.075 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.076 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:04.127 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:09:04.128 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:09:04.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.129 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.129 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:04.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:09:04.178 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-04-03T03:09:04.178 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-04-03T03:09:04.179 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:09:04.179 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:09:04.180 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:04.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.269 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.270 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:04.317 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:09:04.318 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:09:04.319 INFO:teuthology.orchestra.run:waiting for 900 2021-04-03T03:09:04.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:09:04.320 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:04.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.375 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.375 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:09:04.386 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:09:04.387 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:09:04.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.388 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.389 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:09:04.439 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:09:04.441 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:09:04.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.442 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.442 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:09:04.490 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2021-04-03T03:09:04.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:09:04.491 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2021-04-03T03:09:04.492 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:09:04.492 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:09:04.493 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:09:04.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.577 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.577 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:09:04.626 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:09:04.627 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:09:04.627 INFO:teuthology.orchestra.run:waiting for 900 2021-04-03T03:09:04.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:09:04.628 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:09:04.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.681 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.682 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:09:04.692 INFO:teuthology.orchestra.run.gibba031.stdout:fuseblk 2021-04-03T03:09:04.693 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2021-04-03T03:09:04.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.694 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.695 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:09:04.746 INFO:teuthology.orchestra.run.gibba031.stdout:fuseblk 2021-04-03T03:09:04.747 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2021-04-03T03:09:04.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.748 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.748 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:09:04.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:09:04.796 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': Transport endpoint is not connected 2021-04-03T03:09:04.796 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.2 2021-04-03T03:09:04.796 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba031.front.sepia.ceph.com... 2021-04-03T03:09:04.797 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:09:04.797 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:09:04.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:04.891 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:04.892 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:09:04.901 INFO:teuthology.orchestra.run.gibba031.stdout:ext2/ext3 2021-04-03T03:09:04.902 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:09:04.902 INFO:teuthology.orchestra.run:waiting for 900 2021-04-03T03:09:04.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:09:04.903 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:09:04.958 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:09:05.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f5100cb9700 1 Processor -- start 2021-04-03T03:09:05.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f5100cb9700 1 -- start start 2021-04-03T03:09:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f5100cb9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f50f40d75f0 0x7f50f40d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f5100cb9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f403c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f5100cb9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f403f930 con 0x7f50f403cce0 2021-04-03T03:09:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f5100cb9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f40d8390 con 0x7f50f40d75f0 2021-04-03T03:09:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f5100cb9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40402d0 con 0x7f50f403d6a0 2021-04-03T03:09:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fb7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f50f40d75f0 0x7f50f40d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fbfff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f50f403cce0 0x7f50f40d3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51264/0 (socket says 172.21.2.111:51264) 2021-04-03T03:09:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fbfff700 1 -- 172.21.2.111:0/150889887 learned_addr learned my addr 172.21.2.111:0/150889887 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50faffd700 1 --2- 172.21.2.111:0/150889887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f403c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fb7fe700 1 -- 172.21.2.111:0/150889887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 msgr2=0x7f50f403c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fb7fe700 1 --2- 172.21.2.111:0/150889887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f403c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fb7fe700 1 -- 172.21.2.111:0/150889887 >> v1:172.21.2.111:6789/0 conn(0x7f50f403cce0 legacy=0x7f50f40d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fb7fe700 1 -- 172.21.2.111:0/150889887 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f403d130 con 0x7f50f40d75f0 2021-04-03T03:09:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50faffd700 1 --2- 172.21.2.111:0/150889887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f403c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.011+0000 7f50fb7fe700 1 --2- 172.21.2.111:0/150889887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d7ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f50fc069660 tx=0x7f50e8006dd0).ready entity=mon.0 client_cookie=e640f00a499e3f35 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fa7fc700 1 -- 172.21.2.111:0/150889887 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50e80096f0 con 0x7f50f40d75f0 2021-04-03T03:09:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fa7fc700 1 -- 172.21.2.111:0/150889887 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e8009850 con 0x7f50f40d75f0 2021-04-03T03:09:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fa7fc700 1 -- 172.21.2.111:0/150889887 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50e8009bb0 con 0x7f50f40d75f0 2021-04-03T03:09:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/150889887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 msgr2=0x7f50f40d7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 --2- 172.21.2.111:0/150889887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d7ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f50fc069660 tx=0x7f50e8006dd0).stop 2021-04-03T03:09:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/150889887 shutdown_connections 2021-04-03T03:09:05.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 --2- 172.21.2.111:0/150889887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d7ea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 --2- 172.21.2.111:0/150889887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f403c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/150889887 >> 172.21.2.111:0/150889887 conn(0x7f50f40462c0 msgr2=0x7f50f40466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:05.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/150889887 shutdown_connections 2021-04-03T03:09:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/150889887 wait complete. 2021-04-03T03:09:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 Processor -- start 2021-04-03T03:09:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- start start 2021-04-03T03:09:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50f403cce0 0x7f50f4044cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f40d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40cfff0 con 0x7f50f40d75f0 2021-04-03T03:09:05.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f403c4c0 con 0x7f50f403cce0 2021-04-03T03:09:05.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f403b970 con 0x7f50f403d6a0 2021-04-03T03:09:05.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50480/0 (socket says 172.21.2.111:50480) 2021-04-03T03:09:05.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 -- 172.21.2.111:0/1894422587 learned_addr learned my addr 172.21.2.111:0/1894422587 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:05.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50faffd700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f40d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fb7fe700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50f403cce0 0x7f50f4044cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 -- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 msgr2=0x7f50f40d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f40d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 -- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50f403cce0 msgr2=0x7f50f4044cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50f403cce0 0x7f50f4044cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 -- 172.21.2.111:0/1894422587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f40d1f30 con 0x7f50f40d75f0 2021-04-03T03:09:05.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fbfff700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d5360 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f50fc068af0 tx=0x7f50fc05ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50f8ff9700 1 -- 172.21.2.111:0/1894422587 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50fc05dfc0 con 0x7f50f40d75f0 2021-04-03T03:09:05.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50f8ff9700 1 -- 172.21.2.111:0/1894422587 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc0699f0 con 0x7f50f40d75f0 2021-04-03T03:09:05.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f4039590 con 0x7f50f40d75f0 2021-04-03T03:09:05.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50f8ff9700 1 -- 172.21.2.111:0/1894422587 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50fc06b5d0 con 0x7f50f40d75f0 2021-04-03T03:09:05.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f40d59e0 con 0x7f50f40d75f0 2021-04-03T03:09:05.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f4039d80 con 0x7f50f40d75f0 2021-04-03T03:09:05.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50f8ff9700 1 -- 172.21.2.111:0/1894422587 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f50fc061110 con 0x7f50f40d75f0 2021-04-03T03:09:05.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50f8ff9700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f50e4057050 0x7f50e4059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50f8ff9700 1 -- 172.21.2.111:0/1894422587 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f50fc0de850 con 0x7f50f40d75f0 2021-04-03T03:09:05.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.015+0000 7f50fb7fe700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f50e4057050 0x7f50e4059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.019+0000 7f50fb7fe700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f50e4057050 0x7f50e4059510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f50e8017000 tx=0x7f50e8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.019+0000 7f50f8ff9700 1 -- 172.21.2.111:0/1894422587 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f50fc0b3c30 con 0x7f50f40d75f0 2021-04-03T03:09:05.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f50f40d20c0 con 0x7f50f40d75f0 2021-04-03T03:09:05.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f50f8ff9700 1 -- 172.21.2.111:0/1894422587 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+34065 (secure 0 0 0) 0x7f50fc08a020 con 0x7f50f40d75f0 2021-04-03T03:09:05.121 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:05.121 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:09:05.121 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 24, 2021-04-03T03:09:05.121 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:09:05.122 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:09:05.122 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:08:45.426626+0000", 2021-04-03T03:09:05.122 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:09:05.122 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:09:05.122 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:09:05.123 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:09:05.123 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:09:05.123 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:09:05.123 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:09:05.124 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:09:05.124 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:09:05.124 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:09:05.124 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:09:05.124 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:09:05.125 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:09:05.125 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:09:05.125 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:09:05.125 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 4, 2021-04-03T03:09:05.126 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:09:05.126 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:09:05.126 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:09:05.126 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:09:05.126 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:09:05.127 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.127 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:09:05.127 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:09:05.127 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:09:05.127 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:09:05.128 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:09:05.128 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:09:05.128 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:09:05.128 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:09:05.129 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:09:05.129 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:09:05.129 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:09:05.129 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:09:05.129 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:09:05.130 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:09:05.130 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:09:05.130 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:09:05.130 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:09:05.131 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:09:05.131 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:09:05.131 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:09:05.131 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:09:05.131 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:09:05.132 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:09:05.132 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.132 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:09:05.132 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:09:05.133 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:09:05.133 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:09:05.133 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:09:05.133 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:09:05.133 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:09:05.134 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:09:05.134 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:09:05.134 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:09:05.134 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:09:05.134 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:09:05.135 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:09:05.135 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:09:05.135 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:09:05.135 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:09:05.136 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:09:05.136 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:09:05.136 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:09:05.136 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:09:05.136 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:09:05.137 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:09:05.137 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:09:05.137 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:09:05.137 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:09:05.137 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:09:05.138 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:09:05.138 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.138 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:09:05.138 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:09:05.138 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:09:05.139 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:09:05.139 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:09:05.139 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:09:05.139 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:09:05.140 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:09:05.140 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:09:05.140 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:09:05.140 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:09:05.140 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:09:05.141 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:09:05.141 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.141 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:09:05.141 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:09:05.141 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.142 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.142 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.142 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:09:05.142 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:09:05.143 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:09:05.143 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:09:05.143 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:09:05.143 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:09:05.143 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:09:05.144 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:09:05.144 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:09:05.144 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:09:05.144 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:09:05.145 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:09:05.145 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:09:05.145 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:09:05.145 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:09:05.145 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:09:05.146 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:09:05.146 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:09:05.146 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:09:05.146 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:09:05.146 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:09:05.147 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:09:05.147 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:09:05.147 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.147 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:09:05.148 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:09:05.148 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:09:05.148 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:09:05.148 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:09:05.148 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:09:05.149 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:09:05.149 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:09:05.149 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:09:05.149 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:09:05.149 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:09:05.150 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:09:05.150 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:09:05.150 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:09:05.150 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:09:05.151 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:09:05.151 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:09:05.151 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:09:05.151 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:09:05.151 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:09:05.152 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:09:05.152 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:09:05.152 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:09:05.152 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:09:05.152 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:09:05.153 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:09:05.153 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:09:05.153 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.153 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:09:05.153 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:09:05.154 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:09:05.154 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:09:05.154 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:09:05.154 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:09:05.155 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:09:05.155 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:09:05.155 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:09:05.155 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:09:05.155 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:09:05.156 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:09:05.156 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:09:05.156 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:09:05.156 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.156 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.157 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.157 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 3, 2021-04-03T03:09:05.157 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:09:05.157 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:40.074197+0000", 2021-04-03T03:09:05.158 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:09:05.158 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:09:05.158 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:09:05.158 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:09:05.158 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:09:05.159 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:09:05.159 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:09:05.159 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:09:05.159 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:09:05.159 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:09:05.160 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:09:05.160 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:09:05.160 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:09:05.160 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:09:05.160 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:09:05.161 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:09:05.161 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:09:05.161 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:09:05.161 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:09:05.162 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:09:05.162 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.162 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "23", 2021-04-03T03:09:05.162 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:09:05.162 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:09:05.163 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:09:05.163 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:09:05.163 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:09:05.163 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:09:05.163 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:09:05.164 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:09:05.164 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:09:05.164 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:09:05.164 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:09:05.165 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:09:05.165 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:09:05.165 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:09:05.165 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:09:05.165 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:09:05.166 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:09:05.166 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:09:05.166 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:09:05.166 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:09:05.166 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:09:05.167 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:09:05.167 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:09:05.167 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:09:05.167 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:09:05.167 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:09:05.168 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.168 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:09:05.168 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:09:05.168 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:09:05.169 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:09:05.169 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:09:05.169 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:09:05.169 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:09:05.169 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:09:05.170 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:09:05.170 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:09:05.170 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:09:05.170 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:09:05.170 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:09:05.171 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:09:05.171 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:09:05.171 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.171 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:09:05.172 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:09:05.172 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:09:05.172 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.172 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.172 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.173 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.173 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 4, 2021-04-03T03:09:05.173 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:09:05.173 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:42.209560+0000", 2021-04-03T03:09:05.174 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 5, 2021-04-03T03:09:05.174 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool,ec_overwrites", 2021-04-03T03:09:05.174 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 3, 2021-04-03T03:09:05.174 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 4, 2021-04-03T03:09:05.174 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 3, 2021-04-03T03:09:05.175 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 1, 2021-04-03T03:09:05.175 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:09:05.175 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:09:05.175 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:09:05.176 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:09:05.176 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:09:05.176 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:09:05.176 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:09:05.176 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:09:05.177 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:09:05.177 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:09:05.177 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:09:05.177 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:09:05.177 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:09:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:09:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "24", 2021-04-03T03:09:05.178 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:09:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:09:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:09:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:09:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:09:05.179 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:09:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:09:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:09:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:09:05.180 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:09:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:09:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:09:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:09:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:09:05.181 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:09:05.182 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:09:05.182 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:09:05.182 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:09:05.182 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:09:05.182 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:09:05.183 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:09:05.183 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:09:05.183 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:09:05.183 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "cephfs_data", 2021-04-03T03:09:05.184 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:09:05.184 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:09:05.184 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.184 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:09:05.184 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:09:05.185 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:09:05.185 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:09:05.185 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:09:05.185 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:09:05.186 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:09:05.186 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:09:05.186 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 8192, 2021-04-03T03:09:05.186 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:09:05.186 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:09:05.187 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:09:05.187 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:09:05.187 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:09:05.187 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:09:05.187 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.188 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.188 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.188 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:09:05.188 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:09:05.188 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.189 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:09:05.189 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:09:05.189 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.189 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.190 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.190 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.190 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.190 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.190 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.191 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 21, 2021-04-03T03:09:05.191 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.191 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.191 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.192 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.192 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.192 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.192 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:09:05.192 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.193 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.193 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.193 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.193 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:09:05.193 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.194 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.194 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.194 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.194 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.194 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.195 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.195 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.195 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:09:05.195 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.196 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.196 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.196 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.196 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:09:05.196 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.197 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.197 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.197 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.197 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.198 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.198 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.198 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.198 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:09:05.198 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.199 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.199 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.199 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.199 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:09:05.200 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.200 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.200 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.200 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.200 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.201 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.201 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.201 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.201 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:09:05.201 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.202 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.202 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.202 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.202 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:09:05.202 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:09:05.203 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.203 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.203 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.203 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:09:05.204 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:09:05.204 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:09:05.204 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:09:05.204 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.204 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.205 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.205 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.205 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.205 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.205 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:09:05.206 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:09:05.206 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.206 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.206 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.207 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.207 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.207 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.207 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.207 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 21, 2021-04-03T03:09:05.208 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.208 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.208 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.208 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.208 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.209 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.209 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:09:05.209 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.209 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.210 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.210 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.210 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:09:05.210 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.210 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.211 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.211 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.211 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.211 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.211 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.212 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.212 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:09:05.212 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.212 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.212 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.213 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.213 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:09:05.213 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.213 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.214 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.214 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.214 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.214 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.214 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.215 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.215 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:09:05.215 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.215 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.216 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.216 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.216 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:09:05.216 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.216 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.217 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.217 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.217 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.217 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.217 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.218 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.218 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:09:05.218 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.218 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.219 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.219 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.219 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:09:05.219 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:09:05.219 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.220 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.220 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.220 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:09:05.220 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:09:05.220 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:09:05.221 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:09:05.221 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.221 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.221 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.222 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.222 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.222 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.222 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:09:05.222 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:09:05.223 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.223 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.223 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.223 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.223 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.224 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.224 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.224 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 16, 2021-04-03T03:09:05.224 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.224 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.225 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.225 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.225 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.225 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.226 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:09:05.226 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.226 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.226 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.226 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.227 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:09:05.227 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.227 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.227 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.228 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.228 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.228 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.228 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.228 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.229 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:09:05.229 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.229 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.229 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.230 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.230 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:09:05.230 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.230 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.230 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.231 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.231 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.231 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.231 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.232 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.232 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:09:05.232 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.232 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.232 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.233 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.233 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:09:05.233 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.233 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.233 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.234 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.234 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.234 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.234 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.234 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.235 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:09:05.235 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.235 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.235 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.236 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.236 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:09:05.236 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:09:05.236 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.236 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.237 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.237 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:09:05.237 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:09:05.237 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:09:05.237 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:09:05.238 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.238 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.238 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.238 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.239 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.239 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.239 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:09:05.239 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:09:05.239 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.240 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.240 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.240 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.240 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.240 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.241 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.241 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 21, 2021-04-03T03:09:05.241 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.241 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.242 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.242 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.242 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.242 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.242 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:09:05.243 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.243 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.243 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.243 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.243 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:09:05.244 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.244 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.244 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.244 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.245 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.245 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.245 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.245 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.245 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:09:05.246 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.246 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.246 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.246 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.246 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:09:05.247 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.247 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.247 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.248 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.248 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.248 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.248 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.248 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:09:05.249 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.249 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.249 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.249 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.249 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:09:05.250 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.250 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.250 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.250 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.250 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.251 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.251 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.251 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.251 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:09:05.252 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.252 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.252 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.252 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.252 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:09:05.253 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:09:05.253 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.253 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.253 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.253 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:09:05.254 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:09:05.254 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:09:05.254 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:09:05.254 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.255 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.255 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.255 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.255 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.255 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.256 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:09:05.256 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:09:05.256 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.256 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.257 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.257 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.257 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.257 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.257 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.258 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 21, 2021-04-03T03:09:05.258 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.258 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.258 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.258 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.259 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.259 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.259 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:09:05.259 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.260 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.260 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.260 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.260 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:09:05.260 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.261 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.261 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.261 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.261 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.261 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.262 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.262 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.262 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:09:05.262 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.263 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.263 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.263 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.263 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:09:05.263 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.264 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.264 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.264 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.265 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.265 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.265 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.265 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:09:05.265 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.266 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.266 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.266 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.266 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:09:05.266 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.267 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.267 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.267 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.267 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.267 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.268 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.268 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.268 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:09:05.268 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.269 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.269 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.269 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.269 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:09:05.269 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:09:05.270 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.270 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.270 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.270 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:09:05.270 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:09:05.271 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:09:05.271 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:09:05.271 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.271 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.272 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.272 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.272 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.272 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.272 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:09:05.273 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:09:05.273 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.273 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.273 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.274 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.274 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.274 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.274 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.274 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 18, 2021-04-03T03:09:05.275 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.275 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.275 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.275 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.275 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.276 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.276 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:09:05.276 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.276 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.277 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.277 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.277 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:09:05.277 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.277 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.278 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.278 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.278 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.278 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.278 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.279 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.279 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:09:05.279 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.279 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.280 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.280 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.280 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:09:05.280 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.280 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.281 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.281 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.281 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.281 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.282 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.282 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.282 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:09:05.282 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.282 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.283 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.283 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.283 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:09:05.283 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.284 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.284 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.284 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.284 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.284 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.285 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.285 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.285 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:09:05.285 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:09:05.286 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:09:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.287 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:09:05.288 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:09:05.288 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:09:05.288 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:09:05.288 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.288 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.289 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.289 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.289 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.289 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.289 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:09:05.290 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:09:05.290 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.290 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.290 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.291 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.291 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.291 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.291 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.291 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 21, 2021-04-03T03:09:05.292 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.292 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.292 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.292 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.292 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.293 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.293 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:09:05.293 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.293 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.294 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.294 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.294 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:09:05.294 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.294 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.295 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.295 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.295 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.295 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.295 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.296 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.296 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:09:05.296 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.296 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.297 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.297 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.297 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:09:05.297 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.297 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.298 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.298 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.298 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.298 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.298 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.299 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.299 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:09:05.299 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.299 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.300 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.300 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.300 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:09:05.300 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.300 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.301 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.301 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.301 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.301 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.301 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.302 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.302 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:09:05.302 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.302 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.303 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.303 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.303 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:09:05.303 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:09:05.303 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.304 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.304 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.304 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:09:05.304 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:09:05.304 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:09:05.305 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:09:05.305 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.305 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.305 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.306 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.306 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.306 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.306 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:09:05.306 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:09:05.307 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:09:05.307 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:09:05.307 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:09:05.307 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:09:05.307 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:09:05.308 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:09:05.308 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:09:05.308 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 16, 2021-04-03T03:09:05.308 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:09:05.309 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:09:05.309 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:09:05.309 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.309 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.309 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.310 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:09:05.310 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.310 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.310 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.310 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.311 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:09:05.311 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.311 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.311 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.311 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.312 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:09:05.312 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.312 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.312 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.313 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:09:05.313 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.313 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.313 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.313 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.314 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:09:05.314 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.314 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.314 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.314 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.315 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:09:05.315 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.315 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.315 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.316 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:09:05.316 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.316 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.316 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.316 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.317 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:09:05.317 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.317 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.317 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.318 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.318 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:09:05.318 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:09:05.318 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.318 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:09:05.319 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:09:05.319 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.319 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.319 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.319 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:09:05.320 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:09:05.320 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:09:05.320 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.320 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.321 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.321 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:09:05.321 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:09:05.321 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:09:05.321 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:09:05.322 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:09:05.322 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:09:05.322 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:09:05.322 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:05.322 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.323 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:09:05.323 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:09:05.323 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.323 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:09:05.323 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.324 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.324 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.324 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.324 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.325 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:09:05.325 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.325 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.325 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.325 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:09:05.326 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.326 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.326 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.326 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.326 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.327 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:09:05.327 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.327 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.327 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.328 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:09:05.328 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.328 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.328 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.328 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.329 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.329 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:09:05.329 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.329 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.329 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.330 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:09:05.330 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.330 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.330 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.330 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.331 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.331 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:09:05.331 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.331 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.332 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.332 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:09:05.332 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.332 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.332 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.333 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.333 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.333 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:09:05.333 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.333 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.334 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.334 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:09:05.334 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.334 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.334 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.335 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.335 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.335 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:09:05.335 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.336 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.336 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.336 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:09:05.336 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.336 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.337 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.337 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.337 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.337 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:09:05.338 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.338 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.338 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:05.338 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:09:05.338 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:09:05.339 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:09:05.339 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:09:05.339 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:09:05.339 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:09:05.339 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:09:05.340 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:09:05.340 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.340 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:09:05.340 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:09:05.341 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:09:05.341 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:09:05.341 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:09:05.341 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:09:05.341 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:09:05.342 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:09:05.342 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:09:05.342 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:09:05.342 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:09:05.343 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:09:05.343 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:09:05.343 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:09:05.343 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:09:05.343 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:09:05.344 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:09:05.344 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:09:05.344 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.344 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:09:05.344 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:09:05.345 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:09:05.345 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:09:05.345 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:09:05.345 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:09:05.346 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:09:05.346 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:05.346 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:05.346 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:09:05.346 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:09:05.347 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:09:05.347 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:09:05.347 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:09:05.347 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:09:05.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f50e4057050 msgr2=0x7f50e4059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f50e4057050 0x7f50e4059510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f50e8017000 tx=0x7f50e8015000).stop 2021-04-03T03:09:05.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 msgr2=0x7f50f40d5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d5360 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f50fc068af0 tx=0x7f50fc05ad70).stop 2021-04-03T03:09:05.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 shutdown_connections 2021-04-03T03:09:05.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50f40d75f0 0x7f50f40d5360 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f50e4057050 0x7f50e4059510 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50f403cce0 0x7f50f4044cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 --2- 172.21.2.111:0/1894422587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50f403d6a0 0x7f50f40d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 >> 172.21.2.111:0/1894422587 conn(0x7f50f40462c0 msgr2=0x7f50f4006840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:05.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 shutdown_connections 2021-04-03T03:09:05.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.111+0000 7f5100cb9700 1 -- 172.21.2.111:0/1894422587 wait complete. 2021-04-03T03:09:05.352 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:05.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.403+0000 7f935ac4c700 1 Processor -- start 2021-04-03T03:09:05.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.403+0000 7f935ac4c700 1 -- start start 2021-04-03T03:09:05.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.403+0000 7f935ac4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0dbb90 0x7f934c042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.403+0000 7f935ac4c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f934c006bc0 0x7f934c0dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.403+0000 7f935ac4c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934c0dc6e0 con 0x7f934c03d360 2021-04-03T03:09:05.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.403+0000 7f935ac4c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934c0d9560 con 0x7f934c006bc0 2021-04-03T03:09:05.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.403+0000 7f935ac4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c03dcb0 con 0x7f934c0dbb90 2021-04-03T03:09:05.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f934c03d360 0x7f934c0064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51276/0 (socket says 172.21.2.111:51276) 2021-04-03T03:09:05.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 -- 172.21.2.111:0/2665012562 learned_addr learned my addr 172.21.2.111:0/2665012562 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:05.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359c4a700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0dbb90 0x7f934c042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148321681 0 0) 0x7f934c0dc6e0 con 0x7f934c03d360 2021-04-03T03:09:05.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f933c003240 con 0x7f934c03d360 2021-04-03T03:09:05.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935a44b700 1 --2- 172.21.2.111:0/2665012562 >> v2:172.21.2.115:3300/0 conn(0x7f934c006bc0 0x7f934c0dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 603023042 0 0) 0x7f933c003240 con 0x7f934c03d360 2021-04-03T03:09:05.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0dbb90 msgr2=0x7f934c042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0dbb90 0x7f934c042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 >> v2:172.21.2.115:3300/0 conn(0x7f934c006bc0 msgr2=0x7f934c0dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 --2- 172.21.2.111:0/2665012562 >> v2:172.21.2.115:3300/0 conn(0x7f934c006bc0 0x7f934c0dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0dc460 con 0x7f934c03d360 2021-04-03T03:09:05.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f93440016d0 con 0x7f934c03d360 2021-04-03T03:09:05.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 >> v1:172.21.2.111:6789/0 conn(0x7f934c03d360 legacy=0x7f934c0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f933c0037e0 0x7f933c003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f933c0044e0 0x7f933c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f933c004e50 0x7f933c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0d9560 con 0x7f933c0044e0 2021-04-03T03:09:05.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0052d0 con 0x7f933c0037e0 2021-04-03T03:09:05.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c00bc20 con 0x7f933c004e50 2021-04-03T03:09:05.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359c4a700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0dbb90 0x7f934c042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:09:05.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f933c0044e0 0x7f933c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 -- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f933c004e50 msgr2=0x7f933c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f933c004e50 0x7f933c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 -- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f933c0037e0 msgr2=0x7f933c003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f933c0037e0 0x7f933c003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 -- 172.21.2.111:0/2665012562 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c003240 con 0x7f933c0044e0 2021-04-03T03:09:05.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359449700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f933c0044e0 0x7f933c004960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f934c03e820 tx=0x7f934c03ec80).ready entity=mon.0 client_cookie=b386e4f9ec2dea4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934400ce30 con 0x7f933c0044e0 2021-04-03T03:09:05.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93440070d0 con 0x7f933c0044e0 2021-04-03T03:09:05.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9358c48700 1 -- 172.21.2.111:0/2665012562 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9344007430 con 0x7f933c0044e0 2021-04-03T03:09:05.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- 172.21.2.111:0/2665012562 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f933c0044e0 msgr2=0x7f933c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 --2- 172.21.2.111:0/2665012562 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f933c0044e0 0x7f933c004960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f934c03e820 tx=0x7f934c03ec80).stop 2021-04-03T03:09:05.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935a44b700 1 -- 172.21.2.111:0/2665012562 reap_dead start 2021-04-03T03:09:05.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- 172.21.2.111:0/2665012562 shutdown_connections 2021-04-03T03:09:05.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- 172.21.2.111:0/2665012562 >> 172.21.2.111:0/2665012562 conn(0x7f934c0462c0 msgr2=0x7f934c0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:05.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- 172.21.2.111:0/2665012562 shutdown_connections 2021-04-03T03:09:05.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- 172.21.2.111:0/2665012562 wait complete. 2021-04-03T03:09:05.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 Processor -- start 2021-04-03T03:09:05.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- start start 2021-04-03T03:09:05.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f934c039960 0x7f934c039d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0da2c0 0x7f934c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f934c0db330 0x7f934c0d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c03a940 con 0x7f934c039960 2021-04-03T03:09:05.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c03c4f0 con 0x7f934c0db330 2021-04-03T03:09:05.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f935ac4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c03a230 con 0x7f934c0da2c0 2021-04-03T03:09:05.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359c4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f934c039960 0x7f934c039d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359c4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f934c039960 0x7f934c039d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50494/0 (socket says 172.21.2.111:50494) 2021-04-03T03:09:05.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.407+0000 7f9359c4a700 1 -- 172.21.2.111:0/3213831906 learned_addr learned my addr 172.21.2.111:0/3213831906 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:05.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359c4a700 1 -- 172.21.2.111:0/3213831906 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0da2c0 msgr2=0x7f934c0d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359c4a700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0da2c0 0x7f934c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359c4a700 1 -- 172.21.2.111:0/3213831906 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f934c0db330 msgr2=0x7f934c0d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359c4a700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f934c0db330 0x7f934c0d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359c4a700 1 -- 172.21.2.111:0/3213831906 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c03c250 con 0x7f934c039960 2021-04-03T03:09:05.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359c4a700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f934c039960 0x7f934c039d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f934c03aca0 tx=0x7f9350008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9342ffd700 1 -- 172.21.2.111:0/3213831906 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f935000f710 con 0x7f934c039960 2021-04-03T03:09:05.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9342ffd700 1 -- 172.21.2.111:0/3213831906 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000f870 con 0x7f934c039960 2021-04-03T03:09:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c0d51d0 con 0x7f934c039960 2021-04-03T03:09:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9342ffd700 1 -- 172.21.2.111:0/3213831906 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f935000fbd0 con 0x7f934c039960 2021-04-03T03:09:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c0e6eb0 con 0x7f934c039960 2021-04-03T03:09:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c0da6c0 con 0x7f934c039960 2021-04-03T03:09:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9342ffd700 1 -- 172.21.2.111:0/3213831906 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f935000fd30 con 0x7f934c039960 2021-04-03T03:09:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9342ffd700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93340570a0 0x7f9334059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9342ffd700 1 -- 172.21.2.111:0/3213831906 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f9350070170 con 0x7f934c039960 2021-04-03T03:09:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359449700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93340570a0 0x7f9334059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.411+0000 7f9359449700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93340570a0 0x7f9334059560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f934c03e820 tx=0x7f9344015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.415+0000 7f9342ffd700 1 -- 172.21.2.111:0/3213831906 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9350042c50 con 0x7f934c039960 2021-04-03T03:09:05.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.535+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934c0db020 con 0x7f934c039960 2021-04-03T03:09:05.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.535+0000 7f9342ffd700 1 -- 172.21.2.111:0/3213831906 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f9350045550 con 0x7f934c039960 2021-04-03T03:09:05.546 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:05.546 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1066299897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1066299897},{"type":"v1","addr":"172.21.2.115:6833","nonce":1066299897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:08:44.422766+0000","modified":"2021-04-03T03:08:45.425728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2149096292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2149096292},{"type":"v1","addr":"172.21.2.111:6835","nonce":2149096292}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-04-03T03:09:05.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93340570a0 msgr2=0x7f9334059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93340570a0 0x7f9334059560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f934c03e820 tx=0x7f9344015000).stop 2021-04-03T03:09:05.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f934c039960 msgr2=0x7f934c039d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f934c039960 0x7f934c039d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f934c03aca0 tx=0x7f9350008040).stop 2021-04-03T03:09:05.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 shutdown_connections 2021-04-03T03:09:05.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93340570a0 0x7f9334059560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f934c0db330 0x7f934c0d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f934c039960 0x7f934c039d40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 --2- 172.21.2.111:0/3213831906 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f934c0da2c0 0x7f934c0d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 >> 172.21.2.111:0/3213831906 conn(0x7f934c0462c0 msgr2=0x7f934c045180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:05.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 shutdown_connections 2021-04-03T03:09:05.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.539+0000 7f935ac4c700 1 -- 172.21.2.111:0/3213831906 wait complete. 2021-04-03T03:09:05.550 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 5 2021-04-03T03:09:05.553 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:09:05.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.599+0000 7f8f86b27700 1 Processor -- start 2021-04-03T03:09:05.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.599+0000 7f8f86b27700 1 -- start start 2021-04-03T03:09:05.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 0x7f8f800b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8f800b44a0 0x7f8f800b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800607d0 con 0x7f8f800b5620 2021-04-03T03:09:05.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f801433c0 con 0x7f8f800b44a0 2021-04-03T03:09:05.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800b3f80 con 0x7f8f800603f0 2021-04-03T03:09:05.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8f800b44a0 0x7f8f800b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8f800b44a0 0x7f8f800b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50506/0 (socket says 172.21.2.111:50506) 2021-04-03T03:09:05.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 -- 172.21.2.111:0/3814080710 learned_addr learned my addr 172.21.2.111:0/3814080710 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:05.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 -- 172.21.2.111:0/3814080710 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 msgr2=0x7f8f800b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 --2- 172.21.2.111:0/3814080710 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 0x7f8f800b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 -- 172.21.2.111:0/3814080710 >> v1:172.21.2.115:6789/0 conn(0x7f8f800b5620 legacy=0x7f8f801469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 -- 172.21.2.111:0/3814080710 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800b4c60 con 0x7f8f800b44a0 2021-04-03T03:09:05.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f85324700 1 --2- 172.21.2.111:0/3814080710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b44a0 0x7f8f800b2970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8f800b5500 tx=0x7f8f74006e90).ready entity=mon.0 client_cookie=6f823a97dfa544c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f84b23700 1 -- 172.21.2.111:0/3814080710 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f740098e0 con 0x7f8f800b44a0 2021-04-03T03:09:05.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f84b23700 1 -- 172.21.2.111:0/3814080710 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f74009a40 con 0x7f8f800b44a0 2021-04-03T03:09:05.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f84b23700 1 -- 172.21.2.111:0/3814080710 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f74009de0 con 0x7f8f800b44a0 2021-04-03T03:09:05.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- 172.21.2.111:0/3814080710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b44a0 msgr2=0x7f8f800b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3814080710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b44a0 0x7f8f800b2970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8f800b5500 tx=0x7f8f74006e90).stop 2021-04-03T03:09:05.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- 172.21.2.111:0/3814080710 shutdown_connections 2021-04-03T03:09:05.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3814080710 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 0x7f8f800b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3814080710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b44a0 0x7f8f800b2970 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- 172.21.2.111:0/3814080710 >> 172.21.2.111:0/3814080710 conn(0x7f8f8005c7d0 msgr2=0x7f8f8005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:05.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- 172.21.2.111:0/3814080710 shutdown_connections 2021-04-03T03:09:05.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- 172.21.2.111:0/3814080710 wait complete. 2021-04-03T03:09:05.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 Processor -- start 2021-04-03T03:09:05.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- start start 2021-04-03T03:09:05.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 0x7f8f80152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8f800b44a0 0x7f8f80145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b5620 0x7f8f800ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80145db0 con 0x7f8f800b5620 2021-04-03T03:09:05.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80144a00 con 0x7f8f800b44a0 2021-04-03T03:09:05.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86b27700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80144ca0 con 0x7f8f800603f0 2021-04-03T03:09:05.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b5620 0x7f8f800ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b5620 0x7f8f800ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50512/0 (socket says 172.21.2.111:50512) 2021-04-03T03:09:05.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 -- 172.21.2.111:0/3725862237 learned_addr learned my addr 172.21.2.111:0/3725862237 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:05.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 -- 172.21.2.111:0/3725862237 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 msgr2=0x7f8f80152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 0x7f8f80152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 -- 172.21.2.111:0/3725862237 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8f800b44a0 msgr2=0x7f8f80145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8f800b44a0 0x7f8f80145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.603+0000 7f8f86326700 1 -- 172.21.2.111:0/3725862237 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800af8c0 con 0x7f8f800b5620 2021-04-03T03:09:05.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f86326700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b5620 0x7f8f800ad170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f8f78003270 tx=0x7f8f780090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f72ffd700 1 -- 172.21.2.111:0/3725862237 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f7800c900 con 0x7f8f800b5620 2021-04-03T03:09:05.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f72ffd700 1 -- 172.21.2.111:0/3725862237 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7800ca60 con 0x7f8f800b5620 2021-04-03T03:09:05.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f72ffd700 1 -- 172.21.2.111:0/3725862237 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f7800cdc0 con 0x7f8f800b5620 2021-04-03T03:09:05.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f801532f0 con 0x7f8f800b5620 2021-04-03T03:09:05.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f80145b60 con 0x7f8f800b5620 2021-04-03T03:09:05.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f80144ea0 con 0x7f8f800b5620 2021-04-03T03:09:05.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f72ffd700 1 -- 172.21.2.111:0/3725862237 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8f78015470 con 0x7f8f800b5620 2021-04-03T03:09:05.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f72ffd700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f60057000 0x7f8f600594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f72ffd700 1 -- 172.21.2.111:0/3725862237 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f8f780700e0 con 0x7f8f800b5620 2021-04-03T03:09:05.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f85b25700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f60057000 0x7f8f600594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.607+0000 7f8f85b25700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f60057000 0x7f8f600594c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8f7c002f30 tx=0x7f8f7c006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.611+0000 7f8f72ffd700 1 -- 172.21.2.111:0/3725862237 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8f80144ea0 con 0x7f8f800b5620 2021-04-03T03:09:05.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.735+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f80144e60 con 0x7f8f800b5620 2021-04-03T03:09:05.801 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:09:05.795+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:09:05.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f72ffd700 1 -- 172.21.2.111:0/3725862237 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f78048e90 con 0x7f8f800b5620 2021-04-03T03:09:05.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f60057000 msgr2=0x7f8f600594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f60057000 0x7f8f600594c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8f7c002f30 tx=0x7f8f7c006a60).stop 2021-04-03T03:09:05.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b5620 msgr2=0x7f8f800ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b5620 0x7f8f800ad170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f8f78003270 tx=0x7f8f780090f0).stop 2021-04-03T03:09:05.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 shutdown_connections 2021-04-03T03:09:05.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8f800b5620 0x7f8f800ad170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8f800603f0 0x7f8f80152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f60057000 0x7f8f600594c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 --2- 172.21.2.111:0/3725862237 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8f800b44a0 0x7f8f80145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 >> 172.21.2.111:0/3725862237 conn(0x7f8f8005c7d0 msgr2=0x7f8f800b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:05.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 shutdown_connections 2021-04-03T03:09:05.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.803+0000 7f8f86b27700 1 -- 172.21.2.111:0/3725862237 wait complete. 2021-04-03T03:09:05.816 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:09:05.820 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:05.825 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:09:05.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.875+0000 7fd787fff700 1 Processor -- start 2021-04-03T03:09:05.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.875+0000 7fd787fff700 1 -- start start 2021-04-03T03:09:05.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd780006a30 0x7fd7800d3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd780006640 con 0x7fd780006260 2021-04-03T03:09:05.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78003fb20 con 0x7fd7800d44e0 2021-04-03T03:09:05.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78003fdc0 con 0x7fd780006a30 2021-04-03T03:09:05.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd786ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd780006260 0x7fd7800d38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51320/0 (socket says 172.21.2.111:51320) 2021-04-03T03:09:05.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd786ffd700 1 -- 172.21.2.111:0/2646612739 learned_addr learned my addr 172.21.2.111:0/2646612739 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:05.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229421820 0 0) 0x7fd780006640 con 0x7fd780006260 2021-04-03T03:09:05.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd768003240 con 0x7fd780006260 2021-04-03T03:09:05.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 --2- 172.21.2.111:0/2646612739 >> v2:172.21.2.115:3300/0 conn(0x7fd780006a30 0x7fd7800d3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 178897757 0 0) 0x7fd768003240 con 0x7fd780006260 2021-04-03T03:09:05.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 >> v2:172.21.2.115:3300/0 conn(0x7fd780006a30 msgr2=0x7fd7800d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 --2- 172.21.2.111:0/2646612739 >> v2:172.21.2.115:3300/0 conn(0x7fd780006a30 0x7fd7800d3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 >> v1:172.21.2.115:6789/0 conn(0x7fd7800d44e0 legacy=0x7fd7800d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780040060 con 0x7fd780006260 2021-04-03T03:09:05.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fd7780016d0 con 0x7fd780006260 2021-04-03T03:09:05.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 >> v1:172.21.2.111:6789/0 conn(0x7fd780006260 legacy=0x7fd7800d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768003780 0x7fd768003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd768004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd768004e50 0x7fd76800c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78003fdc0 con 0x7fd7680044e0 2021-04-03T03:09:05.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd768005300 con 0x7fd768003780 2021-04-03T03:09:05.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800cd30 con 0x7fd768004e50 2021-04-03T03:09:05.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd768004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7877fe700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd768004e50 0x7fd76800c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd786ffd700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768003780 0x7fd768003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 -- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd768004e50 msgr2=0x7fd76800c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd768004e50 0x7fd76800c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 -- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768003780 msgr2=0x7fd768003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:05.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd768003780 0x7fd768003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 -- 172.21.2.111:0/2646612739 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768003240 con 0x7fd7680044e0 2021-04-03T03:09:05.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7877fe700 1 -- 172.21.2.111:0/2646612739 reap_dead start 2021-04-03T03:09:05.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd7867fc700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd768004960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fd77000a000 tx=0x7fd770008040).ready entity=mon.0 client_cookie=d6126984cd54fb85 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd77000b130 con 0x7fd7680044e0 2021-04-03T03:09:05.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77000da00 con 0x7fd7680044e0 2021-04-03T03:09:05.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd785ffb700 1 -- 172.21.2.111:0/2646612739 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd77000dd60 con 0x7fd7680044e0 2021-04-03T03:09:05.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- 172.21.2.111:0/2646612739 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 msgr2=0x7fd768004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:05.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd768004960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fd77000a000 tx=0x7fd770008040).stop 2021-04-03T03:09:05.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- 172.21.2.111:0/2646612739 shutdown_connections 2021-04-03T03:09:05.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 --2- 172.21.2.111:0/2646612739 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd768004960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:05.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- 172.21.2.111:0/2646612739 >> 172.21.2.111:0/2646612739 conn(0x7fd7800462c0 msgr2=0x7fd7800466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:05.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- 172.21.2.111:0/2646612739 shutdown_connections 2021-04-03T03:09:05.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- 172.21.2.111:0/2646612739 wait complete. 2021-04-03T03:09:05.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 Processor -- start 2021-04-03T03:09:05.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- start start 2021-04-03T03:09:05.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd7800a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7800ce3d0 0x7fd7800e1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7800e1560 0x7fd7800e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800e0c40 con 0x7fd7680044e0 2021-04-03T03:09:05.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800a0bd0 con 0x7fd7800ce3d0 2021-04-03T03:09:05.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd787fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800e1a10 con 0x7fd7800e1560 2021-04-03T03:09:05.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd786ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd7800a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd786ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd7800a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50540/0 (socket says 172.21.2.111:50540) 2021-04-03T03:09:05.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.879+0000 7fd786ffd700 1 -- 172.21.2.111:0/2337186195 learned_addr learned my addr 172.21.2.111:0/2337186195 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:05.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd786ffd700 1 -- 172.21.2.111:0/2337186195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7800e1560 msgr2=0x7fd7800e4100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:05.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd786ffd700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7800e1560 0x7fd7800e4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd786ffd700 1 -- 172.21.2.111:0/2337186195 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7800ce3d0 msgr2=0x7fd7800e1070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:05.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd786ffd700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7800ce3d0 0x7fd7800e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:05.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd786ffd700 1 -- 172.21.2.111:0/2337186195 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7800e1c60 con 0x7fd7680044e0 2021-04-03T03:09:05.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd786ffd700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd7800a4cd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd77800b000 tx=0x7fd780046e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd77ffff700 1 -- 172.21.2.111:0/2337186195 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd77800c9e0 con 0x7fd7680044e0 2021-04-03T03:09:05.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd77ffff700 1 -- 172.21.2.111:0/2337186195 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77800cb40 con 0x7fd7680044e0 2021-04-03T03:09:05.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7800a0e70 con 0x7fd7680044e0 2021-04-03T03:09:05.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7800ce850 con 0x7fd7680044e0 2021-04-03T03:09:05.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd77ffff700 1 -- 172.21.2.111:0/2337186195 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd77800cea0 con 0x7fd7680044e0 2021-04-03T03:09:05.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7800e0db0 con 0x7fd7680044e0 2021-04-03T03:09:05.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd77ffff700 1 -- 172.21.2.111:0/2337186195 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd778014430 con 0x7fd7680044e0 2021-04-03T03:09:05.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd77ffff700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd76c0570a0 0x7fd76c059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:05.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd77ffff700 1 -- 172.21.2.111:0/2337186195 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fd7780702a0 con 0x7fd7680044e0 2021-04-03T03:09:05.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd7867fc700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd76c0570a0 0x7fd76c059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:05.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.883+0000 7fd7867fc700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd76c0570a0 0x7fd76c059560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fd77000a000 tx=0x7fd770008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:05.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:05.887+0000 7fd77ffff700 1 -- 172.21.2.111:0/2337186195 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd7800e0db0 con 0x7fd7680044e0 2021-04-03T03:09:06.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7800e0d80 con 0x7fd7680044e0 2021-04-03T03:09:06.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd77ffff700 1 -- 172.21.2.111:0/2337186195 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1774 (secure 0 0 0) 0x7fd7780454b0 con 0x7fd7680044e0 2021-04-03T03:09:06.020 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:06.021 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1066299897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1066299897},{"type":"v1","addr":"172.21.2.115:6833","nonce":1066299897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:08:44.422766+0000","modified":"2021-04-03T03:09:05.798241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-04-03T03:09:06.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd76c0570a0 msgr2=0x7fd76c059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd76c0570a0 0x7fd76c059560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fd77000a000 tx=0x7fd770008040).stop 2021-04-03T03:09:06.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 msgr2=0x7fd7800a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd7800a4cd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd77800b000 tx=0x7fd780046e50).stop 2021-04-03T03:09:06.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 shutdown_connections 2021-04-03T03:09:06.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7800e1560 0x7fd7800e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7680044e0 0x7fd7800a4cd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:06.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd76c0570a0 0x7fd76c059560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:06.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 --2- 172.21.2.111:0/2337186195 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7800ce3d0 0x7fd7800e1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 >> 172.21.2.111:0/2337186195 conn(0x7fd7800462c0 msgr2=0x7fd780004550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:06.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 shutdown_connections 2021-04-03T03:09:06.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.011+0000 7fd787fff700 1 -- 172.21.2.111:0/2337186195 wait complete. 2021-04-03T03:09:06.024 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 6 2021-04-03T03:09:06.028 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:09:06.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 Processor -- start 2021-04-03T03:09:06.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- start start 2021-04-03T03:09:06.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240603f0 0x7fa3241447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3240b17b0 0x7fa324144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3240607d0 con 0x7fa3240add50 2021-04-03T03:09:06.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3241451c0 con 0x7fa3240b17b0 2021-04-03T03:09:06.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32414b090 con 0x7fa3240603f0 2021-04-03T03:09:06.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3240b17b0 0x7fa324144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:06.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3240b17b0 0x7fa324144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50550/0 (socket says 172.21.2.111:50550) 2021-04-03T03:09:06.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 -- 172.21.2.111:0/2011640213 learned_addr learned my addr 172.21.2.111:0/2011640213 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:06.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 -- 172.21.2.111:0/2011640213 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240603f0 msgr2=0x7fa3241447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:06.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 --2- 172.21.2.111:0/2011640213 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240603f0 0x7fa3241447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 -- 172.21.2.111:0/2011640213 >> v1:172.21.2.115:6789/0 conn(0x7fa3240add50 legacy=0x7fa324147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:06.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 -- 172.21.2.111:0/2011640213 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32414b330 con 0x7fa3240b17b0 2021-04-03T03:09:06.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa3237fe700 1 --2- 172.21.2.111:0/2011640213 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240b17b0 0x7fa324144cd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fa3240b5600 tx=0x7fa31c009270).ready entity=mon.0 client_cookie=ac104b799357bb4e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:06.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa321ffb700 1 -- 172.21.2.111:0/2011640213 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31c00bbc0 con 0x7fa3240b17b0 2021-04-03T03:09:06.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa321ffb700 1 -- 172.21.2.111:0/2011640213 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c00bd20 con 0x7fa3240b17b0 2021-04-03T03:09:06.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa321ffb700 1 -- 172.21.2.111:0/2011640213 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31c011400 con 0x7fa3240b17b0 2021-04-03T03:09:06.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- 172.21.2.111:0/2011640213 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240b17b0 msgr2=0x7fa324144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 --2- 172.21.2.111:0/2011640213 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240b17b0 0x7fa324144cd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fa3240b5600 tx=0x7fa31c009270).stop 2021-04-03T03:09:06.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- 172.21.2.111:0/2011640213 shutdown_connections 2021-04-03T03:09:06.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 --2- 172.21.2.111:0/2011640213 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240b17b0 0x7fa324144cd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:06.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 --2- 172.21.2.111:0/2011640213 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240603f0 0x7fa3241447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- 172.21.2.111:0/2011640213 >> 172.21.2.111:0/2011640213 conn(0x7fa3240b61c0 msgr2=0x7fa32405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:06.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- 172.21.2.111:0/2011640213 shutdown_connections 2021-04-03T03:09:06.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 -- 172.21.2.111:0/2011640213 wait complete. 2021-04-03T03:09:06.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.131+0000 7fa323fff700 1 Processor -- start 2021-04-03T03:09:06.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 -- start start 2021-04-03T03:09:06.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240603f0 0x7fa3240a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240add50 0x7fa324157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3240b17b0 0x7fa324062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240ab9f0 con 0x7fa3240603f0 2021-04-03T03:09:06.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240aad20 con 0x7fa3240b17b0 2021-04-03T03:09:06.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240aaa80 con 0x7fa3240add50 2021-04-03T03:09:06.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240603f0 0x7fa3240a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:06.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240603f0 0x7fa3240a7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50554/0 (socket says 172.21.2.111:50554) 2021-04-03T03:09:06.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 -- 172.21.2.111:0/1515856657 learned_addr learned my addr 172.21.2.111:0/1515856657 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:06.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 -- 172.21.2.111:0/1515856657 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240add50 msgr2=0x7fa324157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240add50 0x7fa324157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 -- 172.21.2.111:0/1515856657 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3240b17b0 msgr2=0x7fa324062d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3240b17b0 0x7fa324062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 -- 172.21.2.111:0/1515856657 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240abe50 con 0x7fa3240603f0 2021-04-03T03:09:06.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa322ffd700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240603f0 0x7fa3240a7eb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fa314002f30 tx=0x7fa314006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:06.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa31bfff700 1 -- 172.21.2.111:0/1515856657 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31400d840 con 0x7fa3240603f0 2021-04-03T03:09:06.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa31bfff700 1 -- 172.21.2.111:0/1515856657 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa314000e90 con 0x7fa3240603f0 2021-04-03T03:09:06.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3240ab420 con 0x7fa3240603f0 2021-04-03T03:09:06.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa31bfff700 1 -- 172.21.2.111:0/1515856657 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31400db90 con 0x7fa3240603f0 2021-04-03T03:09:06.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3240aa770 con 0x7fa3240603f0 2021-04-03T03:09:06.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3240abb60 con 0x7fa3240603f0 2021-04-03T03:09:06.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa31bfff700 1 -- 172.21.2.111:0/1515856657 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa314017030 con 0x7fa3240603f0 2021-04-03T03:09:06.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa31bfff700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa308057000 0x7fa3080594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa31bfff700 1 -- 172.21.2.111:0/1515856657 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fa31406f630 con 0x7fa3240603f0 2021-04-03T03:09:06.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa3227fc700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa308057000 0x7fa3080594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:06.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.135+0000 7fa3227fc700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa308057000 0x7fa3080594c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fa30c002f50 tx=0x7fa30c006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:06.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.139+0000 7fa31bfff700 1 -- 172.21.2.111:0/1515856657 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa314044a60 con 0x7fa3240603f0 2021-04-03T03:09:06.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.263+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa324057660 con 0x7fa3240603f0 2021-04-03T03:09:06.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.815+0000 7fa31bfff700 1 -- 172.21.2.111:0/1515856657 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa314048080 con 0x7fa3240603f0 2021-04-03T03:09:06.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa308057000 msgr2=0x7fa3080594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa308057000 0x7fa3080594c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fa30c002f50 tx=0x7fa30c006990).stop 2021-04-03T03:09:06.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240603f0 msgr2=0x7fa3240a7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240603f0 0x7fa3240a7eb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fa314002f30 tx=0x7fa314006ea0).stop 2021-04-03T03:09:06.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 shutdown_connections 2021-04-03T03:09:06.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa308057000 0x7fa3080594c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:06.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3240b17b0 0x7fa324062d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3240603f0 0x7fa3240a7eb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:06.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 --2- 172.21.2.111:0/1515856657 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3240add50 0x7fa324157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 >> 172.21.2.111:0/1515856657 conn(0x7fa3240b61c0 msgr2=0x7fa3241447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:06.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 shutdown_connections 2021-04-03T03:09:06.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.819+0000 7fa323fff700 1 -- 172.21.2.111:0/1515856657 wait complete. 2021-04-03T03:09:06.834 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:09:06.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 Processor -- start 2021-04-03T03:09:06.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- start start 2021-04-03T03:09:06.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6db40add50 0x7f6db4147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40607d0 con 0x7f6db4144ef0 2021-04-03T03:09:06.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db414b090 con 0x7f6db40603f0 2021-04-03T03:09:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db414b330 con 0x7f6db40add50 2021-04-03T03:09:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6db40add50 0x7f6db4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6db40add50 0x7f6db4147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50564/0 (socket says 172.21.2.111:50564) 2021-04-03T03:09:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 -- 172.21.2.111:0/2218942953 learned_addr learned my addr 172.21.2.111:0/2218942953 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 -- 172.21.2.111:0/2218942953 >> v1:172.21.2.115:6789/0 conn(0x7f6db40603f0 legacy=0x7f6db41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 -- 172.21.2.111:0/2218942953 >> v1:172.21.2.111:6789/0 conn(0x7f6db4144ef0 legacy=0x7f6db4147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 -- 172.21.2.111:0/2218942953 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db414b5d0 con 0x7f6db40add50 2021-04-03T03:09:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 --2- 172.21.2.111:0/2218942953 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db4147740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f6db40b5600 tx=0x7f6da4006e90).ready entity=mon.0 client_cookie=8d1b5c9d4a4f78aa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db37fe700 1 -- 172.21.2.111:0/2218942953 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da4009980 con 0x7f6db40add50 2021-04-03T03:09:06.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db37fe700 1 -- 172.21.2.111:0/2218942953 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da4009ae0 con 0x7f6db40add50 2021-04-03T03:09:06.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db37fe700 1 -- 172.21.2.111:0/2218942953 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da4009e80 con 0x7f6db40add50 2021-04-03T03:09:06.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- 172.21.2.111:0/2218942953 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 msgr2=0x7f6db4147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2218942953 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db4147740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f6db40b5600 tx=0x7f6da4006e90).stop 2021-04-03T03:09:06.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- 172.21.2.111:0/2218942953 shutdown_connections 2021-04-03T03:09:06.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2218942953 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db4147740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:06.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- 172.21.2.111:0/2218942953 >> 172.21.2.111:0/2218942953 conn(0x7f6db40b61c0 msgr2=0x7f6db405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:06.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- 172.21.2.111:0/2218942953 shutdown_connections 2021-04-03T03:09:06.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- 172.21.2.111:0/2218942953 wait complete. 2021-04-03T03:09:06.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 Processor -- start 2021-04-03T03:09:06.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- start start 2021-04-03T03:09:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6db40603f0 0x7f6db40b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db41571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6db4144ef0 0x7f6db41576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40abe00 con 0x7f6db40add50 2021-04-03T03:09:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40657d0 con 0x7f6db40603f0 2021-04-03T03:09:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db9e95700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4065a70 con 0x7f6db4144ef0 2021-04-03T03:09:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db41571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db41571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50568/0 (socket says 172.21.2.111:50568) 2021-04-03T03:09:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.903+0000 7f6db3fff700 1 -- 172.21.2.111:0/2262378937 learned_addr learned my addr 172.21.2.111:0/2262378937 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db3fff700 1 -- 172.21.2.111:0/2262378937 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6db4144ef0 msgr2=0x7f6db41576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db3fff700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6db4144ef0 0x7f6db41576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db3fff700 1 -- 172.21.2.111:0/2262378937 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6db40603f0 msgr2=0x7f6db40b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db3fff700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6db40603f0 0x7f6db40b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db3fff700 1 -- 172.21.2.111:0/2262378937 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40abaf0 con 0x7f6db40add50 2021-04-03T03:09:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db3fff700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db41571c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f6db40b5600 tx=0x7f6da4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db1ffb700 1 -- 172.21.2.111:0/2262378937 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da4011db0 con 0x7f6db40add50 2021-04-03T03:09:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db1ffb700 1 -- 172.21.2.111:0/2262378937 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da401e460 con 0x7f6db40add50 2021-04-03T03:09:06.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db4156b60 con 0x7f6db40add50 2021-04-03T03:09:06.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db40aab70 con 0x7f6db40add50 2021-04-03T03:09:06.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db1ffb700 1 -- 172.21.2.111:0/2262378937 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da401e7c0 con 0x7f6db40add50 2021-04-03T03:09:06.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db4057660 con 0x7f6db40add50 2021-04-03T03:09:06.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db1ffb700 1 -- 172.21.2.111:0/2262378937 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6da4025030 con 0x7f6db40add50 2021-04-03T03:09:06.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db1ffb700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d98056ea0 0x7f6d98059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:06.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db1ffb700 1 -- 172.21.2.111:0/2262378937 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f6da4071730 con 0x7f6db40add50 2021-04-03T03:09:06.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db8e93700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d98056ea0 0x7f6d98059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:06.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db8e93700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d98056ea0 0x7f6d98059360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6d9c002f50 tx=0x7f6d9c006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:06.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:06.907+0000 7f6db1ffb700 1 -- 172.21.2.111:0/2262378937 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6da40469a0 con 0x7f6db40add50 2021-04-03T03:09:07.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.003+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6db4148850 con 0x7f6db40add50 2021-04-03T03:09:07.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db1ffb700 1 -- 172.21.2.111:0/2262378937 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6da4023020 con 0x7f6db40add50 2021-04-03T03:09:07.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d98056ea0 msgr2=0x7f6d98059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:07.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d98056ea0 0x7f6d98059360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6d9c002f50 tx=0x7f6d9c006990).stop 2021-04-03T03:09:07.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 msgr2=0x7f6db41571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:07.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db41571c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f6db40b5600 tx=0x7f6da4015000).stop 2021-04-03T03:09:07.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 shutdown_connections 2021-04-03T03:09:07.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6db4144ef0 0x7f6db41576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:07.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6d98056ea0 0x7f6d98059360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:07.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6db40603f0 0x7f6db40b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:07.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 --2- 172.21.2.111:0/2262378937 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6db40add50 0x7f6db41571c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:07.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 >> 172.21.2.111:0/2262378937 conn(0x7f6db40b61c0 msgr2=0x7f6db40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:07.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 shutdown_connections 2021-04-03T03:09:07.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.823+0000 7f6db9e95700 1 -- 172.21.2.111:0/2262378937 wait complete. 2021-04-03T03:09:07.834 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:09:07.838 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:09:07.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 Processor -- start 2021-04-03T03:09:07.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 -- start start 2021-04-03T03:09:07.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efd300603f0 0x7efd301447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:07.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7efd300add50 0x7efd30144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:07.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd300607d0 con 0x7efd300b17b0 2021-04-03T03:09:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd301451c0 con 0x7efd300603f0 2021-04-03T03:09:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3014b090 con 0x7efd300add50 2021-04-03T03:09:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efd300603f0 0x7efd301447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efd300603f0 0x7efd301447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50574/0 (socket says 172.21.2.111:50574) 2021-04-03T03:09:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 -- 172.21.2.111:0/3481625390 learned_addr learned my addr 172.21.2.111:0/3481625390 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35599700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7efd300b17b0 0x7efd30147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51368/0 (socket says 172.21.2.111:51368) 2021-04-03T03:09:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 -- 172.21.2.111:0/3481625390 >> v2:172.21.2.115:3300/0 conn(0x7efd300add50 msgr2=0x7efd30144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 --2- 172.21.2.111:0/3481625390 >> v2:172.21.2.115:3300/0 conn(0x7efd300add50 0x7efd30144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 -- 172.21.2.111:0/3481625390 >> v1:172.21.2.111:6789/0 conn(0x7efd300b17b0 legacy=0x7efd30147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 -- 172.21.2.111:0/3481625390 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3014b330 con 0x7efd300603f0 2021-04-03T03:09:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd34d98700 1 --2- 172.21.2.111:0/3481625390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd301447e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7efd18002f30 tx=0x7efd18006e90).ready entity=mon.0 client_cookie=99653265200ceabb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd2f7fe700 1 -- 172.21.2.111:0/3481625390 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd180099c0 con 0x7efd300603f0 2021-04-03T03:09:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd2f7fe700 1 -- 172.21.2.111:0/3481625390 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18009b20 con 0x7efd300603f0 2021-04-03T03:09:07.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd2f7fe700 1 -- 172.21.2.111:0/3481625390 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd18009e80 con 0x7efd300603f0 2021-04-03T03:09:07.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 -- 172.21.2.111:0/3481625390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 msgr2=0x7efd301447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:07.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.887+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3481625390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd301447e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7efd18002f30 tx=0x7efd18006e90).stop 2021-04-03T03:09:07.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- 172.21.2.111:0/3481625390 shutdown_connections 2021-04-03T03:09:07.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3481625390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd301447e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:07.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3481625390 >> v2:172.21.2.115:3300/0 conn(0x7efd300add50 0x7efd30144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:07.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- 172.21.2.111:0/3481625390 >> 172.21.2.111:0/3481625390 conn(0x7efd300b61c0 msgr2=0x7efd3005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:07.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- 172.21.2.111:0/3481625390 shutdown_connections 2021-04-03T03:09:07.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- 172.21.2.111:0/3481625390 wait complete. 2021-04-03T03:09:07.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 Processor -- start 2021-04-03T03:09:07.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- start start 2021-04-03T03:09:07.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd300b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:07.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efd300add50 0x7efd30157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:07.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efd300b17b0 0x7efd30157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:07.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300a9330 con 0x7efd300603f0 2021-04-03T03:09:07.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300a7000 con 0x7efd300add50 2021-04-03T03:09:07.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b3fe0 con 0x7efd300b17b0 2021-04-03T03:09:07.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd300b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:07.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd300b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50582/0 (socket says 172.21.2.111:50582) 2021-04-03T03:09:07.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 -- 172.21.2.111:0/3330611696 learned_addr learned my addr 172.21.2.111:0/3330611696 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:07.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35599700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efd300b17b0 0x7efd30157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:07.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 -- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efd300b17b0 msgr2=0x7efd30157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:07.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efd300b17b0 0x7efd30157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:07.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 -- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efd300add50 msgr2=0x7efd30157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:07.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efd300add50 0x7efd30157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:07.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 -- 172.21.2.111:0/3330611696 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300b4280 con 0x7efd300603f0 2021-04-03T03:09:07.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35599700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efd300b17b0 0x7efd30157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:07.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd34d98700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd300b02e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7efd18002f30 tx=0x7efd18015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:07.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2dffb700 1 -- 172.21.2.111:0/3330611696 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1801e460 con 0x7efd300603f0 2021-04-03T03:09:07.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2dffb700 1 -- 172.21.2.111:0/3330611696 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1801e5c0 con 0x7efd300603f0 2021-04-03T03:09:07.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd30156cb0 con 0x7efd300603f0 2021-04-03T03:09:07.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd30067e70 con 0x7efd300603f0 2021-04-03T03:09:07.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2dffb700 1 -- 172.21.2.111:0/3330611696 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1801e920 con 0x7efd300603f0 2021-04-03T03:09:07.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd300b0fa0 con 0x7efd300603f0 2021-04-03T03:09:07.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2dffb700 1 -- 172.21.2.111:0/3330611696 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7efd18025030 con 0x7efd300603f0 2021-04-03T03:09:07.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2dffb700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd140570a0 0x7efd14059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:07.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2dffb700 1 -- 172.21.2.111:0/3330611696 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6075+0+0 (secure 0 0 0) 0x7efd18071770 con 0x7efd300603f0 2021-04-03T03:09:07.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2ffff700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd140570a0 0x7efd14059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:07.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.891+0000 7efd2ffff700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd140570a0 0x7efd14059560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7efd20002f30 tx=0x7efd20006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:07.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.895+0000 7efd2dffb700 1 -- 172.21.2.111:0/3330611696 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7efd18046ba0 con 0x7efd300603f0 2021-04-03T03:09:07.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:07.987+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd300b3e10 con 0x7efd300603f0 2021-04-03T03:09:08.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd2dffb700 1 -- 172.21.2.111:0/3330611696 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7efd1804a1c0 con 0x7efd300603f0 2021-04-03T03:09:08.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd140570a0 msgr2=0x7efd14059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:08.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd140570a0 0x7efd14059560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7efd20002f30 tx=0x7efd20006a60).stop 2021-04-03T03:09:08.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 msgr2=0x7efd300b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:08.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd300b02e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7efd18002f30 tx=0x7efd18015000).stop 2021-04-03T03:09:08.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 shutdown_connections 2021-04-03T03:09:08.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efd300b17b0 0x7efd30157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:08.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efd300603f0 0x7efd300b02e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:08.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd140570a0 0x7efd14059560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:08.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 --2- 172.21.2.111:0/3330611696 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efd300add50 0x7efd30157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:08.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 >> 172.21.2.111:0/3330611696 conn(0x7efd300b61c0 msgr2=0x7efd301441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:08.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 shutdown_connections 2021-04-03T03:09:08.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.835+0000 7efd35d9a700 1 -- 172.21.2.111:0/3330611696 wait complete. 2021-04-03T03:09:08.846 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:09:08.851 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:09:08.851 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:09:08.851 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:09:08.851 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:09:08.852 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:09:08.852 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:09:08.852 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:09:08.853 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:09:08.856 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:09:08.856 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:09:08.858 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:09:08.858 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:09:08.898 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:09:08.899 DEBUG:teuthology.parallel:result is None 2021-04-03T03:09:08.902 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:09:08.902 DEBUG:teuthology.parallel:result is None 2021-04-03T03:09:08.903 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:09:08.938 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:09:08.947 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:09:08.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.955+0000 7ff79fde7700 1 Processor -- start 2021-04-03T03:09:08.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.955+0000 7ff79fde7700 1 -- start start 2021-04-03T03:09:08.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff798041f00 0x7ff798040fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:08.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff79803d600 0x7ff79803c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:08.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff798041610 con 0x7ff79803fdf0 2021-04-03T03:09:08.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980422e0 con 0x7ff798041f00 2021-04-03T03:09:08.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff798042700 con 0x7ff79803d600 2021-04-03T03:09:08.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff798041f00 0x7ff798040fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:08.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff798041f00 0x7ff798040fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50600/0 (socket says 172.21.2.111:50600) 2021-04-03T03:09:08.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 -- 172.21.2.111:0/927288381 learned_addr learned my addr 172.21.2.111:0/927288381 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:08.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79f5e6700 1 --2- 172.21.2.111:0/927288381 >> v2:172.21.2.115:3300/0 conn(0x7ff79803d600 0x7ff79803c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:08.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 -- 172.21.2.111:0/927288381 >> v2:172.21.2.115:3300/0 conn(0x7ff79803d600 msgr2=0x7ff79803c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:08.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 --2- 172.21.2.111:0/927288381 >> v2:172.21.2.115:3300/0 conn(0x7ff79803d600 0x7ff79803c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:08.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 -- 172.21.2.111:0/927288381 >> v1:172.21.2.115:6789/0 conn(0x7ff79803fdf0 legacy=0x7ff7980d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:08.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 -- 172.21.2.111:0/927288381 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798041870 con 0x7ff798041f00 2021-04-03T03:09:08.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79ede5700 1 --2- 172.21.2.111:0/927288381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff798041f00 0x7ff798040fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7ff798040980 tx=0x7ff7a005ad70).ready entity=mon.0 client_cookie=fc6f855c897eb74f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:08.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79dde3700 1 -- 172.21.2.111:0/927288381 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7a005bda0 con 0x7ff798041f00 2021-04-03T03:09:08.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79dde3700 1 -- 172.21.2.111:0/927288381 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a005df60 con 0x7ff798041f00 2021-04-03T03:09:08.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79dde3700 1 -- 172.21.2.111:0/927288381 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7a00699f0 con 0x7ff798041f00 2021-04-03T03:09:08.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/927288381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff798041f00 msgr2=0x7ff798040fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:08.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/927288381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff798041f00 0x7ff798040fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7ff798040980 tx=0x7ff7a005ad70).stop 2021-04-03T03:09:08.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/927288381 shutdown_connections 2021-04-03T03:09:08.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/927288381 >> v2:172.21.2.115:3300/0 conn(0x7ff79803d600 0x7ff79803c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:08.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/927288381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff798041f00 0x7ff798040fa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:08.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/927288381 >> 172.21.2.111:0/927288381 conn(0x7ff7980462b0 msgr2=0x7ff7980466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:08.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/927288381 shutdown_connections 2021-04-03T03:09:08.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/927288381 wait complete. 2021-04-03T03:09:08.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 Processor -- start 2021-04-03T03:09:08.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- start start 2021-04-03T03:09:08.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff79803d600 0x7ff7980e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:08.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff79803fdf0 0x7ff7980d2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:08.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff798041f00 0x7ff7980e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:08.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cda20 con 0x7ff79803fdf0 2021-04-03T03:09:08.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980da130 con 0x7ff798041f00 2021-04-03T03:09:08.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cdc50 con 0x7ff79803d600 2021-04-03T03:09:08.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff79803fdf0 0x7ff7980d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:08.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff79803fdf0 0x7ff7980d2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50610/0 (socket says 172.21.2.111:50610) 2021-04-03T03:09:08.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 -- 172.21.2.111:0/3348218848 learned_addr learned my addr 172.21.2.111:0/3348218848 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:08.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 -- 172.21.2.111:0/3348218848 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff79803d600 msgr2=0x7ff7980e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:08.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff79803d600 0x7ff7980e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:08.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 -- 172.21.2.111:0/3348218848 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff798041f00 msgr2=0x7ff7980e34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:08.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff798041f00 0x7ff7980e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:08.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 -- 172.21.2.111:0/3348218848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980d2a30 con 0x7ff79803fdf0 2021-04-03T03:09:08.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79e5e4700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff79803fdf0 0x7ff7980d2540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7ff794001f60 tx=0x7ff794007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:08.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff78ffff700 1 -- 172.21.2.111:0/3348218848 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff79400e4a0 con 0x7ff79803fdf0 2021-04-03T03:09:08.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff78ffff700 1 -- 172.21.2.111:0/3348218848 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79400e600 con 0x7ff79803fdf0 2021-04-03T03:09:08.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7980d94d0 con 0x7ff79803fdf0 2021-04-03T03:09:08.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff78ffff700 1 -- 172.21.2.111:0/3348218848 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff79400e960 con 0x7ff79803fdf0 2021-04-03T03:09:08.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980d8f70 con 0x7ff79803fdf0 2021-04-03T03:09:08.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.963+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7980cde10 con 0x7ff79803fdf0 2021-04-03T03:09:08.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.963+0000 7ff78ffff700 1 -- 172.21.2.111:0/3348218848 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff794019030 con 0x7ff79803fdf0 2021-04-03T03:09:08.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.963+0000 7ff78ffff700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff77c056f20 0x7ff77c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:08.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.963+0000 7ff78ffff700 1 -- 172.21.2.111:0/3348218848 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5682+0+0 (secure 0 0 0) 0x7ff79406f3c0 con 0x7ff79803fdf0 2021-04-03T03:09:08.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.963+0000 7ff79ede5700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff77c056f20 0x7ff77c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:08.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.963+0000 7ff79ede5700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff77c056f20 0x7ff77c0593e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7ff798040980 tx=0x7ff7a0064030).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:08.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:08.963+0000 7ff78ffff700 1 -- 172.21.2.111:0/3348218848 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff794042c50 con 0x7ff79803fdf0 2021-04-03T03:09:09.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.067+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7ff798004d30 con 0x7ff79803fdf0 2021-04-03T03:09:09.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.955+0000 7ff78ffff700 1 -- 172.21.2.111:0/3348218848 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff794044a20 con 0x7ff79803fdf0 2021-04-03T03:09:09.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff77c056f20 msgr2=0x7ff77c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:09.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff77c056f20 0x7ff77c0593e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7ff798040980 tx=0x7ff7a0064030).stop 2021-04-03T03:09:09.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff79803fdf0 msgr2=0x7ff7980d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:09.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff79803fdf0 0x7ff7980d2540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7ff794001f60 tx=0x7ff794007c10).stop 2021-04-03T03:09:09.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 shutdown_connections 2021-04-03T03:09:09.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff77c056f20 0x7ff77c0593e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:09.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff798041f00 0x7ff7980e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:09.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff79803d600 0x7ff7980e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:09.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 --2- 172.21.2.111:0/3348218848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff79803fdf0 0x7ff7980d2540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:09.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.959+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 >> 172.21.2.111:0/3348218848 conn(0x7ff7980462b0 msgr2=0x7ff79803cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:09.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.963+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 shutdown_connections 2021-04-03T03:09:09.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:09.963+0000 7ff79fde7700 1 -- 172.21.2.111:0/3348218848 wait complete. 2021-04-03T03:09:09.972 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:09:09.981 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:09:09.982 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:09:10.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.091+0000 7f4dc7386700 1 Processor -- start 2021-04-03T03:09:10.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.091+0000 7f4dc7386700 1 -- start start 2021-04-03T03:09:10.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00dbb90 0x7f4dc0042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4dc0006bc0 0x7f4dc00dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc00dc6e0 con 0x7f4dc003d360 2021-04-03T03:09:10.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc00d9560 con 0x7f4dc0006bc0 2021-04-03T03:09:10.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc003dcb0 con 0x7f4dc00dbb90 2021-04-03T03:09:10.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6b85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4dc0006bc0 0x7f4dc00dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc5b83700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4dc003d360 0x7f4dc00064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51410/0 (socket says 172.21.2.111:51410) 2021-04-03T03:09:10.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc5b83700 1 -- 172.21.2.111:0/1975661587 learned_addr learned my addr 172.21.2.111:0/1975661587 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:10.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 --2- 172.21.2.111:0/1975661587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00dbb90 0x7f4dc0042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6b85700 1 -- 172.21.2.111:0/1975661587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00dbb90 msgr2=0x7f4dc0042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6b85700 1 --2- 172.21.2.111:0/1975661587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00dbb90 0x7f4dc0042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6b85700 1 -- 172.21.2.111:0/1975661587 >> v1:172.21.2.111:6789/0 conn(0x7f4dc003d360 legacy=0x7f4dc00064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6b85700 1 -- 172.21.2.111:0/1975661587 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00dc460 con 0x7f4dc0006bc0 2021-04-03T03:09:10.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc5382700 1 -- 172.21.2.111:0/1975661587 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523622021 0 0) 0x7f4dc00dc6e0 con 0x7f4dc003d360 2021-04-03T03:09:10.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6b85700 1 --2- 172.21.2.111:0/1975661587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc0006bc0 0x7f4dc00dbf70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f4dc003e820 tx=0x7f4dc805ad70).ready entity=mon.0 client_cookie=8985b2bcab1e7898 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc5382700 1 -- 172.21.2.111:0/1975661587 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dc805bda0 con 0x7f4dc0006bc0 2021-04-03T03:09:10.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc5382700 1 -- 172.21.2.111:0/1975661587 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc805df60 con 0x7f4dc0006bc0 2021-04-03T03:09:10.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc5382700 1 -- 172.21.2.111:0/1975661587 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dc80699f0 con 0x7f4dc0006bc0 2021-04-03T03:09:10.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- 172.21.2.111:0/1975661587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc0006bc0 msgr2=0x7f4dc00dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- 172.21.2.111:0/1975661587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc0006bc0 0x7f4dc00dbf70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f4dc003e820 tx=0x7f4dc805ad70).stop 2021-04-03T03:09:10.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- 172.21.2.111:0/1975661587 shutdown_connections 2021-04-03T03:09:10.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- 172.21.2.111:0/1975661587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc0006bc0 0x7f4dc00dbf70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- 172.21.2.111:0/1975661587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00dbb90 0x7f4dc0042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- 172.21.2.111:0/1975661587 >> 172.21.2.111:0/1975661587 conn(0x7f4dc00462c0 msgr2=0x7f4dc00466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- 172.21.2.111:0/1975661587 shutdown_connections 2021-04-03T03:09:10.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- 172.21.2.111:0/1975661587 wait complete. 2021-04-03T03:09:10.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 Processor -- start 2021-04-03T03:09:10.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- start start 2021-04-03T03:09:10.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc003d360 0x7f4dc00d7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dc00dbb90 0x7f4dc003af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00a2fc0 0x7f4dc0039a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc003b630 con 0x7f4dc003d360 2021-04-03T03:09:10.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc003b8d0 con 0x7f4dc00dbb90 2021-04-03T03:09:10.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc0039590 con 0x7f4dc00a2fc0 2021-04-03T03:09:10.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc003d360 0x7f4dc00d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc003d360 0x7f4dc00d7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50620/0 (socket says 172.21.2.111:50620) 2021-04-03T03:09:10.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 -- 172.21.2.111:0/3086862681 learned_addr learned my addr 172.21.2.111:0/3086862681 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:10.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 -- 172.21.2.111:0/3086862681 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00a2fc0 msgr2=0x7f4dc0039a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:10.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00a2fc0 0x7f4dc0039a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 -- 172.21.2.111:0/3086862681 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dc00dbb90 msgr2=0x7f4dc003af90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:10.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dc00dbb90 0x7f4dc003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 -- 172.21.2.111:0/3086862681 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc0039f20 con 0x7f4dc003d360 2021-04-03T03:09:10.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc6384700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc003d360 0x7f4dc00d7320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f4db0003070 tx=0x7f4db0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dbf7fe700 1 -- 172.21.2.111:0/3086862681 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4db0009790 con 0x7f4dc003d360 2021-04-03T03:09:10.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dbf7fe700 1 -- 172.21.2.111:0/3086862681 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db00098f0 con 0x7f4dc003d360 2021-04-03T03:09:10.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc009f830 con 0x7f4dc003d360 2021-04-03T03:09:10.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dbf7fe700 1 -- 172.21.2.111:0/3086862681 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4db0009c50 con 0x7f4dc003d360 2021-04-03T03:09:10.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.095+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc009fda0 con 0x7f4dc003d360 2021-04-03T03:09:10.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.099+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc003b480 con 0x7f4dc003d360 2021-04-03T03:09:10.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.099+0000 7f4dbf7fe700 1 -- 172.21.2.111:0/3086862681 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4db0013480 con 0x7f4dc003d360 2021-04-03T03:09:10.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.099+0000 7f4dbf7fe700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dac056f20 0x7f4dac0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.099+0000 7f4dbf7fe700 1 -- 172.21.2.111:0/3086862681 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f4db0070710 con 0x7f4dc003d360 2021-04-03T03:09:10.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.099+0000 7f4dc5b83700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dac056f20 0x7f4dac0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.099+0000 7f4dc5b83700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dac056f20 0x7f4dac0593e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4db8000e90 tx=0x7f4db8007610).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.099+0000 7f4dbf7fe700 1 -- 172.21.2.111:0/3086862681 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4db0045cc0 con 0x7f4dc003d360 2021-04-03T03:09:10.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.227+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4dc0004da0 con 0x7f4dc003d360 2021-04-03T03:09:10.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.227+0000 7f4dbf7fe700 1 -- 172.21.2.111:0/3086862681 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+5627 (secure 0 0 0) 0x7f4db0009db0 con 0x7f4dc003d360 2021-04-03T03:09:10.237 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:10.237 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:09:10.237 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:09:10.238 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.238 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:09:10.238 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:09:10.238 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.238 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:09:10.239 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.239 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.239 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.239 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.240 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.240 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.240 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:09:10.240 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:09:10.240 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.241 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:09:10.241 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.241 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.241 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.242 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.242 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.242 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.242 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:09:10.242 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:09:10.243 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.243 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.243 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.243 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.243 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.244 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.244 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.244 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:09:10.244 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:09:10.244 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.245 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.245 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.245 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.245 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.245 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.246 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.246 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:09:10.246 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:09:10.246 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.247 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.247 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.247 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.247 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.248 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.248 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:09:10.248 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:09:10.248 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.249 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.249 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.249 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.249 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.249 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.250 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.250 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:09:10.250 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:09:10.250 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.250 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.251 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.251 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.251 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.251 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.252 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:09:10.252 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:09:10.252 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.253 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.253 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.253 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.253 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.253 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.254 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:09:10.254 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:09:10.254 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.255 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.255 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.255 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.255 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.255 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:09:10.256 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:09:10.256 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.256 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.257 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.257 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.257 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.257 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:09:10.258 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:09:10.258 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:09:10.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow r", 2021-04-03T03:09:10.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow rw", 2021-04-03T03:09:10.259 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rwx" 2021-04-03T03:09:10.259 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.259 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.259 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.259 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:09:10.260 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:09:10.260 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:09:10.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow r", 2021-04-03T03:09:10.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow rw", 2021-04-03T03:09:10.261 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rwx" 2021-04-03T03:09:10.261 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.261 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.261 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.262 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:09:10.262 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:09:10.262 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:09:10.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow r", 2021-04-03T03:09:10.263 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow rw", 2021-04-03T03:09:10.263 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rwx" 2021-04-03T03:09:10.263 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.263 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.263 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.264 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:09:10.264 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:09:10.264 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:09:10.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:09:10.265 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:09:10.265 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.265 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.265 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.266 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.266 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.bootstrap-mds", 2021-04-03T03:09:10.266 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAq3GdgHdshNBAAuXvvkSpJR4Y+X3tFYFNmEQ==", 2021-04-03T03:09:10.266 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.266 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile bootstrap-mds" 2021-04-03T03:09:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.bootstrap-mgr", 2021-04-03T03:09:10.267 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAq3GdgsOchNBAAKEVIhqv8pcJcexzGj0C3Ig==", 2021-04-03T03:09:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile bootstrap-mgr" 2021-04-03T03:09:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.268 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.269 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.bootstrap-osd", 2021-04-03T03:09:10.269 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAq3GdgzPIhNBAAQhwBH7V2yXzD+LjEH1zktQ==", 2021-04-03T03:09:10.269 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.269 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile bootstrap-osd" 2021-04-03T03:09:10.269 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.270 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.270 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.270 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.bootstrap-rbd", 2021-04-03T03:09:10.270 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAq3Gdgav0hNBAAp/8C0LZq7k7edu5WN72/Zg==", 2021-04-03T03:09:10.270 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.271 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile bootstrap-rbd" 2021-04-03T03:09:10.271 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.271 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.271 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.271 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.bootstrap-rbd-mirror", 2021-04-03T03:09:10.272 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAq3GdgXwkiNBAA14Vmn8S4LkIbSxa7j2367Q==", 2021-04-03T03:09:10.272 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.272 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2021-04-03T03:09:10.272 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.272 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.273 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.273 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.bootstrap-rgw", 2021-04-03T03:09:10.273 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAq3GdgqBQiNBAAyOMA/hEj42WmCylriD6stA==", 2021-04-03T03:09:10.273 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.273 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile bootstrap-rgw" 2021-04-03T03:09:10.274 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.274 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.274 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.274 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:09:10.274 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:09:10.275 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.275 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:09:10.275 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:09:10.275 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.275 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.276 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:10.276 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:10.276 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:09:10.276 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:09:10.276 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:09:10.277 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:09:10.277 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:09:10.277 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:09:10.277 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.277 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:10.278 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:09:10.278 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:09:10.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.227+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dac056f20 msgr2=0x7f4dac0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.227+0000 7f4dc7386700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dac056f20 0x7f4dac0593e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4db8000e90 tx=0x7f4db8007610).stop 2021-04-03T03:09:10.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.227+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc003d360 msgr2=0x7f4dc00d7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.227+0000 7f4dc7386700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc003d360 0x7f4dc00d7320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f4db0003070 tx=0x7f4db0008040).stop 2021-04-03T03:09:10.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 shutdown_connections 2021-04-03T03:09:10.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dc00a2fc0 0x7f4dc0039a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dc003d360 0x7f4dc00d7320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dac056f20 0x7f4dac0593e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 --2- 172.21.2.111:0/3086862681 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dc00dbb90 0x7f4dc003af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 >> 172.21.2.111:0/3086862681 conn(0x7f4dc00462c0 msgr2=0x7f4dc0045380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 shutdown_connections 2021-04-03T03:09:10.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.231+0000 7f4dc7386700 1 -- 172.21.2.111:0/3086862681 wait complete. 2021-04-03T03:09:10.282 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mds 2021-04-03T03:09:10.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890359e700 1 Processor -- start 2021-04-03T03:09:10.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890359e700 1 -- start start 2021-04-03T03:09:10.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f88fc03f880 0x7f88fc0d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc0067e0 0x7f88fc0d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88fc0d1c50 con 0x7f88fc005640 2021-04-03T03:09:10.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88fc0d5ff0 con 0x7f88fc03f880 2021-04-03T03:09:10.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0429c0 con 0x7f88fc0067e0 2021-04-03T03:09:10.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f8902d9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f88fc005640 0x7f88fc0cb760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51422/0 (socket says 172.21.2.111:51422) 2021-04-03T03:09:10.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f8902d9d700 1 -- 172.21.2.111:0/2675366381 learned_addr learned my addr 172.21.2.111:0/2675366381 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:10.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083799160 0 0) 0x7f88fc0d1c50 con 0x7f88fc005640 2021-04-03T03:09:10.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e4003240 con 0x7f88fc005640 2021-04-03T03:09:10.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2122561175 0 0) 0x7f88e4003240 con 0x7f88fc005640 2021-04-03T03:09:10.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc0067e0 msgr2=0x7f88fc0d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc0067e0 0x7f88fc0d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890259c700 1 --2- 172.21.2.111:0/2675366381 >> v2:172.21.2.115:3300/0 conn(0x7f88fc03f880 0x7f88fc0d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 >> v2:172.21.2.115:3300/0 conn(0x7f88fc03f880 msgr2=0x7f88fc0d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 --2- 172.21.2.111:0/2675366381 >> v2:172.21.2.115:3300/0 conn(0x7f88fc03f880 0x7f88fc0d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc03fc60 con 0x7f88fc005640 2021-04-03T03:09:10.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f89040450f0 con 0x7f88fc005640 2021-04-03T03:09:10.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890259c700 1 --2- 172.21.2.111:0/2675366381 >> v2:172.21.2.115:3300/0 conn(0x7f88fc03f880 0x7f88fc0d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:10.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 >> v1:172.21.2.111:6789/0 conn(0x7f88fc005640 legacy=0x7f88fc0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88e40037b0 0x7f88e4003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc0067e0 0x7f88e40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88e40049d0 0x7f88e4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0d5ff0 con 0x7f88e40049d0 2021-04-03T03:09:10.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0429c0 con 0x7f88e40037b0 2021-04-03T03:09:10.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40041b0 con 0x7f88fc0067e0 2021-04-03T03:09:10.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f8902d9d700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88e40049d0 0x7f88e4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f8902d9d700 1 -- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc0067e0 msgr2=0x7f88e40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.343+0000 7f8902d9d700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc0067e0 0x7f88e40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f8902d9d700 1 -- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88e40037b0 msgr2=0x7f88e4003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f8902d9d700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88e40037b0 0x7f88e4003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f8902d9d700 1 -- 172.21.2.111:0/2675366381 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4003240 con 0x7f88e40049d0 2021-04-03T03:09:10.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f8902d9d700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88e40049d0 0x7f88e4004e50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f88e40054b0 tx=0x7f890405ad70).ready entity=mon.0 client_cookie=b3ed310e62b031a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89040699f0 con 0x7f88e40049d0 2021-04-03T03:09:10.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890405b040 con 0x7f88e40049d0 2021-04-03T03:09:10.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890159a700 1 -- 172.21.2.111:0/2675366381 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8904061300 con 0x7f88e40049d0 2021-04-03T03:09:10.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/2675366381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88e40049d0 msgr2=0x7f88e4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 --2- 172.21.2.111:0/2675366381 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88e40049d0 0x7f88e4004e50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f88e40054b0 tx=0x7f890405ad70).stop 2021-04-03T03:09:10.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f8902d9d700 1 -- 172.21.2.111:0/2675366381 reap_dead start 2021-04-03T03:09:10.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/2675366381 shutdown_connections 2021-04-03T03:09:10.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/2675366381 >> 172.21.2.111:0/2675366381 conn(0x7f88fc0462b0 msgr2=0x7f88fc0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/2675366381 shutdown_connections 2021-04-03T03:09:10.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/2675366381 wait complete. 2021-04-03T03:09:10.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 Processor -- start 2021-04-03T03:09:10.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- start start 2021-04-03T03:09:10.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88fc0be350 0x7f88fc03eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc03b4e0 0x7f88fc03b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88fc03be50 0x7f88fc0e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc03cec0 con 0x7f88fc0be350 2021-04-03T03:09:10.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0be730 con 0x7f88fc03be50 2021-04-03T03:09:10.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc03c980 con 0x7f88fc03b4e0 2021-04-03T03:09:10.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88fc0be350 0x7f88fc03eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88fc0be350 0x7f88fc03eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50642/0 (socket says 172.21.2.111:50642) 2021-04-03T03:09:10.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 -- 172.21.2.111:0/3876698686 learned_addr learned my addr 172.21.2.111:0/3876698686 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:10.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 -- 172.21.2.111:0/3876698686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc03b4e0 msgr2=0x7f88fc03b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc03b4e0 0x7f88fc03b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 -- 172.21.2.111:0/3876698686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88fc03be50 msgr2=0x7f88fc0e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88fc03be50 0x7f88fc0e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 -- 172.21.2.111:0/3876698686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc03cc20 con 0x7f88fc0be350 2021-04-03T03:09:10.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890259c700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88fc0be350 0x7f88fc03eb80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f88f4007000 tx=0x7f88f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f88fb7fe700 1 -- 172.21.2.111:0/3876698686 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88f4015070 con 0x7f88fc0be350 2021-04-03T03:09:10.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f88fb7fe700 1 -- 172.21.2.111:0/3876698686 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f4010580 con 0x7f88fc0be350 2021-04-03T03:09:10.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f88fb7fe700 1 -- 172.21.2.111:0/3876698686 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88f4010920 con 0x7f88fc0be350 2021-04-03T03:09:10.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc03f070 con 0x7f88fc0be350 2021-04-03T03:09:10.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc03c650 con 0x7f88fc0be350 2021-04-03T03:09:10.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc039c80 con 0x7f88fc0be350 2021-04-03T03:09:10.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.347+0000 7f88fb7fe700 1 -- 172.21.2.111:0/3876698686 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f88f4010a80 con 0x7f88fc0be350 2021-04-03T03:09:10.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.351+0000 7f88fb7fe700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88e8056f70 0x7f88e8059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.351+0000 7f88fb7fe700 1 -- 172.21.2.111:0/3876698686 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f88f406fe50 con 0x7f88fc0be350 2021-04-03T03:09:10.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.351+0000 7f8901d9b700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88e8056f70 0x7f88e8059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.351+0000 7f8901d9b700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88e8056f70 0x7f88e8059430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f88ec002f30 tx=0x7f88ec0069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.351+0000 7f88fb7fe700 1 -- 172.21.2.111:0/3876698686 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f88f4045400 con 0x7f88fc0be350 2021-04-03T03:09:10.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.479+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f88fc004d30 con 0x7f88fc0be350 2021-04-03T03:09:10.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f88fb7fe700 1 -- 172.21.2.111:0/3876698686 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f88f4048a20 con 0x7f88fc0be350 2021-04-03T03:09:10.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88e8056f70 msgr2=0x7f88e8059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88e8056f70 0x7f88e8059430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f88ec002f30 tx=0x7f88ec0069f0).stop 2021-04-03T03:09:10.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88fc0be350 msgr2=0x7f88fc03eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88fc0be350 0x7f88fc03eb80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f88f4007000 tx=0x7f88f4008040).stop 2021-04-03T03:09:10.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 shutdown_connections 2021-04-03T03:09:10.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88e8056f70 0x7f88e8059430 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88fc03be50 0x7f88fc0e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88fc0be350 0x7f88fc03eb80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 --2- 172.21.2.111:0/3876698686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88fc03b4e0 0x7f88fc03b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 >> 172.21.2.111:0/3876698686 conn(0x7f88fc0462b0 msgr2=0x7f88fc005a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 shutdown_connections 2021-04-03T03:09:10.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.483+0000 7f890359e700 1 -- 172.21.2.111:0/3876698686 wait complete. 2021-04-03T03:09:10.496 INFO:teuthology.orchestra.run.gibba011.stderr:updated 2021-04-03T03:09:10.500 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mgr 2021-04-03T03:09:10.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b3fff700 1 Processor -- start 2021-04-03T03:09:10.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b3fff700 1 -- start start 2021-04-03T03:09:10.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b40603f0 0x7f49b41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b40607d0 con 0x7f49b40b17b0 2021-04-03T03:09:10.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b3fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b414b090 con 0x7f49b40add50 2021-04-03T03:09:10.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b414b330 con 0x7f49b40603f0 2021-04-03T03:09:10.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b37fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f49b40b17b0 0x7f49b4147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51442/0 (socket says 172.21.2.111:51442) 2021-04-03T03:09:10.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b37fe700 1 -- 172.21.2.111:0/3386377349 learned_addr learned my addr 172.21.2.111:0/3386377349 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:10.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287204902 0 0) 0x7f49b40607d0 con 0x7f49b40b17b0 2021-04-03T03:09:10.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4994003240 con 0x7f49b40b17b0 2021-04-03T03:09:10.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1478047023 0 0) 0x7f4994003240 con 0x7f49b40b17b0 2021-04-03T03:09:10.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b40603f0 msgr2=0x7f49b41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b40603f0 0x7f49b41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 >> v1:172.21.2.115:6789/0 conn(0x7f49b40add50 legacy=0x7f49b4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b414b5d0 con 0x7f49b40b17b0 2021-04-03T03:09:10.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f49ac001a50 con 0x7f49b40b17b0 2021-04-03T03:09:10.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 >> v1:172.21.2.111:6789/0 conn(0x7f49b40b17b0 legacy=0x7f49b4147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49940037e0 0x7f4994003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b40603f0 0x7f49940044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49940049d0 0x7f4994004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b414b330 con 0x7f49940049d0 2021-04-03T03:09:10.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940042b0 con 0x7f49940037e0 2021-04-03T03:09:10.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.547+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499400b730 con 0x7f49b40603f0 2021-04-03T03:09:10.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49940049d0 0x7f4994004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 -- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b40603f0 msgr2=0x7f49940044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b40603f0 0x7f49940044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 -- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49940037e0 msgr2=0x7f4994003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49940037e0 0x7f4994003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 -- 172.21.2.111:0/3386377349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994003240 con 0x7f49940049d0 2021-04-03T03:09:10.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49940049d0 0x7f4994004e50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f49b40b5640 tx=0x7f49b414b220).ready entity=mon.0 client_cookie=4fe122d04be61a4e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49ac005b80 con 0x7f49940049d0 2021-04-03T03:09:10.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac005ce0 con 0x7f49940049d0 2021-04-03T03:09:10.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b1ffb700 1 -- 172.21.2.111:0/3386377349 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49ac0072c0 con 0x7f49940049d0 2021-04-03T03:09:10.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/3386377349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49940049d0 msgr2=0x7f4994004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 --2- 172.21.2.111:0/3386377349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49940049d0 0x7f4994004e50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f49b40b5640 tx=0x7f49b414b220).stop 2021-04-03T03:09:10.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b37fe700 1 -- 172.21.2.111:0/3386377349 reap_dead start 2021-04-03T03:09:10.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/3386377349 shutdown_connections 2021-04-03T03:09:10.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/3386377349 >> 172.21.2.111:0/3386377349 conn(0x7f49b40b61c0 msgr2=0x7f49b405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/3386377349 shutdown_connections 2021-04-03T03:09:10.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/3386377349 wait complete. 2021-04-03T03:09:10.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 Processor -- start 2021-04-03T03:09:10.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- start start 2021-04-03T03:09:10.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49b4065e60 0x7f49b4156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b4066240 0x7f49b4156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49b4062a70 0x7f49b4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40ab880 con 0x7f49b4065e60 2021-04-03T03:09:10.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4066970 con 0x7f49b4062a70 2021-04-03T03:09:10.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4157040 con 0x7f49b4066240 2021-04-03T03:09:10.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49b4065e60 0x7f49b4156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49b4065e60 0x7f49b4156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50660/0 (socket says 172.21.2.111:50660) 2021-04-03T03:09:10.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 -- 172.21.2.111:0/1236671895 learned_addr learned my addr 172.21.2.111:0/1236671895 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:10.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 -- 172.21.2.111:0/1236671895 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b4066240 msgr2=0x7f49b4156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b4066240 0x7f49b4156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 -- 172.21.2.111:0/1236671895 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49b4062a70 msgr2=0x7f49b4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49b4062a70 0x7f49b4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 -- 172.21.2.111:0/1236671895 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b4157290 con 0x7f49b4065e60 2021-04-03T03:09:10.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b2ffd700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49b4065e60 0x7f49b4156660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f49a4002f30 tx=0x7f49a4006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49abfff700 1 -- 172.21.2.111:0/1236671895 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a400d7e0 con 0x7f49b4065e60 2021-04-03T03:09:10.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49abfff700 1 -- 172.21.2.111:0/1236671895 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a4000e90 con 0x7f49b4065e60 2021-04-03T03:09:10.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b40666d0 con 0x7f49b4065e60 2021-04-03T03:09:10.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b40aac50 con 0x7f49b4065e60 2021-04-03T03:09:10.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49abfff700 1 -- 172.21.2.111:0/1236671895 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a400db30 con 0x7f49b4065e60 2021-04-03T03:09:10.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b4057660 con 0x7f49b4065e60 2021-04-03T03:09:10.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.551+0000 7f49abfff700 1 -- 172.21.2.111:0/1236671895 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f49a400dc90 con 0x7f49b4065e60 2021-04-03T03:09:10.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.555+0000 7f49abfff700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4998056ea0 0x7f4998059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.555+0000 7f49abfff700 1 -- 172.21.2.111:0/1236671895 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f49a4070000 con 0x7f49b4065e60 2021-04-03T03:09:10.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.555+0000 7f49b27fc700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4998056ea0 0x7f4998059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.555+0000 7f49b27fc700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4998056ea0 0x7f4998059360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f499c001f60 tx=0x7f499c0077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.555+0000 7f49abfff700 1 -- 172.21.2.111:0/1236671895 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f49a4045720 con 0x7f49b4065e60 2021-04-03T03:09:10.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.683+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f49b4148850 con 0x7f49b4065e60 2021-04-03T03:09:10.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49abfff700 1 -- 172.21.2.111:0/1236671895 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7f49a4048d40 con 0x7f49b4065e60 2021-04-03T03:09:10.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4998056ea0 msgr2=0x7f4998059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4998056ea0 0x7f4998059360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f499c001f60 tx=0x7f499c0077d0).stop 2021-04-03T03:09:10.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49b4065e60 msgr2=0x7f49b4156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49b4065e60 0x7f49b4156660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f49a4002f30 tx=0x7f49a4006e50).stop 2021-04-03T03:09:10.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 shutdown_connections 2021-04-03T03:09:10.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4998056ea0 0x7f4998059360 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49b4062a70 0x7f49b4159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49b4065e60 0x7f49b4156660 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 --2- 172.21.2.111:0/1236671895 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49b4066240 0x7f49b4156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 >> 172.21.2.111:0/1236671895 conn(0x7f49b40b61c0 msgr2=0x7f49b4145c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 shutdown_connections 2021-04-03T03:09:10.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.691+0000 7f49b3fff700 1 -- 172.21.2.111:0/1236671895 wait complete. 2021-04-03T03:09:10.702 INFO:teuthology.orchestra.run.gibba011.stderr:updated 2021-04-03T03:09:10.706 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-osd 2021-04-03T03:09:10.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 Processor -- start 2021-04-03T03:09:10.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- start start 2021-04-03T03:09:10.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f441403f120 0x7f4414005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4414042470 0x7f44140d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441403d840 con 0x7f4414041fe0 2021-04-03T03:09:10.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441403faa0 con 0x7f441403f120 2021-04-03T03:09:10.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441403dca0 con 0x7f4414042470 2021-04-03T03:09:10.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4414041fe0 0x7f44140d89a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51464/0 (socket says 172.21.2.111:51464) 2021-04-03T03:09:10.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 -- 172.21.2.111:0/4026980473 learned_addr learned my addr 172.21.2.111:0/4026980473 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:10.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647522158 0 0) 0x7f441403d840 con 0x7f4414041fe0 2021-04-03T03:09:10.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4400003240 con 0x7f4414041fe0 2021-04-03T03:09:10.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441b7fe700 1 --2- 172.21.2.111:0/4026980473 >> v2:172.21.2.115:3300/0 conn(0x7f441403f120 0x7f4414005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441bfff700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4414042470 0x7f44140d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 908802085 0 0) 0x7f4400003240 con 0x7f4414041fe0 2021-04-03T03:09:10.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4414042470 msgr2=0x7f44140d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4414042470 0x7f44140d90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 >> v2:172.21.2.115:3300/0 conn(0x7f441403f120 msgr2=0x7f4414005640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 --2- 172.21.2.111:0/4026980473 >> v2:172.21.2.115:3300/0 conn(0x7f441403f120 0x7f4414005640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414005b30 con 0x7f4414041fe0 2021-04-03T03:09:10.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f44080016d0 con 0x7f4414041fe0 2021-04-03T03:09:10.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 >> v1:172.21.2.111:6789/0 conn(0x7f4414041fe0 legacy=0x7f44140d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4400003730 0x7f4400003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44000044e0 0x7f4400004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4414042470 0x7f4400004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441403faa0 con 0x7f44000044e0 2021-04-03T03:09:10.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441403dca0 con 0x7f4400003730 2021-04-03T03:09:10.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4400004270 con 0x7f4414042470 2021-04-03T03:09:10.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441b7fe700 1 --2- 172.21.2.111:0/4026980473 >> v2:172.21.2.115:3300/0 conn(0x7f441403f120 0x7f4414005640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:09:10.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44000044e0 0x7f4400004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 -- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4414042470 msgr2=0x7f4400004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4414042470 0x7f4400004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 -- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4400003730 msgr2=0x7f4400003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441b7fe700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4400003730 0x7f4400003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4400003730 0x7f4400003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 -- 172.21.2.111:0/4026980473 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400003240 con 0x7f44000044e0 2021-04-03T03:09:10.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441b7fe700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4400003730 0x7f4400003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:10.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441affd700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44000044e0 0x7f4400004960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f44080013e0 tx=0x7f441403ec70).ready entity=mon.0 client_cookie=11ea5f649c905494 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f440800ce00 con 0x7f44000044e0 2021-04-03T03:09:10.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44080072e0 con 0x7f44000044e0 2021-04-03T03:09:10.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441a7fc700 1 -- 172.21.2.111:0/4026980473 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44080055f0 con 0x7f44000044e0 2021-04-03T03:09:10.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- 172.21.2.111:0/4026980473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44000044e0 msgr2=0x7f4400004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 --2- 172.21.2.111:0/4026980473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44000044e0 0x7f4400004960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f44080013e0 tx=0x7f441403ec70).stop 2021-04-03T03:09:10.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f441bfff700 1 -- 172.21.2.111:0/4026980473 reap_dead start 2021-04-03T03:09:10.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- 172.21.2.111:0/4026980473 shutdown_connections 2021-04-03T03:09:10.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- 172.21.2.111:0/4026980473 >> 172.21.2.111:0/4026980473 conn(0x7f44140462b0 msgr2=0x7f44140466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- 172.21.2.111:0/4026980473 shutdown_connections 2021-04-03T03:09:10.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.755+0000 7f4420838700 1 -- 172.21.2.111:0/4026980473 wait complete. 2021-04-03T03:09:10.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 Processor -- start 2021-04-03T03:09:10.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 -- start start 2021-04-03T03:09:10.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44140a2960 0x7f44140a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f441409f4a0 0x7f44140ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44140de140 0x7f44140ea450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441409fb80 con 0x7f441409f4a0 2021-04-03T03:09:10.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44140de5d0 con 0x7f44140de140 2021-04-03T03:09:10.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441403c8b0 con 0x7f44140a2960 2021-04-03T03:09:10.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f441409f4a0 0x7f44140ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f441409f4a0 0x7f44140ddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50680/0 (socket says 172.21.2.111:50680) 2021-04-03T03:09:10.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 -- 172.21.2.111:0/3713922066 learned_addr learned my addr 172.21.2.111:0/3713922066 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:10.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441bfff700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44140de140 0x7f44140ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 -- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44140a2960 msgr2=0x7f44140a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44140a2960 0x7f44140a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 -- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44140de140 msgr2=0x7f44140ea450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44140de140 0x7f44140ea450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 -- 172.21.2.111:0/3713922066 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44140e7a80 con 0x7f441409f4a0 2021-04-03T03:09:10.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441bfff700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44140de140 0x7f44140ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:10.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441affd700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f441409f4a0 0x7f44140ddc50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f44080013e0 tx=0x7f4408013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4418ff9700 1 -- 172.21.2.111:0/3713922066 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f440801a070 con 0x7f441409f4a0 2021-04-03T03:09:10.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4418ff9700 1 -- 172.21.2.111:0/3713922066 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408005e30 con 0x7f441409f4a0 2021-04-03T03:09:10.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4418ff9700 1 -- 172.21.2.111:0/3713922066 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44080016d0 con 0x7f441409f4a0 2021-04-03T03:09:10.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44140a3230 con 0x7f441409f4a0 2021-04-03T03:09:10.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44140eab50 con 0x7f441409f4a0 2021-04-03T03:09:10.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44140e78e0 con 0x7f441409f4a0 2021-04-03T03:09:10.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4418ff9700 1 -- 172.21.2.111:0/3713922066 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4408026070 con 0x7f441409f4a0 2021-04-03T03:09:10.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4418ff9700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f4418ff9700 1 -- 172.21.2.111:0/3713922066 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f4408070680 con 0x7f441409f4a0 2021-04-03T03:09:10.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441b7fe700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.759+0000 7f441b7fe700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4410008000 tx=0x7f4410008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.763+0000 7f4418ff9700 1 -- 172.21.2.111:0/3713922066 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f44140e78e0 con 0x7f441409f4a0 2021-04-03T03:09:10.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.891+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f44140e78e0 con 0x7f441409f4a0 2021-04-03T03:09:10.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4418ff9700 1 -- 172.21.2.111:0/3713922066 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f44140e78e0 con 0x7f441409f4a0 2021-04-03T03:09:10.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 msgr2=0x7f4404059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4410008000 tx=0x7f4410008040).stop 2021-04-03T03:09:10.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f441409f4a0 msgr2=0x7f44140ddc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f441409f4a0 0x7f44140ddc50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f44080013e0 tx=0x7f4408013000).stop 2021-04-03T03:09:10.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 shutdown_connections 2021-04-03T03:09:10.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44140de140 0x7f44140ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44140a2960 0x7f44140a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 --2- 172.21.2.111:0/3713922066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f441409f4a0 0x7f44140ddc50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 >> 172.21.2.111:0/3713922066 conn(0x7f44140462b0 msgr2=0x7f44140434f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 shutdown_connections 2021-04-03T03:09:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.895+0000 7f4420838700 1 -- 172.21.2.111:0/3713922066 wait complete. 2021-04-03T03:09:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:updated 2021-04-03T03:09:10.911 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd 2021-04-03T03:09:10.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 Processor -- start 2021-04-03T03:09:10.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 -- start start 2021-04-03T03:09:10.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa9fc03f880 0x7fa9fc0d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa9fc0067e0 0x7fa9fc0d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9fc0d1c50 con 0x7fa9fc005640 2021-04-03T03:09:10.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d5ff0 con 0x7fa9fc03f880 2021-04-03T03:09:10.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc0429c0 con 0x7fa9fc0067e0 2021-04-03T03:09:10.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0a37e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa9fc005640 0x7fa9fc0cb760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51482/0 (socket says 172.21.2.111:51482) 2021-04-03T03:09:10.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa09b7d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa9fc03f880 0x7fa9fc0d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0a37e700 1 -- 172.21.2.111:0/3033784152 learned_addr learned my addr 172.21.2.111:0/3033784152 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:10.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa09b7d700 1 -- 172.21.2.111:0/3033784152 >> v2:172.21.2.115:3300/0 conn(0x7fa9fc0067e0 msgr2=0x7fa9fc0d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:10.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa09b7d700 1 --2- 172.21.2.111:0/3033784152 >> v2:172.21.2.115:3300/0 conn(0x7fa9fc0067e0 0x7fa9fc0d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa09b7d700 1 -- 172.21.2.111:0/3033784152 >> v1:172.21.2.111:6789/0 conn(0x7fa9fc005640 legacy=0x7fa9fc0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa09b7d700 1 -- 172.21.2.111:0/3033784152 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc03fc60 con 0x7fa9fc03f880 2021-04-03T03:09:10.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa09b7d700 1 --2- 172.21.2.111:0/3033784152 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc03f880 0x7fa9fc0d5b00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7faa00002f30 tx=0x7faa00006dd0).ready entity=mon.0 client_cookie=6623da855f30ea75 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa08b7b700 1 -- 172.21.2.111:0/3033784152 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa000097f0 con 0x7fa9fc03f880 2021-04-03T03:09:10.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa08b7b700 1 -- 172.21.2.111:0/3033784152 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa00009950 con 0x7fa9fc03f880 2021-04-03T03:09:10.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa08b7b700 1 -- 172.21.2.111:0/3033784152 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa00009cb0 con 0x7fa9fc03f880 2021-04-03T03:09:10.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 -- 172.21.2.111:0/3033784152 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc03f880 msgr2=0x7fa9fc0d5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.959+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/3033784152 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc03f880 0x7fa9fc0d5b00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7faa00002f30 tx=0x7faa00006dd0).stop 2021-04-03T03:09:10.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- 172.21.2.111:0/3033784152 shutdown_connections 2021-04-03T03:09:10.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/3033784152 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc03f880 0x7fa9fc0d5b00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:10.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/3033784152 >> v2:172.21.2.115:3300/0 conn(0x7fa9fc0067e0 0x7fa9fc0d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- 172.21.2.111:0/3033784152 >> 172.21.2.111:0/3033784152 conn(0x7fa9fc0462b0 msgr2=0x7fa9fc0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:10.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- 172.21.2.111:0/3033784152 shutdown_connections 2021-04-03T03:09:10.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- 172.21.2.111:0/3033784152 wait complete. 2021-04-03T03:09:10.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 Processor -- start 2021-04-03T03:09:10.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- start start 2021-04-03T03:09:10.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc005640 0x7fa9fc0d4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9fc0067e0 0x7fa9fc03aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9fc03f880 0x7fa9fc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d6840 con 0x7fa9fc005640 2021-04-03T03:09:10.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d6ae0 con 0x7fa9fc0067e0 2021-04-03T03:09:10.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d6d80 con 0x7fa9fc03f880 2021-04-03T03:09:10.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc005640 0x7fa9fc0d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc005640 0x7fa9fc0d4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50694/0 (socket says 172.21.2.111:50694) 2021-04-03T03:09:10.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 -- 172.21.2.111:0/806014380 learned_addr learned my addr 172.21.2.111:0/806014380 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:10.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 -- 172.21.2.111:0/806014380 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9fc03f880 msgr2=0x7fa9fc03b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9fc03f880 0x7fa9fc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 -- 172.21.2.111:0/806014380 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9fc0067e0 msgr2=0x7fa9fc03aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:10.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9fc0067e0 0x7fa9fc03aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:10.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 -- 172.21.2.111:0/806014380 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0d4500 con 0x7fa9fc005640 2021-04-03T03:09:10.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa09b7d700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc005640 0x7fa9fc0d4010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7faa00002f30 tx=0x7faa00015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7fa9f6ffd700 1 -- 172.21.2.111:0/806014380 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa00011ce0 con 0x7fa9fc005640 2021-04-03T03:09:10.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7fa9f6ffd700 1 -- 172.21.2.111:0/806014380 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa00011e40 con 0x7fa9fc005640 2021-04-03T03:09:10.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9fc0d7220 con 0x7fa9fc005640 2021-04-03T03:09:10.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9fc0d63a0 con 0x7fa9fc005640 2021-04-03T03:09:10.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7fa9f6ffd700 1 -- 172.21.2.111:0/806014380 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa0001e650 con 0x7fa9fc005640 2021-04-03T03:09:10.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9fc0d6f40 con 0x7fa9fc005640 2021-04-03T03:09:10.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7fa9f6ffd700 1 -- 172.21.2.111:0/806014380 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7faa00025030 con 0x7fa9fc005640 2021-04-03T03:09:10.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7fa9f6ffd700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9e4056ef0 0x7fa9e40593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:10.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7fa9f6ffd700 1 -- 172.21.2.111:0/806014380 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7faa000713e0 con 0x7fa9fc005640 2021-04-03T03:09:10.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0937c700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9e4056ef0 0x7fa9e40593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:10.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.963+0000 7faa0937c700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9e4056ef0 0x7fa9e40593b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fa9f8002f30 tx=0x7fa9f8006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:10.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:10.967+0000 7fa9f6ffd700 1 -- 172.21.2.111:0/806014380 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7faa00046b00 con 0x7fa9fc005640 2021-04-03T03:09:11.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.091+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa9fc004d30 con 0x7fa9fc005640 2021-04-03T03:09:11.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.095+0000 7fa9f6ffd700 1 -- 172.21.2.111:0/806014380 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7faa000230a0 con 0x7fa9fc005640 2021-04-03T03:09:11.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.095+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9e4056ef0 msgr2=0x7fa9e40593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.095+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9e4056ef0 0x7fa9e40593b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fa9f8002f30 tx=0x7fa9f8006a60).stop 2021-04-03T03:09:11.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc005640 msgr2=0x7fa9fc0d4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc005640 0x7fa9fc0d4010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7faa00002f30 tx=0x7faa00015000).stop 2021-04-03T03:09:11.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 shutdown_connections 2021-04-03T03:09:11.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9fc03f880 0x7fa9fc03b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9fc005640 0x7fa9fc0d4010 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9e4056ef0 0x7fa9e40593b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 --2- 172.21.2.111:0/806014380 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9fc0067e0 0x7fa9fc03aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 >> 172.21.2.111:0/806014380 conn(0x7fa9fc0462b0 msgr2=0x7fa9fc042ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 shutdown_connections 2021-04-03T03:09:11.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.099+0000 7faa0ab7f700 1 -- 172.21.2.111:0/806014380 wait complete. 2021-04-03T03:09:11.109 INFO:teuthology.orchestra.run.gibba011.stderr:updated 2021-04-03T03:09:11.112 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd-mirror 2021-04-03T03:09:11.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.159+0000 7f560d330700 1 Processor -- start 2021-04-03T03:09:11.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.159+0000 7f560d330700 1 -- start start 2021-04-03T03:09:11.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f56080603f0 0x7f560813cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5608143310 0x7f56081437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56080607d0 con 0x7f56080acb30 2021-04-03T03:09:11.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5608147860 con 0x7f5608143310 2021-04-03T03:09:11.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56080af840 con 0x7f56080603f0 2021-04-03T03:09:11.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5608143310 0x7f56081437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5608143310 0x7f56081437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50702/0 (socket says 172.21.2.111:50702) 2021-04-03T03:09:11.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 -- 172.21.2.111:0/321630106 learned_addr learned my addr 172.21.2.111:0/321630106 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 -- 172.21.2.111:0/321630106 >> v2:172.21.2.115:3300/0 conn(0x7f56080603f0 msgr2=0x7f560813cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:11.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 --2- 172.21.2.111:0/321630106 >> v2:172.21.2.115:3300/0 conn(0x7f56080603f0 0x7f560813cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 -- 172.21.2.111:0/321630106 >> v1:172.21.2.111:6789/0 conn(0x7f56080acb30 legacy=0x7f560813d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 -- 172.21.2.111:0/321630106 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56081473c0 con 0x7f5608143310 2021-04-03T03:09:11.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 --2- 172.21.2.111:0/321630106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f56081437a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f56000032b0 tx=0x7f5600009270).ready entity=mon.0 client_cookie=14168c08582d2a37 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f5606ffd700 1 -- 172.21.2.111:0/321630106 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f560000bc00 con 0x7f5608143310 2021-04-03T03:09:11.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f5606ffd700 1 -- 172.21.2.111:0/321630106 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560000bd60 con 0x7f5608143310 2021-04-03T03:09:11.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f5606ffd700 1 -- 172.21.2.111:0/321630106 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5600011400 con 0x7f5608143310 2021-04-03T03:09:11.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- 172.21.2.111:0/321630106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 msgr2=0x7f56081437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- 172.21.2.111:0/321630106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f56081437a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f56000032b0 tx=0x7f5600009270).stop 2021-04-03T03:09:11.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- 172.21.2.111:0/321630106 shutdown_connections 2021-04-03T03:09:11.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- 172.21.2.111:0/321630106 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f56081437a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- 172.21.2.111:0/321630106 >> v2:172.21.2.115:3300/0 conn(0x7f56080603f0 0x7f560813cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- 172.21.2.111:0/321630106 >> 172.21.2.111:0/321630106 conn(0x7f56080b61c0 msgr2=0x7f560805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- 172.21.2.111:0/321630106 shutdown_connections 2021-04-03T03:09:11.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- 172.21.2.111:0/321630106 wait complete. 2021-04-03T03:09:11.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 Processor -- start 2021-04-03T03:09:11.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- start start 2021-04-03T03:09:11.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56080603f0 0x7f5608149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f5608155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56081423e0 0x7f5608155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5608141960 con 0x7f5608143310 2021-04-03T03:09:11.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56081428b0 con 0x7f56081423e0 2021-04-03T03:09:11.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5608142b50 con 0x7f56080603f0 2021-04-03T03:09:11.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f5608155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f5608155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50712/0 (socket says 172.21.2.111:50712) 2021-04-03T03:09:11.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 -- 172.21.2.111:0/1154643677 learned_addr learned my addr 172.21.2.111:0/1154643677 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f5607fff700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56080603f0 0x7f5608149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560cb2f700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56081423e0 0x7f5608155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 -- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56080603f0 msgr2=0x7f5608149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56080603f0 0x7f5608149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 -- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56081423e0 msgr2=0x7f5608155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56081423e0 0x7f5608155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 -- 172.21.2.111:0/1154643677 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608142df0 con 0x7f5608143310 2021-04-03T03:09:11.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56077fe700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f5608155880 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f560806b360 tx=0x7f55fc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56057fa700 1 -- 172.21.2.111:0/1154643677 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55fc00e4a0 con 0x7f5608143310 2021-04-03T03:09:11.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56057fa700 1 -- 172.21.2.111:0/1154643677 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc00e600 con 0x7f5608143310 2021-04-03T03:09:11.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56081402f0 con 0x7f5608143310 2021-04-03T03:09:11.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f56057fa700 1 -- 172.21.2.111:0/1154643677 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55fc00e960 con 0x7f5608143310 2021-04-03T03:09:11.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.163+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560813ffd0 con 0x7f5608143310 2021-04-03T03:09:11.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.167+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5608142f80 con 0x7f5608143310 2021-04-03T03:09:11.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.167+0000 7f56057fa700 1 -- 172.21.2.111:0/1154643677 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f55fc019030 con 0x7f5608143310 2021-04-03T03:09:11.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.167+0000 7f56057fa700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f55e8056fb0 0x7f55e8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.167+0000 7f56057fa700 1 -- 172.21.2.111:0/1154643677 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f55fc06f5c0 con 0x7f5608143310 2021-04-03T03:09:11.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.167+0000 7f5607fff700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f55e8056fb0 0x7f55e8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.167+0000 7f5607fff700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f55e8056fb0 0x7f55e8059470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f55f800a000 tx=0x7f55f8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.167+0000 7f56057fa700 1 -- 172.21.2.111:0/1154643677 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f55fc044bd0 con 0x7f5608143310 2021-04-03T03:09:11.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.295+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f5608057660 con 0x7f5608143310 2021-04-03T03:09:11.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f56057fa700 1 -- 172.21.2.111:0/1154643677 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f55fc0481f0 con 0x7f5608143310 2021-04-03T03:09:11.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f55e8056fb0 msgr2=0x7f55e8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f55e8056fb0 0x7f55e8059470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f55f800a000 tx=0x7f55f8008040).stop 2021-04-03T03:09:11.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 msgr2=0x7f5608155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f5608155880 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f560806b360 tx=0x7f55fc007c30).stop 2021-04-03T03:09:11.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 shutdown_connections 2021-04-03T03:09:11.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f55e8056fb0 0x7f55e8059470 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56081423e0 0x7f5608155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56080603f0 0x7f5608149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 --2- 172.21.2.111:0/1154643677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5608143310 0x7f5608155880 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 >> 172.21.2.111:0/1154643677 conn(0x7f56080b61c0 msgr2=0x7f56080aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 shutdown_connections 2021-04-03T03:09:11.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.299+0000 7f560d330700 1 -- 172.21.2.111:0/1154643677 wait complete. 2021-04-03T03:09:11.311 INFO:teuthology.orchestra.run.gibba011.stderr:updated 2021-04-03T03:09:11.315 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rgw 2021-04-03T03:09:11.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 Processor -- start 2021-04-03T03:09:11.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 -- start start 2021-04-03T03:09:11.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f73640603f0 0x7f73641447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 0x7f7364144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73640607d0 con 0x7f73640b17b0 2021-04-03T03:09:11.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73641451c0 con 0x7f73640603f0 2021-04-03T03:09:11.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736414b090 con 0x7f73640add50 2021-04-03T03:09:11.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7368d73700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f73640b17b0 0x7f7364147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51508/0 (socket says 172.21.2.111:51508) 2021-04-03T03:09:11.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7368d73700 1 -- 172.21.2.111:0/3072315945 learned_addr learned my addr 172.21.2.111:0/3072315945 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:11.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7363fff700 1 --2- 172.21.2.111:0/3072315945 >> v2:172.21.2.111:3300/0 conn(0x7f73640603f0 0x7f73641447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7362ffd700 1 -- 172.21.2.111:0/3072315945 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890464352 0 0) 0x7f73640607d0 con 0x7f73640b17b0 2021-04-03T03:09:11.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7362ffd700 1 -- 172.21.2.111:0/3072315945 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7344003240 con 0x7f73640b17b0 2021-04-03T03:09:11.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7363fff700 1 -- 172.21.2.111:0/3072315945 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 msgr2=0x7f7364144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:11.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7363fff700 1 --2- 172.21.2.111:0/3072315945 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 0x7f7364144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7363fff700 1 -- 172.21.2.111:0/3072315945 >> v1:172.21.2.111:6789/0 conn(0x7f73640b17b0 legacy=0x7f7364147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7363fff700 1 -- 172.21.2.111:0/3072315945 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736414b330 con 0x7f73640603f0 2021-04-03T03:09:11.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7363fff700 1 --2- 172.21.2.111:0/3072315945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73641447e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f7354002f30 tx=0x7f7354006dd0).ready entity=mon.0 client_cookie=11c3c60fd0f1c480 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7362ffd700 1 -- 172.21.2.111:0/3072315945 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7354009760 con 0x7f73640603f0 2021-04-03T03:09:11.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7362ffd700 1 -- 172.21.2.111:0/3072315945 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73540098c0 con 0x7f73640603f0 2021-04-03T03:09:11.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7362ffd700 1 -- 172.21.2.111:0/3072315945 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7354009c20 con 0x7f73640603f0 2021-04-03T03:09:11.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 -- 172.21.2.111:0/3072315945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 msgr2=0x7f73641447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.363+0000 7f7369574700 1 --2- 172.21.2.111:0/3072315945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73641447e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f7354002f30 tx=0x7f7354006dd0).stop 2021-04-03T03:09:11.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- 172.21.2.111:0/3072315945 shutdown_connections 2021-04-03T03:09:11.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 --2- 172.21.2.111:0/3072315945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73641447e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 --2- 172.21.2.111:0/3072315945 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 0x7f7364144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- 172.21.2.111:0/3072315945 >> 172.21.2.111:0/3072315945 conn(0x7f73640b61c0 msgr2=0x7f736405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- 172.21.2.111:0/3072315945 shutdown_connections 2021-04-03T03:09:11.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- 172.21.2.111:0/3072315945 wait complete. 2021-04-03T03:09:11.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 Processor -- start 2021-04-03T03:09:11.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- start start 2021-04-03T03:09:11.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73640a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 0x7f73640b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73640b17b0 0x7f7364156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7364148650 con 0x7f73640603f0 2021-04-03T03:09:11.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73640a70a0 con 0x7f73640b17b0 2021-04-03T03:09:11.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73640a7340 con 0x7f73640add50 2021-04-03T03:09:11.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73640a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73640a7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50722/0 (socket says 172.21.2.111:50722) 2021-04-03T03:09:11.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 -- 172.21.2.111:0/370619507 learned_addr learned my addr 172.21.2.111:0/370619507 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 -- 172.21.2.111:0/370619507 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 msgr2=0x7f73640b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 0x7f73640b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 -- 172.21.2.111:0/370619507 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73640b17b0 msgr2=0x7f7364156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73640b17b0 0x7f7364156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 -- 172.21.2.111:0/370619507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73640a75e0 con 0x7f73640603f0 2021-04-03T03:09:11.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7363fff700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73640a7a80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f7354002f30 tx=0x7f7354015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73617fa700 1 -- 172.21.2.111:0/370619507 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7354011e80 con 0x7f73640603f0 2021-04-03T03:09:11.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73617fa700 1 -- 172.21.2.111:0/370619507 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735401c5c0 con 0x7f73640603f0 2021-04-03T03:09:11.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73617fa700 1 -- 172.21.2.111:0/370619507 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f735401c920 con 0x7f73640603f0 2021-04-03T03:09:11.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73640a7f70 con 0x7f73640603f0 2021-04-03T03:09:11.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73641476b0 con 0x7f73640603f0 2021-04-03T03:09:11.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73640a7770 con 0x7f73640603f0 2021-04-03T03:09:11.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73617fa700 1 -- 172.21.2.111:0/370619507 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f735400f070 con 0x7f73640603f0 2021-04-03T03:09:11.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73617fa700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7348056ef0 0x7f73480593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73617fa700 1 -- 172.21.2.111:0/370619507 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f7354071de0 con 0x7f73640603f0 2021-04-03T03:09:11.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73637fe700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7348056ef0 0x7f73480593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.367+0000 7f73637fe700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7348056ef0 0x7f73480593b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f734c002f30 tx=0x7f734c006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.371+0000 7f73617fa700 1 -- 172.21.2.111:0/370619507 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f73540473f0 con 0x7f73640603f0 2021-04-03T03:09:11.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.503+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f7364057660 con 0x7f73640603f0 2021-04-03T03:09:11.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f73617fa700 1 -- 172.21.2.111:0/370619507 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f735404aa10 con 0x7f73640603f0 2021-04-03T03:09:11.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7348056ef0 msgr2=0x7f73480593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7348056ef0 0x7f73480593b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f734c002f30 tx=0x7f734c006a60).stop 2021-04-03T03:09:11.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 msgr2=0x7f73640a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73640a7a80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f7354002f30 tx=0x7f7354015000).stop 2021-04-03T03:09:11.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 shutdown_connections 2021-04-03T03:09:11.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7348056ef0 0x7f73480593b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73640b17b0 0x7f7364156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73640603f0 0x7f73640a7a80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 --2- 172.21.2.111:0/370619507 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73640add50 0x7f73640b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 >> 172.21.2.111:0/370619507 conn(0x7f73640b61c0 msgr2=0x7f736414a810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 shutdown_connections 2021-04-03T03:09:11.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.507+0000 7f7369574700 1 -- 172.21.2.111:0/370619507 wait complete. 2021-04-03T03:09:11.519 INFO:teuthology.orchestra.run.gibba011.stderr:updated 2021-04-03T03:09:11.523 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:11.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 Processor -- start 2021-04-03T03:09:11.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- start start 2021-04-03T03:09:11.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5bf40b12b0 0x7f5bf41522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf40607d0 con 0x7f5bf40603f0 2021-04-03T03:09:11.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf40b0eb0 con 0x7f5bf414b700 2021-04-03T03:09:11.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf40af400 con 0x7f5bf40b12b0 2021-04-03T03:09:11.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf2b85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5bf40b12b0 0x7f5bf41522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf2384700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5bf40603f0 0x7f5bf40b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51522/0 (socket says 172.21.2.111:51522) 2021-04-03T03:09:11.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf2384700 1 -- 172.21.2.111:0/1728315682 learned_addr learned my addr 172.21.2.111:0/1728315682 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:11.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf1b83700 1 -- 172.21.2.111:0/1728315682 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878657986 0 0) 0x7f5bf40607d0 con 0x7f5bf40603f0 2021-04-03T03:09:11.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf1b83700 1 -- 172.21.2.111:0/1728315682 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bd4003240 con 0x7f5bf40603f0 2021-04-03T03:09:11.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf2b85700 1 -- 172.21.2.111:0/1728315682 >> v1:172.21.2.115:6789/0 conn(0x7f5bf414b700 legacy=0x7f5bf40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:11.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf2b85700 1 -- 172.21.2.111:0/1728315682 >> v1:172.21.2.111:6789/0 conn(0x7f5bf40603f0 legacy=0x7f5bf40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf2b85700 1 -- 172.21.2.111:0/1728315682 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf40af6a0 con 0x7f5bf40b12b0 2021-04-03T03:09:11.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf2b85700 1 --2- 172.21.2.111:0/1728315682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf40b12b0 0x7f5bf41522d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f5bf41486c0 tx=0x7f5bec0091d0).ready entity=mon.0 client_cookie=db9100e5ab636632 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf1b83700 1 -- 172.21.2.111:0/1728315682 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bec00b990 con 0x7f5bf40b12b0 2021-04-03T03:09:11.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf1b83700 1 -- 172.21.2.111:0/1728315682 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec00baf0 con 0x7f5bf40b12b0 2021-04-03T03:09:11.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf1b83700 1 -- 172.21.2.111:0/1728315682 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bec00bc60 con 0x7f5bf40b12b0 2021-04-03T03:09:11.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- 172.21.2.111:0/1728315682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf40b12b0 msgr2=0x7f5bf41522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 --2- 172.21.2.111:0/1728315682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf40b12b0 0x7f5bf41522d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f5bf41486c0 tx=0x7f5bec0091d0).stop 2021-04-03T03:09:11.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- 172.21.2.111:0/1728315682 shutdown_connections 2021-04-03T03:09:11.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 --2- 172.21.2.111:0/1728315682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf40b12b0 0x7f5bf41522d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- 172.21.2.111:0/1728315682 >> 172.21.2.111:0/1728315682 conn(0x7f5bf40b61c0 msgr2=0x7f5bf405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- 172.21.2.111:0/1728315682 shutdown_connections 2021-04-03T03:09:11.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- 172.21.2.111:0/1728315682 wait complete. 2021-04-03T03:09:11.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 Processor -- start 2021-04-03T03:09:11.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.571+0000 7f5bf3386700 1 -- start start 2021-04-03T03:09:11.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf40603f0 0x7f5bf40ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf40b12b0 0x7f5bf4156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf414b700 0x7f5bf4156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40a9860 con 0x7f5bf414b700 2021-04-03T03:09:11.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf4064570 con 0x7f5bf40603f0 2021-04-03T03:09:11.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf4064810 con 0x7f5bf40b12b0 2021-04-03T03:09:11.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf414b700 0x7f5bf4156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf414b700 0x7f5bf4156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50736/0 (socket says 172.21.2.111:50736) 2021-04-03T03:09:11.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 -- 172.21.2.111:0/898203840 learned_addr learned my addr 172.21.2.111:0/898203840 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bebfff700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf40b12b0 0x7f5bf4156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 -- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf40b12b0 msgr2=0x7f5bf4156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2384700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf40603f0 0x7f5bf40ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf40b12b0 0x7f5bf4156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 -- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf40603f0 msgr2=0x7f5bf40ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf40603f0 0x7f5bf40ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 -- 172.21.2.111:0/898203840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf40a9aa0 con 0x7f5bf414b700 2021-04-03T03:09:11.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2384700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf40603f0 0x7f5bf40ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:11.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2b85700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf414b700 0x7f5bf4156a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f5bf41486c0 tx=0x7f5bec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5beb7fe700 1 -- 172.21.2.111:0/898203840 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bec00be90 con 0x7f5bf414b700 2021-04-03T03:09:11.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5beb7fe700 1 -- 172.21.2.111:0/898203840 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec01f3f0 con 0x7f5bf414b700 2021-04-03T03:09:11.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf40a9f40 con 0x7f5bf414b700 2021-04-03T03:09:11.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5beb7fe700 1 -- 172.21.2.111:0/898203840 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bec01f750 con 0x7f5bf414b700 2021-04-03T03:09:11.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf40abb70 con 0x7f5bf414b700 2021-04-03T03:09:11.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf40aad40 con 0x7f5bf414b700 2021-04-03T03:09:11.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5beb7fe700 1 -- 172.21.2.111:0/898203840 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5bec00f070 con 0x7f5bf414b700 2021-04-03T03:09:11.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5beb7fe700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056f20 0x7f5bd80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5beb7fe700 1 -- 172.21.2.111:0/898203840 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f5bec070b00 con 0x7f5bf414b700 2021-04-03T03:09:11.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2384700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056f20 0x7f5bd80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.575+0000 7f5bf2384700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056f20 0x7f5bd80593e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5be4008000 tx=0x7f5be4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.579+0000 7f5beb7fe700 1 -- 172.21.2.111:0/898203840 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5bec046110 con 0x7f5bf414b700 2021-04-03T03:09:11.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bf4057660 con 0x7f5bf414b700 2021-04-03T03:09:11.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5beb7fe700 1 -- 172.21.2.111:0/898203840 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f5bec024090 con 0x7f5bf414b700 2021-04-03T03:09:11.679 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:11.679 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:09:09.951299+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:11.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056f20 msgr2=0x7f5bd80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056f20 0x7f5bd80593e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5be4008000 tx=0x7f5be4009040).stop 2021-04-03T03:09:11.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf414b700 msgr2=0x7f5bf4156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf414b700 0x7f5bf4156a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f5bf41486c0 tx=0x7f5bec016000).stop 2021-04-03T03:09:11.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 shutdown_connections 2021-04-03T03:09:11.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bf414b700 0x7f5bf4156a10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bd8056f20 0x7f5bd80593e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bf40603f0 0x7f5bf40ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 --2- 172.21.2.111:0/898203840 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bf40b12b0 0x7f5bf4156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 >> 172.21.2.111:0/898203840 conn(0x7f5bf40b61c0 msgr2=0x7f5bf40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 shutdown_connections 2021-04-03T03:09:11.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.671+0000 7f5bf3386700 1 -- 172.21.2.111:0/898203840 wait complete. 2021-04-03T03:09:11.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:09:11.688 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:09:11.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.739+0000 7fd773fff700 1 Processor -- start 2021-04-03T03:09:11.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.739+0000 7fd773fff700 1 -- start start 2021-04-03T03:09:11.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd7740603f0 0x7fd7740b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740b44a0 0x7fd7740b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740607d0 con 0x7fd7740b5620 2021-04-03T03:09:11.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7741433c0 con 0x7fd7740603f0 2021-04-03T03:09:11.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740b3f80 con 0x7fd7740b44a0 2021-04-03T03:09:11.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd7740603f0 0x7fd7740b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd7740603f0 0x7fd7740b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50744/0 (socket says 172.21.2.111:50744) 2021-04-03T03:09:11.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 -- 172.21.2.111:0/1741107964 learned_addr learned my addr 172.21.2.111:0/1741107964 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 -- 172.21.2.111:0/1741107964 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740b44a0 msgr2=0x7fd7740b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:11.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 --2- 172.21.2.111:0/1741107964 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740b44a0 0x7fd7740b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 -- 172.21.2.111:0/1741107964 >> v1:172.21.2.111:6789/0 conn(0x7fd7740b5620 legacy=0x7fd7741469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:11.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 -- 172.21.2.111:0/1741107964 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740b4c60 con 0x7fd7740603f0 2021-04-03T03:09:11.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd772ffd700 1 --2- 172.21.2.111:0/1741107964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740603f0 0x7fd7740b2480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd7740b5500 tx=0x7fd764006e90).ready entity=mon.0 client_cookie=c09e39f2a5672daa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd771ffb700 1 -- 172.21.2.111:0/1741107964 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7640098e0 con 0x7fd7740603f0 2021-04-03T03:09:11.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd771ffb700 1 -- 172.21.2.111:0/1741107964 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764009a40 con 0x7fd7740603f0 2021-04-03T03:09:11.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd771ffb700 1 -- 172.21.2.111:0/1741107964 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd764009de0 con 0x7fd7740603f0 2021-04-03T03:09:11.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- 172.21.2.111:0/1741107964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740603f0 msgr2=0x7fd7740b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- 172.21.2.111:0/1741107964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740603f0 0x7fd7740b2480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd7740b5500 tx=0x7fd764006e90).stop 2021-04-03T03:09:11.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- 172.21.2.111:0/1741107964 shutdown_connections 2021-04-03T03:09:11.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- 172.21.2.111:0/1741107964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740603f0 0x7fd7740b2480 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- 172.21.2.111:0/1741107964 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740b44a0 0x7fd7740b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- 172.21.2.111:0/1741107964 >> 172.21.2.111:0/1741107964 conn(0x7fd77405c7d0 msgr2=0x7fd77405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- 172.21.2.111:0/1741107964 shutdown_connections 2021-04-03T03:09:11.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- 172.21.2.111:0/1741107964 wait complete. 2021-04-03T03:09:11.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 Processor -- start 2021-04-03T03:09:11.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- start start 2021-04-03T03:09:11.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740603f0 0x7fd774147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740b44a0 0x7fd7741523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7740b5620 0x7fd7741559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774145e70 con 0x7fd7740b44a0 2021-04-03T03:09:11.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774147ef0 con 0x7fd7740b5620 2021-04-03T03:09:11.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd774145bd0 con 0x7fd7740603f0 2021-04-03T03:09:11.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740b44a0 0x7fd7741523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740b44a0 0x7fd7741523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50752/0 (socket says 172.21.2.111:50752) 2021-04-03T03:09:11.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 -- 172.21.2.111:0/3346864200 learned_addr learned my addr 172.21.2.111:0/3346864200 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 -- 172.21.2.111:0/3346864200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740603f0 msgr2=0x7fd774147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740603f0 0x7fd774147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 -- 172.21.2.111:0/3346864200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7740b5620 msgr2=0x7fd7741559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7740b5620 0x7fd7741559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 -- 172.21.2.111:0/3346864200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774147c00 con 0x7fd7740b44a0 2021-04-03T03:09:11.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd7727fc700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740b44a0 0x7fd7741523c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fd768002f30 tx=0x7fd768006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd763fff700 1 -- 172.21.2.111:0/3346864200 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd768014070 con 0x7fd7740b44a0 2021-04-03T03:09:11.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd763fff700 1 -- 172.21.2.111:0/3346864200 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76800f780 con 0x7fd7740b44a0 2021-04-03T03:09:11.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7740aecf0 con 0x7fd7740b44a0 2021-04-03T03:09:11.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7740afc40 con 0x7fd7740b44a0 2021-04-03T03:09:11.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.743+0000 7fd763fff700 1 -- 172.21.2.111:0/3346864200 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd76800fae0 con 0x7fd7740b44a0 2021-04-03T03:09:11.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.747+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774148060 con 0x7fd7740b44a0 2021-04-03T03:09:11.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.747+0000 7fd763fff700 1 -- 172.21.2.111:0/3346864200 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd76800fc40 con 0x7fd7740b44a0 2021-04-03T03:09:11.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.747+0000 7fd763fff700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750057000 0x7fd7500594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.747+0000 7fd763fff700 1 -- 172.21.2.111:0/3346864200 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fd76806fd50 con 0x7fd7740b44a0 2021-04-03T03:09:11.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.747+0000 7fd772ffd700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750057000 0x7fd7500594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.747+0000 7fd772ffd700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750057000 0x7fd7500594c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd7740b5500 tx=0x7fd764015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.747+0000 7fd763fff700 1 -- 172.21.2.111:0/3346864200 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd768045360 con 0x7fd7740b44a0 2021-04-03T03:09:11.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.839+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd774057660 con 0x7fd7740b44a0 2021-04-03T03:09:11.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.839+0000 7fd763fff700 1 -- 172.21.2.111:0/3346864200 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v29) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd768048980 con 0x7fd7740b44a0 2021-04-03T03:09:11.849 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:09:11.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750057000 msgr2=0x7fd7500594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750057000 0x7fd7500594c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd7740b5500 tx=0x7fd764015000).stop 2021-04-03T03:09:11.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740b44a0 msgr2=0x7fd7741523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740b44a0 0x7fd7741523c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fd768002f30 tx=0x7fd768006e80).stop 2021-04-03T03:09:11.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 shutdown_connections 2021-04-03T03:09:11.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd750057000 0x7fd7500594c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd7740b5620 0x7fd7741559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd7740603f0 0x7fd774147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 --2- 172.21.2.111:0/3346864200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd7740b44a0 0x7fd7741523c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 >> 172.21.2.111:0/3346864200 conn(0x7fd77405c7d0 msgr2=0x7fd774143510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 shutdown_connections 2021-04-03T03:09:11.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.843+0000 7fd773fff700 1 -- 172.21.2.111:0/3346864200 wait complete. 2021-04-03T03:09:11.857 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:09:11.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 Processor -- start 2021-04-03T03:09:11.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- start start 2021-04-03T03:09:11.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0cf40b8350 0x7f0cf4141d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0cf4129ed0 0x7f0cf412a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40ad880 con 0x7f0cf40b24d0 2021-04-03T03:09:11.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf40b8730 con 0x7f0cf4129ed0 2021-04-03T03:09:11.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf4141350 con 0x7f0cf40b8350 2021-04-03T03:09:11.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0cf4129ed0 0x7f0cf412a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0cf4129ed0 0x7f0cf412a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50758/0 (socket says 172.21.2.111:50758) 2021-04-03T03:09:11.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 -- 172.21.2.111:0/3023253530 learned_addr learned my addr 172.21.2.111:0/3023253530 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 -- 172.21.2.111:0/3023253530 >> v2:172.21.2.115:3300/0 conn(0x7f0cf40b8350 msgr2=0x7f0cf4141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:11.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 --2- 172.21.2.111:0/3023253530 >> v2:172.21.2.115:3300/0 conn(0x7f0cf40b8350 0x7f0cf4141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 -- 172.21.2.111:0/3023253530 >> v1:172.21.2.115:6789/0 conn(0x7f0cf40b24d0 legacy=0x7f0cf40ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:11.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 -- 172.21.2.111:0/3023253530 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf41415f0 con 0x7f0cf4129ed0 2021-04-03T03:09:11.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 --2- 172.21.2.111:0/3023253530 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf412a320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0cf4064060 tx=0x7f0cec009270).ready entity=mon.0 client_cookie=9f93d1edc529e807 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cf8947700 1 -- 172.21.2.111:0/3023253530 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cec00bbc0 con 0x7f0cf4129ed0 2021-04-03T03:09:11.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cf8947700 1 -- 172.21.2.111:0/3023253530 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec00bd20 con 0x7f0cf4129ed0 2021-04-03T03:09:11.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cf8947700 1 -- 172.21.2.111:0/3023253530 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cec00be90 con 0x7f0cf4129ed0 2021-04-03T03:09:11.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- 172.21.2.111:0/3023253530 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 msgr2=0x7f0cf412a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/3023253530 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf412a320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0cf4064060 tx=0x7f0cec009270).stop 2021-04-03T03:09:11.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- 172.21.2.111:0/3023253530 shutdown_connections 2021-04-03T03:09:11.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/3023253530 >> v2:172.21.2.115:3300/0 conn(0x7f0cf40b8350 0x7f0cf4141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:11.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/3023253530 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf412a320 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- 172.21.2.111:0/3023253530 >> 172.21.2.111:0/3023253530 conn(0x7f0cf4060490 msgr2=0x7f0cf40b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:11.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- 172.21.2.111:0/3023253530 shutdown_connections 2021-04-03T03:09:11.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- 172.21.2.111:0/3023253530 wait complete. 2021-04-03T03:09:11.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 Processor -- start 2021-04-03T03:09:11.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- start start 2021-04-03T03:09:11.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cf40b24d0 0x7f0cf4125f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cf40b8350 0x7f0cf40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf4146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4144140 con 0x7f0cf4129ed0 2021-04-03T03:09:11.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4142840 con 0x7f0cf40b8350 2021-04-03T03:09:11.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa94b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4142ac0 con 0x7f0cf40b24d0 2021-04-03T03:09:11.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf4146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf4146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50766/0 (socket says 172.21.2.111:50766) 2021-04-03T03:09:11.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cfa14a700 1 -- 172.21.2.111:0/4247905714 learned_addr learned my addr 172.21.2.111:0/4247905714 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:11.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cf9148700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cf40b8350 0x7f0cf40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.907+0000 7f0cf9949700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cf40b24d0 0x7f0cf4125f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa14a700 1 -- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cf40b24d0 msgr2=0x7f0cf4125f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa14a700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cf40b24d0 0x7f0cf4125f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa14a700 1 -- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cf40b8350 msgr2=0x7f0cf40b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:11.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa14a700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cf40b8350 0x7f0cf40b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:11.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa14a700 1 -- 172.21.2.111:0/4247905714 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4142d60 con 0x7f0cf4129ed0 2021-04-03T03:09:11.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cf9148700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cf40b8350 0x7f0cf40b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:09:11.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa14a700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf4146120 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f0cf4064060 tx=0x7f0cec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0ce6ffd700 1 -- 172.21.2.111:0/4247905714 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cec020620 con 0x7f0cf4129ed0 2021-04-03T03:09:11.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0ce6ffd700 1 -- 172.21.2.111:0/4247905714 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec020780 con 0x7f0cf4129ed0 2021-04-03T03:09:11.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf4146610 con 0x7f0cf4129ed0 2021-04-03T03:09:11.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0ce6ffd700 1 -- 172.21.2.111:0/4247905714 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cec020ae0 con 0x7f0cf4129ed0 2021-04-03T03:09:11.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf4143f60 con 0x7f0cf4129ed0 2021-04-03T03:09:11.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf41442b0 con 0x7f0cf4129ed0 2021-04-03T03:09:11.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0ce6ffd700 1 -- 172.21.2.111:0/4247905714 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0cec026020 con 0x7f0cf4129ed0 2021-04-03T03:09:11.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0ce6ffd700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cd40570a0 0x7f0cd4059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:11.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0ce6ffd700 1 -- 172.21.2.111:0/4247905714 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f0cec071360 con 0x7f0cf4129ed0 2021-04-03T03:09:11.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cf9949700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cd40570a0 0x7f0cd4059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:11.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.911+0000 7f0cf9949700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cd40570a0 0x7f0cd4059560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0ce800a000 tx=0x7f0ce8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:11.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:11.915+0000 7f0ce6ffd700 1 -- 172.21.2.111:0/4247905714 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0cf41442b0 con 0x7f0cf4129ed0 2021-04-03T03:09:12.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0cf4144280 con 0x7f0cf4129ed0 2021-04-03T03:09:12.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0ce6ffd700 1 -- 172.21.2.111:0/4247905714 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0cec0150a0 con 0x7f0cf4129ed0 2021-04-03T03:09:12.016 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:12.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cd40570a0 msgr2=0x7f0cd4059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cd40570a0 0x7f0cd4059560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0ce800a000 tx=0x7f0ce8008040).stop 2021-04-03T03:09:12.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 msgr2=0x7f0cf4146120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf4146120 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f0cf4064060 tx=0x7f0cec016000).stop 2021-04-03T03:09:12.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 shutdown_connections 2021-04-03T03:09:12.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cf4129ed0 0x7f0cf4146120 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cf40b24d0 0x7f0cf4125f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cd40570a0 0x7f0cd4059560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 --2- 172.21.2.111:0/4247905714 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cf40b8350 0x7f0cf40b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 >> 172.21.2.111:0/4247905714 conn(0x7f0cf4060490 msgr2=0x7f0cf40b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:12.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 shutdown_connections 2021-04-03T03:09:12.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.007+0000 7f0cfa94b700 1 -- 172.21.2.111:0/4247905714 wait complete. 2021-04-03T03:09:12.024 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:09:12.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5b59e700 1 Processor -- start 2021-04-03T03:09:12.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5b59e700 1 -- start start 2021-04-03T03:09:12.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fae5c0b0610 0x7fae5c144150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13f680 0x7fae5c13fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5b59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0b1460 con 0x7fae5c13ffd0 2021-04-03T03:09:12.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c144640 con 0x7fae5c0b0610 2021-04-03T03:09:12.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0b6ab0 con 0x7fae5c13f680 2021-04-03T03:09:12.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae59d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13f680 0x7fae5c13fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5a59c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fae5c0b0610 0x7fae5c144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5ad9d700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fae5c13ffd0 0x7fae5c0af280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:35868/0 (socket says 172.21.2.111:35868) 2021-04-03T03:09:12.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5a59c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fae5c0b0610 0x7fae5c144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:40478/0 (socket says 172.21.2.111:40478) 2021-04-03T03:09:12.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae59d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13f680 0x7fae5c13fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:34122/0 (socket says 172.21.2.111:34122) 2021-04-03T03:09:12.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.071+0000 7fae5ad9d700 1 -- 172.21.2.111:0/2538516819 learned_addr learned my addr 172.21.2.111:0/2538516819 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:12.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5a59c700 1 -- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13f680 msgr2=0x7fae5c13fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:12.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5a59c700 1 --2- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13f680 0x7fae5c13fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5a59c700 1 -- 172.21.2.111:0/2538516819 >> v1:172.21.2.115:6789/0 conn(0x7fae5c13ffd0 legacy=0x7fae5c0af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5a59c700 1 -- 172.21.2.111:0/2538516819 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0b6d50 con 0x7fae5c0b0610 2021-04-03T03:09:12.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 --2- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13f680 0x7fae5c13fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:09:12.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5a59c700 1 --2- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c144150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fae4c002f30 tx=0x7fae4c006dd0).ready entity=mon.1 client_cookie=a4efd28a66123043 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5959a700 1 -- 172.21.2.111:0/2538516819 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae4c009760 con 0x7fae5c0b0610 2021-04-03T03:09:12.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5959a700 1 -- 172.21.2.111:0/2538516819 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c0098c0 con 0x7fae5c0b0610 2021-04-03T03:09:12.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5959a700 1 -- 172.21.2.111:0/2538516819 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae4c009c20 con 0x7fae5c0b0610 2021-04-03T03:09:12.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 msgr2=0x7fae5c144150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 --2- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c144150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fae4c002f30 tx=0x7fae4c006dd0).stop 2021-04-03T03:09:12.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- 172.21.2.111:0/2538516819 shutdown_connections 2021-04-03T03:09:12.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 --2- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c144150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 --2- 172.21.2.111:0/2538516819 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13f680 0x7fae5c13fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- 172.21.2.111:0/2538516819 >> 172.21.2.111:0/2538516819 conn(0x7fae5c063d70 msgr2=0x7fae5c064180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:12.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- 172.21.2.111:0/2538516819 shutdown_connections 2021-04-03T03:09:12.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- 172.21.2.111:0/2538516819 wait complete. 2021-04-03T03:09:12.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 Processor -- start 2021-04-03T03:09:12.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- start start 2021-04-03T03:09:12.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae5c13f680 0x7fae5c0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13ffd0 0x7fae5c0aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c067c20 con 0x7fae5c13f680 2021-04-03T03:09:12.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c067980 con 0x7fae5c0b0610 2021-04-03T03:09:12.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c065ab0 con 0x7fae5c13ffd0 2021-04-03T03:09:12.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae5c13f680 0x7fae5c0aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae5c13f680 0x7fae5c0aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50778/0 (socket says 172.21.2.111:50778) 2021-04-03T03:09:12.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 -- 172.21.2.111:0/3497753919 learned_addr learned my addr 172.21.2.111:0/3497753919 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:12.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5ad9d700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13ffd0 0x7fae5c0aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5a59c700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 -- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13ffd0 msgr2=0x7fae5c0aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13ffd0 0x7fae5c0aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 -- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 msgr2=0x7fae5c141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c141b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 -- 172.21.2.111:0/3497753919 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0acf00 con 0x7fae5c13f680 2021-04-03T03:09:12.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5a59c700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:12.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae59d9b700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae5c13f680 0x7fae5c0aca10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fae4400c000 tx=0x7fae44006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae537fe700 1 -- 172.21.2.111:0/3497753919 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae4400cda0 con 0x7fae5c13f680 2021-04-03T03:09:12.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae537fe700 1 -- 172.21.2.111:0/3497753919 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae440123f0 con 0x7fae5c13f680 2021-04-03T03:09:12.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c0ad2f0 con 0x7fae5c13f680 2021-04-03T03:09:12.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae537fe700 1 -- 172.21.2.111:0/3497753919 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae44012750 con 0x7fae5c13f680 2021-04-03T03:09:12.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.075+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c065730 con 0x7fae5c13f680 2021-04-03T03:09:12.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.079+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c0677c0 con 0x7fae5c13f680 2021-04-03T03:09:12.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.079+0000 7fae537fe700 1 -- 172.21.2.111:0/3497753919 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fae440128b0 con 0x7fae5c13f680 2021-04-03T03:09:12.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.079+0000 7fae537fe700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae40056f20 0x7fae400593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.079+0000 7fae537fe700 1 -- 172.21.2.111:0/3497753919 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fae44070600 con 0x7fae5c13f680 2021-04-03T03:09:12.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.079+0000 7fae5a59c700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae40056f20 0x7fae400593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.079+0000 7fae5a59c700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae40056f20 0x7fae400593e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fae4c016000 tx=0x7fae4c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.079+0000 7fae537fe700 1 -- 172.21.2.111:0/3497753919 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fae44045d20 con 0x7fae5c13f680 2021-04-03T03:09:12.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fae5c065c70 con 0x7fae5c13f680 2021-04-03T03:09:12.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae537fe700 1 -- 172.21.2.111:0/3497753919 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+143 (secure 0 0 0) 0x7fae44049340 con 0x7fae5c13f680 2021-04-03T03:09:12.183 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:12.183 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:09:12.183 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:12.183 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:09:12.184 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:09:12.184 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:09:12.184 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:09:12.184 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:09:12.184 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:09:12.185 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:09:12.185 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:09:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae40056f20 msgr2=0x7fae400593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae40056f20 0x7fae400593e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fae4c016000 tx=0x7fae4c015000).stop 2021-04-03T03:09:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae5c13f680 msgr2=0x7fae5c0aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae5c13f680 0x7fae5c0aca10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fae4400c000 tx=0x7fae44006dd0).stop 2021-04-03T03:09:12.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 shutdown_connections 2021-04-03T03:09:12.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae5c13ffd0 0x7fae5c0aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae40056f20 0x7fae400593e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae5c0b0610 0x7fae5c141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 --2- 172.21.2.111:0/3497753919 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae5c13f680 0x7fae5c0aca10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 >> 172.21.2.111:0/3497753919 conn(0x7fae5c063d70 msgr2=0x7fae5c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 shutdown_connections 2021-04-03T03:09:12.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.175+0000 7fae5b59e700 1 -- 172.21.2.111:0/3497753919 wait complete. 2021-04-03T03:09:12.190 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:09:12.190 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:09:12.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 Processor -- start 2021-04-03T03:09:12.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 -- start start 2021-04-03T03:09:12.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f31e40603f0 0x7f31e41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f31e40b17b0 0x7f31e4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e40607d0 con 0x7f31e40add50 2021-04-03T03:09:12.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e41451c0 con 0x7f31e40b17b0 2021-04-03T03:09:12.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e414b090 con 0x7f31e40603f0 2021-04-03T03:09:12.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e37fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f31e40add50 0x7f31e4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51576/0 (socket says 172.21.2.111:51576) 2021-04-03T03:09:12.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e8eab700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f31e40b17b0 0x7f31e4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e37fe700 1 -- 172.21.2.111:0/4061025099 learned_addr learned my addr 172.21.2.111:0/4061025099 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:12.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e2ffd700 1 -- 172.21.2.111:0/4061025099 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064587175 0 0) 0x7f31e40607d0 con 0x7f31e40add50 2021-04-03T03:09:12.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e2ffd700 1 -- 172.21.2.111:0/4061025099 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31c4003240 con 0x7f31e40add50 2021-04-03T03:09:12.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e8eab700 1 -- 172.21.2.111:0/4061025099 >> v2:172.21.2.115:3300/0 conn(0x7f31e40603f0 msgr2=0x7f31e41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:12.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e8eab700 1 --2- 172.21.2.111:0/4061025099 >> v2:172.21.2.115:3300/0 conn(0x7f31e40603f0 0x7f31e41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e8eab700 1 -- 172.21.2.111:0/4061025099 >> v1:172.21.2.111:6789/0 conn(0x7f31e40add50 legacy=0x7f31e4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e8eab700 1 -- 172.21.2.111:0/4061025099 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e414b330 con 0x7f31e40b17b0 2021-04-03T03:09:12.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e8eab700 1 --2- 172.21.2.111:0/4061025099 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40b17b0 0x7f31e4144cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f31dc0032b0 tx=0x7f31dc0091d0).ready entity=mon.0 client_cookie=c71e1565f3a8be38 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e2ffd700 1 -- 172.21.2.111:0/4061025099 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31dc00b9a0 con 0x7f31e40b17b0 2021-04-03T03:09:12.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e2ffd700 1 -- 172.21.2.111:0/4061025099 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc00bb00 con 0x7f31e40b17b0 2021-04-03T03:09:12.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e2ffd700 1 -- 172.21.2.111:0/4061025099 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31dc00bc70 con 0x7f31e40b17b0 2021-04-03T03:09:12.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 -- 172.21.2.111:0/4061025099 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40b17b0 msgr2=0x7f31e4144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.239+0000 7f31e96ac700 1 --2- 172.21.2.111:0/4061025099 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40b17b0 0x7f31e4144cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f31dc0032b0 tx=0x7f31dc0091d0).stop 2021-04-03T03:09:12.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- 172.21.2.111:0/4061025099 shutdown_connections 2021-04-03T03:09:12.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 --2- 172.21.2.111:0/4061025099 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40b17b0 0x7f31e4144cd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 --2- 172.21.2.111:0/4061025099 >> v2:172.21.2.115:3300/0 conn(0x7f31e40603f0 0x7f31e41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- 172.21.2.111:0/4061025099 >> 172.21.2.111:0/4061025099 conn(0x7f31e40b61c0 msgr2=0x7f31e405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:12.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- 172.21.2.111:0/4061025099 shutdown_connections 2021-04-03T03:09:12.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- 172.21.2.111:0/4061025099 wait complete. 2021-04-03T03:09:12.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 Processor -- start 2021-04-03T03:09:12.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- start start 2021-04-03T03:09:12.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31e40603f0 0x7f31e40aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40add50 0x7f31e4065350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31e40b17b0 0x7f31e4062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40aaa20 con 0x7f31e40add50 2021-04-03T03:09:12.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e4064c10 con 0x7f31e40603f0 2021-04-03T03:09:12.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e4064eb0 con 0x7f31e40b17b0 2021-04-03T03:09:12.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40add50 0x7f31e4065350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40add50 0x7f31e4065350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50792/0 (socket says 172.21.2.111:50792) 2021-04-03T03:09:12.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 -- 172.21.2.111:0/2898223360 learned_addr learned my addr 172.21.2.111:0/2898223360 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:12.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 -- 172.21.2.111:0/2898223360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31e40b17b0 msgr2=0x7f31e4062c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31e40b17b0 0x7f31e4062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 -- 172.21.2.111:0/2898223360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31e40603f0 msgr2=0x7f31e40aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31e40603f0 0x7f31e40aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 -- 172.21.2.111:0/2898223360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4065840 con 0x7f31e40add50 2021-04-03T03:09:12.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e37fe700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40add50 0x7f31e4065350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f31d40010d0 tx=0x7f31d4007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e17fa700 1 -- 172.21.2.111:0/2898223360 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d40072e0 con 0x7f31e40add50 2021-04-03T03:09:12.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e17fa700 1 -- 172.21.2.111:0/2898223360 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d4007440 con 0x7f31e40add50 2021-04-03T03:09:12.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e17fa700 1 -- 172.21.2.111:0/2898223360 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d40057e0 con 0x7f31e40add50 2021-04-03T03:09:12.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e40630f0 con 0x7f31e40add50 2021-04-03T03:09:12.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e4156be0 con 0x7f31e40add50 2021-04-03T03:09:12.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e4064a60 con 0x7f31e40add50 2021-04-03T03:09:12.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e17fa700 1 -- 172.21.2.111:0/2898223360 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f31d4010070 con 0x7f31e40add50 2021-04-03T03:09:12.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e17fa700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c8057000 0x7f31c80594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e17fa700 1 -- 172.21.2.111:0/2898223360 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f31d406f6e0 con 0x7f31e40add50 2021-04-03T03:09:12.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e3fff700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c8057000 0x7f31c80594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.243+0000 7f31e3fff700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c8057000 0x7f31c80594c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f31cc002f30 tx=0x7f31cc006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.247+0000 7f31e17fa700 1 -- 172.21.2.111:0/2898223360 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f31d4044cf0 con 0x7f31e40add50 2021-04-03T03:09:12.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.339+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f31e4065070 con 0x7f31e40add50 2021-04-03T03:09:12.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.863+0000 7f31e17fa700 1 -- 172.21.2.111:0/2898223360 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f31d4048310 con 0x7f31e40add50 2021-04-03T03:09:12.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c8057000 msgr2=0x7f31c80594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c8057000 0x7f31c80594c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f31cc002f30 tx=0x7f31cc006a60).stop 2021-04-03T03:09:12.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40add50 msgr2=0x7f31e4065350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40add50 0x7f31e4065350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f31d40010d0 tx=0x7f31d4007a60).stop 2021-04-03T03:09:12.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 shutdown_connections 2021-04-03T03:09:12.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31e40b17b0 0x7f31e4062c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c8057000 0x7f31c80594c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31e40603f0 0x7f31e40aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 --2- 172.21.2.111:0/2898223360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31e40add50 0x7f31e4065350 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 >> 172.21.2.111:0/2898223360 conn(0x7f31e40b61c0 msgr2=0x7f31e405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:12.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 shutdown_connections 2021-04-03T03:09:12.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.867+0000 7f31e96ac700 1 -- 172.21.2.111:0/2898223360 wait complete. 2021-04-03T03:09:12.878 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:09:12.882 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:09:12.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 Processor -- start 2021-04-03T03:09:12.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- start start 2021-04-03T03:09:12.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f79700603f0 0x7f79700b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79700b5620 0x7f7970142310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79700607d0 con 0x7f79700b44a0 2021-04-03T03:09:12.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79701433c0 con 0x7f79700b5620 2021-04-03T03:09:12.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79700b3f80 con 0x7f79700603f0 2021-04-03T03:09:12.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79700b5620 0x7f7970142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79700b5620 0x7f7970142310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50798/0 (socket says 172.21.2.111:50798) 2021-04-03T03:09:12.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 -- 172.21.2.111:0/1832738615 learned_addr learned my addr 172.21.2.111:0/1832738615 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:12.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975690700 1 --2- 172.21.2.111:0/1832738615 >> v2:172.21.2.115:3300/0 conn(0x7f79700603f0 0x7f79700b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 -- 172.21.2.111:0/1832738615 >> v2:172.21.2.115:3300/0 conn(0x7f79700603f0 msgr2=0x7f79700b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:12.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 --2- 172.21.2.111:0/1832738615 >> v2:172.21.2.115:3300/0 conn(0x7f79700603f0 0x7f79700b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 -- 172.21.2.111:0/1832738615 >> v1:172.21.2.115:6789/0 conn(0x7f79700b44a0 legacy=0x7f79700b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:12.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 -- 172.21.2.111:0/1832738615 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700b4c60 con 0x7f79700b5620 2021-04-03T03:09:12.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7975e91700 1 --2- 172.21.2.111:0/1832738615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f7970142310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f79700b2320 tx=0x7f7968009270).ready entity=mon.0 client_cookie=438b431bc999c454 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7967fff700 1 -- 172.21.2.111:0/1832738615 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796800bb80 con 0x7f79700b5620 2021-04-03T03:09:12.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7967fff700 1 -- 172.21.2.111:0/1832738615 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800bce0 con 0x7f79700b5620 2021-04-03T03:09:12.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7967fff700 1 -- 172.21.2.111:0/1832738615 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796800be50 con 0x7f79700b5620 2021-04-03T03:09:12.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- 172.21.2.111:0/1832738615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 msgr2=0x7f7970142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 --2- 172.21.2.111:0/1832738615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f7970142310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f79700b2320 tx=0x7f7968009270).stop 2021-04-03T03:09:12.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- 172.21.2.111:0/1832738615 shutdown_connections 2021-04-03T03:09:12.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 --2- 172.21.2.111:0/1832738615 >> v2:172.21.2.115:3300/0 conn(0x7f79700603f0 0x7f79700b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 --2- 172.21.2.111:0/1832738615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f7970142310 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:12.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- 172.21.2.111:0/1832738615 >> 172.21.2.111:0/1832738615 conn(0x7f797005c7d0 msgr2=0x7f797005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:12.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- 172.21.2.111:0/1832738615 shutdown_connections 2021-04-03T03:09:12.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.931+0000 7f7976692700 1 -- 172.21.2.111:0/1832738615 wait complete. 2021-04-03T03:09:12.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 Processor -- start 2021-04-03T03:09:12.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 -- start start 2021-04-03T03:09:12.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79700603f0 0x7f7970113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79700b44a0 0x7f79700ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f79701406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970147140 con 0x7f79700b5620 2021-04-03T03:09:12.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701473e0 con 0x7f79700b44a0 2021-04-03T03:09:12.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79700adc10 con 0x7f79700603f0 2021-04-03T03:09:12.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f79701406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f79701406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50806/0 (socket says 172.21.2.111:50806) 2021-04-03T03:09:12.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 -- 172.21.2.111:0/2906434180 learned_addr learned my addr 172.21.2.111:0/2906434180 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:12.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 -- 172.21.2.111:0/2906434180 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79700603f0 msgr2=0x7f7970113820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79700603f0 0x7f7970113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 -- 172.21.2.111:0/2906434180 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79700b44a0 msgr2=0x7f79700ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:12.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79700b44a0 0x7f79700ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:12.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 -- 172.21.2.111:0/2906434180 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700ae0b0 con 0x7f79700b5620 2021-04-03T03:09:12.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7975e91700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f79701406e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f79700b2320 tx=0x7f7968016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f79667fc700 1 -- 172.21.2.111:0/2906434180 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79680032b0 con 0x7f79700b5620 2021-04-03T03:09:12.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f79667fc700 1 -- 172.21.2.111:0/2906434180 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796801f420 con 0x7f79700b5620 2021-04-03T03:09:12.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f79667fc700 1 -- 172.21.2.111:0/2906434180 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796801f780 con 0x7f79700b5620 2021-04-03T03:09:12.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79700ad170 con 0x7f79700b5620 2021-04-03T03:09:12.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7970140bd0 con 0x7f79700b5620 2021-04-03T03:09:12.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.935+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79700addd0 con 0x7f79700b5620 2021-04-03T03:09:12.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.939+0000 7f79667fc700 1 -- 172.21.2.111:0/2906434180 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7968026030 con 0x7f79700b5620 2021-04-03T03:09:12.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.939+0000 7f79667fc700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79580570a0 0x7f7958059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:12.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.939+0000 7f79667fc700 1 -- 172.21.2.111:0/2906434180 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5950+0+0 (secure 0 0 0) 0x7f7968071030 con 0x7f79700b5620 2021-04-03T03:09:12.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.939+0000 7f7975690700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79580570a0 0x7f7958059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:12.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.939+0000 7f7975690700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79580570a0 0x7f7958059560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f796c00a000 tx=0x7f796c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:12.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:12.939+0000 7f79667fc700 1 -- 172.21.2.111:0/2906434180 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f79680465f0 con 0x7f79700b5620 2021-04-03T03:09:13.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.035+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f7970057660 con 0x7f79700b5620 2021-04-03T03:09:13.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f79667fc700 1 -- 172.21.2.111:0/2906434180 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v31) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7968024070 con 0x7f79700b5620 2021-04-03T03:09:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79580570a0 msgr2=0x7f7958059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79580570a0 0x7f7958059560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f796c00a000 tx=0x7f796c008040).stop 2021-04-03T03:09:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 msgr2=0x7f79701406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f79701406e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f79700b2320 tx=0x7f7968016000).stop 2021-04-03T03:09:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 shutdown_connections 2021-04-03T03:09:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79700b5620 0x7f79701406e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79700603f0 0x7f7970113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.875+0000 7f7976692700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79580570a0 0x7f7958059560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.879+0000 7f7976692700 1 --2- 172.21.2.111:0/2906434180 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79700b44a0 0x7f79700ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.879+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 >> 172.21.2.111:0/2906434180 conn(0x7f797005c7d0 msgr2=0x7f79700b63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.879+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 shutdown_connections 2021-04-03T03:09:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.879+0000 7f7976692700 1 -- 172.21.2.111:0/2906434180 wait complete. 2021-04-03T03:09:13.889 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:09:13.894 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:09:13.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 Processor -- start 2021-04-03T03:09:13.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 -- start start 2021-04-03T03:09:13.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb3900dbba0 0x7fb3900dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:13.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb39003d560 0x7fb390041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:13.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900dc560 con 0x7fb390041060 2021-04-03T03:09:13.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39003de60 con 0x7fb39003d560 2021-04-03T03:09:13.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900dc800 con 0x7fb3900dbba0 2021-04-03T03:09:13.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb39003d560 0x7fb390041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:13.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb39003d560 0x7fb390041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50816/0 (socket says 172.21.2.111:50816) 2021-04-03T03:09:13.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 -- 172.21.2.111:0/540358442 learned_addr learned my addr 172.21.2.111:0/540358442 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:13.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39cbe7700 1 --2- 172.21.2.111:0/540358442 >> v2:172.21.2.115:3300/0 conn(0x7fb3900dbba0 0x7fb3900dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:13.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 -- 172.21.2.111:0/540358442 >> v2:172.21.2.115:3300/0 conn(0x7fb3900dbba0 msgr2=0x7fb3900dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:13.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 --2- 172.21.2.111:0/540358442 >> v2:172.21.2.115:3300/0 conn(0x7fb3900dbba0 0x7fb3900dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:13.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 -- 172.21.2.111:0/540358442 >> v1:172.21.2.111:6789/0 conn(0x7fb390041060 legacy=0x7fb39003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:13.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 -- 172.21.2.111:0/540358442 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39003da30 con 0x7fb39003d560 2021-04-03T03:09:13.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39d3e8700 1 --2- 172.21.2.111:0/540358442 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb39003d560 0x7fb390041650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fb39806b5d0 tx=0x7fb39805ad70).ready entity=mon.0 client_cookie=f2bf89d52ee9a1b4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:13.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb3977fe700 1 -- 172.21.2.111:0/540358442 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39805bda0 con 0x7fb39003d560 2021-04-03T03:09:13.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb3977fe700 1 -- 172.21.2.111:0/540358442 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39805df60 con 0x7fb39003d560 2021-04-03T03:09:13.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb3977fe700 1 -- 172.21.2.111:0/540358442 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3980699f0 con 0x7fb39003d560 2021-04-03T03:09:13.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 -- 172.21.2.111:0/540358442 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb39003d560 msgr2=0x7fb390041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:13.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.943+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/540358442 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb39003d560 0x7fb390041650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fb39806b5d0 tx=0x7fb39805ad70).stop 2021-04-03T03:09:13.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- 172.21.2.111:0/540358442 shutdown_connections 2021-04-03T03:09:13.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/540358442 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb39003d560 0x7fb390041650 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:13.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/540358442 >> v2:172.21.2.115:3300/0 conn(0x7fb3900dbba0 0x7fb3900dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:13.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- 172.21.2.111:0/540358442 >> 172.21.2.111:0/540358442 conn(0x7fb3900462b0 msgr2=0x7fb3900466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:13.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- 172.21.2.111:0/540358442 shutdown_connections 2021-04-03T03:09:13.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- 172.21.2.111:0/540358442 wait complete. 2021-04-03T03:09:13.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 Processor -- start 2021-04-03T03:09:13.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- start start 2021-04-03T03:09:13.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb39003d560 0x7fb3900dd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:13.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb390041060 0x7fb3900ea730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:13.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3900dbba0 0x7fb3900eac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:13.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900a0730 con 0x7fb390041060 2021-04-03T03:09:13.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390039580 con 0x7fb3900dbba0 2021-04-03T03:09:13.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900a0960 con 0x7fb39003d560 2021-04-03T03:09:13.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb390041060 0x7fb3900ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:13.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb390041060 0x7fb3900ea730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50822/0 (socket says 172.21.2.111:50822) 2021-04-03T03:09:13.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 -- 172.21.2.111:0/2340202417 learned_addr learned my addr 172.21.2.111:0/2340202417 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:13.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 -- 172.21.2.111:0/2340202417 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb39003d560 msgr2=0x7fb3900dd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:13.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb39003d560 0x7fb3900dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:13.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 -- 172.21.2.111:0/2340202417 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3900dbba0 msgr2=0x7fb3900eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:13.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3900dbba0 0x7fb3900eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:13.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 -- 172.21.2.111:0/2340202417 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390039b70 con 0x7fb390041060 2021-04-03T03:09:13.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb397fff700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb390041060 0x7fb3900ea730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fb3900087d0 tx=0x7fb38c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:13.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb395ffb700 1 -- 172.21.2.111:0/2340202417 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb38c00e4a0 con 0x7fb390041060 2021-04-03T03:09:13.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb395ffb700 1 -- 172.21.2.111:0/2340202417 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c00e600 con 0x7fb390041060 2021-04-03T03:09:13.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb395ffb700 1 -- 172.21.2.111:0/2340202417 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb38c00e960 con 0x7fb390041060 2021-04-03T03:09:13.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3900e7050 con 0x7fb390041060 2021-04-03T03:09:13.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb390039e20 con 0x7fb390041060 2021-04-03T03:09:13.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb390004db0 con 0x7fb390041060 2021-04-03T03:09:13.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb395ffb700 1 -- 172.21.2.111:0/2340202417 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb38c019030 con 0x7fb390041060 2021-04-03T03:09:13.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb395ffb700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3780570a0 0x7fb378059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:13.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb395ffb700 1 -- 172.21.2.111:0/2340202417 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6300+0+0 (secure 0 0 0) 0x7fb38c06f900 con 0x7fb390041060 2021-04-03T03:09:13.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.947+0000 7fb39cbe7700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3780570a0 0x7fb378059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:13.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.951+0000 7fb39cbe7700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3780570a0 0x7fb378059560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb38800a000 tx=0x7fb388008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:13.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:13.951+0000 7fb395ffb700 1 -- 172.21.2.111:0/2340202417 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb38c044c50 con 0x7fb390041060 2021-04-03T03:09:14.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.075+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fb3900083c0 con 0x7fb390041060 2021-04-03T03:09:14.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb395ffb700 1 -- 172.21.2.111:0/2340202417 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v10) v1 ==== 187+0+0 (secure 0 0 0) 0x7fb38c048270 con 0x7fb390041060 2021-04-03T03:09:14.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3780570a0 msgr2=0x7fb378059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3780570a0 0x7fb378059560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb38800a000 tx=0x7fb388008040).stop 2021-04-03T03:09:14.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb390041060 msgr2=0x7fb3900ea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb390041060 0x7fb3900ea730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fb3900087d0 tx=0x7fb38c007c30).stop 2021-04-03T03:09:14.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 shutdown_connections 2021-04-03T03:09:14.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3780570a0 0x7fb378059560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3900dbba0 0x7fb3900eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb39003d560 0x7fb3900dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 --2- 172.21.2.111:0/2340202417 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb390041060 0x7fb3900ea730 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 >> 172.21.2.111:0/2340202417 conn(0x7fb3900462b0 msgr2=0x7fb39003ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:14.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 shutdown_connections 2021-04-03T03:09:14.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.083+0000 7fb39dbe9700 1 -- 172.21.2.111:0/2340202417 wait complete. 2021-04-03T03:09:14.097 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 5 and data pool 6 2021-04-03T03:09:14.101 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:14.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f962b2c3700 1 Processor -- start 2021-04-03T03:09:14.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f962b2c3700 1 -- start start 2021-04-03T03:09:14.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f962b2c3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240b12b0 0x7f96241522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f962b2c3700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96240607d0 con 0x7f962414b700 2021-04-03T03:09:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f962b2c3700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96240b0eb0 con 0x7f96240603f0 2021-04-03T03:09:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f962b2c3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96240af400 con 0x7f96240b12b0 2021-04-03T03:09:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f9629ac0700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f962414b700 0x7f96240b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51636/0 (socket says 172.21.2.111:51636) 2021-04-03T03:09:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.147+0000 7f9629ac0700 1 -- 172.21.2.111:0/447160018 learned_addr learned my addr 172.21.2.111:0/447160018 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753435808 0 0) 0x7f96240607d0 con 0x7f962414b700 2021-04-03T03:09:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f960c003240 con 0x7f962414b700 2021-04-03T03:09:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240b12b0 0x7f96241522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2647147803 0 0) 0x7f960c003240 con 0x7f962414b700 2021-04-03T03:09:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240b12b0 msgr2=0x7f96241522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240b12b0 0x7f96241522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 >> v1:172.21.2.115:6789/0 conn(0x7f96240603f0 legacy=0x7f96240b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240af6a0 con 0x7f962414b700 2021-04-03T03:09:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f96180016d0 con 0x7f962414b700 2021-04-03T03:09:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 >> v1:172.21.2.111:6789/0 conn(0x7f962414b700 legacy=0x7f96240b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f960c003760 0x7f960c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240b12b0 0x7f960c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f960c0049d0 0x7f960c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240af400 con 0x7f960c003760 2021-04-03T03:09:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c004200 con 0x7f960c0049d0 2021-04-03T03:09:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c00b730 con 0x7f96240b12b0 2021-04-03T03:09:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f960c003760 0x7f960c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 -- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240b12b0 msgr2=0x7f960c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240b12b0 0x7f960c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 -- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f960c0049d0 msgr2=0x7f960c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f960c0049d0 0x7f960c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 -- 172.21.2.111:0/447160018 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c003240 con 0x7f960c003760 2021-04-03T03:09:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f960c003760 0x7f960c003b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f960c0054b0 tx=0x7f9620007b70).ready entity=mon.0 client_cookie=ec8ca39f5865af62 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f962000c620 con 0x7f960c003760 2021-04-03T03:09:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962000c780 con 0x7f960c003760 2021-04-03T03:09:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96292bf700 1 -- 172.21.2.111:0/447160018 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f962000cae0 con 0x7f960c003760 2021-04-03T03:09:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- 172.21.2.111:0/447160018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f960c003760 msgr2=0x7f960c003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 --2- 172.21.2.111:0/447160018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f960c003760 0x7f960c003b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f960c0054b0 tx=0x7f9620007b70).stop 2021-04-03T03:09:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 -- 172.21.2.111:0/447160018 reap_dead start 2021-04-03T03:09:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- 172.21.2.111:0/447160018 shutdown_connections 2021-04-03T03:09:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- 172.21.2.111:0/447160018 >> 172.21.2.111:0/447160018 conn(0x7f96240b61c0 msgr2=0x7f962405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- 172.21.2.111:0/447160018 shutdown_connections 2021-04-03T03:09:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- 172.21.2.111:0/447160018 wait complete. 2021-04-03T03:09:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 Processor -- start 2021-04-03T03:09:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- start start 2021-04-03T03:09:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96240ad620 0x7f9624159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240abc40 0x7f9624064230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9624064720 0x7f9624146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240ab050 con 0x7f9624064720 2021-04-03T03:09:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240ada00 con 0x7f96240ad620 2021-04-03T03:09:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96240aa110 con 0x7f96240abc40 2021-04-03T03:09:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9624064720 0x7f9624146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9624064720 0x7f9624146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50854/0 (socket says 172.21.2.111:50854) 2021-04-03T03:09:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 -- 172.21.2.111:0/4214256741 learned_addr learned my addr 172.21.2.111:0/4214256741 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f9629ac0700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240abc40 0x7f9624064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96240ad620 0x7f9624159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 -- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240abc40 msgr2=0x7f9624064230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240abc40 0x7f9624064230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 -- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96240ad620 msgr2=0x7f9624159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96240ad620 0x7f9624159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 -- 172.21.2.111:0/4214256741 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240aad40 con 0x7f9624064720 2021-04-03T03:09:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962a2c1700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96240ad620 0x7f9624159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962aac2700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9624064720 0x7f9624146930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f961c000b20 tx=0x7f961c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96177fe700 1 -- 172.21.2.111:0/4214256741 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c0097d0 con 0x7f9624064720 2021-04-03T03:09:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96177fe700 1 -- 172.21.2.111:0/4214256741 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c009930 con 0x7f9624064720 2021-04-03T03:09:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624064bb0 con 0x7f9624064720 2021-04-03T03:09:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f96177fe700 1 -- 172.21.2.111:0/4214256741 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c009c90 con 0x7f9624064720 2021-04-03T03:09:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.151+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96240a9db0 con 0x7f9624064720 2021-04-03T03:09:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.155+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96240aa310 con 0x7f9624064720 2021-04-03T03:09:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.155+0000 7f96177fe700 1 -- 172.21.2.111:0/4214256741 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f961c012420 con 0x7f9624064720 2021-04-03T03:09:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.155+0000 7f96177fe700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9604056ec0 0x7f9604059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.155+0000 7f96177fe700 1 -- 172.21.2.111:0/4214256741 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f961c070130 con 0x7f9624064720 2021-04-03T03:09:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.155+0000 7f962a2c1700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9604056ec0 0x7f9604059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.155+0000 7f962a2c1700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9604056ec0 0x7f9604059380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9620002000 tx=0x7f9620016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.155+0000 7f96177fe700 1 -- 172.21.2.111:0/4214256741 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f961c0454c0 con 0x7f9624064720 2021-04-03T03:09:14.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.251+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9624057660 con 0x7f9624064720 2021-04-03T03:09:14.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.251+0000 7f96177fe700 1 -- 172.21.2.111:0/4214256741 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f961c048ae0 con 0x7f9624064720 2021-04-03T03:09:14.266 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:14.266 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":32,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:09:14.082867+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:09:12.356149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-04-03T03:09:13.051346+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:14.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9604056ec0 msgr2=0x7f9604059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9604056ec0 0x7f9604059380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9620002000 tx=0x7f9620016000).stop 2021-04-03T03:09:14.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9624064720 msgr2=0x7f9624146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9624064720 0x7f9624146930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f961c000b20 tx=0x7f961c008040).stop 2021-04-03T03:09:14.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 shutdown_connections 2021-04-03T03:09:14.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9624064720 0x7f9624146930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9604056ec0 0x7f9604059380 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96240ad620 0x7f9624159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 --2- 172.21.2.111:0/4214256741 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96240abc40 0x7f9624064230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 >> 172.21.2.111:0/4214256741 conn(0x7f96240b61c0 msgr2=0x7f962414a640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:14.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 shutdown_connections 2021-04-03T03:09:14.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.255+0000 7f962b2c3700 1 -- 172.21.2.111:0/4214256741 wait complete. 2021-04-03T03:09:14.276 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:14.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 Processor -- start 2021-04-03T03:09:14.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- start start 2021-04-03T03:09:14.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f14a014b700 0x7f14a00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f14a00b0c90 0x7f14a0147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a00af400 con 0x7f14a00603f0 2021-04-03T03:09:14.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a0060860 con 0x7f14a00b0c90 2021-04-03T03:09:14.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a00af6a0 con 0x7f14a014b700 2021-04-03T03:09:14.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f14a00b0c90 0x7f14a0147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f14a00b0c90 0x7f14a0147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50864/0 (socket says 172.21.2.111:50864) 2021-04-03T03:09:14.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 -- 172.21.2.111:0/1936769935 learned_addr learned my addr 172.21.2.111:0/1936769935 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:14.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 -- 172.21.2.111:0/1936769935 >> v2:172.21.2.115:3300/0 conn(0x7f14a014b700 msgr2=0x7f14a00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 --2- 172.21.2.111:0/1936769935 >> v2:172.21.2.115:3300/0 conn(0x7f14a014b700 0x7f14a00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 -- 172.21.2.111:0/1936769935 >> v1:172.21.2.115:6789/0 conn(0x7f14a00603f0 legacy=0x7f14a00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 -- 172.21.2.111:0/1936769935 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a00afb40 con 0x7f14a00b0c90 2021-04-03T03:09:14.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149ffff700 1 --2- 172.21.2.111:0/1936769935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00b0c90 0x7f14a0147680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f14a006a3b0 tx=0x7f1490006e90).ready entity=mon.0 client_cookie=2bb971de08e03452 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149e7fc700 1 -- 172.21.2.111:0/1936769935 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1490009930 con 0x7f14a00b0c90 2021-04-03T03:09:14.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149e7fc700 1 -- 172.21.2.111:0/1936769935 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1490009a90 con 0x7f14a00b0c90 2021-04-03T03:09:14.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f149e7fc700 1 -- 172.21.2.111:0/1936769935 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1490009e30 con 0x7f14a00b0c90 2021-04-03T03:09:14.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- 172.21.2.111:0/1936769935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00b0c90 msgr2=0x7f14a0147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/1936769935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00b0c90 0x7f14a0147680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f14a006a3b0 tx=0x7f1490006e90).stop 2021-04-03T03:09:14.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- 172.21.2.111:0/1936769935 shutdown_connections 2021-04-03T03:09:14.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/1936769935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00b0c90 0x7f14a0147680 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/1936769935 >> v2:172.21.2.115:3300/0 conn(0x7f14a014b700 0x7f14a00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- 172.21.2.111:0/1936769935 >> 172.21.2.111:0/1936769935 conn(0x7f14a00b61c0 msgr2=0x7f14a005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:14.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- 172.21.2.111:0/1936769935 shutdown_connections 2021-04-03T03:09:14.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- 172.21.2.111:0/1936769935 wait complete. 2021-04-03T03:09:14.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 Processor -- start 2021-04-03T03:09:14.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 -- start start 2021-04-03T03:09:14.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00603f0 0x7f14a00ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.383+0000 7f14a4f5a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14a00b0c90 0x7f14a00675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f14a4f5a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14a014b700 0x7f14a0067a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f14a4f5a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a0157910 con 0x7f14a00603f0 2021-04-03T03:09:14.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f14a4f5a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00a9ac0 con 0x7f14a014b700 2021-04-03T03:09:14.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f14a4f5a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a0064340 con 0x7f14a00b0c90 2021-04-03T03:09:14.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00603f0 0x7f14a00ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00603f0 0x7f14a00ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50866/0 (socket says 172.21.2.111:50866) 2021-04-03T03:09:14.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 -- 172.21.2.111:0/2619278568 learned_addr learned my addr 172.21.2.111:0/2619278568 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:14.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 -- 172.21.2.111:0/2619278568 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14a00b0c90 msgr2=0x7f14a00675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14a00b0c90 0x7f14a00675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 -- 172.21.2.111:0/2619278568 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14a014b700 msgr2=0x7f14a0067a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14a014b700 0x7f14a0067a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 -- 172.21.2.111:0/2619278568 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a0064570 con 0x7f14a00603f0 2021-04-03T03:09:14.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149f7fe700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00603f0 0x7f14a00ab6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f148c002f50 tx=0x7f148c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149cff9700 1 -- 172.21.2.111:0/2619278568 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f148c00d4a0 con 0x7f14a00603f0 2021-04-03T03:09:14.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149cff9700 1 -- 172.21.2.111:0/2619278568 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c00d600 con 0x7f14a00603f0 2021-04-03T03:09:14.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a0064810 con 0x7f14a00603f0 2021-04-03T03:09:14.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a00abed0 con 0x7f14a00603f0 2021-04-03T03:09:14.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149cff9700 1 -- 172.21.2.111:0/2619278568 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f148c00d960 con 0x7f14a00603f0 2021-04-03T03:09:14.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a0057660 con 0x7f14a00603f0 2021-04-03T03:09:14.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149cff9700 1 -- 172.21.2.111:0/2619278568 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f148c00f070 con 0x7f14a00603f0 2021-04-03T03:09:14.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149cff9700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f147c056ef0 0x7f147c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149cff9700 1 -- 172.21.2.111:0/2619278568 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f148c06fe40 con 0x7f14a00603f0 2021-04-03T03:09:14.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149effd700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f147c056ef0 0x7f147c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.387+0000 7f149effd700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f147c056ef0 0x7f147c0593b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f14980032b0 tx=0x7f1498006d70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.391+0000 7f149cff9700 1 -- 172.21.2.111:0/2619278568 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f148c045120 con 0x7f14a00603f0 2021-04-03T03:09:14.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14a00b1550 con 0x7f14a00603f0 2021-04-03T03:09:14.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f149cff9700 1 -- 172.21.2.111:0/2619278568 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f148c048740 con 0x7f14a00603f0 2021-04-03T03:09:14.492 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:14.493 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":32,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:09:14.082867+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:09:12.356149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-04-03T03:09:13.051346+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:14.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f147c056ef0 msgr2=0x7f147c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f147c056ef0 0x7f147c0593b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f14980032b0 tx=0x7f1498006d70).stop 2021-04-03T03:09:14.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00603f0 msgr2=0x7f14a00ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00603f0 0x7f14a00ab6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f148c002f50 tx=0x7f148c006dd0).stop 2021-04-03T03:09:14.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 shutdown_connections 2021-04-03T03:09:14.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f147c056ef0 0x7f147c0593b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f14a014b700 0x7f14a0067a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f14a00603f0 0x7f14a00ab6d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 --2- 172.21.2.111:0/2619278568 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f14a00b0c90 0x7f14a00675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:14.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 >> 172.21.2.111:0/2619278568 conn(0x7f14a00b61c0 msgr2=0x7f14a00acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:14.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 shutdown_connections 2021-04-03T03:09:14.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.483+0000 7f14a4f5a700 1 -- 172.21.2.111:0/2619278568 wait complete. 2021-04-03T03:09:14.499 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:09:14.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446fde7700 1 Processor -- start 2021-04-03T03:09:14.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446fde7700 1 -- start start 2021-04-03T03:09:14.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 0x7f4470147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446fde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44700607d0 con 0x7f44700603f0 2021-04-03T03:09:14.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446fde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447014b090 con 0x7f44700add50 2021-04-03T03:09:14.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447014b330 con 0x7f4470144ef0 2021-04-03T03:09:14.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446ede5700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f44700603f0 0x7f44701447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51666/0 (socket says 172.21.2.111:51666) 2021-04-03T03:09:14.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446ede5700 1 -- 172.21.2.111:0/2371996511 learned_addr learned my addr 172.21.2.111:0/2371996511 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:14.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30056736 0 0) 0x7f44700607d0 con 0x7f44700603f0 2021-04-03T03:09:14.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4450003240 con 0x7f44700603f0 2021-04-03T03:09:14.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446f5e6700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 0x7f4470147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1277365168 0 0) 0x7f4450003240 con 0x7f44700603f0 2021-04-03T03:09:14.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 msgr2=0x7f4470147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 0x7f4470147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 >> v1:172.21.2.115:6789/0 conn(0x7f44700add50 legacy=0x7f4470147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447014b5d0 con 0x7f44700603f0 2021-04-03T03:09:14.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f44600016d0 con 0x7f44700603f0 2021-04-03T03:09:14.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 >> v1:172.21.2.111:6789/0 conn(0x7f44700603f0 legacy=0x7f44701447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 0x7f44500037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4450003d40 0x7f44500044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44500049d0 0x7f4450004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447014b330 con 0x7f4450003d40 2021-04-03T03:09:14.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44500042d0 con 0x7f44500049d0 2021-04-03T03:09:14.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.563+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445000b730 con 0x7f4470144ef0 2021-04-03T03:09:14.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4450003d40 0x7f44500044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44500049d0 0x7f4450004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446ede5700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 0x7f44500037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 -- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 msgr2=0x7f44500037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4470144ef0 0x7f44500037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 -- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44500049d0 msgr2=0x7f4450004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:14.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44500049d0 0x7f4450004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 -- 172.21.2.111:0/2371996511 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450003240 con 0x7f4450003d40 2021-04-03T03:09:14.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4450003d40 0x7f44500044e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f44700b5640 tx=0x7f4458007b70).ready entity=mon.0 client_cookie=321d3f1c73c08a71 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4458001de0 con 0x7f4450003d40 2021-04-03T03:09:14.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445800c600 con 0x7f4450003d40 2021-04-03T03:09:14.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446dde3700 1 -- 172.21.2.111:0/2371996511 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445800c960 con 0x7f4450003d40 2021-04-03T03:09:14.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/2371996511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4450003d40 msgr2=0x7f44500044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 --2- 172.21.2.111:0/2371996511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4450003d40 0x7f44500044e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f44700b5640 tx=0x7f4458007b70).stop 2021-04-03T03:09:14.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 -- 172.21.2.111:0/2371996511 reap_dead start 2021-04-03T03:09:14.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/2371996511 shutdown_connections 2021-04-03T03:09:14.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/2371996511 >> 172.21.2.111:0/2371996511 conn(0x7f44700b61c0 msgr2=0x7f447005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:14.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/2371996511 shutdown_connections 2021-04-03T03:09:14.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/2371996511 wait complete. 2021-04-03T03:09:14.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 Processor -- start 2021-04-03T03:09:14.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- start start 2021-04-03T03:09:14.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44700a9b70 0x7f44701462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4470147a40 0x7f44701467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44701481d0 0x7f447015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44701577b0 con 0x7f44701481d0 2021-04-03T03:09:14.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44700abef0 con 0x7f4470147a40 2021-04-03T03:09:14.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44701574d0 con 0x7f44700a9b70 2021-04-03T03:09:14.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4470147a40 0x7f44701467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4470147a40 0x7f44701467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:40596/0 (socket says 172.21.2.111:40596) 2021-04-03T03:09:14.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446e5e4700 1 -- 172.21.2.111:0/3496084344 learned_addr learned my addr 172.21.2.111:0/3496084344 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:14.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44701481d0 0x7f447015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446ede5700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44700a9b70 0x7f44701462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 -- 172.21.2.111:0/3496084344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44700a9b70 msgr2=0x7f44701462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44700a9b70 0x7f44701462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 -- 172.21.2.111:0/3496084344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4470147a40 msgr2=0x7f44701467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:14.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4470147a40 0x7f44701467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:14.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 -- 172.21.2.111:0/3496084344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44700a9f50 con 0x7f44701481d0 2021-04-03T03:09:14.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446f5e6700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44701481d0 0x7f447015a060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f446800d000 tx=0x7f4468006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f4467fff700 1 -- 172.21.2.111:0/3496084344 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4468018070 con 0x7f44701481d0 2021-04-03T03:09:14.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f4467fff700 1 -- 172.21.2.111:0/3496084344 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800c040 con 0x7f44701481d0 2021-04-03T03:09:14.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f4467fff700 1 -- 172.21.2.111:0/3496084344 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446800db50 con 0x7f44701481d0 2021-04-03T03:09:14.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4470146cc0 con 0x7f44701481d0 2021-04-03T03:09:14.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447015a6e0 con 0x7f44701481d0 2021-04-03T03:09:14.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.567+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4470146e50 con 0x7f44701481d0 2021-04-03T03:09:14.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.571+0000 7f4467fff700 1 -- 172.21.2.111:0/3496084344 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f446800dcb0 con 0x7f44701481d0 2021-04-03T03:09:14.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.571+0000 7f4467fff700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4454056ea0 0x7f4454059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:14.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.571+0000 7f4467fff700 1 -- 172.21.2.111:0/3496084344 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f4468070520 con 0x7f44701481d0 2021-04-03T03:09:14.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.571+0000 7f446ede5700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4454056ea0 0x7f4454059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:14.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.571+0000 7f446ede5700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4454056ea0 0x7f4454059360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f4460005000 tx=0x7f447014b220).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:14.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.571+0000 7f4467fff700 1 -- 172.21.2.111:0/3496084344 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f44680458b0 con 0x7f44701481d0 2021-04-03T03:09:14.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:14.695+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4470057660 con 0x7f44701481d0 2021-04-03T03:09:15.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f4467fff700 1 -- 172.21.2.111:0/3496084344 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4468048ed0 con 0x7f44701481d0 2021-04-03T03:09:15.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4454056ea0 msgr2=0x7f4454059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4454056ea0 0x7f4454059360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f4460005000 tx=0x7f447014b220).stop 2021-04-03T03:09:15.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44701481d0 msgr2=0x7f447015a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44701481d0 0x7f447015a060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f446800d000 tx=0x7f4468006f90).stop 2021-04-03T03:09:15.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 shutdown_connections 2021-04-03T03:09:15.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44701481d0 0x7f447015a060 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44700a9b70 0x7f44701462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4454056ea0 0x7f4454059360 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 --2- 172.21.2.111:0/3496084344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4470147a40 0x7f44701467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 >> 172.21.2.111:0/3496084344 conn(0x7f44700b61c0 msgr2=0x7f447014a570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 shutdown_connections 2021-04-03T03:09:15.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.091+0000 7f446fde7700 1 -- 172.21.2.111:0/3496084344 wait complete. 2021-04-03T03:09:15.108 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:15.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 Processor -- start 2021-04-03T03:09:15.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 -- start start 2021-04-03T03:09:15.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0603f0 0x7fe71c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe71c14b700 0x7fe71c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0607d0 con 0x7fe71c0b12b0 2021-04-03T03:09:15.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c0af400 con 0x7fe71c14b700 2021-04-03T03:09:15.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0af6a0 con 0x7fe71c0603f0 2021-04-03T03:09:15.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe71c0b12b0 0x7fe71c1465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51686/0 (socket says 172.21.2.111:51686) 2021-04-03T03:09:15.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 -- 172.21.2.111:0/2235471796 learned_addr learned my addr 172.21.2.111:0/2235471796 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:15.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe719d9b700 1 --2- 172.21.2.111:0/2235471796 >> v2:172.21.2.115:3300/0 conn(0x7fe71c14b700 0x7fe71c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235030519 0 0) 0x7fe71c0607d0 con 0x7fe71c0b12b0 2021-04-03T03:09:15.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6fc003240 con 0x7fe71c0b12b0 2021-04-03T03:09:15.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71a59c700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0603f0 0x7fe71c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3129245680 0 0) 0x7fe6fc003240 con 0x7fe71c0b12b0 2021-04-03T03:09:15.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0603f0 msgr2=0x7fe71c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0603f0 0x7fe71c0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 >> v2:172.21.2.115:3300/0 conn(0x7fe71c14b700 msgr2=0x7fe71c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 --2- 172.21.2.111:0/2235471796 >> v2:172.21.2.115:3300/0 conn(0x7fe71c14b700 0x7fe71c0b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0afb40 con 0x7fe71c0b12b0 2021-04-03T03:09:15.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe714001a50 con 0x7fe71c0b12b0 2021-04-03T03:09:15.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 >> v1:172.21.2.111:6789/0 conn(0x7fe71c0b12b0 legacy=0x7fe71c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0603f0 0x7fe6fc003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6fc003c40 0x7fe6fc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6fc0049d0 0x7fe6fc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0af400 con 0x7fe6fc0049d0 2021-04-03T03:09:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0af6a0 con 0x7fe6fc003c40 2021-04-03T03:09:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc004200 con 0x7fe71c0603f0 2021-04-03T03:09:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6fc0049d0 0x7fe6fc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 -- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0603f0 msgr2=0x7fe6fc003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0603f0 0x7fe6fc003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 -- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6fc003c40 msgr2=0x7fe6fc0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6fc003c40 0x7fe6fc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 -- 172.21.2.111:0/2235471796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc003240 con 0x7fe6fc0049d0 2021-04-03T03:09:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6fc0049d0 0x7fe6fc004e50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fe714001760 tx=0x7fe714007b50).ready entity=mon.0 client_cookie=7ddb24ea053e32fb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe714005b80 con 0x7fe6fc0049d0 2021-04-03T03:09:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe714005ce0 con 0x7fe6fc0049d0 2021-04-03T03:09:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71959a700 1 -- 172.21.2.111:0/2235471796 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7140072c0 con 0x7fe6fc0049d0 2021-04-03T03:09:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 -- 172.21.2.111:0/2235471796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6fc0049d0 msgr2=0x7fe6fc004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71b59e700 1 --2- 172.21.2.111:0/2235471796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6fc0049d0 0x7fe6fc004e50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fe714001760 tx=0x7fe714007b50).stop 2021-04-03T03:09:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.159+0000 7fe71ad9d700 1 -- 172.21.2.111:0/2235471796 reap_dead start 2021-04-03T03:09:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- 172.21.2.111:0/2235471796 shutdown_connections 2021-04-03T03:09:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- 172.21.2.111:0/2235471796 >> 172.21.2.111:0/2235471796 conn(0x7fe71c0b61c0 msgr2=0x7fe71c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- 172.21.2.111:0/2235471796 shutdown_connections 2021-04-03T03:09:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- 172.21.2.111:0/2235471796 wait complete. 2021-04-03T03:09:15.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 Processor -- start 2021-04-03T03:09:15.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- start start 2021-04-03T03:09:15.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe71c14a790 0x7fe71c065a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0abb30 0x7fe71c159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe71c15a2d0 0x7fe71c0a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c065f20 con 0x7fe71c15a2d0 2021-04-03T03:09:15.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c14ab70 con 0x7fe71c14a790 2021-04-03T03:09:15.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0ab1b0 con 0x7fe71c0abb30 2021-04-03T03:09:15.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe71c15a2d0 0x7fe71c0a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe71c15a2d0 0x7fe71c0a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50908/0 (socket says 172.21.2.111:50908) 2021-04-03T03:09:15.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 -- 172.21.2.111:0/3455299034 learned_addr learned my addr 172.21.2.111:0/3455299034 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:15.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 -- 172.21.2.111:0/3455299034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0abb30 msgr2=0x7fe71c159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0abb30 0x7fe71c159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 -- 172.21.2.111:0/3455299034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe71c14a790 msgr2=0x7fe71c065a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe71c14a790 0x7fe71c065a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 -- 172.21.2.111:0/3455299034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0ab450 con 0x7fe71c15a2d0 2021-04-03T03:09:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71ad9d700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe71c15a2d0 0x7fe71c0a8f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fe71c0620f0 tx=0x7fe714013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe7137fe700 1 -- 172.21.2.111:0/3455299034 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe714013450 con 0x7fe71c15a2d0 2021-04-03T03:09:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe7137fe700 1 -- 172.21.2.111:0/3455299034 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71400b070 con 0x7fe71c15a2d0 2021-04-03T03:09:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c15a780 con 0x7fe71c15a2d0 2021-04-03T03:09:15.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe7137fe700 1 -- 172.21.2.111:0/3455299034 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7140137a0 con 0x7fe71c15a2d0 2021-04-03T03:09:15.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c0a95b0 con 0x7fe71c15a2d0 2021-04-03T03:09:15.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c0ab5e0 con 0x7fe71c15a2d0 2021-04-03T03:09:15.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe7137fe700 1 -- 172.21.2.111:0/3455299034 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe714026070 con 0x7fe71c15a2d0 2021-04-03T03:09:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe7137fe700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe700056f20 0x7fe7000593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe7137fe700 1 -- 172.21.2.111:0/3455299034 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe714070b90 con 0x7fe71c15a2d0 2021-04-03T03:09:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71a59c700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe700056f20 0x7fe7000593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.163+0000 7fe71a59c700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe700056f20 0x7fe7000593e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe70400a000 tx=0x7fe704008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.167+0000 7fe7137fe700 1 -- 172.21.2.111:0/3455299034 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe714045f80 con 0x7fe71c15a2d0 2021-04-03T03:09:15.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.291+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe71c057660 con 0x7fe71c15a2d0 2021-04-03T03:09:15.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.291+0000 7fe7137fe700 1 -- 172.21.2.111:0/3455299034 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fe7140495a0 con 0x7fe71c15a2d0 2021-04-03T03:09:15.301 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:15.302 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:09:15.086110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1150523943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1150523943},{"type":"v1","addr":"172.21.2.111:6835","nonce":1150523943}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:09:15.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe700056f20 msgr2=0x7fe7000593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe700056f20 0x7fe7000593e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe70400a000 tx=0x7fe704008040).stop 2021-04-03T03:09:15.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe71c15a2d0 msgr2=0x7fe71c0a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe71c15a2d0 0x7fe71c0a8f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fe71c0620f0 tx=0x7fe714013000).stop 2021-04-03T03:09:15.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 shutdown_connections 2021-04-03T03:09:15.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe71c15a2d0 0x7fe71c0a8f30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe700056f20 0x7fe7000593e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe71c14a790 0x7fe71c065a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 --2- 172.21.2.111:0/3455299034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe71c0abb30 0x7fe71c159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 >> 172.21.2.111:0/3455299034 conn(0x7fe71c0b61c0 msgr2=0x7fe71c145fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 shutdown_connections 2021-04-03T03:09:15.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.295+0000 7fe71b59e700 1 -- 172.21.2.111:0/3455299034 wait complete. 2021-04-03T03:09:15.306 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 11 2021-04-03T03:09:15.309 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:15.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.355+0000 7f77dd689700 1 Processor -- start 2021-04-03T03:09:15.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.355+0000 7f77dd689700 1 -- start start 2021-04-03T03:09:15.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dd689700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f77d814b700 0x7f77d80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dd689700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dd689700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d80607d0 con 0x7f77d80603f0 2021-04-03T03:09:15.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dd689700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d80af400 con 0x7f77d814b700 2021-04-03T03:09:15.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dd689700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80af6a0 con 0x7f77d80b0c90 2021-04-03T03:09:15.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dce88700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77d7fff700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f77d80603f0 0x7f77d80b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:36010/0 (socket says 172.21.2.111:36010) 2021-04-03T03:09:15.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77d7fff700 1 -- 172.21.2.111:0/2723394170 learned_addr learned my addr 172.21.2.111:0/2723394170 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:15.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77d77fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f77d814b700 0x7f77d80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dce88700 1 -- 172.21.2.111:0/2723394170 >> v2:172.21.2.115:3300/0 conn(0x7f77d814b700 msgr2=0x7f77d80b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dce88700 1 --2- 172.21.2.111:0/2723394170 >> v2:172.21.2.115:3300/0 conn(0x7f77d814b700 0x7f77d80b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dce88700 1 -- 172.21.2.111:0/2723394170 >> v1:172.21.2.115:6789/0 conn(0x7f77d80603f0 legacy=0x7f77d80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dce88700 1 -- 172.21.2.111:0/2723394170 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80afb40 con 0x7f77d80b0c90 2021-04-03T03:09:15.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77d77fe700 1 --2- 172.21.2.111:0/2723394170 >> v2:172.21.2.115:3300/0 conn(0x7f77d814b700 0x7f77d80b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:09:15.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dce88700 1 --2- 172.21.2.111:0/2723394170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8147680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f77d00032b0 tx=0x7f77d00091d0).ready entity=mon.2 client_cookie=bcf39b71c6f913f6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77d6ffd700 1 -- 172.21.2.111:0/2723394170 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d000b9a0 con 0x7f77d80b0c90 2021-04-03T03:09:15.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77d6ffd700 1 -- 172.21.2.111:0/2723394170 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d000bb00 con 0x7f77d80b0c90 2021-04-03T03:09:15.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77d6ffd700 1 -- 172.21.2.111:0/2723394170 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d000bc70 con 0x7f77d80b0c90 2021-04-03T03:09:15.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dd689700 1 -- 172.21.2.111:0/2723394170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 msgr2=0x7f77d8147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.359+0000 7f77dd689700 1 --2- 172.21.2.111:0/2723394170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8147680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f77d00032b0 tx=0x7f77d00091d0).stop 2021-04-03T03:09:15.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- 172.21.2.111:0/2723394170 shutdown_connections 2021-04-03T03:09:15.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 --2- 172.21.2.111:0/2723394170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8147680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 --2- 172.21.2.111:0/2723394170 >> v2:172.21.2.115:3300/0 conn(0x7f77d814b700 0x7f77d80b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- 172.21.2.111:0/2723394170 >> 172.21.2.111:0/2723394170 conn(0x7f77d80b61c0 msgr2=0x7f77d805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- 172.21.2.111:0/2723394170 shutdown_connections 2021-04-03T03:09:15.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- 172.21.2.111:0/2723394170 wait complete. 2021-04-03T03:09:15.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 Processor -- start 2021-04-03T03:09:15.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- start start 2021-04-03T03:09:15.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77d80603f0 0x7f77d80aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8067500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77d814b700 0x7f77d80679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80630b0 con 0x7f77d814b700 2021-04-03T03:09:15.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8064f90 con 0x7f77d80603f0 2021-04-03T03:09:15.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8062970 con 0x7f77d80b0c90 2021-04-03T03:09:15.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77d814b700 0x7f77d80679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77d814b700 0x7f77d80679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50924/0 (socket says 172.21.2.111:50924) 2021-04-03T03:09:15.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 -- 172.21.2.111:0/738302699 learned_addr learned my addr 172.21.2.111:0/738302699 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:15.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77d7fff700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77d80603f0 0x7f77d80aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77d77fe700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 -- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 msgr2=0x7f77d8067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 -- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77d80603f0 msgr2=0x7f77d80aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77d80603f0 0x7f77d80aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 -- 172.21.2.111:0/738302699 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8067ee0 con 0x7f77d814b700 2021-04-03T03:09:15.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77d77fe700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:09:15.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dce88700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77d814b700 0x7f77d80679f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f77d00032b0 tx=0x7f77d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77d57fa700 1 -- 172.21.2.111:0/738302699 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d0011c70 con 0x7f77d814b700 2021-04-03T03:09:15.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77d57fa700 1 -- 172.21.2.111:0/738302699 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d0011dd0 con 0x7f77d814b700 2021-04-03T03:09:15.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77d57fa700 1 -- 172.21.2.111:0/738302699 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d0020630 con 0x7f77d814b700 2021-04-03T03:09:15.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d80a95e0 con 0x7f77d814b700 2021-04-03T03:09:15.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.363+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d8064d30 con 0x7f77d814b700 2021-04-03T03:09:15.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.367+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d80ab610 con 0x7f77d814b700 2021-04-03T03:09:15.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.367+0000 7f77d57fa700 1 -- 172.21.2.111:0/738302699 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f77d0026020 con 0x7f77d814b700 2021-04-03T03:09:15.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.367+0000 7f77d57fa700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77bc056ea0 0x7f77bc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.367+0000 7f77d57fa700 1 -- 172.21.2.111:0/738302699 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f77d0071030 con 0x7f77d814b700 2021-04-03T03:09:15.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.367+0000 7f77d7fff700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77bc056ea0 0x7f77bc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.367+0000 7f77d7fff700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77bc056ea0 0x7f77bc059360 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f77c800d000 tx=0x7f77c8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.367+0000 7f77d57fa700 1 -- 172.21.2.111:0/738302699 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f77d0046310 con 0x7f77d814b700 2021-04-03T03:09:15.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77d8057660 con 0x7f77d814b700 2021-04-03T03:09:15.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77d57fa700 1 -- 172.21.2.111:0/738302699 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f77d0015020 con 0x7f77d814b700 2021-04-03T03:09:15.468 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:15.469 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:09:15.086914+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:09:12.356149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-04-03T03:09:13.051346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:15.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77bc056ea0 msgr2=0x7f77bc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77bc056ea0 0x7f77bc059360 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f77c800d000 tx=0x7f77c8009040).stop 2021-04-03T03:09:15.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77d814b700 msgr2=0x7f77d80679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77d814b700 0x7f77d80679f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f77d00032b0 tx=0x7f77d0016000).stop 2021-04-03T03:09:15.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 shutdown_connections 2021-04-03T03:09:15.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77d814b700 0x7f77d80679f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77bc056ea0 0x7f77bc059360 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77d80603f0 0x7f77d80aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 --2- 172.21.2.111:0/738302699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77d80b0c90 0x7f77d8067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 >> 172.21.2.111:0/738302699 conn(0x7f77d80b61c0 msgr2=0x7f77d814ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 shutdown_connections 2021-04-03T03:09:15.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.459+0000 7f77dd689700 1 -- 172.21.2.111:0/738302699 wait complete. 2021-04-03T03:09:15.476 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:09:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8966776700 1 Processor -- start 2021-04-03T03:09:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8966776700 1 -- start start 2021-04-03T03:09:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8966776700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 0x7f8958042380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8966776700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8958041240 con 0x7f895803d350 2021-04-03T03:09:15.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8966776700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89580f1850 con 0x7f8958046ce0 2021-04-03T03:09:15.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8966776700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8958042870 con 0x7f895803f650 2021-04-03T03:09:15.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8965774700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f895803d350 0x7f89580f51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51724/0 (socket says 172.21.2.111:51724) 2021-04-03T03:09:15.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8965774700 1 -- 172.21.2.111:0/932237867 learned_addr learned my addr 172.21.2.111:0/932237867 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:15.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8967bf0700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 0x7f8958042380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400708290 0 0) 0x7f8958041240 con 0x7f895803d350 2021-04-03T03:09:15.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f894c003240 con 0x7f895803d350 2021-04-03T03:09:15.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 146864014 0 0) 0x7f894c003240 con 0x7f895803d350 2021-04-03T03:09:15.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 msgr2=0x7f8958042380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 0x7f8958042380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 >> v1:172.21.2.115:6789/0 conn(0x7f8958046ce0 legacy=0x7f8958041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580470c0 con 0x7f895803d350 2021-04-03T03:09:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f895c0016d0 con 0x7f895803d350 2021-04-03T03:09:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.527+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 >> v1:172.21.2.111:6789/0 conn(0x7f895803d350 legacy=0x7f89580f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f894c003760 0x7f894c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 0x7f894c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f894c0049d0 0x7f894c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8958042870 con 0x7f894c0049d0 2021-04-03T03:09:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c004200 con 0x7f894c003760 2021-04-03T03:09:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c04f740 con 0x7f895803f650 2021-04-03T03:09:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f894c0049d0 0x7f894c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965f75700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f894c003760 0x7f894c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 0x7f894c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 -- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 msgr2=0x7f894c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f895803f650 0x7f894c0044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 -- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f894c003760 msgr2=0x7f894c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f894c003760 0x7f894c003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 -- 172.21.2.111:0/932237867 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c003240 con 0x7f894c0049d0 2021-04-03T03:09:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965f75700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f894c003760 0x7f894c003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f894c0049d0 0x7f894c004e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f8960071000 tx=0x7f896005ad70).ready entity=mon.0 client_cookie=8b6ff2fd17b60eb4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896005df60 con 0x7f894c0049d0 2021-04-03T03:09:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89600699f0 con 0x7f894c0049d0 2021-04-03T03:09:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8964f73700 1 -- 172.21.2.111:0/932237867 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896006b610 con 0x7f894c0049d0 2021-04-03T03:09:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- 172.21.2.111:0/932237867 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f894c0049d0 msgr2=0x7f894c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 --2- 172.21.2.111:0/932237867 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f894c0049d0 0x7f894c004e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f8960071000 tx=0x7f896005ad70).stop 2021-04-03T03:09:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8967bf0700 1 -- 172.21.2.111:0/932237867 reap_dead start 2021-04-03T03:09:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- 172.21.2.111:0/932237867 shutdown_connections 2021-04-03T03:09:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- 172.21.2.111:0/932237867 >> 172.21.2.111:0/932237867 conn(0x7f895800d0c0 msgr2=0x7f89580462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- 172.21.2.111:0/932237867 shutdown_connections 2021-04-03T03:09:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- 172.21.2.111:0/932237867 wait complete. 2021-04-03T03:09:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 Processor -- start 2021-04-03T03:09:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- start start 2021-04-03T03:09:15.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89580cf200 0x7f895803be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89580d2020 0x7f8958045380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8958045870 0x7f89580d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895803a070 con 0x7f89580d2020 2021-04-03T03:09:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580cf5e0 con 0x7f8958045870 2021-04-03T03:09:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8958044380 con 0x7f89580cf200 2021-04-03T03:09:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89580d2020 0x7f8958045380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89580d2020 0x7f8958045380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50944/0 (socket says 172.21.2.111:50944) 2021-04-03T03:09:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 -- 172.21.2.111:0/742460027 learned_addr learned my addr 172.21.2.111:0/742460027 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 -- 172.21.2.111:0/742460027 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89580cf200 msgr2=0x7f895803be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89580cf200 0x7f895803be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 -- 172.21.2.111:0/742460027 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8958045870 msgr2=0x7f89580d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8958045870 0x7f89580d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 -- 172.21.2.111:0/742460027 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958044620 con 0x7f89580d2020 2021-04-03T03:09:15.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8965774700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89580d2020 0x7f8958045380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f895c008000 tx=0x7f895c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f89527fc700 1 -- 172.21.2.111:0/742460027 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895c007170 con 0x7f89580d2020 2021-04-03T03:09:15.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f89527fc700 1 -- 172.21.2.111:0/742460027 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c0072d0 con 0x7f89580d2020 2021-04-03T03:09:15.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895803c320 con 0x7f89580d2020 2021-04-03T03:09:15.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f89527fc700 1 -- 172.21.2.111:0/742460027 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895c00baa0 con 0x7f89580d2020 2021-04-03T03:09:15.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.531+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89580d2610 con 0x7f89580d2020 2021-04-03T03:09:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.535+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8958044a30 con 0x7f89580d2020 2021-04-03T03:09:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.535+0000 7f89527fc700 1 -- 172.21.2.111:0/742460027 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f895c00bc00 con 0x7f89580d2020 2021-04-03T03:09:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.535+0000 7f89527fc700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8948056ee0 0x7f89480593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.535+0000 7f89527fc700 1 -- 172.21.2.111:0/742460027 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f895c0705b0 con 0x7f89580d2020 2021-04-03T03:09:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.535+0000 7f8965f75700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8948056ee0 0x7f89480593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.535+0000 7f8965f75700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8948056ee0 0x7f89480593a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8954008000 tx=0x7f8954009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.535+0000 7f89527fc700 1 -- 172.21.2.111:0/742460027 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8958044a30 con 0x7f89580d2020 2021-04-03T03:09:15.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.663+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8958044a30 con 0x7f89580d2020 2021-04-03T03:09:15.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.667+0000 7f89527fc700 1 -- 172.21.2.111:0/742460027 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8958044a30 con 0x7f89580d2020 2021-04-03T03:09:15.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8948056ee0 msgr2=0x7f89480593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8948056ee0 0x7f89480593a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8954008000 tx=0x7f8954009040).stop 2021-04-03T03:09:15.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89580d2020 msgr2=0x7f8958045380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89580d2020 0x7f8958045380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f895c008000 tx=0x7f895c009040).stop 2021-04-03T03:09:15.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 shutdown_connections 2021-04-03T03:09:15.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8948056ee0 0x7f89480593a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8958045870 0x7f89580d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89580cf200 0x7f895803be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 --2- 172.21.2.111:0/742460027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89580d2020 0x7f8958045380 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 >> 172.21.2.111:0/742460027 conn(0x7f895800d0c0 msgr2=0x7f8958041430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 shutdown_connections 2021-04-03T03:09:15.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.671+0000 7f8966776700 1 -- 172.21.2.111:0/742460027 wait complete. 2021-04-03T03:09:15.682 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:09:15.686 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:09:15.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.735+0000 7f85f8937700 1 Processor -- start 2021-04-03T03:09:15.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.735+0000 7f85f8937700 1 -- start start 2021-04-03T03:09:15.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f85ec03d350 0x7f85ec03f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ec041240 con 0x7f85ec042380 2021-04-03T03:09:15.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ec0f1850 con 0x7f85ec046ce0 2021-04-03T03:09:15.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ec042810 con 0x7f85ec03d350 2021-04-03T03:09:15.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f9db1700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f85ec042380 0x7f85ec0f51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51742/0 (socket says 172.21.2.111:51742) 2021-04-03T03:09:15.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f9db1700 1 -- 172.21.2.111:0/670409647 learned_addr learned my addr 172.21.2.111:0/670409647 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:15.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f37fe700 1 --2- 172.21.2.111:0/670409647 >> v2:172.21.2.115:3300/0 conn(0x7f85ec03d350 0x7f85ec03f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f2ffd700 1 -- 172.21.2.111:0/670409647 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098030966 0 0) 0x7f85ec041240 con 0x7f85ec042380 2021-04-03T03:09:15.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f2ffd700 1 -- 172.21.2.111:0/670409647 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e4003240 con 0x7f85ec042380 2021-04-03T03:09:15.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f37fe700 1 -- 172.21.2.111:0/670409647 >> v1:172.21.2.115:6789/0 conn(0x7f85ec046ce0 legacy=0x7f85ec041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f37fe700 1 -- 172.21.2.111:0/670409647 >> v1:172.21.2.111:6789/0 conn(0x7f85ec042380 legacy=0x7f85ec0f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f37fe700 1 -- 172.21.2.111:0/670409647 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec0470c0 con 0x7f85ec03d350 2021-04-03T03:09:15.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f37fe700 1 --2- 172.21.2.111:0/670409647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec03d350 0x7f85ec03f650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f85e0002f30 tx=0x7f85e0006dd0).ready entity=mon.1 client_cookie=9b08563e948d95c8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f2ffd700 1 -- 172.21.2.111:0/670409647 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e0009800 con 0x7f85ec03d350 2021-04-03T03:09:15.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f2ffd700 1 -- 172.21.2.111:0/670409647 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0009960 con 0x7f85ec03d350 2021-04-03T03:09:15.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f2ffd700 1 -- 172.21.2.111:0/670409647 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e0009cc0 con 0x7f85ec03d350 2021-04-03T03:09:15.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- 172.21.2.111:0/670409647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec03d350 msgr2=0x7f85ec03f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 --2- 172.21.2.111:0/670409647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec03d350 0x7f85ec03f650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f85e0002f30 tx=0x7f85e0006dd0).stop 2021-04-03T03:09:15.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- 172.21.2.111:0/670409647 shutdown_connections 2021-04-03T03:09:15.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 --2- 172.21.2.111:0/670409647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec03d350 0x7f85ec03f650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- 172.21.2.111:0/670409647 >> 172.21.2.111:0/670409647 conn(0x7f85ec00d0c0 msgr2=0x7f85ec0462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- 172.21.2.111:0/670409647 shutdown_connections 2021-04-03T03:09:15.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- 172.21.2.111:0/670409647 wait complete. 2021-04-03T03:09:15.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 Processor -- start 2021-04-03T03:09:15.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- start start 2021-04-03T03:09:15.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f85ec03d350 0x7f85ec03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec042380 0x7f85ec03c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f85ec046ce0 0x7f85ec0396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0442f0 con 0x7f85ec03d350 2021-04-03T03:09:15.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec046240 con 0x7f85ec042380 2021-04-03T03:09:15.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.739+0000 7f85f8937700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0464e0 con 0x7f85ec046ce0 2021-04-03T03:09:15.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f85ec03d350 0x7f85ec03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f85ec03d350 0x7f85ec03b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50956/0 (socket says 172.21.2.111:50956) 2021-04-03T03:09:15.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 -- 172.21.2.111:0/3796837580 learned_addr learned my addr 172.21.2.111:0/3796837580 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:15.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f37fe700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec042380 0x7f85ec03c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f9db1700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f85ec046ce0 0x7f85ec0396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 -- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f85ec046ce0 msgr2=0x7f85ec0396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f85ec046ce0 0x7f85ec0396b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 -- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec042380 msgr2=0x7f85ec03c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec042380 0x7f85ec03c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 -- 172.21.2.111:0/3796837580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec046780 con 0x7f85ec03d350 2021-04-03T03:09:15.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f3fff700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f85ec03d350 0x7f85ec03b7f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f85ec00c2d0 tx=0x7f85e8007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f0ff9700 1 -- 172.21.2.111:0/3796837580 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e8007240 con 0x7f85ec03d350 2021-04-03T03:09:15.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f0ff9700 1 -- 172.21.2.111:0/3796837580 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e80073a0 con 0x7f85ec03d350 2021-04-03T03:09:15.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f0ff9700 1 -- 172.21.2.111:0/3796837580 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e80066e0 con 0x7f85ec03d350 2021-04-03T03:09:15.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec039ba0 con 0x7f85ec03d350 2021-04-03T03:09:15.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec045c60 con 0x7f85ec03d350 2021-04-03T03:09:15.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec0453d0 con 0x7f85ec03d350 2021-04-03T03:09:15.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f0ff9700 1 -- 172.21.2.111:0/3796837580 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f85e8010070 con 0x7f85ec03d350 2021-04-03T03:09:15.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f0ff9700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f85d4056e90 0x7f85d4059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f0ff9700 1 -- 172.21.2.111:0/3796837580 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f85e8001e40 con 0x7f85ec03d350 2021-04-03T03:09:15.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f37fe700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f85d4056e90 0x7f85d4059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.743+0000 7f85f37fe700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f85d4056e90 0x7f85d4059350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f85e001b000 tx=0x7f85e0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.747+0000 7f85f0ff9700 1 -- 172.21.2.111:0/3796837580 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f85e8044cf0 con 0x7f85ec03d350 2021-04-03T03:09:15.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.871+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f85ec004d90 con 0x7f85ec03d350 2021-04-03T03:09:15.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f0ff9700 1 -- 172.21.2.111:0/3796837580 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f85e8048310 con 0x7f85ec03d350 2021-04-03T03:09:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f85d4056e90 msgr2=0x7f85d4059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f85d4056e90 0x7f85d4059350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f85e001b000 tx=0x7f85e0015000).stop 2021-04-03T03:09:15.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f85ec03d350 msgr2=0x7f85ec03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f85ec03d350 0x7f85ec03b7f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f85ec00c2d0 tx=0x7f85e8007be0).stop 2021-04-03T03:09:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 shutdown_connections 2021-04-03T03:09:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f85ec046ce0 0x7f85ec0396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f85ec03d350 0x7f85ec03b7f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f85d4056e90 0x7f85d4059350 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 --2- 172.21.2.111:0/3796837580 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f85ec042380 0x7f85ec03c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 >> 172.21.2.111:0/3796837580 conn(0x7f85ec00d0c0 msgr2=0x7f85ec041c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 shutdown_connections 2021-04-03T03:09:15.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.879+0000 7f85f8937700 1 -- 172.21.2.111:0/3796837580 wait complete. 2021-04-03T03:09:15.892 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:09:15.896 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:09:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 Processor -- start 2021-04-03T03:09:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- start start 2021-04-03T03:09:15.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7824153440 0x7f7824080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 0x7f78240af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78240b3ca0 con 0x7f78240b38c0 2021-04-03T03:09:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7824025b90 con 0x7f7824153440 2021-04-03T03:09:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7824025e30 con 0x7f7824025480 2021-04-03T03:09:15.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782a7e0700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f78240b38c0 0x7f7824080550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51754/0 (socket says 172.21.2.111:51754) 2021-04-03T03:09:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782a7e0700 1 -- 172.21.2.111:0/235930927 learned_addr learned my addr 172.21.2.111:0/235930927 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f7829fdf700 1 --2- 172.21.2.111:0/235930927 >> v2:172.21.2.111:3300/0 conn(0x7f7824153440 0x7f7824080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f7829fdf700 1 -- 172.21.2.111:0/235930927 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 msgr2=0x7f78240af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f7829fdf700 1 --2- 172.21.2.111:0/235930927 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 0x7f78240af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f7829fdf700 1 -- 172.21.2.111:0/235930927 >> v1:172.21.2.111:6789/0 conn(0x7f78240b38c0 legacy=0x7f7824080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f7829fdf700 1 -- 172.21.2.111:0/235930927 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78240260d0 con 0x7f7824153440 2021-04-03T03:09:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f7829fdf700 1 --2- 172.21.2.111:0/235930927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f7824080c60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f78180010d0 tx=0x7f7820006dd0).ready entity=mon.0 client_cookie=fa84e71d6cb2ba22 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f78297de700 1 -- 172.21.2.111:0/235930927 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78200096b0 con 0x7f7824153440 2021-04-03T03:09:15.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f78297de700 1 -- 172.21.2.111:0/235930927 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820009810 con 0x7f7824153440 2021-04-03T03:09:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f78297de700 1 -- 172.21.2.111:0/235930927 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7820009b70 con 0x7f7824153440 2021-04-03T03:09:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- 172.21.2.111:0/235930927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 msgr2=0x7f7824080c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 --2- 172.21.2.111:0/235930927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f7824080c60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f78180010d0 tx=0x7f7820006dd0).stop 2021-04-03T03:09:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- 172.21.2.111:0/235930927 shutdown_connections 2021-04-03T03:09:15.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 --2- 172.21.2.111:0/235930927 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 0x7f78240af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 --2- 172.21.2.111:0/235930927 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f7824080c60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- 172.21.2.111:0/235930927 >> 172.21.2.111:0/235930927 conn(0x7f7824061610 msgr2=0x7f7824061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- 172.21.2.111:0/235930927 shutdown_connections 2021-04-03T03:09:15.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- 172.21.2.111:0/235930927 wait complete. 2021-04-03T03:09:15.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 Processor -- start 2021-04-03T03:09:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.947+0000 7f782afe1700 1 -- start start 2021-04-03T03:09:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 0x7f7824063f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78240b38c0 0x7f7824158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f78240ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7824064430 con 0x7f7824153440 2021-04-03T03:09:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78240ab070 con 0x7f78240b38c0 2021-04-03T03:09:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78240ad6e0 con 0x7f7824025480 2021-04-03T03:09:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f78240ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f78240ab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50972/0 (socket says 172.21.2.111:50972) 2021-04-03T03:09:15.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 -- 172.21.2.111:0/2815694592 learned_addr learned my addr 172.21.2.111:0/2815694592 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f7829fdf700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78240b38c0 0x7f7824158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 -- 172.21.2.111:0/2815694592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 msgr2=0x7f7824063f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 0x7f7824063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:15.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 -- 172.21.2.111:0/2815694592 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78240b38c0 msgr2=0x7f7824158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78240b38c0 0x7f7824158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 -- 172.21.2.111:0/2815694592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78240aabd0 con 0x7f7824153440 2021-04-03T03:09:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782c45b700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f78240ab5a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f781c0032b0 tx=0x7f781c009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f7816ffd700 1 -- 172.21.2.111:0/2815694592 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f781c00ca60 con 0x7f7824153440 2021-04-03T03:09:15.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f7816ffd700 1 -- 172.21.2.111:0/2815694592 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c00cbc0 con 0x7f7824153440 2021-04-03T03:09:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7824159b00 con 0x7f7824153440 2021-04-03T03:09:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7824159230 con 0x7f7824153440 2021-04-03T03:09:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f7816ffd700 1 -- 172.21.2.111:0/2815694592 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f781c015420 con 0x7f7824153440 2021-04-03T03:09:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78240aad60 con 0x7f7824153440 2021-04-03T03:09:15.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f7816ffd700 1 -- 172.21.2.111:0/2815694592 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f781c01c070 con 0x7f7824153440 2021-04-03T03:09:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f7816ffd700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f780c057030 0x7f780c0594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f7816ffd700 1 -- 172.21.2.111:0/2815694592 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f781c06fe80 con 0x7f7824153440 2021-04-03T03:09:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782a7e0700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f780c057030 0x7f780c0594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.951+0000 7f782a7e0700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f780c057030 0x7f780c0594f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7818001190 tx=0x7f7818007770).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:15.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:15.955+0000 7f7816ffd700 1 -- 172.21.2.111:0/2815694592 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f781c045160 con 0x7f7824153440 2021-04-03T03:09:16.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.079+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7824057660 con 0x7f7824153440 2021-04-03T03:09:16.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.083+0000 7f7816ffd700 1 -- 172.21.2.111:0/2815694592 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f781c048780 con 0x7f7824153440 2021-04-03T03:09:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f780c057030 msgr2=0x7f780c0594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f780c057030 0x7f780c0594f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7818001190 tx=0x7f7818007770).stop 2021-04-03T03:09:16.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 msgr2=0x7f78240ab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f78240ab5a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f781c0032b0 tx=0x7f781c009220).stop 2021-04-03T03:09:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 shutdown_connections 2021-04-03T03:09:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7824153440 0x7f78240ab5a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7824025480 0x7f7824063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f780c057030 0x7f780c0594f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 --2- 172.21.2.111:0/2815694592 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78240b38c0 0x7f7824158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 >> 172.21.2.111:0/2815694592 conn(0x7f7824061610 msgr2=0x7f782415a000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 shutdown_connections 2021-04-03T03:09:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.087+0000 7f782afe1700 1 -- 172.21.2.111:0/2815694592 wait complete. 2021-04-03T03:09:16.097 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:09:16.102 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198db13700 1 Processor -- start 2021-04-03T03:09:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198db13700 1 -- start start 2021-04-03T03:09:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198db13700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f19880b8350 0x7f19880b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198db13700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f19880b3a90 0x7f19881446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198db13700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19880b24d0 con 0x7f19880b30c0 2021-04-03T03:09:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198db13700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1988142090 con 0x7f19880b3a90 2021-04-03T03:09:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198db13700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880b8730 con 0x7f19880b8350 2021-04-03T03:09:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f1987fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f19880b3a90 0x7f19881446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f1987fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f19880b3a90 0x7f19881446f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50976/0 (socket says 172.21.2.111:50976) 2021-04-03T03:09:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f1987fff700 1 -- 172.21.2.111:0/4186430235 learned_addr learned my addr 172.21.2.111:0/4186430235 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.147+0000 7f198d312700 1 --1- 172.21.2.111:0/4186430235 >> v1:172.21.2.111:6789/0 conn(0x7f19880b30c0 0x7f1988144be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51772/0 (socket says 172.21.2.111:51772) 2021-04-03T03:09:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198cb11700 1 --2- 172.21.2.111:0/4186430235 >> v2:172.21.2.115:3300/0 conn(0x7f19880b8350 0x7f19880b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1987fff700 1 -- 172.21.2.111:0/4186430235 >> v2:172.21.2.115:3300/0 conn(0x7f19880b8350 msgr2=0x7f19880b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1987fff700 1 --2- 172.21.2.111:0/4186430235 >> v2:172.21.2.115:3300/0 conn(0x7f19880b8350 0x7f19880b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1987fff700 1 -- 172.21.2.111:0/4186430235 >> v1:172.21.2.111:6789/0 conn(0x7f19880b30c0 legacy=0x7f1988144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f19877fe700 1 -- 172.21.2.111:0/4186430235 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618925832 0 0) 0x7f19880b24d0 con 0x7f19880b30c0 2021-04-03T03:09:16.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1987fff700 1 -- 172.21.2.111:0/4186430235 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19881413c0 con 0x7f19880b3a90 2021-04-03T03:09:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198cb11700 1 --2- 172.21.2.111:0/4186430235 >> v2:172.21.2.115:3300/0 conn(0x7f19880b8350 0x7f19880b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:09:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1987fff700 1 --2- 172.21.2.111:0/4186430235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b3a90 0x7f19881446f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f19800032b0 tx=0x7f1980009270).ready entity=mon.0 client_cookie=377645ae0dd8ea7c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f19877fe700 1 -- 172.21.2.111:0/4186430235 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f198000bc00 con 0x7f19880b3a90 2021-04-03T03:09:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f19877fe700 1 -- 172.21.2.111:0/4186430235 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198000bd60 con 0x7f19880b3a90 2021-04-03T03:09:16.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f19877fe700 1 -- 172.21.2.111:0/4186430235 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1980011400 con 0x7f19880b3a90 2021-04-03T03:09:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/4186430235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b3a90 msgr2=0x7f19881446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 --2- 172.21.2.111:0/4186430235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b3a90 0x7f19881446f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f19800032b0 tx=0x7f1980009270).stop 2021-04-03T03:09:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/4186430235 shutdown_connections 2021-04-03T03:09:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 --2- 172.21.2.111:0/4186430235 >> v2:172.21.2.115:3300/0 conn(0x7f19880b8350 0x7f19880b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 --2- 172.21.2.111:0/4186430235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b3a90 0x7f19881446f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/4186430235 >> 172.21.2.111:0/4186430235 conn(0x7f1988060490 msgr2=0x7f19880b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/4186430235 shutdown_connections 2021-04-03T03:09:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/4186430235 wait complete. 2021-04-03T03:09:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 Processor -- start 2021-04-03T03:09:16.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- start start 2021-04-03T03:09:16.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19880b30c0 0x7f1988129db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19880b3a90 0x7f19881446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b8350 0x7f198813c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198813e5c0 con 0x7f19880b8350 2021-04-03T03:09:16.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988142f70 con 0x7f19880b30c0 2021-04-03T03:09:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198812a2a0 con 0x7f19880b3a90 2021-04-03T03:09:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b8350 0x7f198813c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b8350 0x7f198813c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50984/0 (socket says 172.21.2.111:50984) 2021-04-03T03:09:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 -- 172.21.2.111:0/2469092025 learned_addr learned my addr 172.21.2.111:0/2469092025 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:16.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198cb11700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19880b30c0 0x7f1988129db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 -- 172.21.2.111:0/2469092025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19880b3a90 msgr2=0x7f19881446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19880b3a90 0x7f19881446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 -- 172.21.2.111:0/2469092025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19880b30c0 msgr2=0x7f1988129db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19880b30c0 0x7f1988129db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 -- 172.21.2.111:0/2469092025 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988143970 con 0x7f19880b8350 2021-04-03T03:09:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198d312700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b8350 0x7f198813c820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f1978000b80 tx=0x7f19780079c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1985ffb700 1 -- 172.21.2.111:0/2469092025 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19780072e0 con 0x7f19880b8350 2021-04-03T03:09:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1985ffb700 1 -- 172.21.2.111:0/2469092025 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978005530 con 0x7f19880b8350 2021-04-03T03:09:16.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19881429f0 con 0x7f19880b8350 2021-04-03T03:09:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1988144d30 con 0x7f19880b8350 2021-04-03T03:09:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1985ffb700 1 -- 172.21.2.111:0/2469092025 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1978005890 con 0x7f19880b8350 2021-04-03T03:09:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19881430e0 con 0x7f19880b8350 2021-04-03T03:09:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.151+0000 7f1985ffb700 1 -- 172.21.2.111:0/2469092025 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f197801a060 con 0x7f19880b8350 2021-04-03T03:09:16.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.155+0000 7f1985ffb700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19680570a0 0x7f1968059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.155+0000 7f1985ffb700 1 -- 172.21.2.111:0/2469092025 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f197806fa00 con 0x7f19880b8350 2021-04-03T03:09:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.155+0000 7f198cb11700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19680570a0 0x7f1968059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.155+0000 7f198cb11700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19680570a0 0x7f1968059560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f197c008000 tx=0x7f197c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.155+0000 7f1985ffb700 1 -- 172.21.2.111:0/2469092025 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f19881430e0 con 0x7f19880b8350 2021-04-03T03:09:16.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.279+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19881430e0 con 0x7f19880b8350 2021-04-03T03:09:16.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f1985ffb700 1 -- 172.21.2.111:0/2469092025 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f19881430e0 con 0x7f19880b8350 2021-04-03T03:09:16.290 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:16.290 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:09:15.086110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1150523943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1150523943},{"type":"v1","addr":"172.21.2.111:6835","nonce":1150523943}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:09:16.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19680570a0 msgr2=0x7f1968059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19680570a0 0x7f1968059560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f197c008000 tx=0x7f197c008040).stop 2021-04-03T03:09:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b8350 msgr2=0x7f198813c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b8350 0x7f198813c820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f1978000b80 tx=0x7f19780079c0).stop 2021-04-03T03:09:16.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 shutdown_connections 2021-04-03T03:09:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19880b8350 0x7f198813c820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19680570a0 0x7f1968059560 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19880b30c0 0x7f1988129db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 --2- 172.21.2.111:0/2469092025 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19880b3a90 0x7f19881446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 >> 172.21.2.111:0/2469092025 conn(0x7f1988060490 msgr2=0x7f19880b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 shutdown_connections 2021-04-03T03:09:16.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.283+0000 7f198db13700 1 -- 172.21.2.111:0/2469092025 wait complete. 2021-04-03T03:09:16.294 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 11 2021-04-03T03:09:16.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:09:14.082969+0000', 'modified': '2021-04-03T03:09:15.086110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4881}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4881': {'gid': 4881, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1150523943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1150523943}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1150523943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:09:16.500 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:09:16.500 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:09:16.571 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":4881,"want_state":"up:active","state":"up:active","rank_uptime":2.4769863010000002,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":7.6267202159999998} 2021-04-03T03:09:16.572 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 4881, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 2.476986301, "state": "up:active", "uptime": 7.626720216, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:09:16.572 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:09:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 Processor -- start 2021-04-03T03:09:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- start start 2021-04-03T03:09:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f127014b700 0x7f12700b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700af400 con 0x7f12700b0c90 2021-04-03T03:09:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700af6a0 con 0x7f12700603f0 2021-04-03T03:09:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1270060860 con 0x7f127014b700 2021-04-03T03:09:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12750ef700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f127014b700 0x7f12700b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12750ef700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f127014b700 0x7f12700b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:50992/0 (socket says 172.21.2.111:50992) 2021-04-03T03:09:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12760f1700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f12700b0c90 0x7f1270147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51784/0 (socket says 172.21.2.111:51784) 2021-04-03T03:09:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12750ef700 1 -- 172.21.2.111:0/2432613933 learned_addr learned my addr 172.21.2.111:0/2432613933 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12750ef700 1 -- 172.21.2.111:0/2432613933 >> v1:172.21.2.115:6789/0 conn(0x7f12700603f0 legacy=0x7f12700b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12750ef700 1 -- 172.21.2.111:0/2432613933 >> v1:172.21.2.111:6789/0 conn(0x7f12700b0c90 legacy=0x7f1270147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12750ef700 1 -- 172.21.2.111:0/2432613933 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700afb40 con 0x7f127014b700 2021-04-03T03:09:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12750ef700 1 --2- 172.21.2.111:0/2432613933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700b07a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f12700620f0 tx=0x7f126c006e90).ready entity=mon.0 client_cookie=7e520e2480f68dde server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12748ee700 1 -- 172.21.2.111:0/2432613933 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f126c009980 con 0x7f127014b700 2021-04-03T03:09:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12748ee700 1 -- 172.21.2.111:0/2432613933 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c009ae0 con 0x7f127014b700 2021-04-03T03:09:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12748ee700 1 -- 172.21.2.111:0/2432613933 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f126c009e80 con 0x7f127014b700 2021-04-03T03:09:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- 172.21.2.111:0/2432613933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 msgr2=0x7f12700b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 --2- 172.21.2.111:0/2432613933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700b07a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f12700620f0 tx=0x7f126c006e90).stop 2021-04-03T03:09:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- 172.21.2.111:0/2432613933 shutdown_connections 2021-04-03T03:09:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 --2- 172.21.2.111:0/2432613933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700b07a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- 172.21.2.111:0/2432613933 >> 172.21.2.111:0/2432613933 conn(0x7f12700b61c0 msgr2=0x7f127005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- 172.21.2.111:0/2432613933 shutdown_connections 2021-04-03T03:09:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.619+0000 7f12768f2700 1 -- 172.21.2.111:0/2432613933 wait complete. 2021-04-03T03:09:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 Processor -- start 2021-04-03T03:09:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 -- start start 2021-04-03T03:09:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12700603f0 0x7f12700a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12700b0c90 0x7f1270159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127015a360 con 0x7f127014b700 2021-04-03T03:09:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270067fe0 con 0x7f12700b0c90 2021-04-03T03:09:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1270067d40 con 0x7f12700603f0 2021-04-03T03:09:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51000/0 (socket says 172.21.2.111:51000) 2021-04-03T03:09:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 -- 172.21.2.111:0/711578849 learned_addr learned my addr 172.21.2.111:0/711578849 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 -- 172.21.2.111:0/711578849 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12700603f0 msgr2=0x7f12700a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12700603f0 0x7f12700a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 -- 172.21.2.111:0/711578849 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12700b0c90 msgr2=0x7f1270159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12700b0c90 0x7f1270159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 -- 172.21.2.111:0/711578849 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127015a550 con 0x7f127014b700 2021-04-03T03:09:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12760f1700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700aaff0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f127006a3b0 tx=0x7f1268007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f1262ffd700 1 -- 172.21.2.111:0/711578849 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12680020b0 con 0x7f127014b700 2021-04-03T03:09:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f1262ffd700 1 -- 172.21.2.111:0/711578849 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1268005ac0 con 0x7f127014b700 2021-04-03T03:09:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f1262ffd700 1 -- 172.21.2.111:0/711578849 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1268005e20 con 0x7f127014b700 2021-04-03T03:09:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12700a9290 con 0x7f127014b700 2021-04-03T03:09:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12700ab4e0 con 0x7f127014b700 2021-04-03T03:09:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f1262ffd700 1 -- 172.21.2.111:0/711578849 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1268010030 con 0x7f127014b700 2021-04-03T03:09:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127015a6e0 con 0x7f127014b700 2021-04-03T03:09:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f1262ffd700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1250056ea0 0x7f1250059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f1262ffd700 1 -- 172.21.2.111:0/711578849 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1268070180 con 0x7f127014b700 2021-04-03T03:09:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12758f0700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1250056ea0 0x7f1250059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.623+0000 7f12758f0700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1250056ea0 0x7f1250059360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f1264001f60 tx=0x7f12640077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.627+0000 7f1262ffd700 1 -- 172.21.2.111:0/711578849 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1268045570 con 0x7f127014b700 2021-04-03T03:09:16.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.719+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1270057660 con 0x7f127014b700 2021-04-03T03:09:16.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.719+0000 7f1262ffd700 1 -- 172.21.2.111:0/711578849 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f1268048b90 con 0x7f127014b700 2021-04-03T03:09:16.729 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:16.729 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:09:15.086914+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:09:12.356149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-04-03T03:09:13.051346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:09:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1250056ea0 msgr2=0x7f1250059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1250056ea0 0x7f1250059360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f1264001f60 tx=0x7f12640077d0).stop 2021-04-03T03:09:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 msgr2=0x7f12700aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700aaff0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f127006a3b0 tx=0x7f1268007ee0).stop 2021-04-03T03:09:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 shutdown_connections 2021-04-03T03:09:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127014b700 0x7f12700aaff0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12700603f0 0x7f12700a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1250056ea0 0x7f1250059360 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 --2- 172.21.2.111:0/711578849 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12700b0c90 0x7f1270159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 >> 172.21.2.111:0/711578849 conn(0x7f12700b61c0 msgr2=0x7f12700b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 shutdown_connections 2021-04-03T03:09:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.723+0000 7f12768f2700 1 -- 172.21.2.111:0/711578849 wait complete. 2021-04-03T03:09:16.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:09:12.356149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:09:13.051346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:09:16.738 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:09:16.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 Processor -- start 2021-04-03T03:09:16.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 -- start start 2021-04-03T03:09:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 0x7f473c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f473c0add50 0x7f473c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473c0607d0 con 0x7f473c0b17b0 2021-04-03T03:09:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c1451c0 con 0x7f473c0add50 2021-04-03T03:09:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c14b090 con 0x7f473c0603f0 2021-04-03T03:09:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473affd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f473c0add50 0x7f473c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473bfff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f473c0b17b0 0x7f473c147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51796/0 (socket says 172.21.2.111:51796) 2021-04-03T03:09:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473bfff700 1 -- 172.21.2.111:0/2053636354 learned_addr learned my addr 172.21.2.111:0/2053636354 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473affd700 1 -- 172.21.2.111:0/2053636354 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 msgr2=0x7f473c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473affd700 1 --2- 172.21.2.111:0/2053636354 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 0x7f473c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473affd700 1 -- 172.21.2.111:0/2053636354 >> v1:172.21.2.111:6789/0 conn(0x7f473c0b17b0 legacy=0x7f473c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473affd700 1 -- 172.21.2.111:0/2053636354 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c14b330 con 0x7f473c0add50 2021-04-03T03:09:16.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473affd700 1 --2- 172.21.2.111:0/2053636354 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0add50 0x7f473c144cd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f4724002f30 tx=0x7f4724006dd0).ready entity=mon.0 client_cookie=1a5e8576fd9ed619 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473a7fc700 1 -- 172.21.2.111:0/2053636354 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47240097f0 con 0x7f473c0add50 2021-04-03T03:09:16.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473a7fc700 1 -- 172.21.2.111:0/2053636354 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4724009950 con 0x7f473c0add50 2021-04-03T03:09:16.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f473a7fc700 1 -- 172.21.2.111:0/2053636354 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4724009cb0 con 0x7f473c0add50 2021-04-03T03:09:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 -- 172.21.2.111:0/2053636354 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0add50 msgr2=0x7f473c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.791+0000 7f4740b34700 1 --2- 172.21.2.111:0/2053636354 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0add50 0x7f473c144cd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f4724002f30 tx=0x7f4724006dd0).stop 2021-04-03T03:09:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- 172.21.2.111:0/2053636354 shutdown_connections 2021-04-03T03:09:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 --2- 172.21.2.111:0/2053636354 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 0x7f473c1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 --2- 172.21.2.111:0/2053636354 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0add50 0x7f473c144cd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- 172.21.2.111:0/2053636354 >> 172.21.2.111:0/2053636354 conn(0x7f473c0b61c0 msgr2=0x7f473c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- 172.21.2.111:0/2053636354 shutdown_connections 2021-04-03T03:09:16.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- 172.21.2.111:0/2053636354 wait complete. 2021-04-03T03:09:16.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 Processor -- start 2021-04-03T03:09:16.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- start start 2021-04-03T03:09:16.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 0x7f473c14e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f473c0add50 0x7f473c1462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0b17b0 0x7f473c146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0ab8a0 con 0x7f473c0b17b0 2021-04-03T03:09:16.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0abb00 con 0x7f473c0add50 2021-04-03T03:09:16.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0aaf00 con 0x7f473c0603f0 2021-04-03T03:09:16.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0b17b0 0x7f473c146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0b17b0 0x7f473c146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51008/0 (socket says 172.21.2.111:51008) 2021-04-03T03:09:16.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 -- 172.21.2.111:0/3583693553 learned_addr learned my addr 172.21.2.111:0/3583693553 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:16.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 -- 172.21.2.111:0/3583693553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 msgr2=0x7f473c14e9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:16.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 0x7f473c14e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 -- 172.21.2.111:0/3583693553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f473c0add50 msgr2=0x7f473c1462a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:09:16.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f473c0add50 0x7f473c1462a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 -- 172.21.2.111:0/3583693553 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0abe20 con 0x7f473c0b17b0 2021-04-03T03:09:16.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473bfff700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0b17b0 0x7f473c146790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f473c0b5600 tx=0x7f4734007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4738ff9700 1 -- 172.21.2.111:0/3583693553 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4734002040 con 0x7f473c0b17b0 2021-04-03T03:09:16.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4738ff9700 1 -- 172.21.2.111:0/3583693553 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4734005b80 con 0x7f473c0b17b0 2021-04-03T03:09:16.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c0a7430 con 0x7f473c0b17b0 2021-04-03T03:09:16.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c0a70b0 con 0x7f473c0b17b0 2021-04-03T03:09:16.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4738ff9700 1 -- 172.21.2.111:0/3583693553 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4734007110 con 0x7f473c0b17b0 2021-04-03T03:09:16.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c0ab0c0 con 0x7f473c0b17b0 2021-04-03T03:09:16.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4738ff9700 1 -- 172.21.2.111:0/3583693553 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4734010030 con 0x7f473c0b17b0 2021-04-03T03:09:16.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4738ff9700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4720056ef0 0x7f47200593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f4738ff9700 1 -- 172.21.2.111:0/3583693553 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f4734070090 con 0x7f473c0b17b0 2021-04-03T03:09:16.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473b7fe700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4720056ef0 0x7f47200593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.795+0000 7f473b7fe700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4720056ef0 0x7f47200593b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f472c002f30 tx=0x7f472c0069e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.799+0000 7f4738ff9700 1 -- 172.21.2.111:0/3583693553 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4734045480 con 0x7f473c0b17b0 2021-04-03T03:09:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f473c0a7710 con 0x7f473c0b17b0 2021-04-03T03:09:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4738ff9700 1 -- 172.21.2.111:0/3583693553 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v33) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4734048aa0 con 0x7f473c0b17b0 2021-04-03T03:09:16.900 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:09:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4720056ef0 msgr2=0x7f47200593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4720056ef0 0x7f47200593b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f472c002f30 tx=0x7f472c0069e0).stop 2021-04-03T03:09:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0b17b0 msgr2=0x7f473c146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0b17b0 0x7f473c146790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f473c0b5600 tx=0x7f4734007f40).stop 2021-04-03T03:09:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 shutdown_connections 2021-04-03T03:09:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f473c0b17b0 0x7f473c146790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f473c0603f0 0x7f473c14e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4720056ef0 0x7f47200593b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 --2- 172.21.2.111:0/3583693553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f473c0add50 0x7f473c1462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 >> 172.21.2.111:0/3583693553 conn(0x7f473c0b61c0 msgr2=0x7f473c1447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 shutdown_connections 2021-04-03T03:09:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.891+0000 7f4740b34700 1 -- 172.21.2.111:0/3583693553 wait complete. 2021-04-03T03:09:16.907 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:09:16.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962184e700 1 Processor -- start 2021-04-03T03:09:16.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962184e700 1 -- start start 2021-04-03T03:09:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962184e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f96140dbba0 0x7f96140dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962184e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962184e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140dc560 con 0x7f9614041060 2021-04-03T03:09:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962184e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961403de60 con 0x7f96140dbba0 2021-04-03T03:09:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962184e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140dc800 con 0x7f961403d560 2021-04-03T03:09:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962084c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f96140dbba0 0x7f96140dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962084c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f96140dbba0 0x7f96140dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51016/0 (socket says 172.21.2.111:51016) 2021-04-03T03:09:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.955+0000 7f962084c700 1 -- 172.21.2.111:0/3235567725 learned_addr learned my addr 172.21.2.111:0/3235567725 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 --2- 172.21.2.111:0/3235567725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962084c700 1 -- 172.21.2.111:0/3235567725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 msgr2=0x7f9614041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962084c700 1 --2- 172.21.2.111:0/3235567725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962084c700 1 -- 172.21.2.111:0/3235567725 >> v1:172.21.2.115:6789/0 conn(0x7f9614041060 legacy=0x7f961403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962084c700 1 -- 172.21.2.111:0/3235567725 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961403da30 con 0x7f96140dbba0 2021-04-03T03:09:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962084c700 1 --2- 172.21.2.111:0/3235567725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140dbf80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f96140087d0 tx=0x7f9610006e90).ready entity=mon.0 client_cookie=5f9fda78f1027aa9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f961b7fe700 1 -- 172.21.2.111:0/3235567725 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96100098e0 con 0x7f96140dbba0 2021-04-03T03:09:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f961b7fe700 1 -- 172.21.2.111:0/3235567725 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9610009a40 con 0x7f96140dbba0 2021-04-03T03:09:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f961b7fe700 1 -- 172.21.2.111:0/3235567725 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9610009de0 con 0x7f96140dbba0 2021-04-03T03:09:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- 172.21.2.111:0/3235567725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 msgr2=0x7f96140dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 --2- 172.21.2.111:0/3235567725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140dbf80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f96140087d0 tx=0x7f9610006e90).stop 2021-04-03T03:09:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- 172.21.2.111:0/3235567725 shutdown_connections 2021-04-03T03:09:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 --2- 172.21.2.111:0/3235567725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 --2- 172.21.2.111:0/3235567725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140dbf80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- 172.21.2.111:0/3235567725 >> 172.21.2.111:0/3235567725 conn(0x7f96140462b0 msgr2=0x7f96140466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- 172.21.2.111:0/3235567725 shutdown_connections 2021-04-03T03:09:16.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- 172.21.2.111:0/3235567725 wait complete. 2021-04-03T03:09:16.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 Processor -- start 2021-04-03T03:09:16.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- start start 2021-04-03T03:09:16.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f961403d560 0x7f9614044a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9614041060 0x7f96140d7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140d7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140df410 con 0x7f96140dbba0 2021-04-03T03:09:16.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961409fa30 con 0x7f961403d560 2021-04-03T03:09:16.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140df170 con 0x7f9614041060 2021-04-03T03:09:16.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140d7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51026/0 (socket says 172.21.2.111:51026) 2021-04-03T03:09:16.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 -- 172.21.2.111:0/2765171011 learned_addr learned my addr 172.21.2.111:0/2765171011 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:16.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 -- 172.21.2.111:0/2765171011 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9614041060 msgr2=0x7f96140d7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9614041060 0x7f96140d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 -- 172.21.2.111:0/2765171011 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f961403d560 msgr2=0x7f9614044a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:16.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f961403d560 0x7f9614044a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:16.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 -- 172.21.2.111:0/2765171011 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140de0c0 con 0x7f96140dbba0 2021-04-03T03:09:16.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962104d700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140d7860 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f961c063000 tx=0x7f961c064040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f9619ffb700 1 -- 172.21.2.111:0/2765171011 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c05df60 con 0x7f96140dbba0 2021-04-03T03:09:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f9619ffb700 1 -- 172.21.2.111:0/2765171011 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c0699f0 con 0x7f96140dbba0 2021-04-03T03:09:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961409f6b0 con 0x7f96140dbba0 2021-04-03T03:09:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96140df740 con 0x7f96140dbba0 2021-04-03T03:09:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.959+0000 7f9619ffb700 1 -- 172.21.2.111:0/2765171011 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c05f4b0 con 0x7f96140dbba0 2021-04-03T03:09:16.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.963+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614004db0 con 0x7f96140dbba0 2021-04-03T03:09:16.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.963+0000 7f9619ffb700 1 -- 172.21.2.111:0/2765171011 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f961c08c030 con 0x7f96140dbba0 2021-04-03T03:09:16.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.963+0000 7f9619ffb700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056fb0 0x7f95fc059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:16.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.963+0000 7f9619ffb700 1 -- 172.21.2.111:0/2765171011 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f961c0df0d0 con 0x7f96140dbba0 2021-04-03T03:09:16.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.963+0000 7f962084c700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056fb0 0x7f95fc059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:16.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.963+0000 7f962084c700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056fb0 0x7f95fc059470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f96140087d0 tx=0x7f9610015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:16.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:16.963+0000 7f9619ffb700 1 -- 172.21.2.111:0/2765171011 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f961c0b4460 con 0x7f96140dbba0 2021-04-03T03:09:17.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f96140083c0 con 0x7f96140dbba0 2021-04-03T03:09:17.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f9619ffb700 1 -- 172.21.2.111:0/2765171011 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f961c08a070 con 0x7f96140dbba0 2021-04-03T03:09:17.065 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:17.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056fb0 msgr2=0x7f95fc059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056fb0 0x7f95fc059470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f96140087d0 tx=0x7f9610015000).stop 2021-04-03T03:09:17.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 msgr2=0x7f96140d7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140d7860 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f961c063000 tx=0x7f961c064040).stop 2021-04-03T03:09:17.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 shutdown_connections 2021-04-03T03:09:17.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96140dbba0 0x7f96140d7860 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056fb0 0x7f95fc059470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f961403d560 0x7f9614044a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 --2- 172.21.2.111:0/2765171011 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9614041060 0x7f96140d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 >> 172.21.2.111:0/2765171011 conn(0x7f96140462b0 msgr2=0x7f961403f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:17.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 shutdown_connections 2021-04-03T03:09:17.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.055+0000 7f962184e700 1 -- 172.21.2.111:0/2765171011 wait complete. 2021-04-03T03:09:17.072 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:09:17.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.119+0000 7f6b7f30e700 1 Processor -- start 2021-04-03T03:09:17.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.119+0000 7f6b7f30e700 1 -- start start 2021-04-03T03:09:17.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b70007b00 0x7f6b70008fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b700492d0 con 0x7f6b70048ef0 2021-04-03T03:09:17.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b70043430 con 0x7f6b7003f560 2021-04-03T03:09:17.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b700d7fb0 con 0x7f6b70007b00 2021-04-03T03:09:17.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b70007b00 0x7f6b70008fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7e30c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6b70048ef0 0x7f6b70044810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51822/0 (socket says 172.21.2.111:51822) 2021-04-03T03:09:17.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7e30c700 1 -- 172.21.2.111:0/2465802868 learned_addr learned my addr 172.21.2.111:0/2465802868 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:17.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 -- 172.21.2.111:0/2465802868 >> v1:172.21.2.115:6789/0 conn(0x7f6b7003f560 legacy=0x7f6b7003e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:17.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 -- 172.21.2.111:0/2465802868 >> v1:172.21.2.111:6789/0 conn(0x7f6b70048ef0 legacy=0x7f6b70044810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 -- 172.21.2.111:0/2465802868 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700d8250 con 0x7f6b70007b00 2021-04-03T03:09:17.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 --2- 172.21.2.111:0/2465802868 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70007b00 0x7f6b70008fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f6b70042bc0 tx=0x7f6b7805ad70).ready entity=mon.0 client_cookie=cf854a7d31abf85 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7d30a700 1 -- 172.21.2.111:0/2465802868 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7805bda0 con 0x7f6b70007b00 2021-04-03T03:09:17.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7d30a700 1 -- 172.21.2.111:0/2465802868 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7805df60 con 0x7f6b70007b00 2021-04-03T03:09:17.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7d30a700 1 -- 172.21.2.111:0/2465802868 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b780699f0 con 0x7f6b70007b00 2021-04-03T03:09:17.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- 172.21.2.111:0/2465802868 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70007b00 msgr2=0x7f6b70008fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/2465802868 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70007b00 0x7f6b70008fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f6b70042bc0 tx=0x7f6b7805ad70).stop 2021-04-03T03:09:17.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- 172.21.2.111:0/2465802868 shutdown_connections 2021-04-03T03:09:17.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/2465802868 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70007b00 0x7f6b70008fa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- 172.21.2.111:0/2465802868 >> 172.21.2.111:0/2465802868 conn(0x7f6b7000f2d0 msgr2=0x7f6b700484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:17.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- 172.21.2.111:0/2465802868 shutdown_connections 2021-04-03T03:09:17.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- 172.21.2.111:0/2465802868 wait complete. 2021-04-03T03:09:17.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 Processor -- start 2021-04-03T03:09:17.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- start start 2021-04-03T03:09:17.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b70007b00 0x7f6b700d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b7003f560 0x7f6b700d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70048ef0 0x7f6b700cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700ded20 con 0x7f6b70048ef0 2021-04-03T03:09:17.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700d26d0 con 0x7f6b7003f560 2021-04-03T03:09:17.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7f30e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700d2970 con 0x7f6b70007b00 2021-04-03T03:09:17.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70048ef0 0x7f6b700cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70048ef0 0x7f6b700cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51038/0 (socket says 172.21.2.111:51038) 2021-04-03T03:09:17.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 -- 172.21.2.111:0/596487255 learned_addr learned my addr 172.21.2.111:0/596487255 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:17.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 -- 172.21.2.111:0/596487255 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b70007b00 msgr2=0x7f6b700d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b70007b00 0x7f6b700d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 -- 172.21.2.111:0/596487255 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b7003f560 msgr2=0x7f6b700d98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b7003f560 0x7f6b700d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 -- 172.21.2.111:0/596487255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700d2c10 con 0x7f6b70048ef0 2021-04-03T03:09:17.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.123+0000 7f6b7eb0d700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70048ef0 0x7f6b700cccc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f6b70042bc0 tx=0x7f6b78064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b6b7fe700 1 -- 172.21.2.111:0/596487255 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b780441b0 con 0x7f6b70048ef0 2021-04-03T03:09:17.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b6b7fe700 1 -- 172.21.2.111:0/596487255 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7805f2c0 con 0x7f6b70048ef0 2021-04-03T03:09:17.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b700d6f80 con 0x7f6b70048ef0 2021-04-03T03:09:17.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b700cd1b0 con 0x7f6b70048ef0 2021-04-03T03:09:17.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b6b7fe700 1 -- 172.21.2.111:0/596487255 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7805f620 con 0x7f6b70048ef0 2021-04-03T03:09:17.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b700cd7d0 con 0x7f6b70048ef0 2021-04-03T03:09:17.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b6b7fe700 1 -- 172.21.2.111:0/596487255 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6b7808c020 con 0x7f6b70048ef0 2021-04-03T03:09:17.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b6b7fe700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b58056f70 0x7f6b58059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b6b7fe700 1 -- 172.21.2.111:0/596487255 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f6b780e0d70 con 0x7f6b70048ef0 2021-04-03T03:09:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b7e30c700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b58056f70 0x7f6b58059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b7e30c700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b58056f70 0x7f6b58059430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6b70049520 tx=0x7f6b740077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.127+0000 7f6b6b7fe700 1 -- 172.21.2.111:0/596487255 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6b700cd7d0 con 0x7f6b70048ef0 2021-04-03T03:09:17.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b700cd7d0 con 0x7f6b70048ef0 2021-04-03T03:09:17.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b6b7fe700 1 -- 172.21.2.111:0/596487255 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6b700cd7d0 con 0x7f6b70048ef0 2021-04-03T03:09:17.232 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:17.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b58056f70 msgr2=0x7f6b58059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b58056f70 0x7f6b58059430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6b70049520 tx=0x7f6b740077d0).stop 2021-04-03T03:09:17.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70048ef0 msgr2=0x7f6b700cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70048ef0 0x7f6b700cccc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f6b70042bc0 tx=0x7f6b78064030).stop 2021-04-03T03:09:17.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 shutdown_connections 2021-04-03T03:09:17.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b70048ef0 0x7f6b700cccc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b70007b00 0x7f6b700d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b58056f70 0x7f6b58059430 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 --2- 172.21.2.111:0/596487255 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b7003f560 0x7f6b700d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 >> 172.21.2.111:0/596487255 conn(0x7f6b7000f2d0 msgr2=0x7f6b700472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:17.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 shutdown_connections 2021-04-03T03:09:17.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.223+0000 7f6b7f30e700 1 -- 172.21.2.111:0/596487255 wait complete. 2021-04-03T03:09:17.239 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:09:17.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.287+0000 7fa223fff700 1 Processor -- start 2021-04-03T03:09:17.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.287+0000 7fa223fff700 1 -- start start 2021-04-03T03:09:17.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 0x7fa224147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa224144ef0 0x7fa224147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2240607d0 con 0x7fa2240603f0 2021-04-03T03:09:17.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22414b090 con 0x7fa224144ef0 2021-04-03T03:09:17.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22414b330 con 0x7fa2240add50 2021-04-03T03:09:17.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa224144ef0 0x7fa224147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa224144ef0 0x7fa224147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51048/0 (socket says 172.21.2.111:51048) 2021-04-03T03:09:17.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 -- 172.21.2.111:0/3749929232 learned_addr learned my addr 172.21.2.111:0/3749929232 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:17.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 -- 172.21.2.111:0/3749929232 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 msgr2=0x7fa224147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:17.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 --2- 172.21.2.111:0/3749929232 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 0x7fa224147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 -- 172.21.2.111:0/3749929232 >> v1:172.21.2.115:6789/0 conn(0x7fa2240603f0 legacy=0x7fa2241447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:17.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 -- 172.21.2.111:0/3749929232 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22414b5d0 con 0x7fa224144ef0 2021-04-03T03:09:17.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa2237fe700 1 --2- 172.21.2.111:0/3749929232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa224144ef0 0x7fa224147c30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fa2240b5600 tx=0x7fa21c009270).ready entity=mon.0 client_cookie=766380b523a19500 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa221ffb700 1 -- 172.21.2.111:0/3749929232 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa21c00bbc0 con 0x7fa224144ef0 2021-04-03T03:09:17.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa221ffb700 1 -- 172.21.2.111:0/3749929232 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c00bd20 con 0x7fa224144ef0 2021-04-03T03:09:17.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa221ffb700 1 -- 172.21.2.111:0/3749929232 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa21c011400 con 0x7fa224144ef0 2021-04-03T03:09:17.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- 172.21.2.111:0/3749929232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa224144ef0 msgr2=0x7fa224147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- 172.21.2.111:0/3749929232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa224144ef0 0x7fa224147c30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fa2240b5600 tx=0x7fa21c009270).stop 2021-04-03T03:09:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- 172.21.2.111:0/3749929232 shutdown_connections 2021-04-03T03:09:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- 172.21.2.111:0/3749929232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa224144ef0 0x7fa224147c30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- 172.21.2.111:0/3749929232 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 0x7fa224147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- 172.21.2.111:0/3749929232 >> 172.21.2.111:0/3749929232 conn(0x7fa2240b61c0 msgr2=0x7fa22405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- 172.21.2.111:0/3749929232 shutdown_connections 2021-04-03T03:09:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- 172.21.2.111:0/3749929232 wait complete. 2021-04-03T03:09:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 Processor -- start 2021-04-03T03:09:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- start start 2021-04-03T03:09:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2240603f0 0x7fa224145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 0x7fa2240aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa224144ef0 0x7fa2240a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa224067cd0 con 0x7fa2240603f0 2021-04-03T03:09:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240ab090 con 0x7fa224144ef0 2021-04-03T03:09:17.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240a9700 con 0x7fa2240add50 2021-04-03T03:09:17.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2240603f0 0x7fa224145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2240603f0 0x7fa224145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51052/0 (socket says 172.21.2.111:51052) 2021-04-03T03:09:17.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 -- 172.21.2.111:0/1141413805 learned_addr learned my addr 172.21.2.111:0/1141413805 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:17.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 -- 172.21.2.111:0/1141413805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 msgr2=0x7fa2240aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 0x7fa2240aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 -- 172.21.2.111:0/1141413805 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa224144ef0 msgr2=0x7fa2240a7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa224144ef0 0x7fa2240a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 -- 172.21.2.111:0/1141413805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240a76a0 con 0x7fa2240603f0 2021-04-03T03:09:17.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa222ffd700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2240603f0 0x7fa224145cb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fa218002f50 tx=0x7fa218006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa213fff700 1 -- 172.21.2.111:0/1141413805 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa21800d4a0 con 0x7fa2240603f0 2021-04-03T03:09:17.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa213fff700 1 -- 172.21.2.111:0/1141413805 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21800d600 con 0x7fa2240603f0 2021-04-03T03:09:17.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa213fff700 1 -- 172.21.2.111:0/1141413805 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa21800d960 con 0x7fa2240603f0 2021-04-03T03:09:17.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa224067ec0 con 0x7fa2240603f0 2021-04-03T03:09:17.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.291+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2240abd40 con 0x7fa2240603f0 2021-04-03T03:09:17.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.295+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2240a7830 con 0x7fa2240603f0 2021-04-03T03:09:17.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.295+0000 7fa213fff700 1 -- 172.21.2.111:0/1141413805 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa21800f070 con 0x7fa2240603f0 2021-04-03T03:09:17.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.295+0000 7fa213fff700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa200056f20 0x7fa2000593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.295+0000 7fa213fff700 1 -- 172.21.2.111:0/1141413805 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa21806f980 con 0x7fa2240603f0 2021-04-03T03:09:17.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.295+0000 7fa2227fc700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa200056f20 0x7fa2000593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.295+0000 7fa2227fc700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa200056f20 0x7fa2000593e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fa214002f30 tx=0x7fa214006a70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.295+0000 7fa213fff700 1 -- 172.21.2.111:0/1141413805 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa218044d70 con 0x7fa2240603f0 2021-04-03T03:09:17.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa224057660 con 0x7fa2240603f0 2021-04-03T03:09:17.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa213fff700 1 -- 172.21.2.111:0/1141413805 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa218018020 con 0x7fa2240603f0 2021-04-03T03:09:17.400 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:09:17.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa200056f20 msgr2=0x7fa2000593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa200056f20 0x7fa2000593e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fa214002f30 tx=0x7fa214006a70).stop 2021-04-03T03:09:17.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2240603f0 msgr2=0x7fa224145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2240603f0 0x7fa224145cb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fa218002f50 tx=0x7fa218006dd0).stop 2021-04-03T03:09:17.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 shutdown_connections 2021-04-03T03:09:17.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa200056f20 0x7fa2000593e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa224144ef0 0x7fa2240a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2240603f0 0x7fa224145cb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 --2- 172.21.2.111:0/1141413805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2240add50 0x7fa2240aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 >> 172.21.2.111:0/1141413805 conn(0x7fa2240b61c0 msgr2=0x7fa2240b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:17.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 shutdown_connections 2021-04-03T03:09:17.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.391+0000 7fa223fff700 1 -- 172.21.2.111:0/1141413805 wait complete. 2021-04-03T03:09:17.407 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:09:17.408 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:17.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd6911ea700 1 Processor -- start 2021-04-03T03:09:17.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd6911ea700 1 -- start start 2021-04-03T03:09:17.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd6911ea700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd68c0603f0 0x7fd68c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd6911ea700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 0x7fd68c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd6911ea700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c0607d0 con 0x7fd68c0b12b0 2021-04-03T03:09:17.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd6911ea700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68c0af400 con 0x7fd68c0603f0 2021-04-03T03:09:17.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd6911ea700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0af6a0 con 0x7fd68c14b700 2021-04-03T03:09:17.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd68c0603f0 0x7fd68c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd68c0603f0 0x7fd68c0b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51058/0 (socket says 172.21.2.111:51058) 2021-04-03T03:09:17.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 -- 172.21.2.111:0/2091423779 learned_addr learned my addr 172.21.2.111:0/2091423779 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:17.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 -- 172.21.2.111:0/2091423779 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 msgr2=0x7fd68c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:17.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 --2- 172.21.2.111:0/2091423779 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 0x7fd68c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 -- 172.21.2.111:0/2091423779 >> v1:172.21.2.111:6789/0 conn(0x7fd68c0b12b0 legacy=0x7fd68c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 -- 172.21.2.111:0/2091423779 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c0afb40 con 0x7fd68c0603f0 2021-04-03T03:09:17.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68bfff700 1 --2- 172.21.2.111:0/2091423779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0b0090 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fd67c002f30 tx=0x7fd67c006e90).ready entity=mon.0 client_cookie=c923a994e419a2e1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68affd700 1 -- 172.21.2.111:0/2091423779 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd67c009970 con 0x7fd68c0603f0 2021-04-03T03:09:17.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.455+0000 7fd68affd700 1 -- 172.21.2.111:0/2091423779 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c009ad0 con 0x7fd68c0603f0 2021-04-03T03:09:17.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68affd700 1 -- 172.21.2.111:0/2091423779 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd67c009e30 con 0x7fd68c0603f0 2021-04-03T03:09:17.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/2091423779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 msgr2=0x7fd68c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 --2- 172.21.2.111:0/2091423779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0b0090 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fd67c002f30 tx=0x7fd67c006e90).stop 2021-04-03T03:09:17.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/2091423779 shutdown_connections 2021-04-03T03:09:17.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 --2- 172.21.2.111:0/2091423779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0b0090 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 --2- 172.21.2.111:0/2091423779 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 0x7fd68c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/2091423779 >> 172.21.2.111:0/2091423779 conn(0x7fd68c0b61c0 msgr2=0x7fd68c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:17.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/2091423779 shutdown_connections 2021-04-03T03:09:17.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/2091423779 wait complete. 2021-04-03T03:09:17.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 Processor -- start 2021-04-03T03:09:17.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- start start 2021-04-03T03:09:17.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd68c0b12b0 0x7fd68c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 0x7fd68c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0aa600 con 0x7fd68c0603f0 2021-04-03T03:09:17.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c062da0 con 0x7fd68c0b12b0 2021-04-03T03:09:17.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c063040 con 0x7fd68c14b700 2021-04-03T03:09:17.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0a81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51066/0 (socket says 172.21.2.111:51066) 2021-04-03T03:09:17.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 -- 172.21.2.111:0/3301128642 learned_addr learned my addr 172.21.2.111:0/3301128642 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:17.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 -- 172.21.2.111:0/3301128642 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 msgr2=0x7fd68c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 0x7fd68c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 -- 172.21.2.111:0/3301128642 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd68c0b12b0 msgr2=0x7fd68c15a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd68c0b12b0 0x7fd68c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 -- 172.21.2.111:0/3301128642 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c0addd0 con 0x7fd68c0603f0 2021-04-03T03:09:17.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68bfff700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0a81d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fd67c002f30 tx=0x7fd67c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6897fa700 1 -- 172.21.2.111:0/3301128642 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd67c011db0 con 0x7fd68c0603f0 2021-04-03T03:09:17.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6897fa700 1 -- 172.21.2.111:0/3301128642 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c01e460 con 0x7fd68c0603f0 2021-04-03T03:09:17.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68c0aa080 con 0x7fd68c0603f0 2021-04-03T03:09:17.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6897fa700 1 -- 172.21.2.111:0/3301128642 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd67c01e7c0 con 0x7fd68c0603f0 2021-04-03T03:09:17.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68c159c30 con 0x7fd68c0603f0 2021-04-03T03:09:17.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68c057660 con 0x7fd68c0603f0 2021-04-03T03:09:17.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6897fa700 1 -- 172.21.2.111:0/3301128642 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd67c025030 con 0x7fd68c0603f0 2021-04-03T03:09:17.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6897fa700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd670056ec0 0x7fd670059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:17.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd6897fa700 1 -- 172.21.2.111:0/3301128642 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd67c0716d0 con 0x7fd68c0603f0 2021-04-03T03:09:17.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68b7fe700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd670056ec0 0x7fd670059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:17.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.459+0000 7fd68b7fe700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd670056ec0 0x7fd670059380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd674002f30 tx=0x7fd6740069c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:17.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.463+0000 7fd6897fa700 1 -- 172.21.2.111:0/3301128642 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd67c046ac0 con 0x7fd68c0603f0 2021-04-03T03:09:17.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.583+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd68c0686d0 con 0x7fd68c0603f0 2021-04-03T03:09:17.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.583+0000 7fd6897fa700 1 -- 172.21.2.111:0/3301128642 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fd67c023090 con 0x7fd68c0603f0 2021-04-03T03:09:17.593 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:17.593 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:09:15.086110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1150523943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1150523943},{"type":"v1","addr":"172.21.2.111:6835","nonce":1150523943}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:09:17.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.583+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd670056ec0 msgr2=0x7fd670059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.583+0000 7fd6911ea700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd670056ec0 0x7fd670059380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd674002f30 tx=0x7fd6740069c0).stop 2021-04-03T03:09:17.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.583+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 msgr2=0x7fd68c0a81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:17.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.583+0000 7fd6911ea700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0a81d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fd67c002f30 tx=0x7fd67c015000).stop 2021-04-03T03:09:17.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 shutdown_connections 2021-04-03T03:09:17.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd68c14b700 0x7fd68c15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd68c0603f0 0x7fd68c0a81d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd670056ec0 0x7fd670059380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:17.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 --2- 172.21.2.111:0/3301128642 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd68c0b12b0 0x7fd68c15a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:17.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 >> 172.21.2.111:0/3301128642 conn(0x7fd68c0b61c0 msgr2=0x7fd68c0b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:17.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 shutdown_connections 2021-04-03T03:09:17.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:17.587+0000 7fd6911ea700 1 -- 172.21.2.111:0/3301128642 wait complete. 2021-04-03T03:09:17.597 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 11 2021-04-03T03:09:17.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:09:14.082969+0000', 'modified': '2021-04-03T03:09:15.086110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4881}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4881': {'gid': 4881, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1150523943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1150523943}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1150523943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:09:17.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:09:17.803 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:09:17.889 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":4881,"want_state":"up:active","state":"up:active","rank_uptime":3.7949307480000001,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":8.9446642280000006} 2021-04-03T03:09:17.890 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 4881, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 3.794930748, "state": "up:active", "uptime": 8.944664228, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:09:17.890 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:09:17.891 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:09:17.892 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:09:17.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:17.892 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:17.903 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:09:17.955 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:09:17.956 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:09:18.005 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:09:18.005 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:09:18.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:18.007 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:09:18.057 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:09:18.058 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:18.103 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:09:18.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:09:18.110 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:09:18.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:09:18.131+0000 7fa92b3520c0 -1 init, newargv = 0x556f317e9ba0 newargc=9 2021-04-03T03:09:18.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[14947]: starting ceph client 2021-04-03T03:09:18.146 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:09:18.146 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:09:18.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:18.147 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:09:18.151 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[14947]: starting fuse 2021-04-03T03:09:18.153 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:09:18.154 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:09:18.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:18.155 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:09:18.155 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:09:18.155 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:09:18.156 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:09:18.156 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:09:18.157 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:09:18.157 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:09:18.157 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:09:18.158 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:09:18.158 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:09:18.159 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:09:18.159 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:09:18.159 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:09:18.160 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:09:18.160 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:09:18.161 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:09:18.161 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:09:18.161 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:09:18.162 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:09:18.162 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:09:18.163 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:09:18.163 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:09:18.163 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:09:18.223 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.14947.asok 2021-04-03T03:09:18.225 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14947.asok 2021-04-03T03:09:18.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:18.226 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14947.asok status 2021-04-03T03:09:18.326 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:09:18.326 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:09:18.327 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:09:18.327 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:09:18.327 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:09:18.327 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:09:18.328 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:09:18.328 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "14947", 2021-04-03T03:09:18.328 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:09:18.328 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:09:18.328 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:09:18.329 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:09:18.329 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4531, 2021-04-03T03:09:18.329 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:09:18.329 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:09:18.330 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:09:18.330 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4531 2021-04-03T03:09:18.330 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:09:18.330 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:09:18.330 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:09:18.331 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:09:18.331 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 4121642412 2021-04-03T03:09:18.331 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:09:18.331 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:09:18.332 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:09:18.332 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:09:18.332 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:09:18.332 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 4121642412 2021-04-03T03:09:18.332 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:09:18.333 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4531 172.21.2.126:0/4121642412", 2021-04-03T03:09:18.333 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/4121642412", 2021-04-03T03:09:18.333 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:09:18.333 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 11, 2021-04-03T03:09:18.334 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 33, 2021-04-03T03:09:18.334 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:09:18.334 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:09:18.335 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:09:18.335 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:09:18.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:18.336 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:18.337 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:18.345 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:09:18.346 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:09:18.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:18.347 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:18.413 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2021-04-03T03:09:18.478 INFO:teuthology.orchestra.run.gibba011.stdout:{"mds_min_caps_per_client":"100"} 2021-04-03T03:09:18.478 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2021-04-03T03:09:18.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:09:18.479 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:09:18.480 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:09:18.490 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:09:18.492 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:09:18.492 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:09:18.493 DEBUG:teuthology.orchestra.run.gibba026:> import sys 2021-04-03T03:09:18.493 DEBUG:teuthology.orchestra.run.gibba026:> import time 2021-04-03T03:09:18.493 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:09:18.494 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:09:18.494 DEBUG:teuthology.orchestra.run.gibba026:> n = 1000 2021-04-03T03:09:18.495 DEBUG:teuthology.orchestra.run.gibba026:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-04-03T03:09:18.495 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:09:18.495 DEBUG:teuthology.orchestra.run.gibba026:> if not os.path.exists(os.path.dirname(abs_path)): 2021-04-03T03:09:18.496 DEBUG:teuthology.orchestra.run.gibba026:> os.makedirs(os.path.dirname(abs_path)) 2021-04-03T03:09:18.496 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:09:18.496 DEBUG:teuthology.orchestra.run.gibba026:> for i in range(0, n): 2021-04-03T03:09:18.497 DEBUG:teuthology.orchestra.run.gibba026:> fname = "{0}_{1}".format(abs_path, i) 2021-04-03T03:09:18.497 DEBUG:teuthology.orchestra.run.gibba026:> with open(fname, '"'"'w'"'"') as f: 2021-04-03T03:09:18.498 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content'"'"') 2021-04-03T03:09:18.498 DEBUG:teuthology.orchestra.run.gibba026:> if True: 2021-04-03T03:09:18.498 DEBUG:teuthology.orchestra.run.gibba026:> f.flush() 2021-04-03T03:09:18.499 DEBUG:teuthology.orchestra.run.gibba026:> os.fsync(f.fileno()) 2021-04-03T03:09:18.499 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:09:27.780 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:27.783 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:27.786 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:27.788 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:27.792 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:34.604 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:34.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 Processor -- start 2021-04-03T03:09:34.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 -- start start 2021-04-03T03:09:34.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1348048f00 0x7f13480d5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f13480d24a0 0x7f134803e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1348044a00 con 0x7f134803f940 2021-04-03T03:09:34.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13480492e0 con 0x7f13480d24a0 2021-04-03T03:09:34.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480d5380 con 0x7f1348048f00 2021-04-03T03:09:34.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134ed9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f13480d24a0 0x7f134803e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:34.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134dd9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f134803f940 0x7f13480d1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51872/0 (socket says 172.21.2.111:51872) 2021-04-03T03:09:34.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134ed9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f13480d24a0 0x7f134803e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51082/0 (socket says 172.21.2.111:51082) 2021-04-03T03:09:34.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134dd9b700 1 -- 172.21.2.111:0/11048493 learned_addr learned my addr 172.21.2.111:0/11048493 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:09:34.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134e59c700 1 --2- 172.21.2.111:0/11048493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1348048f00 0x7f13480d5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:34.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134ed9d700 1 -- 172.21.2.111:0/11048493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1348048f00 msgr2=0x7f13480d5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:34.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134ed9d700 1 --2- 172.21.2.111:0/11048493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1348048f00 0x7f13480d5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:34.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134ed9d700 1 -- 172.21.2.111:0/11048493 >> v1:172.21.2.111:6789/0 conn(0x7f134803f940 legacy=0x7f13480d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134ed9d700 1 -- 172.21.2.111:0/11048493 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480d5620 con 0x7f13480d24a0 2021-04-03T03:09:34.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134ed9d700 1 --2- 172.21.2.111:0/11048493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803e910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f135006b5d0 tx=0x7f135005ad70).ready entity=mon.0 client_cookie=9fb140245d11a766 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:34.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134d59a700 1 -- 172.21.2.111:0/11048493 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f135005bda0 con 0x7f13480d24a0 2021-04-03T03:09:34.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134d59a700 1 -- 172.21.2.111:0/11048493 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135005df60 con 0x7f13480d24a0 2021-04-03T03:09:34.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134d59a700 1 -- 172.21.2.111:0/11048493 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13500699f0 con 0x7f13480d24a0 2021-04-03T03:09:34.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 -- 172.21.2.111:0/11048493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 msgr2=0x7f134803e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.655+0000 7f134f59e700 1 --2- 172.21.2.111:0/11048493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803e910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f135006b5d0 tx=0x7f135005ad70).stop 2021-04-03T03:09:34.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- 172.21.2.111:0/11048493 shutdown_connections 2021-04-03T03:09:34.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 --2- 172.21.2.111:0/11048493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803e910 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:34.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 --2- 172.21.2.111:0/11048493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1348048f00 0x7f13480d5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:34.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- 172.21.2.111:0/11048493 >> 172.21.2.111:0/11048493 conn(0x7f134800f320 msgr2=0x7f13480484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:34.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- 172.21.2.111:0/11048493 shutdown_connections 2021-04-03T03:09:34.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- 172.21.2.111:0/11048493 wait complete. 2021-04-03T03:09:34.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 Processor -- start 2021-04-03T03:09:34.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- start start 2021-04-03T03:09:34.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f134803f940 0x7f13480a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1348048f00 0x7f134803e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134803f0f0 con 0x7f13480d24a0 2021-04-03T03:09:34.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480a4250 con 0x7f1348048f00 2021-04-03T03:09:34.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480a44f0 con 0x7f134803f940 2021-04-03T03:09:34.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:34.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51086/0 (socket says 172.21.2.111:51086) 2021-04-03T03:09:34.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 -- 172.21.2.111:0/3295442637 learned_addr learned my addr 172.21.2.111:0/3295442637 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:34.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 -- 172.21.2.111:0/3295442637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f134803f940 msgr2=0x7f13480a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f134803f940 0x7f13480a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 -- 172.21.2.111:0/3295442637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1348048f00 msgr2=0x7f134803e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1348048f00 0x7f134803e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 -- 172.21.2.111:0/3295442637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480a4790 con 0x7f13480d24a0 2021-04-03T03:09:34.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134ed9d700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803ec00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f135006b5d0 tx=0x7f1350064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:34.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f133f7fe700 1 -- 172.21.2.111:0/3295442637 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13500699f0 con 0x7f13480d24a0 2021-04-03T03:09:34.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f133f7fe700 1 -- 172.21.2.111:0/3295442637 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13500441b0 con 0x7f13480d24a0 2021-04-03T03:09:34.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13480a2c70 con 0x7f13480d24a0 2021-04-03T03:09:34.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f133f7fe700 1 -- 172.21.2.111:0/3295442637 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f135005f4b0 con 0x7f13480d24a0 2021-04-03T03:09:34.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13480a1dc0 con 0x7f13480d24a0 2021-04-03T03:09:34.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13480a2e00 con 0x7f13480d24a0 2021-04-03T03:09:34.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f133f7fe700 1 -- 172.21.2.111:0/3295442637 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f135008c020 con 0x7f13480d24a0 2021-04-03T03:09:34.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f133f7fe700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f20 0x7f132c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f133f7fe700 1 -- 172.21.2.111:0/3295442637 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f13500e0960 con 0x7f13480d24a0 2021-04-03T03:09:34.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.659+0000 7f134e59c700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f20 0x7f132c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:34.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.663+0000 7f134e59c700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f20 0x7f132c0593e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f134000a000 tx=0x7f1340008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:34.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.663+0000 7f133f7fe700 1 -- 172.21.2.111:0/3295442637 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f13500b5c40 con 0x7f13480d24a0 2021-04-03T03:09:34.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.783+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13480a4920 con 0x7f13480d24a0 2021-04-03T03:09:34.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f133f7fe700 1 -- 172.21.2.111:0/3295442637 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f13500b9260 con 0x7f13480d24a0 2021-04-03T03:09:34.795 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:34.795 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:09:15.086110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1150523943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1150523943},{"type":"v1","addr":"172.21.2.111:6835","nonce":1150523943}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:09:34.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f20 msgr2=0x7f132c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f20 0x7f132c0593e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f134000a000 tx=0x7f1340008040).stop 2021-04-03T03:09:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 msgr2=0x7f134803ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803ec00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f135006b5d0 tx=0x7f1350064030).stop 2021-04-03T03:09:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 shutdown_connections 2021-04-03T03:09:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13480d24a0 0x7f134803ec00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f134803f940 0x7f13480a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f20 0x7f132c0593e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 --2- 172.21.2.111:0/3295442637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1348048f00 0x7f134803e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 >> 172.21.2.111:0/3295442637 conn(0x7f134800f320 msgr2=0x7f13480d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 shutdown_connections 2021-04-03T03:09:34.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.787+0000 7f134f59e700 1 -- 172.21.2.111:0/3295442637 wait complete. 2021-04-03T03:09:34.799 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 11 2021-04-03T03:09:34.802 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2021-04-03T03:09:34.866 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-04-03T03:09:34.867 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-04-03T03:09:34.867 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:34.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 Processor -- start 2021-04-03T03:09:34.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- start start 2021-04-03T03:09:34.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadb005fb50 0x7fadb0067c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fadb014ad80 0x7fadb00af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb00b0960 0x7fadb00b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00b0530 con 0x7fadb005fb50 2021-04-03T03:09:34.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00afe00 con 0x7fadb014ad80 2021-04-03T03:09:34.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00b00a0 con 0x7fadb00b0960 2021-04-03T03:09:34.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadb005fb50 0x7fadb0067c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:34.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadb005fb50 0x7fadb0067c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51094/0 (socket says 172.21.2.111:51094) 2021-04-03T03:09:34.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 -- 172.21.2.111:0/1052445128 learned_addr learned my addr 172.21.2.111:0/1052445128 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:34.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 -- 172.21.2.111:0/1052445128 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb00b0960 msgr2=0x7fadb00b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:34.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- 172.21.2.111:0/1052445128 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb00b0960 0x7fadb00b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 -- 172.21.2.111:0/1052445128 >> v2:172.21.2.115:3300/0 conn(0x7fadb014ad80 msgr2=0x7fadb00af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:34.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- 172.21.2.111:0/1052445128 >> v2:172.21.2.115:3300/0 conn(0x7fadb014ad80 0x7fadb00af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 -- 172.21.2.111:0/1052445128 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00b12e0 con 0x7fadb005fb50 2021-04-03T03:09:34.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- 172.21.2.111:0/1052445128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb0067c60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fadac002f30 tx=0x7fadac006e90).ready entity=mon.0 client_cookie=df05274b1a9c7394 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:34.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb4dac700 1 -- 172.21.2.111:0/1052445128 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac009920 con 0x7fadb005fb50 2021-04-03T03:09:34.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb4dac700 1 -- 172.21.2.111:0/1052445128 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac009a80 con 0x7fadb005fb50 2021-04-03T03:09:34.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb4dac700 1 -- 172.21.2.111:0/1052445128 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac009de0 con 0x7fadb005fb50 2021-04-03T03:09:34.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- 172.21.2.111:0/1052445128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 msgr2=0x7fadb0067c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- 172.21.2.111:0/1052445128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb0067c60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fadac002f30 tx=0x7fadac006e90).stop 2021-04-03T03:09:34.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- 172.21.2.111:0/1052445128 shutdown_connections 2021-04-03T03:09:34.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- 172.21.2.111:0/1052445128 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb00b0960 0x7fadb00b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- 172.21.2.111:0/1052445128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb0067c60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:34.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- 172.21.2.111:0/1052445128 >> v2:172.21.2.115:3300/0 conn(0x7fadb014ad80 0x7fadb00af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- 172.21.2.111:0/1052445128 >> 172.21.2.111:0/1052445128 conn(0x7fadb00b5b50 msgr2=0x7fadb00b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:34.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- 172.21.2.111:0/1052445128 shutdown_connections 2021-04-03T03:09:34.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- 172.21.2.111:0/1052445128 wait complete. 2021-04-03T03:09:34.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 Processor -- start 2021-04-03T03:09:34.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- start start 2021-04-03T03:09:34.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb00a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadb00b0960 0x7fadb00a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb014ad80 0x7fadb00aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00a9a70 con 0x7fadb005fb50 2021-04-03T03:09:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00a7e50 con 0x7fadb00b0960 2021-04-03T03:09:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb6db0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00a80f0 con 0x7fadb014ad80 2021-04-03T03:09:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb00a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb00a8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51102/0 (socket says 172.21.2.111:51102) 2021-04-03T03:09:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 -- 172.21.2.111:0/3272383571 learned_addr learned my addr 172.21.2.111:0/3272383571 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 -- 172.21.2.111:0/3272383571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb014ad80 msgr2=0x7fadb00aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb014ad80 0x7fadb00aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 -- 172.21.2.111:0/3272383571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadb00b0960 msgr2=0x7fadb00a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:34.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.915+0000 7fadb5dae700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadb00b0960 0x7fadb00a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:34.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fadb5dae700 1 -- 172.21.2.111:0/3272383571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00a8390 con 0x7fadb005fb50 2021-04-03T03:09:34.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fadb5dae700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb00a8fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fadac002f30 tx=0x7fadac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:34.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fada2ffd700 1 -- 172.21.2.111:0/3272383571 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac011dc0 con 0x7fadb005fb50 2021-04-03T03:09:34.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fada2ffd700 1 -- 172.21.2.111:0/3272383571 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac01e460 con 0x7fadb005fb50 2021-04-03T03:09:34.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb00a7b40 con 0x7fadb005fb50 2021-04-03T03:09:34.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fada2ffd700 1 -- 172.21.2.111:0/3272383571 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac01e7c0 con 0x7fadb005fb50 2021-04-03T03:09:34.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb00aaf80 con 0x7fadb005fb50 2021-04-03T03:09:34.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb00aa7f0 con 0x7fadb005fb50 2021-04-03T03:09:34.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fada2ffd700 1 -- 172.21.2.111:0/3272383571 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fadac025030 con 0x7fadb005fb50 2021-04-03T03:09:34.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fada2ffd700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad90056fb0 0x7fad90059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:34.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fada2ffd700 1 -- 172.21.2.111:0/3272383571 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fadac0716e0 con 0x7fadb005fb50 2021-04-03T03:09:34.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fadb55ad700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad90056fb0 0x7fad90059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:34.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.919+0000 7fadb55ad700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad90056fb0 0x7fad90059470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fada4002f30 tx=0x7fada4006950).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:34.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:34.923+0000 7fada2ffd700 1 -- 172.21.2.111:0/3272383571 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fadb00aa7f0 con 0x7fadb005fb50 2021-04-03T03:09:35.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.047+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadb00aa7f0 con 0x7fadb005fb50 2021-04-03T03:09:35.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.047+0000 7fada2ffd700 1 -- 172.21.2.111:0/3272383571 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fadb00aa7f0 con 0x7fadb005fb50 2021-04-03T03:09:35.058 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:35.058 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:09:15.086110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1150523943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1150523943},{"type":"v1","addr":"172.21.2.111:6835","nonce":1150523943}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:09:35.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad90056fb0 msgr2=0x7fad90059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad90056fb0 0x7fad90059470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fada4002f30 tx=0x7fada4006950).stop 2021-04-03T03:09:35.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 msgr2=0x7fadb00a8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb00a8fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fadac002f30 tx=0x7fadac015000).stop 2021-04-03T03:09:35.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 shutdown_connections 2021-04-03T03:09:35.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadb014ad80 0x7fadb00aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadb005fb50 0x7fadb00a8fd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad90056fb0 0x7fad90059470 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 --2- 172.21.2.111:0/3272383571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadb00b0960 0x7fadb00a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 >> 172.21.2.111:0/3272383571 conn(0x7fadb00b5b50 msgr2=0x7fadb00b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:35.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 shutdown_connections 2021-04-03T03:09:35.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.051+0000 7fadb6db0700 1 -- 172.21.2.111:0/3272383571 wait complete. 2021-04-03T03:09:35.062 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 11 2021-04-03T03:09:35.065 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2021-04-03T03:09:35.130 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-04-03T03:09:35.131 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-04-03T03:09:35.131 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:09:35.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 Processor -- start 2021-04-03T03:09:35.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 -- start start 2021-04-03T03:09:35.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c0603f0 0x7f6b9c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6b9c14b700 0x7f6b9c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b9c0b12b0 0x7f6b9c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0b0f60 con 0x7f6b9c0b12b0 2021-04-03T03:09:35.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0607d0 con 0x7f6b9c14b700 2021-04-03T03:09:35.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6ba0dca700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0af400 con 0x7f6b9c0603f0 2021-04-03T03:09:35.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6b9bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b9c0b12b0 0x7f6b9c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6b9bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b9c0b12b0 0x7f6b9c0b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51106/0 (socket says 172.21.2.111:51106) 2021-04-03T03:09:35.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.179+0000 7f6b9bfff700 1 -- 172.21.2.111:0/697439513 learned_addr learned my addr 172.21.2.111:0/697439513 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9bfff700 1 -- 172.21.2.111:0/697439513 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c0603f0 msgr2=0x7f6b9c0b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:09:35.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9bfff700 1 --2- 172.21.2.111:0/697439513 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c0603f0 0x7f6b9c0b0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9bfff700 1 -- 172.21.2.111:0/697439513 >> v2:172.21.2.115:3300/0 conn(0x7f6b9c14b700 msgr2=0x7f6b9c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9bfff700 1 --2- 172.21.2.111:0/697439513 >> v2:172.21.2.115:3300/0 conn(0x7f6b9c14b700 0x7f6b9c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9bfff700 1 -- 172.21.2.111:0/697439513 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0af6a0 con 0x7f6b9c0b12b0 2021-04-03T03:09:35.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9bfff700 1 --2- 172.21.2.111:0/697439513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0b12b0 0x7f6b9c0b0a70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f6b9c062740 tx=0x7f6b94009270).ready entity=mon.0 client_cookie=49a7c7c125b8d2f0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9a7fc700 1 -- 172.21.2.111:0/697439513 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b9400bbc0 con 0x7f6b9c0b12b0 2021-04-03T03:09:35.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9a7fc700 1 -- 172.21.2.111:0/697439513 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9400bd20 con 0x7f6b9c0b12b0 2021-04-03T03:09:35.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9a7fc700 1 -- 172.21.2.111:0/697439513 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b94011400 con 0x7f6b9c0b12b0 2021-04-03T03:09:35.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/697439513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0b12b0 msgr2=0x7f6b9c0b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/697439513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0b12b0 0x7f6b9c0b0a70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f6b9c062740 tx=0x7f6b94009270).stop 2021-04-03T03:09:35.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/697439513 shutdown_connections 2021-04-03T03:09:35.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/697439513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0b12b0 0x7f6b9c0b0a70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/697439513 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c0603f0 0x7f6b9c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/697439513 >> v2:172.21.2.115:3300/0 conn(0x7f6b9c14b700 0x7f6b9c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/697439513 >> 172.21.2.111:0/697439513 conn(0x7f6b9c0b61c0 msgr2=0x7f6b9c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:35.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/697439513 shutdown_connections 2021-04-03T03:09:35.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/697439513 wait complete. 2021-04-03T03:09:35.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 Processor -- start 2021-04-03T03:09:35.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- start start 2021-04-03T03:09:35.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0603f0 0x7f6b9c0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b9c0b12b0 0x7f6b9c146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c14b700 0x7f6b9c146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0a8c40 con 0x7f6b9c0603f0 2021-04-03T03:09:35.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0ab700 con 0x7f6b9c0b12b0 2021-04-03T03:09:35.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0aa060 con 0x7f6b9c14b700 2021-04-03T03:09:35.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0603f0 0x7f6b9c0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0603f0 0x7f6b9c0aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51114/0 (socket says 172.21.2.111:51114) 2021-04-03T03:09:35.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 -- 172.21.2.111:0/2954634370 learned_addr learned my addr 172.21.2.111:0/2954634370 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 -- 172.21.2.111:0/2954634370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c14b700 msgr2=0x7f6b9c146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c14b700 0x7f6b9c146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 -- 172.21.2.111:0/2954634370 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b9c0b12b0 msgr2=0x7f6b9c146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b9c0b12b0 0x7f6b9c146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 -- 172.21.2.111:0/2954634370 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0ab930 con 0x7f6b9c0603f0 2021-04-03T03:09:35.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b9b7fe700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0603f0 0x7f6b9c0aae00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f6b9c0a8d80 tx=0x7f6b8c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b98ff9700 1 -- 172.21.2.111:0/2954634370 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8c00d570 con 0x7f6b9c0603f0 2021-04-03T03:09:35.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b98ff9700 1 -- 172.21.2.111:0/2954634370 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c00d6d0 con 0x7f6b9c0603f0 2021-04-03T03:09:35.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6b98ff9700 1 -- 172.21.2.111:0/2954634370 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8c00da30 con 0x7f6b9c0603f0 2021-04-03T03:09:35.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9c147310 con 0x7f6b9c0603f0 2021-04-03T03:09:35.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c0a9790 con 0x7f6b9c0603f0 2021-04-03T03:09:35.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.183+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9c057660 con 0x7f6b9c0603f0 2021-04-03T03:09:35.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.187+0000 7f6b98ff9700 1 -- 172.21.2.111:0/2954634370 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6b8c00f070 con 0x7f6b9c0603f0 2021-04-03T03:09:35.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.187+0000 7f6b98ff9700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b7c056ea0 0x7f6b7c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.187+0000 7f6b98ff9700 1 -- 172.21.2.111:0/2954634370 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f6b8c06fb00 con 0x7f6b9c0603f0 2021-04-03T03:09:35.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.187+0000 7f6b9affd700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b7c056ea0 0x7f6b7c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.187+0000 7f6b9affd700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b7c056ea0 0x7f6b7c059360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f6b90002f30 tx=0x7f6b900069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.187+0000 7f6b98ff9700 1 -- 172.21.2.111:0/2954634370 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6b8c071e60 con 0x7f6b9c0603f0 2021-04-03T03:09:35.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b9c0ab480 con 0x7f6b9c0603f0 2021-04-03T03:09:35.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6b98ff9700 1 -- 172.21.2.111:0/2954634370 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f6b8c018030 con 0x7f6b9c0603f0 2021-04-03T03:09:35.325 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:09:35.326 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:09:15.086110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1150523943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1150523943},{"type":"v1","addr":"172.21.2.111:6835","nonce":1150523943}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:09:35.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b7c056ea0 msgr2=0x7f6b7c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b7c056ea0 0x7f6b7c059360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f6b90002f30 tx=0x7f6b900069f0).stop 2021-04-03T03:09:35.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0603f0 msgr2=0x7f6b9c0aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0603f0 0x7f6b9c0aae00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f6b9c0a8d80 tx=0x7f6b8c006dd0).stop 2021-04-03T03:09:35.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 shutdown_connections 2021-04-03T03:09:35.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b9c14b700 0x7f6b9c146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b9c0603f0 0x7f6b9c0aae00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b7c056ea0 0x7f6b7c059360 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 --2- 172.21.2.111:0/2954634370 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b9c0b12b0 0x7f6b9c146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.315+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 >> 172.21.2.111:0/2954634370 conn(0x7f6b9c0b61c0 msgr2=0x7f6b9c0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:35.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.319+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 shutdown_connections 2021-04-03T03:09:35.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.319+0000 7f6ba0dca700 1 -- 172.21.2.111:0/2954634370 wait complete. 2021-04-03T03:09:35.330 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 11 2021-04-03T03:09:35.333 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 2021-04-03T03:09:35.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.383+0000 7f9c20de9700 1 Processor -- start 2021-04-03T03:09:35.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.383+0000 7f9c20de9700 1 -- start start 2021-04-03T03:09:35.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.383+0000 7f9c20de9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9c1c0b5620 0x7f9c1c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.383+0000 7f9c20de9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c0607d0 con 0x7f9c1c0603f0 2021-04-03T03:09:35.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.383+0000 7f9c20de9700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c0b3f80 con 0x7f9c1c0b44a0 2021-04-03T03:09:35.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.383+0000 7f9c20de9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c1433c0 con 0x7f9c1c0b5620 2021-04-03T03:09:35.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9c1c0b5620 0x7f9c1c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9c1c0b5620 0x7f9c1c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51124/0 (socket says 172.21.2.111:51124) 2021-04-03T03:09:35.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3060234498 learned_addr learned my addr 172.21.2.111:0/3060234498 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3060234498 >> v1:172.21.2.115:6789/0 conn(0x7f9c1c0b44a0 legacy=0x7f9c1c142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3060234498 >> v1:172.21.2.111:6789/0 conn(0x7f9c1c0603f0 legacy=0x7f9c1c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3060234498 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c0b4c60 con 0x7f9c1c0b5620 2021-04-03T03:09:35.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1bfff700 1 --2- 172.21.2.111:0/3060234498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0b5620 0x7f9c1c0b2b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9c0c0010d0 tx=0x7f9c14009270).ready entity=mon.0 client_cookie=ec8bd2956d31247d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1a7fc700 1 -- 172.21.2.111:0/3060234498 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c1400bac0 con 0x7f9c1c0b5620 2021-04-03T03:09:35.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1a7fc700 1 -- 172.21.2.111:0/3060234498 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1400bc20 con 0x7f9c1c0b5620 2021-04-03T03:09:35.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1a7fc700 1 -- 172.21.2.111:0/3060234498 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c1400bd90 con 0x7f9c1c0b5620 2021-04-03T03:09:35.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- 172.21.2.111:0/3060234498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0b5620 msgr2=0x7f9c1c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3060234498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0b5620 0x7f9c1c0b2b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9c0c0010d0 tx=0x7f9c14009270).stop 2021-04-03T03:09:35.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- 172.21.2.111:0/3060234498 shutdown_connections 2021-04-03T03:09:35.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3060234498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0b5620 0x7f9c1c0b2b90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- 172.21.2.111:0/3060234498 >> 172.21.2.111:0/3060234498 conn(0x7f9c1c05c7d0 msgr2=0x7f9c1c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:35.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- 172.21.2.111:0/3060234498 shutdown_connections 2021-04-03T03:09:35.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- 172.21.2.111:0/3060234498 wait complete. 2021-04-03T03:09:35.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 Processor -- start 2021-04-03T03:09:35.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- start start 2021-04-03T03:09:35.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0603f0 0x7f9c1c0b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9c1c0b44a0 0x7f9c1c156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9c1c0b5620 0x7f9c1c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c1578e0 con 0x7f9c1c0603f0 2021-04-03T03:09:35.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c064230 con 0x7f9c1c0b5620 2021-04-03T03:09:35.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0644d0 con 0x7f9c1c0b44a0 2021-04-03T03:09:35.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0603f0 0x7f9c1c0b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0603f0 0x7f9c1c0b8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51126/0 (socket says 172.21.2.111:51126) 2021-04-03T03:09:35.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 -- 172.21.2.111:0/3724970985 learned_addr learned my addr 172.21.2.111:0/3724970985 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 -- 172.21.2.111:0/3724970985 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9c1c0b44a0 msgr2=0x7f9c1c156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9c1c0b44a0 0x7f9c1c156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 -- 172.21.2.111:0/3724970985 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9c1c0b5620 msgr2=0x7f9c1c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9c1c0b5620 0x7f9c1c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 -- 172.21.2.111:0/3724970985 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c0ae390 con 0x7f9c1c0603f0 2021-04-03T03:09:35.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c1b7fe700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0603f0 0x7f9c1c0b8d70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f9c0c000b80 tx=0x7f9c0c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c18ff9700 1 -- 172.21.2.111:0/3724970985 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c0c007020 con 0x7f9c1c0603f0 2021-04-03T03:09:35.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c18ff9700 1 -- 172.21.2.111:0/3724970985 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c007180 con 0x7f9c1c0603f0 2021-04-03T03:09:35.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c0b9260 con 0x7f9c1c0603f0 2021-04-03T03:09:35.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c156aa0 con 0x7f9c1c0603f0 2021-04-03T03:09:35.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.387+0000 7f9c18ff9700 1 -- 172.21.2.111:0/3724970985 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c0c0072f0 con 0x7f9c1c0603f0 2021-04-03T03:09:35.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c18ff9700 1 -- 172.21.2.111:0/3724970985 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9c0c018020 con 0x7f9c1c0603f0 2021-04-03T03:09:35.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c18ff9700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9bfc057050 0x7f9bfc059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c18ff9700 1 -- 172.21.2.111:0/3724970985 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9c0c06f570 con 0x7f9c1c0603f0 2021-04-03T03:09:35.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1affd700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9bfc057050 0x7f9bfc059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1affd700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9bfc057050 0x7f9bfc059510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f9c10001f60 tx=0x7f9c100077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 Processor -- start 2021-04-03T03:09:35.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- start start 2021-04-03T03:09:35.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 0x1daa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1c13e90 0x1c8f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1b93800 0x1d735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cbdf50 con 0x1b93800 2021-04-03T03:09:35.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cbe090 con 0x1d2b390 2021-04-03T03:09:35.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cbf4c0 con 0x1c13e90 2021-04-03T03:09:35.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1b93800 0x1d735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1b93800 0x1d735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51138/0 (socket says 172.21.2.111:51138) 2021-04-03T03:09:35.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3610153293 learned_addr learned my addr 172.21.2.111:0/3610153293 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3610153293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1c13e90 msgr2=0x1c8f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 --2- 172.21.2.111:0/3610153293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1c13e90 0x1c8f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3610153293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 msgr2=0x1daa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 --2- 172.21.2.111:0/3610153293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 0x1daa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 -- 172.21.2.111:0/3610153293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1caec70 con 0x1b93800 2021-04-03T03:09:35.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c1bfff700 1 --2- 172.21.2.111:0/3610153293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1b93800 0x1d735a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f9c0c0010d0 tx=0x7f9c14016000).ready entity=mon.0 client_cookie=f45e9ee20eab6020 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c20de9700 1 -- 172.21.2.111:0/3610153293 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c140091e0 con 0x1b93800 2021-04-03T03:09:35.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c20de9700 1 -- 172.21.2.111:0/3610153293 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c14003270 con 0x1b93800 2021-04-03T03:09:35.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c20de9700 1 -- 172.21.2.111:0/3610153293 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c1401f610 con 0x1b93800 2021-04-03T03:09:35.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- 172.21.2.111:0/3610153293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1b93800 msgr2=0x1d735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 --2- 172.21.2.111:0/3610153293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1b93800 0x1d735a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f9c0c0010d0 tx=0x7f9c14016000).stop 2021-04-03T03:09:35.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- 172.21.2.111:0/3610153293 shutdown_connections 2021-04-03T03:09:35.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 --2- 172.21.2.111:0/3610153293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1b93800 0x1d735a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 --2- 172.21.2.111:0/3610153293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 0x1daa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 --2- 172.21.2.111:0/3610153293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1c13e90 0x1c8f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- 172.21.2.111:0/3610153293 >> 172.21.2.111:0/3610153293 conn(0x1d5abf0 msgr2=0x1d715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:35.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- 172.21.2.111:0/3610153293 shutdown_connections 2021-04-03T03:09:35.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.391+0000 7f9c30867740 1 -- 172.21.2.111:0/3610153293 wait complete. 2021-04-03T03:09:35.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- start start 2021-04-03T03:09:35.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 Processor -- start 2021-04-03T03:09:35.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 0x1d59d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1db90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 0x1d7d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ca0410 con 0x1c13e90 2021-04-03T03:09:35.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d7aa00 con 0x1b93800 2021-04-03T03:09:35.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d7ac30 con 0x1d2b390 2021-04-03T03:09:35.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 10 client.0 ll_register_callbacks cb 0x1d5ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:09:35.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1db90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1db90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51144/0 (socket says 172.21.2.111:51144) 2021-04-03T03:09:35.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 -- 172.21.2.111:0/3954565241 learned_addr learned my addr 172.21.2.111:0/3954565241 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 -- 172.21.2.111:0/3954565241 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 msgr2=0x1d7d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 0x1d7d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 -- 172.21.2.111:0/3954565241 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 msgr2=0x1d59d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 0x1d59d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 -- 172.21.2.111:0/3954565241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1cb74a0 con 0x1c13e90 2021-04-03T03:09:35.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1affd700 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1db90a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9c10002b70 tx=0x7f9c10010000).ready entity=mon.0 client_cookie=657a7f010467e9a0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:09:35.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 1 -- 172.21.2.111:0/3954565241 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c10007240 con 0x1c13e90 2021-04-03T03:09:35.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 1 -- 172.21.2.111:0/3954565241 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9c100073a0 con 0x1c13e90 2021-04-03T03:09:35.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1bb88b0 con 0x1c13e90 2021-04-03T03:09:35.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 1 -- 172.21.2.111:0/3954565241 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c100064f0 con 0x1c13e90 2021-04-03T03:09:35.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 10 client.5139 fetch_fsmap learned FSMap version 11 2021-04-03T03:09:35.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x1b43770 con 0x1c13e90 2021-04-03T03:09:35.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 1 -- 172.21.2.111:0/3954565241 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1bb88b0 con 0x1c13e90 2021-04-03T03:09:35.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 1 -- 172.21.2.111:0/3954565241 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 11) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f9c10006ba0 con 0x1c13e90 2021-04-03T03:09:35.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 10 client.5139 fetch_fsmap finished waiting for FSMap version 11 2021-04-03T03:09:35.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 10 client.5139 resolve_mds: resolved ID 'a' to GID 4881 2021-04-03T03:09:35.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 20 client.5139 populate_metadata read hostname 'gibba011' 2021-04-03T03:09:35.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1b25c00 0x1dbbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 4 client.5139 mds_command: new command op to 4881 tid=0[{"prefix": "get_command_descriptions"}] 2021-04-03T03:09:35.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 --> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1b25150 con 0x1b25c00 2021-04-03T03:09:35.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1b7fe700 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1b25c00 0x1dbbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 10 client.5139 ms_handle_connect on v2:172.21.2.111:6834/1150523943 2021-04-03T03:09:35.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c1b7fe700 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1b25c00 0x1dbbe70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f5d92e14b35fdf63 server_cookie=d498179cc80f94f1 in_seq=0 out_seq=0 2021-04-03T03:09:35.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 1 -- 172.21.2.111:0/3954565241 <== mds.0 v2:172.21.2.111:6834/1150523943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1b25150 con 0x1b25c00 2021-04-03T03:09:35.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 10 client.5139 handle_command_reply: tid=0 2021-04-03T03:09:35.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 client.5139 shutdown 2021-04-03T03:09:35.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 msgr2=0x1db90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1db90a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9c10002b70 tx=0x7f9c10010000).stop 2021-04-03T03:09:35.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 shutdown_connections 2021-04-03T03:09:35.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 0x1d7d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 0x1d59d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1b25c00 0x1dbbe70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 --2- 172.21.2.111:0/3954565241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1db90a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 >> 172.21.2.111:0/3954565241 conn(0x1d5abf0 msgr2=0x1d715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:35.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9bfa7fc700 0 client.5139 ms_handle_reset on v2:172.21.2.111:6834/1150523943 2021-04-03T03:09:35.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 shutdown_connections 2021-04-03T03:09:35.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 1 -- 172.21.2.111:0/3954565241 wait complete. 2021-04-03T03:09:35.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.395+0000 7f9c30867740 20 client.5139 trim_cache size 0 max 16384 2021-04-03T03:09:35.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 Processor -- start 2021-04-03T03:09:35.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- start start 2021-04-03T03:09:35.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1b93800 0x1d74000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d7b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 0x1d7b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d72fa0 con 0x1c13e90 2021-04-03T03:09:35.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dae530 con 0x1d2b390 2021-04-03T03:09:35.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x1da9da0 con 0x1b93800 2021-04-03T03:09:35.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d7b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51148/0 (socket says 172.21.2.111:51148) 2021-04-03T03:09:35.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 -- 172.21.2.111:0/47763575 learned_addr learned my addr 172.21.2.111:0/47763575 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 -- 172.21.2.111:0/47763575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1b93800 msgr2=0x1d74000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 --2- 172.21.2.111:0/47763575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1b93800 0x1d74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 -- 172.21.2.111:0/47763575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 msgr2=0x1d7b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 --2- 172.21.2.111:0/47763575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 0x1d7b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 -- 172.21.2.111:0/47763575 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4008850 con 0x1c13e90 2021-04-03T03:09:35.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c1affd700 1 --2- 172.21.2.111:0/47763575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d7b170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x1d730e0 tx=0x7f9c1001c000).ready entity=mon.0 client_cookie=ce06875c652ada0f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c00ff9700 1 -- 172.21.2.111:0/47763575 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c10011bf0 con 0x1c13e90 2021-04-03T03:09:35.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c00ff9700 1 -- 172.21.2.111:0/47763575 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10011d50 con 0x1c13e90 2021-04-03T03:09:35.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c00ff9700 1 -- 172.21.2.111:0/47763575 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c100066e0 con 0x1c13e90 2021-04-03T03:09:35.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- 172.21.2.111:0/47763575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 msgr2=0x1d7b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:09:35.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 --2- 172.21.2.111:0/47763575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d7b170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x1d730e0 tx=0x7f9c1001c000).stop 2021-04-03T03:09:35.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- 172.21.2.111:0/47763575 shutdown_connections 2021-04-03T03:09:35.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 --2- 172.21.2.111:0/47763575 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1d2b390 0x1d7b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 --2- 172.21.2.111:0/47763575 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1b93800 0x1d74000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 --2- 172.21.2.111:0/47763575 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d7b170 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:09:35.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- 172.21.2.111:0/47763575 >> 172.21.2.111:0/47763575 conn(0x1d5abf0 msgr2=0x1d715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:09:35.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- 172.21.2.111:0/47763575 shutdown_connections 2021-04-03T03:09:35.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.403+0000 7f9c30867740 1 -- 172.21.2.111:0/47763575 wait complete. 2021-04-03T03:09:35.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 -- start start 2021-04-03T03:09:35.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 Processor -- start 2021-04-03T03:09:35.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 0x1e0f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d77f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 0x1d78460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e0eab0 con 0x1c13e90 2021-04-03T03:09:35.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e0ed50 con 0x1b93800 2021-04-03T03:09:35.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dabea0 con 0x1d2b390 2021-04-03T03:09:35.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 10 client.0 ll_register_callbacks cb 0x1d7d170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:09:35.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d77f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d77f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51156/0 (socket says 172.21.2.111:51156) 2021-04-03T03:09:35.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 -- 172.21.2.111:0/1501166296 learned_addr learned my addr 172.21.2.111:0/1501166296 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:09:35.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 -- 172.21.2.111:0/1501166296 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 msgr2=0x1d78460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 0x1d78460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 -- 172.21.2.111:0/1501166296 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 msgr2=0x1e0f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:09:35.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 0x1e0f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:09:35.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 -- 172.21.2.111:0/1501166296 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b4ee50 con 0x1c13e90 2021-04-03T03:09:35.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1affd700 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d77f70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x1d730e0 tx=0x7f9c1002d040).ready entity=mon.0 client_cookie=7c8413cb9451a9f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:09:35.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9bfb7fe700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:09:35.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9bfb7fe700 1 -- 172.21.2.111:0/1501166296 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c100066e0 con 0x1c13e90 2021-04-03T03:09:35.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9bfb7fe700 1 -- 172.21.2.111:0/1501166296 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9c100229e0 con 0x1c13e90 2021-04-03T03:09:35.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1bd7710 con 0x1c13e90 2021-04-03T03:09:35.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9bfb7fe700 1 -- 172.21.2.111:0/1501166296 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c10022ba0 con 0x1c13e90 2021-04-03T03:09:35.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 10 client.5145 fetch_fsmap learned FSMap version 11 2021-04-03T03:09:35.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x1d74740 con 0x1c13e90 2021-04-03T03:09:35.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9bfb7fe700 1 -- 172.21.2.111:0/1501166296 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1bd7710 con 0x1c13e90 2021-04-03T03:09:35.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9bfb7fe700 1 -- 172.21.2.111:0/1501166296 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 11) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f9c1003d030 con 0x1c13e90 2021-04-03T03:09:35.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 10 client.5145 fetch_fsmap finished waiting for FSMap version 11 2021-04-03T03:09:35.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 10 client.5145 resolve_mds: resolved ID 'a' to GID 4881 2021-04-03T03:09:35.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 20 client.5145 populate_metadata read hostname 'gibba011' 2021-04-03T03:09:35.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1db2320 0x1d78950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:09:35.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 4 client.5145 mds_command: new command op to 4881 tid=0[{"prefix": "cache drop"}] 2021-04-03T03:09:35.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 --> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1bae420 con 0x1db2320 2021-04-03T03:09:35.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1b7fe700 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1db2320 0x1d78950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:09:35.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9bfb7fe700 10 client.5145 ms_handle_connect on v2:172.21.2.111:6834/1150523943 2021-04-03T03:09:35.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:09:35.407+0000 7f9c1b7fe700 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1db2320 0x1d78950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=784bb90a8595a02c server_cookie=6a3e028ff13e22d9 in_seq=0 out_seq=0 2021-04-03T03:09:57.840 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:57.843 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:57.846 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:57.849 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:09:57.852 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:19.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9bfb7fe700 1 -- 172.21.2.111:0/1501166296 <== mds.0 v2:172.21.2.111:6834/1150523943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x1bae420 con 0x1db2320 2021-04-03T03:10:19.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9bfb7fe700 10 client.5145 handle_command_reply: tid=0 2021-04-03T03:10:19.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 client.5145 shutdown 2021-04-03T03:10:19.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 msgr2=0x1d77f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:19.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d77f70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x1d730e0 tx=0x7f9c1002d040).stop 2021-04-03T03:10:19.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 shutdown_connections 2021-04-03T03:10:19.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1d2b390 0x1d78460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:19.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1b93800 0x1e0f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:19.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:6834/1150523943,v1:172.21.2.111:6835/1150523943] conn(0x1db2320 0x1d78950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:19.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 --2- 172.21.2.111:0/1501166296 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1c13e90 0x1d77f70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:19.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9bfb7fe700 0 client.5145 ms_handle_reset on v2:172.21.2.111:6834/1150523943 2021-04-03T03:10:19.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 >> 172.21.2.111:0/1501166296 conn(0x1d5abf0 msgr2=0x1d737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:19.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 shutdown_connections 2021-04-03T03:10:19.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 1 -- 172.21.2.111:0/1501166296 wait complete. 2021-04-03T03:10:19.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c30867740 20 client.5145 trim_cache size 0 max 16384 2021-04-03T03:10:19.786 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:10:19.786 INFO:teuthology.orchestra.run.gibba011.stdout: "client_recall": { 2021-04-03T03:10:19.787 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": 0, 2021-04-03T03:10:19.787 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "(0) Success", 2021-04-03T03:10:19.787 INFO:teuthology.orchestra.run.gibba011.stdout: "recalled": 920 2021-04-03T03:10:19.788 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:19.788 INFO:teuthology.orchestra.run.gibba011.stdout: "flush_journal": { 2021-04-03T03:10:19.789 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": 0, 2021-04-03T03:10:19.789 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "" 2021-04-03T03:10:19.789 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:19.790 INFO:teuthology.orchestra.run.gibba011.stdout: "trim_cache": { 2021-04-03T03:10:19.790 INFO:teuthology.orchestra.run.gibba011.stdout: "trimmed": 900 2021-04-03T03:10:19.791 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:19.791 INFO:teuthology.orchestra.run.gibba011.stdout: "cache": { 2021-04-03T03:10:19.791 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": { 2021-04-03T03:10:19.792 INFO:teuthology.orchestra.run.gibba011.stdout: "items": 2519, 2021-04-03T03:10:19.792 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes": 431872 2021-04-03T03:10:19.793 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:19.793 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:19.793 INFO:teuthology.orchestra.run.gibba011.stdout: "duration": 44.358668831999999 2021-04-03T03:10:19.794 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:10:19.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9bfc057050 msgr2=0x7f9bfc059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:19.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9bfc057050 0x7f9bfc059510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f9c10001f60 tx=0x7f9c100077d0).stop 2021-04-03T03:10:19.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0603f0 msgr2=0x7f9c1c0b8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:19.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0603f0 0x7f9c1c0b8d70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f9c0c000b80 tx=0x7f9c0c007bd0).stop 2021-04-03T03:10:19.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 shutdown_connections 2021-04-03T03:10:19.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9bfc057050 0x7f9bfc059510 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:19.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9c1c0b5620 0x7f9c1c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:19.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9c1c0603f0 0x7f9c1c0b8d70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:19.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 --2- 172.21.2.111:0/3724970985 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9c1c0b44a0 0x7f9c1c156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:19.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 >> 172.21.2.111:0/3724970985 conn(0x7f9c1c05c7d0 msgr2=0x7f9c1c142210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:19.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 shutdown_connections 2021-04-03T03:10:19.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:19.767+0000 7f9c20de9700 1 -- 172.21.2.111:0/3724970985 wait complete. 2021-04-03T03:10:19.802 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:10:19.804 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:19.826 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:19.834 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:19.847 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:19.860 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:19.873 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:10:19.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:19.875 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:19.875 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:19.896 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:10:19.897 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:10:19.897 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:10:19.898 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:10:19.899 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:19.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[14947]: fuse finished with error 0 and tester_r 0 2021-04-03T03:10:19.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:19.973 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:19.974 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:20.018 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:10:20.019 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:10:20.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:10:20.084 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:20.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:20.090 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:20.091 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:10:20.100 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:10:20.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:10:20.103 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:10:20.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:10:20.105 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:10:20.152 INFO:teuthology.orchestra.run.gibba028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:10:20.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:10:20.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:20.154 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:20.154 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:10:20.160 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:10:20.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:10:20.161 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:10:20.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:10:20.162 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:10:20.212 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:10:20.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:10:20.213 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:10:20.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 Processor -- start 2021-04-03T03:10:20.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- start start 2021-04-03T03:10:20.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f35dc144ef0 0x7f35dc147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc0607d0 con 0x7f35dc0603f0 2021-04-03T03:10:20.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc14b090 con 0x7f35dc0add50 2021-04-03T03:10:20.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc14b330 con 0x7f35dc144ef0 2021-04-03T03:10:20.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35da59c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f35dc0603f0 0x7f35dc1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51956/0 (socket says 172.21.2.111:51956) 2021-04-03T03:10:20.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35da59c700 1 -- 172.21.2.111:0/1963293374 learned_addr learned my addr 172.21.2.111:0/1963293374 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:20.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387650365 0 0) 0x7f35dc0607d0 con 0x7f35dc0603f0 2021-04-03T03:10:20.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35bc003240 con 0x7f35dc0603f0 2021-04-03T03:10:20.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35dad9d700 1 --2- 172.21.2.111:0/1963293374 >> v2:172.21.2.115:3300/0 conn(0x7f35dc144ef0 0x7f35dc147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 527204093 0 0) 0x7f35bc003240 con 0x7f35dc0603f0 2021-04-03T03:10:20.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 >> v2:172.21.2.115:3300/0 conn(0x7f35dc144ef0 msgr2=0x7f35dc147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 --2- 172.21.2.111:0/1963293374 >> v2:172.21.2.115:3300/0 conn(0x7f35dc144ef0 0x7f35dc147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 >> v1:172.21.2.115:6789/0 conn(0x7f35dc0add50 legacy=0x7f35dc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc14b5d0 con 0x7f35dc0603f0 2021-04-03T03:10:20.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f35cc0016d0 con 0x7f35dc0603f0 2021-04-03T03:10:20.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 >> v1:172.21.2.111:6789/0 conn(0x7f35dc0603f0 legacy=0x7f35dc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35bc003730 0x7f35bc003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35bc0044e0 0x7f35bc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc004e50 0x7f35bc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc14b330 con 0x7f35bc0044e0 2021-04-03T03:10:20.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc004240 con 0x7f35bc003730 2021-04-03T03:10:20.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc00bc20 con 0x7f35bc004e50 2021-04-03T03:10:20.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d9d9b700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35bc0044e0 0x7f35bc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35da59c700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35bc003730 0x7f35bc003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35dad9d700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc004e50 0x7f35bc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d9d9b700 1 -- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc004e50 msgr2=0x7f35bc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d9d9b700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc004e50 0x7f35bc00b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d9d9b700 1 -- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35bc003730 msgr2=0x7f35bc003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d9d9b700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35bc003730 0x7f35bc003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d9d9b700 1 -- 172.21.2.111:0/1963293374 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc003240 con 0x7f35bc0044e0 2021-04-03T03:10:20.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35da59c700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35bc003730 0x7f35bc003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:20.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35dad9d700 1 -- 172.21.2.111:0/1963293374 reap_dead start 2021-04-03T03:10:20.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d9d9b700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35bc0044e0 0x7f35bc004960 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f35c4001f60 tx=0x7f35c4007b70).ready entity=mon.0 client_cookie=4adb9f0f05e1fab8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35c400c870 con 0x7f35bc0044e0 2021-04-03T03:10:20.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c400c9d0 con 0x7f35bc0044e0 2021-04-03T03:10:20.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35d959a700 1 -- 172.21.2.111:0/1963293374 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35c400cd30 con 0x7f35bc0044e0 2021-04-03T03:10:20.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- 172.21.2.111:0/1963293374 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35bc0044e0 msgr2=0x7f35bc004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35bc0044e0 0x7f35bc004960 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f35c4001f60 tx=0x7f35c4007b70).stop 2021-04-03T03:10:20.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- 172.21.2.111:0/1963293374 shutdown_connections 2021-04-03T03:10:20.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 --2- 172.21.2.111:0/1963293374 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35bc0044e0 0x7f35bc004960 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- 172.21.2.111:0/1963293374 >> 172.21.2.111:0/1963293374 conn(0x7f35dc0b61c0 msgr2=0x7f35dc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:20.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- 172.21.2.111:0/1963293374 shutdown_connections 2021-04-03T03:10:20.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- 172.21.2.111:0/1963293374 wait complete. 2021-04-03T03:10:20.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 Processor -- start 2021-04-03T03:10:20.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.267+0000 7f35db59e700 1 -- start start 2021-04-03T03:10:20.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc0044e0 0x7f35dc1452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35dc1570c0 0x7f35dc0a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35dc157540 0x7f35dc0a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0a75f0 con 0x7f35dc157540 2021-04-03T03:10:20.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0a7230 con 0x7f35dc1570c0 2021-04-03T03:10:20.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc1476e0 con 0x7f35bc0044e0 2021-04-03T03:10:20.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35dc157540 0x7f35dc0a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35dc157540 0x7f35dc0a7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51178/0 (socket says 172.21.2.111:51178) 2021-04-03T03:10:20.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 -- 172.21.2.111:0/242634148 learned_addr learned my addr 172.21.2.111:0/242634148 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:20.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 -- 172.21.2.111:0/242634148 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc0044e0 msgr2=0x7f35dc1452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc0044e0 0x7f35dc1452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 -- 172.21.2.111:0/242634148 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35dc1570c0 msgr2=0x7f35dc0a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35dc1570c0 0x7f35dc0a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 -- 172.21.2.111:0/242634148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc147980 con 0x7f35dc157540 2021-04-03T03:10:20.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35dad9d700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35dc157540 0x7f35dc0a7fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f35d40011d0 tx=0x7f35d4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35d37fe700 1 -- 172.21.2.111:0/242634148 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d40135e0 con 0x7f35dc157540 2021-04-03T03:10:20.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35d37fe700 1 -- 172.21.2.111:0/242634148 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d4013740 con 0x7f35dc157540 2021-04-03T03:10:20.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc0a6f20 con 0x7f35dc157540 2021-04-03T03:10:20.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35d37fe700 1 -- 172.21.2.111:0/242634148 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d4013aa0 con 0x7f35dc157540 2021-04-03T03:10:20.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc148480 con 0x7f35dc157540 2021-04-03T03:10:20.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc147d40 con 0x7f35dc157540 2021-04-03T03:10:20.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35d37fe700 1 -- 172.21.2.111:0/242634148 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f35d401a030 con 0x7f35dc157540 2021-04-03T03:10:20.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35d37fe700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f35c0056f20 0x7f35c00593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35d37fe700 1 -- 172.21.2.111:0/242634148 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f35d406f9d0 con 0x7f35dc157540 2021-04-03T03:10:20.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35da59c700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f35c0056f20 0x7f35c00593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.271+0000 7f35da59c700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f35c0056f20 0x7f35c00593e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f35cc008000 tx=0x7f35cc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.275+0000 7f35d37fe700 1 -- 172.21.2.111:0/242634148 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f35d4044dc0 con 0x7f35dc157540 2021-04-03T03:10:20.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.367+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f35dc057660 con 0x7f35dc157540 2021-04-03T03:10:20.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35d37fe700 1 -- 172.21.2.111:0/242634148 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f35d40483e0 con 0x7f35dc157540 2021-04-03T03:10:20.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f35c0056f20 msgr2=0x7f35c00593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f35c0056f20 0x7f35c00593e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f35cc008000 tx=0x7f35cc009040).stop 2021-04-03T03:10:20.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35dc157540 msgr2=0x7f35dc0a7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35dc157540 0x7f35dc0a7fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f35d40011d0 tx=0x7f35d4008040).stop 2021-04-03T03:10:20.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 shutdown_connections 2021-04-03T03:10:20.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f35dc157540 0x7f35dc0a7fa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f35bc0044e0 0x7f35dc1452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f35c0056f20 0x7f35c00593e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 --2- 172.21.2.111:0/242634148 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f35dc1570c0 0x7f35dc0a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 >> 172.21.2.111:0/242634148 conn(0x7f35dc0b61c0 msgr2=0x7f35dc0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:20.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 shutdown_connections 2021-04-03T03:10:20.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.371+0000 7f35db59e700 1 -- 172.21.2.111:0/242634148 wait complete. 2021-04-03T03:10:20.389 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2021-04-03T03:10:20.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 Processor -- start 2021-04-03T03:10:20.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- start start 2021-04-03T03:10:20.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07880603f0 0x7f07880b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f078814b700 0x7f07880b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880b12b0 0x7f07880b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07880b0f60 con 0x7f07880603f0 2021-04-03T03:10:20.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07880607d0 con 0x7f078814b700 2021-04-03T03:10:20.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07880af400 con 0x7f07880b12b0 2021-04-03T03:10:20.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07880603f0 0x7f07880b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07880603f0 0x7f07880b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51186/0 (socket says 172.21.2.111:51186) 2021-04-03T03:10:20.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 -- 172.21.2.111:0/1793400724 learned_addr learned my addr 172.21.2.111:0/1793400724 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:20.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 -- 172.21.2.111:0/1793400724 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880b12b0 msgr2=0x7f07880b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 --2- 172.21.2.111:0/1793400724 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880b12b0 0x7f07880b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 -- 172.21.2.111:0/1793400724 >> v2:172.21.2.115:3300/0 conn(0x7f078814b700 msgr2=0x7f07880b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f0785d9b700 1 --2- 172.21.2.111:0/1793400724 >> v2:172.21.2.115:3300/0 conn(0x7f078814b700 0x7f07880b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 --2- 172.21.2.111:0/1793400724 >> v2:172.21.2.115:3300/0 conn(0x7f078814b700 0x7f07880b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 -- 172.21.2.111:0/1793400724 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880af6a0 con 0x7f07880603f0 2021-04-03T03:10:20.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f0785d9b700 1 --2- 172.21.2.111:0/1793400724 >> v2:172.21.2.115:3300/0 conn(0x7f078814b700 0x7f07880b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:20.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078659c700 1 --2- 172.21.2.111:0/1793400724 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880603f0 0x7f07880b0090 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f0778002f30 tx=0x7f0778006e90).ready entity=mon.0 client_cookie=d641e556a76b3c6a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078559a700 1 -- 172.21.2.111:0/1793400724 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0778009920 con 0x7f07880603f0 2021-04-03T03:10:20.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078559a700 1 -- 172.21.2.111:0/1793400724 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778009a80 con 0x7f07880603f0 2021-04-03T03:10:20.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078559a700 1 -- 172.21.2.111:0/1793400724 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0778009de0 con 0x7f07880603f0 2021-04-03T03:10:20.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- 172.21.2.111:0/1793400724 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880603f0 msgr2=0x7f07880b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 --2- 172.21.2.111:0/1793400724 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880603f0 0x7f07880b0090 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f0778002f30 tx=0x7f0778006e90).stop 2021-04-03T03:10:20.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- 172.21.2.111:0/1793400724 shutdown_connections 2021-04-03T03:10:20.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 --2- 172.21.2.111:0/1793400724 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880b12b0 0x7f07880b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 --2- 172.21.2.111:0/1793400724 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880603f0 0x7f07880b0090 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 --2- 172.21.2.111:0/1793400724 >> v2:172.21.2.115:3300/0 conn(0x7f078814b700 0x7f07880b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- 172.21.2.111:0/1793400724 >> 172.21.2.111:0/1793400724 conn(0x7f07880b61c0 msgr2=0x7f078805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:20.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- 172.21.2.111:0/1793400724 shutdown_connections 2021-04-03T03:10:20.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.435+0000 7f078759e700 1 -- 172.21.2.111:0/1793400724 wait complete. 2021-04-03T03:10:20.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 Processor -- start 2021-04-03T03:10:20.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 -- start start 2021-04-03T03:10:20.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880603f0 0x7f07880a7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880b12b0 0x7f0788159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f078814b700 0x7f0788064430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07880aad90 con 0x7f07880b12b0 2021-04-03T03:10:20.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788063fb0 con 0x7f078814b700 2021-04-03T03:10:20.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07880aafc0 con 0x7f07880603f0 2021-04-03T03:10:20.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880b12b0 0x7f0788159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880b12b0 0x7f0788159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51194/0 (socket says 172.21.2.111:51194) 2021-04-03T03:10:20.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 -- 172.21.2.111:0/3422205470 learned_addr learned my addr 172.21.2.111:0/3422205470 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:20.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 -- 172.21.2.111:0/3422205470 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880603f0 msgr2=0x7f07880a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880603f0 0x7f07880a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 -- 172.21.2.111:0/3422205470 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f078814b700 msgr2=0x7f0788064430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f078814b700 0x7f0788064430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 -- 172.21.2.111:0/3422205470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880a9d30 con 0x7f07880b12b0 2021-04-03T03:10:20.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f0785d9b700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880b12b0 0x7f0788159a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f0770007000 tx=0x7f0770008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f077f7fe700 1 -- 172.21.2.111:0/3422205470 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077000ce90 con 0x7f07880b12b0 2021-04-03T03:10:20.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f077f7fe700 1 -- 172.21.2.111:0/3422205470 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770007710 con 0x7f07880b12b0 2021-04-03T03:10:20.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f077f7fe700 1 -- 172.21.2.111:0/3422205470 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0770007a70 con 0x7f07880b12b0 2021-04-03T03:10:20.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07880ab470 con 0x7f07880b12b0 2021-04-03T03:10:20.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07880aa150 con 0x7f07880b12b0 2021-04-03T03:10:20.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07880ab780 con 0x7f07880b12b0 2021-04-03T03:10:20.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f077f7fe700 1 -- 172.21.2.111:0/3422205470 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0770007bd0 con 0x7f07880b12b0 2021-04-03T03:10:20.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f077f7fe700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f076c056fe0 0x7f076c0594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f077f7fe700 1 -- 172.21.2.111:0/3422205470 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f07700703d0 con 0x7f07880b12b0 2021-04-03T03:10:20.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078659c700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f076c056fe0 0x7f076c0594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.439+0000 7f078659c700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f076c056fe0 0x7f076c0594a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f0778002f30 tx=0x7f0778015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.443+0000 7f077f7fe700 1 -- 172.21.2.111:0/3422205470 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f07700457c0 con 0x7f07880b12b0 2021-04-03T03:10:20.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.571+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f0788064120 con 0x7f07880b12b0 2021-04-03T03:10:20.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f077f7fe700 1 -- 172.21.2.111:0/3422205470 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v84) v1 ==== 139+0+0 (secure 0 0 0) 0x7f0770048de0 con 0x7f07880b12b0 2021-04-03T03:10:20.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f076c056fe0 msgr2=0x7f076c0594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f076c056fe0 0x7f076c0594a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f0778002f30 tx=0x7f0778015000).stop 2021-04-03T03:10:20.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880b12b0 msgr2=0x7f0788159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880b12b0 0x7f0788159a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f0770007000 tx=0x7f0770008040).stop 2021-04-03T03:10:20.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 shutdown_connections 2021-04-03T03:10:20.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f076c056fe0 0x7f076c0594a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f078814b700 0x7f0788064430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07880603f0 0x7f07880a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 --2- 172.21.2.111:0/3422205470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07880b12b0 0x7f0788159a20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 >> 172.21.2.111:0/3422205470 conn(0x7f07880b61c0 msgr2=0x7f07880b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:20.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 shutdown_connections 2021-04-03T03:10:20.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.831+0000 7f078759e700 1 -- 172.21.2.111:0/3422205470 wait complete. 2021-04-03T03:10:20.848 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-04-03T03:10:20.849 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2021-04-03T03:10:20.849 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2021-04-03T03:10:20.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.895+0000 7f8b02e79700 1 Processor -- start 2021-04-03T03:10:20.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.895+0000 7f8b02e79700 1 -- start start 2021-04-03T03:10:20.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.895+0000 7f8b02e79700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8afc0603f0 0x7f8afc0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.895+0000 7f8b02e79700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc0af400 con 0x7f8afc14b700 2021-04-03T03:10:20.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.895+0000 7f8b02e79700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc0af6a0 con 0x7f8afc0b0c90 2021-04-03T03:10:20.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.895+0000 7f8b02e79700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc060860 con 0x7f8afc0603f0 2021-04-03T03:10:20.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.895+0000 7f8b01676700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8afc14b700 0x7f8afc0b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:51992/0 (socket says 172.21.2.111:51992) 2021-04-03T03:10:20.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01676700 1 -- 172.21.2.111:0/470424552 learned_addr learned my addr 172.21.2.111:0/470424552 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:20.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/470424552 >> v2:172.21.2.115:3300/0 conn(0x7f8afc0603f0 0x7f8afc0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675269839 0 0) 0x7f8afc0af400 con 0x7f8afc14b700 2021-04-03T03:10:20.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae4003240 con 0x7f8afc14b700 2021-04-03T03:10:20.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1321308844 0 0) 0x7f8ae4003240 con 0x7f8afc14b700 2021-04-03T03:10:20.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 >> v2:172.21.2.115:3300/0 conn(0x7f8afc0603f0 msgr2=0x7f8afc0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 --2- 172.21.2.111:0/470424552 >> v2:172.21.2.115:3300/0 conn(0x7f8afc0603f0 0x7f8afc0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 >> v1:172.21.2.115:6789/0 conn(0x7f8afc0b0c90 legacy=0x7f8afc147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0afb40 con 0x7f8afc14b700 2021-04-03T03:10:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8aec0016d0 con 0x7f8afc14b700 2021-04-03T03:10:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 >> v1:172.21.2.111:6789/0 conn(0x7f8afc14b700 legacy=0x7f8afc0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8ae4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ae40044e0 0x7f8ae4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ae4004e50 0x7f8ae400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc060860 con 0x7f8ae4003730 2021-04-03T03:10:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4004260 con 0x7f8ae40044e0 2021-04-03T03:10:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae400bc20 con 0x7f8ae4004e50 2021-04-03T03:10:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8ae4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 -- 172.21.2.111:0/470424552 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ae4004e50 msgr2=0x7f8ae400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ae4004e50 0x7f8ae400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 -- 172.21.2.111:0/470424552 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ae40044e0 msgr2=0x7f8ae4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:20.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ae40044e0 0x7f8ae4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 -- 172.21.2.111:0/470424552 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0afb40 con 0x7f8ae4003730 2021-04-03T03:10:20.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02678700 1 -- 172.21.2.111:0/470424552 reap_dead start 2021-04-03T03:10:20.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8ae4003b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f8af400c000 tx=0x7f8af40091d0).ready entity=mon.0 client_cookie=e5598a8f1dccf7ba server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af400ed00 con 0x7f8ae4003730 2021-04-03T03:10:20.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af400ee60 con 0x7f8ae4003730 2021-04-03T03:10:20.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b00e75700 1 -- 172.21.2.111:0/470424552 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af40125e0 con 0x7f8ae4003730 2021-04-03T03:10:20.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/470424552 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 msgr2=0x7f8ae4003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8ae4003b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f8af400c000 tx=0x7f8af40091d0).stop 2021-04-03T03:10:20.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/470424552 shutdown_connections 2021-04-03T03:10:20.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 --2- 172.21.2.111:0/470424552 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8ae4003b50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:20.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/470424552 >> 172.21.2.111:0/470424552 conn(0x7f8afc0b61c0 msgr2=0x7f8afc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:20.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/470424552 shutdown_connections 2021-04-03T03:10:20.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/470424552 wait complete. 2021-04-03T03:10:20.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 Processor -- start 2021-04-03T03:10:20.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- start start 2021-04-03T03:10:20.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8afc062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8afc156620 0x7f8afc067370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8afc0aa690 0x7f8afc067860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc0a74e0 con 0x7f8ae4003730 2021-04-03T03:10:20.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc0a7780 con 0x7f8afc156620 2021-04-03T03:10:20.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc0a9f80 con 0x7f8afc0aa690 2021-04-03T03:10:20.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8afc062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8afc062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51210/0 (socket says 172.21.2.111:51210) 2021-04-03T03:10:20.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 -- 172.21.2.111:0/1963579313 learned_addr learned my addr 172.21.2.111:0/1963579313 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:20.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 -- 172.21.2.111:0/1963579313 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8afc0aa690 msgr2=0x7f8afc067860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8afc0aa690 0x7f8afc067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 -- 172.21.2.111:0/1963579313 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8afc156620 msgr2=0x7f8afc067370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:20.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8afc156620 0x7f8afc067370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:20.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 -- 172.21.2.111:0/1963579313 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc067d50 con 0x7f8ae4003730 2021-04-03T03:10:20.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b01e77700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8afc062bf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f8af400c000 tx=0x7f8af401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8aeaffd700 1 -- 172.21.2.111:0/1963579313 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af40127f0 con 0x7f8ae4003730 2021-04-03T03:10:20.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8aeaffd700 1 -- 172.21.2.111:0/1963579313 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af4012950 con 0x7f8ae4003730 2021-04-03T03:10:20.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8aeaffd700 1 -- 172.21.2.111:0/1963579313 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af4012cb0 con 0x7f8ae4003730 2021-04-03T03:10:20.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc067f40 con 0x7f8ae4003730 2021-04-03T03:10:20.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc0abc90 con 0x7f8ae4003730 2021-04-03T03:10:20.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.899+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc057660 con 0x7f8ae4003730 2021-04-03T03:10:20.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.903+0000 7f8aeaffd700 1 -- 172.21.2.111:0/1963579313 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8af40243f0 con 0x7f8ae4003730 2021-04-03T03:10:20.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.903+0000 7f8aeaffd700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8adc056ea0 0x7f8adc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:20.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.903+0000 7f8aeaffd700 1 -- 172.21.2.111:0/1963579313 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8af40720a0 con 0x7f8ae4003730 2021-04-03T03:10:20.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.903+0000 7f8b01676700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8adc056ea0 0x7f8adc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:20.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.903+0000 7f8b01676700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8adc056ea0 0x7f8adc059360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f8aec0013e0 tx=0x7f8aec001330).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:20.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:20.903+0000 7f8aeaffd700 1 -- 172.21.2.111:0/1963579313 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8af40473b0 con 0x7f8ae4003730 2021-04-03T03:10:21.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.031+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f8afc14afe0 con 0x7f8ae4003730 2021-04-03T03:10:21.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8aeaffd700 1 -- 172.21.2.111:0/1963579313 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v85) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8af40160d0 con 0x7f8ae4003730 2021-04-03T03:10:21.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8adc056ea0 msgr2=0x7f8adc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:21.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8adc056ea0 0x7f8adc059360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f8aec0013e0 tx=0x7f8aec001330).stop 2021-04-03T03:10:21.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 msgr2=0x7f8afc062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:21.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8afc062bf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f8af400c000 tx=0x7f8af401a000).stop 2021-04-03T03:10:21.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 shutdown_connections 2021-04-03T03:10:21.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8afc0aa690 0x7f8afc067860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:21.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae4003730 0x7f8afc062bf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:21.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8adc056ea0 0x7f8adc059360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:21.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 --2- 172.21.2.111:0/1963579313 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8afc156620 0x7f8afc067370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:21.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 >> 172.21.2.111:0/1963579313 conn(0x7f8afc0b61c0 msgr2=0x7f8afc0b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:21.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 shutdown_connections 2021-04-03T03:10:21.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.835+0000 7f8b02e79700 1 -- 172.21.2.111:0/1963579313 wait complete. 2021-04-03T03:10:21.855 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:10:21.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b7fff700 1 Processor -- start 2021-04-03T03:10:21.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b7fff700 1 -- start start 2021-04-03T03:10:21.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 0x7f06b8147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b7fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b80607d0 con 0x7f06b8144ef0 2021-04-03T03:10:21.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b7fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b814b090 con 0x7f06b80603f0 2021-04-03T03:10:21.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b814b330 con 0x7f06b80add50 2021-04-03T03:10:21.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b77fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f06b8144ef0 0x7f06b8147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52014/0 (socket says 172.21.2.111:52014) 2021-04-03T03:10:21.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b77fe700 1 -- 172.21.2.111:0/2575875545 learned_addr learned my addr 172.21.2.111:0/2575875545 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:21.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725446455 0 0) 0x7f06b80607d0 con 0x7f06b8144ef0 2021-04-03T03:10:21.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0698003240 con 0x7f06b8144ef0 2021-04-03T03:10:21.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b67fc700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 0x7f06b8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3345357407 0 0) 0x7f0698003240 con 0x7f06b8144ef0 2021-04-03T03:10:21.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 msgr2=0x7f06b8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:21.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 0x7f06b8147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:21.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 >> v1:172.21.2.115:6789/0 conn(0x7f06b80603f0 legacy=0x7f06b81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:21.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b814b5d0 con 0x7f06b8144ef0 2021-04-03T03:10:21.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f06b0001a50 con 0x7f06b8144ef0 2021-04-03T03:10:21.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 >> v1:172.21.2.111:6789/0 conn(0x7f06b8144ef0 legacy=0x7f06b8147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:21.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 0x7f0698003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0698003cc0 0x7f06980044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06980049d0 0x7f0698004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b814b330 con 0x7f06980049d0 2021-04-03T03:10:21.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698004250 con 0x7f0698003cc0 2021-04-03T03:10:21.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069800b730 con 0x7f06b80add50 2021-04-03T03:10:21.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.903+0000 7f06b77fe700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06980049d0 0x7f0698004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0698003cc0 0x7f06980044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b6ffd700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 0x7f0698003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 -- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 msgr2=0x7f0698003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:21.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80add50 0x7f0698003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:21.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 -- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0698003cc0 msgr2=0x7f06980044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:21.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0698003cc0 0x7f06980044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:21.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 -- 172.21.2.111:0/2575875545 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0698003240 con 0x7f06980049d0 2021-04-03T03:10:21.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0698003cc0 0x7f06980044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:10:21.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06980049d0 0x7f0698004e50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f06b0001760 tx=0x7f06b814b760).ready entity=mon.0 client_cookie=c581d7a488e4de22 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:21.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b0005b80 con 0x7f06980049d0 2021-04-03T03:10:21.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b0005ce0 con 0x7f06980049d0 2021-04-03T03:10:21.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b5ffb700 1 -- 172.21.2.111:0/2575875545 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b00072c0 con 0x7f06980049d0 2021-04-03T03:10:21.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/2575875545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06980049d0 msgr2=0x7f0698004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:21.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 --2- 172.21.2.111:0/2575875545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06980049d0 0x7f0698004e50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f06b0001760 tx=0x7f06b814b760).stop 2021-04-03T03:10:21.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 -- 172.21.2.111:0/2575875545 reap_dead start 2021-04-03T03:10:21.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/2575875545 shutdown_connections 2021-04-03T03:10:21.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/2575875545 >> 172.21.2.111:0/2575875545 conn(0x7f06b80b61c0 msgr2=0x7f06b805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:21.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/2575875545 shutdown_connections 2021-04-03T03:10:21.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/2575875545 wait complete. 2021-04-03T03:10:21.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 Processor -- start 2021-04-03T03:10:21.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- start start 2021-04-03T03:10:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06b80b3b90 0x7f06b80a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06b80a9700 0x7f06b815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80a85e0 0x7f06b815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b815ac50 con 0x7f06b80a9700 2021-04-03T03:10:21.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80a8ca0 con 0x7f06b80b3b90 2021-04-03T03:10:21.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b8063960 con 0x7f06b80a85e0 2021-04-03T03:10:21.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06b80a9700 0x7f06b815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06b80a9700 0x7f06b815a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51232/0 (socket says 172.21.2.111:51232) 2021-04-03T03:10:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 -- 172.21.2.111:0/3822417766 learned_addr learned my addr 172.21.2.111:0/3822417766 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b6ffd700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06b80b3b90 0x7f06b80a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b77fe700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80a85e0 0x7f06b815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 -- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80a85e0 msgr2=0x7f06b815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80a85e0 0x7f06b815a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 -- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06b80b3b90 msgr2=0x7f06b80a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06b80b3b90 0x7f06b80a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:21.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 -- 172.21.2.111:0/3822417766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b80636c0 con 0x7f06b80a9700 2021-04-03T03:10:21.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b6ffd700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06b80b3b90 0x7f06b80a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:10:21.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b67fc700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06b80a9700 0x7f06b815a270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f06b815ad90 tx=0x7f06a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:21.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06affff700 1 -- 172.21.2.111:0/3822417766 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a000e840 con 0x7f06b80a9700 2021-04-03T03:10:21.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06affff700 1 -- 172.21.2.111:0/3822417766 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a000e9a0 con 0x7f06b80a9700 2021-04-03T03:10:21.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b8159dd0 con 0x7f06b80a9700 2021-04-03T03:10:21.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06affff700 1 -- 172.21.2.111:0/3822417766 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a000ed00 con 0x7f06b80a9700 2021-04-03T03:10:21.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b8155b10 con 0x7f06b80a9700 2021-04-03T03:10:21.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.907+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b8159f60 con 0x7f06b80a9700 2021-04-03T03:10:21.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.911+0000 7f06affff700 1 -- 172.21.2.111:0/3822417766 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f06a0024070 con 0x7f06b80a9700 2021-04-03T03:10:21.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.911+0000 7f06affff700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f069c056f20 0x7f069c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:21.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.911+0000 7f06affff700 1 -- 172.21.2.111:0/3822417766 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f06a0070990 con 0x7f06b80a9700 2021-04-03T03:10:21.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.911+0000 7f06b6ffd700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f069c056f20 0x7f069c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:21.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.911+0000 7f06b6ffd700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f069c056f20 0x7f069c0593e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f06a800d000 tx=0x7f06a8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:21.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:21.911+0000 7f06affff700 1 -- 172.21.2.111:0/3822417766 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f06a0045d80 con 0x7f06b80a9700 2021-04-03T03:10:22.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.003+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f06b8057660 con 0x7f06b80a9700 2021-04-03T03:10:22.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.007+0000 7f06affff700 1 -- 172.21.2.111:0/3822417766 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f06a00493a0 con 0x7f06b80a9700 2021-04-03T03:10:22.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f069c056f20 msgr2=0x7f069c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f069c056f20 0x7f069c0593e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f06a800d000 tx=0x7f06a8009040).stop 2021-04-03T03:10:22.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06b80a9700 msgr2=0x7f06b815a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06b80a9700 0x7f06b815a270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f06b815ad90 tx=0x7f06a0008040).stop 2021-04-03T03:10:22.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 shutdown_connections 2021-04-03T03:10:22.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06b80a85e0 0x7f06b815a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f069c056f20 0x7f069c0593e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06b80b3b90 0x7f06b80a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 --2- 172.21.2.111:0/3822417766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f06b80a9700 0x7f06b815a270 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 >> 172.21.2.111:0/3822417766 conn(0x7f06b80b61c0 msgr2=0x7f06b8144510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:22.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 shutdown_connections 2021-04-03T03:10:22.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.011+0000 7f06b7fff700 1 -- 172.21.2.111:0/3822417766 wait complete. 2021-04-03T03:10:22.028 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:10:22.028 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:22.036 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:22.061 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:22.074 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:22.087 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:10:22.101 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:10:22.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:22.102 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:22.103 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:22.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:10:22.124 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:10:22.124 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:10:22.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:22.126 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:22.127 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:10:22.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:10:22.133 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:10:22.134 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:10:22.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:22.135 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:22.136 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:10:22.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:10:22.149 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:10:22.150 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:10:22.150 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:10:22.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 Processor -- start 2021-04-03T03:10:22.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 -- start start 2021-04-03T03:10:22.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f89b414b700 0x7f89b40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f89b40b0c90 0x7f89b4147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40af400 con 0x7f89b40603f0 2021-04-03T03:10:22.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b4060860 con 0x7f89b414b700 2021-04-03T03:10:22.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40af6a0 con 0x7f89b40b0c90 2021-04-03T03:10:22.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f89b414b700 0x7f89b40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f89b414b700 0x7f89b40b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51240/0 (socket says 172.21.2.111:51240) 2021-04-03T03:10:22.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 -- 172.21.2.111:0/2935343974 learned_addr learned my addr 172.21.2.111:0/2935343974 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:22.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 -- 172.21.2.111:0/2935343974 >> v2:172.21.2.115:3300/0 conn(0x7f89b40b0c90 msgr2=0x7f89b4147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:22.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 --2- 172.21.2.111:0/2935343974 >> v2:172.21.2.115:3300/0 conn(0x7f89b40b0c90 0x7f89b4147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 -- 172.21.2.111:0/2935343974 >> v1:172.21.2.111:6789/0 conn(0x7f89b40603f0 legacy=0x7f89b40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 -- 172.21.2.111:0/2935343974 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40afb40 con 0x7f89b414b700 2021-04-03T03:10:22.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1b83700 1 --2- 172.21.2.111:0/2935343974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b40b07a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f89a4002f30 tx=0x7f89a4006e90).ready entity=mon.0 client_cookie=c7efa00a6a2d001c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1382700 1 -- 172.21.2.111:0/2935343974 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89a40099c0 con 0x7f89b414b700 2021-04-03T03:10:22.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1382700 1 -- 172.21.2.111:0/2935343974 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4009b20 con 0x7f89b414b700 2021-04-03T03:10:22.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b1382700 1 -- 172.21.2.111:0/2935343974 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89a4009e80 con 0x7f89b414b700 2021-04-03T03:10:22.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 -- 172.21.2.111:0/2935343974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 msgr2=0x7f89b40b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.207+0000 7f89b3386700 1 --2- 172.21.2.111:0/2935343974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b40b07a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f89a4002f30 tx=0x7f89a4006e90).stop 2021-04-03T03:10:22.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- 172.21.2.111:0/2935343974 shutdown_connections 2021-04-03T03:10:22.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 --2- 172.21.2.111:0/2935343974 >> v2:172.21.2.115:3300/0 conn(0x7f89b40b0c90 0x7f89b4147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 --2- 172.21.2.111:0/2935343974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b40b07a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- 172.21.2.111:0/2935343974 >> 172.21.2.111:0/2935343974 conn(0x7f89b40b61c0 msgr2=0x7f89b405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:22.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- 172.21.2.111:0/2935343974 shutdown_connections 2021-04-03T03:10:22.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- 172.21.2.111:0/2935343974 wait complete. 2021-04-03T03:10:22.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 Processor -- start 2021-04-03T03:10:22.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- start start 2021-04-03T03:10:22.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f89b40603f0 0x7f89b41573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89b40b0c90 0x7f89b4062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b4067070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4156810 con 0x7f89b414b700 2021-04-03T03:10:22.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4156a00 con 0x7f89b40603f0 2021-04-03T03:10:22.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4156ca0 con 0x7f89b40b0c90 2021-04-03T03:10:22.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b4067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b4067070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51248/0 (socket says 172.21.2.111:51248) 2021-04-03T03:10:22.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 -- 172.21.2.111:0/2014945216 learned_addr learned my addr 172.21.2.111:0/2014945216 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:22.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 -- 172.21.2.111:0/2014945216 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89b40b0c90 msgr2=0x7f89b4062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89b40b0c90 0x7f89b4062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 -- 172.21.2.111:0/2014945216 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f89b40603f0 msgr2=0x7f89b41573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f89b40603f0 0x7f89b41573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 -- 172.21.2.111:0/2014945216 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40ab850 con 0x7f89b414b700 2021-04-03T03:10:22.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b2b85700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b4067070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f89ac0032b0 tx=0x7f89ac009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89ab7fe700 1 -- 172.21.2.111:0/2014945216 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ac00ca60 con 0x7f89b414b700 2021-04-03T03:10:22.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89ab7fe700 1 -- 172.21.2.111:0/2014945216 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac00cbc0 con 0x7f89b414b700 2021-04-03T03:10:22.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89ab7fe700 1 -- 172.21.2.111:0/2014945216 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ac015420 con 0x7f89b414b700 2021-04-03T03:10:22.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b40a9c90 con 0x7f89b414b700 2021-04-03T03:10:22.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b40aa220 con 0x7f89b414b700 2021-04-03T03:10:22.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b40a9eb0 con 0x7f89b414b700 2021-04-03T03:10:22.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89ab7fe700 1 -- 172.21.2.111:0/2014945216 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f89ac01c070 con 0x7f89b414b700 2021-04-03T03:10:22.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89ab7fe700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8998056f70 0x7f8998059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.211+0000 7f89ab7fe700 1 -- 172.21.2.111:0/2014945216 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f89ac06fda0 con 0x7f89b414b700 2021-04-03T03:10:22.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.215+0000 7f89b2384700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8998056f70 0x7f8998059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.215+0000 7f89b2384700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8998056f70 0x7f8998059430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f89b406a3b0 tx=0x7f899c0077b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.215+0000 7f89ab7fe700 1 -- 172.21.2.111:0/2014945216 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f89ac045130 con 0x7f89b414b700 2021-04-03T03:10:22.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.307+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f89b4057660 con 0x7f89b414b700 2021-04-03T03:10:22.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.307+0000 7f89ab7fe700 1 -- 172.21.2.111:0/2014945216 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v33) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f89ac01a030 con 0x7f89b414b700 2021-04-03T03:10:22.321 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:22.322 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:10:22.322 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 33, 2021-04-03T03:10:22.323 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:10:22.323 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:10:22.324 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:09:15.086914+0000", 2021-04-03T03:10:22.324 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:10:22.325 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:10:22.325 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:10:22.326 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:10:22.326 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:10:22.326 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:10:22.327 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:10:22.327 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:10:22.328 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:10:22.328 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:10:22.329 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:10:22.329 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:10:22.330 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:10:22.330 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:10:22.331 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:10:22.331 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 6, 2021-04-03T03:10:22.331 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:10:22.332 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:10:22.332 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:10:22.333 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:10:22.333 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:10:22.334 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.334 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:10:22.335 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:10:22.335 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:10:22.336 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:10:22.336 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:10:22.336 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:10:22.337 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:10:22.337 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:10:22.338 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:10:22.338 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:10:22.338 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:10:22.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:10:22.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:10:22.340 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:10:22.340 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:10:22.341 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:10:22.341 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:10:22.341 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:10:22.342 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:10:22.342 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:10:22.343 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:10:22.343 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:10:22.344 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:10:22.344 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.344 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:10:22.345 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:10:22.345 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:10:22.346 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:10:22.346 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:10:22.347 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:10:22.347 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:10:22.348 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:10:22.348 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:10:22.348 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:10:22.349 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:10:22.349 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:10:22.350 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:10:22.350 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:10:22.351 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:10:22.351 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:10:22.351 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:10:22.352 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:10:22.352 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:10:22.353 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:10:22.353 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:10:22.353 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:10:22.354 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:10:22.354 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:10:22.355 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:10:22.355 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:10:22.356 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:10:22.356 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.356 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:10:22.357 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:10:22.357 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:10:22.358 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:10:22.358 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:10:22.358 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:10:22.359 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:10:22.359 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:10:22.360 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:10:22.360 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:10:22.360 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:10:22.361 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:10:22.361 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:10:22.361 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.362 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:10:22.362 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:10:22.363 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.363 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:10:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:10:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:10:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:10:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:10:22.366 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:10:22.366 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:10:22.367 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:10:22.367 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:10:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:10:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:10:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:10:22.369 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:10:22.369 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:10:22.370 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:10:22.370 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:10:22.370 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:10:22.371 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:10:22.371 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:10:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:10:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:10:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:10:22.373 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:10:22.373 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:10:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:10:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:10:22.375 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:10:22.375 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:10:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:10:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:10:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:10:22.377 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:10:22.377 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:10:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:10:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:10:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:10:22.379 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:10:22.379 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:10:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:10:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:10:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:10:22.381 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:10:22.381 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:10:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:10:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:10:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:10:22.383 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:10:22.383 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:10:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:10:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:10:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:10:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:10:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:10:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:10:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:10:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:10:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:10:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:10:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:10:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:10:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:10:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:10:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:10:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:10:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 5, 2021-04-03T03:10:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:10:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:09:12.356149+0000", 2021-04-03T03:10:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:10:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:10:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:10:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:10:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:10:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:10:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:10:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:10:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:10:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:10:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:10:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:10:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:10:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:10:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:10:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:10:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:10:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:10:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:10:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:10:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "32", 2021-04-03T03:10:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:10:22.394 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:10:22.394 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:10:22.394 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:10:22.394 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:10:22.394 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:10:22.395 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:10:22.395 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:10:22.395 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:10:22.395 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:10:22.396 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:10:22.396 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:10:22.396 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:10:22.396 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:10:22.396 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:10:22.397 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:10:22.397 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:10:22.397 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:10:22.397 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:10:22.397 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:10:22.398 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:10:22.398 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:10:22.398 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:10:22.398 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:10:22.398 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:10:22.399 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:10:22.399 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.399 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:10:22.399 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:10:22.400 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:10:22.400 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:10:22.400 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:10:22.400 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:10:22.400 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:10:22.401 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:10:22.401 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:10:22.401 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:10:22.401 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:10:22.401 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:10:22.402 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:10:22.402 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:10:22.402 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:10:22.402 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.402 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:10:22.403 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:10:22.403 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:10:22.403 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.403 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.404 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.404 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.404 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 6, 2021-04-03T03:10:22.404 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:10:22.404 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:09:13.051346+0000", 2021-04-03T03:10:22.405 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:10:22.405 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:10:22.405 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:10:22.405 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:10:22.406 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:10:22.406 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:10:22.406 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:10:22.406 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:10:22.406 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:10:22.407 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:10:22.407 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:10:22.407 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:10:22.407 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:10:22.408 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:10:22.408 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:10:22.408 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:10:22.408 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:10:22.408 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:10:22.409 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:10:22.409 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:10:22.409 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.409 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "33", 2021-04-03T03:10:22.410 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:10:22.410 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:10:22.410 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:10:22.410 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:10:22.411 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:10:22.411 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:10:22.411 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:10:22.411 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:10:22.412 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:10:22.412 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:10:22.412 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:10:22.412 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:10:22.413 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:10:22.413 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:10:22.413 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:10:22.413 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:10:22.414 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:10:22.414 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:10:22.414 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:10:22.414 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:10:22.415 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:10:22.415 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:10:22.415 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:10:22.415 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:10:22.416 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:10:22.416 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:10:22.416 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.416 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:10:22.416 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:10:22.417 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:10:22.417 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:10:22.417 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:10:22.417 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:10:22.418 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:10:22.418 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:10:22.418 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:10:22.418 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:10:22.418 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:10:22.419 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:10:22.419 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:10:22.419 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:10:22.419 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:10:22.420 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.420 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.420 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.420 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:10:22.420 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:10:22.421 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.421 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:10:22.421 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:10:22.422 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.422 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.422 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.422 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.422 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.423 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.423 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.423 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 31, 2021-04-03T03:10:22.423 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.424 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.424 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.424 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.424 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.424 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.425 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:10:22.425 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.425 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.425 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.425 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.426 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:10:22.426 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.426 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.426 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.427 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.427 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.427 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.427 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.427 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.428 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:10:22.428 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.428 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.429 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.429 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.429 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:10:22.429 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.429 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.430 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.430 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.430 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.430 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.431 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.431 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.431 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:10:22.431 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.431 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.432 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.432 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.432 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:10:22.432 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.432 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.433 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.433 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.433 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.433 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.434 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.434 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.434 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:10:22.434 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.434 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.435 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.435 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.435 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:10:22.435 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:10:22.436 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.436 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.436 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.436 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:10:22.437 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:10:22.437 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:10:22.437 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:10:22.437 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.437 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.438 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.438 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.438 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.438 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.438 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:10:22.439 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:10:22.439 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.439 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.439 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.440 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.440 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.440 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.440 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.440 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 31, 2021-04-03T03:10:22.441 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.441 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.441 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.441 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.441 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.442 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.442 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:10:22.442 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.443 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.443 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.443 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.443 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:10:22.444 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.444 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.444 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.444 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.444 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.445 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.445 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.445 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.445 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:10:22.446 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.446 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.446 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.446 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.446 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:10:22.447 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.447 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.447 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.447 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.447 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.448 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.448 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.448 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.449 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:10:22.449 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.449 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.449 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.449 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.450 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:10:22.450 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.450 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.450 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.451 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.451 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.451 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.451 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.451 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.452 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:10:22.452 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.452 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.452 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.452 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.453 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:10:22.453 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:10:22.453 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.453 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.454 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.454 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:10:22.454 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:10:22.454 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:10:22.454 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:10:22.455 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.455 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.455 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.456 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.456 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.456 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.456 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:10:22.456 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:10:22.457 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.457 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.457 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.457 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.457 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.458 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.458 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.458 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 30, 2021-04-03T03:10:22.458 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.458 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.459 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.459 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.459 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.459 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.460 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:10:22.460 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.460 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.460 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.460 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.461 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:10:22.461 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.461 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.461 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.462 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.462 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.462 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.462 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.463 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.463 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:10:22.463 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.463 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.464 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.464 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.464 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:10:22.464 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.464 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.465 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.465 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.465 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.465 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.466 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.466 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.466 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:10:22.466 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.466 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.467 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.467 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.467 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:10:22.467 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.468 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.468 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.468 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.468 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.468 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.469 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.469 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.469 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:10:22.470 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.470 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.470 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.470 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.470 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:10:22.471 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:10:22.471 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.471 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.471 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.471 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:10:22.472 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:10:22.472 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:10:22.472 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:10:22.472 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.473 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.473 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.473 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.473 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.474 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:10:22.474 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:10:22.474 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.474 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.474 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.475 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.475 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.475 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.475 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.476 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 31, 2021-04-03T03:10:22.476 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.476 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.477 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.477 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.477 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.477 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.477 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:10:22.478 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.478 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.478 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.478 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.479 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:10:22.479 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.479 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.479 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.479 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.480 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.480 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.480 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.480 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.481 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:10:22.481 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.481 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.481 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.481 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.482 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:10:22.482 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.482 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.482 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.483 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.483 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.483 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.484 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.484 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.484 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:10:22.484 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.484 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.485 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.485 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.485 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:10:22.485 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.486 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.486 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.486 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.486 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.486 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.487 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.487 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.487 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:10:22.487 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.488 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.488 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.488 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.488 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:10:22.488 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:10:22.489 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.489 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.489 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.490 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:10:22.490 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:10:22.490 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:10:22.490 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:10:22.491 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.491 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.491 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.491 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.492 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.492 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:10:22.492 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:10:22.492 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.493 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.493 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.493 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.493 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.493 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.494 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.494 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 31, 2021-04-03T03:10:22.494 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.494 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.495 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.495 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.495 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.495 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.495 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:10:22.496 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.496 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.496 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.497 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.497 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:10:22.497 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.497 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.497 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.498 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.498 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.498 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.498 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.499 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.499 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:10:22.499 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.499 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.499 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.500 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.500 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:10:22.500 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.500 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.500 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.501 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.501 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.501 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.501 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.502 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.502 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:10:22.502 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.502 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.502 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.503 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.503 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:10:22.503 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.504 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.504 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.504 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.504 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.505 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.505 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.505 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.505 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:10:22.505 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.506 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.506 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.506 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.506 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:10:22.506 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:10:22.507 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.507 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.507 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.507 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:10:22.507 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:10:22.508 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:10:22.508 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:10:22.508 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.508 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.509 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.509 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.509 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.509 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.510 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:10:22.510 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:10:22.510 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.510 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.511 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.511 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.511 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.511 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.511 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.512 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 31, 2021-04-03T03:10:22.512 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.512 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.512 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.513 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.513 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.513 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.513 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:10:22.513 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.514 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.514 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.514 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.514 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:10:22.514 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.515 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.515 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.515 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.515 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.516 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.516 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.516 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.516 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:10:22.516 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.517 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.517 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.517 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.517 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:10:22.517 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.518 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.518 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.518 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.518 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.519 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.519 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.519 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.519 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:10:22.519 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.520 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.520 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.520 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.520 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:10:22.521 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.521 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.521 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.521 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.521 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.522 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.522 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.522 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.522 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:10:22.523 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.523 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.523 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.523 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.523 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:10:22.524 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:10:22.524 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.524 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.524 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.524 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:10:22.525 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:10:22.525 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:10:22.525 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:10:22.525 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.526 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.526 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.526 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.526 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.526 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.527 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:10:22.527 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:10:22.527 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.527 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.527 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.528 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.528 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.528 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.528 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.528 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 30, 2021-04-03T03:10:22.529 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.529 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.529 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.529 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.530 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.530 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.530 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:10:22.530 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.530 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.531 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.531 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.531 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:10:22.531 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.531 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.532 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.532 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.532 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.532 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.532 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.533 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.533 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:10:22.533 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.533 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.533 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.534 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.534 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:10:22.534 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.534 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.535 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.535 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.535 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.535 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.536 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.536 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.536 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:10:22.536 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.536 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.537 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.537 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.537 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:10:22.537 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.538 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.538 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.538 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.538 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.538 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.539 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.539 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.539 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:10:22.539 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.539 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.540 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.540 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.540 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:10:22.540 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:10:22.540 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.541 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.541 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.541 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:10:22.541 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:10:22.542 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:10:22.542 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:10:22.542 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.542 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.542 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.543 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.543 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.543 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.543 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:10:22.543 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:10:22.544 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:10:22.544 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:10:22.544 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:10:22.544 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:10:22.544 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:10:22.545 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:10:22.545 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:10:22.545 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 31, 2021-04-03T03:10:22.545 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:10:22.546 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:10:22.546 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:10:22.546 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.546 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.546 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.547 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:10:22.547 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.547 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.547 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.547 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.548 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:10:22.548 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.548 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.548 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.548 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.549 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:10:22.549 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.549 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.549 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.550 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:10:22.550 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.550 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.550 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.550 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.551 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:10:22.551 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.551 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.551 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.551 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.552 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:10:22.552 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.552 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.552 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.553 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:10:22.553 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.553 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.553 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.553 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.554 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:10:22.554 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.554 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.554 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.554 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.555 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:10:22.555 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:10:22.555 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.555 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:10:22.555 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:10:22.556 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.556 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.556 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.556 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:10:22.557 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:10:22.557 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:10:22.557 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.557 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.557 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.558 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:10:22.558 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:10:22.558 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:10:22.558 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:10:22.558 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:10:22.559 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:10:22.559 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:10:22.559 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:22.559 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.559 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:10:22.560 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:10:22.560 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.560 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:10:22.560 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.560 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.561 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.561 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.561 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.561 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:10:22.562 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.562 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.562 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.562 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:10:22.562 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.563 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.563 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.563 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.563 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.563 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:10:22.564 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.564 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.564 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.564 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:10:22.565 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.565 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.565 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.565 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.565 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.566 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:10:22.566 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.566 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.566 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.566 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:10:22.567 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.567 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.567 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.567 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.568 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.568 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:10:22.568 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.568 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.568 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.569 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:10:22.569 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.569 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.569 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.569 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.570 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.570 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:10:22.570 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.570 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.570 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.571 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:10:22.571 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.571 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.571 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.572 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.572 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.572 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:10:22.572 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.572 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.573 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.573 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:10:22.573 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.573 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.573 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.574 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.574 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.574 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:10:22.574 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.574 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.575 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:22.575 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:10:22.575 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:10:22.575 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:10:22.576 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:10:22.576 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:10:22.576 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:10:22.576 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:10:22.576 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:10:22.577 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.577 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:10:22.577 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:10:22.577 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:10:22.577 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:10:22.578 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:10:22.578 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:10:22.578 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:10:22.578 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:10:22.579 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:10:22.579 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:10:22.579 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:10:22.580 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:10:22.580 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:10:22.580 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:10:22.580 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:10:22.580 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:10:22.581 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:10:22.581 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:10:22.581 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.581 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:10:22.582 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:10:22.582 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:10:22.582 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:10:22.582 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:10:22.583 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:10:22.583 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:10:22.583 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:22.583 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:22.584 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:10:22.584 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:10:22.584 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:10:22.584 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:10:22.584 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:10:22.585 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:10:22.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8998056f70 msgr2=0x7f8998059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8998056f70 0x7f8998059430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f89b406a3b0 tx=0x7f899c0077b0).stop 2021-04-03T03:10:22.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 msgr2=0x7f89b4067070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b4067070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f89ac0032b0 tx=0x7f89ac009220).stop 2021-04-03T03:10:22.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 shutdown_connections 2021-04-03T03:10:22.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89b414b700 0x7f89b4067070 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8998056f70 0x7f8998059430 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f89b40603f0 0x7f89b41573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 --2- 172.21.2.111:0/2014945216 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89b40b0c90 0x7f89b4062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 >> 172.21.2.111:0/2014945216 conn(0x7f89b40b61c0 msgr2=0x7f89b414d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:22.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 shutdown_connections 2021-04-03T03:10:22.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.311+0000 7f89b3386700 1 -- 172.21.2.111:0/2014945216 wait complete. 2021-04-03T03:10:22.590 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:22.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea29ef700 1 Processor -- start 2021-04-03T03:10:22.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea29ef700 1 -- start start 2021-04-03T03:10:22.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea29ef700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9e940db7d0 0x7f9e94044450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea29ef700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9e940d81d0 0x7f9e94006330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea29ef700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e940d8660 con 0x7f9e94044070 2021-04-03T03:10:22.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea29ef700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e940d2780 con 0x7f9e940db7d0 2021-04-03T03:10:22.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea29ef700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e94008120 con 0x7f9e940d81d0 2021-04-03T03:10:22.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea19ed700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f9e94044070 0x7f9e94007a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52044/0 (socket says 172.21.2.111:52044) 2021-04-03T03:10:22.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea19ed700 1 -- 172.21.2.111:0/4042137845 learned_addr learned my addr 172.21.2.111:0/4042137845 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:22.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea11ec700 1 --2- 172.21.2.111:0/4042137845 >> v2:172.21.2.111:3300/0 conn(0x7f9e940db7d0 0x7f9e94044450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea09eb700 1 -- 172.21.2.111:0/4042137845 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095743194 0 0) 0x7f9e8c0010d0 con 0x7f9e94044070 2021-04-03T03:10:22.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.639+0000 7f9ea09eb700 1 -- 172.21.2.111:0/4042137845 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e84003240 con 0x7f9e94044070 2021-04-03T03:10:22.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea11ec700 1 -- 172.21.2.111:0/4042137845 >> v2:172.21.2.115:3300/0 conn(0x7f9e940d81d0 msgr2=0x7f9e94006330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:22.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea11ec700 1 --2- 172.21.2.111:0/4042137845 >> v2:172.21.2.115:3300/0 conn(0x7f9e940d81d0 0x7f9e94006330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea11ec700 1 -- 172.21.2.111:0/4042137845 >> v1:172.21.2.111:6789/0 conn(0x7f9e94044070 legacy=0x7f9e94007a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea11ec700 1 -- 172.21.2.111:0/4042137845 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940d88a0 con 0x7f9e940db7d0 2021-04-03T03:10:22.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea11ec700 1 --2- 172.21.2.111:0/4042137845 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e94044450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f9e98002f30 tx=0x7f9e98006dd0).ready entity=mon.0 client_cookie=dea61a2edb6d1375 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea09eb700 1 -- 172.21.2.111:0/4042137845 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e980097b0 con 0x7f9e940db7d0 2021-04-03T03:10:22.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea09eb700 1 -- 172.21.2.111:0/4042137845 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98009910 con 0x7f9e940db7d0 2021-04-03T03:10:22.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea09eb700 1 -- 172.21.2.111:0/4042137845 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e98009c70 con 0x7f9e940db7d0 2021-04-03T03:10:22.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/4042137845 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 msgr2=0x7f9e94044450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/4042137845 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e94044450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f9e98002f30 tx=0x7f9e98006dd0).stop 2021-04-03T03:10:22.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/4042137845 shutdown_connections 2021-04-03T03:10:22.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/4042137845 >> v2:172.21.2.115:3300/0 conn(0x7f9e940d81d0 0x7f9e94006330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/4042137845 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e94044450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/4042137845 >> 172.21.2.111:0/4042137845 conn(0x7f9e9400d730 msgr2=0x7f9e9400db10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:22.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/4042137845 shutdown_connections 2021-04-03T03:10:22.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/4042137845 wait complete. 2021-04-03T03:10:22.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 Processor -- start 2021-04-03T03:10:22.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- start start 2021-04-03T03:10:22.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e94044070 0x7f9e940d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e940d81d0 0x7f9e94047d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e940dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940dc180 con 0x7f9e940db7d0 2021-04-03T03:10:22.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940df100 con 0x7f9e94044070 2021-04-03T03:10:22.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940df330 con 0x7f9e940d81d0 2021-04-03T03:10:22.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e940dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e940dbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51258/0 (socket says 172.21.2.111:51258) 2021-04-03T03:10:22.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 -- 172.21.2.111:0/3041489121 learned_addr learned my addr 172.21.2.111:0/3041489121 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:22.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 -- 172.21.2.111:0/3041489121 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e940d81d0 msgr2=0x7f9e94047d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e940d81d0 0x7f9e94047d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 -- 172.21.2.111:0/3041489121 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e94044070 msgr2=0x7f9e940d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e94044070 0x7f9e940d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 -- 172.21.2.111:0/3041489121 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940d3350 con 0x7f9e940db7d0 2021-04-03T03:10:22.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea21ee700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e940dbc90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f9e9c06b5d0 tx=0x7f9e9c05ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9e8affd700 1 -- 172.21.2.111:0/3041489121 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9c05f2c0 con 0x7f9e940db7d0 2021-04-03T03:10:22.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9e8affd700 1 -- 172.21.2.111:0/3041489121 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c05f420 con 0x7f9e940db7d0 2021-04-03T03:10:22.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e940d5650 con 0x7f9e940db7d0 2021-04-03T03:10:22.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e940d37f0 con 0x7f9e940db7d0 2021-04-03T03:10:22.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9e8affd700 1 -- 172.21.2.111:0/3041489121 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9c05f780 con 0x7f9e940db7d0 2021-04-03T03:10:22.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.643+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e940d34e0 con 0x7f9e940db7d0 2021-04-03T03:10:22.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.647+0000 7f9e8affd700 1 -- 172.21.2.111:0/3041489121 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9e9c08b020 con 0x7f9e940db7d0 2021-04-03T03:10:22.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.647+0000 7f9e8affd700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e7c056fb0 0x7f9e7c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.647+0000 7f9e8affd700 1 -- 172.21.2.111:0/3041489121 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9e9c0df2c0 con 0x7f9e940db7d0 2021-04-03T03:10:22.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.647+0000 7f9ea19ed700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e7c056fb0 0x7f9e7c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.647+0000 7f9ea19ed700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e7c056fb0 0x7f9e7c059470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9e8c001260 tx=0x7f9e8c001140).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.647+0000 7f9e8affd700 1 -- 172.21.2.111:0/3041489121 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9e9c0b45a0 con 0x7f9e940db7d0 2021-04-03T03:10:22.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.771+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e940051f0 con 0x7f9e940db7d0 2021-04-03T03:10:22.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.771+0000 7f9e8affd700 1 -- 172.21.2.111:0/3041489121 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f9e9c0b7bc0 con 0x7f9e940db7d0 2021-04-03T03:10:22.783 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:22.783 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:09:15.086110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1150523943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1150523943},{"type":"v1","addr":"172.21.2.111:6835","nonce":1150523943}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:10:22.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.771+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e7c056fb0 msgr2=0x7f9e7c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.771+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e7c056fb0 0x7f9e7c059470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9e8c001260 tx=0x7f9e8c001140).stop 2021-04-03T03:10:22.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.771+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 msgr2=0x7f9e940dbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.771+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e940dbc90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f9e9c06b5d0 tx=0x7f9e9c05ad70).stop 2021-04-03T03:10:22.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 shutdown_connections 2021-04-03T03:10:22.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e940db7d0 0x7f9e940dbc90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e7c056fb0 0x7f9e7c059470 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e94044070 0x7f9e940d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 --2- 172.21.2.111:0/3041489121 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e940d81d0 0x7f9e94047d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 >> 172.21.2.111:0/3041489121 conn(0x7f9e9400d730 msgr2=0x7f9e94041d80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:22.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 shutdown_connections 2021-04-03T03:10:22.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.775+0000 7f9ea29ef700 1 -- 172.21.2.111:0/3041489121 wait complete. 2021-04-03T03:10:22.787 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 11 2021-04-03T03:10:22.790 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:10:22.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.835+0000 7ff489913700 1 Processor -- start 2021-04-03T03:10:22.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- start start 2021-04-03T03:10:22.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4840603f0 0x7ff4841447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840607d0 con 0x7ff4840add50 2021-04-03T03:10:22.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48414b090 con 0x7ff4840b17b0 2021-04-03T03:10:22.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48414b330 con 0x7ff4840603f0 2021-04-03T03:10:22.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4840603f0 0x7ff4841447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4840603f0 0x7ff4841447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51266/0 (socket says 172.21.2.111:51266) 2021-04-03T03:10:22.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/3542300205 learned_addr learned my addr 172.21.2.111:0/3542300205 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:22.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/3542300205 >> v1:172.21.2.115:6789/0 conn(0x7ff4840b17b0 legacy=0x7ff484147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:22.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/3542300205 >> v1:172.21.2.111:6789/0 conn(0x7ff4840add50 legacy=0x7ff484147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/3542300205 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48414b5d0 con 0x7ff4840603f0 2021-04-03T03:10:22.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- 172.21.2.111:0/3542300205 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4841447e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7ff4840b5600 tx=0x7ff47c009270).ready entity=mon.0 client_cookie=9de3cf0eb6d0f4d6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff4837fe700 1 -- 172.21.2.111:0/3542300205 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47c00bbc0 con 0x7ff4840603f0 2021-04-03T03:10:22.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff4837fe700 1 -- 172.21.2.111:0/3542300205 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00bd20 con 0x7ff4840603f0 2021-04-03T03:10:22.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff4837fe700 1 -- 172.21.2.111:0/3542300205 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47c00be90 con 0x7ff4840603f0 2021-04-03T03:10:22.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- 172.21.2.111:0/3542300205 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 msgr2=0x7ff4841447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 --2- 172.21.2.111:0/3542300205 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4841447e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7ff4840b5600 tx=0x7ff47c009270).stop 2021-04-03T03:10:22.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- 172.21.2.111:0/3542300205 shutdown_connections 2021-04-03T03:10:22.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 --2- 172.21.2.111:0/3542300205 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4841447e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:22.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- 172.21.2.111:0/3542300205 >> 172.21.2.111:0/3542300205 conn(0x7ff4840b61c0 msgr2=0x7ff48405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:22.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- 172.21.2.111:0/3542300205 shutdown_connections 2021-04-03T03:10:22.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- 172.21.2.111:0/3542300205 wait complete. 2021-04-03T03:10:22.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 Processor -- start 2021-04-03T03:10:22.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- start start 2021-04-03T03:10:22.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4840b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4840add50 0x7ff4840b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4840b17b0 0x7ff484063b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff484145660 con 0x7ff4840603f0 2021-04-03T03:10:22.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48415b130 con 0x7ff4840b17b0 2021-04-03T03:10:22.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48415abf0 con 0x7ff4840add50 2021-04-03T03:10:22.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4840b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4840b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51272/0 (socket says 172.21.2.111:51272) 2021-04-03T03:10:22.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/990738184 learned_addr learned my addr 172.21.2.111:0/990738184 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:22.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/990738184 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4840add50 msgr2=0x7ff4840b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4840add50 0x7ff4840b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/990738184 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4840b17b0 msgr2=0x7ff484063b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:22.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4840b17b0 0x7ff484063b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:22.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 -- 172.21.2.111:0/990738184 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48415ae90 con 0x7ff4840603f0 2021-04-03T03:10:22.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff488911700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4840b4390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ff484062cd0 tx=0x7ff47c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff481ffb700 1 -- 172.21.2.111:0/990738184 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47c01f420 con 0x7ff4840603f0 2021-04-03T03:10:22.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff481ffb700 1 -- 172.21.2.111:0/990738184 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c01f580 con 0x7ff4840603f0 2021-04-03T03:10:22.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48414a980 con 0x7ff4840603f0 2021-04-03T03:10:22.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff481ffb700 1 -- 172.21.2.111:0/990738184 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47c01f8e0 con 0x7ff4840603f0 2021-04-03T03:10:22.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.839+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4840aa010 con 0x7ff4840603f0 2021-04-03T03:10:22.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.843+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff484057660 con 0x7ff4840603f0 2021-04-03T03:10:22.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.843+0000 7ff481ffb700 1 -- 172.21.2.111:0/990738184 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff47c026030 con 0x7ff4840603f0 2021-04-03T03:10:22.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.843+0000 7ff481ffb700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff464056ec0 0x7ff464059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:22.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.843+0000 7ff481ffb700 1 -- 172.21.2.111:0/990738184 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff47c070ef0 con 0x7ff4840603f0 2021-04-03T03:10:22.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.843+0000 7ff483fff700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff464056ec0 0x7ff464059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:22.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.843+0000 7ff483fff700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff464056ec0 0x7ff464059380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7ff4840ae520 tx=0x7ff4780077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:22.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.843+0000 7ff481ffb700 1 -- 172.21.2.111:0/990738184 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff47c0462e0 con 0x7ff4840603f0 2021-04-03T03:10:22.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:22.971+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff484148850 con 0x7ff4840603f0 2021-04-03T03:10:23.844 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:10:23.835+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:10:23.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff481ffb700 1 -- 172.21.2.111:0/990738184 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v12) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff47c015090 con 0x7ff4840603f0 2021-04-03T03:10:23.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff464056ec0 msgr2=0x7ff464059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:23.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff464056ec0 0x7ff464059380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7ff4840ae520 tx=0x7ff4780077f0).stop 2021-04-03T03:10:23.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 msgr2=0x7ff4840b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:23.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4840b4390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ff484062cd0 tx=0x7ff47c016000).stop 2021-04-03T03:10:23.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 shutdown_connections 2021-04-03T03:10:23.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff464056ec0 0x7ff464059380 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:23.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4840b17b0 0x7ff484063b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:23.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4840603f0 0x7ff4840b4390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:23.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 --2- 172.21.2.111:0/990738184 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4840add50 0x7ff4840b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:23.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 >> 172.21.2.111:0/990738184 conn(0x7ff4840b61c0 msgr2=0x7ff484147540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:23.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 shutdown_connections 2021-04-03T03:10:23.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.847+0000 7ff489913700 1 -- 172.21.2.111:0/990738184 wait complete. 2021-04-03T03:10:23.863 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:10:23.866 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:23.878 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:10:23.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.911+0000 7f3895165700 1 Processor -- start 2021-04-03T03:10:23.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.911+0000 7f3895165700 1 -- start start 2021-04-03T03:10:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 0x7f3890142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38900607d0 con 0x7f38900acb30 2021-04-03T03:10:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38900af800 con 0x7f38900603f0 2021-04-03T03:10:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38900b03a0 con 0x7f38901473a0 2021-04-03T03:10:23.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f38900acb30 0x7f38900ae350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52084/0 (socket says 172.21.2.111:52084) 2021-04-03T03:10:23.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 -- 172.21.2.111:0/1247490026 learned_addr learned my addr 172.21.2.111:0/1247490026 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:23.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3894964700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 0x7f3890142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:23.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445612346 0 0) 0x7f38900607d0 con 0x7f38900acb30 2021-04-03T03:10:23.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3870003240 con 0x7f38900acb30 2021-04-03T03:10:23.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3284421383 0 0) 0x7f3870003240 con 0x7f38900acb30 2021-04-03T03:10:23.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 msgr2=0x7f3890142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:23.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 0x7f3890142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:23.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 >> v1:172.21.2.115:6789/0 conn(0x7f38900603f0 legacy=0x7f38901427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:23.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900afaa0 con 0x7f38900acb30 2021-04-03T03:10:23.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f38800016d0 con 0x7f38900acb30 2021-04-03T03:10:23.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 >> v1:172.21.2.111:6789/0 conn(0x7f38900acb30 legacy=0x7f38900ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:23.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3870003760 0x7f3870003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38700044e0 0x7f3870004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 0x7f3870004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3870005360 con 0x7f38700044e0 2021-04-03T03:10:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700042a0 con 0x7f3870003760 2021-04-03T03:10:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387000b730 con 0x7f38901473a0 2021-04-03T03:10:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38700044e0 0x7f3870004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388ffff700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3870003760 0x7f3870003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3894964700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 0x7f3870004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 -- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 msgr2=0x7f3870004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38901473a0 0x7f3870004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 -- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3870003760 msgr2=0x7f3870003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3870003760 0x7f3870003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 -- 172.21.2.111:0/1247490026 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900afaa0 con 0x7f38700044e0 2021-04-03T03:10:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388ffff700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3870003760 0x7f3870003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:10:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388f7fe700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38700044e0 0x7f3870004960 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f38800013e0 tx=0x7f3880007750).ready entity=mon.0 client_cookie=329f98d8cd59f0f4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38800070d0 con 0x7f38700044e0 2021-04-03T03:10:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3880007230 con 0x7f38700044e0 2021-04-03T03:10:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f388effd700 1 -- 172.21.2.111:0/1247490026 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38800055f0 con 0x7f38700044e0 2021-04-03T03:10:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- 172.21.2.111:0/1247490026 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38700044e0 msgr2=0x7f3870004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 --2- 172.21.2.111:0/1247490026 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38700044e0 0x7f3870004960 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f38800013e0 tx=0x7f3880007750).stop 2021-04-03T03:10:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3894964700 1 -- 172.21.2.111:0/1247490026 reap_dead start 2021-04-03T03:10:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- 172.21.2.111:0/1247490026 shutdown_connections 2021-04-03T03:10:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- 172.21.2.111:0/1247490026 >> 172.21.2.111:0/1247490026 conn(0x7f38900b61c0 msgr2=0x7f389005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- 172.21.2.111:0/1247490026 shutdown_connections 2021-04-03T03:10:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- 172.21.2.111:0/1247490026 wait complete. 2021-04-03T03:10:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 Processor -- start 2021-04-03T03:10:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- start start 2021-04-03T03:10:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389013f170 0x7f3890151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3890149740 0x7f3890152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3890147d40 0x7f3890152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389013f550 con 0x7f3890147d40 2021-04-03T03:10:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3890149c50 con 0x7f3890149740 2021-04-03T03:10:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3895165700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3890149ef0 con 0x7f389013f170 2021-04-03T03:10:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3894964700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3890147d40 0x7f3890152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3894964700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3890147d40 0x7f3890152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51306/0 (socket says 172.21.2.111:51306) 2021-04-03T03:10:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.915+0000 7f3894964700 1 -- 172.21.2.111:0/1640686989 learned_addr learned my addr 172.21.2.111:0/1640686989 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3894964700 1 -- 172.21.2.111:0/1640686989 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389013f170 msgr2=0x7f3890151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3894964700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389013f170 0x7f3890151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3894964700 1 -- 172.21.2.111:0/1640686989 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3890149740 msgr2=0x7f3890152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388f7fe700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3890149740 0x7f3890152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3894964700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3890149740 0x7f3890152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3894964700 1 -- 172.21.2.111:0/1640686989 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389014a190 con 0x7f3890147d40 2021-04-03T03:10:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3894964700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3890147d40 0x7f3890152910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f388800b000 tx=0x7f3888008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388d7fa700 1 -- 172.21.2.111:0/1640686989 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f388800eb60 con 0x7f3890147d40 2021-04-03T03:10:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388d7fa700 1 -- 172.21.2.111:0/1640686989 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800ecc0 con 0x7f3890147d40 2021-04-03T03:10:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388d7fa700 1 -- 172.21.2.111:0/1640686989 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3888014660 con 0x7f3890147d40 2021-04-03T03:10:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3890147160 con 0x7f3890147d40 2021-04-03T03:10:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3890155480 con 0x7f3890147d40 2021-04-03T03:10:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389014a320 con 0x7f3890147d40 2021-04-03T03:10:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388d7fa700 1 -- 172.21.2.111:0/1640686989 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f38880147c0 con 0x7f3890147d40 2021-04-03T03:10:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388d7fa700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3874057000 0x7f38740594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388d7fa700 1 -- 172.21.2.111:0/1640686989 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f3888070d30 con 0x7f3890147d40 2021-04-03T03:10:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388ffff700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3874057000 0x7f38740594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.919+0000 7f388ffff700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3874057000 0x7f38740594c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f387800b000 tx=0x7f3878009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:23.923+0000 7f388d7fa700 1 -- 172.21.2.111:0/1640686989 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f38880460c0 con 0x7f3890147d40 2021-04-03T03:10:24.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3890057660 con 0x7f3890147d40 2021-04-03T03:10:24.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f388d7fa700 1 -- 172.21.2.111:0/1640686989 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+1773 (secure 0 0 0) 0x7f38880496e0 con 0x7f3890147d40 2021-04-03T03:10:24.058 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:24.059 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/272245571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":272245571},{"type":"v1","addr":"172.21.2.115:6833","nonce":272245571}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":12,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:09:14.082969+0000","modified":"2021-04-03T03:10:23.839281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-04-03T03:10:24.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3874057000 msgr2=0x7f38740594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3874057000 0x7f38740594c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f387800b000 tx=0x7f3878009040).stop 2021-04-03T03:10:24.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3890147d40 msgr2=0x7f3890152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3890147d40 0x7f3890152910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f388800b000 tx=0x7f3888008040).stop 2021-04-03T03:10:24.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 shutdown_connections 2021-04-03T03:10:24.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3890147d40 0x7f3890152910 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f389013f170 0x7f3890151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:24.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3874057000 0x7f38740594c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 --2- 172.21.2.111:0/1640686989 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3890149740 0x7f3890152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 >> 172.21.2.111:0/1640686989 conn(0x7f38900b61c0 msgr2=0x7f38900b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:24.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 shutdown_connections 2021-04-03T03:10:24.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.047+0000 7f3895165700 1 -- 172.21.2.111:0/1640686989 wait complete. 2021-04-03T03:10:24.062 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 12 2021-04-03T03:10:24.066 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:10:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e36b6c700 1 Processor -- start 2021-04-03T03:10:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e36b6c700 1 -- start start 2021-04-03T03:10:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e36b6c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1e2803f120 0x7f1e28005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e36b6c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 0x7f1e280d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e36b6c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2803d840 con 0x7f1e28042470 2021-04-03T03:10:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e36b6c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e2803faa0 con 0x7f1e2803f120 2021-04-03T03:10:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e36b6c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2803dca0 con 0x7f1e28041fe0 2021-04-03T03:10:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1e28042470 0x7f1e280d8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52106/0 (socket says 172.21.2.111:52106) 2021-04-03T03:10:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 -- 172.21.2.111:0/3270931074 learned_addr learned my addr 172.21.2.111:0/3270931074 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615274436 0 0) 0x7f1e2803d840 con 0x7f1e28042470 2021-04-03T03:10:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e18003240 con 0x7f1e28042470 2021-04-03T03:10:24.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e35b6a700 1 --2- 172.21.2.111:0/3270931074 >> v2:172.21.2.115:3300/0 conn(0x7f1e2803f120 0x7f1e28005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e35369700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 0x7f1e280d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1067121445 0 0) 0x7f1e18003240 con 0x7f1e28042470 2021-04-03T03:10:24.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 msgr2=0x7f1e280d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 0x7f1e280d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 >> v2:172.21.2.115:3300/0 conn(0x7f1e2803f120 msgr2=0x7f1e28005640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 --2- 172.21.2.111:0/3270931074 >> v2:172.21.2.115:3300/0 conn(0x7f1e2803f120 0x7f1e28005640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e28005b30 con 0x7f1e28042470 2021-04-03T03:10:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f1e300450f0 con 0x7f1e28042470 2021-04-03T03:10:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e35369700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 0x7f1e280d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:10:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 >> v1:172.21.2.111:6789/0 conn(0x7f1e28042470 legacy=0x7f1e280d8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 0x7f1e180037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e18003ca0 0x7f1e180044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e180049d0 0x7f1e18004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2803faa0 con 0x7f1e180049d0 2021-04-03T03:10:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2803dca0 con 0x7f1e18003ca0 2021-04-03T03:10:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180041c0 con 0x7f1e28041fe0 2021-04-03T03:10:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e180049d0 0x7f1e18004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e35b6a700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 0x7f1e180037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 -- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 msgr2=0x7f1e180037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e28041fe0 0x7f1e180037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 -- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e18003ca0 msgr2=0x7f1e180044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e18003ca0 0x7f1e180044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 -- 172.21.2.111:0/3270931074 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e18003240 con 0x7f1e180049d0 2021-04-03T03:10:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.155+0000 7f1e3636b700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e180049d0 0x7f1e18004e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1e30069660 tx=0x7f1e2803ec70).ready entity=mon.0 client_cookie=c54c0cb3727ee141 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e3005df60 con 0x7f1e180049d0 2021-04-03T03:10:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e300699f0 con 0x7f1e180049d0 2021-04-03T03:10:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e34b68700 1 -- 172.21.2.111:0/3270931074 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e30061110 con 0x7f1e180049d0 2021-04-03T03:10:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/3270931074 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e180049d0 msgr2=0x7f1e18004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 --2- 172.21.2.111:0/3270931074 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e180049d0 0x7f1e18004e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1e30069660 tx=0x7f1e2803ec70).stop 2021-04-03T03:10:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e3636b700 1 -- 172.21.2.111:0/3270931074 reap_dead start 2021-04-03T03:10:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/3270931074 shutdown_connections 2021-04-03T03:10:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/3270931074 >> 172.21.2.111:0/3270931074 conn(0x7f1e280462b0 msgr2=0x7f1e280466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/3270931074 shutdown_connections 2021-04-03T03:10:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/3270931074 wait complete. 2021-04-03T03:10:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 Processor -- start 2021-04-03T03:10:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- start start 2021-04-03T03:10:24.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e280d7490 0x7f1e280dbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e280d4110 0x7f1e280d4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e2803b2a0 0x7f1e2803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280d7870 con 0x7f1e280d4110 2021-04-03T03:10:24.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e28039700 con 0x7f1e280d7490 2021-04-03T03:10:24.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280399a0 con 0x7f1e2803b2a0 2021-04-03T03:10:24.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e280d4110 0x7f1e280d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e280d4110 0x7f1e280d4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51322/0 (socket says 172.21.2.111:51322) 2021-04-03T03:10:24.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 -- 172.21.2.111:0/4143635631 learned_addr learned my addr 172.21.2.111:0/4143635631 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:24.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 -- 172.21.2.111:0/4143635631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e2803b2a0 msgr2=0x7f1e2803b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e2803b2a0 0x7f1e2803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:24.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 -- 172.21.2.111:0/4143635631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e280d7490 msgr2=0x7f1e280dbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e280d7490 0x7f1e280dbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:24.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 -- 172.21.2.111:0/4143635631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e28039e40 con 0x7f1e280d4110 2021-04-03T03:10:24.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35369700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e280d4110 0x7f1e280d4590 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f1e20006a60 tx=0x7f1e20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:24.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e1effd700 1 -- 172.21.2.111:0/4143635631 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e20016070 con 0x7f1e280d4110 2021-04-03T03:10:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e1effd700 1 -- 172.21.2.111:0/4143635631 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20011850 con 0x7f1e280d4110 2021-04-03T03:10:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e280d4a80 con 0x7f1e280d4110 2021-04-03T03:10:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e1effd700 1 -- 172.21.2.111:0/4143635631 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e20011bb0 con 0x7f1e280d4110 2021-04-03T03:10:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e280dbbc0 con 0x7f1e280d4110 2021-04-03T03:10:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e28039b60 con 0x7f1e280d4110 2021-04-03T03:10:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e1effd700 1 -- 172.21.2.111:0/4143635631 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1e20011d10 con 0x7f1e280d4110 2021-04-03T03:10:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e1effd700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e10056f20 0x7f1e100593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e1effd700 1 -- 172.21.2.111:0/4143635631 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f1e200703b0 con 0x7f1e280d4110 2021-04-03T03:10:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35b6a700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e10056f20 0x7f1e100593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.159+0000 7f1e35b6a700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e10056f20 0x7f1e100593e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f1e2c008000 tx=0x7f1e2c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:24.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.163+0000 7f1e1effd700 1 -- 172.21.2.111:0/4143635631 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1e20045630 con 0x7f1e280d4110 2021-04-03T03:10:24.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.287+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1e280dbbc0 con 0x7f1e280d4110 2021-04-03T03:10:24.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e1effd700 1 -- 172.21.2.111:0/4143635631 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e280dbbc0 con 0x7f1e280d4110 2021-04-03T03:10:24.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e10056f20 msgr2=0x7f1e100593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e10056f20 0x7f1e100593e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f1e2c008000 tx=0x7f1e2c008040).stop 2021-04-03T03:10:24.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e280d4110 msgr2=0x7f1e280d4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e280d4110 0x7f1e280d4590 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f1e20006a60 tx=0x7f1e20008040).stop 2021-04-03T03:10:24.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 shutdown_connections 2021-04-03T03:10:24.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1e2803b2a0 0x7f1e2803b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:24.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1e10056f20 0x7f1e100593e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1e280d7490 0x7f1e280dbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:24.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 --2- 172.21.2.111:0/4143635631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1e280d4110 0x7f1e280d4590 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 >> 172.21.2.111:0/4143635631 conn(0x7f1e280462b0 msgr2=0x7f1e28045790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:24.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 shutdown_connections 2021-04-03T03:10:24.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.851+0000 7f1e36b6c700 1 -- 172.21.2.111:0/4143635631 wait complete. 2021-04-03T03:10:24.870 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:10:24.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 Processor -- start 2021-04-03T03:10:24.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 -- start start 2021-04-03T03:10:24.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7feeac0603f0 0x7feeac1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 0x7feeac144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0607d0 con 0x7feeac0b17b0 2021-04-03T03:10:24.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac1451c0 con 0x7feeac0603f0 2021-04-03T03:10:24.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac14b090 con 0x7feeac0add50 2021-04-03T03:10:24.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7feeac0b17b0 0x7feeac147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52124/0 (socket says 172.21.2.111:52124) 2021-04-03T03:10:24.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 -- 172.21.2.111:0/3719162488 learned_addr learned my addr 172.21.2.111:0/3719162488 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:24.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb0bec700 1 --2- 172.21.2.111:0/3719162488 >> v2:172.21.2.115:3300/0 conn(0x7feeac0603f0 0x7feeac1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeabfff700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 0x7feeac144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610506527 0 0) 0x7feeac0607d0 con 0x7feeac0b17b0 2021-04-03T03:10:24.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee8c003240 con 0x7feeac0b17b0 2021-04-03T03:10:24.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 366813694 0 0) 0x7fee8c003240 con 0x7feeac0b17b0 2021-04-03T03:10:24.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 msgr2=0x7feeac144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 0x7feeac144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 >> v2:172.21.2.115:3300/0 conn(0x7feeac0603f0 msgr2=0x7feeac1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 --2- 172.21.2.111:0/3719162488 >> v2:172.21.2.115:3300/0 conn(0x7feeac0603f0 0x7feeac1447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac14b330 con 0x7feeac0b17b0 2021-04-03T03:10:24.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7feea4001a50 con 0x7feeac0b17b0 2021-04-03T03:10:24.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 >> v1:172.21.2.111:6789/0 conn(0x7feeac0b17b0 legacy=0x7feeac147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee8c0037b0 0x7fee8c003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 0x7fee8c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee8c0049d0 0x7fee8c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac14b090 con 0x7fee8c0049d0 2021-04-03T03:10:24.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c0042a0 con 0x7fee8c0037b0 2021-04-03T03:10:24.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c00b730 con 0x7feeac0add50 2021-04-03T03:10:24.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee8c0049d0 0x7fee8c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeabfff700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 0x7fee8c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb0bec700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee8c0037b0 0x7fee8c003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 -- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 msgr2=0x7fee8c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0add50 0x7fee8c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 -- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee8c0037b0 msgr2=0x7fee8c003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:24.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee8c0037b0 0x7fee8c003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 -- 172.21.2.111:0/3719162488 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c003240 con 0x7fee8c0049d0 2021-04-03T03:10:24.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee8c0049d0 0x7fee8c004e50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7feea4001760 tx=0x7feeac0628a0).ready entity=mon.0 client_cookie=38bc68934ffa72a2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:24.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea4005b80 con 0x7fee8c0049d0 2021-04-03T03:10:24.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4005ce0 con 0x7fee8c0049d0 2021-04-03T03:10:24.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeab7fe700 1 -- 172.21.2.111:0/3719162488 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea40072c0 con 0x7fee8c0049d0 2021-04-03T03:10:24.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 -- 172.21.2.111:0/3719162488 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee8c0049d0 msgr2=0x7fee8c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb1bee700 1 --2- 172.21.2.111:0/3719162488 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee8c0049d0 0x7fee8c004e50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7feea4001760 tx=0x7feeac0628a0).stop 2021-04-03T03:10:24.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.919+0000 7feeb13ed700 1 -- 172.21.2.111:0/3719162488 reap_dead start 2021-04-03T03:10:24.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- 172.21.2.111:0/3719162488 shutdown_connections 2021-04-03T03:10:24.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- 172.21.2.111:0/3719162488 >> 172.21.2.111:0/3719162488 conn(0x7feeac0b61c0 msgr2=0x7feeac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:24.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- 172.21.2.111:0/3719162488 shutdown_connections 2021-04-03T03:10:24.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- 172.21.2.111:0/3719162488 wait complete. 2021-04-03T03:10:24.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 Processor -- start 2021-04-03T03:10:24.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- start start 2021-04-03T03:10:24.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feeac0a9370 0x7feeac156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0aa3a0 0x7feeac1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feeac1577e0 0x7feeac15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0abdc0 con 0x7feeac1577e0 2021-04-03T03:10:24.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0a9750 con 0x7feeac0a9370 2021-04-03T03:10:24.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0abb20 con 0x7feeac0aa3a0 2021-04-03T03:10:24.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feeac1577e0 0x7feeac15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feeac1577e0 0x7feeac15a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51346/0 (socket says 172.21.2.111:51346) 2021-04-03T03:10:24.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 -- 172.21.2.111:0/2090913631 learned_addr learned my addr 172.21.2.111:0/2090913631 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:24.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb0bec700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feeac0a9370 0x7feeac156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 -- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0aa3a0 msgr2=0x7feeac1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeabfff700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0aa3a0 0x7feeac1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0aa3a0 0x7feeac1572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 -- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feeac0a9370 msgr2=0x7feeac156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:24.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feeac0a9370 0x7feeac156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:24.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 -- 172.21.2.111:0/2090913631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0a7f30 con 0x7feeac1577e0 2021-04-03T03:10:24.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeabfff700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0aa3a0 0x7feeac1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:24.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb0bec700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feeac0a9370 0x7feeac156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:10:24.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb13ed700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feeac1577e0 0x7feeac15a050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7feeac0af1d0 tx=0x7feea4013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:24.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feea9ffb700 1 -- 172.21.2.111:0/2090913631 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea401e470 con 0x7feeac1577e0 2021-04-03T03:10:24.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feea9ffb700 1 -- 172.21.2.111:0/2090913631 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4018030 con 0x7feeac1577e0 2021-04-03T03:10:24.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac0b3780 con 0x7feeac1577e0 2021-04-03T03:10:24.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feea9ffb700 1 -- 172.21.2.111:0/2090913631 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea401e7c0 con 0x7feeac1577e0 2021-04-03T03:10:24.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac15a540 con 0x7feeac1577e0 2021-04-03T03:10:24.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac057660 con 0x7feeac1577e0 2021-04-03T03:10:24.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feea9ffb700 1 -- 172.21.2.111:0/2090913631 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7feea4025030 con 0x7feeac1577e0 2021-04-03T03:10:24.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feea9ffb700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee90056ea0 0x7fee90059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:24.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feea9ffb700 1 -- 172.21.2.111:0/2090913631 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6502+0+0 (secure 0 0 0) 0x7feea4071040 con 0x7feeac1577e0 2021-04-03T03:10:24.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb0bec700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee90056ea0 0x7fee90059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:24.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.923+0000 7feeb0bec700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee90056ea0 0x7fee90059360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fee94008000 tx=0x7fee94008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:24.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:24.927+0000 7feea9ffb700 1 -- 172.21.2.111:0/2090913631 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7feea40463d0 con 0x7feeac1577e0 2021-04-03T03:10:25.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.019+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feeac0b1510 con 0x7feeac1577e0 2021-04-03T03:10:25.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feea9ffb700 1 -- 172.21.2.111:0/2090913631 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 185+0+0 (secure 0 0 0) 0x7feea4023030 con 0x7feeac1577e0 2021-04-03T03:10:25.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee90056ea0 msgr2=0x7fee90059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:25.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee90056ea0 0x7fee90059360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fee94008000 tx=0x7fee94008040).stop 2021-04-03T03:10:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feeac1577e0 msgr2=0x7feeac15a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feeac1577e0 0x7feeac15a050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7feeac0af1d0 tx=0x7feea4013000).stop 2021-04-03T03:10:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 shutdown_connections 2021-04-03T03:10:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feeac1577e0 0x7feeac15a050 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee90056ea0 0x7fee90059360 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feeac0a9370 0x7feeac156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 --2- 172.21.2.111:0/2090913631 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feeac0aa3a0 0x7feeac1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 >> 172.21.2.111:0/2090913631 conn(0x7feeac0b61c0 msgr2=0x7feeac145a60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 shutdown_connections 2021-04-03T03:10:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.859+0000 7feeb1bee700 1 -- 172.21.2.111:0/2090913631 wait complete. 2021-04-03T03:10:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:10:25.878 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:10:25.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 Processor -- start 2021-04-03T03:10:25.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 -- start start 2021-04-03T03:10:25.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:25.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7febd40add50 0x7febd4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:25.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40607d0 con 0x7febd40b17b0 2021-04-03T03:10:25.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd41451c0 con 0x7febd40add50 2021-04-03T03:10:25.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd414b090 con 0x7febd40603f0 2021-04-03T03:10:25.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7febd40add50 0x7febd4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:25.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7febd40add50 0x7febd4144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51354/0 (socket says 172.21.2.111:51354) 2021-04-03T03:10:25.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 -- 172.21.2.111:0/3525491279 learned_addr learned my addr 172.21.2.111:0/3525491279 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:25.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd37fe700 1 --2- 172.21.2.111:0/3525491279 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:25.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 -- 172.21.2.111:0/3525491279 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 msgr2=0x7febd41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:25.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 --2- 172.21.2.111:0/3525491279 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd41447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 -- 172.21.2.111:0/3525491279 >> v1:172.21.2.115:6789/0 conn(0x7febd40b17b0 legacy=0x7febd4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:25.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 -- 172.21.2.111:0/3525491279 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd414b330 con 0x7febd40add50 2021-04-03T03:10:25.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd2ffd700 1 --2- 172.21.2.111:0/3525491279 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4144cd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7febbc002f30 tx=0x7febbc006e90).ready entity=mon.0 client_cookie=8f99f6444b889c25 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:25.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd27fc700 1 -- 172.21.2.111:0/3525491279 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febbc009970 con 0x7febd40add50 2021-04-03T03:10:25.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd27fc700 1 -- 172.21.2.111:0/3525491279 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc009ad0 con 0x7febd40add50 2021-04-03T03:10:25.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd27fc700 1 -- 172.21.2.111:0/3525491279 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febbc009e30 con 0x7febd40add50 2021-04-03T03:10:25.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 -- 172.21.2.111:0/3525491279 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 msgr2=0x7febd4144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:25.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.927+0000 7febd8be4700 1 --2- 172.21.2.111:0/3525491279 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4144cd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7febbc002f30 tx=0x7febbc006e90).stop 2021-04-03T03:10:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- 172.21.2.111:0/3525491279 shutdown_connections 2021-04-03T03:10:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 --2- 172.21.2.111:0/3525491279 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 --2- 172.21.2.111:0/3525491279 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4144cd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- 172.21.2.111:0/3525491279 >> 172.21.2.111:0/3525491279 conn(0x7febd40b61c0 msgr2=0x7febd405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- 172.21.2.111:0/3525491279 shutdown_connections 2021-04-03T03:10:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- 172.21.2.111:0/3525491279 wait complete. 2021-04-03T03:10:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 Processor -- start 2021-04-03T03:10:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- start start 2021-04-03T03:10:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd40b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4063210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febd40b17b0 0x7febd4063700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40a8fd0 con 0x7febd40add50 2021-04-03T03:10:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40a9dc0 con 0x7febd40b17b0 2021-04-03T03:10:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40aa060 con 0x7febd40603f0 2021-04-03T03:10:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4063210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4063210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51358/0 (socket says 172.21.2.111:51358) 2021-04-03T03:10:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 -- 172.21.2.111:0/2948678673 learned_addr learned my addr 172.21.2.111:0/2948678673 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd37fe700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd40b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd3fff700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febd40b17b0 0x7febd4063700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 -- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 msgr2=0x7febd40b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd40b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 -- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febd40b17b0 msgr2=0x7febd4063700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febd40b17b0 0x7febd4063700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 -- 172.21.2.111:0/2948678673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40a8af0 con 0x7febd40add50 2021-04-03T03:10:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd2ffd700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4063210 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7febbc002f30 tx=0x7febbc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd0ff9700 1 -- 172.21.2.111:0/2948678673 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febbc011ca0 con 0x7febd40add50 2021-04-03T03:10:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd0ff9700 1 -- 172.21.2.111:0/2948678673 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc011e00 con 0x7febd40add50 2021-04-03T03:10:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40b4af0 con 0x7febd40add50 2021-04-03T03:10:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd40abfb0 con 0x7febd40add50 2021-04-03T03:10:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd0ff9700 1 -- 172.21.2.111:0/2948678673 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febbc01e650 con 0x7febd40add50 2021-04-03T03:10:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd40a8e10 con 0x7febd40add50 2021-04-03T03:10:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd0ff9700 1 -- 172.21.2.111:0/2948678673 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7febbc025030 con 0x7febd40add50 2021-04-03T03:10:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.931+0000 7febd0ff9700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7febb8056ea0 0x7febb8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.935+0000 7febd37fe700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7febb8056ea0 0x7febb8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.935+0000 7febd0ff9700 1 -- 172.21.2.111:0/2948678673 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6064+0+0 (secure 0 0 0) 0x7febbc0712d0 con 0x7febd40add50 2021-04-03T03:10:25.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.935+0000 7febd37fe700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7febb8056ea0 0x7febb8059360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7febc400c000 tx=0x7febc4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:25.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:25.935+0000 7febd0ff9700 1 -- 172.21.2.111:0/2948678673 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7febbc046820 con 0x7febd40add50 2021-04-03T03:10:26.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.031+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7febd4057660 con 0x7febd40add50 2021-04-03T03:10:26.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd0ff9700 1 -- 172.21.2.111:0/2948678673 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 173+0+0 (secure 0 0 0) 0x7febbc0230d0 con 0x7febd40add50 2021-04-03T03:10:26.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7febb8056ea0 msgr2=0x7febb8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:26.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7febb8056ea0 0x7febb8059360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7febc400c000 tx=0x7febc4008040).stop 2021-04-03T03:10:26.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 msgr2=0x7febd4063210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:26.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4063210 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7febbc002f30 tx=0x7febbc015000).stop 2021-04-03T03:10:26.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 shutdown_connections 2021-04-03T03:10:26.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7febb8056ea0 0x7febb8059360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:26.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febd40b17b0 0x7febd4063700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:26.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febd40603f0 0x7febd40b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:26.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 --2- 172.21.2.111:0/2948678673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febd40add50 0x7febd4063210 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:26.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 >> 172.21.2.111:0/2948678673 conn(0x7febd40b61c0 msgr2=0x7febd40b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:26.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 shutdown_connections 2021-04-03T03:10:26.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.871+0000 7febd8be4700 1 -- 172.21.2.111:0/2948678673 wait complete. 2021-04-03T03:10:26.886 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:10:26.890 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:10:26.891 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:10:26.891 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:10:26.891 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:10:26.892 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:10:26.892 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:10:26.892 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:10:26.893 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:10:26.896 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:10:26.896 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:10:26.898 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:10:26.898 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:10:26.941 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:10:26.941 DEBUG:teuthology.parallel:result is None 2021-04-03T03:10:26.942 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:10:26.942 DEBUG:teuthology.parallel:result is None 2021-04-03T03:10:26.943 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:10:26.979 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:10:26.980 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:10:27.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 Processor -- start 2021-04-03T03:10:27.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- start start 2021-04-03T03:10:27.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f523803f120 0x7f5238005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:27.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5238041fe0 0x7f5238042440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:27.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52380d89a0 0x7f52380d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:27.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523803faa0 con 0x7f52380d89a0 2021-04-03T03:10:27.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523803dca0 con 0x7f5238041fe0 2021-04-03T03:10:27.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523803d840 con 0x7f523803f120 2021-04-03T03:10:27.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52380d89a0 0x7f52380d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:27.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52380d89a0 0x7f52380d8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51378/0 (socket says 172.21.2.111:51378) 2021-04-03T03:10:27.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 -- 172.21.2.111:0/977709487 learned_addr learned my addr 172.21.2.111:0/977709487 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:27.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 --2- 172.21.2.111:0/977709487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f523803f120 0x7f5238005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:27.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5244da2700 1 --2- 172.21.2.111:0/977709487 >> v2:172.21.2.115:3300/0 conn(0x7f5238041fe0 0x7f5238042440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:27.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 -- 172.21.2.111:0/977709487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f523803f120 msgr2=0x7f5238005640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:27.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 --2- 172.21.2.111:0/977709487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f523803f120 0x7f5238005640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:27.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 -- 172.21.2.111:0/977709487 >> v2:172.21.2.115:3300/0 conn(0x7f5238041fe0 msgr2=0x7f5238042440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:27.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 --2- 172.21.2.111:0/977709487 >> v2:172.21.2.115:3300/0 conn(0x7f5238041fe0 0x7f5238042440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:27.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 -- 172.21.2.111:0/977709487 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5238005b30 con 0x7f52380d89a0 2021-04-03T03:10:27.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 --2- 172.21.2.111:0/977709487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f523803f120 0x7f5238005640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:10:27.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5244da2700 1 --2- 172.21.2.111:0/977709487 >> v2:172.21.2.115:3300/0 conn(0x7f5238041fe0 0x7f5238042440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:10:27.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 --2- 172.21.2.111:0/977709487 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52380d89a0 0x7f52380d8e30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f523803d2c0 tx=0x7f523c006e90).ready entity=mon.0 client_cookie=8e5e512ceacfdb96 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:27.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f522ffff700 1 -- 172.21.2.111:0/977709487 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f523c0098e0 con 0x7f52380d89a0 2021-04-03T03:10:27.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f522ffff700 1 -- 172.21.2.111:0/977709487 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c009a40 con 0x7f52380d89a0 2021-04-03T03:10:27.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f522ffff700 1 -- 172.21.2.111:0/977709487 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f523c009da0 con 0x7f52380d89a0 2021-04-03T03:10:27.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- 172.21.2.111:0/977709487 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52380d89a0 msgr2=0x7f52380d8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:27.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- 172.21.2.111:0/977709487 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52380d89a0 0x7f52380d8e30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f523803d2c0 tx=0x7f523c006e90).stop 2021-04-03T03:10:27.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- 172.21.2.111:0/977709487 shutdown_connections 2021-04-03T03:10:27.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- 172.21.2.111:0/977709487 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52380d89a0 0x7f52380d8e30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:27.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- 172.21.2.111:0/977709487 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f523803f120 0x7f5238005640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:27.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- 172.21.2.111:0/977709487 >> v2:172.21.2.115:3300/0 conn(0x7f5238041fe0 0x7f5238042440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:27.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- 172.21.2.111:0/977709487 >> 172.21.2.111:0/977709487 conn(0x7f52380462b0 msgr2=0x7f52380466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:27.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- 172.21.2.111:0/977709487 shutdown_connections 2021-04-03T03:10:27.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- 172.21.2.111:0/977709487 wait complete. 2021-04-03T03:10:27.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 Processor -- start 2021-04-03T03:10:27.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- start start 2021-04-03T03:10:27.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f523803f120 0x7f52380a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:27.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5238041fe0 0x7f52380d6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:27.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52380d89a0 0x7f52380d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:27.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52380a08c0 con 0x7f523803f120 2021-04-03T03:10:27.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52380a0d50 con 0x7f52380d89a0 2021-04-03T03:10:27.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52465a5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52380e7e60 con 0x7f5238041fe0 2021-04-03T03:10:27.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f523803f120 0x7f52380a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:27.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f523803f120 0x7f52380a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51382/0 (socket says 172.21.2.111:51382) 2021-04-03T03:10:27.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 -- 172.21.2.111:0/3588358599 learned_addr learned my addr 172.21.2.111:0/3588358599 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:27.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5244da2700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5238041fe0 0x7f52380d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:27.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f5245da4700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52380d89a0 0x7f52380d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:27.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 -- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5238041fe0 msgr2=0x7f52380d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:27.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5238041fe0 0x7f52380d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:27.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 -- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52380d89a0 msgr2=0x7f52380d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:27.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52380d89a0 0x7f52380d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:27.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 -- 172.21.2.111:0/3588358599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380e7bf0 con 0x7f523803f120 2021-04-03T03:10:27.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f52455a3700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f523803f120 0x7f52380a03d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f5230006920 tx=0x7f5230008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:27.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f522e7fc700 1 -- 172.21.2.111:0/3588358599 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5230009980 con 0x7f523803f120 2021-04-03T03:10:27.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.995+0000 7f522e7fc700 1 -- 172.21.2.111:0/3588358599 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5230009ae0 con 0x7f523803f120 2021-04-03T03:10:27.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f522e7fc700 1 -- 172.21.2.111:0/3588358599 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5230009e40 con 0x7f523803f120 2021-04-03T03:10:27.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52380a0ab0 con 0x7f523803f120 2021-04-03T03:10:27.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52380dce10 con 0x7f523803f120 2021-04-03T03:10:27.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52380e8020 con 0x7f523803f120 2021-04-03T03:10:27.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f522e7fc700 1 -- 172.21.2.111:0/3588358599 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5230013480 con 0x7f523803f120 2021-04-03T03:10:27.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f522e7fc700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5220056f20 0x7f52200593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:27.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f522e7fc700 1 -- 172.21.2.111:0/3588358599 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f52300706b0 con 0x7f523803f120 2021-04-03T03:10:27.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f5244da2700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5220056f20 0x7f52200593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:27.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f5244da2700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5220056f20 0x7f52200593e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5240067000 tx=0x7f5240044f70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:27.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:26.999+0000 7f522e7fc700 1 -- 172.21.2.111:0/3588358599 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f52380e8020 con 0x7f523803f120 2021-04-03T03:10:27.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.107+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5238004d30 con 0x7f523803f120 2021-04-03T03:10:27.884 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:27.897 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:27.900 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:27.905 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:27.929 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:28.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.983+0000 7f522e7fc700 1 -- 172.21.2.111:0/3588358599 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5230045d10 con 0x7f523803f120 2021-04-03T03:10:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5220056f20 msgr2=0x7f52200593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5220056f20 0x7f52200593e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5240067000 tx=0x7f5240044f70).stop 2021-04-03T03:10:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f523803f120 msgr2=0x7f52380a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f523803f120 0x7f52380a03d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f5230006920 tx=0x7f5230008040).stop 2021-04-03T03:10:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 shutdown_connections 2021-04-03T03:10:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5220056f20 0x7f52200593e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52380d89a0 0x7f52380d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f523803f120 0x7f52380a03d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 --2- 172.21.2.111:0/3588358599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5238041fe0 0x7f52380d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 >> 172.21.2.111:0/3588358599 conn(0x7f52380462b0 msgr2=0x7f5238041610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:28.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 shutdown_connections 2021-04-03T03:10:28.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:27.987+0000 7f52465a5700 1 -- 172.21.2.111:0/3588358599 wait complete. 2021-04-03T03:10:28.622 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:10:28.623 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:10:28.623 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:10:28.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa443fff700 1 Processor -- start 2021-04-03T03:10:28.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa443fff700 1 -- start start 2021-04-03T03:10:28.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa443fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa4440b12b0 0x7fa4441522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa443fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440607d0 con 0x7fa44414b700 2021-04-03T03:10:28.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa443fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440b0eb0 con 0x7fa4440603f0 2021-04-03T03:10:28.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa443fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4440af400 con 0x7fa4440b12b0 2021-04-03T03:10:28.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa4427fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa44414b700 0x7fa4440b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52186/0 (socket says 172.21.2.111:52186) 2021-04-03T03:10:28.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa4427fc700 1 -- 172.21.2.111:0/1040557678 learned_addr learned my addr 172.21.2.111:0/1040557678 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:28.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa4437fe700 1 --2- 172.21.2.111:0/1040557678 >> v2:172.21.2.115:3300/0 conn(0x7fa4440b12b0 0x7fa4441522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987962288 0 0) 0x7fa4440607d0 con 0x7fa44414b700 2021-04-03T03:10:28.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa424003240 con 0x7fa44414b700 2021-04-03T03:10:28.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1452702232 0 0) 0x7fa424003240 con 0x7fa44414b700 2021-04-03T03:10:28.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 >> v2:172.21.2.115:3300/0 conn(0x7fa4440b12b0 msgr2=0x7fa4441522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:28.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 --2- 172.21.2.111:0/1040557678 >> v2:172.21.2.115:3300/0 conn(0x7fa4440b12b0 0x7fa4441522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 >> v1:172.21.2.115:6789/0 conn(0x7fa4440603f0 legacy=0x7fa4440b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4440af6a0 con 0x7fa44414b700 2021-04-03T03:10:28.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa42c0016d0 con 0x7fa44414b700 2021-04-03T03:10:28.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 >> v1:172.21.2.111:6789/0 conn(0x7fa44414b700 legacy=0x7fa4440b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa424003760 0x7fa424003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa4240044e0 0x7fa424004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa424004e50 0x7fa42400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440af400 con 0x7fa424003760 2021-04-03T03:10:28.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa424004290 con 0x7fa424004e50 2021-04-03T03:10:28.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42400bc20 con 0x7fa4240044e0 2021-04-03T03:10:28.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.683+0000 7fa442ffd700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa424003760 0x7fa424003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa442ffd700 1 -- 172.21.2.111:0/1040557678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa4240044e0 msgr2=0x7fa424004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:28.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa442ffd700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa4240044e0 0x7fa424004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:28.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa442ffd700 1 -- 172.21.2.111:0/1040557678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa424004e50 msgr2=0x7fa42400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:28.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa442ffd700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa424004e50 0x7fa42400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:28.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa442ffd700 1 -- 172.21.2.111:0/1040557678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa424003240 con 0x7fa424003760 2021-04-03T03:10:28.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4437fe700 1 -- 172.21.2.111:0/1040557678 reap_dead start 2021-04-03T03:10:28.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa442ffd700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa424003760 0x7fa424003b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fa424004450 tx=0x7fa434007b70).ready entity=mon.0 client_cookie=1f457747161e72e6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:28.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43400c620 con 0x7fa424003760 2021-04-03T03:10:28.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43400c780 con 0x7fa424003760 2021-04-03T03:10:28.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa441ffb700 1 -- 172.21.2.111:0/1040557678 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43400cae0 con 0x7fa424003760 2021-04-03T03:10:28.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/1040557678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa424003760 msgr2=0x7fa424003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa424003760 0x7fa424003b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fa424004450 tx=0x7fa434007b70).stop 2021-04-03T03:10:28.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/1040557678 shutdown_connections 2021-04-03T03:10:28.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 --2- 172.21.2.111:0/1040557678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa424003760 0x7fa424003b80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/1040557678 >> 172.21.2.111:0/1040557678 conn(0x7fa4440b61c0 msgr2=0x7fa44405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:28.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/1040557678 shutdown_connections 2021-04-03T03:10:28.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/1040557678 wait complete. 2021-04-03T03:10:28.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 Processor -- start 2021-04-03T03:10:28.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- start start 2021-04-03T03:10:28.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa424003760 0x7fa444156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4440628d0 0x7fa4440ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa444062e20 0x7fa444067480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440a9e20 con 0x7fa4440628d0 2021-04-03T03:10:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440aa0c0 con 0x7fa424003760 2021-04-03T03:10:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440aa360 con 0x7fa444062e20 2021-04-03T03:10:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4440628d0 0x7fa4440ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4440628d0 0x7fa4440ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51408/0 (socket says 172.21.2.111:51408) 2021-04-03T03:10:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 -- 172.21.2.111:0/2654390928 learned_addr learned my addr 172.21.2.111:0/2654390928 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4437fe700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa444062e20 0x7fa444067480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 -- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa444062e20 msgr2=0x7fa444067480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa444062e20 0x7fa444067480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 -- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa424003760 msgr2=0x7fa444156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa424003760 0x7fa444156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 -- 172.21.2.111:0/2654390928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4440abc80 con 0x7fa4440628d0 2021-04-03T03:10:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4437fe700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa444062e20 0x7fa444067480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa4427fc700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4440628d0 0x7fa4440ad910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fa42c0013e0 tx=0x7fa4440af830).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:28.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa43bfff700 1 -- 172.21.2.111:0/2654390928 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42c007e40 con 0x7fa4440628d0 2021-04-03T03:10:28.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa43bfff700 1 -- 172.21.2.111:0/2654390928 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c007170 con 0x7fa4440628d0 2021-04-03T03:10:28.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa444067970 con 0x7fa4440628d0 2021-04-03T03:10:28.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa43bfff700 1 -- 172.21.2.111:0/2654390928 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42c00a7f0 con 0x7fa4440628d0 2021-04-03T03:10:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa444067e10 con 0x7fa4440628d0 2021-04-03T03:10:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa444057660 con 0x7fa4440628d0 2021-04-03T03:10:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.687+0000 7fa43bfff700 1 -- 172.21.2.111:0/2654390928 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa42c010070 con 0x7fa4440628d0 2021-04-03T03:10:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.691+0000 7fa43bfff700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa428056f20 0x7fa4280593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.691+0000 7fa43bfff700 1 -- 172.21.2.111:0/2654390928 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fa42c06faa0 con 0x7fa4440628d0 2021-04-03T03:10:28.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.691+0000 7fa442ffd700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa428056f20 0x7fa4280593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.691+0000 7fa442ffd700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa428056f20 0x7fa4280593e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fa424004450 tx=0x7fa434016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:28.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.691+0000 7fa43bfff700 1 -- 172.21.2.111:0/2654390928 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa42c0450b0 con 0x7fa4440628d0 2021-04-03T03:10:28.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa4440686d0 con 0x7fa4440628d0 2021-04-03T03:10:28.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa43bfff700 1 -- 172.21.2.111:0/2654390928 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fa42c0486d0 con 0x7fa4440628d0 2021-04-03T03:10:28.829 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:28.830 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:10:28.830 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:10:28.830 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.830 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:10:28.830 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:10:28.831 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.831 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:10:28.831 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.831 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.832 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.832 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.832 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.832 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.832 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:10:28.833 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:10:28.833 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.833 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:10:28.833 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.834 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.834 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.834 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.834 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.835 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.835 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:10:28.835 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:10:28.835 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.835 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.836 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.836 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.836 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.836 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.837 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.837 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:10:28.837 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:10:28.837 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.837 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.838 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.838 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.838 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.838 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.838 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.839 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:10:28.839 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:10:28.839 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.839 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.840 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.840 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.840 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.840 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.841 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.841 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:10:28.841 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:10:28.841 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.841 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.842 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.842 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.842 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.842 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.842 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.843 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:10:28.843 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:10:28.843 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.843 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.844 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.844 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.844 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.844 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.844 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.845 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:10:28.845 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:10:28.845 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.845 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.846 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.846 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.846 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.846 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.846 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.847 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:10:28.847 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:10:28.847 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.847 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.847 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.848 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.848 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.848 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.848 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.848 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:10:28.849 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:10:28.849 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.849 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.849 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.850 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.850 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.850 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.850 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.850 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:10:28.851 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:10:28.851 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.851 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:10:28.851 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:10:28.851 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:10:28.852 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.852 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.852 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.852 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:10:28.853 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:10:28.853 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.853 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:10:28.853 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:10:28.853 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:10:28.854 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.854 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.854 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.854 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:10:28.854 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:10:28.855 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.855 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:10:28.855 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:10:28.855 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:10:28.856 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.856 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.856 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.856 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:10:28.856 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:10:28.857 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.857 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:10:28.857 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:10:28.857 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:10:28.857 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.858 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.858 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.858 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.858 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:10:28.858 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:10:28.859 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.859 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:10:28.859 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:10:28.859 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.860 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.860 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:28.860 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:28.860 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:10:28.860 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:10:28.861 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:10:28.861 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:10:28.861 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:10:28.861 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:10:28.861 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.862 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:28.862 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:10:28.862 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:10:28.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa428056f20 msgr2=0x7fa4280593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa428056f20 0x7fa4280593e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fa424004450 tx=0x7fa434016000).stop 2021-04-03T03:10:28.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4440628d0 msgr2=0x7fa4440ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4440628d0 0x7fa4440ad910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fa42c0013e0 tx=0x7fa4440af830).stop 2021-04-03T03:10:28.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 shutdown_connections 2021-04-03T03:10:28.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa444062e20 0x7fa444067480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa428056f20 0x7fa4280593e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa424003760 0x7fa444156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:28.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 --2- 172.21.2.111:0/2654390928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4440628d0 0x7fa4440ad910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 >> 172.21.2.111:0/2654390928 conn(0x7fa4440b61c0 msgr2=0x7fa4440b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:28.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 shutdown_connections 2021-04-03T03:10:28.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.819+0000 7fa443fff700 1 -- 172.21.2.111:0/2654390928 wait complete. 2021-04-03T03:10:28.867 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:10:28.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 Processor -- start 2021-04-03T03:10:28.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- start start 2021-04-03T03:10:28.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5ee405f9c0 0x7f5ee405fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee41439c0 0x7f5ee40b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee4143e60 con 0x7f5ee40b1b70 2021-04-03T03:10:28.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee40b04e0 con 0x7f5ee405f9c0 2021-04-03T03:10:28.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40afda0 con 0x7f5ee41439c0 2021-04-03T03:10:28.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee3fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5ee40b1b70 0x7f5ee40b0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52206/0 (socket says 172.21.2.111:52206) 2021-04-03T03:10:28.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee37fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5ee405f9c0 0x7f5ee405fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee3fff700 1 -- 172.21.2.111:0/1117392637 learned_addr learned my addr 172.21.2.111:0/1117392637 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:28.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee2ffd700 1 --2- 172.21.2.111:0/1117392637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee41439c0 0x7f5ee40b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee2ffd700 1 -- 172.21.2.111:0/1117392637 >> v2:172.21.2.115:3300/0 conn(0x7f5ee405f9c0 msgr2=0x7f5ee405fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:28.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee2ffd700 1 --2- 172.21.2.111:0/1117392637 >> v2:172.21.2.115:3300/0 conn(0x7f5ee405f9c0 0x7f5ee405fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee2ffd700 1 -- 172.21.2.111:0/1117392637 >> v1:172.21.2.111:6789/0 conn(0x7f5ee40b1b70 legacy=0x7f5ee40b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee2ffd700 1 -- 172.21.2.111:0/1117392637 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40b0040 con 0x7f5ee41439c0 2021-04-03T03:10:28.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee2ffd700 1 --2- 172.21.2.111:0/1117392637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee41439c0 0x7f5ee40b0950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5edc0032b0 tx=0x7f5edc0091d0).ready entity=mon.2 client_cookie=13af661371efd1fb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:28.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee27fc700 1 -- 172.21.2.111:0/1117392637 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5edc00b9a0 con 0x7f5ee41439c0 2021-04-03T03:10:28.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee27fc700 1 -- 172.21.2.111:0/1117392637 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc00bb00 con 0x7f5ee41439c0 2021-04-03T03:10:28.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee27fc700 1 -- 172.21.2.111:0/1117392637 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5edc00bc70 con 0x7f5ee41439c0 2021-04-03T03:10:28.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- 172.21.2.111:0/1117392637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee41439c0 msgr2=0x7f5ee40b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/1117392637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee41439c0 0x7f5ee40b0950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5edc0032b0 tx=0x7f5edc0091d0).stop 2021-04-03T03:10:28.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- 172.21.2.111:0/1117392637 shutdown_connections 2021-04-03T03:10:28.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/1117392637 >> v2:172.21.2.115:3300/0 conn(0x7f5ee405f9c0 0x7f5ee405fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/1117392637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee41439c0 0x7f5ee40b0950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- 172.21.2.111:0/1117392637 >> 172.21.2.111:0/1117392637 conn(0x7f5ee40b5fb0 msgr2=0x7f5ee40b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:28.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- 172.21.2.111:0/1117392637 shutdown_connections 2021-04-03T03:10:28.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- 172.21.2.111:0/1117392637 wait complete. 2021-04-03T03:10:28.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 Processor -- start 2021-04-03T03:10:28.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.919+0000 7f5ee8e91700 1 -- start start 2021-04-03T03:10:28.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee405f9c0 0x7f5ee4065660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee40b1b70 0x7f5ee4157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee41439c0 0x7f5ee40a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4156800 con 0x7f5ee40b1b70 2021-04-03T03:10:28.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4066380 con 0x7f5ee41439c0 2021-04-03T03:10:28.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee411ec80 con 0x7f5ee405f9c0 2021-04-03T03:10:28.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee40b1b70 0x7f5ee4157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee40b1b70 0x7f5ee4157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51418/0 (socket says 172.21.2.111:51418) 2021-04-03T03:10:28.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 -- 172.21.2.111:0/449909745 learned_addr learned my addr 172.21.2.111:0/449909745 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:28.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee37fe700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee405f9c0 0x7f5ee4065660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee3fff700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee41439c0 0x7f5ee40a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 -- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee405f9c0 msgr2=0x7f5ee4065660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee405f9c0 0x7f5ee4065660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 -- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee41439c0 msgr2=0x7f5ee40a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:28.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee41439c0 0x7f5ee40a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:28.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 -- 172.21.2.111:0/449909745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee411e180 con 0x7f5ee40b1b70 2021-04-03T03:10:28.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee2ffd700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee40b1b70 0x7f5ee4157120 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f5edc0032b0 tx=0x7f5edc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:28.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee0ff9700 1 -- 172.21.2.111:0/449909745 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5edc0205e0 con 0x7f5ee40b1b70 2021-04-03T03:10:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee0ff9700 1 -- 172.21.2.111:0/449909745 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc020740 con 0x7f5ee40b1b70 2021-04-03T03:10:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee0ff9700 1 -- 172.21.2.111:0/449909745 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5edc020aa0 con 0x7f5ee40b1b70 2021-04-03T03:10:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee41569e0 con 0x7f5ee40b1b70 2021-04-03T03:10:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee4157990 con 0x7f5ee40b1b70 2021-04-03T03:10:28.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee41577a0 con 0x7f5ee40b1b70 2021-04-03T03:10:28.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee0ff9700 1 -- 172.21.2.111:0/449909745 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5edc01e070 con 0x7f5ee40b1b70 2021-04-03T03:10:28.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee0ff9700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ec4056fb0 0x7f5ec4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:28.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee37fe700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ec4056fb0 0x7f5ec4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:28.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee0ff9700 1 -- 172.21.2.111:0/449909745 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f5edc070bb0 con 0x7f5ee40b1b70 2021-04-03T03:10:28.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.923+0000 7f5ee37fe700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ec4056fb0 0x7f5ec4059470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f5ed800c000 tx=0x7f5ed800f040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:28.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:28.927+0000 7f5ee0ff9700 1 -- 172.21.2.111:0/449909745 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5edc077080 con 0x7f5ee40b1b70 2021-04-03T03:10:29.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ee411df90 con 0x7f5ee40b1b70 2021-04-03T03:10:29.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee0ff9700 1 -- 172.21.2.111:0/449909745 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f5edc031480 con 0x7f5ee40b1b70 2021-04-03T03:10:29.031 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:29.031 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":38,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:10:27.980818+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:10:29.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ec4056fb0 msgr2=0x7f5ec4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ec4056fb0 0x7f5ec4059470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f5ed800c000 tx=0x7f5ed800f040).stop 2021-04-03T03:10:29.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee40b1b70 msgr2=0x7f5ee4157120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee40b1b70 0x7f5ee4157120 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f5edc0032b0 tx=0x7f5edc016000).stop 2021-04-03T03:10:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 shutdown_connections 2021-04-03T03:10:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ec4056fb0 0x7f5ec4059470 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ee41439c0 0x7f5ee40a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ee405f9c0 0x7f5ee4065660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 --2- 172.21.2.111:0/449909745 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ee40b1b70 0x7f5ee4157120 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 >> 172.21.2.111:0/449909745 conn(0x7f5ee40b5fb0 msgr2=0x7f5ee40ad600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 shutdown_connections 2021-04-03T03:10:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.019+0000 7f5ee8e91700 1 -- 172.21.2.111:0/449909745 wait complete. 2021-04-03T03:10:29.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:10:29.040 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:10:29.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.135+0000 7f9457fff700 1 Processor -- start 2021-04-03T03:10:29.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.135+0000 7f9457fff700 1 -- start start 2021-04-03T03:10:29.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.135+0000 7f9457fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.135+0000 7f9457fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9458066250 0x7f94580666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.135+0000 7f9457fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94580b3ec0 con 0x7f94580b3ae0 2021-04-03T03:10:29.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.135+0000 7f9457fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94580b2d70 con 0x7f9458066250 2021-04-03T03:10:29.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.135+0000 7f9457fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9458064250 con 0x7f9458063c90 2021-04-03T03:10:29.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f94567fc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f94577fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9458066250 0x7f94580666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9456ffd700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f94580b3ae0 0x7f94581435c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:36522/0 (socket says 172.21.2.111:36522) 2021-04-03T03:10:29.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9456ffd700 1 -- 172.21.2.111:0/1201513650 learned_addr learned my addr 172.21.2.111:0/1201513650 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:29.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f94577fe700 1 -- 172.21.2.111:0/1201513650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 msgr2=0x7f94580b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:29.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f94577fe700 1 --2- 172.21.2.111:0/1201513650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f94577fe700 1 -- 172.21.2.111:0/1201513650 >> v1:172.21.2.115:6789/0 conn(0x7f94580b3ae0 legacy=0x7f94581435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f94577fe700 1 -- 172.21.2.111:0/1201513650 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458065db0 con 0x7f9458066250 2021-04-03T03:10:29.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f94577fe700 1 --2- 172.21.2.111:0/1201513650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9458066250 0x7f94580666b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f94500032b0 tx=0x7f945000b460).ready entity=mon.1 client_cookie=b627424c3e4917a6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9455ffb700 1 -- 172.21.2.111:0/1201513650 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9450010660 con 0x7f9458066250 2021-04-03T03:10:29.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9455ffb700 1 -- 172.21.2.111:0/1201513650 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94500107c0 con 0x7f9458066250 2021-04-03T03:10:29.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9455ffb700 1 -- 172.21.2.111:0/1201513650 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9450010b20 con 0x7f9458066250 2021-04-03T03:10:29.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 -- 172.21.2.111:0/1201513650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9458066250 msgr2=0x7f94580666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 --2- 172.21.2.111:0/1201513650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9458066250 0x7f94580666b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f94500032b0 tx=0x7f945000b460).stop 2021-04-03T03:10:29.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 -- 172.21.2.111:0/1201513650 shutdown_connections 2021-04-03T03:10:29.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 --2- 172.21.2.111:0/1201513650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9458066250 0x7f94580666b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 --2- 172.21.2.111:0/1201513650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 -- 172.21.2.111:0/1201513650 >> 172.21.2.111:0/1201513650 conn(0x7f94580622a0 msgr2=0x7f9458060ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 -- 172.21.2.111:0/1201513650 shutdown_connections 2021-04-03T03:10:29.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 -- 172.21.2.111:0/1201513650 wait complete. 2021-04-03T03:10:29.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 Processor -- start 2021-04-03T03:10:29.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.139+0000 7f9457fff700 1 -- start start 2021-04-03T03:10:29.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9458066250 0x7f9458066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f94580b3ae0 0x7f945813f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580ad880 con 0x7f9458066250 2021-04-03T03:10:29.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580adb20 con 0x7f94580b3ae0 2021-04-03T03:10:29.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580addc0 con 0x7f9458063c90 2021-04-03T03:10:29.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9458066250 0x7f9458066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9458066250 0x7f9458066d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51436/0 (socket says 172.21.2.111:51436) 2021-04-03T03:10:29.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 -- 172.21.2.111:0/2779147363 learned_addr learned my addr 172.21.2.111:0/2779147363 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:29.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9456ffd700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 -- 172.21.2.111:0/2779147363 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 msgr2=0x7f94580ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 -- 172.21.2.111:0/2779147363 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f94580b3ae0 msgr2=0x7f945813f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f94580b3ae0 0x7f945813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 -- 172.21.2.111:0/2779147363 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458067220 con 0x7f9458066250 2021-04-03T03:10:29.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f94567fc700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9458066250 0x7f9458066d30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f944000c000 tx=0x7f9440006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f944ffff700 1 -- 172.21.2.111:0/2779147363 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f944000cae0 con 0x7f9458066250 2021-04-03T03:10:29.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f944ffff700 1 -- 172.21.2.111:0/2779147363 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944000cc40 con 0x7f9458066250 2021-04-03T03:10:29.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945813eb70 con 0x7f9458066250 2021-04-03T03:10:29.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f944ffff700 1 -- 172.21.2.111:0/2779147363 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94400123f0 con 0x7f9458066250 2021-04-03T03:10:29.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94581406e0 con 0x7f9458066250 2021-04-03T03:10:29.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9458057660 con 0x7f9458066250 2021-04-03T03:10:29.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f944ffff700 1 -- 172.21.2.111:0/2779147363 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9440012550 con 0x7f9458066250 2021-04-03T03:10:29.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f944ffff700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f943c057000 0x7f943c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f944ffff700 1 -- 172.21.2.111:0/2779147363 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f94400703b0 con 0x7f9458066250 2021-04-03T03:10:29.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9456ffd700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f943c057000 0x7f943c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.143+0000 7f9456ffd700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f943c057000 0x7f943c0594c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9448008000 tx=0x7f9448009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.147+0000 7f944ffff700 1 -- 172.21.2.111:0/2779147363 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9440045a70 con 0x7f9458066250 2021-04-03T03:10:29.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f94580685b0 con 0x7f9458066250 2021-04-03T03:10:29.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f944ffff700 1 -- 172.21.2.111:0/2779147363 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v38) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9440049090 con 0x7f9458066250 2021-04-03T03:10:29.249 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:10:29.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f943c057000 msgr2=0x7f943c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f943c057000 0x7f943c0594c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9448008000 tx=0x7f9448009040).stop 2021-04-03T03:10:29.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9458066250 msgr2=0x7f9458066d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9458066250 0x7f9458066d30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f944000c000 tx=0x7f9440006dd0).stop 2021-04-03T03:10:29.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 shutdown_connections 2021-04-03T03:10:29.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f943c057000 0x7f943c0594c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f94580b3ae0 0x7f945813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9458063c90 0x7f94580ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 --2- 172.21.2.111:0/2779147363 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9458066250 0x7f9458066d30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 >> 172.21.2.111:0/2779147363 conn(0x7f94580622a0 msgr2=0x7f945813d710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 shutdown_connections 2021-04-03T03:10:29.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.239+0000 7f9457fff700 1 -- 172.21.2.111:0/2779147363 wait complete. 2021-04-03T03:10:29.257 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:10:29.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 Processor -- start 2021-04-03T03:10:29.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 -- start start 2021-04-03T03:10:29.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f38bc043120 0x7f38bc006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc0091a0 con 0x7f38bc048ef0 2021-04-03T03:10:29.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc0492d0 con 0x7f38bc006610 2021-04-03T03:10:29.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc0dc3b0 con 0x7f38bc043120 2021-04-03T03:10:29.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c259c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f38bc048ef0 0x7f38bc005a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52232/0 (socket says 172.21.2.111:52232) 2021-04-03T03:10:29.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c259c700 1 -- 172.21.2.111:0/690382615 learned_addr learned my addr 172.21.2.111:0/690382615 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:29.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c1d9b700 1 --2- 172.21.2.111:0/690382615 >> v2:172.21.2.115:3300/0 conn(0x7f38bc043120 0x7f38bc006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c159a700 1 -- 172.21.2.111:0/690382615 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792978034 0 0) 0x7f38bc0492d0 con 0x7f38bc006610 2021-04-03T03:10:29.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c159a700 1 -- 172.21.2.111:0/690382615 --> v1:172.21.2.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38a8003240 con 0x7f38bc006610 2021-04-03T03:10:29.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c159a700 1 -- 172.21.2.111:0/690382615 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066684820 0 0) 0x7f38bc0091a0 con 0x7f38bc048ef0 2021-04-03T03:10:29.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c159a700 1 -- 172.21.2.111:0/690382615 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38bc0492d0 con 0x7f38bc048ef0 2021-04-03T03:10:29.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c1d9b700 1 -- 172.21.2.111:0/690382615 >> v1:172.21.2.115:6789/0 conn(0x7f38bc006610 legacy=0x7f38bc0438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c1d9b700 1 -- 172.21.2.111:0/690382615 >> v1:172.21.2.111:6789/0 conn(0x7f38bc048ef0 legacy=0x7f38bc005a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c1d9b700 1 -- 172.21.2.111:0/690382615 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0dc650 con 0x7f38bc043120 2021-04-03T03:10:29.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c1d9b700 1 --2- 172.21.2.111:0/690382615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc043120 0x7f38bc006120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f38b8002f30 tx=0x7f38b8006dd0).ready entity=mon.1 client_cookie=26cf214db5eaf402 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c159a700 1 -- 172.21.2.111:0/690382615 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38b8009740 con 0x7f38bc043120 2021-04-03T03:10:29.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c159a700 1 -- 172.21.2.111:0/690382615 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b80098a0 con 0x7f38bc043120 2021-04-03T03:10:29.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c159a700 1 -- 172.21.2.111:0/690382615 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38b8009c00 con 0x7f38bc043120 2021-04-03T03:10:29.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 -- 172.21.2.111:0/690382615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc043120 msgr2=0x7f38bc006120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 --2- 172.21.2.111:0/690382615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc043120 0x7f38bc006120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f38b8002f30 tx=0x7f38b8006dd0).stop 2021-04-03T03:10:29.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.311+0000 7f38c359e700 1 -- 172.21.2.111:0/690382615 shutdown_connections 2021-04-03T03:10:29.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 --2- 172.21.2.111:0/690382615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc043120 0x7f38bc006120 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- 172.21.2.111:0/690382615 >> 172.21.2.111:0/690382615 conn(0x7f38bc00f350 msgr2=0x7f38bc0484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- 172.21.2.111:0/690382615 shutdown_connections 2021-04-03T03:10:29.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- 172.21.2.111:0/690382615 wait complete. 2021-04-03T03:10:29.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 Processor -- start 2021-04-03T03:10:29.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- start start 2021-04-03T03:10:29.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc006610 0x7f38bc0d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38bc043120 0x7f38bc03b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38bc048ef0 0x7f38bc0eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0ea5b0 con 0x7f38bc043120 2021-04-03T03:10:29.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc041b10 con 0x7f38bc006610 2021-04-03T03:10:29.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0ea7e0 con 0x7f38bc048ef0 2021-04-03T03:10:29.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38bc043120 0x7f38bc03b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38bc043120 0x7f38bc03b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51446/0 (socket says 172.21.2.111:51446) 2021-04-03T03:10:29.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 -- 172.21.2.111:0/1479339978 learned_addr learned my addr 172.21.2.111:0/1479339978 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:29.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c259c700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc006610 0x7f38bc0d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c2d9d700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38bc048ef0 0x7f38bc0eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 -- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38bc048ef0 msgr2=0x7f38bc0eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38bc048ef0 0x7f38bc0eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 -- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc006610 msgr2=0x7f38bc0d81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc006610 0x7f38bc0d81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 -- 172.21.2.111:0/1479339978 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc03e9a0 con 0x7f38bc043120 2021-04-03T03:10:29.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c1d9b700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38bc043120 0x7f38bc03b9a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f38b8002f30 tx=0x7f38b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38b37fe700 1 -- 172.21.2.111:0/1479339978 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38b8011c10 con 0x7f38bc043120 2021-04-03T03:10:29.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38b37fe700 1 -- 172.21.2.111:0/1479339978 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b8011d70 con 0x7f38bc043120 2021-04-03T03:10:29.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc03be90 con 0x7f38bc043120 2021-04-03T03:10:29.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38b37fe700 1 -- 172.21.2.111:0/1479339978 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38b801e650 con 0x7f38bc043120 2021-04-03T03:10:29.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc0417a0 con 0x7f38bc043120 2021-04-03T03:10:29.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc041960 con 0x7f38bc043120 2021-04-03T03:10:29.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38b37fe700 1 -- 172.21.2.111:0/1479339978 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f38b8025030 con 0x7f38bc043120 2021-04-03T03:10:29.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38b37fe700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f38a0056f20 0x7f38a00593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38b37fe700 1 -- 172.21.2.111:0/1479339978 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f38b8071180 con 0x7f38bc043120 2021-04-03T03:10:29.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c259c700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f38a0056f20 0x7f38a00593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.315+0000 7f38c259c700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f38a0056f20 0x7f38a00593e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f38b400d000 tx=0x7f38b4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.319+0000 7f38b37fe700 1 -- 172.21.2.111:0/1479339978 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f38b8046790 con 0x7f38bc043120 2021-04-03T03:10:29.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.411+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38bc042550 con 0x7f38bc043120 2021-04-03T03:10:29.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.411+0000 7f38b37fe700 1 -- 172.21.2.111:0/1479339978 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38b80230a0 con 0x7f38bc043120 2021-04-03T03:10:29.424 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:10:29.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f38a0056f20 msgr2=0x7f38a00593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f38a0056f20 0x7f38a00593e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f38b400d000 tx=0x7f38b4009040).stop 2021-04-03T03:10:29.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38bc043120 msgr2=0x7f38bc03b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38bc043120 0x7f38bc03b9a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f38b8002f30 tx=0x7f38b8015000).stop 2021-04-03T03:10:29.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 shutdown_connections 2021-04-03T03:10:29.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f38bc048ef0 0x7f38bc0eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f38a0056f20 0x7f38a00593e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f38bc006610 0x7f38bc0d81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 --2- 172.21.2.111:0/1479339978 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f38bc043120 0x7f38bc03b9a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 >> 172.21.2.111:0/1479339978 conn(0x7f38bc00f350 msgr2=0x7f38bc0d9940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 shutdown_connections 2021-04-03T03:10:29.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.415+0000 7f38c359e700 1 -- 172.21.2.111:0/1479339978 wait complete. 2021-04-03T03:10:29.432 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:10:29.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf3de7700 1 Processor -- start 2021-04-03T03:10:29.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf3de7700 1 -- start start 2021-04-03T03:10:29.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf3de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b0710 0x7f5cf40b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf3de7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5cf40b3740 0x7f5cf4143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf3de7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf40aa120 con 0x7f5cf40600e0 2021-04-03T03:10:29.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf3de7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf40b1f00 con 0x7f5cf40b3740 2021-04-03T03:10:29.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf3de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40b0af0 con 0x7f5cf40b0710 2021-04-03T03:10:29.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf35e6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5cf40b3740 0x7f5cf4143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf25e4700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5cf40600e0 0x7f5cf40b3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52248/0 (socket says 172.21.2.111:52248) 2021-04-03T03:10:29.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.483+0000 7f5cf25e4700 1 -- 172.21.2.111:0/373642314 learned_addr learned my addr 172.21.2.111:0/373642314 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:29.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf2de5700 1 --2- 172.21.2.111:0/373642314 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b0710 0x7f5cf40b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf35e6700 1 -- 172.21.2.111:0/373642314 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b0710 msgr2=0x7f5cf40b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:29.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf35e6700 1 --2- 172.21.2.111:0/373642314 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b0710 0x7f5cf40b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf35e6700 1 -- 172.21.2.111:0/373642314 >> v1:172.21.2.111:6789/0 conn(0x7f5cf40600e0 legacy=0x7f5cf40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf35e6700 1 -- 172.21.2.111:0/373642314 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40b12d0 con 0x7f5cf40b3740 2021-04-03T03:10:29.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf35e6700 1 --2- 172.21.2.111:0/373642314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b3740 0x7f5cf4143ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f5ce40010d0 tx=0x7f5cec0091d0).ready entity=mon.0 client_cookie=4e2fac7c7f0ce849 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf1de3700 1 -- 172.21.2.111:0/373642314 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cec00b910 con 0x7f5cf40b3740 2021-04-03T03:10:29.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf1de3700 1 -- 172.21.2.111:0/373642314 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec00ba70 con 0x7f5cf40b3740 2021-04-03T03:10:29.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf1de3700 1 -- 172.21.2.111:0/373642314 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cec00bbe0 con 0x7f5cf40b3740 2021-04-03T03:10:29.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- 172.21.2.111:0/373642314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b3740 msgr2=0x7f5cf4143ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/373642314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b3740 0x7f5cf4143ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f5ce40010d0 tx=0x7f5cec0091d0).stop 2021-04-03T03:10:29.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- 172.21.2.111:0/373642314 shutdown_connections 2021-04-03T03:10:29.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/373642314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b3740 0x7f5cf4143ce0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/373642314 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b0710 0x7f5cf40b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- 172.21.2.111:0/373642314 >> 172.21.2.111:0/373642314 conn(0x7f5cf4063d70 msgr2=0x7f5cf4064180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- 172.21.2.111:0/373642314 shutdown_connections 2021-04-03T03:10:29.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- 172.21.2.111:0/373642314 wait complete. 2021-04-03T03:10:29.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 Processor -- start 2021-04-03T03:10:29.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- start start 2021-04-03T03:10:29.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5cf40600e0 0x7f5cf4155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b0710 0x7f5cf4065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b3740 0x7f5cf40660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4157340 con 0x7f5cf40b0710 2021-04-03T03:10:29.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41575c0 con 0x7f5cf40600e0 2021-04-03T03:10:29.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4157860 con 0x7f5cf40b3740 2021-04-03T03:10:29.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b0710 0x7f5cf4065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b0710 0x7f5cf4065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51464/0 (socket says 172.21.2.111:51464) 2021-04-03T03:10:29.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 -- 172.21.2.111:0/3172248389 learned_addr learned my addr 172.21.2.111:0/3172248389 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:29.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 -- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b3740 msgr2=0x7f5cf40660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf35e6700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b3740 0x7f5cf40660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b3740 0x7f5cf40660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 -- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5cf40600e0 msgr2=0x7f5cf4155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf2de5700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5cf40600e0 0x7f5cf4155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5cf40600e0 0x7f5cf4155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 -- 172.21.2.111:0/3172248389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40665b0 con 0x7f5cf40b0710 2021-04-03T03:10:29.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf25e4700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b0710 0x7f5cf4065bd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f5ce4001190 tx=0x7f5ce4007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cebfff700 1 -- 172.21.2.111:0/3172248389 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ce4007320 con 0x7f5cf40b0710 2021-04-03T03:10:29.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cebfff700 1 -- 172.21.2.111:0/3172248389 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4007480 con 0x7f5cf40b0710 2021-04-03T03:10:29.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf41564a0 con 0x7f5cf40b0710 2021-04-03T03:10:29.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cebfff700 1 -- 172.21.2.111:0/3172248389 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ce40057e0 con 0x7f5cf40b0710 2021-04-03T03:10:29.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.487+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40aae70 con 0x7f5cf40b0710 2021-04-03T03:10:29.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.491+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf4057660 con 0x7f5cf40b0710 2021-04-03T03:10:29.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.491+0000 7f5cebfff700 1 -- 172.21.2.111:0/3172248389 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5ce4010030 con 0x7f5cf40b0710 2021-04-03T03:10:29.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.491+0000 7f5cebfff700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5cd8056f60 0x7f5cd8059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.491+0000 7f5cebfff700 1 -- 172.21.2.111:0/3172248389 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f5ce406fcb0 con 0x7f5cf40b0710 2021-04-03T03:10:29.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.491+0000 7f5cf2de5700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5cd8056f60 0x7f5cd8059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.491+0000 7f5cf2de5700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5cd8056f60 0x7f5cd8059420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5cdc008000 tx=0x7f5cdc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.491+0000 7f5cebfff700 1 -- 172.21.2.111:0/3172248389 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5ce4045480 con 0x7f5cf40b0710 2021-04-03T03:10:29.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5cf40a9970 con 0x7f5cf40b0710 2021-04-03T03:10:29.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cebfff700 1 -- 172.21.2.111:0/3172248389 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5ce4048aa0 con 0x7f5cf40b0710 2021-04-03T03:10:29.598 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:29.598 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:10:29.598 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:29.599 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:10:29.599 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:10:29.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:10:29.599 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:10:29.600 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:10:29.600 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:10:29.600 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:10:29.600 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:10:29.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5cd8056f60 msgr2=0x7f5cd8059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5cd8056f60 0x7f5cd8059420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5cdc008000 tx=0x7f5cdc008040).stop 2021-04-03T03:10:29.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b0710 msgr2=0x7f5cf4065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b0710 0x7f5cf4065bd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f5ce4001190 tx=0x7f5ce4007be0).stop 2021-04-03T03:10:29.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 shutdown_connections 2021-04-03T03:10:29.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5cf40b3740 0x7f5cf40660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5cd8056f60 0x7f5cd8059420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5cf40600e0 0x7f5cf4155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 --2- 172.21.2.111:0/3172248389 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5cf40b0710 0x7f5cf4065bd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 >> 172.21.2.111:0/3172248389 conn(0x7f5cf4063d70 msgr2=0x7f5cf405c960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 shutdown_connections 2021-04-03T03:10:29.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.587+0000 7f5cf3de7700 1 -- 172.21.2.111:0/3172248389 wait complete. 2021-04-03T03:10:29.605 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:10:29.606 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:10:29.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f896534d700 1 Processor -- start 2021-04-03T03:10:29.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f896534d700 1 -- start start 2021-04-03T03:10:29.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f896534d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f89600add50 0x7f8960147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f896534d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8960144ef0 0x7f8960147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f896534d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89600607d0 con 0x7f89600603f0 2021-04-03T03:10:29.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f896534d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896014b090 con 0x7f89600add50 2021-04-03T03:10:29.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f896534d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896014b330 con 0x7f8960144ef0 2021-04-03T03:10:29.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f895f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f89600add50 0x7f8960147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f895f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f89600add50 0x7f8960147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51470/0 (socket says 172.21.2.111:51470) 2021-04-03T03:10:29.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.655+0000 7f895f7fe700 1 -- 172.21.2.111:0/70896012 learned_addr learned my addr 172.21.2.111:0/70896012 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:29.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 -- 172.21.2.111:0/70896012 >> v2:172.21.2.115:3300/0 conn(0x7f8960144ef0 msgr2=0x7f8960147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:29.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 --2- 172.21.2.111:0/70896012 >> v2:172.21.2.115:3300/0 conn(0x7f8960144ef0 0x7f8960147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 -- 172.21.2.111:0/70896012 >> v1:172.21.2.115:6789/0 conn(0x7f89600603f0 legacy=0x7f89601447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:29.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 -- 172.21.2.111:0/70896012 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896014b5d0 con 0x7f89600add50 2021-04-03T03:10:29.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 --2- 172.21.2.111:0/70896012 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f8960147740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f89600ae520 tx=0x7f8948006e90).ready entity=mon.0 client_cookie=d57c6a976085c1c4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895effd700 1 -- 172.21.2.111:0/70896012 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8948009930 con 0x7f89600add50 2021-04-03T03:10:29.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895effd700 1 -- 172.21.2.111:0/70896012 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8948009a90 con 0x7f89600add50 2021-04-03T03:10:29.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895effd700 1 -- 172.21.2.111:0/70896012 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8948009e30 con 0x7f89600add50 2021-04-03T03:10:29.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- 172.21.2.111:0/70896012 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 msgr2=0x7f8960147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 --2- 172.21.2.111:0/70896012 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f8960147740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f89600ae520 tx=0x7f8948006e90).stop 2021-04-03T03:10:29.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- 172.21.2.111:0/70896012 shutdown_connections 2021-04-03T03:10:29.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 --2- 172.21.2.111:0/70896012 >> v2:172.21.2.115:3300/0 conn(0x7f8960144ef0 0x7f8960147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 --2- 172.21.2.111:0/70896012 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f8960147740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- 172.21.2.111:0/70896012 >> 172.21.2.111:0/70896012 conn(0x7f89600b61c0 msgr2=0x7f896005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- 172.21.2.111:0/70896012 shutdown_connections 2021-04-03T03:10:29.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- 172.21.2.111:0/70896012 wait complete. 2021-04-03T03:10:29.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 Processor -- start 2021-04-03T03:10:29.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- start start 2021-04-03T03:10:29.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89600603f0 0x7f89601455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f896015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8960144ef0 0x7f89600a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600a8840 con 0x7f89600add50 2021-04-03T03:10:29.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601474a0 con 0x7f8960144ef0 2021-04-03T03:10:29.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8960147690 con 0x7f89600603f0 2021-04-03T03:10:29.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f896015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f896015aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51474/0 (socket says 172.21.2.111:51474) 2021-04-03T03:10:29.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 -- 172.21.2.111:0/406207365 learned_addr learned my addr 172.21.2.111:0/406207365 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:29.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 -- 172.21.2.111:0/406207365 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89600603f0 msgr2=0x7f89601455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89600603f0 0x7f89601455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 -- 172.21.2.111:0/406207365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8960144ef0 msgr2=0x7f89600a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8960144ef0 0x7f89600a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 -- 172.21.2.111:0/406207365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960147930 con 0x7f89600add50 2021-04-03T03:10:29.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895f7fe700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f896015aad0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f89600ae520 tx=0x7f8948015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895d7fa700 1 -- 172.21.2.111:0/406207365 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8948011e10 con 0x7f89600add50 2021-04-03T03:10:29.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895d7fa700 1 -- 172.21.2.111:0/406207365 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894801e460 con 0x7f89600add50 2021-04-03T03:10:29.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8960067ec0 con 0x7f89600add50 2021-04-03T03:10:29.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f895d7fa700 1 -- 172.21.2.111:0/406207365 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f894801e7c0 con 0x7f89600add50 2021-04-03T03:10:29.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.659+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89600a8570 con 0x7f89600add50 2021-04-03T03:10:29.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.663+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8960057660 con 0x7f89600add50 2021-04-03T03:10:29.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.663+0000 7f895d7fa700 1 -- 172.21.2.111:0/406207365 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8948025030 con 0x7f89600add50 2021-04-03T03:10:29.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.663+0000 7f895d7fa700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8944056f20 0x7f89440593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.663+0000 7f895d7fa700 1 -- 172.21.2.111:0/406207365 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f8948071210 con 0x7f89600add50 2021-04-03T03:10:29.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.663+0000 7f895ffff700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8944056f20 0x7f89440593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.663+0000 7f895ffff700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8944056f20 0x7f89440593e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f8950002f50 tx=0x7f8950006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.663+0000 7f895d7fa700 1 -- 172.21.2.111:0/406207365 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8948046930 con 0x7f89600add50 2021-04-03T03:10:29.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.755+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f89600a7770 con 0x7f89600add50 2021-04-03T03:10:29.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.895+0000 7f895d7fa700 1 -- 172.21.2.111:0/406207365 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v39) v1 ==== 133+0+0 (secure 0 0 0) 0x7f89480230a0 con 0x7f89600add50 2021-04-03T03:10:29.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8944056f20 msgr2=0x7f89440593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8944056f20 0x7f89440593e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f8950002f50 tx=0x7f8950006990).stop 2021-04-03T03:10:29.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 msgr2=0x7f896015aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f896015aad0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f89600ae520 tx=0x7f8948015000).stop 2021-04-03T03:10:29.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 shutdown_connections 2021-04-03T03:10:29.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8944056f20 0x7f89440593e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8960144ef0 0x7f89600a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f89600603f0 0x7f89601455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 --2- 172.21.2.111:0/406207365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f89600add50 0x7f896015aad0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 >> 172.21.2.111:0/406207365 conn(0x7f89600b61c0 msgr2=0x7f896005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 shutdown_connections 2021-04-03T03:10:29.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.899+0000 7f896534d700 1 -- 172.21.2.111:0/406207365 wait complete. 2021-04-03T03:10:29.914 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:10:29.916 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:10:29.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 Processor -- start 2021-04-03T03:10:29.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- start start 2021-04-03T03:10:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f31dc0dbba0 0x7f31dc0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f31dc041060 0x7f31dc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc0dc560 con 0x7f31dc03d560 2021-04-03T03:10:29.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31dc03de60 con 0x7f31dc041060 2021-04-03T03:10:29.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31dc0dc800 con 0x7f31dc0dbba0 2021-04-03T03:10:29.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f31dc041060 0x7f31dc03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f31dc041060 0x7f31dc03fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51486/0 (socket says 172.21.2.111:51486) 2021-04-03T03:10:29.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 -- 172.21.2.111:0/4038640519 learned_addr learned my addr 172.21.2.111:0/4038640519 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:29.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e8ccb700 1 --2- 172.21.2.111:0/4038640519 >> v2:172.21.2.115:3300/0 conn(0x7f31dc0dbba0 0x7f31dc0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 -- 172.21.2.111:0/4038640519 >> v2:172.21.2.115:3300/0 conn(0x7f31dc0dbba0 msgr2=0x7f31dc0dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:29.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 --2- 172.21.2.111:0/4038640519 >> v2:172.21.2.115:3300/0 conn(0x7f31dc0dbba0 0x7f31dc0dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 -- 172.21.2.111:0/4038640519 >> v1:172.21.2.115:6789/0 conn(0x7f31dc03d560 legacy=0x7f31dc041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:29.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 -- 172.21.2.111:0/4038640519 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc03da30 con 0x7f31dc041060 2021-04-03T03:10:29.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e8ccb700 1 --2- 172.21.2.111:0/4038640519 >> v2:172.21.2.115:3300/0 conn(0x7f31dc0dbba0 0x7f31dc0dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:29.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e3fff700 1 --2- 172.21.2.111:0/4038640519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc041060 0x7f31dc03fa80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f31dc0087d0 tx=0x7f31d4006e90).ready entity=mon.0 client_cookie=c8853ee925b8c886 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e37fe700 1 -- 172.21.2.111:0/4038640519 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d4009930 con 0x7f31dc041060 2021-04-03T03:10:29.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e37fe700 1 -- 172.21.2.111:0/4038640519 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d4009a90 con 0x7f31dc041060 2021-04-03T03:10:29.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e37fe700 1 -- 172.21.2.111:0/4038640519 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d4009e30 con 0x7f31dc041060 2021-04-03T03:10:29.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- 172.21.2.111:0/4038640519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc041060 msgr2=0x7f31dc03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/4038640519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc041060 0x7f31dc03fa80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f31dc0087d0 tx=0x7f31d4006e90).stop 2021-04-03T03:10:29.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- 172.21.2.111:0/4038640519 shutdown_connections 2021-04-03T03:10:29.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/4038640519 >> v2:172.21.2.115:3300/0 conn(0x7f31dc0dbba0 0x7f31dc0dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/4038640519 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc041060 0x7f31dc03fa80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- 172.21.2.111:0/4038640519 >> 172.21.2.111:0/4038640519 conn(0x7f31dc0462b0 msgr2=0x7f31dc0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:29.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- 172.21.2.111:0/4038640519 shutdown_connections 2021-04-03T03:10:29.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- 172.21.2.111:0/4038640519 wait complete. 2021-04-03T03:10:29.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 Processor -- start 2021-04-03T03:10:29.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 -- start start 2021-04-03T03:10:29.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.967+0000 7f31e9ccd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc03d560 0x7f31dc0d6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31dc041060 0x7f31dc0d6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31dc0dbba0 0x7f31dc0de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0dd2d0 con 0x7f31dc03d560 2021-04-03T03:10:29.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0dd030 con 0x7f31dc041060 2021-04-03T03:10:29.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0a0c70 con 0x7f31dc0dbba0 2021-04-03T03:10:29.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc03d560 0x7f31dc0d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc03d560 0x7f31dc0d6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51488/0 (socket says 172.21.2.111:51488) 2021-04-03T03:10:29.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 -- 172.21.2.111:0/3607971534 learned_addr learned my addr 172.21.2.111:0/3607971534 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:29.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e94cc700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31dc0dbba0 0x7f31dc0de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 -- 172.21.2.111:0/3607971534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31dc0dbba0 msgr2=0x7f31dc0de6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31dc0dbba0 0x7f31dc0de6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:29.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 -- 172.21.2.111:0/3607971534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31dc041060 msgr2=0x7f31dc0d6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:29.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31dc041060 0x7f31dc0d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:29.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 -- 172.21.2.111:0/3607971534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc0de200 con 0x7f31dc03d560 2021-04-03T03:10:29.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e8ccb700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc03d560 0x7f31dc0d6360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f31d8007000 tx=0x7f31d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e1ffb700 1 -- 172.21.2.111:0/3607971534 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d800ce30 con 0x7f31dc03d560 2021-04-03T03:10:29.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e1ffb700 1 -- 172.21.2.111:0/3607971534 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8007750 con 0x7f31dc03d560 2021-04-03T03:10:29.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e1ffb700 1 -- 172.21.2.111:0/3607971534 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d8007ab0 con 0x7f31dc03d560 2021-04-03T03:10:29.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc0ddc40 con 0x7f31dc03d560 2021-04-03T03:10:29.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc0dedc0 con 0x7f31dc03d560 2021-04-03T03:10:29.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31dc004db0 con 0x7f31dc03d560 2021-04-03T03:10:29.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e1ffb700 1 -- 172.21.2.111:0/3607971534 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f31d8007c10 con 0x7f31dc03d560 2021-04-03T03:10:29.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e1ffb700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c4056f70 0x7f31c4059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:29.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e1ffb700 1 -- 172.21.2.111:0/3607971534 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5950+0+0 (secure 0 0 0) 0x7f31d80702d0 con 0x7f31dc03d560 2021-04-03T03:10:29.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e3fff700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c4056f70 0x7f31c4059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:29.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.971+0000 7f31e3fff700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c4056f70 0x7f31c4059430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f31dc0087d0 tx=0x7f31d4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:29.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:29.975+0000 7f31e1ffb700 1 -- 172.21.2.111:0/3607971534 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f31d8045780 con 0x7f31dc03d560 2021-04-03T03:10:30.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.067+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f31dc042020 con 0x7f31dc03d560 2021-04-03T03:10:30.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.907+0000 7f31e1ffb700 1 -- 172.21.2.111:0/3607971534 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v40) v1 ==== 125+0+0 (secure 0 0 0) 0x7f31d8048da0 con 0x7f31dc03d560 2021-04-03T03:10:30.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c4056f70 msgr2=0x7f31c4059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:30.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c4056f70 0x7f31c4059430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f31dc0087d0 tx=0x7f31d4015000).stop 2021-04-03T03:10:30.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc03d560 msgr2=0x7f31dc0d6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:30.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc03d560 0x7f31dc0d6360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f31d8007000 tx=0x7f31d8008040).stop 2021-04-03T03:10:30.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 shutdown_connections 2021-04-03T03:10:30.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31dc0dbba0 0x7f31dc0de6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:30.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31dc03d560 0x7f31dc0d6360 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:30.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f31c4056f70 0x7f31c4059430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:30.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 --2- 172.21.2.111:0/3607971534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31dc041060 0x7f31dc0d6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:30.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 >> 172.21.2.111:0/3607971534 conn(0x7f31dc0462b0 msgr2=0x7f31dc041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:30.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 shutdown_connections 2021-04-03T03:10:30.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.911+0000 7f31e9ccd700 1 -- 172.21.2.111:0/3607971534 wait complete. 2021-04-03T03:10:30.924 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:10:30.928 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:10:30.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 Processor -- start 2021-04-03T03:10:30.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- start start 2021-04-03T03:10:30.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1ce40603f0 0x7f1ce41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:30.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1ce40add50 0x7f1ce4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:30.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce40607d0 con 0x7f1ce40b17b0 2021-04-03T03:10:30.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce41451c0 con 0x7f1ce40603f0 2021-04-03T03:10:30.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce414b090 con 0x7f1ce40add50 2021-04-03T03:10:30.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea1c9700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1ce40b17b0 0x7f1ce4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52290/0 (socket says 172.21.2.111:52290) 2021-04-03T03:10:30.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea1c9700 1 -- 172.21.2.111:0/2502635999 learned_addr learned my addr 172.21.2.111:0/2502635999 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:30.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce99c8700 1 --2- 172.21.2.111:0/2502635999 >> v2:172.21.2.111:3300/0 conn(0x7f1ce40603f0 0x7f1ce41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:30.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce89c6700 1 -- 172.21.2.111:0/2502635999 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052552819 0 0) 0x7f1ce40607d0 con 0x7f1ce40b17b0 2021-04-03T03:10:30.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce89c6700 1 -- 172.21.2.111:0/2502635999 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ccc003240 con 0x7f1ce40b17b0 2021-04-03T03:10:30.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce91c7700 1 --2- 172.21.2.111:0/2502635999 >> v2:172.21.2.115:3300/0 conn(0x7f1ce40add50 0x7f1ce4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:30.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce99c8700 1 -- 172.21.2.111:0/2502635999 >> v2:172.21.2.115:3300/0 conn(0x7f1ce40add50 msgr2=0x7f1ce4144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:30.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce99c8700 1 --2- 172.21.2.111:0/2502635999 >> v2:172.21.2.115:3300/0 conn(0x7f1ce40add50 0x7f1ce4144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:30.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce99c8700 1 -- 172.21.2.111:0/2502635999 >> v1:172.21.2.111:6789/0 conn(0x7f1ce40b17b0 legacy=0x7f1ce4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:30.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce99c8700 1 -- 172.21.2.111:0/2502635999 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce414b330 con 0x7f1ce40603f0 2021-04-03T03:10:30.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce91c7700 1 --2- 172.21.2.111:0/2502635999 >> v2:172.21.2.115:3300/0 conn(0x7f1ce40add50 0x7f1ce4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:30.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce99c8700 1 --2- 172.21.2.111:0/2502635999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40603f0 0x7f1ce41447e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f1cd4002f30 tx=0x7f1cd4006dd0).ready entity=mon.0 client_cookie=9cd2e27ebedee21e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:30.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce89c6700 1 -- 172.21.2.111:0/2502635999 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd40097b0 con 0x7f1ce40603f0 2021-04-03T03:10:30.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce89c6700 1 -- 172.21.2.111:0/2502635999 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd4009910 con 0x7f1ce40603f0 2021-04-03T03:10:30.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1ce89c6700 1 -- 172.21.2.111:0/2502635999 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd4009c70 con 0x7f1ce40603f0 2021-04-03T03:10:30.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2502635999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40603f0 msgr2=0x7f1ce41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:30.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2502635999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40603f0 0x7f1ce41447e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f1cd4002f30 tx=0x7f1cd4006dd0).stop 2021-04-03T03:10:30.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2502635999 shutdown_connections 2021-04-03T03:10:30.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2502635999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40603f0 0x7f1ce41447e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:30.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2502635999 >> v2:172.21.2.115:3300/0 conn(0x7f1ce40add50 0x7f1ce4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:30.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2502635999 >> 172.21.2.111:0/2502635999 conn(0x7f1ce40b61c0 msgr2=0x7f1ce405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:30.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2502635999 shutdown_connections 2021-04-03T03:10:30.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.975+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2502635999 wait complete. 2021-04-03T03:10:30.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 Processor -- start 2021-04-03T03:10:30.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 -- start start 2021-04-03T03:10:30.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1ce40603f0 0x7f1ce415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:30.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40add50 0x7f1ce4143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:30.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1ce40b17b0 0x7f1ce4156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:30.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce415aca0 con 0x7f1ce40add50 2021-04-03T03:10:30.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4159dd0 con 0x7f1ce40603f0 2021-04-03T03:10:30.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce415a070 con 0x7f1ce40b17b0 2021-04-03T03:10:30.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40add50 0x7f1ce4143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:30.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40add50 0x7f1ce4143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51504/0 (socket says 172.21.2.111:51504) 2021-04-03T03:10:30.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 -- 172.21.2.111:0/2302202549 learned_addr learned my addr 172.21.2.111:0/2302202549 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:30.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea1c9700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1ce40b17b0 0x7f1ce4156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:30.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 -- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1ce40b17b0 msgr2=0x7f1ce4156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:30.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1ce40b17b0 0x7f1ce4156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:30.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 -- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1ce40603f0 msgr2=0x7f1ce415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:30.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1ce40603f0 0x7f1ce415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:30.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 -- 172.21.2.111:0/2302202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce415a310 con 0x7f1ce40add50 2021-04-03T03:10:30.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea1c9700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1ce40b17b0 0x7f1ce4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:30.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce91c7700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40add50 0x7f1ce4143110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f1cdc007000 tx=0x7f1cdc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cd2ffd700 1 -- 172.21.2.111:0/2302202549 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cdc007900 con 0x7f1ce40add50 2021-04-03T03:10:31.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cd2ffd700 1 -- 172.21.2.111:0/2302202549 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cdc007a60 con 0x7f1ce40add50 2021-04-03T03:10:31.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce4143600 con 0x7f1ce40add50 2021-04-03T03:10:31.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce40b3c80 con 0x7f1ce40add50 2021-04-03T03:10:31.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cd2ffd700 1 -- 172.21.2.111:0/2302202549 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cdc007dc0 con 0x7f1ce40add50 2021-04-03T03:10:31.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc00033f0 con 0x7f1ce40add50 2021-04-03T03:10:31.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cd2ffd700 1 -- 172.21.2.111:0/2302202549 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1cdc01a410 con 0x7f1ce40add50 2021-04-03T03:10:31.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cd2ffd700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cc4056f70 0x7f1cc4059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1cd2ffd700 1 -- 172.21.2.111:0/2302202549 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f1cdc070420 con 0x7f1ce40add50 2021-04-03T03:10:31.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce99c8700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cc4056f70 0x7f1cc4059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.979+0000 7f1ce99c8700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cc4056f70 0x7f1cc4059430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1cd4002f30 tx=0x7f1cd4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:30.983+0000 7f1cd2ffd700 1 -- 172.21.2.111:0/2302202549 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1cdc045710 con 0x7f1ce40add50 2021-04-03T03:10:31.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.111+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f1cc0003c20 con 0x7f1ce40add50 2021-04-03T03:10:31.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cd2ffd700 1 -- 172.21.2.111:0/2302202549 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v15) v1 ==== 187+0+0 (secure 0 0 0) 0x7f1cdc048d30 con 0x7f1ce40add50 2021-04-03T03:10:31.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cc4056f70 msgr2=0x7f1cc4059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cc4056f70 0x7f1cc4059430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1cd4002f30 tx=0x7f1cd4015000).stop 2021-04-03T03:10:31.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40add50 msgr2=0x7f1ce4143110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40add50 0x7f1ce4143110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f1cdc007000 tx=0x7f1cdc008040).stop 2021-04-03T03:10:31.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 shutdown_connections 2021-04-03T03:10:31.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1ce40b17b0 0x7f1ce4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cc4056f70 0x7f1cc4059430 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1ce40603f0 0x7f1ce415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 --2- 172.21.2.111:0/2302202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1ce40add50 0x7f1ce4143110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 >> 172.21.2.111:0/2302202549 conn(0x7f1ce40b61c0 msgr2=0x7f1ce405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:31.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 shutdown_connections 2021-04-03T03:10:31.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.115+0000 7f1cea9ca700 1 -- 172.21.2.111:0/2302202549 wait complete. 2021-04-03T03:10:31.133 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 7 and data pool 8 2021-04-03T03:10:31.136 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:10:31.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 Processor -- start 2021-04-03T03:10:31.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- start start 2021-04-03T03:10:31.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe370006bc0 0x7fe370042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3700dc6e0 con 0x7fe3700dbb90 2021-04-03T03:10:31.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe370042010 con 0x7fe37003d360 2021-04-03T03:10:31.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3700d9560 con 0x7fe370006bc0 2021-04-03T03:10:31.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37d81d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe3700dbb90 0x7fe3700064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52312/0 (socket says 172.21.2.111:52312) 2021-04-03T03:10:31.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37d81d700 1 -- 172.21.2.111:0/2527103662 learned_addr learned my addr 172.21.2.111:0/2527103662 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:31.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962222799 0 0) 0x7fe3700dc6e0 con 0x7fe3700dbb90 2021-04-03T03:10:31.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe360003240 con 0x7fe3700dbb90 2021-04-03T03:10:31.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37d01c700 1 --2- 172.21.2.111:0/2527103662 >> v2:172.21.2.115:3300/0 conn(0x7fe370006bc0 0x7fe370042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3358970574 0 0) 0x7fe360003240 con 0x7fe3700dbb90 2021-04-03T03:10:31.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 >> v2:172.21.2.115:3300/0 conn(0x7fe370006bc0 msgr2=0x7fe370042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:31.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 --2- 172.21.2.111:0/2527103662 >> v2:172.21.2.115:3300/0 conn(0x7fe370006bc0 0x7fe370042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 >> v1:172.21.2.115:6789/0 conn(0x7fe37003d360 legacy=0x7fe3700056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:31.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3700422b0 con 0x7fe3700dbb90 2021-04-03T03:10:31.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe36c0016d0 con 0x7fe3700dbb90 2021-04-03T03:10:31.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 >> v1:172.21.2.111:6789/0 conn(0x7fe3700dbb90 legacy=0x7fe3700064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360003800 0x7fe360003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3600044e0 0x7fe360004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe360004e50 0x7fe36000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700d9560 con 0x7fe360004e50 2021-04-03T03:10:31.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360005300 con 0x7fe360003800 2021-04-03T03:10:31.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36000bc20 con 0x7fe3600044e0 2021-04-03T03:10:31.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe360004e50 0x7fe36000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37d81d700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360003800 0x7fe360003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37d01c700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3600044e0 0x7fe360004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 -- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3600044e0 msgr2=0x7fe360004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:31.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3600044e0 0x7fe360004960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 -- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360003800 msgr2=0x7fe360003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:31.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360003800 0x7fe360003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 -- 172.21.2.111:0/2527103662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360003240 con 0x7fe360004e50 2021-04-03T03:10:31.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 -- 172.21.2.111:0/2527103662 reap_dead start 2021-04-03T03:10:31.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37d81d700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360003800 0x7fe360003c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:31.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e01e700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe360004e50 0x7fe36000b730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fe37806b5d0 tx=0x7fe37805ad70).ready entity=mon.0 client_cookie=576c88e04f6bd2ab server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe37805bda0 con 0x7fe360004e50 2021-04-03T03:10:31.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37805df60 con 0x7fe360004e50 2021-04-03T03:10:31.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37c81b700 1 -- 172.21.2.111:0/2527103662 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3780699f0 con 0x7fe360004e50 2021-04-03T03:10:31.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- 172.21.2.111:0/2527103662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe360004e50 msgr2=0x7fe36000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe360004e50 0x7fe36000b730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fe37806b5d0 tx=0x7fe37805ad70).stop 2021-04-03T03:10:31.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- 172.21.2.111:0/2527103662 shutdown_connections 2021-04-03T03:10:31.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2527103662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe360004e50 0x7fe36000b730 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- 172.21.2.111:0/2527103662 >> 172.21.2.111:0/2527103662 conn(0x7fe3700462c0 msgr2=0x7fe3700466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:31.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- 172.21.2.111:0/2527103662 shutdown_connections 2021-04-03T03:10:31.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.183+0000 7fe37e81f700 1 -- 172.21.2.111:0/2527103662 wait complete. 2021-04-03T03:10:31.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 Processor -- start 2021-04-03T03:10:31.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 -- start start 2021-04-03T03:10:31.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360004e50 0x7fe3700ea440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe370039b90 0x7fe3700dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe37003b490 0x7fe37003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37003be30 con 0x7fe370039b90 2021-04-03T03:10:31.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37003ab80 con 0x7fe360004e50 2021-04-03T03:10:31.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700dd920 con 0x7fe37003b490 2021-04-03T03:10:31.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe370039b90 0x7fe3700dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe370039b90 0x7fe3700dd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51534/0 (socket says 172.21.2.111:51534) 2021-04-03T03:10:31.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 -- 172.21.2.111:0/2554872311 learned_addr learned my addr 172.21.2.111:0/2554872311 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:31.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 -- 172.21.2.111:0/2554872311 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe37003b490 msgr2=0x7fe37003b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe37003b490 0x7fe37003b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 -- 172.21.2.111:0/2554872311 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360004e50 msgr2=0x7fe3700ea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d81d700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360004e50 0x7fe3700ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360004e50 0x7fe3700ea440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 -- 172.21.2.111:0/2554872311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37003adb0 con 0x7fe370039b90 2021-04-03T03:10:31.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d01c700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe370039b90 0x7fe3700dd0e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fe374008000 tx=0x7fe374008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe36affd700 1 -- 172.21.2.111:0/2554872311 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3740104a0 con 0x7fe370039b90 2021-04-03T03:10:31.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe36affd700 1 -- 172.21.2.111:0/2554872311 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374010600 con 0x7fe370039b90 2021-04-03T03:10:31.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3700a0360 con 0x7fe370039b90 2021-04-03T03:10:31.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe36affd700 1 -- 172.21.2.111:0/2554872311 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe374010960 con 0x7fe370039b90 2021-04-03T03:10:31.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3700dd5d0 con 0x7fe370039b90 2021-04-03T03:10:31.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe370004da0 con 0x7fe370039b90 2021-04-03T03:10:31.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe36affd700 1 -- 172.21.2.111:0/2554872311 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe374012030 con 0x7fe370039b90 2021-04-03T03:10:31.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe36affd700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe358056f60 0x7fe358059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe36affd700 1 -- 172.21.2.111:0/2554872311 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe37406fb30 con 0x7fe370039b90 2021-04-03T03:10:31.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d81d700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe358056f60 0x7fe358059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.187+0000 7fe37d81d700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe358056f60 0x7fe358059420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fe36c009000 tx=0x7fe36c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.191+0000 7fe36affd700 1 -- 172.21.2.111:0/2554872311 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe374044f20 con 0x7fe370039b90 2021-04-03T03:10:31.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.283+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe370046ff0 con 0x7fe370039b90 2021-04-03T03:10:31.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe36affd700 1 -- 172.21.2.111:0/2554872311 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16297 (secure 0 0 0) 0x7fe374048540 con 0x7fe370039b90 2021-04-03T03:10:31.296 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:31.297 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":42,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:10:31.115991+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:10:29.772278+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-04-03T03:10:30.083040+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:10:31.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe358056f60 msgr2=0x7fe358059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe358056f60 0x7fe358059420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fe36c009000 tx=0x7fe36c009040).stop 2021-04-03T03:10:31.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe370039b90 msgr2=0x7fe3700dd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe370039b90 0x7fe3700dd0e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fe374008000 tx=0x7fe374008040).stop 2021-04-03T03:10:31.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 shutdown_connections 2021-04-03T03:10:31.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe37003b490 0x7fe37003b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe358056f60 0x7fe358059420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe360004e50 0x7fe3700ea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 --2- 172.21.2.111:0/2554872311 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe370039b90 0x7fe3700dd0e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 >> 172.21.2.111:0/2554872311 conn(0x7fe3700462c0 msgr2=0x7fe3700d7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:31.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 shutdown_connections 2021-04-03T03:10:31.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.287+0000 7fe37e81f700 1 -- 172.21.2.111:0/2554872311 wait complete. 2021-04-03T03:10:31.305 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:10:31.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 Processor -- start 2021-04-03T03:10:31.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 -- start start 2021-04-03T03:10:31.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480603f0 0x7fc8480b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc84814b700 0x7fc8480b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc8480b12b0 0x7fc8480b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8480b0f60 con 0x7fc84814b700 2021-04-03T03:10:31.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8480607d0 con 0x7fc8480b12b0 2021-04-03T03:10:31.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480af400 con 0x7fc8480603f0 2021-04-03T03:10:31.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc84814b700 0x7fc8480b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc84814b700 0x7fc8480b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51542/0 (socket says 172.21.2.111:51542) 2021-04-03T03:10:31.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 -- 172.21.2.111:0/600486019 learned_addr learned my addr 172.21.2.111:0/600486019 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:31.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 -- 172.21.2.111:0/600486019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480603f0 msgr2=0x7fc8480b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:31.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 --2- 172.21.2.111:0/600486019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480603f0 0x7fc8480b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 -- 172.21.2.111:0/600486019 >> v2:172.21.2.115:3300/0 conn(0x7fc8480b12b0 msgr2=0x7fc8480b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:31.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 --2- 172.21.2.111:0/600486019 >> v2:172.21.2.115:3300/0 conn(0x7fc8480b12b0 0x7fc8480b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 -- 172.21.2.111:0/600486019 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8480af6a0 con 0x7fc84814b700 2021-04-03T03:10:31.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc8467fc700 1 --2- 172.21.2.111:0/600486019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8480b0580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fc848062740 tx=0x7fc83c006e90).ready entity=mon.0 client_cookie=8c4e8d0ac0e37fff server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc845ffb700 1 -- 172.21.2.111:0/600486019 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc83c0098e0 con 0x7fc84814b700 2021-04-03T03:10:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc845ffb700 1 -- 172.21.2.111:0/600486019 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c009a40 con 0x7fc84814b700 2021-04-03T03:10:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc845ffb700 1 -- 172.21.2.111:0/600486019 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc83c009da0 con 0x7fc84814b700 2021-04-03T03:10:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 -- 172.21.2.111:0/600486019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 msgr2=0x7fc8480b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.355+0000 7fc847fff700 1 --2- 172.21.2.111:0/600486019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8480b0580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fc848062740 tx=0x7fc83c006e90).stop 2021-04-03T03:10:31.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- 172.21.2.111:0/600486019 shutdown_connections 2021-04-03T03:10:31.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 --2- 172.21.2.111:0/600486019 >> v2:172.21.2.115:3300/0 conn(0x7fc8480b12b0 0x7fc8480b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 --2- 172.21.2.111:0/600486019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480603f0 0x7fc8480b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 --2- 172.21.2.111:0/600486019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8480b0580 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- 172.21.2.111:0/600486019 >> 172.21.2.111:0/600486019 conn(0x7fc8480b61c0 msgr2=0x7fc84805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:31.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- 172.21.2.111:0/600486019 shutdown_connections 2021-04-03T03:10:31.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- 172.21.2.111:0/600486019 wait complete. 2021-04-03T03:10:31.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 Processor -- start 2021-04-03T03:10:31.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- start start 2021-04-03T03:10:31.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc8480603f0 0x7fc848159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480b12b0 0x7fc848148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8481455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848064d20 con 0x7fc84814b700 2021-04-03T03:10:31.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848064fc0 con 0x7fc8480603f0 2021-04-03T03:10:31.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc848065260 con 0x7fc8480b12b0 2021-04-03T03:10:31.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8481455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8481455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51550/0 (socket says 172.21.2.111:51550) 2021-04-03T03:10:31.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 -- 172.21.2.111:0/2800435987 learned_addr learned my addr 172.21.2.111:0/2800435987 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:31.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 -- 172.21.2.111:0/2800435987 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480b12b0 msgr2=0x7fc848148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480b12b0 0x7fc848148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 -- 172.21.2.111:0/2800435987 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc8480603f0 msgr2=0x7fc848159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc8480603f0 0x7fc848159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 -- 172.21.2.111:0/2800435987 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc848063b60 con 0x7fc84814b700 2021-04-03T03:10:31.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc8477fe700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8481455c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fc8400032b0 tx=0x7fc8400091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc837fff700 1 -- 172.21.2.111:0/2800435987 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc84000c9e0 con 0x7fc84814b700 2021-04-03T03:10:31.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc837fff700 1 -- 172.21.2.111:0/2800435987 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000cb40 con 0x7fc84814b700 2021-04-03T03:10:31.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc837fff700 1 -- 172.21.2.111:0/2800435987 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc84000cea0 con 0x7fc84814b700 2021-04-03T03:10:31.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc848149420 con 0x7fc84814b700 2021-04-03T03:10:31.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc848067750 con 0x7fc84814b700 2021-04-03T03:10:31.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc848064210 con 0x7fc84814b700 2021-04-03T03:10:31.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc837fff700 1 -- 172.21.2.111:0/2800435987 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc84001c070 con 0x7fc84814b700 2021-04-03T03:10:31.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc837fff700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc824056ec0 0x7fc824059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc837fff700 1 -- 172.21.2.111:0/2800435987 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc84006fde0 con 0x7fc84814b700 2021-04-03T03:10:31.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.359+0000 7fc846ffd700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc824056ec0 0x7fc824059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.363+0000 7fc846ffd700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc824056ec0 0x7fc824059380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc838002f30 tx=0x7fc838006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.363+0000 7fc837fff700 1 -- 172.21.2.111:0/2800435987 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc8400450c0 con 0x7fc84814b700 2021-04-03T03:10:31.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.455+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc848057660 con 0x7fc84814b700 2021-04-03T03:10:31.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.455+0000 7fc837fff700 1 -- 172.21.2.111:0/2800435987 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16297 (secure 0 0 0) 0x7fc8400486e0 con 0x7fc84814b700 2021-04-03T03:10:31.468 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:31.468 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":42,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:10:31.115991+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:10:29.772278+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-04-03T03:10:30.083040+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:10:31.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc824056ec0 msgr2=0x7fc824059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc824056ec0 0x7fc824059380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc838002f30 tx=0x7fc838006a60).stop 2021-04-03T03:10:31.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 msgr2=0x7fc8481455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8481455c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fc8400032b0 tx=0x7fc8400091a0).stop 2021-04-03T03:10:31.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 shutdown_connections 2021-04-03T03:10:31.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc84814b700 0x7fc8481455c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc824056ec0 0x7fc824059380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc8480603f0 0x7fc848159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 --2- 172.21.2.111:0/2800435987 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc8480b12b0 0x7fc848148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 >> 172.21.2.111:0/2800435987 conn(0x7fc8480b61c0 msgr2=0x7fc848067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:31.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 shutdown_connections 2021-04-03T03:10:31.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.459+0000 7fc847fff700 1 -- 172.21.2.111:0/2800435987 wait complete. 2021-04-03T03:10:31.476 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:10:31.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 Processor -- start 2021-04-03T03:10:31.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- start start 2021-04-03T03:10:31.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200add50 0x7ff020147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff020144ef0 0x7ff020147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0200607d0 con 0x7ff0200603f0 2021-04-03T03:10:31.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02014b090 con 0x7ff020144ef0 2021-04-03T03:10:31.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014b330 con 0x7ff0200add50 2021-04-03T03:10:31.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff02572b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff020144ef0 0x7ff020147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff024f2a700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff0200603f0 0x7ff0201447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52346/0 (socket says 172.21.2.111:52346) 2021-04-03T03:10:31.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff024f2a700 1 -- 172.21.2.111:0/4037390229 learned_addr learned my addr 172.21.2.111:0/4037390229 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:31.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff02572b700 1 -- 172.21.2.111:0/4037390229 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200add50 msgr2=0x7ff020147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:31.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff02572b700 1 --2- 172.21.2.111:0/4037390229 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200add50 0x7ff020147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff02572b700 1 -- 172.21.2.111:0/4037390229 >> v1:172.21.2.111:6789/0 conn(0x7ff0200603f0 legacy=0x7ff0201447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff02572b700 1 -- 172.21.2.111:0/4037390229 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02014b5d0 con 0x7ff020144ef0 2021-04-03T03:10:31.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff02572b700 1 --2- 172.21.2.111:0/4037390229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020147c30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7ff0180032b0 tx=0x7ff0180091d0).ready entity=mon.0 client_cookie=66d0a6507d6cf67a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff01f7fe700 1 -- 172.21.2.111:0/4037390229 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01800ba50 con 0x7ff020144ef0 2021-04-03T03:10:31.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff01f7fe700 1 -- 172.21.2.111:0/4037390229 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01800bbb0 con 0x7ff020144ef0 2021-04-03T03:10:31.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff01f7fe700 1 -- 172.21.2.111:0/4037390229 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01800bd20 con 0x7ff020144ef0 2021-04-03T03:10:31.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- 172.21.2.111:0/4037390229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 msgr2=0x7ff020147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 --2- 172.21.2.111:0/4037390229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020147c30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7ff0180032b0 tx=0x7ff0180091d0).stop 2021-04-03T03:10:31.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- 172.21.2.111:0/4037390229 shutdown_connections 2021-04-03T03:10:31.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 --2- 172.21.2.111:0/4037390229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020147c30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:31.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 --2- 172.21.2.111:0/4037390229 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200add50 0x7ff020147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- 172.21.2.111:0/4037390229 >> 172.21.2.111:0/4037390229 conn(0x7ff0200b61c0 msgr2=0x7ff02005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:31.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- 172.21.2.111:0/4037390229 shutdown_connections 2021-04-03T03:10:31.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.527+0000 7ff025f2c700 1 -- 172.21.2.111:0/4037390229 wait complete. 2021-04-03T03:10:31.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 Processor -- start 2021-04-03T03:10:31.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 -- start start 2021-04-03T03:10:31.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200603f0 0x7ff0200a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0200add50 0x7ff020156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020063d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff020156f10 con 0x7ff020144ef0 2021-04-03T03:10:31.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff020157170 con 0x7ff0200add50 2021-04-03T03:10:31.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff020157410 con 0x7ff0200603f0 2021-04-03T03:10:31.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020063d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020063d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51562/0 (socket says 172.21.2.111:51562) 2021-04-03T03:10:31.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 -- 172.21.2.111:0/711982924 learned_addr learned my addr 172.21.2.111:0/711982924 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:31.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 -- 172.21.2.111:0/711982924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200603f0 msgr2=0x7ff0200a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200603f0 0x7ff0200a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 -- 172.21.2.111:0/711982924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0200add50 msgr2=0x7ff020156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:31.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0200add50 0x7ff020156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:31.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 -- 172.21.2.111:0/711982924 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0201576b0 con 0x7ff020144ef0 2021-04-03T03:10:31.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff02572b700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020063d00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7ff0180032b0 tx=0x7ff018016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff01dffb700 1 -- 172.21.2.111:0/711982924 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0180205f0 con 0x7ff020144ef0 2021-04-03T03:10:31.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff01dffb700 1 -- 172.21.2.111:0/711982924 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018020750 con 0x7ff020144ef0 2021-04-03T03:10:31.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0200641f0 con 0x7ff020144ef0 2021-04-03T03:10:31.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff01dffb700 1 -- 172.21.2.111:0/711982924 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff018020ab0 con 0x7ff020144ef0 2021-04-03T03:10:31.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff020064650 con 0x7ff020144ef0 2021-04-03T03:10:31.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff020157840 con 0x7ff020144ef0 2021-04-03T03:10:31.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff01dffb700 1 -- 172.21.2.111:0/711982924 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff018027030 con 0x7ff020144ef0 2021-04-03T03:10:31.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff01dffb700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff004056fb0 0x7ff004059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:31.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff01dffb700 1 -- 172.21.2.111:0/711982924 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff0180716d0 con 0x7ff020144ef0 2021-04-03T03:10:31.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff024f2a700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff004056fb0 0x7ff004059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:31.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.531+0000 7ff024f2a700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff004056fb0 0x7ff004059470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff0200b5600 tx=0x7ff0100077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:31.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.535+0000 7ff01dffb700 1 -- 172.21.2.111:0/711982924 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff018046ac0 con 0x7ff020144ef0 2021-04-03T03:10:31.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:31.659+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff020057660 con 0x7ff020144ef0 2021-04-03T03:10:32.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff01dffb700 1 -- 172.21.2.111:0/711982924 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff0180250b0 con 0x7ff020144ef0 2021-04-03T03:10:32.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff004056fb0 msgr2=0x7ff004059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff004056fb0 0x7ff004059470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff0200b5600 tx=0x7ff0100077e0).stop 2021-04-03T03:10:32.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 msgr2=0x7ff020063d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020063d00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7ff0180032b0 tx=0x7ff018016000).stop 2021-04-03T03:10:32.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 shutdown_connections 2021-04-03T03:10:32.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff020144ef0 0x7ff020063d00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0200603f0 0x7ff0200a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff004056fb0 0x7ff004059470 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 --2- 172.21.2.111:0/711982924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0200add50 0x7ff020156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 >> 172.21.2.111:0/711982924 conn(0x7ff0200b61c0 msgr2=0x7ff020143e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 shutdown_connections 2021-04-03T03:10:32.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.123+0000 7ff025f2c700 1 -- 172.21.2.111:0/711982924 wait complete. 2021-04-03T03:10:32.142 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:32.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe361e66700 1 Processor -- start 2021-04-03T03:10:32.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe361e66700 1 -- start start 2021-04-03T03:10:32.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe361e66700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe35c0603f0 0x7fe35c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe361e66700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe35c14b700 0x7fe35c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe361e66700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0607d0 con 0x7fe35c0b12b0 2021-04-03T03:10:32.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe361e66700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c0af400 con 0x7fe35c0603f0 2021-04-03T03:10:32.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe361e66700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c0af6a0 con 0x7fe35c14b700 2021-04-03T03:10:32.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe35c0603f0 0x7fe35c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe35c0603f0 0x7fe35c0b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51568/0 (socket says 172.21.2.111:51568) 2021-04-03T03:10:32.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 -- 172.21.2.111:0/2474678362 learned_addr learned my addr 172.21.2.111:0/2474678362 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 -- 172.21.2.111:0/2474678362 >> v2:172.21.2.115:3300/0 conn(0x7fe35c14b700 msgr2=0x7fe35c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 --2- 172.21.2.111:0/2474678362 >> v2:172.21.2.115:3300/0 conn(0x7fe35c14b700 0x7fe35c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 -- 172.21.2.111:0/2474678362 >> v1:172.21.2.115:6789/0 conn(0x7fe35c0b12b0 legacy=0x7fe35c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 -- 172.21.2.111:0/2474678362 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0afb40 con 0x7fe35c0603f0 2021-04-03T03:10:32.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe360e64700 1 --2- 172.21.2.111:0/2474678362 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0b0090 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fe35c06a3b0 tx=0x7fe34c006e90).ready entity=mon.0 client_cookie=65738c0776182 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.187+0000 7fe35b7fe700 1 -- 172.21.2.111:0/2474678362 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c009930 con 0x7fe35c0603f0 2021-04-03T03:10:32.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe35b7fe700 1 -- 172.21.2.111:0/2474678362 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c009a90 con 0x7fe35c0603f0 2021-04-03T03:10:32.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe35b7fe700 1 -- 172.21.2.111:0/2474678362 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c009e30 con 0x7fe35c0603f0 2021-04-03T03:10:32.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/2474678362 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 msgr2=0x7fe35c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 --2- 172.21.2.111:0/2474678362 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0b0090 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fe35c06a3b0 tx=0x7fe34c006e90).stop 2021-04-03T03:10:32.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/2474678362 shutdown_connections 2021-04-03T03:10:32.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 --2- 172.21.2.111:0/2474678362 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0b0090 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 --2- 172.21.2.111:0/2474678362 >> v2:172.21.2.115:3300/0 conn(0x7fe35c14b700 0x7fe35c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/2474678362 >> 172.21.2.111:0/2474678362 conn(0x7fe35c0b61c0 msgr2=0x7fe35c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/2474678362 shutdown_connections 2021-04-03T03:10:32.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/2474678362 wait complete. 2021-04-03T03:10:32.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 Processor -- start 2021-04-03T03:10:32.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- start start 2021-04-03T03:10:32.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe35c0b12b0 0x7fe35c062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe35c14b700 0x7fe35c064960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0a9ce0 con 0x7fe35c0603f0 2021-04-03T03:10:32.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c063490 con 0x7fe35c14b700 2021-04-03T03:10:32.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0628d0 con 0x7fe35c0b12b0 2021-04-03T03:10:32.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51576/0 (socket says 172.21.2.111:51576) 2021-04-03T03:10:32.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 -- 172.21.2.111:0/3252267653 learned_addr learned my addr 172.21.2.111:0/3252267653 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361665700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe35c14b700 0x7fe35c064960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 -- 172.21.2.111:0/3252267653 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe35c0b12b0 msgr2=0x7fe35c062fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe35c0b12b0 0x7fe35c062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 -- 172.21.2.111:0/3252267653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe35c14b700 msgr2=0x7fe35c064960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe35c14b700 0x7fe35c064960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 -- 172.21.2.111:0/3252267653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0656f0 con 0x7fe35c0603f0 2021-04-03T03:10:32.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe360e64700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0ab7b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fe35c06a3b0 tx=0x7fe34c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe359ffb700 1 -- 172.21.2.111:0/3252267653 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c011d70 con 0x7fe35c0603f0 2021-04-03T03:10:32.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe359ffb700 1 -- 172.21.2.111:0/3252267653 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c01e460 con 0x7fe35c0603f0 2021-04-03T03:10:32.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c062b00 con 0x7fe35c0603f0 2021-04-03T03:10:32.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe359ffb700 1 -- 172.21.2.111:0/3252267653 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c01e7c0 con 0x7fe35c0603f0 2021-04-03T03:10:32.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0aaa70 con 0x7fe35c0603f0 2021-04-03T03:10:32.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c057660 con 0x7fe35c0603f0 2021-04-03T03:10:32.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe359ffb700 1 -- 172.21.2.111:0/3252267653 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe34c025030 con 0x7fe35c0603f0 2021-04-03T03:10:32.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe359ffb700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe33c056f10 0x7fe33c0593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe359ffb700 1 -- 172.21.2.111:0/3252267653 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe34c071770 con 0x7fe35c0603f0 2021-04-03T03:10:32.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe35bfff700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe33c056f10 0x7fe33c0593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.191+0000 7fe35bfff700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe33c056f10 0x7fe33c0593d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fe350002f30 tx=0x7fe3500069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.195+0000 7fe359ffb700 1 -- 172.21.2.111:0/3252267653 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe34c046a50 con 0x7fe35c0603f0 2021-04-03T03:10:32.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe35c062c90 con 0x7fe35c0603f0 2021-04-03T03:10:32.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe359ffb700 1 -- 172.21.2.111:0/3252267653 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fe34c023870 con 0x7fe35c0603f0 2021-04-03T03:10:32.334 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:32.334 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:10:32.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe33c056f10 msgr2=0x7fe33c0593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe33c056f10 0x7fe33c0593d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fe350002f30 tx=0x7fe3500069f0).stop 2021-04-03T03:10:32.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 msgr2=0x7fe35c0ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0ab7b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fe35c06a3b0 tx=0x7fe34c015000).stop 2021-04-03T03:10:32.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 shutdown_connections 2021-04-03T03:10:32.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe33c056f10 0x7fe33c0593d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe35c14b700 0x7fe35c064960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe35c0603f0 0x7fe35c0ab7b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 --2- 172.21.2.111:0/3252267653 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe35c0b12b0 0x7fe35c062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 >> 172.21.2.111:0/3252267653 conn(0x7fe35c0b61c0 msgr2=0x7fe35c0ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 shutdown_connections 2021-04-03T03:10:32.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.323+0000 7fe361e66700 1 -- 172.21.2.111:0/3252267653 wait complete. 2021-04-03T03:10:32.338 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:10:32.342 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:10:32.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 Processor -- start 2021-04-03T03:10:32.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- start start 2021-04-03T03:10:32.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc3900603f0 0x7fc3900b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc39014b700 0x7fc3900b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3900607d0 con 0x7fc3900b12b0 2021-04-03T03:10:32.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3900af400 con 0x7fc3900603f0 2021-04-03T03:10:32.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3900af6a0 con 0x7fc39014b700 2021-04-03T03:10:32.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc3900603f0 0x7fc3900b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc3900603f0 0x7fc3900b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51582/0 (socket says 172.21.2.111:51582) 2021-04-03T03:10:32.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 -- 172.21.2.111:0/1558375685 learned_addr learned my addr 172.21.2.111:0/1558375685 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 -- 172.21.2.111:0/1558375685 >> v2:172.21.2.115:3300/0 conn(0x7fc39014b700 msgr2=0x7fc3900b0580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:10:32.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 --2- 172.21.2.111:0/1558375685 >> v2:172.21.2.115:3300/0 conn(0x7fc39014b700 0x7fc3900b0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 -- 172.21.2.111:0/1558375685 >> v1:172.21.2.115:6789/0 conn(0x7fc3900b12b0 legacy=0x7fc3901465d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:10:32.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 -- 172.21.2.111:0/1558375685 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3900afb40 con 0x7fc3900603f0 2021-04-03T03:10:32.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc395ed4700 1 --2- 172.21.2.111:0/1558375685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc3900b0090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc39006a3b0 tx=0x7fc38c006e90).ready entity=mon.0 client_cookie=bfbdc20a87288a0d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc394ed2700 1 -- 172.21.2.111:0/1558375685 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38c009930 con 0x7fc3900603f0 2021-04-03T03:10:32.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc394ed2700 1 -- 172.21.2.111:0/1558375685 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c009a90 con 0x7fc3900603f0 2021-04-03T03:10:32.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc394ed2700 1 -- 172.21.2.111:0/1558375685 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38c009e30 con 0x7fc3900603f0 2021-04-03T03:10:32.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- 172.21.2.111:0/1558375685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 msgr2=0x7fc3900b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 --2- 172.21.2.111:0/1558375685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc3900b0090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc39006a3b0 tx=0x7fc38c006e90).stop 2021-04-03T03:10:32.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- 172.21.2.111:0/1558375685 shutdown_connections 2021-04-03T03:10:32.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 --2- 172.21.2.111:0/1558375685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc3900b0090 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 --2- 172.21.2.111:0/1558375685 >> v2:172.21.2.115:3300/0 conn(0x7fc39014b700 0x7fc3900b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- 172.21.2.111:0/1558375685 >> 172.21.2.111:0/1558375685 conn(0x7fc3900b61c0 msgr2=0x7fc39005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- 172.21.2.111:0/1558375685 shutdown_connections 2021-04-03T03:10:32.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- 172.21.2.111:0/1558375685 wait complete. 2021-04-03T03:10:32.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 Processor -- start 2021-04-03T03:10:32.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.387+0000 7fc396ed6700 1 -- start start 2021-04-03T03:10:32.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc390063c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc3900b12b0 0x7fc390064140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc39014b700 0x7fc3900ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900a8700 con 0x7fc3900603f0 2021-04-03T03:10:32.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900a98b0 con 0x7fc39014b700 2021-04-03T03:10:32.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc390157570 con 0x7fc3900b12b0 2021-04-03T03:10:32.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc390063c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc390063c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51588/0 (socket says 172.21.2.111:51588) 2021-04-03T03:10:32.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 -- 172.21.2.111:0/592905975 learned_addr learned my addr 172.21.2.111:0/592905975 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 -- 172.21.2.111:0/592905975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc3900b12b0 msgr2=0x7fc390064140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc3900b12b0 0x7fc390064140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 -- 172.21.2.111:0/592905975 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc39014b700 msgr2=0x7fc3900ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc39014b700 0x7fc3900ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 -- 172.21.2.111:0/592905975 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390157810 con 0x7fc3900603f0 2021-04-03T03:10:32.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc395ed4700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc390063c50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fc39006a3b0 tx=0x7fc38c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc382ffd700 1 -- 172.21.2.111:0/592905975 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38c011d50 con 0x7fc3900603f0 2021-04-03T03:10:32.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc382ffd700 1 -- 172.21.2.111:0/592905975 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c01e460 con 0x7fc3900603f0 2021-04-03T03:10:32.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc382ffd700 1 -- 172.21.2.111:0/592905975 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38c01e7c0 con 0x7fc3900603f0 2021-04-03T03:10:32.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3900a8e40 con 0x7fc3900603f0 2021-04-03T03:10:32.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3900a9fc0 con 0x7fc3900603f0 2021-04-03T03:10:32.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc390057660 con 0x7fc3900603f0 2021-04-03T03:10:32.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc382ffd700 1 -- 172.21.2.111:0/592905975 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc38c01a040 con 0x7fc3900603f0 2021-04-03T03:10:32.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc382ffd700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc370056ea0 0x7fc370059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc382ffd700 1 -- 172.21.2.111:0/592905975 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc38c0712c0 con 0x7fc3900603f0 2021-04-03T03:10:32.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc3956d3700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc370056ea0 0x7fc370059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.391+0000 7fc3956d3700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc370056ea0 0x7fc370059360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc384000e90 tx=0x7fc384006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.395+0000 7fc382ffd700 1 -- 172.21.2.111:0/592905975 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc38c0466b0 con 0x7fc3900603f0 2021-04-03T03:10:32.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc390147f70 con 0x7fc3900603f0 2021-04-03T03:10:32.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc382ffd700 1 -- 172.21.2.111:0/592905975 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fc38c023090 con 0x7fc3900603f0 2021-04-03T03:10:32.497 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:32.498 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:10:32.119441+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:10:29.772278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-04-03T03:10:30.083040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:10:32.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc370056ea0 msgr2=0x7fc370059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc370056ea0 0x7fc370059360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc384000e90 tx=0x7fc384006910).stop 2021-04-03T03:10:32.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 msgr2=0x7fc390063c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc390063c50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fc39006a3b0 tx=0x7fc38c015000).stop 2021-04-03T03:10:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 shutdown_connections 2021-04-03T03:10:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc370056ea0 0x7fc370059360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc39014b700 0x7fc3900ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc3900603f0 0x7fc390063c50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 --2- 172.21.2.111:0/592905975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc3900b12b0 0x7fc390064140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 >> 172.21.2.111:0/592905975 conn(0x7fc3900b61c0 msgr2=0x7fc3900b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 shutdown_connections 2021-04-03T03:10:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.487+0000 7fc396ed6700 1 -- 172.21.2.111:0/592905975 wait complete. 2021-04-03T03:10:32.506 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:10:32.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 Processor -- start 2021-04-03T03:10:32.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 -- start start 2021-04-03T03:10:32.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f2f88080c60 0x7f2f88025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f88081090 con 0x7f2f880b38c0 2021-04-03T03:10:32.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f880b3ca0 con 0x7f2f88153440 2021-04-03T03:10:32.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f88025970 con 0x7f2f88080c60 2021-04-03T03:10:32.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f86d9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f2f880b38c0 0x7f2f88080550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52388/0 (socket says 172.21.2.111:52388) 2021-04-03T03:10:32.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f86d9d700 1 -- 172.21.2.111:0/1191922517 learned_addr learned my addr 172.21.2.111:0/1191922517 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:32.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895710737 0 0) 0x7f2f780010d0 con 0x7f2f880b38c0 2021-04-03T03:10:32.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f74003240 con 0x7f2f880b38c0 2021-04-03T03:10:32.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1332542398 0 0) 0x7f2f74003240 con 0x7f2f880b38c0 2021-04-03T03:10:32.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 >> v2:172.21.2.115:3300/0 conn(0x7f2f88080c60 msgr2=0x7f2f88025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 --2- 172.21.2.111:0/1191922517 >> v2:172.21.2.115:3300/0 conn(0x7f2f88080c60 0x7f2f88025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 >> v1:172.21.2.115:6789/0 conn(0x7f2f88153440 legacy=0x7f2f880af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88025b90 con 0x7f2f880b38c0 2021-04-03T03:10:32.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f2f78001860 con 0x7f2f880b38c0 2021-04-03T03:10:32.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 >> v1:172.21.2.111:6789/0 conn(0x7f2f880b38c0 legacy=0x7f2f88080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74003730 0x7f2f74003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f740044e0 0x7f2f74004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f74004e50 0x7f2f7400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88025970 con 0x7f2f74004e50 2021-04-03T03:10:32.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74004240 con 0x7f2f74003730 2021-04-03T03:10:32.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7400bc20 con 0x7f2f740044e0 2021-04-03T03:10:32.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f74004e50 0x7f2f7400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8659c700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f740044e0 0x7f2f74004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f86d9d700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74003730 0x7f2f74003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 -- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f740044e0 msgr2=0x7f2f74004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f740044e0 0x7f2f74004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 -- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74003730 msgr2=0x7f2f74003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74003730 0x7f2f74003b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 -- 172.21.2.111:0/1191922517 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74003240 con 0x7f2f74004e50 2021-04-03T03:10:32.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8659c700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f740044e0 0x7f2f74004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:32.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 -- 172.21.2.111:0/1191922517 reap_dead start 2021-04-03T03:10:32.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8cf50700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f74004e50 0x7f2f7400b730 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f2f800032b0 tx=0x7f2f8004d220).ready entity=mon.0 client_cookie=a43773a5d837d3f9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f8004fb20 con 0x7f2f74004e50 2021-04-03T03:10:32.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8004fc80 con 0x7f2f74004e50 2021-04-03T03:10:32.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f85d9b700 1 -- 172.21.2.111:0/1191922517 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f80055440 con 0x7f2f74004e50 2021-04-03T03:10:32.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 -- 172.21.2.111:0/1191922517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f74004e50 msgr2=0x7f2f7400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.555+0000 7f2f8759e700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f74004e50 0x7f2f7400b730 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f2f800032b0 tx=0x7f2f8004d220).stop 2021-04-03T03:10:32.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- 172.21.2.111:0/1191922517 shutdown_connections 2021-04-03T03:10:32.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 --2- 172.21.2.111:0/1191922517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f74004e50 0x7f2f7400b730 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- 172.21.2.111:0/1191922517 >> 172.21.2.111:0/1191922517 conn(0x7f2f88061610 msgr2=0x7f2f88061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- 172.21.2.111:0/1191922517 shutdown_connections 2021-04-03T03:10:32.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- 172.21.2.111:0/1191922517 wait complete. 2021-04-03T03:10:32.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 Processor -- start 2021-04-03T03:10:32.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- start start 2021-04-03T03:10:32.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74004e50 0x7f2f8813f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f880ad230 0x7f2f88141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f88080550 0x7f2f880aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88068010 con 0x7f2f880ad230 2021-04-03T03:10:32.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881427a0 con 0x7f2f74004e50 2021-04-03T03:10:32.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88142a40 con 0x7f2f88080550 2021-04-03T03:10:32.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f880ad230 0x7f2f88141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f880ad230 0x7f2f88141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51608/0 (socket says 172.21.2.111:51608) 2021-04-03T03:10:32.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 -- 172.21.2.111:0/2625264414 learned_addr learned my addr 172.21.2.111:0/2625264414 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 -- 172.21.2.111:0/2625264414 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f88080550 msgr2=0x7f2f880aaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f88080550 0x7f2f880aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 -- 172.21.2.111:0/2625264414 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74004e50 msgr2=0x7f2f8813f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f86d9d700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74004e50 0x7f2f8813f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74004e50 0x7f2f8813f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 -- 172.21.2.111:0/2625264414 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8813f790 con 0x7f2f880ad230 2021-04-03T03:10:32.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8659c700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f880ad230 0x7f2f88141f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f2f70008000 tx=0x7f2f70009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f7f7fe700 1 -- 172.21.2.111:0/2625264414 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7000f510 con 0x7f2f880ad230 2021-04-03T03:10:32.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f7f7fe700 1 -- 172.21.2.111:0/2625264414 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7000f670 con 0x7f2f880ad230 2021-04-03T03:10:32.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f88142420 con 0x7f2f880ad230 2021-04-03T03:10:32.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f7f7fe700 1 -- 172.21.2.111:0/2625264414 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7000f9d0 con 0x7f2f880ad230 2021-04-03T03:10:32.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f880ab150 con 0x7f2f880ad230 2021-04-03T03:10:32.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f88057660 con 0x7f2f880ad230 2021-04-03T03:10:32.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.559+0000 7f2f7f7fe700 1 -- 172.21.2.111:0/2625264414 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2f7000fb30 con 0x7f2f880ad230 2021-04-03T03:10:32.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.563+0000 7f2f7f7fe700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2f6c057070 0x7f2f6c059530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.563+0000 7f2f7f7fe700 1 -- 172.21.2.111:0/2625264414 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f2f7006fcc0 con 0x7f2f880ad230 2021-04-03T03:10:32.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.563+0000 7f2f86d9d700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2f6c057070 0x7f2f6c059530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.563+0000 7f2f86d9d700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2f6c057070 0x7f2f6c059530 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f2f78009000 tx=0x7f2f78009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.563+0000 7f2f7f7fe700 1 -- 172.21.2.111:0/2625264414 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2f70042c50 con 0x7f2f880ad230 2021-04-03T03:10:32.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.691+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2f881425b0 con 0x7f2f880ad230 2021-04-03T03:10:32.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f7f7fe700 1 -- 172.21.2.111:0/2625264414 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2f70048b90 con 0x7f2f880ad230 2021-04-03T03:10:32.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2f6c057070 msgr2=0x7f2f6c059530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2f6c057070 0x7f2f6c059530 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f2f78009000 tx=0x7f2f78009040).stop 2021-04-03T03:10:32.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f880ad230 msgr2=0x7f2f88141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f880ad230 0x7f2f88141f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f2f70008000 tx=0x7f2f70009040).stop 2021-04-03T03:10:32.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 shutdown_connections 2021-04-03T03:10:32.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2f88080550 0x7f2f880aaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2f6c057070 0x7f2f6c059530 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2f74004e50 0x7f2f8813f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 --2- 172.21.2.111:0/2625264414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2f880ad230 0x7f2f88141f30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 >> 172.21.2.111:0/2625264414 conn(0x7f2f88061610 msgr2=0x7f2f880b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 shutdown_connections 2021-04-03T03:10:32.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.695+0000 7f2f8759e700 1 -- 172.21.2.111:0/2625264414 wait complete. 2021-04-03T03:10:32.708 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:10:32.712 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:10:32.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 Processor -- start 2021-04-03T03:10:32.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- start start 2021-04-03T03:10:32.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b40b38c0 0x7fa2b4025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa2b4153440 0x7fa2b4080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa2b4080a40 0x7fa2b414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b4025970 con 0x7fa2b4153440 2021-04-03T03:10:32.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b40b3ca0 con 0x7fa2b4080a40 2021-04-03T03:10:32.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4025b90 con 0x7fa2b40b38c0 2021-04-03T03:10:32.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa2b4153440 0x7fa2b4080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa2b4153440 0x7fa2b4080550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51614/0 (socket says 172.21.2.111:51614) 2021-04-03T03:10:32.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/97499609 learned_addr learned my addr 172.21.2.111:0/97499609 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b3fff700 1 --2- 172.21.2.111:0/97499609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b40b38c0 0x7fa2b4025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/97499609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b40b38c0 msgr2=0x7fa2b4025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- 172.21.2.111:0/97499609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b40b38c0 0x7fa2b4025480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/97499609 >> v2:172.21.2.115:3300/0 conn(0x7fa2b4080a40 msgr2=0x7fa2b414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- 172.21.2.111:0/97499609 >> v2:172.21.2.115:3300/0 conn(0x7fa2b4080a40 0x7fa2b414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/97499609 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4025e30 con 0x7fa2b4153440 2021-04-03T03:10:32.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- 172.21.2.111:0/97499609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b4153440 0x7fa2b4080550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fa2a4002f30 tx=0x7fa2a4006e90).ready entity=mon.0 client_cookie=3e9d6b3b2761353a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b2ffd700 1 -- 172.21.2.111:0/97499609 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a4009920 con 0x7fa2b4153440 2021-04-03T03:10:32.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b2ffd700 1 -- 172.21.2.111:0/97499609 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a4009a80 con 0x7fa2b4153440 2021-04-03T03:10:32.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b2ffd700 1 -- 172.21.2.111:0/97499609 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a4009de0 con 0x7fa2b4153440 2021-04-03T03:10:32.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/97499609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b4153440 msgr2=0x7fa2b4080550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/97499609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b4153440 0x7fa2b4080550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fa2a4002f30 tx=0x7fa2a4006e90).stop 2021-04-03T03:10:32.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/97499609 shutdown_connections 2021-04-03T03:10:32.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/97499609 >> v2:172.21.2.115:3300/0 conn(0x7fa2b4080a40 0x7fa2b414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/97499609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b40b38c0 0x7fa2b4025480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/97499609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b4153440 0x7fa2b4080550 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/97499609 >> 172.21.2.111:0/97499609 conn(0x7fa2b4061610 msgr2=0x7fa2b4061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/97499609 shutdown_connections 2021-04-03T03:10:32.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/97499609 wait complete. 2021-04-03T03:10:32.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 Processor -- start 2021-04-03T03:10:32.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- start start 2021-04-03T03:10:32.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b4080a40 0x7fa2b41519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b40b38c0 0x7fa2b4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2b4153440 0x7fa2b4150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4128040 con 0x7fa2b40b38c0 2021-04-03T03:10:32.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b412b100 con 0x7fa2b4153440 2021-04-03T03:10:32.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b412b3a0 con 0x7fa2b4080a40 2021-04-03T03:10:32.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b40b38c0 0x7fa2b4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b40b38c0 0x7fa2b4150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51618/0 (socket says 172.21.2.111:51618) 2021-04-03T03:10:32.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/1883247282 learned_addr learned my addr 172.21.2.111:0/1883247282 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/1883247282 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b4080a40 msgr2=0x7fa2b41519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b4080a40 0x7fa2b41519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/1883247282 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2b4153440 msgr2=0x7fa2b4150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2b4153440 0x7fa2b4150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 -- 172.21.2.111:0/1883247282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4128730 con 0x7fa2b40b38c0 2021-04-03T03:10:32.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b37fe700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b40b38c0 0x7fa2b4150340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa2a4002f30 tx=0x7fa2a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b0ff9700 1 -- 172.21.2.111:0/1883247282 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a4011d70 con 0x7fa2b40b38c0 2021-04-03T03:10:32.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b0ff9700 1 -- 172.21.2.111:0/1883247282 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a401e460 con 0x7fa2b40b38c0 2021-04-03T03:10:32.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b4152420 con 0x7fa2b40b38c0 2021-04-03T03:10:32.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.763+0000 7fa2b0ff9700 1 -- 172.21.2.111:0/1883247282 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a401e7c0 con 0x7fa2b40b38c0 2021-04-03T03:10:32.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b4150d20 con 0x7fa2b40b38c0 2021-04-03T03:10:32.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b412b560 con 0x7fa2b40b38c0 2021-04-03T03:10:32.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b0ff9700 1 -- 172.21.2.111:0/1883247282 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa2a4025030 con 0x7fa2b40b38c0 2021-04-03T03:10:32.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b0ff9700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa294056f20 0x7fa2940593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b0ff9700 1 -- 172.21.2.111:0/1883247282 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa2a4071810 con 0x7fa2b40b38c0 2021-04-03T03:10:32.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b3fff700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa294056f20 0x7fa2940593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b3fff700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa294056f20 0x7fa2940593e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa2a8007000 tx=0x7fa2a8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.767+0000 7fa2b0ff9700 1 -- 172.21.2.111:0/1883247282 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa2a4046af0 con 0x7fa2b40b38c0 2021-04-03T03:10:32.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.895+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa2b4057660 con 0x7fa2b40b38c0 2021-04-03T03:10:32.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.899+0000 7fa2b0ff9700 1 -- 172.21.2.111:0/1883247282 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa2a40230e0 con 0x7fa2b40b38c0 2021-04-03T03:10:32.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa294056f20 msgr2=0x7fa2940593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa294056f20 0x7fa2940593e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa2a8007000 tx=0x7fa2a8008040).stop 2021-04-03T03:10:32.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b40b38c0 msgr2=0x7fa2b4150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b40b38c0 0x7fa2b4150340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa2a4002f30 tx=0x7fa2a4015000).stop 2021-04-03T03:10:32.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 shutdown_connections 2021-04-03T03:10:32.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa294056f20 0x7fa2940593e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2b4153440 0x7fa2b4150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2b4080a40 0x7fa2b41519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:32.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 --2- 172.21.2.111:0/1883247282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2b40b38c0 0x7fa2b4150340 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 >> 172.21.2.111:0/1883247282 conn(0x7fa2b4061610 msgr2=0x7fa2b40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 shutdown_connections 2021-04-03T03:10:32.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.903+0000 7fa2b8a4b700 1 -- 172.21.2.111:0/1883247282 wait complete. 2021-04-03T03:10:32.916 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:10:32.920 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:10:32.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.967+0000 7fbdb5f35700 1 Processor -- start 2021-04-03T03:10:32.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.967+0000 7fbdb5f35700 1 -- start start 2021-04-03T03:10:32.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbda8046ce0 0x7fbda803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbda803d350 0x7fbda8041c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda8041240 con 0x7fbda8042160 2021-04-03T03:10:32.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80f1850 con 0x7fbda8046ce0 2021-04-03T03:10:32.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80470c0 con 0x7fbda803d350 2021-04-03T03:10:32.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb73af700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fbda8042160 0x7fbda80f51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52422/0 (socket says 172.21.2.111:52422) 2021-04-03T03:10:32.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5734700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbda8046ce0 0x7fbda803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb73af700 1 -- 172.21.2.111:0/670116637 learned_addr learned my addr 172.21.2.111:0/670116637 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:32.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 --2- 172.21.2.111:0/670116637 >> v2:172.21.2.115:3300/0 conn(0x7fbda803d350 0x7fbda8041c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5734700 1 -- 172.21.2.111:0/670116637 >> v2:172.21.2.115:3300/0 conn(0x7fbda803d350 msgr2=0x7fbda8041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:32.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5734700 1 --2- 172.21.2.111:0/670116637 >> v2:172.21.2.115:3300/0 conn(0x7fbda803d350 0x7fbda8041c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5734700 1 -- 172.21.2.111:0/670116637 >> v1:172.21.2.111:6789/0 conn(0x7fbda8042160 legacy=0x7fbda80f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5734700 1 -- 172.21.2.111:0/670116637 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80400d0 con 0x7fbda8046ce0 2021-04-03T03:10:32.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5734700 1 --2- 172.21.2.111:0/670116637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8046ce0 0x7fbda803f650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fbda0002f30 tx=0x7fbda0006dd0).ready entity=mon.0 client_cookie=d330943ee7ea8054 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbda7fff700 1 -- 172.21.2.111:0/670116637 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda00097b0 con 0x7fbda8046ce0 2021-04-03T03:10:32.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbda7fff700 1 -- 172.21.2.111:0/670116637 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0009910 con 0x7fbda8046ce0 2021-04-03T03:10:32.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbda7fff700 1 -- 172.21.2.111:0/670116637 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda0009c70 con 0x7fbda8046ce0 2021-04-03T03:10:32.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- 172.21.2.111:0/670116637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8046ce0 msgr2=0x7fbda803f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/670116637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8046ce0 0x7fbda803f650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fbda0002f30 tx=0x7fbda0006dd0).stop 2021-04-03T03:10:32.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- 172.21.2.111:0/670116637 shutdown_connections 2021-04-03T03:10:32.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/670116637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8046ce0 0x7fbda803f650 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/670116637 >> v2:172.21.2.115:3300/0 conn(0x7fbda803d350 0x7fbda8041c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- 172.21.2.111:0/670116637 >> 172.21.2.111:0/670116637 conn(0x7fbda800d0c0 msgr2=0x7fbda80462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:32.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- 172.21.2.111:0/670116637 shutdown_connections 2021-04-03T03:10:32.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- 172.21.2.111:0/670116637 wait complete. 2021-04-03T03:10:32.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 Processor -- start 2021-04-03T03:10:32.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- start start 2021-04-03T03:10:32.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbda803d350 0x7fbda80398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8042160 0x7fbda80be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbda8046ce0 0x7fbda80be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80bd990 con 0x7fbda8042160 2021-04-03T03:10:32.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80bdc30 con 0x7fbda803d350 2021-04-03T03:10:32.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5f35700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80f4880 con 0x7fbda8046ce0 2021-04-03T03:10:32.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8042160 0x7fbda80be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8042160 0x7fbda80be0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51636/0 (socket says 172.21.2.111:51636) 2021-04-03T03:10:32.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 -- 172.21.2.111:0/3405410721 learned_addr learned my addr 172.21.2.111:0/3405410721 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:32.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb5734700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbda803d350 0x7fbda80398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb73af700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbda8046ce0 0x7fbda80be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 -- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbda8046ce0 msgr2=0x7fbda80be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbda8046ce0 0x7fbda80be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 -- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbda803d350 msgr2=0x7fbda80398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:32.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbda803d350 0x7fbda80398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:32.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 -- 172.21.2.111:0/3405410721 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda803ba20 con 0x7fbda8042160 2021-04-03T03:10:32.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.971+0000 7fbdb4f33700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8042160 0x7fbda80be0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fbdac00c000 tx=0x7fbdac006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbda5ffb700 1 -- 172.21.2.111:0/3405410721 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdac00cae0 con 0x7fbda8042160 2021-04-03T03:10:32.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbda5ffb700 1 -- 172.21.2.111:0/3405410721 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00cc40 con 0x7fbda8042160 2021-04-03T03:10:32.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda803bcd0 con 0x7fbda8042160 2021-04-03T03:10:32.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbda5ffb700 1 -- 172.21.2.111:0/3405410721 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdac0123f0 con 0x7fbda8042160 2021-04-03T03:10:32.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80d02f0 con 0x7fbda8042160 2021-04-03T03:10:32.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbda5ffb700 1 -- 172.21.2.111:0/3405410721 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbdac012550 con 0x7fbda8042160 2021-04-03T03:10:32.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda8004d90 con 0x7fbda8042160 2021-04-03T03:10:32.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbda5ffb700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056fb0 0x7fbd90059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:32.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbda5ffb700 1 -- 172.21.2.111:0/3405410721 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fbdac0706e0 con 0x7fbda8042160 2021-04-03T03:10:32.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbdb5734700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056fb0 0x7fbd90059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:32.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbdb5734700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056fb0 0x7fbd90059470 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fbda0019000 tx=0x7fbda0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:32.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:32.975+0000 7fbda5ffb700 1 -- 172.21.2.111:0/3405410721 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbdac045a70 con 0x7fbda8042160 2021-04-03T03:10:33.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.107+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbda80083c0 con 0x7fbda8042160 2021-04-03T03:10:33.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbda5ffb700 1 -- 172.21.2.111:0/3405410721 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbdac049090 con 0x7fbda8042160 2021-04-03T03:10:33.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056fb0 msgr2=0x7fbd90059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056fb0 0x7fbd90059470 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fbda0019000 tx=0x7fbda0015000).stop 2021-04-03T03:10:33.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8042160 msgr2=0x7fbda80be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8042160 0x7fbda80be0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fbdac00c000 tx=0x7fbdac006dd0).stop 2021-04-03T03:10:33.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 shutdown_connections 2021-04-03T03:10:33.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbda8046ce0 0x7fbda80be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbd90056fb0 0x7fbd90059470 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbda803d350 0x7fbda80398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 --2- 172.21.2.111:0/3405410721 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbda8042160 0x7fbda80be0d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 >> 172.21.2.111:0/3405410721 conn(0x7fbda800d0c0 msgr2=0x7fbda80f25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:33.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 shutdown_connections 2021-04-03T03:10:33.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.111+0000 7fbdb5f35700 1 -- 172.21.2.111:0/3405410721 wait complete. 2021-04-03T03:10:33.125 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:10:33.129 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:33.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 Processor -- start 2021-04-03T03:10:33.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- start start 2021-04-03T03:10:33.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c0d4b70 0x7f753c03ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c0d0b90 con 0x7f753c047e00 2021-04-03T03:10:33.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c0481e0 con 0x7f753c03e680 2021-04-03T03:10:33.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c042420 con 0x7f753c0d4b70 2021-04-03T03:10:33.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754989e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f753c047e00 0x7f753c0d1540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52432/0 (socket says 172.21.2.111:52432) 2021-04-03T03:10:33.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754989e700 1 -- 172.21.2.111:0/2290281396 learned_addr learned my addr 172.21.2.111:0/2290281396 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:33.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224101878 0 0) 0x7f753c0d0b90 con 0x7f753c047e00 2021-04-03T03:10:33.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c003240 con 0x7f753c047e00 2021-04-03T03:10:33.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c0d4b70 0x7f753c03ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2421626967 0 0) 0x7f752c003240 con 0x7f753c047e00 2021-04-03T03:10:33.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c0d4b70 msgr2=0x7f753c03ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c0d4b70 0x7f753c03ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 >> v1:172.21.2.115:6789/0 conn(0x7f753c03e680 legacy=0x7f753c040410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0d0e00 con 0x7f753c047e00 2021-04-03T03:10:33.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f75440450f0 con 0x7f753c047e00 2021-04-03T03:10:33.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 >> v1:172.21.2.111:6789/0 conn(0x7f753c047e00 legacy=0x7f753c0d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f752c0037e0 0x7f752c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f752c0044e0 0x7f752c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c0d4b70 0x7f752c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c005360 con 0x7f752c0044e0 2021-04-03T03:10:33.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c004260 con 0x7f752c0037e0 2021-04-03T03:10:33.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c00b730 con 0x7f753c0d4b70 2021-04-03T03:10:33.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f752c0044e0 0x7f752c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 -- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c0d4b70 msgr2=0x7f752c004e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:10:33.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c0d4b70 0x7f752c004e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 -- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f752c0037e0 msgr2=0x7f752c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f752c0037e0 0x7f752c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 -- 172.21.2.111:0/2290281396 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c003240 con 0x7f752c0044e0 2021-04-03T03:10:33.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f752c0044e0 0x7f752c004960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f753400a000 tx=0x7f7534008040).ready entity=mon.0 client_cookie=7663cc5d4a6ead36 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f753400da60 con 0x7f752c0044e0 2021-04-03T03:10:33.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753400dbc0 con 0x7f752c0044e0 2021-04-03T03:10:33.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754889c700 1 -- 172.21.2.111:0/2290281396 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7534013480 con 0x7f752c0044e0 2021-04-03T03:10:33.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- 172.21.2.111:0/2290281396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f752c0044e0 msgr2=0x7f752c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 --2- 172.21.2.111:0/2290281396 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f752c0044e0 0x7f752c004960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f753400a000 tx=0x7f7534008040).stop 2021-04-03T03:10:33.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a09f700 1 -- 172.21.2.111:0/2290281396 reap_dead start 2021-04-03T03:10:33.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- 172.21.2.111:0/2290281396 shutdown_connections 2021-04-03T03:10:33.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- 172.21.2.111:0/2290281396 >> 172.21.2.111:0/2290281396 conn(0x7f753c00e220 msgr2=0x7f753c0473d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:33.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- 172.21.2.111:0/2290281396 shutdown_connections 2021-04-03T03:10:33.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- 172.21.2.111:0/2290281396 wait complete. 2021-04-03T03:10:33.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 Processor -- start 2021-04-03T03:10:33.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- start start 2021-04-03T03:10:33.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c03b910 0x7f753c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f753c03c210 0x7f753c0cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f753c0cc440 0x7f753c0df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0a13f0 con 0x7f753c03c210 2021-04-03T03:10:33.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c03c030 con 0x7f753c0cc440 2021-04-03T03:10:33.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754a8a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0cc8f0 con 0x7f753c03b910 2021-04-03T03:10:33.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f753c03c210 0x7f753c0cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.175+0000 7f754909d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f753c03c210 0x7f753c0cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51652/0 (socket says 172.21.2.111:51652) 2021-04-03T03:10:33.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754909d700 1 -- 172.21.2.111:0/1209883649 learned_addr learned my addr 172.21.2.111:0/1209883649 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:33.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754909d700 1 -- 172.21.2.111:0/1209883649 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c03b910 msgr2=0x7f753c0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754909d700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c03b910 0x7f753c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754909d700 1 -- 172.21.2.111:0/1209883649 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f753c0cc440 msgr2=0x7f753c0df8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754909d700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f753c0cc440 0x7f753c0df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754909d700 1 -- 172.21.2.111:0/1209883649 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c03bcf0 con 0x7f753c03c210 2021-04-03T03:10:33.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754909d700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f753c03c210 0x7f753c0cbf50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f753400a000 tx=0x7f7534008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f7532ffd700 1 -- 172.21.2.111:0/1209883649 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7534021070 con 0x7f753c03c210 2021-04-03T03:10:33.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f7532ffd700 1 -- 172.21.2.111:0/1209883649 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753401d630 con 0x7f753c03c210 2021-04-03T03:10:33.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c0a1150 con 0x7f753c03c210 2021-04-03T03:10:33.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f7532ffd700 1 -- 172.21.2.111:0/1209883649 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f753401d990 con 0x7f753c03c210 2021-04-03T03:10:33.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c03ac50 con 0x7f753c03c210 2021-04-03T03:10:33.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c0a0f90 con 0x7f753c03c210 2021-04-03T03:10:33.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f7532ffd700 1 -- 172.21.2.111:0/1209883649 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f753401daf0 con 0x7f753c03c210 2021-04-03T03:10:33.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f7532ffd700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7524056ea0 0x7f7524059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f7532ffd700 1 -- 172.21.2.111:0/1209883649 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f75340714c0 con 0x7f753c03c210 2021-04-03T03:10:33.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754989e700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7524056ea0 0x7f7524059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.179+0000 7f754989e700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7524056ea0 0x7f7524059360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f753c03fdc0 tx=0x7f753c0d50f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.183+0000 7f7532ffd700 1 -- 172.21.2.111:0/1209883649 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f75340468b0 con 0x7f753c03c210 2021-04-03T03:10:33.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f753c03a940 con 0x7f753c03c210 2021-04-03T03:10:33.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f7532ffd700 1 -- 172.21.2.111:0/1209883649 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f7534049ed0 con 0x7f753c03c210 2021-04-03T03:10:33.318 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:33.318 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:10:33.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7524056ea0 msgr2=0x7f7524059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7524056ea0 0x7f7524059360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f753c03fdc0 tx=0x7f753c0d50f0).stop 2021-04-03T03:10:33.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f753c03c210 msgr2=0x7f753c0cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f753c03c210 0x7f753c0cbf50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f753400a000 tx=0x7f7534008040).stop 2021-04-03T03:10:33.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 shutdown_connections 2021-04-03T03:10:33.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7524056ea0 0x7f7524059360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f753c0cc440 0x7f753c0df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f753c03b910 0x7f753c0cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 --2- 172.21.2.111:0/1209883649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f753c03c210 0x7f753c0cbf50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 >> 172.21.2.111:0/1209883649 conn(0x7f753c00e220 msgr2=0x7f753c03fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:33.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 shutdown_connections 2021-04-03T03:10:33.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.307+0000 7f754a8a0700 1 -- 172.21.2.111:0/1209883649 wait complete. 2021-04-03T03:10:33.322 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:10:33.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:10:31.116099+0000', 'modified': '2021-04-03T03:10:32.118491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5241': {'gid': 5241, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/798702406', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 798702406}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 798702406}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:10:33.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:10:33.530 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:10:33.601 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5241,"want_state":"up:active","state":"up:active","rank_uptime":2.4721264710000002,"mdsmap_epoch":16,"osdmap_epoch":43,"osdmap_epoch_barrier":42,"uptime":6.6159121089999999} 2021-04-03T03:10:33.602 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5241, "mdsmap_epoch": 16, "osdmap_epoch": 43, "osdmap_epoch_barrier": 42, "rank_uptime": 2.472126471, "state": "up:active", "uptime": 6.615912109, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:10:33.602 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:10:33.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 Processor -- start 2021-04-03T03:10:33.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 -- start start 2021-04-03T03:10:33.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd408005dc0 0x7fd408042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4080dc6e0 con 0x7fd4080dbb90 2021-04-03T03:10:33.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd408042010 con 0x7fd408006bc0 2021-04-03T03:10:33.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4080d9560 con 0x7fd408005dc0 2021-04-03T03:10:33.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd4080dbb90 0x7fd4080064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52452/0 (socket says 172.21.2.111:52452) 2021-04-03T03:10:33.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 -- 172.21.2.111:0/2020753671 learned_addr learned my addr 172.21.2.111:0/2020753671 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:33.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680448317 0 0) 0x7fd4080dc6e0 con 0x7fd4080dbb90 2021-04-03T03:10:33.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f8003240 con 0x7fd4080dbb90 2021-04-03T03:10:33.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415961700 1 --2- 172.21.2.111:0/2020753671 >> v2:172.21.2.115:3300/0 conn(0x7fd408005dc0 0x7fd408042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2970543767 0 0) 0x7fd3f8003240 con 0x7fd4080dbb90 2021-04-03T03:10:33.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 >> v2:172.21.2.115:3300/0 conn(0x7fd408005dc0 msgr2=0x7fd408042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 --2- 172.21.2.111:0/2020753671 >> v2:172.21.2.115:3300/0 conn(0x7fd408005dc0 0x7fd408042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 >> v1:172.21.2.115:6789/0 conn(0x7fd408006bc0 legacy=0x7fd4080056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080422b0 con 0x7fd4080dbb90 2021-04-03T03:10:33.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fd4040016d0 con 0x7fd4080dbb90 2021-04-03T03:10:33.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 >> v1:172.21.2.111:6789/0 conn(0x7fd4080dbb90 legacy=0x7fd4080064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3f8003730 0x7fd3f8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3f80044e0 0x7fd3f8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3f8004e50 0x7fd3f800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4080d9560 con 0x7fd3f8003730 2021-04-03T03:10:33.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8004260 con 0x7fd3f80044e0 2021-04-03T03:10:33.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f800bc20 con 0x7fd3f8004e50 2021-04-03T03:10:33.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3f8003730 0x7fd3f8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 -- 172.21.2.111:0/2020753671 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3f8004e50 msgr2=0x7fd3f800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3f8004e50 0x7fd3f800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 -- 172.21.2.111:0/2020753671 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3f80044e0 msgr2=0x7fd3f8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3f80044e0 0x7fd3f8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 -- 172.21.2.111:0/2020753671 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080422b0 con 0x7fd3f8003730 2021-04-03T03:10:33.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415961700 1 -- 172.21.2.111:0/2020753671 reap_dead start 2021-04-03T03:10:33.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd415160700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3f8003730 0x7fd3f8003b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fd4040013e0 tx=0x7fd404007750).ready entity=mon.0 client_cookie=65ff84863cae919a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4040070d0 con 0x7fd3f8003730 2021-04-03T03:10:33.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404007230 con 0x7fd3f8003730 2021-04-03T03:10:33.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd403fff700 1 -- 172.21.2.111:0/2020753671 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4040055f0 con 0x7fd3f8003730 2021-04-03T03:10:33.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 -- 172.21.2.111:0/2020753671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3f8003730 msgr2=0x7fd3f8003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.647+0000 7fd416162700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3f8003730 0x7fd3f8003b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fd4040013e0 tx=0x7fd404007750).stop 2021-04-03T03:10:33.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- 172.21.2.111:0/2020753671 shutdown_connections 2021-04-03T03:10:33.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 --2- 172.21.2.111:0/2020753671 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3f8003730 0x7fd3f8003b50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- 172.21.2.111:0/2020753671 >> 172.21.2.111:0/2020753671 conn(0x7fd4080462c0 msgr2=0x7fd4080466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:33.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- 172.21.2.111:0/2020753671 shutdown_connections 2021-04-03T03:10:33.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- 172.21.2.111:0/2020753671 wait complete. 2021-04-03T03:10:33.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 Processor -- start 2021-04-03T03:10:33.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- start start 2021-04-03T03:10:33.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3f8003730 0x7fd40803acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4080df0d0 0x7fd40809f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4080dc8f0 0x7fd4080dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40803a8a0 con 0x7fd4080df0d0 2021-04-03T03:10:33.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4080a0040 con 0x7fd4080dc8f0 2021-04-03T03:10:33.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4080dd290 con 0x7fd3f8003730 2021-04-03T03:10:33.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4080df0d0 0x7fd40809f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4080df0d0 0x7fd40809f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51674/0 (socket says 172.21.2.111:51674) 2021-04-03T03:10:33.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 -- 172.21.2.111:0/1994432246 learned_addr learned my addr 172.21.2.111:0/1994432246 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:33.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 -- 172.21.2.111:0/1994432246 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3f8003730 msgr2=0x7fd40803acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3f8003730 0x7fd40803acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 -- 172.21.2.111:0/1994432246 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4080dc8f0 msgr2=0x7fd4080dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4080dc8f0 0x7fd4080dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 -- 172.21.2.111:0/1994432246 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080de280 con 0x7fd4080df0d0 2021-04-03T03:10:33.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd41495f700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4080df0d0 0x7fd40809f7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fd4080087e0 tx=0x7fd40c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd4027fc700 1 -- 172.21.2.111:0/1994432246 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd40c00e4a0 con 0x7fd4080df0d0 2021-04-03T03:10:33.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd4027fc700 1 -- 172.21.2.111:0/1994432246 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c00e600 con 0x7fd4080df0d0 2021-04-03T03:10:33.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd4027fc700 1 -- 172.21.2.111:0/1994432246 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd40c00e960 con 0x7fd4080df0d0 2021-04-03T03:10:33.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40809fcc0 con 0x7fd4080df0d0 2021-04-03T03:10:33.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd408039720 con 0x7fd4080df0d0 2021-04-03T03:10:33.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd408004da0 con 0x7fd4080df0d0 2021-04-03T03:10:33.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd4027fc700 1 -- 172.21.2.111:0/1994432246 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd40c019030 con 0x7fd4080df0d0 2021-04-03T03:10:33.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd4027fc700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3f0056ea0 0x7fd3f0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd4027fc700 1 -- 172.21.2.111:0/1994432246 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd40c06f7e0 con 0x7fd4080df0d0 2021-04-03T03:10:33.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd415160700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3f0056ea0 0x7fd3f0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.651+0000 7fd415160700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3f0056ea0 0x7fd3f0059360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fd4040013e0 tx=0x7fd404015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.655+0000 7fd4027fc700 1 -- 172.21.2.111:0/1994432246 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd40c044bd0 con 0x7fd4080df0d0 2021-04-03T03:10:33.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd408046ff0 con 0x7fd4080df0d0 2021-04-03T03:10:33.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd4027fc700 1 -- 172.21.2.111:0/1994432246 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fd40c0481f0 con 0x7fd4080df0d0 2021-04-03T03:10:33.763 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:33.763 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:10:32.119441+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:10:29.772278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-04-03T03:10:30.083040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:10:33.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3f0056ea0 msgr2=0x7fd3f0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3f0056ea0 0x7fd3f0059360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fd4040013e0 tx=0x7fd404015000).stop 2021-04-03T03:10:33.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4080df0d0 msgr2=0x7fd40809f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4080df0d0 0x7fd40809f7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fd4080087e0 tx=0x7fd40c007c00).stop 2021-04-03T03:10:33.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 shutdown_connections 2021-04-03T03:10:33.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3f0056ea0 0x7fd3f0059360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4080dc8f0 0x7fd4080dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3f8003730 0x7fd40803acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 --2- 172.21.2.111:0/1994432246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4080df0d0 0x7fd40809f7d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 >> 172.21.2.111:0/1994432246 conn(0x7fd4080462c0 msgr2=0x7fd408041880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:33.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 shutdown_connections 2021-04-03T03:10:33.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.751+0000 7fd416162700 1 -- 172.21.2.111:0/1994432246 wait complete. 2021-04-03T03:10:33.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:10:29.772278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:10:30.083040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:10:33.773 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:10:33.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6b59700 1 Processor -- start 2021-04-03T03:10:33.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6b59700 1 -- start start 2021-04-03T03:10:33.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6b59700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb5f00add50 0x7fb5f0147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6b59700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5f0144ef0 0x7fb5f0147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6b59700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f00607d0 con 0x7fb5f00603f0 2021-04-03T03:10:33.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6b59700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f014b090 con 0x7fb5f0144ef0 2021-04-03T03:10:33.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6b59700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f014b330 con 0x7fb5f00add50 2021-04-03T03:10:33.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5f0144ef0 0x7fb5f0147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5f0144ef0 0x7fb5f0147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51682/0 (socket says 172.21.2.111:51682) 2021-04-03T03:10:33.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 -- 172.21.2.111:0/4125845901 learned_addr learned my addr 172.21.2.111:0/4125845901 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:33.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f5356700 1 --2- 172.21.2.111:0/4125845901 >> v2:172.21.2.115:3300/0 conn(0x7fb5f00add50 0x7fb5f0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 -- 172.21.2.111:0/4125845901 >> v2:172.21.2.115:3300/0 conn(0x7fb5f00add50 msgr2=0x7fb5f0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 --2- 172.21.2.111:0/4125845901 >> v2:172.21.2.115:3300/0 conn(0x7fb5f00add50 0x7fb5f0147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 -- 172.21.2.111:0/4125845901 >> v1:172.21.2.115:6789/0 conn(0x7fb5f00603f0 legacy=0x7fb5f01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:33.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 -- 172.21.2.111:0/4125845901 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f014b5d0 con 0x7fb5f0144ef0 2021-04-03T03:10:33.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f6358700 1 --2- 172.21.2.111:0/4125845901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0147c30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fb5f00ae520 tx=0x7fb5e8009270).ready entity=mon.0 client_cookie=510a855fcf4c077a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.819+0000 7fb5f4b55700 1 -- 172.21.2.111:0/4125845901 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e800bbc0 con 0x7fb5f0144ef0 2021-04-03T03:10:33.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f4b55700 1 -- 172.21.2.111:0/4125845901 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e800bd20 con 0x7fb5f0144ef0 2021-04-03T03:10:33.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f4b55700 1 -- 172.21.2.111:0/4125845901 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e800be90 con 0x7fb5f0144ef0 2021-04-03T03:10:33.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/4125845901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 msgr2=0x7fb5f0147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/4125845901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0147c30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fb5f00ae520 tx=0x7fb5e8009270).stop 2021-04-03T03:10:33.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/4125845901 shutdown_connections 2021-04-03T03:10:33.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/4125845901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0147c30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/4125845901 >> v2:172.21.2.115:3300/0 conn(0x7fb5f00add50 0x7fb5f0147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/4125845901 >> 172.21.2.111:0/4125845901 conn(0x7fb5f00b61c0 msgr2=0x7fb5f005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:33.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/4125845901 shutdown_connections 2021-04-03T03:10:33.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/4125845901 wait complete. 2021-04-03T03:10:33.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 Processor -- start 2021-04-03T03:10:33.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- start start 2021-04-03T03:10:33.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5f00603f0 0x7fb5f00aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5f00add50 0x7fb5f01568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0063e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00a8a80 con 0x7fb5f0144ef0 2021-04-03T03:10:33.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00a9a40 con 0x7fb5f00603f0 2021-04-03T03:10:33.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0156db0 con 0x7fb5f00add50 2021-04-03T03:10:33.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0063e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0063e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51688/0 (socket says 172.21.2.111:51688) 2021-04-03T03:10:33.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 -- 172.21.2.111:0/659963797 learned_addr learned my addr 172.21.2.111:0/659963797 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:33.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 -- 172.21.2.111:0/659963797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5f00add50 msgr2=0x7fb5f01568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5f00add50 0x7fb5f01568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 -- 172.21.2.111:0/659963797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5f00603f0 msgr2=0x7fb5f00aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5f00603f0 0x7fb5f00aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 -- 172.21.2.111:0/659963797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0063990 con 0x7fb5f0144ef0 2021-04-03T03:10:33.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6358700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0063e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fb5f00a9b80 tx=0x7fb5e8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5e2ffd700 1 -- 172.21.2.111:0/659963797 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e8020620 con 0x7fb5f0144ef0 2021-04-03T03:10:33.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5e2ffd700 1 -- 172.21.2.111:0/659963797 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8020780 con 0x7fb5f0144ef0 2021-04-03T03:10:33.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5e2ffd700 1 -- 172.21.2.111:0/659963797 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e8020ae0 con 0x7fb5f0144ef0 2021-04-03T03:10:33.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f00a9c60 con 0x7fb5f0144ef0 2021-04-03T03:10:33.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f00aba70 con 0x7fb5f0144ef0 2021-04-03T03:10:33.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5e2ffd700 1 -- 172.21.2.111:0/659963797 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb5e8026020 con 0x7fb5f0144ef0 2021-04-03T03:10:33.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f0057660 con 0x7fb5f0144ef0 2021-04-03T03:10:33.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5e2ffd700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5d0057000 0x7fb5d00594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:33.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5e2ffd700 1 -- 172.21.2.111:0/659963797 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb5e80715e0 con 0x7fb5f0144ef0 2021-04-03T03:10:33.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f5b57700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5d0057000 0x7fb5d00594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:33.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.823+0000 7fb5f5b57700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5d0057000 0x7fb5d00594c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fb5e4001f60 tx=0x7fb5e40077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:33.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.827+0000 7fb5e2ffd700 1 -- 172.21.2.111:0/659963797 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb5e8046a00 con 0x7fb5f0144ef0 2021-04-03T03:10:33.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.919+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb5f0148850 con 0x7fb5f0144ef0 2021-04-03T03:10:33.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.919+0000 7fb5e2ffd700 1 -- 172.21.2.111:0/659963797 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v43) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb5e80150b0 con 0x7fb5f0144ef0 2021-04-03T03:10:33.932 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:10:33.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5d0057000 msgr2=0x7fb5d00594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5d0057000 0x7fb5d00594c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fb5e4001f60 tx=0x7fb5e40077d0).stop 2021-04-03T03:10:33.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 msgr2=0x7fb5f0063e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:33.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0063e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fb5f00a9b80 tx=0x7fb5e8016000).stop 2021-04-03T03:10:33.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 shutdown_connections 2021-04-03T03:10:33.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5f0144ef0 0x7fb5f0063e20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb5d0057000 0x7fb5d00594c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:33.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5f00603f0 0x7fb5f00aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 --2- 172.21.2.111:0/659963797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5f00add50 0x7fb5f01568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:33.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 >> 172.21.2.111:0/659963797 conn(0x7fb5f00b61c0 msgr2=0x7fb5f005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:33.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 shutdown_connections 2021-04-03T03:10:33.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.923+0000 7fb5f6b59700 1 -- 172.21.2.111:0/659963797 wait complete. 2021-04-03T03:10:33.940 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:10:33.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 Processor -- start 2021-04-03T03:10:33.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 -- start start 2021-04-03T03:10:34.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 0x7fe6f8147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f80607d0 con 0x7fe6f80603f0 2021-04-03T03:10:34.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f814b090 con 0x7fe6f80add50 2021-04-03T03:10:34.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f814b330 con 0x7fe6f8144ef0 2021-04-03T03:10:34.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe6f80603f0 0x7fe6f81447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52486/0 (socket says 172.21.2.111:52486) 2021-04-03T03:10:34.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1407196404 learned_addr learned my addr 172.21.2.111:0/1407196404 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:34.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213193753 0 0) 0x7fe6f80607d0 con 0x7fe6f80603f0 2021-04-03T03:10:34.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e4003240 con 0x7fe6f80603f0 2021-04-03T03:10:34.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fcbac700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 0x7fe6f8147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2757730288 0 0) 0x7fe6e4003240 con 0x7fe6f80603f0 2021-04-03T03:10:34.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 msgr2=0x7fe6f8147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:34.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 0x7fe6f8147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 >> v1:172.21.2.115:6789/0 conn(0x7fe6f80add50 legacy=0x7fe6f8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f814b5d0 con 0x7fe6f80603f0 2021-04-03T03:10:34.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe6ec0016d0 con 0x7fe6f80603f0 2021-04-03T03:10:34.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 >> v1:172.21.2.111:6789/0 conn(0x7fe6f80603f0 legacy=0x7fe6f81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e4003760 0x7fe6e4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 0x7fe6e40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6e40049d0 0x7fe6e4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f814b330 con 0x7fe6e4003760 2021-04-03T03:10:34.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4004250 con 0x7fe6e40049d0 2021-04-03T03:10:34.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e400b730 con 0x7fe6f8144ef0 2021-04-03T03:10:34.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e4003760 0x7fe6e4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f77fe700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 0x7fe6e40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fcbac700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6e40049d0 0x7fe6e4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 msgr2=0x7fe6e40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:34.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8144ef0 0x7fe6e40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6e40049d0 msgr2=0x7fe6e4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:34.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6e40049d0 0x7fe6e4004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1407196404 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e4003240 con 0x7fe6e4003760 2021-04-03T03:10:34.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e4003760 0x7fe6e4003b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe6ec0013e0 tx=0x7fe6f814b220).ready entity=mon.0 client_cookie=b841463f29d3970c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6ec0070d0 con 0x7fe6e4003760 2021-04-03T03:10:34.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec007230 con 0x7fe6e4003760 2021-04-03T03:10:34.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/1407196404 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6ec0055f0 con 0x7fe6e4003760 2021-04-03T03:10:34.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1407196404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e4003760 msgr2=0x7fe6e4003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fd3ad700 1 --2- 172.21.2.111:0/1407196404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6e4003760 0x7fe6e4003b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe6ec0013e0 tx=0x7fe6f814b220).stop 2021-04-03T03:10:34.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.987+0000 7fe6fcbac700 1 -- 172.21.2.111:0/1407196404 reap_dead start 2021-04-03T03:10:34.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1407196404 shutdown_connections 2021-04-03T03:10:34.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1407196404 >> 172.21.2.111:0/1407196404 conn(0x7fe6f80b61c0 msgr2=0x7fe6f805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1407196404 shutdown_connections 2021-04-03T03:10:34.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1407196404 wait complete. 2021-04-03T03:10:34.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 Processor -- start 2021-04-03T03:10:34.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- start start 2021-04-03T03:10:34.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f80ab420 0x7fe6f815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f80abdc0 0x7fe6f815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6f80a6f20 0x7fe6f815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f815b0d0 con 0x7fe6f80abdc0 2021-04-03T03:10:34.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80abbf0 con 0x7fe6f80a6f20 2021-04-03T03:10:34.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80ab800 con 0x7fe6f80ab420 2021-04-03T03:10:34.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f80abdc0 0x7fe6f815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f80abdc0 0x7fe6f815a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51708/0 (socket says 172.21.2.111:51708) 2021-04-03T03:10:34.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 -- 172.21.2.111:0/1508564778 learned_addr learned my addr 172.21.2.111:0/1508564778 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:34.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f80ab420 0x7fe6f815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fcbac700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6f80a6f20 0x7fe6f815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 -- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f80ab420 msgr2=0x7fe6f815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f80ab420 0x7fe6f815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 -- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6f80a6f20 msgr2=0x7fe6f815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6f80a6f20 0x7fe6f815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 -- 172.21.2.111:0/1508564778 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80aa270 con 0x7fe6f80abdc0 2021-04-03T03:10:34.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f77fe700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f80abdc0 0x7fe6f815a6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fe6e800b000 tx=0x7fe6e8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f57fa700 1 -- 172.21.2.111:0/1508564778 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6e8010450 con 0x7fe6f80abdc0 2021-04-03T03:10:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f57fa700 1 -- 172.21.2.111:0/1508564778 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e80105b0 con 0x7fe6f80abdc0 2021-04-03T03:10:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f57fa700 1 -- 172.21.2.111:0/1508564778 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6e8010910 con 0x7fe6f80abdc0 2021-04-03T03:10:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f81456e0 con 0x7fe6f80abdc0 2021-04-03T03:10:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f8141710 con 0x7fe6f80abdc0 2021-04-03T03:10:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f8057660 con 0x7fe6f80abdc0 2021-04-03T03:10:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f57fa700 1 -- 172.21.2.111:0/1508564778 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe6e8010a70 con 0x7fe6f80abdc0 2021-04-03T03:10:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f57fa700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6d8056ea0 0x7fe6d8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f57fa700 1 -- 172.21.2.111:0/1508564778 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe6e806fdd0 con 0x7fe6f80abdc0 2021-04-03T03:10:34.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.991+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6d8056ea0 0x7fe6d8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.995+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6d8056ea0 0x7fe6d8059360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fe6ec01a000 tx=0x7fe6ec013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:33.995+0000 7fe6f57fa700 1 -- 172.21.2.111:0/1508564778 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe6e8045160 con 0x7fe6f80abdc0 2021-04-03T03:10:34.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.087+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe6f80af2b0 con 0x7fe6f80abdc0 2021-04-03T03:10:34.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.087+0000 7fe6f57fa700 1 -- 172.21.2.111:0/1508564778 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6e8048780 con 0x7fe6f80abdc0 2021-04-03T03:10:34.100 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:10:34.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6d8056ea0 msgr2=0x7fe6d8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6d8056ea0 0x7fe6d8059360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fe6ec01a000 tx=0x7fe6ec013000).stop 2021-04-03T03:10:34.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f80abdc0 msgr2=0x7fe6f815a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f80abdc0 0x7fe6f815a6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fe6e800b000 tx=0x7fe6e8009040).stop 2021-04-03T03:10:34.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 shutdown_connections 2021-04-03T03:10:34.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6d8056ea0 0x7fe6d8059360 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6f80a6f20 0x7fe6f815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f80ab420 0x7fe6f815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 --2- 172.21.2.111:0/1508564778 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f80abdc0 0x7fe6f815a6f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 >> 172.21.2.111:0/1508564778 conn(0x7fe6f80b61c0 msgr2=0x7fe6f8142ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 shutdown_connections 2021-04-03T03:10:34.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.091+0000 7fe6fd3ad700 1 -- 172.21.2.111:0/1508564778 wait complete. 2021-04-03T03:10:34.108 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:10:34.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 Processor -- start 2021-04-03T03:10:34.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 -- start start 2021-04-03T03:10:34.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f43fc0603f0 0x7f43fc1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f43fc0add50 0x7f43fc144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc0607d0 con 0x7f43fc0b17b0 2021-04-03T03:10:34.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc1451c0 con 0x7f43fc0add50 2021-04-03T03:10:34.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc14b090 con 0x7f43fc0603f0 2021-04-03T03:10:34.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f43fc0add50 0x7f43fc144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f43fc0add50 0x7f43fc144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51718/0 (socket says 172.21.2.111:51718) 2021-04-03T03:10:34.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 -- 172.21.2.111:0/1331881834 learned_addr learned my addr 172.21.2.111:0/1331881834 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:34.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 -- 172.21.2.111:0/1331881834 >> v2:172.21.2.115:3300/0 conn(0x7f43fc0603f0 msgr2=0x7f43fc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:34.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 --2- 172.21.2.111:0/1331881834 >> v2:172.21.2.115:3300/0 conn(0x7f43fc0603f0 0x7f43fc1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 -- 172.21.2.111:0/1331881834 >> v1:172.21.2.111:6789/0 conn(0x7f43fc0b17b0 legacy=0x7f43fc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 -- 172.21.2.111:0/1331881834 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc14b330 con 0x7f43fc0add50 2021-04-03T03:10:34.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43fb7fe700 1 --2- 172.21.2.111:0/1331881834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0add50 0x7f43fc144cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f43e4002f30 tx=0x7f43e4006e90).ready entity=mon.0 client_cookie=ff3c9124c5cbbba4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43faffd700 1 -- 172.21.2.111:0/1331881834 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43e40099c0 con 0x7f43fc0add50 2021-04-03T03:10:34.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43faffd700 1 -- 172.21.2.111:0/1331881834 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e4009b20 con 0x7f43fc0add50 2021-04-03T03:10:34.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f43faffd700 1 -- 172.21.2.111:0/1331881834 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43e4009e80 con 0x7f43fc0add50 2021-04-03T03:10:34.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 -- 172.21.2.111:0/1331881834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0add50 msgr2=0x7f43fc144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.155+0000 7f440142d700 1 --2- 172.21.2.111:0/1331881834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0add50 0x7f43fc144cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f43e4002f30 tx=0x7f43e4006e90).stop 2021-04-03T03:10:34.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- 172.21.2.111:0/1331881834 shutdown_connections 2021-04-03T03:10:34.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 --2- 172.21.2.111:0/1331881834 >> v2:172.21.2.115:3300/0 conn(0x7f43fc0603f0 0x7f43fc1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 --2- 172.21.2.111:0/1331881834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0add50 0x7f43fc144cd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- 172.21.2.111:0/1331881834 >> 172.21.2.111:0/1331881834 conn(0x7f43fc0b61c0 msgr2=0x7f43fc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- 172.21.2.111:0/1331881834 shutdown_connections 2021-04-03T03:10:34.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- 172.21.2.111:0/1331881834 wait complete. 2021-04-03T03:10:34.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 Processor -- start 2021-04-03T03:10:34.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- start start 2021-04-03T03:10:34.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0603f0 0x7f43fc062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43fc0add50 0x7f43fc063280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43fc0b17b0 0x7f43fc15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0aaaa0 con 0x7f43fc0603f0 2021-04-03T03:10:34.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0aa6e0 con 0x7f43fc0b17b0 2021-04-03T03:10:34.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc065400 con 0x7f43fc0add50 2021-04-03T03:10:34.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0603f0 0x7f43fc062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0603f0 0x7f43fc062d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51720/0 (socket says 172.21.2.111:51720) 2021-04-03T03:10:34.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 -- 172.21.2.111:0/1705254751 learned_addr learned my addr 172.21.2.111:0/1705254751 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:34.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 -- 172.21.2.111:0/1705254751 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43fc0add50 msgr2=0x7f43fc063280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43fc0add50 0x7f43fc063280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 -- 172.21.2.111:0/1705254751 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43fc0b17b0 msgr2=0x7f43fc15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43fc0b17b0 0x7f43fc15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 -- 172.21.2.111:0/1705254751 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0656a0 con 0x7f43fc0603f0 2021-04-03T03:10:34.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fbfff700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0603f0 0x7f43fc062d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f43ec002f30 tx=0x7f43ec006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43f97fa700 1 -- 172.21.2.111:0/1705254751 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43ec00d840 con 0x7f43fc0603f0 2021-04-03T03:10:34.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43f97fa700 1 -- 172.21.2.111:0/1705254751 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec000e90 con 0x7f43fc0603f0 2021-04-03T03:10:34.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc0aa3d0 con 0x7f43fc0603f0 2021-04-03T03:10:34.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc15a6b0 con 0x7f43fc0603f0 2021-04-03T03:10:34.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43f97fa700 1 -- 172.21.2.111:0/1705254751 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43ec00db90 con 0x7f43fc0603f0 2021-04-03T03:10:34.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc057660 con 0x7f43fc0603f0 2021-04-03T03:10:34.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43f97fa700 1 -- 172.21.2.111:0/1705254751 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f43ec017030 con 0x7f43fc0603f0 2021-04-03T03:10:34.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43f97fa700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43e0056f70 0x7f43e0059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43f97fa700 1 -- 172.21.2.111:0/1705254751 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f43ec06f750 con 0x7f43fc0603f0 2021-04-03T03:10:34.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fb7fe700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43e0056f70 0x7f43e0059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.159+0000 7f43fb7fe700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43e0056f70 0x7f43e0059430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f43e4002f30 tx=0x7f43e4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.163+0000 7f43f97fa700 1 -- 172.21.2.111:0/1705254751 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f43ec044ae0 con 0x7f43fc0603f0 2021-04-03T03:10:34.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.255+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43fc148850 con 0x7f43fc0603f0 2021-04-03T03:10:34.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.255+0000 7f43f97fa700 1 -- 172.21.2.111:0/1705254751 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+10 (secure 0 0 0) 0x7f43ec048100 con 0x7f43fc0603f0 2021-04-03T03:10:34.268 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:10:34.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43e0056f70 msgr2=0x7f43e0059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43e0056f70 0x7f43e0059430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f43e4002f30 tx=0x7f43e4015000).stop 2021-04-03T03:10:34.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0603f0 msgr2=0x7f43fc062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0603f0 0x7f43fc062d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f43ec002f30 tx=0x7f43ec006ea0).stop 2021-04-03T03:10:34.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 shutdown_connections 2021-04-03T03:10:34.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43e0056f70 0x7f43e0059430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43fc0b17b0 0x7f43fc15a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43fc0603f0 0x7f43fc062d90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 --2- 172.21.2.111:0/1705254751 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43fc0add50 0x7f43fc063280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 >> 172.21.2.111:0/1705254751 conn(0x7f43fc0b61c0 msgr2=0x7f43fc1449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 shutdown_connections 2021-04-03T03:10:34.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.259+0000 7f440142d700 1 -- 172.21.2.111:0/1705254751 wait complete. 2021-04-03T03:10:34.276 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:10:34.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 Processor -- start 2021-04-03T03:10:34.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- start start 2021-04-03T03:10:34.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbebc0af920 0x7fbebc0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc0afd00 con 0x7fbebc0b25b0 2021-04-03T03:10:34.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc144a80 con 0x7fbebc1420d0 2021-04-03T03:10:34.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbebc141330 con 0x7fbebc0af920 2021-04-03T03:10:34.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebaffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbebc0af920 0x7fbebc0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbeba7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fbebc0b25b0 0x7fbebc1419c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52522/0 (socket says 172.21.2.111:52522) 2021-04-03T03:10:34.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbeba7fc700 1 -- 172.21.2.111:0/326301771 learned_addr learned my addr 172.21.2.111:0/326301771 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:34.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbeb9ffb700 1 -- 172.21.2.111:0/326301771 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726431033 0 0) 0x7fbebc0afd00 con 0x7fbebc0b25b0 2021-04-03T03:10:34.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbeb9ffb700 1 -- 172.21.2.111:0/326301771 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe9c003240 con 0x7fbebc0b25b0 2021-04-03T03:10:34.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebaffd700 1 -- 172.21.2.111:0/326301771 >> v1:172.21.2.115:6789/0 conn(0x7fbebc1420d0 legacy=0x7fbebc13e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:34.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebaffd700 1 -- 172.21.2.111:0/326301771 >> v1:172.21.2.111:6789/0 conn(0x7fbebc0b25b0 legacy=0x7fbebc1419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebaffd700 1 -- 172.21.2.111:0/326301771 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc141520 con 0x7fbebc0af920 2021-04-03T03:10:34.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebaffd700 1 --2- 172.21.2.111:0/326301771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc0af920 0x7fbebc0b0b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fbebc060aa0 tx=0x7fbeac006dd0).ready entity=mon.0 client_cookie=1b295b8c03cd77fb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbeb9ffb700 1 -- 172.21.2.111:0/326301771 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac009770 con 0x7fbebc0af920 2021-04-03T03:10:34.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbeb9ffb700 1 -- 172.21.2.111:0/326301771 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac0098d0 con 0x7fbebc0af920 2021-04-03T03:10:34.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbeb9ffb700 1 -- 172.21.2.111:0/326301771 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac009c70 con 0x7fbebc0af920 2021-04-03T03:10:34.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- 172.21.2.111:0/326301771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc0af920 msgr2=0x7fbebc0b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 --2- 172.21.2.111:0/326301771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc0af920 0x7fbebc0b0b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fbebc060aa0 tx=0x7fbeac006dd0).stop 2021-04-03T03:10:34.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- 172.21.2.111:0/326301771 shutdown_connections 2021-04-03T03:10:34.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 --2- 172.21.2.111:0/326301771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc0af920 0x7fbebc0b0b60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- 172.21.2.111:0/326301771 >> 172.21.2.111:0/326301771 conn(0x7fbebc0b5fb0 msgr2=0x7fbebc0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- 172.21.2.111:0/326301771 shutdown_connections 2021-04-03T03:10:34.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.323+0000 7fbebbfff700 1 -- 172.21.2.111:0/326301771 wait complete. 2021-04-03T03:10:34.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 Processor -- start 2021-04-03T03:10:34.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 -- start start 2021-04-03T03:10:34.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbebc0af920 0x7fbebc066050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbebc0b25b0 0x7fbebc13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc1420d0 0x7fbebc140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc148720 con 0x7fbebc1420d0 2021-04-03T03:10:34.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc146c80 con 0x7fbebc0b25b0 2021-04-03T03:10:34.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1483a0 con 0x7fbebc0af920 2021-04-03T03:10:34.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc1420d0 0x7fbebc140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc1420d0 0x7fbebc140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51738/0 (socket says 172.21.2.111:51738) 2021-04-03T03:10:34.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 -- 172.21.2.111:0/2799165773 learned_addr learned my addr 172.21.2.111:0/2799165773 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:34.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 -- 172.21.2.111:0/2799165773 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbebc0af920 msgr2=0x7fbebc066050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbebc0af920 0x7fbebc066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 -- 172.21.2.111:0/2799165773 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbebc0b25b0 msgr2=0x7fbebc13fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbebc0b25b0 0x7fbebc13fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 -- 172.21.2.111:0/2799165773 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc147f90 con 0x7fbebc1420d0 2021-04-03T03:10:34.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebb7fe700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc1420d0 0x7fbebc140010 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fbeb4003230 tx=0x7fbeb4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbeb3fff700 1 -- 172.21.2.111:0/2799165773 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeb400d680 con 0x7fbebc1420d0 2021-04-03T03:10:34.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbeb3fff700 1 -- 172.21.2.111:0/2799165773 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb400d7e0 con 0x7fbebc1420d0 2021-04-03T03:10:34.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbeb3fff700 1 -- 172.21.2.111:0/2799165773 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeb400db40 con 0x7fbebc1420d0 2021-04-03T03:10:34.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc1477b0 con 0x7fbebc1420d0 2021-04-03T03:10:34.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc140500 con 0x7fbebc1420d0 2021-04-03T03:10:34.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc1481e0 con 0x7fbebc1420d0 2021-04-03T03:10:34.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbeb3fff700 1 -- 172.21.2.111:0/2799165773 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbeb401e070 con 0x7fbebc1420d0 2021-04-03T03:10:34.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbeb3fff700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbea0056fb0 0x7fbea0059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbeb3fff700 1 -- 172.21.2.111:0/2799165773 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fbeb40701d0 con 0x7fbebc1420d0 2021-04-03T03:10:34.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebaffd700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbea0056fb0 0x7fbea0059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.327+0000 7fbebaffd700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbea0056fb0 0x7fbea0059470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fbebc060aa0 tx=0x7fbeac015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.331+0000 7fbeb3fff700 1 -- 172.21.2.111:0/2799165773 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbebc1481e0 con 0x7fbebc1420d0 2021-04-03T03:10:34.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbebc1481e0 con 0x7fbebc1420d0 2021-04-03T03:10:34.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbeb3fff700 1 -- 172.21.2.111:0/2799165773 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbebc1481e0 con 0x7fbebc1420d0 2021-04-03T03:10:34.434 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:10:34.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbea0056fb0 msgr2=0x7fbea0059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbea0056fb0 0x7fbea0059470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fbebc060aa0 tx=0x7fbeac015000).stop 2021-04-03T03:10:34.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc1420d0 msgr2=0x7fbebc140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc1420d0 0x7fbebc140010 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fbeb4003230 tx=0x7fbeb4009060).stop 2021-04-03T03:10:34.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 shutdown_connections 2021-04-03T03:10:34.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbebc1420d0 0x7fbebc140010 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbebc0af920 0x7fbebc066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbea0056fb0 0x7fbea0059470 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 --2- 172.21.2.111:0/2799165773 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbebc0b25b0 0x7fbebc13fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 >> 172.21.2.111:0/2799165773 conn(0x7fbebc0b5fb0 msgr2=0x7fbebc145250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 shutdown_connections 2021-04-03T03:10:34.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.423+0000 7fbebbfff700 1 -- 172.21.2.111:0/2799165773 wait complete. 2021-04-03T03:10:34.441 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:10:34.442 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:34.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 Processor -- start 2021-04-03T03:10:34.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 -- start start 2021-04-03T03:10:34.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f33ec144150 0x7f33ec13fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec0af280 0x7f33ec0af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ec0b1460 con 0x7f33ec0b0610 2021-04-03T03:10:34.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ec0b6ab0 con 0x7f33ec144150 2021-04-03T03:10:34.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0b6d50 con 0x7f33ec0af280 2021-04-03T03:10:34.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f33ec144150 0x7f33ec13fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f33ec144150 0x7f33ec13fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51744/0 (socket says 172.21.2.111:51744) 2021-04-03T03:10:34.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 -- 172.21.2.111:0/2874162900 learned_addr learned my addr 172.21.2.111:0/2874162900 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:34.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 -- 172.21.2.111:0/2874162900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec0af280 msgr2=0x7f33ec0af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:34.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 --2- 172.21.2.111:0/2874162900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec0af280 0x7f33ec0af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 -- 172.21.2.111:0/2874162900 >> v1:172.21.2.115:6789/0 conn(0x7f33ec0b0610 legacy=0x7f33ec13f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:34.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 -- 172.21.2.111:0/2874162900 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0b6ff0 con 0x7f33ec144150 2021-04-03T03:10:34.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33eaffd700 1 --2- 172.21.2.111:0/2874162900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec144150 0x7f33ec13fd90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f33ec0b0fe0 tx=0x7f33d4006f20).ready entity=mon.0 client_cookie=3efd6785e13ac60 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33ea7fc700 1 -- 172.21.2.111:0/2874162900 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d400de10 con 0x7f33ec144150 2021-04-03T03:10:34.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33ea7fc700 1 -- 172.21.2.111:0/2874162900 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d4010470 con 0x7f33ec144150 2021-04-03T03:10:34.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33ea7fc700 1 -- 172.21.2.111:0/2874162900 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d40107d0 con 0x7f33ec144150 2021-04-03T03:10:34.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 -- 172.21.2.111:0/2874162900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec144150 msgr2=0x7f33ec13fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.491+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/2874162900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec144150 0x7f33ec13fd90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f33ec0b0fe0 tx=0x7f33d4006f20).stop 2021-04-03T03:10:34.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- 172.21.2.111:0/2874162900 shutdown_connections 2021-04-03T03:10:34.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/2874162900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec0af280 0x7f33ec0af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/2874162900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec144150 0x7f33ec13fd90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- 172.21.2.111:0/2874162900 >> 172.21.2.111:0/2874162900 conn(0x7f33ec063d70 msgr2=0x7f33ec064180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- 172.21.2.111:0/2874162900 shutdown_connections 2021-04-03T03:10:34.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- 172.21.2.111:0/2874162900 wait complete. 2021-04-03T03:10:34.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 Processor -- start 2021-04-03T03:10:34.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- start start 2021-04-03T03:10:34.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33ec0af280 0x7f33ec066010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec0b0610 0x7f33ec0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec144150 0x7f33ec0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0ac1f0 con 0x7f33ec0b0610 2021-04-03T03:10:34.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0ac3d0 con 0x7f33ec0af280 2021-04-03T03:10:34.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0ac670 con 0x7f33ec144150 2021-04-03T03:10:34.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec0b0610 0x7f33ec0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec0b0610 0x7f33ec0acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51748/0 (socket says 172.21.2.111:51748) 2021-04-03T03:10:34.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 -- 172.21.2.111:0/1918275694 learned_addr learned my addr 172.21.2.111:0/1918275694 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:34.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 -- 172.21.2.111:0/1918275694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec144150 msgr2=0x7f33ec0ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec144150 0x7f33ec0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 -- 172.21.2.111:0/1918275694 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33ec0af280 msgr2=0x7f33ec066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33ec0af280 0x7f33ec066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 -- 172.21.2.111:0/1918275694 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0677a0 con 0x7f33ec0b0610 2021-04-03T03:10:34.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33eaffd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec0b0610 0x7f33ec0acb10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f33ec0b0fe0 tx=0x7f33d4018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33e8ff9700 1 -- 172.21.2.111:0/1918275694 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d401d070 con 0x7f33ec0b0610 2021-04-03T03:10:34.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33e8ff9700 1 -- 172.21.2.111:0/1918275694 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d4018ad0 con 0x7f33ec0b0610 2021-04-03T03:10:34.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33e8ff9700 1 -- 172.21.2.111:0/1918275694 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d4018e30 con 0x7f33ec0b0610 2021-04-03T03:10:34.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec0a9010 con 0x7f33ec0b0610 2021-04-03T03:10:34.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec0ad680 con 0x7f33ec0b0610 2021-04-03T03:10:34.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ec0ac830 con 0x7f33ec0b0610 2021-04-03T03:10:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33e8ff9700 1 -- 172.21.2.111:0/1918275694 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f33d4029070 con 0x7f33ec0b0610 2021-04-03T03:10:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.495+0000 7f33e8ff9700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33d0056ea0 0x7f33d0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.499+0000 7f33e8ff9700 1 -- 172.21.2.111:0/1918275694 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f33d4071990 con 0x7f33ec0b0610 2021-04-03T03:10:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.499+0000 7f33eb7fe700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33d0056ea0 0x7f33d0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.499+0000 7f33eb7fe700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33d0056ea0 0x7f33d0059360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f33dc002f50 tx=0x7f33dc006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:34.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.499+0000 7f33e8ff9700 1 -- 172.21.2.111:0/1918275694 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f33d4046d20 con 0x7f33ec0b0610 2021-04-03T03:10:34.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.623+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33ec057660 con 0x7f33ec0b0610 2021-04-03T03:10:34.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.623+0000 7f33e8ff9700 1 -- 172.21.2.111:0/1918275694 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f33d404a340 con 0x7f33ec0b0610 2021-04-03T03:10:34.636 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:34.636 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:10:34.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33d0056ea0 msgr2=0x7f33d0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33d0056ea0 0x7f33d0059360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f33dc002f50 tx=0x7f33dc006990).stop 2021-04-03T03:10:34.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec0b0610 msgr2=0x7f33ec0acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:34.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec0b0610 0x7f33ec0acb10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f33ec0b0fe0 tx=0x7f33d4018000).stop 2021-04-03T03:10:34.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 shutdown_connections 2021-04-03T03:10:34.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33ec144150 0x7f33ec0ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33d0056ea0 0x7f33d0059360 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33ec0af280 0x7f33ec066010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:34.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 --2- 172.21.2.111:0/1918275694 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33ec0b0610 0x7f33ec0acb10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:34.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 >> 172.21.2.111:0/1918275694 conn(0x7f33ec063d70 msgr2=0x7f33ec0b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:34.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 shutdown_connections 2021-04-03T03:10:34.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:34.627+0000 7f33f0cdd700 1 -- 172.21.2.111:0/1918275694 wait complete. 2021-04-03T03:10:34.640 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:10:34.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:10:31.116099+0000', 'modified': '2021-04-03T03:10:32.118491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5241': {'gid': 5241, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/798702406', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 798702406}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 798702406}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:10:34.846 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:10:34.846 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:10:34.918 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5241,"want_state":"up:active","state":"up:active","rank_uptime":3.7887064850000001,"mdsmap_epoch":16,"osdmap_epoch":43,"osdmap_epoch_barrier":42,"uptime":7.9324930350000002} 2021-04-03T03:10:34.918 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5241, "mdsmap_epoch": 16, "osdmap_epoch": 43, "osdmap_epoch_barrier": 42, "rank_uptime": 3.788706485, "state": "up:active", "uptime": 7.932493035, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:10:34.919 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:10:34.919 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:10:34.919 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:10:34.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:34.920 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:34.931 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:10:34.984 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:10:34.985 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:10:35.032 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:10:35.033 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:10:35.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:35.034 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:10:35.086 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:10:35.086 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:35.132 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:10:35.143 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:10:35.143 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:10:35.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:10:35.163+0000 7fa1ba8bd0c0 -1 init, newargv = 0x563b9e9a6b60 newargc=9 2021-04-03T03:10:35.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15043]: starting ceph client 2021-04-03T03:10:35.179 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:10:35.179 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:10:35.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:35.181 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:10:35.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15043]: starting fuse 2021-04-03T03:10:35.189 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:10:35.189 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:10:35.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:35.190 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:10:35.190 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:10:35.191 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:10:35.191 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:10:35.192 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:10:35.192 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:10:35.192 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:10:35.193 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:10:35.193 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:10:35.194 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:10:35.194 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:10:35.194 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:10:35.195 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:10:35.195 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:10:35.196 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:10:35.196 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:10:35.197 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:10:35.197 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:10:35.197 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:10:35.198 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:10:35.198 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:10:35.199 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:10:35.199 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:10:35.256 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15043.asok 2021-04-03T03:10:35.259 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15043.asok 2021-04-03T03:10:35.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:35.260 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15043.asok status 2021-04-03T03:10:35.357 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:10:35.357 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:10:35.357 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:10:35.358 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:10:35.358 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:10:35.358 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:10:35.358 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:10:35.358 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15043", 2021-04-03T03:10:35.359 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:10:35.359 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:10:35.359 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:10:35.359 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:10:35.359 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4475, 2021-04-03T03:10:35.360 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:10:35.360 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:10:35.360 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:10:35.360 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4475 2021-04-03T03:10:35.360 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:10:35.361 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:10:35.361 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:10:35.361 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:10:35.361 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 1124968257 2021-04-03T03:10:35.362 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:10:35.362 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:10:35.362 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:10:35.362 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:10:35.362 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:10:35.363 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 1124968257 2021-04-03T03:10:35.363 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:10:35.363 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4475 172.21.2.126:0/1124968257", 2021-04-03T03:10:35.363 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/1124968257", 2021-04-03T03:10:35.363 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:10:35.364 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 16, 2021-04-03T03:10:35.364 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 43, 2021-04-03T03:10:35.364 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:10:35.364 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:10:35.364 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:10:35.365 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:10:35.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:35.366 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:35.367 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:35.376 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:10:35.377 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:10:35.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:35.377 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:35.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:10:35.442 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:10:35.442 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:10:35.492 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:10:35.493 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:10:35.494 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:10:35.494 DEBUG:teuthology.orchestra.run.gibba026:> import sys 2021-04-03T03:10:35.495 DEBUG:teuthology.orchestra.run.gibba026:> import time 2021-04-03T03:10:35.495 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:10:35.495 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:10:35.496 DEBUG:teuthology.orchestra.run.gibba026:> n = 1000 2021-04-03T03:10:35.496 DEBUG:teuthology.orchestra.run.gibba026:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-04-03T03:10:35.497 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:10:35.497 DEBUG:teuthology.orchestra.run.gibba026:> if not os.path.exists(os.path.dirname(abs_path)): 2021-04-03T03:10:35.497 DEBUG:teuthology.orchestra.run.gibba026:> os.makedirs(os.path.dirname(abs_path)) 2021-04-03T03:10:35.498 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:10:35.498 DEBUG:teuthology.orchestra.run.gibba026:> for i in range(0, n): 2021-04-03T03:10:35.498 DEBUG:teuthology.orchestra.run.gibba026:> fname = "{0}_{1}".format(abs_path, i) 2021-04-03T03:10:35.499 DEBUG:teuthology.orchestra.run.gibba026:> with open(fname, '"'"'w'"'"') as f: 2021-04-03T03:10:35.499 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content'"'"') 2021-04-03T03:10:35.500 DEBUG:teuthology.orchestra.run.gibba026:> if True: 2021-04-03T03:10:35.500 DEBUG:teuthology.orchestra.run.gibba026:> f.flush() 2021-04-03T03:10:35.500 DEBUG:teuthology.orchestra.run.gibba026:> os.fsync(f.fileno()) 2021-04-03T03:10:35.501 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:10:51.411 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:51.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 Processor -- start 2021-04-03T03:10:51.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- start start 2021-04-03T03:10:51.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd02c005dc0 0x7fd02c042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c0dc6e0 con 0x7fd02c0dbb90 2021-04-03T03:10:51.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c042010 con 0x7fd02c006bc0 2021-04-03T03:10:51.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c0d9560 con 0x7fd02c005dc0 2021-04-03T03:10:51.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd0337fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd02c005dc0 0x7fd02c042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd0337fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd02c005dc0 0x7fd02c042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51760/0 (socket says 172.21.2.111:51760) 2021-04-03T03:10:51.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd0337fe700 1 -- 172.21.2.111:0/1970268695 learned_addr learned my addr 172.21.2.111:0/1970268695 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:51.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd0337fe700 1 -- 172.21.2.111:0/1970268695 >> v1:172.21.2.115:6789/0 conn(0x7fd02c006bc0 legacy=0x7fd02c0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:51.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd0337fe700 1 -- 172.21.2.111:0/1970268695 >> v1:172.21.2.111:6789/0 conn(0x7fd02c0dbb90 legacy=0x7fd02c0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd0337fe700 1 -- 172.21.2.111:0/1970268695 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0422b0 con 0x7fd02c005dc0 2021-04-03T03:10:51.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd0337fe700 1 --2- 172.21.2.111:0/1970268695 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c005dc0 0x7fd02c042750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fd02c0087e0 tx=0x7fd03405ad70).ready entity=mon.0 client_cookie=a4cc958453a87a63 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:51.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd031ffb700 1 -- 172.21.2.111:0/1970268695 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd03405bda0 con 0x7fd02c005dc0 2021-04-03T03:10:51.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd031ffb700 1 -- 172.21.2.111:0/1970268695 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03405df60 con 0x7fd02c005dc0 2021-04-03T03:10:51.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd031ffb700 1 -- 172.21.2.111:0/1970268695 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0340699f0 con 0x7fd02c005dc0 2021-04-03T03:10:51.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- 172.21.2.111:0/1970268695 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c005dc0 msgr2=0x7fd02c042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 --2- 172.21.2.111:0/1970268695 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c005dc0 0x7fd02c042750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fd02c0087e0 tx=0x7fd03405ad70).stop 2021-04-03T03:10:51.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- 172.21.2.111:0/1970268695 shutdown_connections 2021-04-03T03:10:51.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 --2- 172.21.2.111:0/1970268695 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c005dc0 0x7fd02c042750 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- 172.21.2.111:0/1970268695 >> 172.21.2.111:0/1970268695 conn(0x7fd02c0462c0 msgr2=0x7fd02c0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:51.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- 172.21.2.111:0/1970268695 shutdown_connections 2021-04-03T03:10:51.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.475+0000 7fd033fff700 1 -- 172.21.2.111:0/1970268695 wait complete. 2021-04-03T03:10:51.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 Processor -- start 2021-04-03T03:10:51.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 -- start start 2021-04-03T03:10:51.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd02c005dc0 0x7fd02c039ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c0dbb90 0x7fd02c0da150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd02c0a06d0 0x7fd02c0da640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0d9770 con 0x7fd02c0dbb90 2021-04-03T03:10:51.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0d9a10 con 0x7fd02c005dc0 2021-04-03T03:10:51.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0d9cb0 con 0x7fd02c0a06d0 2021-04-03T03:10:51.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c0dbb90 0x7fd02c0da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c0dbb90 0x7fd02c0da150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51766/0 (socket says 172.21.2.111:51766) 2021-04-03T03:10:51.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 -- 172.21.2.111:0/573716264 learned_addr learned my addr 172.21.2.111:0/573716264 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:51.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 -- 172.21.2.111:0/573716264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd02c0a06d0 msgr2=0x7fd02c0da640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd02c0a06d0 0x7fd02c0da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:51.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 -- 172.21.2.111:0/573716264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd02c005dc0 msgr2=0x7fd02c039ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd02c005dc0 0x7fd02c039ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:51.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 -- 172.21.2.111:0/573716264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0a16a0 con 0x7fd02c0dbb90 2021-04-03T03:10:51.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd0327fc700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c0dbb90 0x7fd02c0da150 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fd01c002f50 tx=0x7fd01c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:51.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd02bfff700 1 -- 172.21.2.111:0/573716264 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd01c00d4a0 con 0x7fd02c0dbb90 2021-04-03T03:10:51.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd02bfff700 1 -- 172.21.2.111:0/573716264 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c00d600 con 0x7fd02c0dbb90 2021-04-03T03:10:51.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c0d5c10 con 0x7fd02c0dbb90 2021-04-03T03:10:51.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd02bfff700 1 -- 172.21.2.111:0/573716264 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd01c00d960 con 0x7fd02c0dbb90 2021-04-03T03:10:51.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02c0dab30 con 0x7fd02c0dbb90 2021-04-03T03:10:51.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02c0d9e70 con 0x7fd02c0dbb90 2021-04-03T03:10:51.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd02bfff700 1 -- 172.21.2.111:0/573716264 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd01c00f070 con 0x7fd02c0dbb90 2021-04-03T03:10:51.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd02bfff700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd018056f20 0x7fd0180593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd02bfff700 1 -- 172.21.2.111:0/573716264 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd01c06fca0 con 0x7fd02c0dbb90 2021-04-03T03:10:51.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd032ffd700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd018056f20 0x7fd0180593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.479+0000 7fd032ffd700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd018056f20 0x7fd0180593e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fd02c046210 tx=0x7fd0240077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:51.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.483+0000 7fd02bfff700 1 -- 172.21.2.111:0/573716264 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd01c074070 con 0x7fd02c0dbb90 2021-04-03T03:10:51.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.603+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd02c004da0 con 0x7fd02c0dbb90 2021-04-03T03:10:51.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.603+0000 7fd02bfff700 1 -- 172.21.2.111:0/573716264 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fd01c045090 con 0x7fd02c0dbb90 2021-04-03T03:10:51.616 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:51.617 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:10:51.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd018056f20 msgr2=0x7fd0180593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd018056f20 0x7fd0180593e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fd02c046210 tx=0x7fd0240077d0).stop 2021-04-03T03:10:51.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c0dbb90 msgr2=0x7fd02c0da150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c0dbb90 0x7fd02c0da150 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fd01c002f50 tx=0x7fd01c006dd0).stop 2021-04-03T03:10:51.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 shutdown_connections 2021-04-03T03:10:51.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd02c0a06d0 0x7fd02c0da640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:51.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd018056f20 0x7fd0180593e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd02c005dc0 0x7fd02c039ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:51.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 --2- 172.21.2.111:0/573716264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd02c0dbb90 0x7fd02c0da150 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 >> 172.21.2.111:0/573716264 conn(0x7fd02c0462c0 msgr2=0x7fd02c0442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:51.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 shutdown_connections 2021-04-03T03:10:51.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.607+0000 7fd033fff700 1 -- 172.21.2.111:0/573716264 wait complete. 2021-04-03T03:10:51.620 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:10:51.624 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2021-04-03T03:10:51.693 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-04-03T03:10:51.693 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-04-03T03:10:51.694 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:51.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 Processor -- start 2021-04-03T03:10:51.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- start start 2021-04-03T03:10:51.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f81cc0603f0 0x7f81cc0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0b0c90 0x7f81cc147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc0607d0 con 0x7f81cc14b700 2021-04-03T03:10:51.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc0af400 con 0x7f81cc0603f0 2021-04-03T03:10:51.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0af6a0 con 0x7f81cc0b0c90 2021-04-03T03:10:51.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d15ba700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f81cc14b700 0x7f81cc0b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:36872/0 (socket says 172.21.2.111:36872) 2021-04-03T03:10:51.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d15ba700 1 -- 172.21.2.111:0/1423843637 learned_addr learned my addr 172.21.2.111:0/1423843637 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:10:51.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d1dbb700 1 --2- 172.21.2.111:0/1423843637 >> v2:172.21.2.115:3300/0 conn(0x7f81cc0603f0 0x7f81cc0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d25bc700 1 --2- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0b0c90 0x7f81cc147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d1dbb700 1 -- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0b0c90 msgr2=0x7f81cc147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:51.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d1dbb700 1 --2- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0b0c90 0x7f81cc147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d1dbb700 1 -- 172.21.2.111:0/1423843637 >> v1:172.21.2.115:6789/0 conn(0x7f81cc14b700 legacy=0x7f81cc0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d1dbb700 1 -- 172.21.2.111:0/1423843637 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0afb40 con 0x7f81cc0603f0 2021-04-03T03:10:51.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d1dbb700 1 --2- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc0603f0 0x7f81cc0b0090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f81c0002f30 tx=0x7f81c0006dd0).ready entity=mon.1 client_cookie=8e1a73699cb6b64d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:51.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d0db9700 1 -- 172.21.2.111:0/1423843637 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81c0009760 con 0x7f81cc0603f0 2021-04-03T03:10:51.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d0db9700 1 -- 172.21.2.111:0/1423843637 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c00098c0 con 0x7f81cc0603f0 2021-04-03T03:10:51.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d0db9700 1 -- 172.21.2.111:0/1423843637 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81c0009c20 con 0x7f81cc0603f0 2021-04-03T03:10:51.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc0603f0 msgr2=0x7f81cc0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc0603f0 0x7f81cc0b0090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f81c0002f30 tx=0x7f81c0006dd0).stop 2021-04-03T03:10:51.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1423843637 shutdown_connections 2021-04-03T03:10:51.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc0603f0 0x7f81cc0b0090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1423843637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0b0c90 0x7f81cc147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1423843637 >> 172.21.2.111:0/1423843637 conn(0x7f81cc0b61c0 msgr2=0x7f81cc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:51.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1423843637 shutdown_connections 2021-04-03T03:10:51.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1423843637 wait complete. 2021-04-03T03:10:51.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 Processor -- start 2021-04-03T03:10:51.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.739+0000 7f81d2dbd700 1 -- start start 2021-04-03T03:10:51.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0603f0 0x7f81cc0a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81cc0b0c90 0x7f81cc0ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc14b700 0x7f81cc0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc067fe0 con 0x7f81cc0b0c90 2021-04-03T03:10:51.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc067770 con 0x7f81cc14b700 2021-04-03T03:10:51.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0aa2e0 con 0x7f81cc0603f0 2021-04-03T03:10:51.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d1dbb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0603f0 0x7f81cc0a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d15ba700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81cc0b0c90 0x7f81cc0ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d1dbb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0603f0 0x7f81cc0a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:35126/0 (socket says 172.21.2.111:35126) 2021-04-03T03:10:51.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d1dbb700 1 -- 172.21.2.111:0/1100066599 learned_addr learned my addr 172.21.2.111:0/1100066599 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:51.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d25bc700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc14b700 0x7f81cc0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d15ba700 1 -- 172.21.2.111:0/1100066599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0603f0 msgr2=0x7f81cc0a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d15ba700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0603f0 0x7f81cc0a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d15ba700 1 -- 172.21.2.111:0/1100066599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc14b700 msgr2=0x7f81cc0abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d15ba700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc14b700 0x7f81cc0abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d15ba700 1 -- 172.21.2.111:0/1100066599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc067c60 con 0x7f81cc0b0c90 2021-04-03T03:10:51.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d15ba700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81cc0b0c90 0x7f81cc0ab730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f81cc06a3b0 tx=0x7f81c8007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:51.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81beffd700 1 -- 172.21.2.111:0/1100066599 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81c800ce20 con 0x7f81cc0b0c90 2021-04-03T03:10:51.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81beffd700 1 -- 172.21.2.111:0/1100066599 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c80072e0 con 0x7f81cc0b0c90 2021-04-03T03:10:51.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81beffd700 1 -- 172.21.2.111:0/1100066599 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81c80057e0 con 0x7f81cc0b0c90 2021-04-03T03:10:51.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc0631f0 con 0x7f81cc0b0c90 2021-04-03T03:10:51.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc0ab0e0 con 0x7f81cc0b0c90 2021-04-03T03:10:51.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc057660 con 0x7f81cc0b0c90 2021-04-03T03:10:51.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81beffd700 1 -- 172.21.2.111:0/1100066599 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f81c8010070 con 0x7f81cc0b0c90 2021-04-03T03:10:51.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81beffd700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f81ac056fb0 0x7f81ac059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:51.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81beffd700 1 -- 172.21.2.111:0/1100066599 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f81c806f610 con 0x7f81cc0b0c90 2021-04-03T03:10:51.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d1dbb700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f81ac056fb0 0x7f81ac059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:51.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.743+0000 7f81d1dbb700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f81ac056fb0 0x7f81ac059470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f81c001b000 tx=0x7f81c0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:51.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.747+0000 7f81beffd700 1 -- 172.21.2.111:0/1100066599 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f81c8044a00 con 0x7f81cc0b0c90 2021-04-03T03:10:51.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.871+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81cc0b3de0 con 0x7f81cc0b0c90 2021-04-03T03:10:51.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.871+0000 7f81beffd700 1 -- 172.21.2.111:0/1100066599 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f81c8048020 con 0x7f81cc0b0c90 2021-04-03T03:10:51.884 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:51.884 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:10:51.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f81ac056fb0 msgr2=0x7f81ac059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f81ac056fb0 0x7f81ac059470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f81c001b000 tx=0x7f81c0015000).stop 2021-04-03T03:10:51.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81cc0b0c90 msgr2=0x7f81cc0ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:51.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81cc0b0c90 0x7f81cc0ab730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f81cc06a3b0 tx=0x7f81c8007bc0).stop 2021-04-03T03:10:51.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 shutdown_connections 2021-04-03T03:10:51.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f81ac056fb0 0x7f81ac059470 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81cc14b700 0x7f81cc0abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81cc0603f0 0x7f81cc0a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 --2- 172.21.2.111:0/1100066599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81cc0b0c90 0x7f81cc0ab730 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:51.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 >> 172.21.2.111:0/1100066599 conn(0x7f81cc0b61c0 msgr2=0x7f81cc146e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:51.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 shutdown_connections 2021-04-03T03:10:51.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:51.875+0000 7f81d2dbd700 1 -- 172.21.2.111:0/1100066599 wait complete. 2021-04-03T03:10:51.888 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:10:51.892 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2021-04-03T03:10:51.956 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-04-03T03:10:51.956 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-04-03T03:10:51.957 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:10:51.971 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:10:52.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 Processor -- start 2021-04-03T03:10:52.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- start start 2021-04-03T03:10:52.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f50440603f0 0x7f50440b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f504414b700 0x7f50440b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50440b12b0 0x7f50440b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50440b0f60 con 0x7f504414b700 2021-04-03T03:10:52.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50440607d0 con 0x7f50440603f0 2021-04-03T03:10:52.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440af400 con 0x7f50440b12b0 2021-04-03T03:10:52.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f504414b700 0x7f50440b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f504414b700 0x7f50440b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51788/0 (socket says 172.21.2.111:51788) 2021-04-03T03:10:52.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 -- 172.21.2.111:0/2086290832 learned_addr learned my addr 172.21.2.111:0/2086290832 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5042d9d700 1 --2- 172.21.2.111:0/2086290832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50440b12b0 0x7f50440b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504259c700 1 --2- 172.21.2.111:0/2086290832 >> v2:172.21.2.115:3300/0 conn(0x7f50440603f0 0x7f50440b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 -- 172.21.2.111:0/2086290832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50440b12b0 msgr2=0x7f50440b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 --2- 172.21.2.111:0/2086290832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50440b12b0 0x7f50440b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 -- 172.21.2.111:0/2086290832 >> v2:172.21.2.115:3300/0 conn(0x7f50440603f0 msgr2=0x7f50440b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 --2- 172.21.2.111:0/2086290832 >> v2:172.21.2.115:3300/0 conn(0x7f50440603f0 0x7f50440b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 -- 172.21.2.111:0/2086290832 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440af6a0 con 0x7f504414b700 2021-04-03T03:10:52.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f5041d9b700 1 --2- 172.21.2.111:0/2086290832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f504414b700 0x7f50440b0580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f502c002f30 tx=0x7f502c006e90).ready entity=mon.0 client_cookie=b8dbd54cedb4da11 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504159a700 1 -- 172.21.2.111:0/2086290832 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f502c009920 con 0x7f504414b700 2021-04-03T03:10:52.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504159a700 1 -- 172.21.2.111:0/2086290832 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c009a80 con 0x7f504414b700 2021-04-03T03:10:52.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504159a700 1 -- 172.21.2.111:0/2086290832 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f502c009de0 con 0x7f504414b700 2021-04-03T03:10:52.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- 172.21.2.111:0/2086290832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f504414b700 msgr2=0x7f50440b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 --2- 172.21.2.111:0/2086290832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f504414b700 0x7f50440b0580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f502c002f30 tx=0x7f502c006e90).stop 2021-04-03T03:10:52.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- 172.21.2.111:0/2086290832 shutdown_connections 2021-04-03T03:10:52.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 --2- 172.21.2.111:0/2086290832 >> v2:172.21.2.115:3300/0 conn(0x7f50440603f0 0x7f50440b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 --2- 172.21.2.111:0/2086290832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50440b12b0 0x7f50440b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 --2- 172.21.2.111:0/2086290832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f504414b700 0x7f50440b0580 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- 172.21.2.111:0/2086290832 >> 172.21.2.111:0/2086290832 conn(0x7f50440b61c0 msgr2=0x7f504405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:52.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- 172.21.2.111:0/2086290832 shutdown_connections 2021-04-03T03:10:52.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.019+0000 7f504359e700 1 -- 172.21.2.111:0/2086290832 wait complete. 2021-04-03T03:10:52.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 Processor -- start 2021-04-03T03:10:52.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 -- start start 2021-04-03T03:10:52.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50440603f0 0x7f50441564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50440b12b0 0x7f504414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f504414b700 0x7f50441455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440ab820 con 0x7f50440b12b0 2021-04-03T03:10:52.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504415a950 con 0x7f50440603f0 2021-04-03T03:10:52.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504415a410 con 0x7f504414b700 2021-04-03T03:10:52.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50440b12b0 0x7f504414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50440b12b0 0x7f504414ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51790/0 (socket says 172.21.2.111:51790) 2021-04-03T03:10:52.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 -- 172.21.2.111:0/35791438 learned_addr learned my addr 172.21.2.111:0/35791438 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 -- 172.21.2.111:0/35791438 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f504414b700 msgr2=0x7f50441455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f504414b700 0x7f50441455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 -- 172.21.2.111:0/35791438 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50440603f0 msgr2=0x7f50441564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50440603f0 0x7f50441564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 -- 172.21.2.111:0/35791438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504415a6b0 con 0x7f50440b12b0 2021-04-03T03:10:52.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f5041d9b700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50440b12b0 0x7f504414ddd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f502c002f30 tx=0x7f502c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f503b7fe700 1 -- 172.21.2.111:0/35791438 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f502c011e00 con 0x7f50440b12b0 2021-04-03T03:10:52.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f503b7fe700 1 -- 172.21.2.111:0/35791438 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c01e460 con 0x7f50440b12b0 2021-04-03T03:10:52.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50440a9f90 con 0x7f50440b12b0 2021-04-03T03:10:52.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50440a9990 con 0x7f50440b12b0 2021-04-03T03:10:52.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f503b7fe700 1 -- 172.21.2.111:0/35791438 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f502c01e7c0 con 0x7f50440b12b0 2021-04-03T03:10:52.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50440aa150 con 0x7f50440b12b0 2021-04-03T03:10:52.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f503b7fe700 1 -- 172.21.2.111:0/35791438 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f502c025030 con 0x7f50440b12b0 2021-04-03T03:10:52.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f503b7fe700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5028056fe0 0x7f50280594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f503b7fe700 1 -- 172.21.2.111:0/35791438 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f502c071690 con 0x7f50440b12b0 2021-04-03T03:10:52.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504259c700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5028056fe0 0x7f50280594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.023+0000 7f504259c700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5028056fe0 0x7f50280594a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f503400a000 tx=0x7f5034008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.027+0000 7f503b7fe700 1 -- 172.21.2.111:0/35791438 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f502c046a80 con 0x7f50440b12b0 2021-04-03T03:10:52.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5044057660 con 0x7f50440b12b0 2021-04-03T03:10:52.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f503b7fe700 1 -- 172.21.2.111:0/35791438 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f502c023020 con 0x7f50440b12b0 2021-04-03T03:10:52.163 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:10:52.164 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:10:52.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5028056fe0 msgr2=0x7f50280594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5028056fe0 0x7f50280594a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f503400a000 tx=0x7f5034008040).stop 2021-04-03T03:10:52.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50440b12b0 msgr2=0x7f504414ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50440b12b0 0x7f504414ddd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f502c002f30 tx=0x7f502c015000).stop 2021-04-03T03:10:52.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 shutdown_connections 2021-04-03T03:10:52.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f504414b700 0x7f50441455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5028056fe0 0x7f50280594a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50440603f0 0x7f50441564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 --2- 172.21.2.111:0/35791438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50440b12b0 0x7f504414ddd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 >> 172.21.2.111:0/35791438 conn(0x7f50440b61c0 msgr2=0x7f50440ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:52.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 shutdown_connections 2021-04-03T03:10:52.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.151+0000 7f504359e700 1 -- 172.21.2.111:0/35791438 wait complete. 2021-04-03T03:10:52.168 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:10:52.171 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 2021-04-03T03:10:52.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.215+0000 7efe2ee44700 1 Processor -- start 2021-04-03T03:10:52.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.215+0000 7efe2ee44700 1 -- start start 2021-04-03T03:10:52.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.215+0000 7efe2ee44700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.215+0000 7efe2ee44700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7efe280b17b0 0x7efe28144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.215+0000 7efe2ee44700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe280607d0 con 0x7efe280add50 2021-04-03T03:10:52.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.215+0000 7efe2ee44700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe281451c0 con 0x7efe280b17b0 2021-04-03T03:10:52.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.215+0000 7efe2ee44700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2814b090 con 0x7efe280603f0 2021-04-03T03:10:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:35146/0 (socket says 172.21.2.111:35146) 2021-04-03T03:10:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2d641700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7efe280add50 0x7efe28147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:36894/0 (socket says 172.21.2.111:36894) 2021-04-03T03:10:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 -- 172.21.2.111:0/54908547 learned_addr learned my addr 172.21.2.111:0/54908547 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2e643700 1 --2- 172.21.2.111:0/54908547 >> v2:172.21.2.115:3300/0 conn(0x7efe280b17b0 0x7efe28144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 -- 172.21.2.111:0/54908547 >> v2:172.21.2.115:3300/0 conn(0x7efe280b17b0 msgr2=0x7efe28144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 --2- 172.21.2.111:0/54908547 >> v2:172.21.2.115:3300/0 conn(0x7efe280b17b0 0x7efe28144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 -- 172.21.2.111:0/54908547 >> v1:172.21.2.115:6789/0 conn(0x7efe280add50 legacy=0x7efe28147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 -- 172.21.2.111:0/54908547 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2814b330 con 0x7efe280603f0 2021-04-03T03:10:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2e643700 1 --2- 172.21.2.111:0/54908547 >> v2:172.21.2.115:3300/0 conn(0x7efe280b17b0 0x7efe28144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:10:52.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2de42700 1 --2- 172.21.2.111:0/54908547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281447e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7efe24002f30 tx=0x7efe24006e90).ready entity=mon.2 client_cookie=7e6c6c8d06e59354 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ce40700 1 -- 172.21.2.111:0/54908547 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe24009850 con 0x7efe280603f0 2021-04-03T03:10:52.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ce40700 1 -- 172.21.2.111:0/54908547 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe240099b0 con 0x7efe280603f0 2021-04-03T03:10:52.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ce40700 1 -- 172.21.2.111:0/54908547 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe24009d10 con 0x7efe280603f0 2021-04-03T03:10:52.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 -- 172.21.2.111:0/54908547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 msgr2=0x7efe281447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 --2- 172.21.2.111:0/54908547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281447e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7efe24002f30 tx=0x7efe24006e90).stop 2021-04-03T03:10:52.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 -- 172.21.2.111:0/54908547 shutdown_connections 2021-04-03T03:10:52.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 --2- 172.21.2.111:0/54908547 >> v2:172.21.2.115:3300/0 conn(0x7efe280b17b0 0x7efe28144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 --2- 172.21.2.111:0/54908547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281447e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 -- 172.21.2.111:0/54908547 >> 172.21.2.111:0/54908547 conn(0x7efe280b61c0 msgr2=0x7efe2805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:52.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 -- 172.21.2.111:0/54908547 shutdown_connections 2021-04-03T03:10:52.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.219+0000 7efe2ee44700 1 -- 172.21.2.111:0/54908547 wait complete. 2021-04-03T03:10:52.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 Processor -- start 2021-04-03T03:10:52.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 -- start start 2021-04-03T03:10:52.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efe280add50 0x7efe2815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efe280b17b0 0x7efe2815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe280a6f20 con 0x7efe280add50 2021-04-03T03:10:52.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe280aa980 con 0x7efe280b17b0 2021-04-03T03:10:52.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe280aac20 con 0x7efe280603f0 2021-04-03T03:10:52.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efe280add50 0x7efe2815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efe280add50 0x7efe2815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51802/0 (socket says 172.21.2.111:51802) 2021-04-03T03:10:52.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 -- 172.21.2.111:0/497507689 learned_addr learned my addr 172.21.2.111:0/497507689 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2de42700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 -- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 msgr2=0x7efe281479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2e643700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efe280b17b0 0x7efe2815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 -- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efe280b17b0 msgr2=0x7efe2815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efe280b17b0 0x7efe2815ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 -- 172.21.2.111:0/497507689 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2815a290 con 0x7efe280add50 2021-04-03T03:10:52.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2de42700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2e643700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efe280b17b0 0x7efe2815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:10:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2d641700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efe280add50 0x7efe2815a730 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7efe280b5600 tx=0x7efe1c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe1affd700 1 -- 172.21.2.111:0/497507689 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe1c007320 con 0x7efe280add50 2021-04-03T03:10:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe1affd700 1 -- 172.21.2.111:0/497507689 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c005530 con 0x7efe280add50 2021-04-03T03:10:52.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2815b110 con 0x7efe280add50 2021-04-03T03:10:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe1affd700 1 -- 172.21.2.111:0/497507689 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe1c005890 con 0x7efe280add50 2021-04-03T03:10:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe280b0a20 con 0x7efe280add50 2021-04-03T03:10:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe1affd700 1 -- 172.21.2.111:0/497507689 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7efe1c010030 con 0x7efe280add50 2021-04-03T03:10:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe1affd700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efe08056f20 0x7efe080593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe1affd700 1 -- 172.21.2.111:0/497507689 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7efe1c06ff30 con 0x7efe280add50 2021-04-03T03:10:52.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2de42700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efe08056f20 0x7efe080593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.223+0000 7efe2de42700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efe08056f20 0x7efe080593e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7efe24017000 tx=0x7efe24015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 Processor -- start 2021-04-03T03:10:52.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- start start 2021-04-03T03:10:52.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 0x2041860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 0x1f26930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x200a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f54f50 con 0x1e2a800 2021-04-03T03:10:52.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f55090 con 0x1fc2390 2021-04-03T03:10:52.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f564c0 con 0x1eaae90 2021-04-03T03:10:52.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x200a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x200a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51814/0 (socket says 172.21.2.111:51814) 2021-04-03T03:10:52.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/2330408590 learned_addr learned my addr 172.21.2.111:0/2330408590 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/2330408590 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 msgr2=0x1f26930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- 172.21.2.111:0/2330408590 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 0x1f26930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/2330408590 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 msgr2=0x2041860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- 172.21.2.111:0/2330408590 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 0x2041860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/2330408590 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f45c70 con 0x1e2a800 2021-04-03T03:10:52.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- 172.21.2.111:0/2330408590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x200a5a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7efe2000b000 tx=0x7efe20008040).ready entity=mon.0 client_cookie=9c8bb9e4408615ac server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2ee44700 1 -- 172.21.2.111:0/2330408590 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe20018070 con 0x1e2a800 2021-04-03T03:10:52.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2ee44700 1 -- 172.21.2.111:0/2330408590 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe20014600 con 0x1e2a800 2021-04-03T03:10:52.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2ee44700 1 -- 172.21.2.111:0/2330408590 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe20010ee0 con 0x1e2a800 2021-04-03T03:10:52.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- 172.21.2.111:0/2330408590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 msgr2=0x200a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/2330408590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x200a5a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7efe2000b000 tx=0x7efe20008040).stop 2021-04-03T03:10:52.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- 172.21.2.111:0/2330408590 shutdown_connections 2021-04-03T03:10:52.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/2330408590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x200a5a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/2330408590 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 0x2041860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/2330408590 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 0x1f26930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- 172.21.2.111:0/2330408590 >> 172.21.2.111:0/2330408590 conn(0x1ff1bf0 msgr2=0x20085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:52.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- 172.21.2.111:0/2330408590 shutdown_connections 2021-04-03T03:10:52.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- 172.21.2.111:0/2330408590 wait complete. 2021-04-03T03:10:52.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- start start 2021-04-03T03:10:52.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 Processor -- start 2021-04-03T03:10:52.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1e2a800 0x2014b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 0x20500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1fc2390 0x2042de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f37410 con 0x1fc2390 2021-04-03T03:10:52.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x2010890 con 0x1e2a800 2021-04-03T03:10:52.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x204f970 con 0x1eaae90 2021-04-03T03:10:52.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe3e8c2740 10 client.0 ll_register_callbacks cb 0x1ff1b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:10:52.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1fc2390 0x2042de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1fc2390 0x2042de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51820/0 (socket says 172.21.2.111:51820) 2021-04-03T03:10:52.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/4207760470 learned_addr learned my addr 172.21.2.111:0/4207760470 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/4207760470 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 msgr2=0x20500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 0x20500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/4207760470 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1e2a800 msgr2=0x2014b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1e2a800 0x2014b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 -- 172.21.2.111:0/4207760470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1f4e4a0 con 0x1fc2390 2021-04-03T03:10:52.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe2e643700 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1fc2390 0x2042de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7efe2000b000 tx=0x7efe2001a000).ready entity=mon.0 client_cookie=e72fc0034312c924 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.227+0000 7efe04ff9700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:10:52.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 1 -- 172.21.2.111:0/4207760470 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2001f900 con 0x1fc2390 2021-04-03T03:10:52.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 1 -- 172.21.2.111:0/4207760470 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7efe2001fa60 con 0x1fc2390 2021-04-03T03:10:52.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e4f8b0 con 0x1fc2390 2021-04-03T03:10:52.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 1 -- 172.21.2.111:0/4207760470 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe2001fc20 con 0x1fc2390 2021-04-03T03:10:52.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 10 client.5430 fetch_fsmap learned FSMap version 16 2021-04-03T03:10:52.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x1dda770 con 0x1fc2390 2021-04-03T03:10:52.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 1 -- 172.21.2.111:0/4207760470 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e4f8b0 con 0x1fc2390 2021-04-03T03:10:52.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 1 -- 172.21.2.111:0/4207760470 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 16) v1 ==== 1247+0+0 (secure 0 0 0) 0x7efe20026070 con 0x1fc2390 2021-04-03T03:10:52.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 10 client.5430 fetch_fsmap finished waiting for FSMap version 16 2021-04-03T03:10:52.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 10 client.5430 resolve_mds: resolved ID 'a' to GID 5241 2021-04-03T03:10:52.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 20 client.5430 populate_metadata read hostname 'gibba011' 2021-04-03T03:10:52.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x1dbcc00 0x200f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 4 client.5430 mds_command: new command op to 5241 tid=0[{"prefix": "get_command_descriptions"}] 2021-04-03T03:10:52.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 --> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1dbc150 con 0x1dbcc00 2021-04-03T03:10:52.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe2de42700 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x1dbcc00 0x200f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 10 client.5430 ms_handle_connect on v2:172.21.2.111:6834/798702406 2021-04-03T03:10:52.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe2de42700 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x1dbcc00 0x200f060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bd4f8132681aff79 server_cookie=a25ae455f12074f0 in_seq=0 out_seq=0 2021-04-03T03:10:52.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 1 -- 172.21.2.111:0/4207760470 <== mds.0 v2:172.21.2.111:6834/798702406 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1dbc150 con 0x1dbcc00 2021-04-03T03:10:52.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 10 client.5430 handle_command_reply: tid=0 2021-04-03T03:10:52.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 client.5430 shutdown 2021-04-03T03:10:52.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1fc2390 msgr2=0x2042de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1fc2390 0x2042de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7efe2000b000 tx=0x7efe2001a000).stop 2021-04-03T03:10:52.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 shutdown_connections 2021-04-03T03:10:52.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x1dbcc00 0x200f060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1fc2390 0x2042de0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1e2a800 0x2014b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/4207760470 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1eaae90 0x20500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe04ff9700 0 client.5430 ms_handle_reset on v2:172.21.2.111:6834/798702406 2021-04-03T03:10:52.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 >> 172.21.2.111:0/4207760470 conn(0x1ff1bf0 msgr2=0x20085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:52.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 shutdown_connections 2021-04-03T03:10:52.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 1 -- 172.21.2.111:0/4207760470 wait complete. 2021-04-03T03:10:52.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.231+0000 7efe3e8c2740 20 client.5430 trim_cache size 0 max 16384 2021-04-03T03:10:52.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 Processor -- start 2021-04-03T03:10:52.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- start start 2021-04-03T03:10:52.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1e2a800 0x200a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1eaae90 0x2014730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 0x20124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x2013320 con 0x1eaae90 2021-04-03T03:10:52.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x2043780 con 0x1fc2390 2021-04-03T03:10:52.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x20500b0 con 0x1e2a800 2021-04-03T03:10:52.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1eaae90 0x2014730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1eaae90 0x2014730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51828/0 (socket says 172.21.2.111:51828) 2021-04-03T03:10:52.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 -- 172.21.2.111:0/1212598301 learned_addr learned my addr 172.21.2.111:0/1212598301 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 -- 172.21.2.111:0/1212598301 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1e2a800 msgr2=0x200a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 --2- 172.21.2.111:0/1212598301 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1e2a800 0x200a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 -- 172.21.2.111:0/1212598301 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 msgr2=0x20124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 --2- 172.21.2.111:0/1212598301 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 0x20124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 -- 172.21.2.111:0/1212598301 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28157700 con 0x1eaae90 2021-04-03T03:10:52.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2d641700 1 --2- 172.21.2.111:0/1212598301 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1eaae90 0x2014730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7efe1c06e490 tx=0x7efe1c011000).ready entity=mon.0 client_cookie=31967db4d7d4ab7c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe06ffd700 1 -- 172.21.2.111:0/1212598301 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe1c045580 con 0x1eaae90 2021-04-03T03:10:52.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe06ffd700 1 -- 172.21.2.111:0/1212598301 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c0456e0 con 0x1eaae90 2021-04-03T03:10:52.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe06ffd700 1 -- 172.21.2.111:0/1212598301 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe1c048550 con 0x1eaae90 2021-04-03T03:10:52.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- 172.21.2.111:0/1212598301 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1eaae90 msgr2=0x2014730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:10:52.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/1212598301 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1eaae90 0x2014730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7efe1c06e490 tx=0x7efe1c011000).stop 2021-04-03T03:10:52.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- 172.21.2.111:0/1212598301 shutdown_connections 2021-04-03T03:10:52.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/1212598301 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1fc2390 0x20124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/1212598301 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1e2a800 0x200a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/1212598301 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1eaae90 0x2014730 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:10:52.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- 172.21.2.111:0/1212598301 >> 172.21.2.111:0/1212598301 conn(0x1ff1bf0 msgr2=0x1f27b70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:10:52.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- 172.21.2.111:0/1212598301 shutdown_connections 2021-04-03T03:10:52.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- 172.21.2.111:0/1212598301 wait complete. 2021-04-03T03:10:52.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- start start 2021-04-03T03:10:52.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 Processor -- start 2021-04-03T03:10:52.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x20a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1eaae90 0x20a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1fc2390 0x20a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x2044d90 con 0x1e2a800 2021-04-03T03:10:52.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x2045030 con 0x1eaae90 2021-04-03T03:10:52.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x204f300 con 0x1fc2390 2021-04-03T03:10:52.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 10 client.0 ll_register_callbacks cb 0x2013f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:10:52.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x20a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x20a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51830/0 (socket says 172.21.2.111:51830) 2021-04-03T03:10:52.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 -- 172.21.2.111:0/782117118 learned_addr learned my addr 172.21.2.111:0/782117118 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:10:52.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 -- 172.21.2.111:0/782117118 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1fc2390 msgr2=0x20a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1fc2390 0x20a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 -- 172.21.2.111:0/782117118 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1eaae90 msgr2=0x20a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:10:52.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1eaae90 0x20a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:10:52.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 -- 172.21.2.111:0/782117118 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1de5e50 con 0x1e2a800 2021-04-03T03:10:52.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe2de42700 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x20a4e20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7efe2402d000 tx=0x7efe24013000).ready entity=mon.0 client_cookie=c5b1c8629ef1af3f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:10:52.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe05ffb700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:10:52.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe05ffb700 1 -- 172.21.2.111:0/782117118 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe24030030 con 0x1e2a800 2021-04-03T03:10:52.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe05ffb700 1 -- 172.21.2.111:0/782117118 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7efe24031070 con 0x1e2a800 2021-04-03T03:10:52.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e6e710 con 0x1e2a800 2021-04-03T03:10:52.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe05ffb700 1 -- 172.21.2.111:0/782117118 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe24009e60 con 0x1e2a800 2021-04-03T03:10:52.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 10 client.5436 fetch_fsmap learned FSMap version 16 2021-04-03T03:10:52.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x20419d0 con 0x1e2a800 2021-04-03T03:10:52.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe05ffb700 1 -- 172.21.2.111:0/782117118 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e6e710 con 0x1e2a800 2021-04-03T03:10:52.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe05ffb700 1 -- 172.21.2.111:0/782117118 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 16) v1 ==== 1247+0+0 (secure 0 0 0) 0x7efe2401a030 con 0x1e2a800 2021-04-03T03:10:52.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 10 client.5436 fetch_fsmap finished waiting for FSMap version 16 2021-04-03T03:10:52.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 10 client.5436 resolve_mds: resolved ID 'a' to GID 5241 2021-04-03T03:10:52.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 20 client.5436 populate_metadata read hostname 'gibba011' 2021-04-03T03:10:52.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.239+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x20a5800 0x20a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:10:52.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.243+0000 7efe3e8c2740 4 client.5436 mds_command: new command op to 5241 tid=0[{"prefix": "cache drop"}] 2021-04-03T03:10:52.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.243+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 --> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1e45420 con 0x20a5800 2021-04-03T03:10:52.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.243+0000 7efe2d641700 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x20a5800 0x20a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:10:52.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.243+0000 7efe05ffb700 10 client.5436 ms_handle_connect on v2:172.21.2.111:6834/798702406 2021-04-03T03:10:52.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:10:52.243+0000 7efe2d641700 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x20a5800 0x20a0f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2798ddabab9f81fd server_cookie=fd04c8c996253e10 in_seq=0 out_seq=0 2021-04-03T03:10:58.651 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:58.654 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:58.657 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:58.660 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:10:58.664 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:03.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe05ffb700 1 -- 172.21.2.111:0/782117118 <== mds.0 v2:172.21.2.111:6834/798702406 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x1e45420 con 0x20a5800 2021-04-03T03:11:03.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe05ffb700 10 client.5436 handle_command_reply: tid=0 2021-04-03T03:11:03.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 client.5436 shutdown 2021-04-03T03:11:03.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 msgr2=0x20a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x20a4e20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7efe2402d000 tx=0x7efe24013000).stop 2021-04-03T03:11:03.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 shutdown_connections 2021-04-03T03:11:03.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x1fc2390 0x20a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:6834/798702406,v1:172.21.2.111:6835/798702406] conn(0x20a5800 0x20a0f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x1e2a800 0x20a4e20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 --2- 172.21.2.111:0/782117118 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x1eaae90 0x20a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe05ffb700 0 client.5436 ms_handle_reset on v2:172.21.2.111:6834/798702406 2021-04-03T03:11:03.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 >> 172.21.2.111:0/782117118 conn(0x1ff1bf0 msgr2=0x2014010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:03.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 shutdown_connections 2021-04-03T03:11:03.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 1 -- 172.21.2.111:0/782117118 wait complete. 2021-04-03T03:11:03.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.323+0000 7efe3e8c2740 20 client.5436 trim_cache size 0 max 16384 2021-04-03T03:11:03.341 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:11:03.341 INFO:teuthology.orchestra.run.gibba011.stdout: "client_recall": { 2021-04-03T03:11:03.341 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": 0, 2021-04-03T03:11:03.342 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "(0) Success", 2021-04-03T03:11:03.342 INFO:teuthology.orchestra.run.gibba011.stdout: "recalled": 220 2021-04-03T03:11:03.342 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:03.342 INFO:teuthology.orchestra.run.gibba011.stdout: "flush_journal": { 2021-04-03T03:11:03.343 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": 0, 2021-04-03T03:11:03.343 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "" 2021-04-03T03:11:03.343 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:03.343 INFO:teuthology.orchestra.run.gibba011.stdout: "trim_cache": { 2021-04-03T03:11:03.344 INFO:teuthology.orchestra.run.gibba011.stdout: "trimmed": 0 2021-04-03T03:11:03.344 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:03.344 INFO:teuthology.orchestra.run.gibba011.stdout: "cache": { 2021-04-03T03:11:03.344 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": { 2021-04-03T03:11:03.345 INFO:teuthology.orchestra.run.gibba011.stdout: "items": 19928, 2021-04-03T03:11:03.345 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes": 3095712 2021-04-03T03:11:03.345 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:03.345 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:03.345 INFO:teuthology.orchestra.run.gibba011.stdout: "duration": 11.082850243999999 2021-04-03T03:11:03.346 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:11:03.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efe08056f20 msgr2=0x7efe080593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efe08056f20 0x7efe080593e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7efe24017000 tx=0x7efe24015000).stop 2021-04-03T03:11:03.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efe280add50 msgr2=0x7efe2815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efe280add50 0x7efe2815a730 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7efe280b5600 tx=0x7efe1c007bb0).stop 2021-04-03T03:11:03.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 shutdown_connections 2021-04-03T03:11:03.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efe08056f20 0x7efe080593e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efe280b17b0 0x7efe2815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efe280603f0 0x7efe281479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 --2- 172.21.2.111:0/497507689 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efe280add50 0x7efe2815a730 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 >> 172.21.2.111:0/497507689 conn(0x7efe280b61c0 msgr2=0x7efe281441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:03.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 shutdown_connections 2021-04-03T03:11:03.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.327+0000 7efe2ee44700 1 -- 172.21.2.111:0/497507689 wait complete. 2021-04-03T03:11:03.350 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-04-03T03:11:03.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:03.351 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:03.352 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:03.361 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-04-03T03:11:03.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:03.362 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-04-03T03:11:03.363 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:11:03.363 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:11:03.364 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:03.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:03.453 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:03.453 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:03.458 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:11:03.458 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:11:03.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:11:03.459 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:03.508 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:11:03.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 Processor -- start 2021-04-03T03:11:03.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- start start 2021-04-03T03:11:03.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb66414b700 0x7fb6640b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb6640b0c90 0x7fb664147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6640af400 con 0x7fb6640603f0 2021-04-03T03:11:03.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb664060860 con 0x7fb6640b0c90 2021-04-03T03:11:03.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6640af6a0 con 0x7fb66414b700 2021-04-03T03:11:03.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb6640b0c90 0x7fb664147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb6640b0c90 0x7fb664147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51844/0 (socket says 172.21.2.111:51844) 2021-04-03T03:11:03.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 -- 172.21.2.111:0/2446473757 learned_addr learned my addr 172.21.2.111:0/2446473757 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:03.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 -- 172.21.2.111:0/2446473757 >> v2:172.21.2.115:3300/0 conn(0x7fb66414b700 msgr2=0x7fb6640b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:03.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 --2- 172.21.2.111:0/2446473757 >> v2:172.21.2.115:3300/0 conn(0x7fb66414b700 0x7fb6640b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 -- 172.21.2.111:0/2446473757 >> v1:172.21.2.115:6789/0 conn(0x7fb6640603f0 legacy=0x7fb6640b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:03.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 -- 172.21.2.111:0/2446473757 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6640afb40 con 0x7fb6640b0c90 2021-04-03T03:11:03.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb662d9d700 1 --2- 172.21.2.111:0/2446473757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb664147680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fb66406a3b0 tx=0x7fb65c009270).ready entity=mon.0 client_cookie=4eb30051d8e96507 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66159a700 1 -- 172.21.2.111:0/2446473757 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb65c00bbc0 con 0x7fb6640b0c90 2021-04-03T03:11:03.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66159a700 1 -- 172.21.2.111:0/2446473757 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c00bd20 con 0x7fb6640b0c90 2021-04-03T03:11:03.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66159a700 1 -- 172.21.2.111:0/2446473757 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb65c00be90 con 0x7fb6640b0c90 2021-04-03T03:11:03.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- 172.21.2.111:0/2446473757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 msgr2=0x7fb664147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 --2- 172.21.2.111:0/2446473757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb664147680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fb66406a3b0 tx=0x7fb65c009270).stop 2021-04-03T03:11:03.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- 172.21.2.111:0/2446473757 shutdown_connections 2021-04-03T03:11:03.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 --2- 172.21.2.111:0/2446473757 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb664147680 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 --2- 172.21.2.111:0/2446473757 >> v2:172.21.2.115:3300/0 conn(0x7fb66414b700 0x7fb6640b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- 172.21.2.111:0/2446473757 >> 172.21.2.111:0/2446473757 conn(0x7fb6640b61c0 msgr2=0x7fb66405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:03.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- 172.21.2.111:0/2446473757 shutdown_connections 2021-04-03T03:11:03.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.611+0000 7fb66359e700 1 -- 172.21.2.111:0/2446473757 wait complete. 2021-04-03T03:11:03.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 Processor -- start 2021-04-03T03:11:03.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 -- start start 2021-04-03T03:11:03.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6640603f0 0x7fb664063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb6640a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb66414b700 0x7fb6640a8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664063cf0 con 0x7fb6640b0c90 2021-04-03T03:11:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664063f50 con 0x7fb6640603f0 2021-04-03T03:11:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641571b0 con 0x7fb66414b700 2021-04-03T03:11:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb6640a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb6640a7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51846/0 (socket says 172.21.2.111:51846) 2021-04-03T03:11:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 -- 172.21.2.111:0/713854833 learned_addr learned my addr 172.21.2.111:0/713854833 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 -- 172.21.2.111:0/713854833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb66414b700 msgr2=0x7fb6640a8420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:11:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb66414b700 0x7fb6640a8420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 -- 172.21.2.111:0/713854833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6640603f0 msgr2=0x7fb664063800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:11:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6640603f0 0x7fb664063800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 -- 172.21.2.111:0/713854833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6641576e0 con 0x7fb6640b0c90 2021-04-03T03:11:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb661d9b700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb6640a7f30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fb64c002f30 tx=0x7fb64c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb65b7fe700 1 -- 172.21.2.111:0/713854833 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb64c00d570 con 0x7fb6640b0c90 2021-04-03T03:11:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb65b7fe700 1 -- 172.21.2.111:0/713854833 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c00d6d0 con 0x7fb6640b0c90 2021-04-03T03:11:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6640a9370 con 0x7fb6640b0c90 2021-04-03T03:11:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb65b7fe700 1 -- 172.21.2.111:0/713854833 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb64c00da30 con 0x7fb6640b0c90 2021-04-03T03:11:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6640a8aa0 con 0x7fb6640b0c90 2021-04-03T03:11:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb664157370 con 0x7fb6640b0c90 2021-04-03T03:11:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb65b7fe700 1 -- 172.21.2.111:0/713854833 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb64c018030 con 0x7fb6640b0c90 2021-04-03T03:11:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb65b7fe700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb648056ea0 0x7fb648059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb65b7fe700 1 -- 172.21.2.111:0/713854833 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb64c06f600 con 0x7fb6640b0c90 2021-04-03T03:11:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66259c700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb648056ea0 0x7fb648059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.615+0000 7fb66259c700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb648056ea0 0x7fb648059360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fb654002f50 tx=0x7fb654006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.619+0000 7fb65b7fe700 1 -- 172.21.2.111:0/713854833 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb64c041c50 con 0x7fb6640b0c90 2021-04-03T03:11:03.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb664057660 con 0x7fb6640b0c90 2021-04-03T03:11:03.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb65b7fe700 1 -- 172.21.2.111:0/713854833 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fb64c023fa0 con 0x7fb6640b0c90 2021-04-03T03:11:03.723 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:11:03.723 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:10:32.119441+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:10:29.772278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-04-03T03:10:30.083040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:11:03.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb648056ea0 msgr2=0x7fb648059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb648056ea0 0x7fb648059360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fb654002f50 tx=0x7fb654006910).stop 2021-04-03T03:11:03.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 msgr2=0x7fb6640a7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb6640a7f30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fb64c002f30 tx=0x7fb64c006dd0).stop 2021-04-03T03:11:03.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 shutdown_connections 2021-04-03T03:11:03.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb66414b700 0x7fb6640a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb648056ea0 0x7fb648059360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6640603f0 0x7fb664063800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 --2- 172.21.2.111:0/713854833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6640b0c90 0x7fb6640a7f30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 >> 172.21.2.111:0/713854833 conn(0x7fb6640b61c0 msgr2=0x7fb664146a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:03.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 shutdown_connections 2021-04-03T03:11:03.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.711+0000 7fb66359e700 1 -- 172.21.2.111:0/713854833 wait complete. 2021-04-03T03:11:03.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:10:29.772278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:10:30.083040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:11:03.731 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:11:03.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 Processor -- start 2021-04-03T03:11:03.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 -- start start 2021-04-03T03:11:03.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40603f0 0x7f75d41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f75d40b17b0 0x7f75d4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d40607d0 con 0x7f75d40add50 2021-04-03T03:11:03.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d41451c0 con 0x7f75d40b17b0 2021-04-03T03:11:03.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d414b090 con 0x7f75d40603f0 2021-04-03T03:11:03.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d2de5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40603f0 0x7f75d41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d25e4700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f75d40add50 0x7f75d4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:36952/0 (socket says 172.21.2.111:36952) 2021-04-03T03:11:03.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d35e6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f75d40b17b0 0x7f75d4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d25e4700 1 -- 172.21.2.111:0/2362272096 learned_addr learned my addr 172.21.2.111:0/2362272096 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:03.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d35e6700 1 -- 172.21.2.111:0/2362272096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40603f0 msgr2=0x7f75d41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:03.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d35e6700 1 --2- 172.21.2.111:0/2362272096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40603f0 0x7f75d41447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d35e6700 1 -- 172.21.2.111:0/2362272096 >> v1:172.21.2.115:6789/0 conn(0x7f75d40add50 legacy=0x7f75d4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d35e6700 1 -- 172.21.2.111:0/2362272096 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d414b330 con 0x7f75d40b17b0 2021-04-03T03:11:03.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d35e6700 1 --2- 172.21.2.111:0/2362272096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40b17b0 0x7f75d4144cd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f75cc0032b0 tx=0x7f75cc0091d0).ready entity=mon.1 client_cookie=162173a483d8f367 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d1de3700 1 -- 172.21.2.111:0/2362272096 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75cc00b9a0 con 0x7f75d40b17b0 2021-04-03T03:11:03.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d1de3700 1 -- 172.21.2.111:0/2362272096 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc00bb00 con 0x7f75d40b17b0 2021-04-03T03:11:03.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d1de3700 1 -- 172.21.2.111:0/2362272096 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75cc00bc70 con 0x7f75d40b17b0 2021-04-03T03:11:03.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 -- 172.21.2.111:0/2362272096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40b17b0 msgr2=0x7f75d4144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.779+0000 7f75d3de7700 1 --2- 172.21.2.111:0/2362272096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40b17b0 0x7f75d4144cd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f75cc0032b0 tx=0x7f75cc0091d0).stop 2021-04-03T03:11:03.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- 172.21.2.111:0/2362272096 shutdown_connections 2021-04-03T03:11:03.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 --2- 172.21.2.111:0/2362272096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40b17b0 0x7f75d4144cd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 --2- 172.21.2.111:0/2362272096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40603f0 0x7f75d41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- 172.21.2.111:0/2362272096 >> 172.21.2.111:0/2362272096 conn(0x7f75d40b61c0 msgr2=0x7f75d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:03.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- 172.21.2.111:0/2362272096 shutdown_connections 2021-04-03T03:11:03.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- 172.21.2.111:0/2362272096 wait complete. 2021-04-03T03:11:03.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 Processor -- start 2021-04-03T03:11:03.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- start start 2021-04-03T03:11:03.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75d40603f0 0x7f75d40a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40add50 0x7f75d40ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40b17b0 0x7f75d40ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40abf90 con 0x7f75d40603f0 2021-04-03T03:11:03.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d4064980 con 0x7f75d40add50 2021-04-03T03:11:03.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d4064c20 con 0x7f75d40b17b0 2021-04-03T03:11:03.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75d40603f0 0x7f75d40a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75d40603f0 0x7f75d40a9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51862/0 (socket says 172.21.2.111:51862) 2021-04-03T03:11:03.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 -- 172.21.2.111:0/1501533075 learned_addr learned my addr 172.21.2.111:0/1501533075 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:03.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d35e6700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40b17b0 0x7f75d40ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 -- 172.21.2.111:0/1501533075 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40b17b0 msgr2=0x7f75d40ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40b17b0 0x7f75d40ab9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 -- 172.21.2.111:0/1501533075 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40add50 msgr2=0x7f75d40ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40add50 0x7f75d40ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 -- 172.21.2.111:0/1501533075 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d40aa1f0 con 0x7f75d40603f0 2021-04-03T03:11:03.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d2de5700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75d40603f0 0x7f75d40a9b80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f75c400c000 tx=0x7f75c4006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75cbfff700 1 -- 172.21.2.111:0/1501533075 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c400cae0 con 0x7f75d40603f0 2021-04-03T03:11:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75cbfff700 1 -- 172.21.2.111:0/1501533075 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c400cc40 con 0x7f75d40603f0 2021-04-03T03:11:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d40aad00 con 0x7f75d40603f0 2021-04-03T03:11:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75cbfff700 1 -- 172.21.2.111:0/1501533075 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c40123f0 con 0x7f75d40603f0 2021-04-03T03:11:03.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d415a0a0 con 0x7f75d40603f0 2021-04-03T03:11:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d4057660 con 0x7f75d40603f0 2021-04-03T03:11:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75cbfff700 1 -- 172.21.2.111:0/1501533075 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f75c4012550 con 0x7f75d40603f0 2021-04-03T03:11:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75cbfff700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f75b8056ec0 0x7f75b8059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75cbfff700 1 -- 172.21.2.111:0/1501533075 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f75c40706f0 con 0x7f75d40603f0 2021-04-03T03:11:03.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d25e4700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f75b8056ec0 0x7f75b8059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.783+0000 7f75d25e4700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f75b8056ec0 0x7f75b8059380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f75d40b5600 tx=0x7f75bc0077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.787+0000 7f75cbfff700 1 -- 172.21.2.111:0/1501533075 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f75c4045a80 con 0x7f75d40603f0 2021-04-03T03:11:03.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f75d4148850 con 0x7f75d40603f0 2021-04-03T03:11:03.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75cbfff700 1 -- 172.21.2.111:0/1501533075 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v43) v1 ==== 110+0+10 (secure 0 0 0) 0x7f75c40490a0 con 0x7f75d40603f0 2021-04-03T03:11:03.891 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:11:03.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f75b8056ec0 msgr2=0x7f75b8059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f75b8056ec0 0x7f75b8059380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f75d40b5600 tx=0x7f75bc0077e0).stop 2021-04-03T03:11:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75d40603f0 msgr2=0x7f75d40a9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75d40603f0 0x7f75d40a9b80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f75c400c000 tx=0x7f75c4006dd0).stop 2021-04-03T03:11:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 shutdown_connections 2021-04-03T03:11:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f75d40b17b0 0x7f75d40ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75d40603f0 0x7f75d40a9b80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f75b8056ec0 0x7f75b8059380 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 --2- 172.21.2.111:0/1501533075 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75d40add50 0x7f75d40ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 >> 172.21.2.111:0/1501533075 conn(0x7f75d40b61c0 msgr2=0x7f75d40b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 shutdown_connections 2021-04-03T03:11:03.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.879+0000 7f75d3de7700 1 -- 172.21.2.111:0/1501533075 wait complete. 2021-04-03T03:11:03.899 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:11:03.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.943+0000 7f914759e700 1 Processor -- start 2021-04-03T03:11:03.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.943+0000 7f914759e700 1 -- start start 2021-04-03T03:11:03.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9148144ef0 0x7f9148147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91480607d0 con 0x7f91480603f0 2021-04-03T03:11:03.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914814b090 con 0x7f9148144ef0 2021-04-03T03:11:03.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914814b330 con 0x7f91480add50 2021-04-03T03:11:03.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9146d9d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9148144ef0 0x7f9148147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:35218/0 (socket says 172.21.2.111:35218) 2021-04-03T03:11:03.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 -- 172.21.2.111:0/3030403030 learned_addr learned my addr 172.21.2.111:0/3030403030 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:03.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914659c700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f91480603f0 0x7f91481447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:36964/0 (socket says 172.21.2.111:36964) 2021-04-03T03:11:03.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 -- 172.21.2.111:0/3030403030 >> v2:172.21.2.115:3300/0 conn(0x7f9148144ef0 msgr2=0x7f9148147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:03.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 --2- 172.21.2.111:0/3030403030 >> v2:172.21.2.115:3300/0 conn(0x7f9148144ef0 0x7f9148147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 -- 172.21.2.111:0/3030403030 >> v1:172.21.2.115:6789/0 conn(0x7f91480603f0 legacy=0x7f91481447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 -- 172.21.2.111:0/3030403030 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914814b5d0 con 0x7f91480add50 2021-04-03T03:11:03.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9146d9d700 1 --2- 172.21.2.111:0/3030403030 >> v2:172.21.2.115:3300/0 conn(0x7f9148144ef0 0x7f9148147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:11:03.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f9145d9b700 1 --2- 172.21.2.111:0/3030403030 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f91480add50 0x7f9148147740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9130002f30 tx=0x7f9130006e90).ready entity=mon.2 client_cookie=8bb3ebf2f3c1e816 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914559a700 1 -- 172.21.2.111:0/3030403030 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9130009850 con 0x7f91480add50 2021-04-03T03:11:03.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914559a700 1 -- 172.21.2.111:0/3030403030 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300099b0 con 0x7f91480add50 2021-04-03T03:11:03.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914559a700 1 -- 172.21.2.111:0/3030403030 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9130009d10 con 0x7f91480add50 2021-04-03T03:11:03.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- 172.21.2.111:0/3030403030 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f91480add50 msgr2=0x7f9148147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 --2- 172.21.2.111:0/3030403030 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f91480add50 0x7f9148147740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9130002f30 tx=0x7f9130006e90).stop 2021-04-03T03:11:03.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- 172.21.2.111:0/3030403030 shutdown_connections 2021-04-03T03:11:03.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 --2- 172.21.2.111:0/3030403030 >> v2:172.21.2.115:3300/0 conn(0x7f9148144ef0 0x7f9148147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 --2- 172.21.2.111:0/3030403030 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- 172.21.2.111:0/3030403030 >> 172.21.2.111:0/3030403030 conn(0x7f91480b61c0 msgr2=0x7f914805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:03.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- 172.21.2.111:0/3030403030 shutdown_connections 2021-04-03T03:11:03.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.947+0000 7f914759e700 1 -- 172.21.2.111:0/3030403030 wait complete. 2021-04-03T03:11:03.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 Processor -- start 2021-04-03T03:11:03.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 -- start start 2021-04-03T03:11:03.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f91480603f0 0x7f91480aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9148144ef0 0x7f914815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480a96c0 con 0x7f91480add50 2021-04-03T03:11:03.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480ab150 con 0x7f91480603f0 2021-04-03T03:11:03.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914815a830 con 0x7f9148144ef0 2021-04-03T03:11:03.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51876/0 (socket says 172.21.2.111:51876) 2021-04-03T03:11:03.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 -- 172.21.2.111:0/312158478 learned_addr learned my addr 172.21.2.111:0/312158478 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:03.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914659c700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f91480603f0 0x7f91480aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9146d9d700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9148144ef0 0x7f914815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 -- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9148144ef0 msgr2=0x7f914815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9148144ef0 0x7f914815a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 -- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f91480603f0 msgr2=0x7f91480aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:03.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f91480603f0 0x7f91480aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:03.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 -- 172.21.2.111:0/312158478 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91480a9420 con 0x7f91480add50 2021-04-03T03:11:03.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914659c700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f91480603f0 0x7f91480aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:11:03.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f9145d9b700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f91480add50 0x7f9148147740 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f9130002f30 tx=0x7f9130015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f913f7fe700 1 -- 172.21.2.111:0/312158478 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9130011c60 con 0x7f91480add50 2021-04-03T03:11:03.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f913f7fe700 1 -- 172.21.2.111:0/312158478 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9130011dc0 con 0x7f91480add50 2021-04-03T03:11:03.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9148063170 con 0x7f91480add50 2021-04-03T03:11:03.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91480a90a0 con 0x7f91480add50 2021-04-03T03:11:03.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f913f7fe700 1 -- 172.21.2.111:0/312158478 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f913001e650 con 0x7f91480add50 2021-04-03T03:11:03.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9148063330 con 0x7f91480add50 2021-04-03T03:11:03.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f913f7fe700 1 -- 172.21.2.111:0/312158478 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9130025030 con 0x7f91480add50 2021-04-03T03:11:03.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f913f7fe700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f912c056f70 0x7f912c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:03.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f913f7fe700 1 -- 172.21.2.111:0/312158478 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f91300714f0 con 0x7f91480add50 2021-04-03T03:11:03.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914659c700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f912c056f70 0x7f912c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:03.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.951+0000 7f914659c700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f912c056f70 0x7f912c059430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f9138008000 tx=0x7f9138009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:03.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:03.955+0000 7f913f7fe700 1 -- 172.21.2.111:0/312158478 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f91300467d0 con 0x7f91480add50 2021-04-03T03:11:04.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9148057660 con 0x7f91480add50 2021-04-03T03:11:04.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f913f7fe700 1 -- 172.21.2.111:0/312158478 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f91300230a0 con 0x7f91480add50 2021-04-03T03:11:04.059 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:11:04.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f912c056f70 msgr2=0x7f912c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f912c056f70 0x7f912c059430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f9138008000 tx=0x7f9138009040).stop 2021-04-03T03:11:04.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f91480add50 msgr2=0x7f9148147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f91480add50 0x7f9148147740 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f9130002f30 tx=0x7f9130015000).stop 2021-04-03T03:11:04.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 shutdown_connections 2021-04-03T03:11:04.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9148144ef0 0x7f914815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f912c056f70 0x7f912c059430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f91480603f0 0x7f91480aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 --2- 172.21.2.111:0/312158478 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f91480add50 0x7f9148147740 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 >> 172.21.2.111:0/312158478 conn(0x7f91480b61c0 msgr2=0x7f9148142ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:04.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 shutdown_connections 2021-04-03T03:11:04.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.047+0000 7f914759e700 1 -- 172.21.2.111:0/312158478 wait complete. 2021-04-03T03:11:04.067 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:11:04.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6d8a8700 1 Processor -- start 2021-04-03T03:11:04.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6d8a8700 1 -- start start 2021-04-03T03:11:04.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6d8a8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7b68144ef0 0x7f7b68147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6d8a8700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b680607d0 con 0x7f7b680603f0 2021-04-03T03:11:04.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6d8a8700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6814b090 con 0x7f7b680add50 2021-04-03T03:11:04.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6d8a8700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6814b330 con 0x7f7b68144ef0 2021-04-03T03:11:04.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6c8a6700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7b680603f0 0x7f7b681447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52674/0 (socket says 172.21.2.111:52674) 2021-04-03T03:11:04.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6c8a6700 1 -- 172.21.2.111:0/2877173732 learned_addr learned my addr 172.21.2.111:0/2877173732 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:04.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b6d0a7700 1 --2- 172.21.2.111:0/2877173732 >> v2:172.21.2.111:3300/0 conn(0x7f7b68144ef0 0x7f7b68147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b677fe700 1 -- 172.21.2.111:0/2877173732 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241258525 0 0) 0x7f7b680607d0 con 0x7f7b680603f0 2021-04-03T03:11:04.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.111+0000 7f7b677fe700 1 -- 172.21.2.111:0/2877173732 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b48003240 con 0x7f7b680603f0 2021-04-03T03:11:04.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 -- 172.21.2.111:0/2877173732 >> v1:172.21.2.115:6789/0 conn(0x7f7b680add50 legacy=0x7f7b68147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:04.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 -- 172.21.2.111:0/2877173732 >> v1:172.21.2.111:6789/0 conn(0x7f7b680603f0 legacy=0x7f7b681447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 -- 172.21.2.111:0/2877173732 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6814b5d0 con 0x7f7b68144ef0 2021-04-03T03:11:04.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 --2- 172.21.2.111:0/2877173732 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68147e50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f7b680b5600 tx=0x7f7b600091d0).ready entity=mon.0 client_cookie=e7e32a2a46416e22 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b677fe700 1 -- 172.21.2.111:0/2877173732 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6000b960 con 0x7f7b68144ef0 2021-04-03T03:11:04.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b677fe700 1 -- 172.21.2.111:0/2877173732 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000bac0 con 0x7f7b68144ef0 2021-04-03T03:11:04.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b677fe700 1 -- 172.21.2.111:0/2877173732 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6000bc30 con 0x7f7b68144ef0 2021-04-03T03:11:04.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/2877173732 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 msgr2=0x7f7b68147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/2877173732 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68147e50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f7b680b5600 tx=0x7f7b600091d0).stop 2021-04-03T03:11:04.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/2877173732 shutdown_connections 2021-04-03T03:11:04.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/2877173732 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68147e50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/2877173732 >> 172.21.2.111:0/2877173732 conn(0x7f7b680b61c0 msgr2=0x7f7b6805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:04.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/2877173732 shutdown_connections 2021-04-03T03:11:04.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/2877173732 wait complete. 2021-04-03T03:11:04.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 Processor -- start 2021-04-03T03:11:04.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- start start 2021-04-03T03:11:04.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7b680603f0 0x7f7b680a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7b680add50 0x7f7b681428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68147dd0 con 0x7f7b68144ef0 2021-04-03T03:11:04.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68068010 con 0x7f7b680add50 2021-04-03T03:11:04.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68148000 con 0x7f7b680603f0 2021-04-03T03:11:04.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51890/0 (socket says 172.21.2.111:51890) 2021-04-03T03:11:04.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 -- 172.21.2.111:0/3102826872 learned_addr learned my addr 172.21.2.111:0/3102826872 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:04.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 -- 172.21.2.111:0/3102826872 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7b680603f0 msgr2=0x7f7b680a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7b680603f0 0x7f7b680a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 -- 172.21.2.111:0/3102826872 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7b680add50 msgr2=0x7f7b681428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7b680add50 0x7f7b681428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 -- 172.21.2.111:0/3102826872 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b68067d70 con 0x7f7b68144ef0 2021-04-03T03:11:04.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d0a7700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68142de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f7b680b5600 tx=0x7f7b60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b65ffb700 1 -- 172.21.2.111:0/3102826872 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b60020400 con 0x7f7b68144ef0 2021-04-03T03:11:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b65ffb700 1 -- 172.21.2.111:0/3102826872 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b60020560 con 0x7f7b68144ef0 2021-04-03T03:11:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b65ffb700 1 -- 172.21.2.111:0/3102826872 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b600208c0 con 0x7f7b68144ef0 2021-04-03T03:11:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b680aa8b0 con 0x7f7b68144ef0 2021-04-03T03:11:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.115+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b680a7730 con 0x7f7b68144ef0 2021-04-03T03:11:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.119+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b68057660 con 0x7f7b68144ef0 2021-04-03T03:11:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.119+0000 7f7b65ffb700 1 -- 172.21.2.111:0/3102826872 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7b60027030 con 0x7f7b68144ef0 2021-04-03T03:11:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.119+0000 7f7b65ffb700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7b4c056ef0 0x7f7b4c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.119+0000 7f7b65ffb700 1 -- 172.21.2.111:0/3102826872 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7b600712c0 con 0x7f7b68144ef0 2021-04-03T03:11:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.119+0000 7f7b6c8a6700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7b4c056ef0 0x7f7b4c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.119+0000 7f7b6c8a6700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7b4c056ef0 0x7f7b4c0593b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f7b580010d0 tx=0x7f7b580076a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.119+0000 7f7b65ffb700 1 -- 172.21.2.111:0/3102826872 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7b600465a0 con 0x7f7b68144ef0 2021-04-03T03:11:04.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.211+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b680b1b90 con 0x7f7b68144ef0 2021-04-03T03:11:04.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b65ffb700 1 -- 172.21.2.111:0/3102826872 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7b60049bc0 con 0x7f7b68144ef0 2021-04-03T03:11:04.226 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:11:04.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7b4c056ef0 msgr2=0x7f7b4c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7b4c056ef0 0x7f7b4c0593b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f7b580010d0 tx=0x7f7b580076a0).stop 2021-04-03T03:11:04.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 msgr2=0x7f7b68142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68142de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f7b680b5600 tx=0x7f7b60016000).stop 2021-04-03T03:11:04.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 shutdown_connections 2021-04-03T03:11:04.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7b68144ef0 0x7f7b68142de0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7b680603f0 0x7f7b680a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7b4c056ef0 0x7f7b4c0593b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 --2- 172.21.2.111:0/3102826872 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7b680add50 0x7f7b681428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 >> 172.21.2.111:0/3102826872 conn(0x7f7b680b61c0 msgr2=0x7f7b6805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:04.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 shutdown_connections 2021-04-03T03:11:04.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.215+0000 7f7b6d8a8700 1 -- 172.21.2.111:0/3102826872 wait complete. 2021-04-03T03:11:04.233 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:11:04.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 Processor -- start 2021-04-03T03:11:04.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- start start 2021-04-03T03:11:04.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7cf80603f0 0x7f7cf81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf80607d0 con 0x7f7cf80add50 2021-04-03T03:11:04.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf81451c0 con 0x7f7cf80603f0 2021-04-03T03:11:04.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf814b090 con 0x7f7cf80b17b0 2021-04-03T03:11:04.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfcdde700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7cf80603f0 0x7f7cf81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cf7fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7cf80add50 0x7f7cf8147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52688/0 (socket says 172.21.2.111:52688) 2021-04-03T03:11:04.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cf7fff700 1 -- 172.21.2.111:0/733664860 learned_addr learned my addr 172.21.2.111:0/733664860 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:04.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfcdde700 1 -- 172.21.2.111:0/733664860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 msgr2=0x7f7cf8144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:04.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfd5df700 1 --2- 172.21.2.111:0/733664860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfcdde700 1 --2- 172.21.2.111:0/733664860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf8144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfcdde700 1 -- 172.21.2.111:0/733664860 >> v1:172.21.2.111:6789/0 conn(0x7f7cf80add50 legacy=0x7f7cf8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfcdde700 1 -- 172.21.2.111:0/733664860 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf814b330 con 0x7f7cf80603f0 2021-04-03T03:11:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfd5df700 1 --2- 172.21.2.111:0/733664860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf8144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:11:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfcdde700 1 --2- 172.21.2.111:0/733664860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80603f0 0x7f7cf81447e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7ce8002f30 tx=0x7f7ce8006dd0).ready entity=mon.0 client_cookie=c5dd9f4e9a9893a4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cf77fe700 1 -- 172.21.2.111:0/733664860 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce80097f0 con 0x7f7cf80603f0 2021-04-03T03:11:04.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cf77fe700 1 -- 172.21.2.111:0/733664860 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce8009950 con 0x7f7cf80603f0 2021-04-03T03:11:04.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cf77fe700 1 -- 172.21.2.111:0/733664860 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce8009cb0 con 0x7f7cf80603f0 2021-04-03T03:11:04.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- 172.21.2.111:0/733664860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80603f0 msgr2=0x7f7cf81447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/733664860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80603f0 0x7f7cf81447e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7ce8002f30 tx=0x7f7ce8006dd0).stop 2021-04-03T03:11:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- 172.21.2.111:0/733664860 shutdown_connections 2021-04-03T03:11:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/733664860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf8144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/733664860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80603f0 0x7f7cf81447e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- 172.21.2.111:0/733664860 >> 172.21.2.111:0/733664860 conn(0x7f7cf80b61c0 msgr2=0x7f7cf805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- 172.21.2.111:0/733664860 shutdown_connections 2021-04-03T03:11:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.279+0000 7f7cfdde0700 1 -- 172.21.2.111:0/733664860 wait complete. 2021-04-03T03:11:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 Processor -- start 2021-04-03T03:11:04.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 -- start start 2021-04-03T03:11:04.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7cf80603f0 0x7f7cf8156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80add50 0x7f7cf8157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf80a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8068020 con 0x7f7cf80add50 2021-04-03T03:11:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80a9350 con 0x7f7cf80603f0 2021-04-03T03:11:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8157c40 con 0x7f7cf80b17b0 2021-04-03T03:11:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80add50 0x7f7cf8157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80add50 0x7f7cf8157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51902/0 (socket says 172.21.2.111:51902) 2021-04-03T03:11:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 -- 172.21.2.111:0/2396202332 learned_addr learned my addr 172.21.2.111:0/2396202332 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 -- 172.21.2.111:0/2396202332 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 msgr2=0x7f7cf80a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf80a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 -- 172.21.2.111:0/2396202332 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7cf80603f0 msgr2=0x7f7cf8156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7cf80603f0 0x7f7cf8156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 -- 172.21.2.111:0/2396202332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf81579a0 con 0x7f7cf80add50 2021-04-03T03:11:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf7fff700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80add50 0x7f7cf8157130 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f7cf80b5600 tx=0x7f7ce0007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf5ffb700 1 -- 172.21.2.111:0/2396202332 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce00072e0 con 0x7f7cf80add50 2021-04-03T03:11:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf5ffb700 1 -- 172.21.2.111:0/2396202332 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce0007440 con 0x7f7cf80add50 2021-04-03T03:11:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf5ffb700 1 -- 172.21.2.111:0/2396202332 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce00057e0 con 0x7f7cf80add50 2021-04-03T03:11:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf8147d90 con 0x7f7cf80add50 2021-04-03T03:11:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf8157620 con 0x7f7cf80add50 2021-04-03T03:11:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf8147f20 con 0x7f7cf80add50 2021-04-03T03:11:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf5ffb700 1 -- 172.21.2.111:0/2396202332 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7ce0010070 con 0x7f7cf80add50 2021-04-03T03:11:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf5ffb700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7cdc056fb0 0x7f7cdc059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cf5ffb700 1 -- 172.21.2.111:0/2396202332 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7ce006f8b0 con 0x7f7cf80add50 2021-04-03T03:11:04.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfcdde700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7cdc056fb0 0x7f7cdc059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.283+0000 7f7cfcdde700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7cdc056fb0 0x7f7cdc059470 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f7ce8002f30 tx=0x7f7ce8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.287+0000 7f7cf5ffb700 1 -- 172.21.2.111:0/2396202332 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7ce0044b10 con 0x7f7cf80add50 2021-04-03T03:11:04.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cf8148150 con 0x7f7cf80add50 2021-04-03T03:11:04.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cf5ffb700 1 -- 172.21.2.111:0/2396202332 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7ce0018070 con 0x7f7cf80add50 2021-04-03T03:11:04.391 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:11:04.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7cdc056fb0 msgr2=0x7f7cdc059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7cdc056fb0 0x7f7cdc059470 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f7ce8002f30 tx=0x7f7ce8015000).stop 2021-04-03T03:11:04.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80add50 msgr2=0x7f7cf8157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80add50 0x7f7cf8157130 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f7cf80b5600 tx=0x7f7ce0007bc0).stop 2021-04-03T03:11:04.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 shutdown_connections 2021-04-03T03:11:04.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7cf80b17b0 0x7f7cf80a9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7cdc056fb0 0x7f7cdc059470 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7cf80603f0 0x7f7cf8156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 --2- 172.21.2.111:0/2396202332 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7cf80add50 0x7f7cf8157130 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 >> 172.21.2.111:0/2396202332 conn(0x7f7cf80b61c0 msgr2=0x7f7cf81440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:04.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 shutdown_connections 2021-04-03T03:11:04.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.379+0000 7f7cfdde0700 1 -- 172.21.2.111:0/2396202332 wait complete. 2021-04-03T03:11:04.399 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:11:04.400 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:11:04.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 Processor -- start 2021-04-03T03:11:04.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 -- start start 2021-04-03T03:11:04.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f40740603f0 0x7f40740b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f407414b700 0x7f40740b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40740b12b0 0x7f40740b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40740b0f60 con 0x7f407414b700 2021-04-03T03:11:04.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40740607d0 con 0x7f40740603f0 2021-04-03T03:11:04.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40740af400 con 0x7f40740b12b0 2021-04-03T03:11:04.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f407414b700 0x7f40740b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f407414b700 0x7f40740b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51910/0 (socket says 172.21.2.111:51910) 2021-04-03T03:11:04.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 -- 172.21.2.111:0/171811037 learned_addr learned my addr 172.21.2.111:0/171811037 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:04.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 -- 172.21.2.111:0/171811037 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40740b12b0 msgr2=0x7f40740b0a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:11:04.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 --2- 172.21.2.111:0/171811037 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40740b12b0 0x7f40740b0a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 -- 172.21.2.111:0/171811037 >> v2:172.21.2.115:3300/0 conn(0x7f40740603f0 msgr2=0x7f40740b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:11:04.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 --2- 172.21.2.111:0/171811037 >> v2:172.21.2.115:3300/0 conn(0x7f40740603f0 0x7f40740b0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 -- 172.21.2.111:0/171811037 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40740af6a0 con 0x7f407414b700 2021-04-03T03:11:04.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4073fff700 1 --2- 172.21.2.111:0/171811037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f407414b700 0x7f40740b0580 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f406c002f30 tx=0x7f406c006e90).ready entity=mon.0 client_cookie=4f6666b55a29e902 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f40737fe700 1 -- 172.21.2.111:0/171811037 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f406c009920 con 0x7f407414b700 2021-04-03T03:11:04.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f40737fe700 1 -- 172.21.2.111:0/171811037 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c009a80 con 0x7f407414b700 2021-04-03T03:11:04.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f40737fe700 1 -- 172.21.2.111:0/171811037 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f406c009e20 con 0x7f407414b700 2021-04-03T03:11:04.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 -- 172.21.2.111:0/171811037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f407414b700 msgr2=0x7f40740b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.443+0000 7f4079f06700 1 --2- 172.21.2.111:0/171811037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f407414b700 0x7f40740b0580 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f406c002f30 tx=0x7f406c006e90).stop 2021-04-03T03:11:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- 172.21.2.111:0/171811037 shutdown_connections 2021-04-03T03:11:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 --2- 172.21.2.111:0/171811037 >> v2:172.21.2.115:3300/0 conn(0x7f40740603f0 0x7f40740b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 --2- 172.21.2.111:0/171811037 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40740b12b0 0x7f40740b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 --2- 172.21.2.111:0/171811037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f407414b700 0x7f40740b0580 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- 172.21.2.111:0/171811037 >> 172.21.2.111:0/171811037 conn(0x7f40740b61c0 msgr2=0x7f407405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- 172.21.2.111:0/171811037 shutdown_connections 2021-04-03T03:11:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- 172.21.2.111:0/171811037 wait complete. 2021-04-03T03:11:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 Processor -- start 2021-04-03T03:11:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- start start 2021-04-03T03:11:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40740603f0 0x7f4074062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40740b12b0 0x7f4074067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407414b700 0x7f4074067a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40740abf50 con 0x7f40740603f0 2021-04-03T03:11:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40740ad870 con 0x7f40740b12b0 2021-04-03T03:11:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40740642d0 con 0x7f407414b700 2021-04-03T03:11:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40740603f0 0x7f4074062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40740603f0 0x7f4074062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51914/0 (socket says 172.21.2.111:51914) 2021-04-03T03:11:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 -- 172.21.2.111:0/886899062 learned_addr learned my addr 172.21.2.111:0/886899062 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:04.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 -- 172.21.2.111:0/886899062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407414b700 msgr2=0x7f4074067a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:11:04.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407414b700 0x7f4074067a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 -- 172.21.2.111:0/886899062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40740b12b0 msgr2=0x7f4074067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40740b12b0 0x7f4074067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 -- 172.21.2.111:0/886899062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4074064500 con 0x7f40740603f0 2021-04-03T03:11:04.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4078f04700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40740603f0 0x7f4074062bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f40740ad9b0 tx=0x7f405c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4071ffb700 1 -- 172.21.2.111:0/886899062 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f405c00d570 con 0x7f40740603f0 2021-04-03T03:11:04.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4071ffb700 1 -- 172.21.2.111:0/886899062 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c00d6d0 con 0x7f40740603f0 2021-04-03T03:11:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40740647a0 con 0x7f40740603f0 2021-04-03T03:11:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40740a9f40 con 0x7f40740603f0 2021-04-03T03:11:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4071ffb700 1 -- 172.21.2.111:0/886899062 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f405c00da30 con 0x7f40740603f0 2021-04-03T03:11:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4074057660 con 0x7f40740603f0 2021-04-03T03:11:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4071ffb700 1 -- 172.21.2.111:0/886899062 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f405c00f070 con 0x7f40740603f0 2021-04-03T03:11:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4071ffb700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4058056ef0 0x7f40580593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4071ffb700 1 -- 172.21.2.111:0/886899062 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f405c06fde0 con 0x7f40740603f0 2021-04-03T03:11:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4073fff700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4058056ef0 0x7f40580593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.447+0000 7f4073fff700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4058056ef0 0x7f40580593b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f406c002f30 tx=0x7f406c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:04.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.451+0000 7f4071ffb700 1 -- 172.21.2.111:0/886899062 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f405c0450c0 con 0x7f40740603f0 2021-04-03T03:11:04.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.571+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40740686d0 con 0x7f40740603f0 2021-04-03T03:11:04.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4071ffb700 1 -- 172.21.2.111:0/886899062 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f405c0486e0 con 0x7f40740603f0 2021-04-03T03:11:04.586 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:11:04.586 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:11:04.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4058056ef0 msgr2=0x7f40580593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4058056ef0 0x7f40580593b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f406c002f30 tx=0x7f406c015000).stop 2021-04-03T03:11:04.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40740603f0 msgr2=0x7f4074062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:04.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40740603f0 0x7f4074062bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f40740ad9b0 tx=0x7f405c006dd0).stop 2021-04-03T03:11:04.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 shutdown_connections 2021-04-03T03:11:04.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407414b700 0x7f4074067a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40740603f0 0x7f4074062bf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4058056ef0 0x7f40580593b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:04.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 --2- 172.21.2.111:0/886899062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40740b12b0 0x7f4074067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:04.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 >> 172.21.2.111:0/886899062 conn(0x7f40740b61c0 msgr2=0x7f40740b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:04.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 shutdown_connections 2021-04-03T03:11:04.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:04.575+0000 7f4079f06700 1 -- 172.21.2.111:0/886899062 wait complete. 2021-04-03T03:11:04.591 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:11:04.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:10:31.116099+0000', 'modified': '2021-04-03T03:10:32.118491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5241': {'gid': 5241, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/798702406', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 798702406}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 798702406}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:11:04.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:11:04.796 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:11:04.869 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5241,"want_state":"up:active","state":"up:active","rank_uptime":33.739407995000001,"mdsmap_epoch":16,"osdmap_epoch":43,"osdmap_epoch_barrier":42,"uptime":37.88319508} 2021-04-03T03:11:04.870 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5241, "mdsmap_epoch": 16, "osdmap_epoch": 43, "osdmap_epoch_barrier": 42, "rank_uptime": 33.739407995, "state": "up:active", "uptime": 37.88319508, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:11:04.870 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:11:04.871 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:11:04.871 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:11:04.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:04.872 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:04.882 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:11:04.936 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:11:04.937 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:11:04.984 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:11:04.984 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:11:04.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:04.986 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:11:05.033 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:11:05.033 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:05.076 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:11:05.087 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:11:05.088 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:11:05.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:11:05.111+0000 7fe7d78cb0c0 -1 init, newargv = 0x5628593fa550 newargc=9 2021-04-03T03:11:05.119 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15119]: starting ceph client 2021-04-03T03:11:05.123 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:11:05.124 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:11:05.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:05.125 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:11:05.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15119]: starting fuse 2021-04-03T03:11:05.133 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:11:05.133 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:11:05.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:05.134 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:11:05.134 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:11:05.135 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:11:05.135 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:11:05.136 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:11:05.136 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:11:05.136 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:11:05.137 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:11:05.137 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:11:05.137 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:11:05.138 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:11:05.138 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:11:05.139 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:11:05.139 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:11:05.139 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:11:05.140 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:11:05.140 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:11:05.140 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:11:05.141 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:11:05.141 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:11:05.142 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:11:05.142 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:11:05.142 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:11:05.200 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15119.asok 2021-04-03T03:11:05.203 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15119.asok 2021-04-03T03:11:05.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:05.204 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15119.asok status 2021-04-03T03:11:05.318 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:11:05.318 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:11:05.318 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:11:05.318 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:11:05.319 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:11:05.319 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:11:05.319 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:11:05.319 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15119", 2021-04-03T03:11:05.320 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:11:05.320 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:11:05.320 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:11:05.320 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:11:05.320 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 5481, 2021-04-03T03:11:05.321 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:11:05.321 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:11:05.321 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:11:05.321 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 5481 2021-04-03T03:11:05.321 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:11:05.322 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:11:05.322 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:11:05.322 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:11:05.322 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3761526062 2021-04-03T03:11:05.322 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:11:05.323 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:11:05.323 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:11:05.323 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:11:05.323 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:11:05.324 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3761526062 2021-04-03T03:11:05.324 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:11:05.324 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.5481 172.21.2.126:0/3761526062", 2021-04-03T03:11:05.324 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/3761526062", 2021-04-03T03:11:05.324 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:11:05.325 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 16, 2021-04-03T03:11:05.325 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 43, 2021-04-03T03:11:05.325 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:11:05.325 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:11:05.325 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:11:05.326 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:11:05.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:05.327 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:05.328 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:05.333 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:11:05.333 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:11:05.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:05.334 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:28.694 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:28.698 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:28.701 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:28.704 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:28.707 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:51.994 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:11:51.994 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:52.006 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:52.018 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:52.029 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:52.042 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:52.053 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:11:52.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:52.054 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:52.055 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:52.075 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:11:52.076 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:11:52.076 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:11:52.077 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:11:52.077 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:52.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15119]: fuse finished with error 0 and tester_r 0 2021-04-03T03:11:52.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:52.169 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:52.169 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:52.174 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:11:52.174 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:11:52.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:11:52.258 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:52.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:52.262 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:52.263 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:11:52.272 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:11:52.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:52.273 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:11:52.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:11:52.274 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:11:52.324 INFO:teuthology.orchestra.run.gibba028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:11:52.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:52.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:52.326 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:52.327 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:11:52.336 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:11:52.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:52.338 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:11:52.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:11:52.339 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:11:52.387 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:11:52.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:52.389 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:11:52.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfdd0a700 1 Processor -- start 2021-04-03T03:11:52.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfdd0a700 1 -- start start 2021-04-03T03:11:52.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfdd0a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf00dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfdd0a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9cf00438e0 0x7f9cf00086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfdd0a700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf000b090 con 0x7f9cf0042370 2021-04-03T03:11:52.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfdd0a700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf0042cb0 con 0x7f9cf00438e0 2021-04-03T03:11:52.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfdd0a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0042f50 con 0x7f9cf000acb0 2021-04-03T03:11:52.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfcd08700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf00dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfd509700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9cf00438e0 0x7f9cf00086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cf7fff700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f9cf0042370 0x7f9cf0007fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:37020/0 (socket says 172.21.2.111:37020) 2021-04-03T03:11:52.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cf7fff700 1 -- 172.21.2.111:0/1673139556 learned_addr learned my addr 172.21.2.111:0/1673139556 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:52.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfcd08700 1 -- 172.21.2.111:0/1673139556 >> v2:172.21.2.115:3300/0 conn(0x7f9cf00438e0 msgr2=0x7f9cf00086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:52.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfcd08700 1 --2- 172.21.2.111:0/1673139556 >> v2:172.21.2.115:3300/0 conn(0x7f9cf00438e0 0x7f9cf00086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfcd08700 1 -- 172.21.2.111:0/1673139556 >> v1:172.21.2.115:6789/0 conn(0x7f9cf0042370 legacy=0x7f9cf0007fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfcd08700 1 -- 172.21.2.111:0/1673139556 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00431f0 con 0x7f9cf000acb0 2021-04-03T03:11:52.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.439+0000 7f9cfcd08700 1 --2- 172.21.2.111:0/1673139556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf00dbb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9cec002f30 tx=0x7f9cec006dd0).ready entity=mon.2 client_cookie=d8251d7578d112aa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:52.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cf77fe700 1 -- 172.21.2.111:0/1673139556 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cec009760 con 0x7f9cf000acb0 2021-04-03T03:11:52.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cf77fe700 1 -- 172.21.2.111:0/1673139556 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0098c0 con 0x7f9cf000acb0 2021-04-03T03:11:52.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cf77fe700 1 -- 172.21.2.111:0/1673139556 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cec009c20 con 0x7f9cf000acb0 2021-04-03T03:11:52.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/1673139556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 msgr2=0x7f9cf00dbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/1673139556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf00dbb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9cec002f30 tx=0x7f9cec006dd0).stop 2021-04-03T03:11:52.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/1673139556 shutdown_connections 2021-04-03T03:11:52.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/1673139556 >> v2:172.21.2.115:3300/0 conn(0x7f9cf00438e0 0x7f9cf00086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/1673139556 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf00dbb20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/1673139556 >> 172.21.2.111:0/1673139556 conn(0x7f9cf00474e0 msgr2=0x7f9cf00478f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:52.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/1673139556 shutdown_connections 2021-04-03T03:11:52.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/1673139556 wait complete. 2021-04-03T03:11:52.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 Processor -- start 2021-04-03T03:11:52.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- start start 2021-04-03T03:11:52.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf0027a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cf0042370 0x7f9cf0029720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cf00438e0 0x7f9cf00e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf002cf40 con 0x7f9cf00438e0 2021-04-03T03:11:52.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0029c10 con 0x7f9cf0042370 2021-04-03T03:11:52.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0029e70 con 0x7f9cf000acb0 2021-04-03T03:11:52.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cf00438e0 0x7f9cf00e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cf00438e0 0x7f9cf00e7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51932/0 (socket says 172.21.2.111:51932) 2021-04-03T03:11:52.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 -- 172.21.2.111:0/2098040619 learned_addr learned my addr 172.21.2.111:0/2098040619 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:52.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfcd08700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf0027a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 -- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 msgr2=0x7f9cf0027a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf0027a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cf7fff700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cf0042370 0x7f9cf0029720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 -- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cf0042370 msgr2=0x7f9cf0029720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cf0042370 0x7f9cf0029720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 -- 172.21.2.111:0/2098040619 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf002c3a0 con 0x7f9cf00438e0 2021-04-03T03:11:52.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfd509700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cf00438e0 0x7f9cf00e7520 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f9cf8082000 tx=0x7f9cf805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:52.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cf5ffb700 1 -- 172.21.2.111:0/2098040619 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cf805d7e0 con 0x7f9cf00438e0 2021-04-03T03:11:52.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cf5ffb700 1 -- 172.21.2.111:0/2098040619 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf80441b0 con 0x7f9cf00438e0 2021-04-03T03:11:52.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cf5ffb700 1 -- 172.21.2.111:0/2098040619 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cf805f4b0 con 0x7f9cf00438e0 2021-04-03T03:11:52.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf002c840 con 0x7f9cf00438e0 2021-04-03T03:11:52.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.443+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf002d240 con 0x7f9cf00438e0 2021-04-03T03:11:52.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.447+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf002a030 con 0x7f9cf00438e0 2021-04-03T03:11:52.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.447+0000 7f9cf5ffb700 1 -- 172.21.2.111:0/2098040619 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9cf808c020 con 0x7f9cf00438e0 2021-04-03T03:11:52.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.447+0000 7f9cf5ffb700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ce0057050 0x7f9ce0059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.447+0000 7f9cf5ffb700 1 -- 172.21.2.111:0/2098040619 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9cf80df6d0 con 0x7f9cf00438e0 2021-04-03T03:11:52.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.447+0000 7f9cfcd08700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ce0057050 0x7f9ce0059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.447+0000 7f9cfcd08700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ce0057050 0x7f9ce0059510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f9cec019000 tx=0x7f9cec015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:52.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.447+0000 7f9cf5ffb700 1 -- 172.21.2.111:0/2098040619 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9cf80b49b0 con 0x7f9cf00438e0 2021-04-03T03:11:52.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.543+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9cf0005180 con 0x7f9cf00438e0 2021-04-03T03:11:52.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cf5ffb700 1 -- 172.21.2.111:0/2098040619 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9cf80b7fd0 con 0x7f9cf00438e0 2021-04-03T03:11:52.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ce0057050 msgr2=0x7f9ce0059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ce0057050 0x7f9ce0059510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f9cec019000 tx=0x7f9cec015000).stop 2021-04-03T03:11:52.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cf00438e0 msgr2=0x7f9cf00e7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cf00438e0 0x7f9cf00e7520 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f9cf8082000 tx=0x7f9cf805ad70).stop 2021-04-03T03:11:52.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 shutdown_connections 2021-04-03T03:11:52.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9cf00438e0 0x7f9cf00e7520 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9cf000acb0 0x7f9cf0027a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ce0057050 0x7f9ce0059510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 --2- 172.21.2.111:0/2098040619 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9cf0042370 0x7f9cf0029720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 >> 172.21.2.111:0/2098040619 conn(0x7f9cf00474e0 msgr2=0x7f9cf0025af0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:52.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 shutdown_connections 2021-04-03T03:11:52.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.547+0000 7f9cfdd0a700 1 -- 172.21.2.111:0/2098040619 wait complete. 2021-04-03T03:11:52.569 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2021-04-03T03:11:52.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 Processor -- start 2021-04-03T03:11:52.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 -- start start 2021-04-03T03:11:52.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb5ac14b700 0x7fb5ac0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5ac0b0c90 0x7fb5ac147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ac0af400 con 0x7fb5ac0603f0 2021-04-03T03:11:52.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ac060860 con 0x7fb5ac0b0c90 2021-04-03T03:11:52.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ac0af6a0 con 0x7fb5ac14b700 2021-04-03T03:11:52.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5ac0b0c90 0x7fb5ac147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb5ac0b0c90 0x7fb5ac147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51938/0 (socket says 172.21.2.111:51938) 2021-04-03T03:11:52.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 -- 172.21.2.111:0/490606667 learned_addr learned my addr 172.21.2.111:0/490606667 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:52.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 -- 172.21.2.111:0/490606667 >> v2:172.21.2.115:3300/0 conn(0x7fb5ac14b700 msgr2=0x7fb5ac0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:52.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 --2- 172.21.2.111:0/490606667 >> v2:172.21.2.115:3300/0 conn(0x7fb5ac14b700 0x7fb5ac0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:52.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 -- 172.21.2.111:0/490606667 >> v1:172.21.2.115:6789/0 conn(0x7fb5ac0603f0 legacy=0x7fb5ac0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:52.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 -- 172.21.2.111:0/490606667 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac0afb40 con 0x7fb5ac0b0c90 2021-04-03T03:11:52.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b21f8700 1 --2- 172.21.2.111:0/490606667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0b0c90 0x7fb5ac147680 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fb5ac06a3b0 tx=0x7fb59c009270).ready entity=mon.0 client_cookie=978a2960011e7cbc server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:52.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b09f5700 1 -- 172.21.2.111:0/490606667 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb59c00bbc0 con 0x7fb5ac0b0c90 2021-04-03T03:11:52.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b09f5700 1 -- 172.21.2.111:0/490606667 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c00bd20 con 0x7fb5ac0b0c90 2021-04-03T03:11:52.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b09f5700 1 -- 172.21.2.111:0/490606667 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb59c00be90 con 0x7fb5ac0b0c90 2021-04-03T03:11:52.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 -- 172.21.2.111:0/490606667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0b0c90 msgr2=0x7fb5ac147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.611+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/490606667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0b0c90 0x7fb5ac147680 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fb5ac06a3b0 tx=0x7fb59c009270).stop 2021-04-03T03:11:52.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- 172.21.2.111:0/490606667 shutdown_connections 2021-04-03T03:11:52.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/490606667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0b0c90 0x7fb5ac147680 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:52.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/490606667 >> v2:172.21.2.115:3300/0 conn(0x7fb5ac14b700 0x7fb5ac0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:52.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- 172.21.2.111:0/490606667 >> 172.21.2.111:0/490606667 conn(0x7fb5ac0b61c0 msgr2=0x7fb5ac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:52.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- 172.21.2.111:0/490606667 shutdown_connections 2021-04-03T03:11:52.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- 172.21.2.111:0/490606667 wait complete. 2021-04-03T03:11:52.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 Processor -- start 2021-04-03T03:11:52.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- start start 2021-04-03T03:11:52.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0603f0 0x7fb5ac0a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5ac0b0c90 0x7fb5ac067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5ac14b700 0x7fb5ac067a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0ab060 con 0x7fb5ac0603f0 2021-04-03T03:11:52.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0ab290 con 0x7fb5ac14b700 2021-04-03T03:11:52.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac067f10 con 0x7fb5ac0b0c90 2021-04-03T03:11:52.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0603f0 0x7fb5ac0a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0603f0 0x7fb5ac0a7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51942/0 (socket says 172.21.2.111:51942) 2021-04-03T03:11:52.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 -- 172.21.2.111:0/383016031 learned_addr learned my addr 172.21.2.111:0/383016031 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:52.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 -- 172.21.2.111:0/383016031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5ac0b0c90 msgr2=0x7fb5ac067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5ac0b0c90 0x7fb5ac067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:52.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 -- 172.21.2.111:0/383016031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5ac14b700 msgr2=0x7fb5ac067a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:52.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5ac14b700 0x7fb5ac067a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:52.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 -- 172.21.2.111:0/383016031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac0aa520 con 0x7fb5ac0603f0 2021-04-03T03:11:52.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b19f7700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0603f0 0x7fb5ac0a7620 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fb5a0002f50 tx=0x7fb5a0006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:52.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5aaffd700 1 -- 172.21.2.111:0/383016031 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a000d4a0 con 0x7fb5ac0603f0 2021-04-03T03:11:52.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5aaffd700 1 -- 172.21.2.111:0/383016031 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a000d600 con 0x7fb5ac0603f0 2021-04-03T03:11:52.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5aaffd700 1 -- 172.21.2.111:0/383016031 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a000d960 con 0x7fb5ac0603f0 2021-04-03T03:11:52.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac0a9f20 con 0x7fb5ac0603f0 2021-04-03T03:11:52.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac156600 con 0x7fb5ac0603f0 2021-04-03T03:11:52.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ac0aa710 con 0x7fb5ac0603f0 2021-04-03T03:11:52.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5aaffd700 1 -- 172.21.2.111:0/383016031 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb5a000f070 con 0x7fb5ac0603f0 2021-04-03T03:11:52.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5aaffd700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb58c056ef0 0x7fb58c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:52.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5aaffd700 1 -- 172.21.2.111:0/383016031 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb5a006fe10 con 0x7fb5ac0603f0 2021-04-03T03:11:52.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.615+0000 7fb5b11f6700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb58c056ef0 0x7fb58c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:52.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.619+0000 7fb5b11f6700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb58c056ef0 0x7fb58c0593b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb5a4002f30 tx=0x7fb5a40069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:52.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.619+0000 7fb5aaffd700 1 -- 172.21.2.111:0/383016031 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb5a00450f0 con 0x7fb5ac0603f0 2021-04-03T03:11:52.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:52.747+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7fb5ac0abb60 con 0x7fb5ac0603f0 2021-04-03T03:11:53.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5aaffd700 1 -- 172.21.2.111:0/383016031 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v141) v1 ==== 144+0+0 (secure 0 0 0) 0x7fb5a0048710 con 0x7fb5ac0603f0 2021-04-03T03:11:53.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb58c056ef0 msgr2=0x7fb58c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:53.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb58c056ef0 0x7fb58c0593b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb5a4002f30 tx=0x7fb5a40069f0).stop 2021-04-03T03:11:53.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0603f0 msgr2=0x7fb5ac0a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:53.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0603f0 0x7fb5ac0a7620 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fb5a0002f50 tx=0x7fb5a0006dd0).stop 2021-04-03T03:11:53.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 shutdown_connections 2021-04-03T03:11:53.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb58c056ef0 0x7fb58c0593b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:53.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb5ac14b700 0x7fb5ac067a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:53.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb5ac0603f0 0x7fb5ac0a7620 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:53.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 --2- 172.21.2.111:0/383016031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb5ac0b0c90 0x7fb5ac067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:53.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 >> 172.21.2.111:0/383016031 conn(0x7fb5ac0b61c0 msgr2=0x7fb5ac14dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:53.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 shutdown_connections 2021-04-03T03:11:53.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.723+0000 7fb5b29f9700 1 -- 172.21.2.111:0/383016031 wait complete. 2021-04-03T03:11:53.745 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-04-03T03:11:53.746 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2021-04-03T03:11:53.746 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2021-04-03T03:11:53.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 Processor -- start 2021-04-03T03:11:53.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- start start 2021-04-03T03:11:53.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 0x7f1d2c042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:53.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1d2c006bc0 0x7f1d2c0dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:53.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c0dc6e0 con 0x7f1d2c03d360 2021-04-03T03:11:53.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c0d9560 con 0x7f1d2c006bc0 2021-04-03T03:11:53.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c03dcb0 con 0x7f1d2c0dbb90 2021-04-03T03:11:53.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1d2c006bc0 0x7f1d2c0dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:53.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1d2c006bc0 0x7f1d2c0dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51950/0 (socket says 172.21.2.111:51950) 2021-04-03T03:11:53.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 -- 172.21.2.111:0/3866495029 learned_addr learned my addr 172.21.2.111:0/3866495029 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:53.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 -- 172.21.2.111:0/3866495029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 msgr2=0x7f1d2c042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:53.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 --2- 172.21.2.111:0/3866495029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 0x7f1d2c042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:53.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 -- 172.21.2.111:0/3866495029 >> v1:172.21.2.111:6789/0 conn(0x7f1d2c03d360 legacy=0x7f1d2c0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:53.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 -- 172.21.2.111:0/3866495029 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0dc460 con 0x7f1d2c006bc0 2021-04-03T03:11:53.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d327fc700 1 -- 172.21.2.111:0/3866495029 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063584774 0 0) 0x7f1d2c0dc6e0 con 0x7f1d2c03d360 2021-04-03T03:11:53.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d33fff700 1 --2- 172.21.2.111:0/3866495029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c006bc0 0x7f1d2c0dbf70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1d2c03e820 tx=0x7f1d3405ad70).ready entity=mon.0 client_cookie=96a868a73a3e5d00 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:53.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d327fc700 1 -- 172.21.2.111:0/3866495029 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3405bda0 con 0x7f1d2c006bc0 2021-04-03T03:11:53.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d327fc700 1 -- 172.21.2.111:0/3866495029 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3405df60 con 0x7f1d2c006bc0 2021-04-03T03:11:53.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d327fc700 1 -- 172.21.2.111:0/3866495029 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d340699f0 con 0x7f1d2c006bc0 2021-04-03T03:11:53.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- 172.21.2.111:0/3866495029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c006bc0 msgr2=0x7f1d2c0dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:53.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3866495029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c006bc0 0x7f1d2c0dbf70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1d2c03e820 tx=0x7f1d3405ad70).stop 2021-04-03T03:11:53.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- 172.21.2.111:0/3866495029 shutdown_connections 2021-04-03T03:11:53.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3866495029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c006bc0 0x7f1d2c0dbf70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:53.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3866495029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 0x7f1d2c042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:53.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- 172.21.2.111:0/3866495029 >> 172.21.2.111:0/3866495029 conn(0x7f1d2c0462c0 msgr2=0x7f1d2c0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:53.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- 172.21.2.111:0/3866495029 shutdown_connections 2021-04-03T03:11:53.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- 172.21.2.111:0/3866495029 wait complete. 2021-04-03T03:11:53.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 Processor -- start 2021-04-03T03:11:53.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- start start 2021-04-03T03:11:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d2c03d360 0x7f1d2c0ded20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 0x7f1d2c0d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c0dde60 0x7f1d2c03b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:53.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c03c0e0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0d4e80 con 0x7f1d2c03d360 2021-04-03T03:11:53.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.791+0000 7f1d38d70700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c03c330 con 0x7f1d2c0dbb90 2021-04-03T03:11:53.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c0dde60 0x7f1d2c03b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:53.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c0dde60 0x7f1d2c03b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51956/0 (socket says 172.21.2.111:51956) 2021-04-03T03:11:53.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 -- 172.21.2.111:0/3156615008 learned_addr learned my addr 172.21.2.111:0/3156615008 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:53.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 -- 172.21.2.111:0/3156615008 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 msgr2=0x7f1d2c0d52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:53.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 0x7f1d2c0d52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:53.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 -- 172.21.2.111:0/3156615008 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d2c03d360 msgr2=0x7f1d2c0ded20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:53.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d2c03d360 0x7f1d2c0ded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 -- 172.21.2.111:0/3156615008 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c03c5d0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d33fff700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c0dde60 0x7f1d2c03b0f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f1d2c03e820 tx=0x7f1d34064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d30ff9700 1 -- 172.21.2.111:0/3156615008 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d340441b0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d30ff9700 1 -- 172.21.2.111:0/3156615008 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3405f2c0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c0d57d0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c0df590 con 0x7f1d2c0dde60 2021-04-03T03:11:53.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d30ff9700 1 -- 172.21.2.111:0/3156615008 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3405f620 con 0x7f1d2c0dde60 2021-04-03T03:11:53.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d100033f0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d30ff9700 1 -- 172.21.2.111:0/3156615008 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1d3408c020 con 0x7f1d2c0dde60 2021-04-03T03:11:53.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d30ff9700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1d140570a0 0x7f1d14059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:53.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d30ff9700 1 -- 172.21.2.111:0/3156615008 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1d340e0dc0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d337fe700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1d140570a0 0x7f1d14059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:53.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.795+0000 7f1d337fe700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1d140570a0 0x7f1d14059560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1d24002f30 tx=0x7f1d240069d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:53.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.799+0000 7f1d30ff9700 1 -- 172.21.2.111:0/3156615008 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1d340b60a0 con 0x7f1d2c0dde60 2021-04-03T03:11:53.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:53.923+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f1d100036a0 con 0x7f1d2c0dde60 2021-04-03T03:11:54.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.731+0000 7f1d30ff9700 1 -- 172.21.2.111:0/3156615008 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v142) v1 ==== 155+0+0 (secure 0 0 0) 0x7f1d340b96c0 con 0x7f1d2c0dde60 2021-04-03T03:11:54.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.731+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1d140570a0 msgr2=0x7f1d14059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:54.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.731+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1d140570a0 0x7f1d14059560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1d24002f30 tx=0x7f1d240069d0).stop 2021-04-03T03:11:54.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.731+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c0dde60 msgr2=0x7f1d2c03b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:54.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.731+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c0dde60 0x7f1d2c03b0f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f1d2c03e820 tx=0x7f1d34064030).stop 2021-04-03T03:11:54.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 shutdown_connections 2021-04-03T03:11:54.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d2c0dde60 0x7f1d2c03b0f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:54.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1d140570a0 0x7f1d14059560 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:54.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d2c03d360 0x7f1d2c0ded20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 --2- 172.21.2.111:0/3156615008 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d2c0dbb90 0x7f1d2c0d52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 >> 172.21.2.111:0/3156615008 conn(0x7f1d2c0462c0 msgr2=0x7f1d2c043050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:54.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 shutdown_connections 2021-04-03T03:11:54.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.735+0000 7f1d38d70700 1 -- 172.21.2.111:0/3156615008 wait complete. 2021-04-03T03:11:54.753 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:11:54.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.799+0000 7fefef386700 1 Processor -- start 2021-04-03T03:11:54.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.799+0000 7fefef386700 1 -- start start 2021-04-03T03:11:54.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 0x7feff01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:54.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7feff00b17b0 0x7feff0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:54.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00607d0 con 0x7feff00add50 2021-04-03T03:11:54.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff01451c0 con 0x7feff00b17b0 2021-04-03T03:11:54.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff014b090 con 0x7feff00603f0 2021-04-03T03:11:54.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7feff00b17b0 0x7feff0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:54.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7feff00b17b0 0x7feff0144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51968/0 (socket says 172.21.2.111:51968) 2021-04-03T03:11:54.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 -- 172.21.2.111:0/2362581900 learned_addr learned my addr 172.21.2.111:0/2362581900 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:54.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 -- 172.21.2.111:0/2362581900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 msgr2=0x7feff01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:54.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 --2- 172.21.2.111:0/2362581900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 0x7feff01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 -- 172.21.2.111:0/2362581900 >> v1:172.21.2.115:6789/0 conn(0x7feff00add50 legacy=0x7feff0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:54.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 -- 172.21.2.111:0/2362581900 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff014b330 con 0x7feff00b17b0 2021-04-03T03:11:54.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefeeb85700 1 --2- 172.21.2.111:0/2362581900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00b17b0 0x7feff0144cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7feff00b5600 tx=0x7fefe8009270).ready entity=mon.0 client_cookie=767a43f55020073d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:54.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefed382700 1 -- 172.21.2.111:0/2362581900 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefe800bbc0 con 0x7feff00b17b0 2021-04-03T03:11:54.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefed382700 1 -- 172.21.2.111:0/2362581900 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe800bd20 con 0x7feff00b17b0 2021-04-03T03:11:54.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefed382700 1 -- 172.21.2.111:0/2362581900 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefe8011400 con 0x7feff00b17b0 2021-04-03T03:11:54.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- 172.21.2.111:0/2362581900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00b17b0 msgr2=0x7feff0144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:54.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- 172.21.2.111:0/2362581900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00b17b0 0x7feff0144cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7feff00b5600 tx=0x7fefe8009270).stop 2021-04-03T03:11:54.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- 172.21.2.111:0/2362581900 shutdown_connections 2021-04-03T03:11:54.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- 172.21.2.111:0/2362581900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00b17b0 0x7feff0144cd0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:54.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- 172.21.2.111:0/2362581900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 0x7feff01447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- 172.21.2.111:0/2362581900 >> 172.21.2.111:0/2362581900 conn(0x7feff00b61c0 msgr2=0x7feff005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:54.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- 172.21.2.111:0/2362581900 shutdown_connections 2021-04-03T03:11:54.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- 172.21.2.111:0/2362581900 wait complete. 2021-04-03T03:11:54.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 Processor -- start 2021-04-03T03:11:54.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- start start 2021-04-03T03:11:54.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 0x7feff014a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:54.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00add50 0x7feff015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:54.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feff00b17b0 0x7feff015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:54.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff015a100 con 0x7feff00add50 2021-04-03T03:11:54.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0066af0 con 0x7feff00b17b0 2021-04-03T03:11:54.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff0066d90 con 0x7feff00603f0 2021-04-03T03:11:54.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00add50 0x7feff015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:54.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00add50 0x7feff015a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51974/0 (socket says 172.21.2.111:51974) 2021-04-03T03:11:54.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 -- 172.21.2.111:0/3065691701 learned_addr learned my addr 172.21.2.111:0/3065691701 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:54.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 -- 172.21.2.111:0/3065691701 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 msgr2=0x7feff014a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:54.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 0x7feff014a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 -- 172.21.2.111:0/3065691701 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feff00b17b0 msgr2=0x7feff015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:54.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feff00b17b0 0x7feff015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 -- 172.21.2.111:0/3065691701 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0067030 con 0x7feff00add50 2021-04-03T03:11:54.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefedb83700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00add50 0x7feff015a4c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fefe0002f50 tx=0x7fefe0006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:54.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefe77fe700 1 -- 172.21.2.111:0/3065691701 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefe000d4a0 con 0x7feff00add50 2021-04-03T03:11:54.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefe77fe700 1 -- 172.21.2.111:0/3065691701 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe000d600 con 0x7feff00add50 2021-04-03T03:11:54.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff0066770 con 0x7feff00add50 2021-04-03T03:11:54.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefe77fe700 1 -- 172.21.2.111:0/3065691701 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefe000d960 con 0x7feff00add50 2021-04-03T03:11:54.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.803+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff0065fc0 con 0x7feff00add50 2021-04-03T03:11:54.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.807+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff0067840 con 0x7feff00add50 2021-04-03T03:11:54.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.807+0000 7fefe77fe700 1 -- 172.21.2.111:0/3065691701 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fefe001a060 con 0x7feff00add50 2021-04-03T03:11:54.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.807+0000 7fefe77fe700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefd4056f20 0x7fefd40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:54.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.807+0000 7fefe77fe700 1 -- 172.21.2.111:0/3065691701 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fefe006fbd0 con 0x7feff00add50 2021-04-03T03:11:54.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.807+0000 7fefee384700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefd4056f20 0x7fefd40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:54.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.807+0000 7fefee384700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefd4056f20 0x7fefd40593e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fefd8002f30 tx=0x7fefd8006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:54.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.807+0000 7fefe77fe700 1 -- 172.21.2.111:0/3065691701 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fefe0044eb0 con 0x7feff00add50 2021-04-03T03:11:54.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.899+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7feff00671c0 con 0x7feff00add50 2021-04-03T03:11:54.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.903+0000 7fefe77fe700 1 -- 172.21.2.111:0/3065691701 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fefe00180c0 con 0x7feff00add50 2021-04-03T03:11:54.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefd4056f20 msgr2=0x7fefd40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:54.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefd4056f20 0x7fefd40593e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fefd8002f30 tx=0x7fefd8006a60).stop 2021-04-03T03:11:54.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00add50 msgr2=0x7feff015a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:54.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00add50 0x7feff015a4c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fefe0002f50 tx=0x7fefe0006dc0).stop 2021-04-03T03:11:54.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 shutdown_connections 2021-04-03T03:11:54.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefd4056f20 0x7fefd40593e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:54.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feff00b17b0 0x7feff015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feff00603f0 0x7feff014a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:54.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 --2- 172.21.2.111:0/3065691701 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feff00add50 0x7feff015a4c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:54.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 >> 172.21.2.111:0/3065691701 conn(0x7feff00b61c0 msgr2=0x7feff0147320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:54.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 shutdown_connections 2021-04-03T03:11:54.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:54.907+0000 7fefef386700 1 -- 172.21.2.111:0/3065691701 wait complete. 2021-04-03T03:11:54.925 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:11:54.926 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:54.932 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:54.943 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:54.955 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:54.968 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:11:54.980 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:11:54.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:54.981 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:54.981 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:11:55.002 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:11:55.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:55.003 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:11:55.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:55.004 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:55.005 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:11:55.012 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:11:55.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:55.014 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:11:55.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:11:55.015 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:11:55.015 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:11:55.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:11:55.028 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:11:55.029 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:11:55.029 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:11:55.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f0190b87700 1 Processor -- start 2021-04-03T03:11:55.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f0190b87700 1 -- start start 2021-04-03T03:11:55.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f0190b87700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f018404ae10 0x7f018404b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f0190b87700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f01840d7d90 0x7f01840d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f0190b87700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01840d8300 con 0x7f01840d4330 2021-04-03T03:11:55.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f0190b87700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01840460a0 con 0x7f01840d7d90 2021-04-03T03:11:55.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f0190b87700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0184046340 con 0x7f018404ae10 2021-04-03T03:11:55.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f018affd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f01840d7d90 0x7f01840d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f018affd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f01840d7d90 0x7f01840d3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51978/0 (socket says 172.21.2.111:51978) 2021-04-03T03:11:55.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.079+0000 7f018affd700 1 -- 172.21.2.111:0/591910555 learned_addr learned my addr 172.21.2.111:0/591910555 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:55.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 -- 172.21.2.111:0/591910555 >> v2:172.21.2.115:3300/0 conn(0x7f018404ae10 msgr2=0x7f018404b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:55.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 --2- 172.21.2.111:0/591910555 >> v2:172.21.2.115:3300/0 conn(0x7f018404ae10 0x7f018404b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 -- 172.21.2.111:0/591910555 >> v1:172.21.2.111:6789/0 conn(0x7f01840d4330 legacy=0x7f01840467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 -- 172.21.2.111:0/591910555 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184046ef0 con 0x7f01840d7d90 2021-04-03T03:11:55.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 --2- 172.21.2.111:0/591910555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d7d90 0x7f01840d3e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f017c002f30 tx=0x7f017c006e90).ready entity=mon.0 client_cookie=e8c503c619e52a22 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018a7fc700 1 -- 172.21.2.111:0/591910555 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f017c0099c0 con 0x7f01840d7d90 2021-04-03T03:11:55.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018a7fc700 1 -- 172.21.2.111:0/591910555 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c009b20 con 0x7f01840d7d90 2021-04-03T03:11:55.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018a7fc700 1 -- 172.21.2.111:0/591910555 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f017c009e80 con 0x7f01840d7d90 2021-04-03T03:11:55.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- 172.21.2.111:0/591910555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d7d90 msgr2=0x7f01840d3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 --2- 172.21.2.111:0/591910555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d7d90 0x7f01840d3e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f017c002f30 tx=0x7f017c006e90).stop 2021-04-03T03:11:55.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- 172.21.2.111:0/591910555 shutdown_connections 2021-04-03T03:11:55.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 --2- 172.21.2.111:0/591910555 >> v2:172.21.2.115:3300/0 conn(0x7f018404ae10 0x7f018404b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 --2- 172.21.2.111:0/591910555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d7d90 0x7f01840d3e40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- 172.21.2.111:0/591910555 >> 172.21.2.111:0/591910555 conn(0x7f0184011530 msgr2=0x7f018404a700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:55.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- 172.21.2.111:0/591910555 shutdown_connections 2021-04-03T03:11:55.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- 172.21.2.111:0/591910555 wait complete. 2021-04-03T03:11:55.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 Processor -- start 2021-04-03T03:11:55.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- start start 2021-04-03T03:11:55.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f018404ae10 0x7f0184049110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d4330 0x7f01840e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01840d7d90 0x7f01840e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0184043010 con 0x7f01840d4330 2021-04-03T03:11:55.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0184042af0 con 0x7f01840d7d90 2021-04-03T03:11:55.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0184042d70 con 0x7f018404ae10 2021-04-03T03:11:55.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d4330 0x7f01840e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d4330 0x7f01840e2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:51986/0 (socket says 172.21.2.111:51986) 2021-04-03T03:11:55.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 -- 172.21.2.111:0/3120472385 learned_addr learned my addr 172.21.2.111:0/3120472385 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:55.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 -- 172.21.2.111:0/3120472385 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f018404ae10 msgr2=0x7f0184049110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f018404ae10 0x7f0184049110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 -- 172.21.2.111:0/3120472385 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01840d7d90 msgr2=0x7f01840e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01840d7d90 0x7f01840e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 -- 172.21.2.111:0/3120472385 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01840e6480 con 0x7f01840d4330 2021-04-03T03:11:55.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f018affd700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d4330 0x7f01840e2da0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f017c002f30 tx=0x7f017c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0188ff9700 1 -- 172.21.2.111:0/3120472385 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f017c009e20 con 0x7f01840d4330 2021-04-03T03:11:55.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0188ff9700 1 -- 172.21.2.111:0/3120472385 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c01e460 con 0x7f01840d4330 2021-04-03T03:11:55.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01840d3010 con 0x7f01840d4330 2021-04-03T03:11:55.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01840d2660 con 0x7f01840d4330 2021-04-03T03:11:55.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.083+0000 7f0188ff9700 1 -- 172.21.2.111:0/3120472385 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f017c01e7c0 con 0x7f01840d4330 2021-04-03T03:11:55.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.087+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01840d2d40 con 0x7f01840d4330 2021-04-03T03:11:55.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.087+0000 7f0188ff9700 1 -- 172.21.2.111:0/3120472385 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f017c025030 con 0x7f01840d4330 2021-04-03T03:11:55.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.087+0000 7f0188ff9700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f016c057000 0x7f016c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.087+0000 7f0188ff9700 1 -- 172.21.2.111:0/3120472385 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f017c071720 con 0x7f01840d4330 2021-04-03T03:11:55.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.087+0000 7f018b7fe700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f016c057000 0x7f016c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.087+0000 7f018b7fe700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f016c057000 0x7f016c0594c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f0180002f30 tx=0x7f0180006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.087+0000 7f0188ff9700 1 -- 172.21.2.111:0/3120472385 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f017c046b10 con 0x7f01840d4330 2021-04-03T03:11:55.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.179+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f01840e6610 con 0x7f01840d4330 2021-04-03T03:11:55.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0188ff9700 1 -- 172.21.2.111:0/3120472385 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v43) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f017c04a130 con 0x7f01840d4330 2021-04-03T03:11:55.196 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:11:55.196 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:11:55.196 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 43, 2021-04-03T03:11:55.196 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:11:55.197 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:11:55.197 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:10:32.119441+0000", 2021-04-03T03:11:55.197 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:11:55.197 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:11:55.197 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:11:55.198 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:11:55.198 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:11:55.198 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:11:55.198 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:11:55.199 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:11:55.199 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:11:55.199 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:11:55.200 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:11:55.200 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:11:55.200 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:11:55.200 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:11:55.200 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:11:55.201 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 8, 2021-04-03T03:11:55.201 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:11:55.201 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:11:55.201 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:11:55.202 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:11:55.202 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:11:55.202 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.202 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:11:55.202 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:11:55.203 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:11:55.203 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:11:55.203 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:11:55.203 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:11:55.204 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:11:55.204 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:11:55.204 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:11:55.204 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:11:55.204 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:11:55.205 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:11:55.205 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:11:55.205 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:11:55.205 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:11:55.205 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:11:55.206 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:11:55.206 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:11:55.206 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:11:55.207 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:11:55.207 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:11:55.207 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:11:55.207 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:11:55.208 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.208 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:11:55.208 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:11:55.208 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:11:55.208 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:11:55.209 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:11:55.209 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:11:55.209 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:11:55.209 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:11:55.209 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:11:55.210 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:11:55.210 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:11:55.210 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:11:55.210 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:11:55.211 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:11:55.211 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:11:55.211 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:11:55.211 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:11:55.211 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:11:55.212 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:11:55.212 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:11:55.212 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:11:55.213 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:11:55.213 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:11:55.213 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:11:55.213 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:11:55.214 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:11:55.214 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:11:55.214 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.214 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:11:55.214 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:11:55.215 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:11:55.215 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:11:55.215 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:11:55.215 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:11:55.216 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:11:55.216 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:11:55.216 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:11:55.216 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:11:55.216 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:11:55.217 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:11:55.217 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:11:55.217 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.217 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:11:55.218 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:11:55.218 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.218 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.218 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.218 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:11:55.219 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:11:55.219 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:11:55.219 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:11:55.220 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:11:55.220 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:11:55.220 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:11:55.220 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:11:55.221 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:11:55.221 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:11:55.221 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:11:55.221 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:11:55.221 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:11:55.222 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:11:55.222 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:11:55.222 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:11:55.222 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:11:55.223 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:11:55.223 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:11:55.223 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:11:55.223 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:11:55.223 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:11:55.224 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:11:55.224 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.224 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:11:55.224 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:11:55.225 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:11:55.225 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:11:55.225 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:11:55.225 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:11:55.225 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:11:55.226 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:11:55.226 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:11:55.226 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:11:55.226 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:11:55.226 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:11:55.227 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:11:55.227 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:11:55.227 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:11:55.227 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:11:55.227 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:11:55.228 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:11:55.228 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:11:55.228 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:11:55.228 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:11:55.229 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:11:55.229 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:11:55.229 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:11:55.229 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:11:55.229 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:11:55.230 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:11:55.230 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.230 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:11:55.230 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:11:55.231 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:11:55.231 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:11:55.231 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:11:55.232 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:11:55.232 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:11:55.232 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:11:55.232 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:11:55.232 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:11:55.233 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:11:55.233 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:11:55.233 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:11:55.233 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:11:55.233 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.234 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.234 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.234 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 7, 2021-04-03T03:11:55.234 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:11:55.235 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:10:29.772278+0000", 2021-04-03T03:11:55.235 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:11:55.235 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:11:55.235 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:11:55.235 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:11:55.236 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:11:55.236 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:11:55.236 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:11:55.236 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:11:55.237 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:11:55.237 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:11:55.237 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:11:55.237 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:11:55.237 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:11:55.238 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:11:55.238 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:11:55.238 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:11:55.238 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:11:55.238 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:11:55.239 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:11:55.239 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:11:55.239 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.239 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "43", 2021-04-03T03:11:55.240 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:11:55.240 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:11:55.240 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:11:55.240 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:11:55.240 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:11:55.241 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:11:55.241 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:11:55.241 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:11:55.241 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:11:55.241 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:11:55.242 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:11:55.242 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:11:55.242 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:11:55.242 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:11:55.242 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:11:55.243 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:11:55.243 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:11:55.243 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:11:55.243 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:11:55.244 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:11:55.244 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:11:55.244 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:11:55.244 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:11:55.244 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:11:55.245 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:11:55.245 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:11:55.245 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.245 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:11:55.245 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:11:55.246 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:11:55.246 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:11:55.246 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:11:55.246 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:11:55.247 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:11:55.247 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:11:55.247 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:11:55.247 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:11:55.247 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:11:55.248 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:11:55.248 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:11:55.248 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:11:55.248 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:11:55.248 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.249 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:11:55.249 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:11:55.249 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:11:55.249 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.250 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.250 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.250 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.250 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 8, 2021-04-03T03:11:55.250 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:11:55.251 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:10:30.083040+0000", 2021-04-03T03:11:55.251 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:11:55.251 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:11:55.251 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:11:55.251 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:11:55.252 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:11:55.252 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:11:55.252 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:11:55.252 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:11:55.252 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:11:55.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:11:55.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:11:55.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:11:55.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:11:55.254 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:11:55.254 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:11:55.254 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:11:55.254 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:11:55.254 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:11:55.255 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:11:55.255 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:11:55.255 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.255 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "43", 2021-04-03T03:11:55.256 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:11:55.256 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:11:55.256 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:11:55.256 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:11:55.256 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:11:55.257 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:11:55.257 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:11:55.257 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:11:55.257 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:11:55.257 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:11:55.258 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:11:55.258 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:11:55.258 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:11:55.258 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:11:55.258 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:11:55.259 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:11:55.259 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:11:55.259 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:11:55.259 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:11:55.260 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:11:55.260 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:11:55.260 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:11:55.260 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:11:55.260 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:11:55.261 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:11:55.261 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:11:55.261 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.261 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:11:55.261 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:11:55.262 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:11:55.262 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:11:55.262 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:11:55.262 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:11:55.262 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:11:55.263 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:11:55.263 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:11:55.263 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:11:55.263 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:11:55.264 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:11:55.264 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:11:55.264 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:11:55.264 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:11:55.264 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:11:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:11:55.265 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:11:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:11:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.266 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.267 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.267 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.267 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.267 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.267 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.268 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 40, 2021-04-03T03:11:55.268 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.268 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.268 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.268 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.269 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.269 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.269 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:11:55.269 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.269 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.270 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.270 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.270 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:11:55.270 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.271 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.271 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.271 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.271 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.271 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.272 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.272 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.272 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:11:55.272 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.272 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.273 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.273 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.273 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:11:55.273 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.273 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.274 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.274 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.274 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.274 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.275 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.275 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.275 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:11:55.275 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.275 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.276 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.276 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.276 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:11:55.276 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.276 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.277 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.277 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.277 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.277 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.278 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.278 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.278 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:11:55.278 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.278 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.279 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.279 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.279 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:11:55.279 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:11:55.279 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.280 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.280 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.280 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:11:55.280 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:11:55.280 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:11:55.281 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:11:55.281 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.281 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.281 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.282 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.282 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.282 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.282 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:11:55.282 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:11:55.283 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.283 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.283 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.283 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.283 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.284 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.284 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.284 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 41, 2021-04-03T03:11:55.284 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.285 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.285 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.285 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.285 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.285 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.286 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:11:55.286 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.286 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.286 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.286 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.287 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:11:55.287 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.287 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.287 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.287 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.288 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.288 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.288 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.288 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.289 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:11:55.289 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.289 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.289 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.289 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.290 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:11:55.290 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.290 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.290 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.290 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.291 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.291 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.291 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.291 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.291 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:11:55.292 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.292 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.292 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.292 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.293 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:11:55.293 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.293 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.293 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.293 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.294 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.294 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.294 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.294 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.294 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:11:55.295 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.295 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.295 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.295 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.295 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:11:55.296 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:11:55.296 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.296 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.296 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.297 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:11:55.297 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:11:55.297 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:11:55.297 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:11:55.297 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.298 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.298 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.298 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.298 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.298 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.299 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:11:55.299 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:11:55.299 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.299 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.299 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.300 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.300 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.300 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.300 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.301 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 30, 2021-04-03T03:11:55.301 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.301 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.301 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.301 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.302 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.302 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.302 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:11:55.302 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.302 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.303 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.303 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.303 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:11:55.303 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.303 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.304 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.304 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.304 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.304 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.305 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.305 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.305 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:11:55.305 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.305 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.306 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.306 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.306 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:11:55.306 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.306 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.307 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.307 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.307 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.307 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.308 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.308 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.308 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:11:55.308 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.308 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.309 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.309 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.309 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:11:55.309 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.309 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.310 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.310 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.310 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.310 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.311 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.311 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.311 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:11:55.311 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.311 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.312 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.312 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.312 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:11:55.312 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:11:55.312 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.313 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.313 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.313 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:11:55.313 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:11:55.314 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:11:55.314 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:11:55.314 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.314 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.314 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.315 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.315 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.315 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.315 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:11:55.315 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:11:55.316 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.316 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.316 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.316 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.317 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.317 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.317 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.317 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 40, 2021-04-03T03:11:55.317 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.318 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.318 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.318 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.318 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.318 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.319 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:11:55.319 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.319 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.319 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.320 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.320 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:11:55.320 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.320 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.320 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.321 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.321 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.321 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.321 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.321 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.322 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:11:55.322 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.322 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.322 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.322 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.323 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:11:55.323 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.323 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.323 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.324 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.324 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.324 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.324 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.324 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.325 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:11:55.325 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.325 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.325 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.325 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.326 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:11:55.326 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.326 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.326 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.326 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.327 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.327 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.327 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.327 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.328 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:11:55.328 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.328 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.328 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.328 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.329 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:11:55.329 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:11:55.329 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.329 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.329 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.330 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:11:55.330 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:11:55.330 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:11:55.330 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:11:55.330 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.331 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.331 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.331 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.331 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.332 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.332 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:11:55.332 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:11:55.332 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.332 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.333 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.333 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.333 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.333 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.333 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.334 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 40, 2021-04-03T03:11:55.334 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.334 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.334 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.334 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.335 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.335 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.335 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:11:55.335 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.336 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.336 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.336 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.336 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:11:55.336 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.337 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.337 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.337 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.337 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.337 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.338 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.338 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.338 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:11:55.338 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.338 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.339 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.339 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.339 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:11:55.339 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.340 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.340 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.340 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.340 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.340 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.341 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.341 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.341 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:11:55.341 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.341 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.342 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.342 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.342 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:11:55.342 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.342 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.343 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.343 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.343 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.343 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.344 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.344 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.344 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:11:55.344 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.344 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.345 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.345 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.345 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:11:55.345 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:11:55.345 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.346 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.346 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.346 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:11:55.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:11:55.347 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:11:55.347 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:11:55.347 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.347 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.347 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.348 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.348 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.348 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.348 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:11:55.348 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:11:55.349 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.349 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.349 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.349 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.349 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.350 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.350 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.350 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 40, 2021-04-03T03:11:55.350 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.351 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.351 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.351 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.351 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.351 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.352 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:11:55.352 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.352 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.352 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.352 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.353 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:11:55.353 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.353 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.353 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.353 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.354 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.354 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.354 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.354 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.355 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:11:55.355 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.355 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.355 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.355 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.356 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:11:55.356 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.356 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.356 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.356 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.357 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.357 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.357 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.357 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.357 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:11:55.358 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.358 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.358 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.358 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.359 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:11:55.359 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.359 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.359 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.359 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.360 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.360 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.360 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.360 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.360 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:11:55.361 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.361 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.361 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.361 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.362 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:11:55.362 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:11:55.362 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.362 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.362 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.363 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:11:55.363 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:11:55.363 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:11:55.363 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:11:55.363 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.364 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.364 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.364 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.364 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.365 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.365 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:11:55.365 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:11:55.365 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.365 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.366 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.366 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.366 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.366 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.366 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.367 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 40, 2021-04-03T03:11:55.367 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.367 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.367 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.367 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.368 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.368 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.368 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:11:55.368 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.369 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.369 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.369 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.369 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:11:55.369 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.370 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.370 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.370 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.370 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.370 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.371 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.371 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.371 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:11:55.371 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.372 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.372 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.372 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:11:55.372 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.373 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.373 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.373 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.373 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.373 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.374 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.374 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.374 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:11:55.374 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.374 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.375 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.375 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.375 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:11:55.375 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.376 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.376 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.376 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.376 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.376 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.377 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.377 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.377 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:11:55.377 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.377 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.378 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.378 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.378 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:11:55.378 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:11:55.378 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.379 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.379 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.379 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:11:55.379 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:11:55.380 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:11:55.380 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:11:55.380 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.380 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.380 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.381 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.381 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.381 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.381 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:11:55.381 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:11:55.382 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:11:55.382 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:11:55.382 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:11:55.382 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:11:55.382 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:11:55.383 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:11:55.383 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:11:55.383 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 39, 2021-04-03T03:11:55.383 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:11:55.384 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:11:55.384 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:11:55.384 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.384 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.384 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.385 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:11:55.385 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.385 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.385 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.385 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.386 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:11:55.386 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.386 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.386 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.386 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.387 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:11:55.387 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.387 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.387 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.388 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:11:55.388 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.388 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.388 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.388 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.389 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:11:55.389 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.389 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.389 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.389 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.390 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:11:55.390 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.390 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.390 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.390 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:11:55.391 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.391 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.391 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.391 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.392 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:11:55.392 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.392 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.392 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.392 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.393 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:11:55.393 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:11:55.393 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.393 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:11:55.393 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:11:55.394 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.394 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.394 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.394 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:11:55.394 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:11:55.395 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:11:55.395 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.395 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.395 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.396 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:11:55.396 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:11:55.396 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:11:55.396 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:11:55.396 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:11:55.397 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:11:55.397 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:11:55.397 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:11:55.397 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.397 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:11:55.398 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:11:55.398 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.398 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:11:55.398 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.398 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.399 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.399 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.399 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.399 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:11:55.400 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.400 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.400 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.400 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:11:55.400 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.401 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.401 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.401 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.401 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.401 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:11:55.402 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.402 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.402 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.402 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:11:55.402 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.403 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.403 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.403 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.403 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.404 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:11:55.404 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.404 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.404 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.404 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:11:55.405 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.405 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.405 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.405 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.405 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.406 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:11:55.406 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.406 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.406 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.407 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:11:55.407 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.407 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.407 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.407 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.408 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.408 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:11:55.408 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.408 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.408 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.409 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:11:55.409 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.409 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.409 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.410 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.410 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.410 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:11:55.410 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.410 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.411 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.411 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:11:55.411 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.411 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.411 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.412 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.412 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.412 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:11:55.412 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.412 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.413 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:11:55.413 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:11:55.413 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:11:55.413 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:11:55.414 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:11:55.414 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:11:55.414 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:11:55.414 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:11:55.414 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:11:55.415 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.415 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:11:55.415 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:11:55.415 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:11:55.415 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:11:55.416 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:11:55.416 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:11:55.416 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:11:55.416 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:11:55.416 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:11:55.417 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:11:55.417 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:11:55.417 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:11:55.417 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:11:55.418 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:11:55.418 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:11:55.418 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:11:55.418 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:11:55.418 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:11:55.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.419 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:11:55.419 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:11:55.419 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:11:55.419 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:11:55.420 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:11:55.420 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:11:55.420 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:11:55.420 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:11:55.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:11:55.421 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:11:55.421 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:11:55.421 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:11:55.421 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:11:55.422 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:11:55.422 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:11:55.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f016c057000 msgr2=0x7f016c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f016c057000 0x7f016c0594c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f0180002f30 tx=0x7f0180006a60).stop 2021-04-03T03:11:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d4330 msgr2=0x7f01840e2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d4330 0x7f01840e2da0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f017c002f30 tx=0x7f017c015000).stop 2021-04-03T03:11:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 shutdown_connections 2021-04-03T03:11:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f016c057000 0x7f016c0594c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01840d7d90 0x7f01840e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f018404ae10 0x7f0184049110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 --2- 172.21.2.111:0/3120472385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01840d4330 0x7f01840e2da0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 >> 172.21.2.111:0/3120472385 conn(0x7f0184011530 msgr2=0x7f018404a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 shutdown_connections 2021-04-03T03:11:55.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.183+0000 7f0190b87700 1 -- 172.21.2.111:0/3120472385 wait complete. 2021-04-03T03:11:55.426 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:11:55.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 Processor -- start 2021-04-03T03:11:55.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- start start 2021-04-03T03:11:55.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f57d80b12b0 0x7f57d81522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d80607d0 con 0x7f57d814b700 2021-04-03T03:11:55.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d80b0eb0 con 0x7f57d80603f0 2021-04-03T03:11:55.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d80af400 con 0x7f57d80b12b0 2021-04-03T03:11:55.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57dcf36700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f57d814b700 0x7f57d80b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52786/0 (socket says 172.21.2.111:52786) 2021-04-03T03:11:55.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57dcf36700 1 -- 172.21.2.111:0/60280725 learned_addr learned my addr 172.21.2.111:0/60280725 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:55.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57ddf38700 1 --2- 172.21.2.111:0/60280725 >> v2:172.21.2.111:3300/0 conn(0x7f57d80b12b0 0x7f57d81522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57cbfff700 1 -- 172.21.2.111:0/60280725 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766253699 0 0) 0x7f57d80607d0 con 0x7f57d814b700 2021-04-03T03:11:55.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57cbfff700 1 -- 172.21.2.111:0/60280725 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c0003240 con 0x7f57d814b700 2021-04-03T03:11:55.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57ddf38700 1 -- 172.21.2.111:0/60280725 >> v1:172.21.2.115:6789/0 conn(0x7f57d80603f0 legacy=0x7f57d80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:55.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57ddf38700 1 -- 172.21.2.111:0/60280725 >> v1:172.21.2.111:6789/0 conn(0x7f57d814b700 legacy=0x7f57d80b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57ddf38700 1 -- 172.21.2.111:0/60280725 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80af6a0 con 0x7f57d80b12b0 2021-04-03T03:11:55.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57ddf38700 1 --2- 172.21.2.111:0/60280725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d80b12b0 0x7f57d81522d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f57d81486c0 tx=0x7f57d00091d0).ready entity=mon.0 client_cookie=2bf5a0d3e64da9ac server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57cbfff700 1 -- 172.21.2.111:0/60280725 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d000b990 con 0x7f57d80b12b0 2021-04-03T03:11:55.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57cbfff700 1 -- 172.21.2.111:0/60280725 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d000baf0 con 0x7f57d80b12b0 2021-04-03T03:11:55.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57cbfff700 1 -- 172.21.2.111:0/60280725 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d000bc60 con 0x7f57d80b12b0 2021-04-03T03:11:55.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- 172.21.2.111:0/60280725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d80b12b0 msgr2=0x7f57d81522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 --2- 172.21.2.111:0/60280725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d80b12b0 0x7f57d81522d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f57d81486c0 tx=0x7f57d00091d0).stop 2021-04-03T03:11:55.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- 172.21.2.111:0/60280725 shutdown_connections 2021-04-03T03:11:55.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 --2- 172.21.2.111:0/60280725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d80b12b0 0x7f57d81522d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- 172.21.2.111:0/60280725 >> 172.21.2.111:0/60280725 conn(0x7f57d80b61c0 msgr2=0x7f57d805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:55.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- 172.21.2.111:0/60280725 shutdown_connections 2021-04-03T03:11:55.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- 172.21.2.111:0/60280725 wait complete. 2021-04-03T03:11:55.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 Processor -- start 2021-04-03T03:11:55.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.491+0000 7f57de739700 1 -- start start 2021-04-03T03:11:55.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57d80603f0 0x7f57d80a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57d80b12b0 0x7f57d8065770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d814b700 0x7f57d8066f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d80666b0 con 0x7f57d814b700 2021-04-03T03:11:55.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8066950 con 0x7f57d80603f0 2021-04-03T03:11:55.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8065050 con 0x7f57d80b12b0 2021-04-03T03:11:55.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d814b700 0x7f57d8066f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d814b700 0x7f57d8066f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52002/0 (socket says 172.21.2.111:52002) 2021-04-03T03:11:55.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 -- 172.21.2.111:0/2318685289 learned_addr learned my addr 172.21.2.111:0/2318685289 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:55.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57dcf36700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57d80b12b0 0x7f57d8065770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 -- 172.21.2.111:0/2318685289 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57d80b12b0 msgr2=0x7f57d8065770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57d80b12b0 0x7f57d8065770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 -- 172.21.2.111:0/2318685289 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57d80603f0 msgr2=0x7f57d80a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57d80603f0 0x7f57d80a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 -- 172.21.2.111:0/2318685289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80654f0 con 0x7f57d814b700 2021-04-03T03:11:55.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ddf38700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d814b700 0x7f57d8066f10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f57d81486c0 tx=0x7f57d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ca7fc700 1 -- 172.21.2.111:0/2318685289 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d000be90 con 0x7f57d814b700 2021-04-03T03:11:55.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ca7fc700 1 -- 172.21.2.111:0/2318685289 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d001f3f0 con 0x7f57d814b700 2021-04-03T03:11:55.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d8146a90 con 0x7f57d814b700 2021-04-03T03:11:55.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d8146f30 con 0x7f57d814b700 2021-04-03T03:11:55.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ca7fc700 1 -- 172.21.2.111:0/2318685289 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d001f750 con 0x7f57d814b700 2021-04-03T03:11:55.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d8065210 con 0x7f57d814b700 2021-04-03T03:11:55.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ca7fc700 1 -- 172.21.2.111:0/2318685289 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f57d000f070 con 0x7f57d814b700 2021-04-03T03:11:55.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ca7fc700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f57b8056f70 0x7f57b8059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57ca7fc700 1 -- 172.21.2.111:0/2318685289 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f57d0070dd0 con 0x7f57d814b700 2021-04-03T03:11:55.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57dd737700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f57b8056f70 0x7f57b8059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.495+0000 7f57dd737700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f57b8056f70 0x7f57b8059430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f57cc001f60 tx=0x7f57cc0077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.499+0000 7f57ca7fc700 1 -- 172.21.2.111:0/2318685289 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f57d00460b0 con 0x7f57d814b700 2021-04-03T03:11:55.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.619+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57d8057660 con 0x7f57d814b700 2021-04-03T03:11:55.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57ca7fc700 1 -- 172.21.2.111:0/2318685289 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f57d0024870 con 0x7f57d814b700 2021-04-03T03:11:55.635 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:11:55.635 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:10:32.118491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/798702406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":798702406},{"type":"v1","addr":"172.21.2.111:6835","nonce":798702406}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:11:55.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f57b8056f70 msgr2=0x7f57b8059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f57b8056f70 0x7f57b8059430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f57cc001f60 tx=0x7f57cc0077d0).stop 2021-04-03T03:11:55.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d814b700 msgr2=0x7f57d8066f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d814b700 0x7f57d8066f10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f57d81486c0 tx=0x7f57d0016000).stop 2021-04-03T03:11:55.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 shutdown_connections 2021-04-03T03:11:55.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f57d814b700 0x7f57d8066f10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f57b8056f70 0x7f57b8059430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f57d80603f0 0x7f57d80a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 --2- 172.21.2.111:0/2318685289 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f57d80b12b0 0x7f57d8065770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 >> 172.21.2.111:0/2318685289 conn(0x7f57d80b61c0 msgr2=0x7f57d80b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:55.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 shutdown_connections 2021-04-03T03:11:55.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.623+0000 7f57de739700 1 -- 172.21.2.111:0/2318685289 wait complete. 2021-04-03T03:11:55.639 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 16 2021-04-03T03:11:55.643 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:11:55.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 Processor -- start 2021-04-03T03:11:55.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- start start 2021-04-03T03:11:55.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240603f0 0x7f21241447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f21240add50 0x7f2124144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21240607d0 con 0x7f21240b17b0 2021-04-03T03:11:55.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21241451c0 con 0x7f21240add50 2021-04-03T03:11:55.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212414b090 con 0x7f21240603f0 2021-04-03T03:11:55.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129821700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f21240add50 0x7f2124144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212a823700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f21240b17b0 0x7f2124147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52798/0 (socket says 172.21.2.111:52798) 2021-04-03T03:11:55.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212a823700 1 -- 172.21.2.111:0/4031006931 learned_addr learned my addr 172.21.2.111:0/4031006931 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:55.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129821700 1 -- 172.21.2.111:0/4031006931 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240603f0 msgr2=0x7f21241447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:55.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129821700 1 --2- 172.21.2.111:0/4031006931 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240603f0 0x7f21241447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129821700 1 -- 172.21.2.111:0/4031006931 >> v1:172.21.2.111:6789/0 conn(0x7f21240b17b0 legacy=0x7f2124147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129821700 1 -- 172.21.2.111:0/4031006931 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212414b330 con 0x7f21240add50 2021-04-03T03:11:55.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129821700 1 --2- 172.21.2.111:0/4031006931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240add50 0x7f2124144cd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f2118002f30 tx=0x7f2118006dd0).ready entity=mon.0 client_cookie=f2b12d83d1857f3c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129020700 1 -- 172.21.2.111:0/4031006931 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21180097f0 con 0x7f21240add50 2021-04-03T03:11:55.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129020700 1 -- 172.21.2.111:0/4031006931 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2118009950 con 0x7f21240add50 2021-04-03T03:11:55.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f2129020700 1 -- 172.21.2.111:0/4031006931 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2118009cb0 con 0x7f21240add50 2021-04-03T03:11:55.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- 172.21.2.111:0/4031006931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240add50 msgr2=0x7f2124144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 --2- 172.21.2.111:0/4031006931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240add50 0x7f2124144cd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f2118002f30 tx=0x7f2118006dd0).stop 2021-04-03T03:11:55.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- 172.21.2.111:0/4031006931 shutdown_connections 2021-04-03T03:11:55.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 --2- 172.21.2.111:0/4031006931 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240603f0 0x7f21241447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 --2- 172.21.2.111:0/4031006931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240add50 0x7f2124144cd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:55.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- 172.21.2.111:0/4031006931 >> 172.21.2.111:0/4031006931 conn(0x7f21240b61c0 msgr2=0x7f212405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:55.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- 172.21.2.111:0/4031006931 shutdown_connections 2021-04-03T03:11:55.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.691+0000 7f212b024700 1 -- 172.21.2.111:0/4031006931 wait complete. 2021-04-03T03:11:55.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 Processor -- start 2021-04-03T03:11:55.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 -- start start 2021-04-03T03:11:55.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21240603f0 0x7f21240a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240add50 0x7f21241569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240b17b0 0x7f21240aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21241570a0 con 0x7f21240b17b0 2021-04-03T03:11:55.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124157340 con 0x7f21240603f0 2021-04-03T03:11:55.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21241575e0 con 0x7f21240add50 2021-04-03T03:11:55.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240b17b0 0x7f21240aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240b17b0 0x7f21240aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52016/0 (socket says 172.21.2.111:52016) 2021-04-03T03:11:55.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 -- 172.21.2.111:0/3544360843 learned_addr learned my addr 172.21.2.111:0/3544360843 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:55.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 -- 172.21.2.111:0/3544360843 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240add50 msgr2=0x7f21241569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240add50 0x7f21241569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 -- 172.21.2.111:0/3544360843 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21240603f0 msgr2=0x7f21240a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:55.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21240603f0 0x7f21240a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:55.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 -- 172.21.2.111:0/3544360843 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124157a80 con 0x7f21240b17b0 2021-04-03T03:11:55.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a823700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240b17b0 0x7f21240aa500 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f21240b5600 tx=0x7f211c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f21177fe700 1 -- 172.21.2.111:0/3544360843 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f211c002060 con 0x7f21240b17b0 2021-04-03T03:11:55.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f21177fe700 1 -- 172.21.2.111:0/3544360843 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c005b80 con 0x7f21240b17b0 2021-04-03T03:11:55.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21240a9f20 con 0x7f21240b17b0 2021-04-03T03:11:55.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f21177fe700 1 -- 172.21.2.111:0/3544360843 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f211c007110 con 0x7f21240b17b0 2021-04-03T03:11:55.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21240aac00 con 0x7f21240b17b0 2021-04-03T03:11:55.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f21177fe700 1 -- 172.21.2.111:0/3544360843 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f211c010030 con 0x7f21240b17b0 2021-04-03T03:11:55.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2124157cd0 con 0x7f21240b17b0 2021-04-03T03:11:55.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f21177fe700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2104056f20 0x7f21040593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:55.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f21177fe700 1 -- 172.21.2.111:0/3544360843 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f211c070090 con 0x7f21240b17b0 2021-04-03T03:11:55.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a022700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2104056f20 0x7f21040593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:55.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.695+0000 7f212a022700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2104056f20 0x7f21040593e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f2120002f30 tx=0x7f2120006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:55.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.699+0000 7f21177fe700 1 -- 172.21.2.111:0/3544360843 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2124157cd0 con 0x7f21240b17b0 2021-04-03T03:11:55.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:55.827+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2124157cd0 con 0x7f21240b17b0 2021-04-03T03:11:56.748 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:11:56.735+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:11:56.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.743+0000 7f21177fe700 1 -- 172.21.2.111:0/3544360843 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2124157cd0 con 0x7f21240b17b0 2021-04-03T03:11:56.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2104056f20 msgr2=0x7f21040593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2104056f20 0x7f21040593e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f2120002f30 tx=0x7f2120006a60).stop 2021-04-03T03:11:56.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240b17b0 msgr2=0x7f21240aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240b17b0 0x7f21240aa500 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f21240b5600 tx=0x7f211c007f90).stop 2021-04-03T03:11:56.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 shutdown_connections 2021-04-03T03:11:56.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21240b17b0 0x7f21240aa500 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2104056f20 0x7f21040593e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21240603f0 0x7f21240a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:56.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 --2- 172.21.2.111:0/3544360843 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21240add50 0x7f21241569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:56.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 >> 172.21.2.111:0/3544360843 conn(0x7f21240b61c0 msgr2=0x7f21241447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:56.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 shutdown_connections 2021-04-03T03:11:56.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.747+0000 7f212b024700 1 -- 172.21.2.111:0/3544360843 wait complete. 2021-04-03T03:11:56.762 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:11:56.766 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:11:56.778 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:11:56.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 Processor -- start 2021-04-03T03:11:56.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- start start 2021-04-03T03:11:56.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8fa40603f0 0x7f8fa40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:56.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa414b700 0x7f8fa40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:56.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa40607d0 con 0x7f8fa40b12b0 2021-04-03T03:11:56.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa40af400 con 0x7f8fa40603f0 2021-04-03T03:11:56.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40af6a0 con 0x7f8fa414b700 2021-04-03T03:11:56.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa8b28700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8fa40b12b0 0x7f8fa41465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52826/0 (socket says 172.21.2.111:52826) 2021-04-03T03:11:56.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa8b28700 1 -- 172.21.2.111:0/2516937873 learned_addr learned my addr 172.21.2.111:0/2516937873 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:56.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa3fff700 1 --2- 172.21.2.111:0/2516937873 >> v2:172.21.2.111:3300/0 conn(0x7f8fa40603f0 0x7f8fa40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:56.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa3fff700 1 -- 172.21.2.111:0/2516937873 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa414b700 msgr2=0x7f8fa40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:56.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa3fff700 1 --2- 172.21.2.111:0/2516937873 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa414b700 0x7f8fa40b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:56.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa3fff700 1 -- 172.21.2.111:0/2516937873 >> v1:172.21.2.111:6789/0 conn(0x7f8fa40b12b0 legacy=0x7f8fa41465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa3fff700 1 -- 172.21.2.111:0/2516937873 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa40afb40 con 0x7f8fa40603f0 2021-04-03T03:11:56.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa3fff700 1 --2- 172.21.2.111:0/2516937873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa40603f0 0x7f8fa40b0090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f8f9c001450 tx=0x7f8f94006dd0).ready entity=mon.0 client_cookie=442f197e4faa24f5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:56.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa2ffd700 1 -- 172.21.2.111:0/2516937873 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f940096b0 con 0x7f8fa40603f0 2021-04-03T03:11:56.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa2ffd700 1 -- 172.21.2.111:0/2516937873 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f94009810 con 0x7f8fa40603f0 2021-04-03T03:11:56.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa2ffd700 1 -- 172.21.2.111:0/2516937873 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f94009b70 con 0x7f8fa40603f0 2021-04-03T03:11:56.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- 172.21.2.111:0/2516937873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa40603f0 msgr2=0x7f8fa40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2516937873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa40603f0 0x7f8fa40b0090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f8f9c001450 tx=0x7f8f94006dd0).stop 2021-04-03T03:11:56.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- 172.21.2.111:0/2516937873 shutdown_connections 2021-04-03T03:11:56.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2516937873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa40603f0 0x7f8fa40b0090 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2516937873 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa414b700 0x7f8fa40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:56.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- 172.21.2.111:0/2516937873 >> 172.21.2.111:0/2516937873 conn(0x7f8fa40b61c0 msgr2=0x7f8fa405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:56.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- 172.21.2.111:0/2516937873 shutdown_connections 2021-04-03T03:11:56.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.811+0000 7f8fa9329700 1 -- 172.21.2.111:0/2516937873 wait complete. 2021-04-03T03:11:56.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 Processor -- start 2021-04-03T03:11:56.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 -- start start 2021-04-03T03:11:56.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fa40603f0 0x7f8fa40a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:56.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa40b12b0 0x7f8fa40abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:56.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa414b700 0x7f8fa4067490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:56.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40ac140 con 0x7f8fa414b700 2021-04-03T03:11:56.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40ab400 con 0x7f8fa40603f0 2021-04-03T03:11:56.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40ab6a0 con 0x7f8fa40b12b0 2021-04-03T03:11:56.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa414b700 0x7f8fa4067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:56.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa414b700 0x7f8fa4067490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52042/0 (socket says 172.21.2.111:52042) 2021-04-03T03:11:56.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 -- 172.21.2.111:0/2494061908 learned_addr learned my addr 172.21.2.111:0/2494061908 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:56.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa37fe700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa40b12b0 0x7f8fa40abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:56.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa3fff700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fa40603f0 0x7f8fa40a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:56.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 -- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa40b12b0 msgr2=0x7f8fa40abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa40b12b0 0x7f8fa40abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 -- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fa40603f0 msgr2=0x7f8fa40a7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fa40603f0 0x7f8fa40a7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 -- 172.21.2.111:0/2494061908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa41599b0 con 0x7f8fa414b700 2021-04-03T03:11:56.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa37fe700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa40b12b0 0x7f8fa40abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:11:56.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa3fff700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fa40603f0 0x7f8fa40a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:11:56.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa8b28700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa414b700 0x7f8fa4067490 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8f9c001510 tx=0x7f8f9c007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:56.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa17fa700 1 -- 172.21.2.111:0/2494061908 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f9c005b80 con 0x7f8fa414b700 2021-04-03T03:11:56.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa17fa700 1 -- 172.21.2.111:0/2494061908 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c005ce0 con 0x7f8fa414b700 2021-04-03T03:11:56.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa17fa700 1 -- 172.21.2.111:0/2494061908 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f9c007300 con 0x7f8fa414b700 2021-04-03T03:11:56.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa40aa1b0 con 0x7f8fa414b700 2021-04-03T03:11:56.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa4067980 con 0x7f8fa414b700 2021-04-03T03:11:56.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa4159ba0 con 0x7f8fa414b700 2021-04-03T03:11:56.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa17fa700 1 -- 172.21.2.111:0/2494061908 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8f9c010030 con 0x7f8fa414b700 2021-04-03T03:11:56.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa17fa700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f88056ef0 0x7f8f880593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:56.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa17fa700 1 -- 172.21.2.111:0/2494061908 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f8f9c0703b0 con 0x7f8fa414b700 2021-04-03T03:11:56.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa3fff700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f88056ef0 0x7f8f880593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:56.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.815+0000 7f8fa3fff700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f88056ef0 0x7f8f880593b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f8f94017000 tx=0x7f8f94015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:56.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.819+0000 7f8fa17fa700 1 -- 172.21.2.111:0/2494061908 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8f9c045740 con 0x7f8fa414b700 2021-04-03T03:11:56.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.943+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fa4057690 con 0x7f8fa414b700 2021-04-03T03:11:56.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.943+0000 7f8fa17fa700 1 -- 172.21.2.111:0/2494061908 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+1777 (secure 0 0 0) 0x7f8f9c048d60 con 0x7f8fa414b700 2021-04-03T03:11:56.958 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:11:56.958 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4439,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/1477453008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1477453008},{"type":"v1","addr":"172.21.2.115:6833","nonce":1477453008}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:10:31.116099+0000","modified":"2021-04-03T03:11:56.739751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-04-03T03:11:56.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f88056ef0 msgr2=0x7f8f880593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f88056ef0 0x7f8f880593b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f8f94017000 tx=0x7f8f94015000).stop 2021-04-03T03:11:56.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa414b700 msgr2=0x7f8fa4067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:56.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa414b700 0x7f8fa4067490 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8f9c001510 tx=0x7f8f9c007ee0).stop 2021-04-03T03:11:56.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 shutdown_connections 2021-04-03T03:11:56.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fa414b700 0x7f8fa4067490 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8f88056ef0 0x7f8f880593b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fa40603f0 0x7f8fa40a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 --2- 172.21.2.111:0/2494061908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fa40b12b0 0x7f8fa40abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:56.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 >> 172.21.2.111:0/2494061908 conn(0x7f8fa40b61c0 msgr2=0x7f8fa4147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:56.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 shutdown_connections 2021-04-03T03:11:56.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:56.947+0000 7f8fa9329700 1 -- 172.21.2.111:0/2494061908 wait complete. 2021-04-03T03:11:56.962 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 17 2021-04-03T03:11:56.965 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:11:57.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 Processor -- start 2021-04-03T03:11:57.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- start start 2021-04-03T03:11:57.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcfe80603f0 0x7fcfe81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe80607d0 con 0x7fcfe80b17b0 2021-04-03T03:11:57.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe814b090 con 0x7fcfe80add50 2021-04-03T03:11:57.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe814b330 con 0x7fcfe80603f0 2021-04-03T03:11:57.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfecf55700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcfe80603f0 0x7fcfe81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:57.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfecf55700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcfe80603f0 0x7fcfe81447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52046/0 (socket says 172.21.2.111:52046) 2021-04-03T03:11:57.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfecf55700 1 -- 172.21.2.111:0/4055794960 learned_addr learned my addr 172.21.2.111:0/4055794960 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:57.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfecf55700 1 -- 172.21.2.111:0/4055794960 >> v1:172.21.2.115:6789/0 conn(0x7fcfe80add50 legacy=0x7fcfe8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:57.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfecf55700 1 -- 172.21.2.111:0/4055794960 >> v1:172.21.2.111:6789/0 conn(0x7fcfe80b17b0 legacy=0x7fcfe8147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfecf55700 1 -- 172.21.2.111:0/4055794960 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe814b5d0 con 0x7fcfe80603f0 2021-04-03T03:11:57.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfecf55700 1 --2- 172.21.2.111:0/4055794960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80603f0 0x7fcfe81447e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fcfe80b5600 tx=0x7fcfd8006e90).ready entity=mon.0 client_cookie=c2d50fc3d8aaf159 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:57.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfe77fe700 1 -- 172.21.2.111:0/4055794960 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfd8009980 con 0x7fcfe80603f0 2021-04-03T03:11:57.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfe77fe700 1 -- 172.21.2.111:0/4055794960 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8009ae0 con 0x7fcfe80603f0 2021-04-03T03:11:57.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfe77fe700 1 -- 172.21.2.111:0/4055794960 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfd8009e80 con 0x7fcfe80603f0 2021-04-03T03:11:57.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- 172.21.2.111:0/4055794960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80603f0 msgr2=0x7fcfe81447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 --2- 172.21.2.111:0/4055794960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80603f0 0x7fcfe81447e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fcfe80b5600 tx=0x7fcfd8006e90).stop 2021-04-03T03:11:57.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- 172.21.2.111:0/4055794960 shutdown_connections 2021-04-03T03:11:57.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 --2- 172.21.2.111:0/4055794960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80603f0 0x7fcfe81447e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:57.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- 172.21.2.111:0/4055794960 >> 172.21.2.111:0/4055794960 conn(0x7fcfe80b61c0 msgr2=0x7fcfe805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:57.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- 172.21.2.111:0/4055794960 shutdown_connections 2021-04-03T03:11:57.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- 172.21.2.111:0/4055794960 wait complete. 2021-04-03T03:11:57.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 Processor -- start 2021-04-03T03:11:57.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- start start 2021-04-03T03:11:57.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfe80603f0 0x7fcfe8066490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80add50 0x7fcfe8146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfe80b17b0 0x7fcfe8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8066980 con 0x7fcfe80add50 2021-04-03T03:11:57.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80abde0 con 0x7fcfe80b17b0 2021-04-03T03:11:57.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfedf57700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8156ee0 con 0x7fcfe80603f0 2021-04-03T03:11:57.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfe7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80add50 0x7fcfe8146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:57.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfe7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80add50 0x7fcfe8146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52052/0 (socket says 172.21.2.111:52052) 2021-04-03T03:11:57.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.051+0000 7fcfe7fff700 1 -- 172.21.2.111:0/3141659756 learned_addr learned my addr 172.21.2.111:0/3141659756 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:57.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe7fff700 1 -- 172.21.2.111:0/3141659756 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfe80603f0 msgr2=0x7fcfe8066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe7fff700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfe80603f0 0x7fcfe8066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:57.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe7fff700 1 -- 172.21.2.111:0/3141659756 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfe80b17b0 msgr2=0x7fcfe8157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe7fff700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfe80b17b0 0x7fcfe8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:57.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe7fff700 1 -- 172.21.2.111:0/3141659756 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8156c00 con 0x7fcfe80add50 2021-04-03T03:11:57.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe7fff700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80add50 0x7fcfe8146a80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fcfd0002f50 tx=0x7fcfd0006db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:57.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe5ffb700 1 -- 172.21.2.111:0/3141659756 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfd000d4a0 con 0x7fcfe80add50 2021-04-03T03:11:57.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe5ffb700 1 -- 172.21.2.111:0/3141659756 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd000d600 con 0x7fcfe80add50 2021-04-03T03:11:57.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe5ffb700 1 -- 172.21.2.111:0/3141659756 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfd000d960 con 0x7fcfe80add50 2021-04-03T03:11:57.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe8146f70 con 0x7fcfe80add50 2021-04-03T03:11:57.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe80aba70 con 0x7fcfe80add50 2021-04-03T03:11:57.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe81570a0 con 0x7fcfe80add50 2021-04-03T03:11:57.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe5ffb700 1 -- 172.21.2.111:0/3141659756 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fcfd000f070 con 0x7fcfe80add50 2021-04-03T03:11:57.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe5ffb700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfcc056ea0 0x7fcfcc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfe5ffb700 1 -- 172.21.2.111:0/3141659756 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fcfd006fab0 con 0x7fcfe80add50 2021-04-03T03:11:57.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfecf55700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfcc056ea0 0x7fcfcc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:57.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.055+0000 7fcfecf55700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfcc056ea0 0x7fcfcc059360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fcfe80b5600 tx=0x7fcfd8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:57.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.059+0000 7fcfe5ffb700 1 -- 172.21.2.111:0/3141659756 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fcfd0044d30 con 0x7fcfe80add50 2021-04-03T03:11:57.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.179+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcfe8057660 con 0x7fcfe80add50 2021-04-03T03:11:57.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfe5ffb700 1 -- 172.21.2.111:0/3141659756 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcfd0048350 con 0x7fcfe80add50 2021-04-03T03:11:57.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfcc056ea0 msgr2=0x7fcfcc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfcc056ea0 0x7fcfcc059360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fcfe80b5600 tx=0x7fcfd8015000).stop 2021-04-03T03:11:57.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80add50 msgr2=0x7fcfe8146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80add50 0x7fcfe8146a80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fcfd0002f50 tx=0x7fcfd0006db0).stop 2021-04-03T03:11:57.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 shutdown_connections 2021-04-03T03:11:57.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfcc056ea0 0x7fcfcc059360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:57.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfe80b17b0 0x7fcfe8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:57.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfe80603f0 0x7fcfe8066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:57.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 --2- 172.21.2.111:0/3141659756 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfe80add50 0x7fcfe8146a80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:57.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 >> 172.21.2.111:0/3141659756 conn(0x7fcfe80b61c0 msgr2=0x7fcfe81479c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:57.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 shutdown_connections 2021-04-03T03:11:57.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.755+0000 7fcfedf57700 1 -- 172.21.2.111:0/3141659756 wait complete. 2021-04-03T03:11:57.776 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:11:57.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa3281700 1 Processor -- start 2021-04-03T03:11:57.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa3281700 1 -- start start 2021-04-03T03:11:57.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa3281700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c0add50 0x7f0f9c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa3281700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0f9c144ef0 0x7f0f9c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa3281700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9c0607d0 con 0x7f0f9c0603f0 2021-04-03T03:11:57.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa3281700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9c14b090 con 0x7f0f9c144ef0 2021-04-03T03:11:57.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa3281700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c14b330 con 0x7f0f9c0add50 2021-04-03T03:11:57.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa227f700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f0f9c0603f0 0x7f0f9c1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:37156/0 (socket says 172.21.2.111:37156) 2021-04-03T03:11:57.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa227f700 1 -- 172.21.2.111:0/4148939647 learned_addr learned my addr 172.21.2.111:0/4148939647 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:11:57.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa1a7e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c0add50 0x7f0f9c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:57.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.823+0000 7f0fa2a80700 1 --2- 172.21.2.111:0/4148939647 >> v2:172.21.2.115:3300/0 conn(0x7f0f9c144ef0 0x7f0f9c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:57.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa2a80700 1 -- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c0add50 msgr2=0x7f0f9c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:57.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa2a80700 1 --2- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c0add50 0x7f0f9c147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:57.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa2a80700 1 -- 172.21.2.111:0/4148939647 >> v1:172.21.2.115:6789/0 conn(0x7f0f9c0603f0 legacy=0x7f0f9c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa2a80700 1 -- 172.21.2.111:0/4148939647 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c14b5d0 con 0x7f0f9c144ef0 2021-04-03T03:11:57.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa1a7e700 1 --2- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c0add50 0x7f0f9c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:11:57.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa2a80700 1 --2- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c144ef0 0x7f0f9c147c30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0f940032b0 tx=0x7f0f940091d0).ready entity=mon.1 client_cookie=ea6626ef077750a8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:57.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa127d700 1 -- 172.21.2.111:0/4148939647 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f9400b9a0 con 0x7f0f9c144ef0 2021-04-03T03:11:57.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa127d700 1 -- 172.21.2.111:0/4148939647 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9400bb00 con 0x7f0f9c144ef0 2021-04-03T03:11:57.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa127d700 1 -- 172.21.2.111:0/4148939647 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f9400bc70 con 0x7f0f9c144ef0 2021-04-03T03:11:57.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c144ef0 msgr2=0x7f0f9c147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 --2- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c144ef0 0x7f0f9c147c30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0f940032b0 tx=0x7f0f940091d0).stop 2021-04-03T03:11:57.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- 172.21.2.111:0/4148939647 shutdown_connections 2021-04-03T03:11:57.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 --2- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c144ef0 0x7f0f9c147c30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:57.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 --2- 172.21.2.111:0/4148939647 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c0add50 0x7f0f9c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:57.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- 172.21.2.111:0/4148939647 >> 172.21.2.111:0/4148939647 conn(0x7f0f9c0b61c0 msgr2=0x7f0f9c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:57.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- 172.21.2.111:0/4148939647 shutdown_connections 2021-04-03T03:11:57.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- 172.21.2.111:0/4148939647 wait complete. 2021-04-03T03:11:57.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 Processor -- start 2021-04-03T03:11:57.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- start start 2021-04-03T03:11:57.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0f9c0603f0 0x7f0f9c1475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c0add50 0x7f0f9c0a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c144ef0 0x7f0f9c0ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0aa8f0 con 0x7f0f9c0603f0 2021-04-03T03:11:57.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c067ff0 con 0x7f0f9c0add50 2021-04-03T03:11:57.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa3281700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c147a90 con 0x7f0f9c144ef0 2021-04-03T03:11:57.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0f9c0603f0 0x7f0f9c1475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:57.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0f9c0603f0 0x7f0f9c1475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52068/0 (socket says 172.21.2.111:52068) 2021-04-03T03:11:57.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 -- 172.21.2.111:0/3564549408 learned_addr learned my addr 172.21.2.111:0/3564549408 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:57.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 -- 172.21.2.111:0/3564549408 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c144ef0 msgr2=0x7f0f9c0ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c144ef0 0x7f0f9c0ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:57.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 -- 172.21.2.111:0/3564549408 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c0add50 msgr2=0x7f0f9c0a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:57.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c0add50 0x7f0f9c0a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:57.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 -- 172.21.2.111:0/3564549408 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c067d50 con 0x7f0f9c0603f0 2021-04-03T03:11:57.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.827+0000 7f0fa227f700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0f9c0603f0 0x7f0f9c1475a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f0f9c0a8af0 tx=0x7f0f98007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:57.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0f8b7fe700 1 -- 172.21.2.111:0/3564549408 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f980072e0 con 0x7f0f9c0603f0 2021-04-03T03:11:57.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0f8b7fe700 1 -- 172.21.2.111:0/3564549408 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98007440 con 0x7f0f9c0603f0 2021-04-03T03:11:57.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0f8b7fe700 1 -- 172.21.2.111:0/3564549408 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f980057e0 con 0x7f0f9c0603f0 2021-04-03T03:11:57.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c0aa450 con 0x7f0f9c0603f0 2021-04-03T03:11:57.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c0ab840 con 0x7f0f9c0603f0 2021-04-03T03:11:57.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c147fd0 con 0x7f0f9c0603f0 2021-04-03T03:11:57.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0f8b7fe700 1 -- 172.21.2.111:0/3564549408 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0f98010070 con 0x7f0f9c0603f0 2021-04-03T03:11:57.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0f8b7fe700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f7c056fb0 0x7f0f7c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:57.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0f8b7fe700 1 -- 172.21.2.111:0/3564549408 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0f9806f800 con 0x7f0f9c0603f0 2021-04-03T03:11:57.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0fa1a7e700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f7c056fb0 0x7f0f7c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:57.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0fa1a7e700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f7c056fb0 0x7f0f7c059470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f0f8c00c000 tx=0x7f0f8c006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:57.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.831+0000 7f0f8b7fe700 1 -- 172.21.2.111:0/3564549408 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0f98044b90 con 0x7f0f9c0603f0 2021-04-03T03:11:57.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:57.927+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f9c0aa5e0 con 0x7f0f9c0603f0 2021-04-03T03:11:58.753 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:58.755 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:58.758 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:58.761 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:58.764 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:11:58.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0f8b7fe700 1 -- 172.21.2.111:0/3564549408 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 185+0+0 (secure 0 0 0) 0x7f0f98018020 con 0x7f0f9c0603f0 2021-04-03T03:11:58.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f7c056fb0 msgr2=0x7f0f7c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:58.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f7c056fb0 0x7f0f7c059470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f0f8c00c000 tx=0x7f0f8c006dd0).stop 2021-04-03T03:11:58.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0f9c0603f0 msgr2=0x7f0f9c1475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:58.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0f9c0603f0 0x7f0f9c1475a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f0f9c0a8af0 tx=0x7f0f98007be0).stop 2021-04-03T03:11:58.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 shutdown_connections 2021-04-03T03:11:58.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0f9c144ef0 0x7f0f9c0ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:58.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0f9c0603f0 0x7f0f9c1475a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:58.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0f7c056fb0 0x7f0f7c059470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:58.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 --2- 172.21.2.111:0/3564549408 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0f9c0add50 0x7f0f9c0a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:58.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 >> 172.21.2.111:0/3564549408 conn(0x7f0f9c0b61c0 msgr2=0x7f0f9c0b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:58.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 shutdown_connections 2021-04-03T03:11:58.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.767+0000 7f0fa3281700 1 -- 172.21.2.111:0/3564549408 wait complete. 2021-04-03T03:11:58.785 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:11:58.790 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:11:58.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 Processor -- start 2021-04-03T03:11:58.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- start start 2021-04-03T03:11:58.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbf700603f0 0x7fbf701447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:58.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 0x7fbf70144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:58.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf700607d0 con 0x7fbf700b17b0 2021-04-03T03:11:58.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf701451c0 con 0x7fbf700603f0 2021-04-03T03:11:58.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7014b090 con 0x7fbf700add50 2021-04-03T03:11:58.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbf700603f0 0x7fbf701447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:58.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbf700603f0 0x7fbf701447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52074/0 (socket says 172.21.2.111:52074) 2021-04-03T03:11:58.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 -- 172.21.2.111:0/1502288824 learned_addr learned my addr 172.21.2.111:0/1502288824 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:58.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 -- 172.21.2.111:0/1502288824 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 msgr2=0x7fbf70144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:58.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 --2- 172.21.2.111:0/1502288824 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 0x7fbf70144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:58.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 -- 172.21.2.111:0/1502288824 >> v1:172.21.2.115:6789/0 conn(0x7fbf700b17b0 legacy=0x7fbf70147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:58.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 -- 172.21.2.111:0/1502288824 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7014b330 con 0x7fbf700603f0 2021-04-03T03:11:58.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6effd700 1 --2- 172.21.2.111:0/1502288824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf701447e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fbf700b5600 tx=0x7fbf60006e90).ready entity=mon.0 client_cookie=83eba6e53887c00e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:58.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6dffb700 1 -- 172.21.2.111:0/1502288824 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf600098e0 con 0x7fbf700603f0 2021-04-03T03:11:58.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6dffb700 1 -- 172.21.2.111:0/1502288824 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60009a40 con 0x7fbf700603f0 2021-04-03T03:11:58.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6dffb700 1 -- 172.21.2.111:0/1502288824 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf60009de0 con 0x7fbf700603f0 2021-04-03T03:11:58.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- 172.21.2.111:0/1502288824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 msgr2=0x7fbf701447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:58.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/1502288824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf701447e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fbf700b5600 tx=0x7fbf60006e90).stop 2021-04-03T03:11:58.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- 172.21.2.111:0/1502288824 shutdown_connections 2021-04-03T03:11:58.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/1502288824 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf701447e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:58.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/1502288824 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 0x7fbf70144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:58.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- 172.21.2.111:0/1502288824 >> 172.21.2.111:0/1502288824 conn(0x7fbf700b61c0 msgr2=0x7fbf7005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:58.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- 172.21.2.111:0/1502288824 shutdown_connections 2021-04-03T03:11:58.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- 172.21.2.111:0/1502288824 wait complete. 2021-04-03T03:11:58.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 Processor -- start 2021-04-03T03:11:58.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.899+0000 7fbf6ffff700 1 -- start start 2021-04-03T03:11:58.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf700aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:58.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 0x7fbf70149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:58.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbf700b17b0 0x7fbf7015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:58.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70159f30 con 0x7fbf700603f0 2021-04-03T03:11:58.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70148270 con 0x7fbf700b17b0 2021-04-03T03:11:58.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70148510 con 0x7fbf700add50 2021-04-03T03:11:58.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf700aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:58.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf700aa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52080/0 (socket says 172.21.2.111:52080) 2021-04-03T03:11:58.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 -- 172.21.2.111:0/3386278973 learned_addr learned my addr 172.21.2.111:0/3386278973 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:58.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 -- 172.21.2.111:0/3386278973 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 msgr2=0x7fbf70149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:58.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 0x7fbf70149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:58.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 -- 172.21.2.111:0/3386278973 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbf700b17b0 msgr2=0x7fbf7015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:58.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbf700b17b0 0x7fbf7015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:58.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 -- 172.21.2.111:0/3386278973 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70149d10 con 0x7fbf700603f0 2021-04-03T03:11:58.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6effd700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf700aa1a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fbf700b5600 tx=0x7fbf60015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:58.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf67fff700 1 -- 172.21.2.111:0/3386278973 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf60011e30 con 0x7fbf700603f0 2021-04-03T03:11:58.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf67fff700 1 -- 172.21.2.111:0/3386278973 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6001e460 con 0x7fbf700603f0 2021-04-03T03:11:58.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf701479d0 con 0x7fbf700603f0 2021-04-03T03:11:58.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf67fff700 1 -- 172.21.2.111:0/3386278973 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6001e7c0 con 0x7fbf700603f0 2021-04-03T03:11:58.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7015a850 con 0x7fbf700603f0 2021-04-03T03:11:58.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7015a0a0 con 0x7fbf700603f0 2021-04-03T03:11:58.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf67fff700 1 -- 172.21.2.111:0/3386278973 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbf60025030 con 0x7fbf700603f0 2021-04-03T03:11:58.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf67fff700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbf54056ec0 0x7fbf54059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:58.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf67fff700 1 -- 172.21.2.111:0/3386278973 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6064+0+0 (secure 0 0 0) 0x7fbf60071420 con 0x7fbf700603f0 2021-04-03T03:11:58.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6e7fc700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbf54056ec0 0x7fbf54059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:58.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.903+0000 7fbf6e7fc700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbf54056ec0 0x7fbf54059380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fbf58002f30 tx=0x7fbf58006a90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:58.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:58.907+0000 7fbf67fff700 1 -- 172.21.2.111:0/3386278973 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbf7015a0a0 con 0x7fbf700603f0 2021-04-03T03:11:59.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.003+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbf7015a070 con 0x7fbf700603f0 2021-04-03T03:11:59.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.775+0000 7fbf67fff700 1 -- 172.21.2.111:0/3386278973 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbf600230d0 con 0x7fbf700603f0 2021-04-03T03:11:59.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbf54056ec0 msgr2=0x7fbf54059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:59.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbf54056ec0 0x7fbf54059380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fbf58002f30 tx=0x7fbf58006a90).stop 2021-04-03T03:11:59.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 msgr2=0x7fbf700aa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:59.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf700aa1a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fbf700b5600 tx=0x7fbf60015000).stop 2021-04-03T03:11:59.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 shutdown_connections 2021-04-03T03:11:59.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbf54056ec0 0x7fbf54059380 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:59.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbf700b17b0 0x7fbf7015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:59.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbf700603f0 0x7fbf700aa1a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:59.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 --2- 172.21.2.111:0/3386278973 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbf700add50 0x7fbf70149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:11:59.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 >> 172.21.2.111:0/3386278973 conn(0x7fbf700b61c0 msgr2=0x7fbf7014a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:59.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 shutdown_connections 2021-04-03T03:11:59.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.779+0000 7fbf6ffff700 1 -- 172.21.2.111:0/3386278973 wait complete. 2021-04-03T03:11:59.795 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:11:59.800 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:11:59.800 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:11:59.801 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:11:59.801 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:11:59.801 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:11:59.801 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:11:59.802 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:11:59.802 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:11:59.806 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:11:59.806 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:11:59.807 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:11:59.808 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:11:59.851 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:11:59.851 DEBUG:teuthology.parallel:result is None 2021-04-03T03:11:59.853 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:11:59.853 DEBUG:teuthology.parallel:result is None 2021-04-03T03:11:59.854 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:11:59.889 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:11:59.893 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:11:59.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 Processor -- start 2021-04-03T03:11:59.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- start start 2021-04-03T03:11:59.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0630047600 0x7f06300453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:59.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0630046550 0x7f0630046a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:59.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0630044150 con 0x7f06300d7670 2021-04-03T03:11:59.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0630044430 con 0x7f0630047600 2021-04-03T03:11:59.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06300446d0 con 0x7f0630046550 2021-04-03T03:11:59.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0630047600 0x7f06300453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:59.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0630047600 0x7f06300453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52100/0 (socket says 172.21.2.111:52100) 2021-04-03T03:11:59.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 -- 172.21.2.111:0/3828354004 learned_addr learned my addr 172.21.2.111:0/3828354004 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:59.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f06377fe700 1 --2- 172.21.2.111:0/3828354004 >> v2:172.21.2.115:3300/0 conn(0x7f0630046550 0x7f0630046a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:59.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 -- 172.21.2.111:0/3828354004 >> v2:172.21.2.115:3300/0 conn(0x7f0630046550 msgr2=0x7f0630046a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:11:59.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 --2- 172.21.2.111:0/3828354004 >> v2:172.21.2.115:3300/0 conn(0x7f0630046550 0x7f0630046a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:59.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 -- 172.21.2.111:0/3828354004 >> v1:172.21.2.111:6789/0 conn(0x7f06300d7670 legacy=0x7f06300d7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:59.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 -- 172.21.2.111:0/3828354004 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630041f40 con 0x7f0630047600 2021-04-03T03:11:59.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0637fff700 1 --2- 172.21.2.111:0/3828354004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300453b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f0638069660 tx=0x7f062c006e90).ready entity=mon.0 client_cookie=4fe4b25e1f6301b8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:59.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0636ffd700 1 -- 172.21.2.111:0/3828354004 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f062c009820 con 0x7f0630047600 2021-04-03T03:11:59.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0636ffd700 1 -- 172.21.2.111:0/3828354004 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c009980 con 0x7f0630047600 2021-04-03T03:11:59.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f0636ffd700 1 -- 172.21.2.111:0/3828354004 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f062c009ce0 con 0x7f0630047600 2021-04-03T03:11:59.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- 172.21.2.111:0/3828354004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 msgr2=0x7f06300453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:59.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 --2- 172.21.2.111:0/3828354004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300453b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f0638069660 tx=0x7f062c006e90).stop 2021-04-03T03:11:59.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- 172.21.2.111:0/3828354004 shutdown_connections 2021-04-03T03:11:59.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 --2- 172.21.2.111:0/3828354004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300453b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:59.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 --2- 172.21.2.111:0/3828354004 >> v2:172.21.2.115:3300/0 conn(0x7f0630046550 0x7f0630046a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:59.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- 172.21.2.111:0/3828354004 >> 172.21.2.111:0/3828354004 conn(0x7f0630010a50 msgr2=0x7f0630010e60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:11:59.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- 172.21.2.111:0/3828354004 shutdown_connections 2021-04-03T03:11:59.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.903+0000 7f063d77d700 1 -- 172.21.2.111:0/3828354004 wait complete. 2021-04-03T03:11:59.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 Processor -- start 2021-04-03T03:11:59.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 -- start start 2021-04-03T03:11:59.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0630046550 0x7f06300d6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:59.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:59.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06300d7670 0x7f06300ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:59.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630041360 con 0x7f0630047600 2021-04-03T03:11:59.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630040e20 con 0x7f06300d7670 2021-04-03T03:11:59.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06300410c0 con 0x7f0630046550 2021-04-03T03:11:59.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:59.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52106/0 (socket says 172.21.2.111:52106) 2021-04-03T03:11:59.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 -- 172.21.2.111:0/1388657931 learned_addr learned my addr 172.21.2.111:0/1388657931 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:11:59.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f0637fff700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0630046550 0x7f06300d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:59.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063cf7c700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06300d7670 0x7f06300ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:59.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 -- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0630046550 msgr2=0x7f06300d6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:59.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0630046550 0x7f06300d6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:59.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 -- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06300d7670 msgr2=0x7f06300ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:11:59.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06300d7670 0x7f06300ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:11:59.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 -- 172.21.2.111:0/1388657931 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300ceab0 con 0x7f0630047600 2021-04-03T03:11:59.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063cf7c700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06300d7670 0x7f06300ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:11:59.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06377fe700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300ce0d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f0628003070 tx=0x7f0628008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:59.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06357fa700 1 -- 172.21.2.111:0/1388657931 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0628009980 con 0x7f0630047600 2021-04-03T03:11:59.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06357fa700 1 -- 172.21.2.111:0/1388657931 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0628009ae0 con 0x7f0630047600 2021-04-03T03:11:59.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06357fa700 1 -- 172.21.2.111:0/1388657931 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0628009e40 con 0x7f0630047600 2021-04-03T03:11:59.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063003ebf0 con 0x7f0630047600 2021-04-03T03:11:59.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0630040ab0 con 0x7f0630047600 2021-04-03T03:11:59.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0630004d30 con 0x7f0630047600 2021-04-03T03:11:59.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06357fa700 1 -- 172.21.2.111:0/1388657931 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0628013480 con 0x7f0630047600 2021-04-03T03:11:59.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06357fa700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f06180570a0 0x7f0618059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:11:59.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f06357fa700 1 -- 172.21.2.111:0/1388657931 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f0628070840 con 0x7f0630047600 2021-04-03T03:11:59.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f0637fff700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f06180570a0 0x7f0618059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:11:59.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.907+0000 7f0637fff700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f06180570a0 0x7f0618059560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f062c01a000 tx=0x7f062c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:11:59.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:11:59.911+0000 7f06357fa700 1 -- 172.21.2.111:0/1388657931 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0628045d90 con 0x7f0630047600 2021-04-03T03:12:00.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.015+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f06300a4a00 con 0x7f0630047600 2021-04-03T03:12:00.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f06357fa700 1 -- 172.21.2.111:0/1388657931 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7f06280493b0 con 0x7f0630047600 2021-04-03T03:12:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f06180570a0 msgr2=0x7f0618059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f06180570a0 0x7f0618059560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f062c01a000 tx=0x7f062c015000).stop 2021-04-03T03:12:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 msgr2=0x7f06300ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:00.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300ce0d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f0628003070 tx=0x7f0628008040).stop 2021-04-03T03:12:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 shutdown_connections 2021-04-03T03:12:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f06180570a0 0x7f0618059560 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f06300d7670 0x7f06300ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:00.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0630046550 0x7f06300d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 --2- 172.21.2.111:0/1388657931 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0630047600 0x7f06300ce0d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 >> 172.21.2.111:0/1388657931 conn(0x7f0630010a50 msgr2=0x7f0630010e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 shutdown_connections 2021-04-03T03:12:00.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:00.895+0000 7f063d77d700 1 -- 172.21.2.111:0/1388657931 wait complete. 2021-04-03T03:12:00.911 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:12:00.915 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:12:00.916 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:12:01.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.031+0000 7ffa57fff700 1 Processor -- start 2021-04-03T03:12:01.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.031+0000 7ffa57fff700 1 -- start start 2021-04-03T03:12:01.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.031+0000 7ffa57fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ffa580b12b0 0x7ffa581522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.031+0000 7ffa57fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580607d0 con 0x7ffa5814b700 2021-04-03T03:12:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.031+0000 7ffa57fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580b0eb0 con 0x7ffa580603f0 2021-04-03T03:12:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.031+0000 7ffa57fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa580af400 con 0x7ffa580b12b0 2021-04-03T03:12:01.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa567fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ffa5814b700 0x7ffa580b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52908/0 (socket says 172.21.2.111:52908) 2021-04-03T03:12:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa567fc700 1 -- 172.21.2.111:0/2982030232 learned_addr learned my addr 172.21.2.111:0/2982030232 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871460525 0 0) 0x7ffa580607d0 con 0x7ffa5814b700 2021-04-03T03:12:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa38003240 con 0x7ffa5814b700 2021-04-03T03:12:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1075851068 0 0) 0x7ffa38003240 con 0x7ffa5814b700 2021-04-03T03:12:01.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 >> v2:172.21.2.115:3300/0 conn(0x7ffa580b12b0 msgr2=0x7ffa581522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 --2- 172.21.2.111:0/2982030232 >> v2:172.21.2.115:3300/0 conn(0x7ffa580b12b0 0x7ffa581522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 >> v1:172.21.2.115:6789/0 conn(0x7ffa580603f0 legacy=0x7ffa580b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580af6a0 con 0x7ffa5814b700 2021-04-03T03:12:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ffa480016d0 con 0x7ffa5814b700 2021-04-03T03:12:01.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 >> v1:172.21.2.111:6789/0 conn(0x7ffa5814b700 legacy=0x7ffa580b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa38003830 0x7ffa38003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa380044e0 0x7ffa38004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa38004e50 0x7ffa3800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580af400 con 0x7ffa38004e50 2021-04-03T03:12:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa38005330 con 0x7ffa38003830 2021-04-03T03:12:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3800bc20 con 0x7ffa380044e0 2021-04-03T03:12:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa38004e50 0x7ffa3800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2982030232 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa380044e0 msgr2=0x7ffa38004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa380044e0 0x7ffa38004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2982030232 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa38003830 msgr2=0x7ffa38003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa38003830 0x7ffa38003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2982030232 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38003240 con 0x7ffa38004e50 2021-04-03T03:12:01.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2982030232 reap_dead start 2021-04-03T03:12:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa38004e50 0x7ffa3800b730 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7ffa500032b0 tx=0x7ffa50009220).ready entity=mon.0 client_cookie=e5ea1925c08d334f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5000bb20 con 0x7ffa38004e50 2021-04-03T03:12:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5000bc80 con 0x7ffa38004e50 2021-04-03T03:12:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa55ffb700 1 -- 172.21.2.111:0/2982030232 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa50011440 con 0x7ffa38004e50 2021-04-03T03:12:01.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- 172.21.2.111:0/2982030232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa38004e50 msgr2=0x7ffa3800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa38004e50 0x7ffa3800b730 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7ffa500032b0 tx=0x7ffa50009220).stop 2021-04-03T03:12:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- 172.21.2.111:0/2982030232 shutdown_connections 2021-04-03T03:12:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2982030232 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa38004e50 0x7ffa3800b730 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- 172.21.2.111:0/2982030232 >> 172.21.2.111:0/2982030232 conn(0x7ffa580b61c0 msgr2=0x7ffa5805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- 172.21.2.111:0/2982030232 shutdown_connections 2021-04-03T03:12:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- 172.21.2.111:0/2982030232 wait complete. 2021-04-03T03:12:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 Processor -- start 2021-04-03T03:12:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- start start 2021-04-03T03:12:01.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa38004e50 0x7ffa580a7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa580aa3f0 0x7ffa581469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa58146eb0 0x7ffa580641c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58062e80 con 0x7ffa58146eb0 2021-04-03T03:12:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580aba50 con 0x7ffa580aa3f0 2021-04-03T03:12:01.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580a8490 con 0x7ffa38004e50 2021-04-03T03:12:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa58146eb0 0x7ffa580641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa58146eb0 0x7ffa580641c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52128/0 (socket says 172.21.2.111:52128) 2021-04-03T03:12:01.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2005816613 learned_addr learned my addr 172.21.2.111:0/2005816613 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa56ffd700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa38004e50 0x7ffa580a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa567fc700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa580aa3f0 0x7ffa581469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa38004e50 msgr2=0x7ffa580a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa38004e50 0x7ffa580a7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa580aa3f0 msgr2=0x7ffa581469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa580aa3f0 0x7ffa581469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 -- 172.21.2.111:0/2005816613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa581596f0 con 0x7ffa58146eb0 2021-04-03T03:12:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa577fe700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa58146eb0 0x7ffa580641c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7ffa500032b0 tx=0x7ffa50016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa4ffff700 1 -- 172.21.2.111:0/2005816613 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5001c610 con 0x7ffa58146eb0 2021-04-03T03:12:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa4ffff700 1 -- 172.21.2.111:0/2005816613 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5001c770 con 0x7ffa58146eb0 2021-04-03T03:12:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa58147360 con 0x7ffa58146eb0 2021-04-03T03:12:01.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa4ffff700 1 -- 172.21.2.111:0/2005816613 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5001cb10 con 0x7ffa58146eb0 2021-04-03T03:12:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.035+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa580aa180 con 0x7ffa58146eb0 2021-04-03T03:12:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.039+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa580ab890 con 0x7ffa58146eb0 2021-04-03T03:12:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.039+0000 7ffa4ffff700 1 -- 172.21.2.111:0/2005816613 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ffa5000f070 con 0x7ffa58146eb0 2021-04-03T03:12:01.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.039+0000 7ffa4ffff700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffa3c056f60 0x7ffa3c059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.039+0000 7ffa4ffff700 1 -- 172.21.2.111:0/2005816613 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5596+0+0 (secure 0 0 0) 0x7ffa50071720 con 0x7ffa58146eb0 2021-04-03T03:12:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.039+0000 7ffa56ffd700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffa3c056f60 0x7ffa3c059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.039+0000 7ffa56ffd700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffa3c056f60 0x7ffa3c059420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ffa40007000 tx=0x7ffa40008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.039+0000 7ffa4ffff700 1 -- 172.21.2.111:0/2005816613 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ffa50046d30 con 0x7ffa58146eb0 2021-04-03T03:12:01.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.167+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffa58057660 con 0x7ffa58146eb0 2021-04-03T03:12:01.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.167+0000 7ffa4ffff700 1 -- 172.21.2.111:0/2005816613 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4242 (secure 0 0 0) 0x7ffa5004a350 con 0x7ffa58146eb0 2021-04-03T03:12:01.309 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:01.309 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:12:01.310 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:12:01.310 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.310 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:12:01.310 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:12:01.310 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.311 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:12:01.311 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.311 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.311 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.312 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.312 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.312 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.313 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:12:01.313 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:12:01.313 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.313 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:12:01.314 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.314 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.314 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.314 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.314 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.315 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.315 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:12:01.315 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:12:01.315 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.316 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.316 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.316 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.316 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.317 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.317 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.317 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:12:01.317 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:12:01.317 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.318 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.318 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.318 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.318 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.319 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.319 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.319 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:12:01.320 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:12:01.320 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.320 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.320 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.321 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.321 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.321 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.321 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.321 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:12:01.322 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:12:01.322 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.322 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.322 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.323 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.323 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.323 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.323 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.324 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:12:01.324 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:12:01.324 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.324 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.325 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.325 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.325 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.325 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.325 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.326 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:12:01.326 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:12:01.327 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.327 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.327 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.327 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.328 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.328 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.328 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.328 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:12:01.328 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:12:01.329 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.329 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.329 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.329 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.330 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.330 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.330 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.330 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:12:01.331 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:12:01.331 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.331 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.331 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.331 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.332 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.332 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.333 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.333 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:12:01.333 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:12:01.333 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.334 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:12:01.334 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:12:01.334 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:12:01.334 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.335 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.335 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.335 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:12:01.335 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:12:01.336 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.336 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:12:01.336 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:12:01.336 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:12:01.336 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.337 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.337 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.337 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:12:01.337 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:12:01.338 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.338 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:12:01.338 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:12:01.338 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:12:01.338 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.339 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.339 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.340 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:12:01.340 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:12:01.340 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.340 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:12:01.341 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:12:01.341 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:12:01.341 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.341 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.342 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.342 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.342 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:12:01.342 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:12:01.343 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.343 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:12:01.343 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:12:01.343 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.344 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.344 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:01.344 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:01.344 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:12:01.344 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:12:01.345 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:12:01.345 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:12:01.345 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:12:01.346 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:12:01.346 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.346 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:01.346 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:12:01.347 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:12:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffa3c056f60 msgr2=0x7ffa3c059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffa3c056f60 0x7ffa3c059420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ffa40007000 tx=0x7ffa40008040).stop 2021-04-03T03:12:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa58146eb0 msgr2=0x7ffa580641c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa58146eb0 0x7ffa580641c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7ffa500032b0 tx=0x7ffa50016000).stop 2021-04-03T03:12:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 shutdown_connections 2021-04-03T03:12:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffa58146eb0 0x7ffa580641c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffa38004e50 0x7ffa580a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffa3c056f60 0x7ffa3c059420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 --2- 172.21.2.111:0/2005816613 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffa580aa3f0 0x7ffa581469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 >> 172.21.2.111:0/2005816613 conn(0x7ffa580b61c0 msgr2=0x7ffa580acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 shutdown_connections 2021-04-03T03:12:01.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.171+0000 7ffa57fff700 1 -- 172.21.2.111:0/2005816613 wait complete. 2021-04-03T03:12:01.352 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:12:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 Processor -- start 2021-04-03T03:12:01.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 -- start start 2021-04-03T03:12:01.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fac1c0600d0 0x7fac1c0604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 0x7fac1c148170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fac1c0ad170 0x7fac1c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c147a30 con 0x7fac1c0600d0 2021-04-03T03:12:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c147cd0 con 0x7fac1c0ad170 2021-04-03T03:12:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c147760 con 0x7fac1c0b54d0 2021-04-03T03:12:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fac1c0600d0 0x7fac1c0604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fac1c0600d0 0x7fac1c0604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52132/0 (socket says 172.21.2.111:52132) 2021-04-03T03:12:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 -- 172.21.2.111:0/2121334960 learned_addr learned my addr 172.21.2.111:0/2121334960 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 -- 172.21.2.111:0/2121334960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 msgr2=0x7fac1c148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 --2- 172.21.2.111:0/2121334960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 0x7fac1c148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 -- 172.21.2.111:0/2121334960 >> v2:172.21.2.115:3300/0 conn(0x7fac1c0ad170 msgr2=0x7fac1c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 --2- 172.21.2.111:0/2121334960 >> v2:172.21.2.115:3300/0 conn(0x7fac1c0ad170 0x7fac1c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 -- 172.21.2.111:0/2121334960 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c148660 con 0x7fac1c0600d0 2021-04-03T03:12:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1a59c700 1 --2- 172.21.2.111:0/2121334960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c0604b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fac0c002f30 tx=0x7fac0c006e90).ready entity=mon.0 client_cookie=4a8bdc5b9e31d803 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1959a700 1 -- 172.21.2.111:0/2121334960 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0c009920 con 0x7fac1c0600d0 2021-04-03T03:12:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1959a700 1 -- 172.21.2.111:0/2121334960 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c009a80 con 0x7fac1c0600d0 2021-04-03T03:12:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1959a700 1 -- 172.21.2.111:0/2121334960 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0c009de0 con 0x7fac1c0600d0 2021-04-03T03:12:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 -- 172.21.2.111:0/2121334960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 msgr2=0x7fac1c0604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.399+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2121334960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c0604b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fac0c002f30 tx=0x7fac0c006e90).stop 2021-04-03T03:12:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- 172.21.2.111:0/2121334960 shutdown_connections 2021-04-03T03:12:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2121334960 >> v2:172.21.2.115:3300/0 conn(0x7fac1c0ad170 0x7fac1c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2121334960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c0604b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2121334960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 0x7fac1c148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- 172.21.2.111:0/2121334960 >> 172.21.2.111:0/2121334960 conn(0x7fac1c05c7d0 msgr2=0x7fac1c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- 172.21.2.111:0/2121334960 shutdown_connections 2021-04-03T03:12:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- 172.21.2.111:0/2121334960 wait complete. 2021-04-03T03:12:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 Processor -- start 2021-04-03T03:12:01.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- start start 2021-04-03T03:12:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c065880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fac1c0ad170 0x7fac1c0637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 0x7fac1c063c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0663e0 con 0x7fac1c0600d0 2021-04-03T03:12:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c066e50 con 0x7fac1c0ad170 2021-04-03T03:12:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c066140 con 0x7fac1c0b54d0 2021-04-03T03:12:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c065880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c065880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52138/0 (socket says 172.21.2.111:52138) 2021-04-03T03:12:01.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 -- 172.21.2.111:0/2959951247 learned_addr learned my addr 172.21.2.111:0/2959951247 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 -- 172.21.2.111:0/2959951247 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 msgr2=0x7fac1c063c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 0x7fac1c063c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 -- 172.21.2.111:0/2959951247 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fac1c0ad170 msgr2=0x7fac1c0637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fac1c0ad170 0x7fac1c0637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 -- 172.21.2.111:0/2959951247 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c14b4d0 con 0x7fac1c0600d0 2021-04-03T03:12:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1a59c700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c065880 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fac0c002f30 tx=0x7fac0c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac0b7fe700 1 -- 172.21.2.111:0/2959951247 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0c011e30 con 0x7fac1c0600d0 2021-04-03T03:12:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac0b7fe700 1 -- 172.21.2.111:0/2959951247 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c01e460 con 0x7fac1c0600d0 2021-04-03T03:12:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac0b7fe700 1 -- 172.21.2.111:0/2959951247 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0c01e7c0 con 0x7fac1c0600d0 2021-04-03T03:12:01.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c064180 con 0x7fac1c0600d0 2021-04-03T03:12:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c0672e0 con 0x7fac1c0600d0 2021-04-03T03:12:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c065f80 con 0x7fac1c0600d0 2021-04-03T03:12:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac0b7fe700 1 -- 172.21.2.111:0/2959951247 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fac0c025030 con 0x7fac1c0600d0 2021-04-03T03:12:01.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac0b7fe700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fabf8057000 0x7fabf80594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac0b7fe700 1 -- 172.21.2.111:0/2959951247 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fac0c0714a0 con 0x7fac1c0600d0 2021-04-03T03:12:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.403+0000 7fac19d9b700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fabf8057000 0x7fabf80594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.407+0000 7fac19d9b700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fabf8057000 0x7fabf80594c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fac10002f30 tx=0x7fac10006970).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.407+0000 7fac0b7fe700 1 -- 172.21.2.111:0/2959951247 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fac0c046bc0 con 0x7fac1c0600d0 2021-04-03T03:12:01.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac1c057660 con 0x7fac1c0600d0 2021-04-03T03:12:01.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac0b7fe700 1 -- 172.21.2.111:0/2959951247 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13414 (secure 0 0 0) 0x7fac0c023090 con 0x7fac1c0600d0 2021-04-03T03:12:01.513 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:01.514 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":48,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:12:00.891276+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:12:01.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fabf8057000 msgr2=0x7fabf80594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fabf8057000 0x7fabf80594c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fac10002f30 tx=0x7fac10006970).stop 2021-04-03T03:12:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 msgr2=0x7fac1c065880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c065880 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fac0c002f30 tx=0x7fac0c015000).stop 2021-04-03T03:12:01.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 shutdown_connections 2021-04-03T03:12:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fac1c0b54d0 0x7fac1c063c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fac1c0600d0 0x7fac1c065880 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fabf8057000 0x7fabf80594c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 --2- 172.21.2.111:0/2959951247 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fac1c0ad170 0x7fac1c0637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 >> 172.21.2.111:0/2959951247 conn(0x7fac1c05c7d0 msgr2=0x7fac1c0b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 shutdown_connections 2021-04-03T03:12:01.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.499+0000 7fac1b59e700 1 -- 172.21.2.111:0/2959951247 wait complete. 2021-04-03T03:12:01.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:12:01.522 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:12:01.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 Processor -- start 2021-04-03T03:12:01.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- start start 2021-04-03T03:12:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc03f880 0x7f7dbc0d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7dbc039dd0 0x7f7dbc03fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dbc0d1c70 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dbc040150 con 0x7f7dbc039dd0 2021-04-03T03:12:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0d2f00 con 0x7f7dbc03f880 2021-04-03T03:12:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc3fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7dbc0ceed0 0x7f7dbc005680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52936/0 (socket says 172.21.2.111:52936) 2021-04-03T03:12:01.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7dbc039dd0 0x7f7dbc03fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc3fff700 1 -- 172.21.2.111:0/1116773464 learned_addr learned my addr 172.21.2.111:0/1116773464 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc2ffd700 1 -- 172.21.2.111:0/1116773464 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc03f880 msgr2=0x7f7dbc0d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc2ffd700 1 --2- 172.21.2.111:0/1116773464 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc03f880 0x7f7dbc0d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc2ffd700 1 -- 172.21.2.111:0/1116773464 >> v1:172.21.2.111:6789/0 conn(0x7f7dbc0ceed0 legacy=0x7f7dbc005680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc2ffd700 1 -- 172.21.2.111:0/1116773464 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0d1ea0 con 0x7f7dbc039dd0 2021-04-03T03:12:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc2ffd700 1 --2- 172.21.2.111:0/1116773464 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc039dd0 0x7f7dbc03fc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f7db4002f30 tx=0x7f7db4006dd0).ready entity=mon.0 client_cookie=ce1de311dfd93d1e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc27fc700 1 -- 172.21.2.111:0/1116773464 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7db40097f0 con 0x7f7dbc039dd0 2021-04-03T03:12:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc27fc700 1 -- 172.21.2.111:0/1116773464 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4009950 con 0x7f7dbc039dd0 2021-04-03T03:12:01.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc27fc700 1 -- 172.21.2.111:0/1116773464 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7db4009cb0 con 0x7f7dbc039dd0 2021-04-03T03:12:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- 172.21.2.111:0/1116773464 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc039dd0 msgr2=0x7f7dbc03fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 --2- 172.21.2.111:0/1116773464 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc039dd0 0x7f7dbc03fc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f7db4002f30 tx=0x7f7db4006dd0).stop 2021-04-03T03:12:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- 172.21.2.111:0/1116773464 shutdown_connections 2021-04-03T03:12:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 --2- 172.21.2.111:0/1116773464 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc03f880 0x7f7dbc0d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 --2- 172.21.2.111:0/1116773464 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc039dd0 0x7f7dbc03fc60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- 172.21.2.111:0/1116773464 >> 172.21.2.111:0/1116773464 conn(0x7f7dbc0462b0 msgr2=0x7f7dbc0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- 172.21.2.111:0/1116773464 shutdown_connections 2021-04-03T03:12:01.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.567+0000 7f7dc897e700 1 -- 172.21.2.111:0/1116773464 wait complete. 2021-04-03T03:12:01.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 Processor -- start 2021-04-03T03:12:01.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 -- start start 2021-04-03T03:12:01.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc039dd0 0x7f7dbc0d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7dbc03f880 0x7f7dbc044ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc0ceed0 0x7f7dbc0eab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0d5090 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0d5330 con 0x7f7dbc03f880 2021-04-03T03:12:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0d5620 con 0x7f7dbc039dd0 2021-04-03T03:12:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc0ceed0 0x7f7dbc0eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc0ceed0 0x7f7dbc0eab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52152/0 (socket says 172.21.2.111:52152) 2021-04-03T03:12:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 -- 172.21.2.111:0/610176227 learned_addr learned my addr 172.21.2.111:0/610176227 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:01.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 -- 172.21.2.111:0/610176227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc039dd0 msgr2=0x7f7dbc0d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc039dd0 0x7f7dbc0d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 -- 172.21.2.111:0/610176227 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7dbc03f880 msgr2=0x7f7dbc044ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7dbc03f880 0x7f7dbc044ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 -- 172.21.2.111:0/610176227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0ce770 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc3fff700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc0ceed0 0x7f7dbc0eab50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f7dbc039850 tx=0x7f7dc405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc0ff9700 1 -- 172.21.2.111:0/610176227 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dc405df60 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc0ff9700 1 -- 172.21.2.111:0/610176227 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc40699f0 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc0ff9700 1 -- 172.21.2.111:0/610176227 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dc406b5d0 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc03d860 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc03d560 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc0d57e0 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc0ff9700 1 -- 172.21.2.111:0/610176227 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7dc4061110 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc0ff9700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7da4056fb0 0x7f7da4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc0ff9700 1 -- 172.21.2.111:0/610176227 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f7dc40dea30 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc37fe700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7da4056fb0 0x7f7da4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.571+0000 7f7dc37fe700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7da4056fb0 0x7f7da4059470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f7db8002f30 tx=0x7f7db8006a50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.575+0000 7f7dc0ff9700 1 -- 172.21.2.111:0/610176227 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7dc40b4040 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7dbc004db0 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc0ff9700 1 -- 172.21.2.111:0/610176227 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v48) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7dc40b7660 con 0x7f7dbc0ceed0 2021-04-03T03:12:01.679 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:12:01.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7da4056fb0 msgr2=0x7f7da4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7da4056fb0 0x7f7da4059470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f7db8002f30 tx=0x7f7db8006a50).stop 2021-04-03T03:12:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc0ceed0 msgr2=0x7f7dbc0eab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc0ceed0 0x7f7dbc0eab50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f7dbc039850 tx=0x7f7dc405ad70).stop 2021-04-03T03:12:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 shutdown_connections 2021-04-03T03:12:01.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7dbc0ceed0 0x7f7dbc0eab50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7dbc039dd0 0x7f7dbc0d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7da4056fb0 0x7f7da4059470 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 --2- 172.21.2.111:0/610176227 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7dbc03f880 0x7f7dbc044ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 >> 172.21.2.111:0/610176227 conn(0x7f7dbc0462b0 msgr2=0x7f7dbc0441b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 shutdown_connections 2021-04-03T03:12:01.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.667+0000 7f7dc897e700 1 -- 172.21.2.111:0/610176227 wait complete. 2021-04-03T03:12:01.687 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:12:01.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8de005700 1 Processor -- start 2021-04-03T03:12:01.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8de005700 1 -- start start 2021-04-03T03:12:01.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8de005700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd8d00dbba0 0x7fd8d00dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8de005700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00dc560 con 0x7fd8d0041060 2021-04-03T03:12:01.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8de005700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00dc800 con 0x7fd8d003d560 2021-04-03T03:12:01.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8de005700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d003de60 con 0x7fd8d00dbba0 2021-04-03T03:12:01.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8dc802700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd8d0041060 0x7fd8d003fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52948/0 (socket says 172.21.2.111:52948) 2021-04-03T03:12:01.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8dc802700 1 -- 172.21.2.111:0/1542256273 learned_addr learned my addr 172.21.2.111:0/1542256273 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:01.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8dd003700 1 --2- 172.21.2.111:0/1542256273 >> v2:172.21.2.111:3300/0 conn(0x7fd8d00dbba0 0x7fd8d00dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8dd003700 1 -- 172.21.2.111:0/1542256273 >> v1:172.21.2.115:6789/0 conn(0x7fd8d003d560 legacy=0x7fd8d0041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8dd003700 1 -- 172.21.2.111:0/1542256273 >> v1:172.21.2.111:6789/0 conn(0x7fd8d0041060 legacy=0x7fd8d003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8dd003700 1 -- 172.21.2.111:0/1542256273 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d003da30 con 0x7fd8d00dbba0 2021-04-03T03:12:01.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.731+0000 7fd8cbfff700 1 -- 172.21.2.111:0/1542256273 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981955782 0 0) 0x7fd8d00dc560 con 0x7fd8d0041060 2021-04-03T03:12:01.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dd003700 1 --2- 172.21.2.111:0/1542256273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d00dbba0 0x7fd8d00dbf80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fd8d0040a80 tx=0x7fd8cc006dd0).ready entity=mon.0 client_cookie=2f7e7d5b74c8f9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8cbfff700 1 -- 172.21.2.111:0/1542256273 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8cc0097b0 con 0x7fd8d00dbba0 2021-04-03T03:12:01.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8cbfff700 1 -- 172.21.2.111:0/1542256273 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc009910 con 0x7fd8d00dbba0 2021-04-03T03:12:01.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8cbfff700 1 -- 172.21.2.111:0/1542256273 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8cc009c70 con 0x7fd8d00dbba0 2021-04-03T03:12:01.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1542256273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d00dbba0 msgr2=0x7fd8d00dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 --2- 172.21.2.111:0/1542256273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d00dbba0 0x7fd8d00dbf80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fd8d0040a80 tx=0x7fd8cc006dd0).stop 2021-04-03T03:12:01.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1542256273 shutdown_connections 2021-04-03T03:12:01.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 --2- 172.21.2.111:0/1542256273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d00dbba0 0x7fd8d00dbf80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1542256273 >> 172.21.2.111:0/1542256273 conn(0x7fd8d00462b0 msgr2=0x7fd8d00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1542256273 shutdown_connections 2021-04-03T03:12:01.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1542256273 wait complete. 2021-04-03T03:12:01.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 Processor -- start 2021-04-03T03:12:01.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- start start 2021-04-03T03:12:01.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd8d003d560 0x7fd8d00de130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d0041060 0x7fd8d00a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8d00dbba0 0x7fd8d00a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d009f5f0 con 0x7fd8d0041060 2021-04-03T03:12:01.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d003bf70 con 0x7fd8d003d560 2021-04-03T03:12:01.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d009f800 con 0x7fd8d00dbba0 2021-04-03T03:12:01.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d0041060 0x7fd8d00a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d0041060 0x7fd8d00a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52164/0 (socket says 172.21.2.111:52164) 2021-04-03T03:12:01.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 -- 172.21.2.111:0/1098272334 learned_addr learned my addr 172.21.2.111:0/1098272334 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:01.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dd804700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8d00dbba0 0x7fd8d00a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 -- 172.21.2.111:0/1098272334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8d00dbba0 msgr2=0x7fd8d00a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8d00dbba0 0x7fd8d00a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 -- 172.21.2.111:0/1098272334 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd8d003d560 msgr2=0x7fd8d00de130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd8d003d560 0x7fd8d00de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 -- 172.21.2.111:0/1098272334 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d009faa0 con 0x7fd8d0041060 2021-04-03T03:12:01.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8dc802700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d0041060 0x7fd8d00a0cb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fd8d4000b80 tx=0x7fd8d4007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8ca7fc700 1 -- 172.21.2.111:0/1098272334 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d4007320 con 0x7fd8d0041060 2021-04-03T03:12:01.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8ca7fc700 1 -- 172.21.2.111:0/1098272334 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4007480 con 0x7fd8d0041060 2021-04-03T03:12:01.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8ca7fc700 1 -- 172.21.2.111:0/1098272334 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d40057e0 con 0x7fd8d0041060 2021-04-03T03:12:01.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d00e6fe0 con 0x7fd8d0041060 2021-04-03T03:12:01.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d00a3230 con 0x7fd8d0041060 2021-04-03T03:12:01.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.735+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d00d79f0 con 0x7fd8d0041060 2021-04-03T03:12:01.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.739+0000 7fd8ca7fc700 1 -- 172.21.2.111:0/1098272334 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd8d4010030 con 0x7fd8d0041060 2021-04-03T03:12:01.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.739+0000 7fd8ca7fc700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd8b8057050 0x7fd8b8059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.739+0000 7fd8ca7fc700 1 -- 172.21.2.111:0/1098272334 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fd8d406fba0 con 0x7fd8d0041060 2021-04-03T03:12:01.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.739+0000 7fd8dd003700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd8b8057050 0x7fd8b8059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.739+0000 7fd8dd003700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd8b8057050 0x7fd8b8059510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fd8d0040a80 tx=0x7fd8cc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.739+0000 7fd8ca7fc700 1 -- 172.21.2.111:0/1098272334 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd8d40452c0 con 0x7fd8d0041060 2021-04-03T03:12:01.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd8d0004db0 con 0x7fd8d0041060 2021-04-03T03:12:01.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8ca7fc700 1 -- 172.21.2.111:0/1098272334 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd8d40488e0 con 0x7fd8d0041060 2021-04-03T03:12:01.849 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:12:01.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd8b8057050 msgr2=0x7fd8b8059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd8b8057050 0x7fd8b8059510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fd8d0040a80 tx=0x7fd8cc015000).stop 2021-04-03T03:12:01.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d0041060 msgr2=0x7fd8d00a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d0041060 0x7fd8d00a0cb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fd8d4000b80 tx=0x7fd8d4007a40).stop 2021-04-03T03:12:01.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 shutdown_connections 2021-04-03T03:12:01.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8d00dbba0 0x7fd8d00a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd8b8057050 0x7fd8b8059510 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd8d003d560 0x7fd8d00de130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 --2- 172.21.2.111:0/1098272334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8d0041060 0x7fd8d00a0cb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 >> 172.21.2.111:0/1098272334 conn(0x7fd8d00462b0 msgr2=0x7fd8d0041570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 shutdown_connections 2021-04-03T03:12:01.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.835+0000 7fd8de005700 1 -- 172.21.2.111:0/1098272334 wait complete. 2021-04-03T03:12:01.857 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:12:01.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.899+0000 7f4dd3fff700 1 Processor -- start 2021-04-03T03:12:01.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.899+0000 7f4dd3fff700 1 -- start start 2021-04-03T03:12:01.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.899+0000 7f4dd3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4dcc026960 0x7f4dcc043cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.899+0000 7f4dd3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4dcc041d70 0x7f4dcc0441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.899+0000 7f4dd3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dcc029cd0 con 0x7f4dcc007e90 2021-04-03T03:12:01.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.899+0000 7f4dd3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dcc008270 con 0x7f4dcc026960 2021-04-03T03:12:01.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.899+0000 7f4dd3fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dcc0286b0 con 0x7f4dcc041d70 2021-04-03T03:12:01.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd27fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4dcc026960 0x7f4dcc043cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd2ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4dcc007e90 0x7f4dcc028df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52962/0 (socket says 172.21.2.111:52962) 2021-04-03T03:12:01.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd2ffd700 1 -- 172.21.2.111:0/636899991 learned_addr learned my addr 172.21.2.111:0/636899991 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:01.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd1ffb700 1 -- 172.21.2.111:0/636899991 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980630849 0 0) 0x7f4dcc029cd0 con 0x7f4dcc007e90 2021-04-03T03:12:01.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd1ffb700 1 -- 172.21.2.111:0/636899991 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db4003240 con 0x7f4dcc007e90 2021-04-03T03:12:01.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd27fc700 1 -- 172.21.2.111:0/636899991 >> v2:172.21.2.115:3300/0 conn(0x7f4dcc041d70 msgr2=0x7f4dcc0441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:01.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd27fc700 1 --2- 172.21.2.111:0/636899991 >> v2:172.21.2.115:3300/0 conn(0x7f4dcc041d70 0x7f4dcc0441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd27fc700 1 -- 172.21.2.111:0/636899991 >> v1:172.21.2.111:6789/0 conn(0x7f4dcc007e90 legacy=0x7f4dcc028df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd27fc700 1 -- 172.21.2.111:0/636899991 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc028950 con 0x7f4dcc026960 2021-04-03T03:12:01.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd27fc700 1 --2- 172.21.2.111:0/636899991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc026960 0x7f4dcc043cf0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f4dc4002f30 tx=0x7f4dc4006dd0).ready entity=mon.0 client_cookie=aae01ba212efa5f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd1ffb700 1 -- 172.21.2.111:0/636899991 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dc40097b0 con 0x7f4dcc026960 2021-04-03T03:12:01.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd1ffb700 1 -- 172.21.2.111:0/636899991 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc4009910 con 0x7f4dcc026960 2021-04-03T03:12:01.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd1ffb700 1 -- 172.21.2.111:0/636899991 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dc4009c70 con 0x7f4dcc026960 2021-04-03T03:12:01.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/636899991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc026960 msgr2=0x7f4dcc043cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:01.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/636899991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc026960 0x7f4dcc043cf0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f4dc4002f30 tx=0x7f4dc4006dd0).stop 2021-04-03T03:12:01.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/636899991 shutdown_connections 2021-04-03T03:12:01.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/636899991 >> v2:172.21.2.115:3300/0 conn(0x7f4dcc041d70 0x7f4dcc0441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/636899991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc026960 0x7f4dcc043cf0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:01.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/636899991 >> 172.21.2.111:0/636899991 conn(0x7f4dcc00e220 msgr2=0x7f4dcc007460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:01.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/636899991 shutdown_connections 2021-04-03T03:12:01.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/636899991 wait complete. 2021-04-03T03:12:01.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 Processor -- start 2021-04-03T03:12:01.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- start start 2021-04-03T03:12:01.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dcc007e90 0x7f4dcc021a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dcc026960 0x7f4dcc01d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc041d70 0x7f4dcc006350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc01e1f0 con 0x7f4dcc041d70 2021-04-03T03:12:01.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc01dcb0 con 0x7f4dcc007e90 2021-04-03T03:12:01.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc01df50 con 0x7f4dcc026960 2021-04-03T03:12:01.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc041d70 0x7f4dcc006350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc041d70 0x7f4dcc006350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52178/0 (socket says 172.21.2.111:52178) 2021-04-03T03:12:01.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 -- 172.21.2.111:0/1323561847 learned_addr learned my addr 172.21.2.111:0/1323561847 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:01.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 -- 172.21.2.111:0/1323561847 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dcc026960 msgr2=0x7f4dcc01d610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:12:01.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dcc026960 0x7f4dcc01d610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 -- 172.21.2.111:0/1323561847 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dcc007e90 msgr2=0x7f4dcc021a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:12:01.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dcc007e90 0x7f4dcc021a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:01.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 -- 172.21.2.111:0/1323561847 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc006840 con 0x7f4dcc041d70 2021-04-03T03:12:01.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd37fe700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc041d70 0x7f4dcc006350 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f4dd406b5d0 tx=0x7f4dd405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dcbfff700 1 -- 172.21.2.111:0/1323561847 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dd40699f0 con 0x7f4dcc041d70 2021-04-03T03:12:01.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dcbfff700 1 -- 172.21.2.111:0/1323561847 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd405f2c0 con 0x7f4dcc041d70 2021-04-03T03:12:01.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc006a90 con 0x7f4dcc041d70 2021-04-03T03:12:01.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dcbfff700 1 -- 172.21.2.111:0/1323561847 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dd405f620 con 0x7f4dcc041d70 2021-04-03T03:12:01.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc0e4120 con 0x7f4dcc041d70 2021-04-03T03:12:01.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc01db00 con 0x7f4dcc041d70 2021-04-03T03:12:01.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.903+0000 7f4dcbfff700 1 -- 172.21.2.111:0/1323561847 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4dd406adb0 con 0x7f4dcc041d70 2021-04-03T03:12:01.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.907+0000 7f4dcbfff700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4db8056fb0 0x7f4db8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:01.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.907+0000 7f4dcbfff700 1 -- 172.21.2.111:0/1323561847 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f4dd40def70 con 0x7f4dcc041d70 2021-04-03T03:12:01.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.907+0000 7f4dd2ffd700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4db8056fb0 0x7f4db8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:01.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.907+0000 7f4dd2ffd700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4db8056fb0 0x7f4db8059470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f4dbc0010d0 tx=0x7f4dbc007610).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:01.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:01.907+0000 7f4dcbfff700 1 -- 172.21.2.111:0/1323561847 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4dd40b45d0 con 0x7f4dcc041d70 2021-04-03T03:12:02.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4dcc01e480 con 0x7f4dcc041d70 2021-04-03T03:12:02.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dcbfff700 1 -- 172.21.2.111:0/1323561847 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4dd40b7bf0 con 0x7f4dcc041d70 2021-04-03T03:12:02.016 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:02.017 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:12:02.017 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:02.017 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:12:02.017 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:12:02.017 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:02.018 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:12:02.018 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:12:02.018 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:12:02.018 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:02.019 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:12:02.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4db8056fb0 msgr2=0x7f4db8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4db8056fb0 0x7f4db8059470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f4dbc0010d0 tx=0x7f4dbc007610).stop 2021-04-03T03:12:02.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc041d70 msgr2=0x7f4dcc006350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc041d70 0x7f4dcc006350 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f4dd406b5d0 tx=0x7f4dd405ad70).stop 2021-04-03T03:12:02.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 shutdown_connections 2021-04-03T03:12:02.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4dcc041d70 0x7f4dcc006350 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4db8056fb0 0x7f4db8059470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4dcc007e90 0x7f4dcc021a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 --2- 172.21.2.111:0/1323561847 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4dcc026960 0x7f4dcc01d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 >> 172.21.2.111:0/1323561847 conn(0x7f4dcc00e220 msgr2=0x7f4dcc02c100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:02.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 shutdown_connections 2021-04-03T03:12:02.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.003+0000 7f4dd3fff700 1 -- 172.21.2.111:0/1323561847 wait complete. 2021-04-03T03:12:02.024 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:12:02.025 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:12:02.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.067+0000 7f3d97386700 1 Processor -- start 2021-04-03T03:12:02.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.067+0000 7f3d97386700 1 -- start start 2021-04-03T03:12:02.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3d980b3ae0 0x7f3d98063c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d980b2880 0x7f3d981435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d980b3ec0 con 0x7f3d98066250 2021-04-03T03:12:02.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d980b2d00 con 0x7f3d980b3ae0 2021-04-03T03:12:02.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98064250 con 0x7f3d980b2880 2021-04-03T03:12:02.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3d980b3ae0 0x7f3d98063c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:02.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3d980b3ae0 0x7f3d98063c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52182/0 (socket says 172.21.2.111:52182) 2021-04-03T03:12:02.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 -- 172.21.2.111:0/2023776347 learned_addr learned my addr 172.21.2.111:0/2023776347 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:02.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 --2- 172.21.2.111:0/2023776347 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d980b2880 0x7f3d981435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:02.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 -- 172.21.2.111:0/2023776347 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d980b2880 msgr2=0x7f3d981435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:02.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 --2- 172.21.2.111:0/2023776347 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d980b2880 0x7f3d981435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 -- 172.21.2.111:0/2023776347 >> v1:172.21.2.115:6789/0 conn(0x7f3d98066250 legacy=0x7f3d9813cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:02.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 -- 172.21.2.111:0/2023776347 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d98065db0 con 0x7f3d980b3ae0 2021-04-03T03:12:02.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d96384700 1 --2- 172.21.2.111:0/2023776347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b3ae0 0x7f3d98063c90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f3d980b45f0 tx=0x7f3d88009270).ready entity=mon.0 client_cookie=c907309749f4af57 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:02.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95382700 1 -- 172.21.2.111:0/2023776347 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8800bbc0 con 0x7f3d980b3ae0 2021-04-03T03:12:02.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95382700 1 -- 172.21.2.111:0/2023776347 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8800bd20 con 0x7f3d980b3ae0 2021-04-03T03:12:02.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95382700 1 -- 172.21.2.111:0/2023776347 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d88011400 con 0x7f3d980b3ae0 2021-04-03T03:12:02.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- 172.21.2.111:0/2023776347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b3ae0 msgr2=0x7f3d98063c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- 172.21.2.111:0/2023776347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b3ae0 0x7f3d98063c90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f3d980b45f0 tx=0x7f3d88009270).stop 2021-04-03T03:12:02.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- 172.21.2.111:0/2023776347 shutdown_connections 2021-04-03T03:12:02.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- 172.21.2.111:0/2023776347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b3ae0 0x7f3d98063c90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- 172.21.2.111:0/2023776347 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d980b2880 0x7f3d981435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- 172.21.2.111:0/2023776347 >> 172.21.2.111:0/2023776347 conn(0x7f3d980622a0 msgr2=0x7f3d98060ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:02.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- 172.21.2.111:0/2023776347 shutdown_connections 2021-04-03T03:12:02.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- 172.21.2.111:0/2023776347 wait complete. 2021-04-03T03:12:02.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 Processor -- start 2021-04-03T03:12:02.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- start start 2021-04-03T03:12:02.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d98066250 0x7f3d98153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b2880 0x7f3d98113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3d980b3ae0 0x7f3d981439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98060410 con 0x7f3d980b2880 2021-04-03T03:12:02.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980606b0 con 0x7f3d980b3ae0 2021-04-03T03:12:02.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9814ff20 con 0x7f3d98066250 2021-04-03T03:12:02.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b2880 0x7f3d98113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:02.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b2880 0x7f3d98113180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52188/0 (socket says 172.21.2.111:52188) 2021-04-03T03:12:02.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 -- 172.21.2.111:0/2988314842 learned_addr learned my addr 172.21.2.111:0/2988314842 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:02.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 -- 172.21.2.111:0/2988314842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d98066250 msgr2=0x7f3d98153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d98066250 0x7f3d98153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 -- 172.21.2.111:0/2988314842 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3d980b3ae0 msgr2=0x7f3d981439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3d980b3ae0 0x7f3d981439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 -- 172.21.2.111:0/2988314842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d981470b0 con 0x7f3d980b2880 2021-04-03T03:12:02.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d95b83700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b2880 0x7f3d98113180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f3d80007000 tx=0x7f3d80008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:02.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d8f7fe700 1 -- 172.21.2.111:0/2988314842 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8000ce60 con 0x7f3d980b2880 2021-04-03T03:12:02.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d8f7fe700 1 -- 172.21.2.111:0/2988314842 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d800076f0 con 0x7f3d980b2880 2021-04-03T03:12:02.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d98060090 con 0x7f3d980b2880 2021-04-03T03:12:02.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d8f7fe700 1 -- 172.21.2.111:0/2988314842 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d80007a50 con 0x7f3d980b2880 2021-04-03T03:12:02.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.071+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d98150580 con 0x7f3d980b2880 2021-04-03T03:12:02.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.075+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d98069320 con 0x7f3d980b2880 2021-04-03T03:12:02.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.075+0000 7f3d8f7fe700 1 -- 172.21.2.111:0/2988314842 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3d80007bb0 con 0x7f3d980b2880 2021-04-03T03:12:02.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.075+0000 7f3d8f7fe700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3d7c0570a0 0x7f3d7c059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.075+0000 7f3d8f7fe700 1 -- 172.21.2.111:0/2988314842 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f3d80070060 con 0x7f3d980b2880 2021-04-03T03:12:02.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.075+0000 7f3d96384700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3d7c0570a0 0x7f3d7c059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:02.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.075+0000 7f3d96384700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3d7c0570a0 0x7f3d7c059560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3d980b45f0 tx=0x7f3d88016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:02.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.075+0000 7f3d8f7fe700 1 -- 172.21.2.111:0/2988314842 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3d98069320 con 0x7f3d980b2880 2021-04-03T03:12:02.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.167+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f3d98060280 con 0x7f3d980b2880 2021-04-03T03:12:02.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.803+0000 7f3d8f7fe700 1 -- 172.21.2.111:0/2988314842 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v49) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3d80045780 con 0x7f3d980b2880 2021-04-03T03:12:02.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3d7c0570a0 msgr2=0x7f3d7c059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3d7c0570a0 0x7f3d7c059560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3d980b45f0 tx=0x7f3d88016000).stop 2021-04-03T03:12:02.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b2880 msgr2=0x7f3d98113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b2880 0x7f3d98113180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f3d80007000 tx=0x7f3d80008040).stop 2021-04-03T03:12:02.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 shutdown_connections 2021-04-03T03:12:02.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3d7c0570a0 0x7f3d7c059560 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3d980b3ae0 0x7f3d981439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3d98066250 0x7f3d98153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 --2- 172.21.2.111:0/2988314842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3d980b2880 0x7f3d98113180 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 >> 172.21.2.111:0/2988314842 conn(0x7f3d980622a0 msgr2=0x7f3d980b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:02.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 shutdown_connections 2021-04-03T03:12:02.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.807+0000 7f3d97386700 1 -- 172.21.2.111:0/2988314842 wait complete. 2021-04-03T03:12:02.825 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:12:02.828 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:12:02.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46f268700 1 Processor -- start 2021-04-03T03:12:02.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46f268700 1 -- start start 2021-04-03T03:12:02.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46f268700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4680add50 0x7ff468147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46f268700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff468144ef0 0x7ff468147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46f268700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680607d0 con 0x7ff4680603f0 2021-04-03T03:12:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46f268700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46814b090 con 0x7ff4680add50 2021-04-03T03:12:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46f268700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46814b330 con 0x7ff468144ef0 2021-04-03T03:12:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46da65700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4680add50 0x7ff468147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:02.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46e266700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff4680603f0 0x7ff4681447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:52990/0 (socket says 172.21.2.111:52990) 2021-04-03T03:12:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46e266700 1 -- 172.21.2.111:0/3443888313 learned_addr learned my addr 172.21.2.111:0/3443888313 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46da65700 1 -- 172.21.2.111:0/3443888313 >> v2:172.21.2.115:3300/0 conn(0x7ff468144ef0 msgr2=0x7ff468147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46da65700 1 --2- 172.21.2.111:0/3443888313 >> v2:172.21.2.115:3300/0 conn(0x7ff468144ef0 0x7ff468147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46da65700 1 -- 172.21.2.111:0/3443888313 >> v1:172.21.2.111:6789/0 conn(0x7ff4680603f0 legacy=0x7ff4681447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46d264700 1 -- 172.21.2.111:0/3443888313 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244382515 0 0) 0x7ff4680607d0 con 0x7ff4680603f0 2021-04-03T03:12:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46da65700 1 -- 172.21.2.111:0/3443888313 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46814b5d0 con 0x7ff4680add50 2021-04-03T03:12:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.871+0000 7ff46da65700 1 --2- 172.21.2.111:0/3443888313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4680add50 0x7ff468147740 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7ff458002f30 tx=0x7ff458006dd0).ready entity=mon.0 client_cookie=e1b1ca8c380dea1c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46d264700 1 -- 172.21.2.111:0/3443888313 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4580097f0 con 0x7ff4680add50 2021-04-03T03:12:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46d264700 1 -- 172.21.2.111:0/3443888313 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458009950 con 0x7ff4680add50 2021-04-03T03:12:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46d264700 1 -- 172.21.2.111:0/3443888313 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff458009cb0 con 0x7ff4680add50 2021-04-03T03:12:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/3443888313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4680add50 msgr2=0x7ff468147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 --2- 172.21.2.111:0/3443888313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4680add50 0x7ff468147740 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7ff458002f30 tx=0x7ff458006dd0).stop 2021-04-03T03:12:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/3443888313 shutdown_connections 2021-04-03T03:12:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 --2- 172.21.2.111:0/3443888313 >> v2:172.21.2.115:3300/0 conn(0x7ff468144ef0 0x7ff468147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 --2- 172.21.2.111:0/3443888313 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4680add50 0x7ff468147740 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:02.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/3443888313 >> 172.21.2.111:0/3443888313 conn(0x7ff4680b61c0 msgr2=0x7ff46805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:02.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/3443888313 shutdown_connections 2021-04-03T03:12:02.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/3443888313 wait complete. 2021-04-03T03:12:02.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 Processor -- start 2021-04-03T03:12:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- start start 2021-04-03T03:12:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4680603f0 0x7ff4680ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4680add50 0x7ff4680aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff468144ef0 0x7ff468159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680a9f40 con 0x7ff468144ef0 2021-04-03T03:12:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681570d0 con 0x7ff4680603f0 2021-04-03T03:12:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680abbc0 con 0x7ff4680add50 2021-04-03T03:12:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff468144ef0 0x7ff468159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:02.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff468144ef0 0x7ff468159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52204/0 (socket says 172.21.2.111:52204) 2021-04-03T03:12:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 -- 172.21.2.111:0/78202549 learned_addr learned my addr 172.21.2.111:0/78202549 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 -- 172.21.2.111:0/78202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4680add50 msgr2=0x7ff4680aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4680add50 0x7ff4680aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 -- 172.21.2.111:0/78202549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4680603f0 msgr2=0x7ff4680ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:02.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4680603f0 0x7ff4680ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 -- 172.21.2.111:0/78202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4681572e0 con 0x7ff468144ef0 2021-04-03T03:12:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46ea67700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff468144ef0 0x7ff468159d90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7ff4680abd50 tx=0x7ff460009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff4577fe700 1 -- 172.21.2.111:0/78202549 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff46000c900 con 0x7ff468144ef0 2021-04-03T03:12:02.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff468157580 con 0x7ff468144ef0 2021-04-03T03:12:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff4577fe700 1 -- 172.21.2.111:0/78202549 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46000ca60 con 0x7ff468144ef0 2021-04-03T03:12:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff4577fe700 1 -- 172.21.2.111:0/78202549 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff46000cdc0 con 0x7ff468144ef0 2021-04-03T03:12:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4680a6fd0 con 0x7ff468144ef0 2021-04-03T03:12:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4680ab410 con 0x7ff468144ef0 2021-04-03T03:12:02.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff4577fe700 1 -- 172.21.2.111:0/78202549 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff460014410 con 0x7ff468144ef0 2021-04-03T03:12:02.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff4577fe700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff448056f20 0x7ff4480593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:02.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff4577fe700 1 -- 172.21.2.111:0/78202549 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5950+0+0 (secure 0 0 0) 0x7ff46006fe20 con 0x7ff468144ef0 2021-04-03T03:12:02.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.875+0000 7ff46e266700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff448056f20 0x7ff4480593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:02.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.879+0000 7ff46e266700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff448056f20 0x7ff4480593e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7ff464000e90 tx=0x7ff4640076a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:02.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.879+0000 7ff4577fe700 1 -- 172.21.2.111:0/78202549 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff4680ab410 con 0x7ff468144ef0 2021-04-03T03:12:02.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:02.975+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7ff4680ab410 con 0x7ff468144ef0 2021-04-03T03:12:03.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.811+0000 7ff4577fe700 1 -- 172.21.2.111:0/78202549 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v50) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff4680ab410 con 0x7ff468144ef0 2021-04-03T03:12:03.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff448056f20 msgr2=0x7ff4480593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:03.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff448056f20 0x7ff4480593e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7ff464000e90 tx=0x7ff4640076a0).stop 2021-04-03T03:12:03.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff468144ef0 msgr2=0x7ff468159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:03.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff468144ef0 0x7ff468159d90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7ff4680abd50 tx=0x7ff460009120).stop 2021-04-03T03:12:03.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 shutdown_connections 2021-04-03T03:12:03.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff468144ef0 0x7ff468159d90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:03.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff448056f20 0x7ff4480593e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:03.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4680603f0 0x7ff4680ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:03.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 --2- 172.21.2.111:0/78202549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4680add50 0x7ff4680aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:03.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 >> 172.21.2.111:0/78202549 conn(0x7ff4680b61c0 msgr2=0x7ff46805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:03.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 shutdown_connections 2021-04-03T03:12:03.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.815+0000 7ff46f268700 1 -- 172.21.2.111:0/78202549 wait complete. 2021-04-03T03:12:03.830 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:12:03.834 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:12:03.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 Processor -- start 2021-04-03T03:12:03.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 -- start start 2021-04-03T03:12:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe9440603f0 0x7fe944144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b17f0 0x7fe944144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:03.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9440b0be0 con 0x7fe9440b0290 2021-04-03T03:12:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9440607d0 con 0x7fe9440603f0 2021-04-03T03:12:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440b0e10 con 0x7fe9440b17f0 2021-04-03T03:12:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94997d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe9440603f0 0x7fe944144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:03.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a17e700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fe9440b0290 0x7fe9440aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:37306/0 (socket says 172.21.2.111:37306) 2021-04-03T03:12:03.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94917c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b17f0 0x7fe944144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:03.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a17e700 1 -- 172.21.2.111:0/1400509650 learned_addr learned my addr 172.21.2.111:0/1400509650 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:03.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94997d700 1 -- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b17f0 msgr2=0x7fe944144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:03.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94997d700 1 --2- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b17f0 0x7fe944144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:03.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94997d700 1 -- 172.21.2.111:0/1400509650 >> v1:172.21.2.115:6789/0 conn(0x7fe9440b0290 legacy=0x7fe9440aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:03.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94997d700 1 -- 172.21.2.111:0/1400509650 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440b10b0 con 0x7fe9440603f0 2021-04-03T03:12:03.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94917c700 1 --2- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b17f0 0x7fe944144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:12:03.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94997d700 1 --2- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440603f0 0x7fe944144700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe9440628d0 tx=0x7fe938006dd0).ready entity=mon.1 client_cookie=567e3447153cd0d7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:03.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94897b700 1 -- 172.21.2.111:0/1400509650 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9380097b0 con 0x7fe9440603f0 2021-04-03T03:12:03.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94897b700 1 -- 172.21.2.111:0/1400509650 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe938009910 con 0x7fe9440603f0 2021-04-03T03:12:03.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 -- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440603f0 msgr2=0x7fe944144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:03.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.879+0000 7fe94a97f700 1 --2- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440603f0 0x7fe944144700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe9440628d0 tx=0x7fe938006dd0).stop 2021-04-03T03:12:03.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- 172.21.2.111:0/1400509650 shutdown_connections 2021-04-03T03:12:03.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 --2- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440603f0 0x7fe944144700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:03.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 --2- 172.21.2.111:0/1400509650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b17f0 0x7fe944144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:03.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- 172.21.2.111:0/1400509650 >> 172.21.2.111:0/1400509650 conn(0x7fe9440b61c0 msgr2=0x7fe94405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:03.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- 172.21.2.111:0/1400509650 shutdown_connections 2021-04-03T03:12:03.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- 172.21.2.111:0/1400509650 wait complete. 2021-04-03T03:12:03.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 Processor -- start 2021-04-03T03:12:03.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- start start 2021-04-03T03:12:03.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9440603f0 0x7fe944066a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:03.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b0290 0x7fe9440af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:03.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440b17f0 0x7fe944155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:03.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440afb80 con 0x7fe9440603f0 2021-04-03T03:12:03.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94413fcb0 con 0x7fe9440b17f0 2021-04-03T03:12:03.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94413ff50 con 0x7fe9440b0290 2021-04-03T03:12:03.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9440603f0 0x7fe944066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:03.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9440603f0 0x7fe944066a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52216/0 (socket says 172.21.2.111:52216) 2021-04-03T03:12:03.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 -- 172.21.2.111:0/2273513559 learned_addr learned my addr 172.21.2.111:0/2273513559 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:03.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 -- 172.21.2.111:0/2273513559 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b0290 msgr2=0x7fe9440af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:03.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b0290 0x7fe9440af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:03.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 -- 172.21.2.111:0/2273513559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440b17f0 msgr2=0x7fe944155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:03.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440b17f0 0x7fe944155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:03.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 -- 172.21.2.111:0/2273513559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9441557f0 con 0x7fe9440603f0 2021-04-03T03:12:03.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94997d700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9440603f0 0x7fe944066a40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fe9440628d0 tx=0x7fe938015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:03.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe936ffd700 1 -- 172.21.2.111:0/2273513559 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe938011ce0 con 0x7fe9440603f0 2021-04-03T03:12:03.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe936ffd700 1 -- 172.21.2.111:0/2273513559 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe938011e40 con 0x7fe9440603f0 2021-04-03T03:12:03.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9440aed50 con 0x7fe9440603f0 2021-04-03T03:12:03.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe944156180 con 0x7fe9440603f0 2021-04-03T03:12:03.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe936ffd700 1 -- 172.21.2.111:0/2273513559 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe93801e650 con 0x7fe9440603f0 2021-04-03T03:12:03.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94413fb10 con 0x7fe9440603f0 2021-04-03T03:12:03.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe936ffd700 1 -- 172.21.2.111:0/2273513559 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe938025030 con 0x7fe9440603f0 2021-04-03T03:12:03.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe936ffd700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe924057000 0x7fe9240594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:03.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe936ffd700 1 -- 172.21.2.111:0/2273513559 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6300+0+0 (secure 0 0 0) 0x7fe938071320 con 0x7fe9440603f0 2021-04-03T03:12:03.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94917c700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe924057000 0x7fe9240594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:03.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.883+0000 7fe94917c700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe924057000 0x7fe9240594c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fe94000c000 tx=0x7fe940006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:03.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:03.887+0000 7fe936ffd700 1 -- 172.21.2.111:0/2273513559 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe94413fb10 con 0x7fe9440603f0 2021-04-03T03:12:04.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.015+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe94413fb10 con 0x7fe9440603f0 2021-04-03T03:12:04.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe936ffd700 1 -- 172.21.2.111:0/2273513559 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v20) v1 ==== 188+0+0 (secure 0 0 0) 0x7fe94413fb10 con 0x7fe9440603f0 2021-04-03T03:12:04.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe924057000 msgr2=0x7fe9240594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe924057000 0x7fe9240594c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fe94000c000 tx=0x7fe940006dd0).stop 2021-04-03T03:12:04.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9440603f0 msgr2=0x7fe944066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9440603f0 0x7fe944066a40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fe9440628d0 tx=0x7fe938015000).stop 2021-04-03T03:12:04.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 shutdown_connections 2021-04-03T03:12:04.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe924057000 0x7fe9240594c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9440b17f0 0x7fe944155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9440603f0 0x7fe944066a40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 --2- 172.21.2.111:0/2273513559 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9440b0290 0x7fe9440af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 >> 172.21.2.111:0/2273513559 conn(0x7fe9440b61c0 msgr2=0x7fe944141380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:04.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 shutdown_connections 2021-04-03T03:12:04.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.019+0000 7fe94a97f700 1 -- 172.21.2.111:0/2273513559 wait complete. 2021-04-03T03:12:04.037 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 9 and data pool 10 2021-04-03T03:12:04.041 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:12:04.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.083+0000 7f099dd10700 1 Processor -- start 2021-04-03T03:12:04.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.083+0000 7f099dd10700 1 -- start start 2021-04-03T03:12:04.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09900dbb90 0x7f0990042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900dc6e0 con 0x7f099003d360 2021-04-03T03:12:04.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0990042010 con 0x7f0990006bc0 2021-04-03T03:12:04.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09900d9560 con 0x7f09900dbb90 2021-04-03T03:12:04.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09900dbb90 0x7f0990042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09900dbb90 0x7f0990042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52234/0 (socket says 172.21.2.111:52234) 2021-04-03T03:12:04.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/1317918408 learned_addr learned my addr 172.21.2.111:0/1317918408 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:04.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/1317918408 >> v1:172.21.2.115:6789/0 conn(0x7f0990006bc0 legacy=0x7f09900056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:04.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/1317918408 >> v1:172.21.2.111:6789/0 conn(0x7f099003d360 legacy=0x7f09900064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/1317918408 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900422b0 con 0x7f09900dbb90 2021-04-03T03:12:04.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- 172.21.2.111:0/1317918408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09900dbb90 0x7f0990042750 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f09900087e0 tx=0x7f0988006e90).ready entity=mon.0 client_cookie=e68e95659badde78 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f09977fe700 1 -- 172.21.2.111:0/1317918408 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0988009980 con 0x7f09900dbb90 2021-04-03T03:12:04.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f09977fe700 1 -- 172.21.2.111:0/1317918408 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988009ae0 con 0x7f09900dbb90 2021-04-03T03:12:04.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f09977fe700 1 -- 172.21.2.111:0/1317918408 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0988009e80 con 0x7f09900dbb90 2021-04-03T03:12:04.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/1317918408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09900dbb90 msgr2=0x7f0990042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 --2- 172.21.2.111:0/1317918408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09900dbb90 0x7f0990042750 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f09900087e0 tx=0x7f0988006e90).stop 2021-04-03T03:12:04.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/1317918408 shutdown_connections 2021-04-03T03:12:04.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 --2- 172.21.2.111:0/1317918408 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09900dbb90 0x7f0990042750 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/1317918408 >> 172.21.2.111:0/1317918408 conn(0x7f09900462c0 msgr2=0x7f09900466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:04.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/1317918408 shutdown_connections 2021-04-03T03:12:04.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/1317918408 wait complete. 2021-04-03T03:12:04.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 Processor -- start 2021-04-03T03:12:04.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- start start 2021-04-03T03:12:04.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f099003d360 0x7f099009f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09900dbb90 0x7f09900e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09900ea3e0 0x7f09900ea980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099009fc20 con 0x7f099003d360 2021-04-03T03:12:04.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099003c750 con 0x7f09900ea3e0 2021-04-03T03:12:04.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099003c9f0 con 0x7f09900dbb90 2021-04-03T03:12:04.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f099003d360 0x7f099009f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f099003d360 0x7f099009f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52240/0 (socket says 172.21.2.111:52240) 2021-04-03T03:12:04.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/2082039995 learned_addr learned my addr 172.21.2.111:0/2082039995 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:04.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/2082039995 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09900dbb90 msgr2=0x7f09900e9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09900dbb90 0x7f09900e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/2082039995 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09900ea3e0 msgr2=0x7f09900ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09900ea3e0 0x7f09900ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 -- 172.21.2.111:0/2082039995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099003c460 con 0x7f099003d360 2021-04-03T03:12:04.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099cd0e700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f099003d360 0x7f099009f730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f09900087e0 tx=0x7f0988015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f0995ffb700 1 -- 172.21.2.111:0/2082039995 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0988011db0 con 0x7f099003d360 2021-04-03T03:12:04.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f0995ffb700 1 -- 172.21.2.111:0/2082039995 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098801e460 con 0x7f099003d360 2021-04-03T03:12:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09900eae70 con 0x7f099003d360 2021-04-03T03:12:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09900eb330 con 0x7f099003d360 2021-04-03T03:12:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f0995ffb700 1 -- 172.21.2.111:0/2082039995 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f098801e7c0 con 0x7f099003d360 2021-04-03T03:12:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09740033f0 con 0x7f099003d360 2021-04-03T03:12:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f0995ffb700 1 -- 172.21.2.111:0/2082039995 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0988025030 con 0x7f099003d360 2021-04-03T03:12:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.087+0000 7f0995ffb700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0978056f70 0x7f0978059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.091+0000 7f0995ffb700 1 -- 172.21.2.111:0/2082039995 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0988071800 con 0x7f099003d360 2021-04-03T03:12:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.091+0000 7f0997fff700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0978056f70 0x7f0978059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.091+0000 7f0997fff700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0978056f70 0x7f0978059430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f0990046210 tx=0x7f098c0077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.091+0000 7f0995ffb700 1 -- 172.21.2.111:0/2082039995 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0988046ae0 con 0x7f099003d360 2021-04-03T03:12:04.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09740036a0 con 0x7f099003d360 2021-04-03T03:12:04.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f0995ffb700 1 -- 172.21.2.111:0/2082039995 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f098804a100 con 0x7f099003d360 2021-04-03T03:12:04.202 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:04.202 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":52,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:12:04.020227+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:12:02.184726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-04-03T03:12:02.990138+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:12:04.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0978056f70 msgr2=0x7f0978059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0978056f70 0x7f0978059430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f0990046210 tx=0x7f098c0077f0).stop 2021-04-03T03:12:04.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f099003d360 msgr2=0x7f099009f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f099003d360 0x7f099009f730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f09900087e0 tx=0x7f0988015000).stop 2021-04-03T03:12:04.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 shutdown_connections 2021-04-03T03:12:04.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0978056f70 0x7f0978059430 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09900ea3e0 0x7f09900ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f099003d360 0x7f099009f730 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 --2- 172.21.2.111:0/2082039995 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09900dbb90 0x7f09900e9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 >> 172.21.2.111:0/2082039995 conn(0x7f09900462c0 msgr2=0x7f0990045540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:04.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 shutdown_connections 2021-04-03T03:12:04.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.187+0000 7f099dd10700 1 -- 172.21.2.111:0/2082039995 wait complete. 2021-04-03T03:12:04.209 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:12:04.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.279+0000 7f48be818700 1 Processor -- start 2021-04-03T03:12:04.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.279+0000 7f48be818700 1 -- start start 2021-04-03T03:12:04.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f48b80603f0 0x7f48b8139940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b80b11a0 0x7f48b8139e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b8067940 con 0x7f48b81414b0 2021-04-03T03:12:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b8068010 con 0x7f48b80603f0 2021-04-03T03:12:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80607d0 con 0x7f48b80b11a0 2021-04-03T03:12:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f48b80603f0 0x7f48b8139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f48b80603f0 0x7f48b8139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52248/0 (socket says 172.21.2.111:52248) 2021-04-03T03:12:04.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/3622054364 learned_addr learned my addr 172.21.2.111:0/3622054364 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:04.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/3622054364 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b80b11a0 msgr2=0x7f48b8139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:04.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- 172.21.2.111:0/3622054364 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b80b11a0 0x7f48b8139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/3622054364 >> v1:172.21.2.115:6789/0 conn(0x7f48b81414b0 legacy=0x7f48b8141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/3622054364 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8140de0 con 0x7f48b80603f0 2021-04-03T03:12:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- 172.21.2.111:0/3622054364 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8139940 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f48b8062fa0 tx=0x7f48b4006f20).ready entity=mon.0 client_cookie=9ec472f36b9e4349 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bc814700 1 -- 172.21.2.111:0/3622054364 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48b400de10 con 0x7f48b80603f0 2021-04-03T03:12:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bc814700 1 -- 172.21.2.111:0/3622054364 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b4010470 con 0x7f48b80603f0 2021-04-03T03:12:04.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bc814700 1 -- 172.21.2.111:0/3622054364 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48b40107d0 con 0x7f48b80603f0 2021-04-03T03:12:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- 172.21.2.111:0/3622054364 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 msgr2=0x7f48b8139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- 172.21.2.111:0/3622054364 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8139940 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f48b8062fa0 tx=0x7f48b4006f20).stop 2021-04-03T03:12:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- 172.21.2.111:0/3622054364 shutdown_connections 2021-04-03T03:12:04.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- 172.21.2.111:0/3622054364 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8139940 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- 172.21.2.111:0/3622054364 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b80b11a0 0x7f48b8139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- 172.21.2.111:0/3622054364 >> 172.21.2.111:0/3622054364 conn(0x7f48b80b61c0 msgr2=0x7f48b805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:04.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- 172.21.2.111:0/3622054364 shutdown_connections 2021-04-03T03:12:04.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- 172.21.2.111:0/3622054364 wait complete. 2021-04-03T03:12:04.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 Processor -- start 2021-04-03T03:12:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- start start 2021-04-03T03:12:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48b80b11a0 0x7f48b813a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b81414b0 0x7f48b813bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8139cd0 con 0x7f48b80603f0 2021-04-03T03:12:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b813a890 con 0x7f48b80b11a0 2021-04-03T03:12:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8066040 con 0x7f48b81414b0 2021-04-03T03:12:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8142dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52254/0 (socket says 172.21.2.111:52254) 2021-04-03T03:12:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/2447426235 learned_addr learned my addr 172.21.2.111:0/2447426235 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/2447426235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b81414b0 msgr2=0x7f48b813bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b81414b0 0x7f48b813bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/2447426235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48b80b11a0 msgr2=0x7f48b813a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48b80b11a0 0x7f48b813a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 -- 172.21.2.111:0/2447426235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8139f00 con 0x7f48b80603f0 2021-04-03T03:12:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48bd816700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8142dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f48b813aa20 tx=0x7f48b4018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48aaffd700 1 -- 172.21.2.111:0/2447426235 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48b401d070 con 0x7f48b80603f0 2021-04-03T03:12:04.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48aaffd700 1 -- 172.21.2.111:0/2447426235 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b4018ad0 con 0x7f48b80603f0 2021-04-03T03:12:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b80662e0 con 0x7f48b80603f0 2021-04-03T03:12:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48aaffd700 1 -- 172.21.2.111:0/2447426235 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48b4018e30 con 0x7f48b80603f0 2021-04-03T03:12:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.283+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b8065c40 con 0x7f48b80603f0 2021-04-03T03:12:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.287+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b813a090 con 0x7f48b80603f0 2021-04-03T03:12:04.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.287+0000 7f48aaffd700 1 -- 172.21.2.111:0/2447426235 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f48b4029070 con 0x7f48b80603f0 2021-04-03T03:12:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.287+0000 7f48aaffd700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4898056fb0 0x7f4898059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.287+0000 7f48aaffd700 1 -- 172.21.2.111:0/2447426235 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f48b4071a70 con 0x7f48b80603f0 2021-04-03T03:12:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.287+0000 7f48bd015700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4898056fb0 0x7f4898059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.287+0000 7f48bd015700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4898056fb0 0x7f4898059470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f48ac002f30 tx=0x7f48ac006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.287+0000 7f48aaffd700 1 -- 172.21.2.111:0/2447426235 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f48b4046e60 con 0x7f48b80603f0 2021-04-03T03:12:04.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.379+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48b8057660 con 0x7f48b80603f0 2021-04-03T03:12:04.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.379+0000 7f48aaffd700 1 -- 172.21.2.111:0/2447426235 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f48b4071e60 con 0x7f48b80603f0 2021-04-03T03:12:04.394 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:04.395 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":52,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:12:04.020227+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:12:02.184726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-04-03T03:12:02.990138+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:12:04.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4898056fb0 msgr2=0x7f4898059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4898056fb0 0x7f4898059470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f48ac002f30 tx=0x7f48ac006a60).stop 2021-04-03T03:12:04.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 msgr2=0x7f48b8142dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8142dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f48b813aa20 tx=0x7f48b4018000).stop 2021-04-03T03:12:04.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 shutdown_connections 2021-04-03T03:12:04.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48b81414b0 0x7f48b813bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48b80603f0 0x7f48b8142dc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4898056fb0 0x7f4898059470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 --2- 172.21.2.111:0/2447426235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48b80b11a0 0x7f48b813a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 >> 172.21.2.111:0/2447426235 conn(0x7f48b80b61c0 msgr2=0x7f48b813f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:04.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 shutdown_connections 2021-04-03T03:12:04.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.383+0000 7f48be818700 1 -- 172.21.2.111:0/2447426235 wait complete. 2021-04-03T03:12:04.402 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:12:04.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 Processor -- start 2021-04-03T03:12:04.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- start start 2021-04-03T03:12:04.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0603f0 0x7fa1cc1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa1cc0b17b0 0x7fa1cc144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc0607d0 con 0x7fa1cc0add50 2021-04-03T03:12:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc1451c0 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc14b090 con 0x7fa1cc0603f0 2021-04-03T03:12:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa1cc0b17b0 0x7fa1cc144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa1cc0b17b0 0x7fa1cc144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52266/0 (socket says 172.21.2.111:52266) 2021-04-03T03:12:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 -- 172.21.2.111:0/1621470115 learned_addr learned my addr 172.21.2.111:0/1621470115 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 -- 172.21.2.111:0/1621470115 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0603f0 msgr2=0x7fa1cc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:04.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 --2- 172.21.2.111:0/1621470115 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0603f0 0x7fa1cc1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 -- 172.21.2.111:0/1621470115 >> v1:172.21.2.111:6789/0 conn(0x7fa1cc0add50 legacy=0x7fa1cc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 -- 172.21.2.111:0/1621470115 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc14b330 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d10b4700 1 --2- 172.21.2.111:0/1621470115 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc144cd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa1c40032b0 tx=0x7fa1c4009270).ready entity=mon.0 client_cookie=c4462357205f15f7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1cb7fe700 1 -- 172.21.2.111:0/1621470115 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c400bc00 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1cb7fe700 1 -- 172.21.2.111:0/1621470115 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c400bd60 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1cb7fe700 1 -- 172.21.2.111:0/1621470115 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c4011400 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- 172.21.2.111:0/1621470115 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 msgr2=0x7fa1cc144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/1621470115 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc144cd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa1c40032b0 tx=0x7fa1c4009270).stop 2021-04-03T03:12:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- 172.21.2.111:0/1621470115 shutdown_connections 2021-04-03T03:12:04.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/1621470115 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc144cd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:04.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/1621470115 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0603f0 0x7fa1cc1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- 172.21.2.111:0/1621470115 >> 172.21.2.111:0/1621470115 conn(0x7fa1cc0b61c0 msgr2=0x7fa1cc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:04.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- 172.21.2.111:0/1621470115 shutdown_connections 2021-04-03T03:12:04.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.447+0000 7fa1d18b5700 1 -- 172.21.2.111:0/1621470115 wait complete. 2021-04-03T03:12:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 Processor -- start 2021-04-03T03:12:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 -- start start 2021-04-03T03:12:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1cc0603f0 0x7fa1cc0a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0add50 0x7fa1cc15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0aa1a0 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0ac010 con 0x7fa1cc0603f0 2021-04-03T03:12:04.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc156d30 con 0x7fa1cc0add50 2021-04-03T03:12:04.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc15a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52272/0 (socket says 172.21.2.111:52272) 2021-04-03T03:12:04.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 -- 172.21.2.111:0/3022318276 learned_addr learned my addr 172.21.2.111:0/3022318276 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:04.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 -- 172.21.2.111:0/3022318276 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0add50 msgr2=0x7fa1cc15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0add50 0x7fa1cc15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 -- 172.21.2.111:0/3022318276 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1cc0603f0 msgr2=0x7fa1cc0a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:04.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1cc0603f0 0x7fa1cc0a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:04.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 -- 172.21.2.111:0/3022318276 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0a72e0 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d10b4700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc15a770 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fa1c40032b0 tx=0x7fa1c4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1c9ffb700 1 -- 172.21.2.111:0/3022318276 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c401f610 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1c9ffb700 1 -- 172.21.2.111:0/3022318276 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c400bed0 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc0a7580 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1c9ffb700 1 -- 172.21.2.111:0/3022318276 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c401f960 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc0a9610 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc0a7710 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1c9ffb700 1 -- 172.21.2.111:0/3022318276 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa1c4026030 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1c9ffb700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa1ac056ea0 0x7fa1ac059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:04.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1c9ffb700 1 -- 172.21.2.111:0/3022318276 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa1c4070eb0 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d08b3700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa1ac056ea0 0x7fa1ac059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:04.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.451+0000 7fa1d08b3700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa1ac056ea0 0x7fa1ac059360 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fa1bc002f30 tx=0x7fa1bc006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:04.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.455+0000 7fa1c9ffb700 1 -- 172.21.2.111:0/3022318276 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa1c4046190 con 0x7fa1cc0b17b0 2021-04-03T03:12:04.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:04.579+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa1cc057660 con 0x7fa1cc0b17b0 2021-04-03T03:12:05.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1c9ffb700 1 -- 172.21.2.111:0/3022318276 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa1c40497b0 con 0x7fa1cc0b17b0 2021-04-03T03:12:05.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa1ac056ea0 msgr2=0x7fa1ac059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa1ac056ea0 0x7fa1ac059360 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fa1bc002f30 tx=0x7fa1bc006a60).stop 2021-04-03T03:12:05.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 msgr2=0x7fa1cc15a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc15a770 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fa1c40032b0 tx=0x7fa1c4016000).stop 2021-04-03T03:12:05.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 shutdown_connections 2021-04-03T03:12:05.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1cc0b17b0 0x7fa1cc15a770 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa1ac056ea0 0x7fa1ac059360 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1cc0603f0 0x7fa1cc0a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 --2- 172.21.2.111:0/3022318276 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1cc0add50 0x7fa1cc15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 >> 172.21.2.111:0/3022318276 conn(0x7fa1cc0b61c0 msgr2=0x7fa1cc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 shutdown_connections 2021-04-03T03:12:05.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.027+0000 7fa1d18b5700 1 -- 172.21.2.111:0/3022318276 wait complete. 2021-04-03T03:12:05.050 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:12:05.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 Processor -- start 2021-04-03T03:12:05.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- start start 2021-04-03T03:12:05.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744041db0 0x7fb7440426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb7440d1610 0x7fb744039e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb744047960 con 0x7fb744047580 2021-04-03T03:12:05.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74403d180 con 0x7fb7440d1610 2021-04-03T03:12:05.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74403d420 con 0x7fb744041db0 2021-04-03T03:12:05.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74b7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fb744047580 0x7fb7440d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53068/0 (socket says 172.21.2.111:53068) 2021-04-03T03:12:05.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74b7fe700 1 -- 172.21.2.111:0/1925962660 learned_addr learned my addr 172.21.2.111:0/1925962660 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:05.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74bfff700 1 --2- 172.21.2.111:0/1925962660 >> v2:172.21.2.111:3300/0 conn(0x7fb7440d1610 0x7fb744039e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74affd700 1 --2- 172.21.2.111:0/1925962660 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744041db0 0x7fb7440426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74bfff700 1 -- 172.21.2.111:0/1925962660 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744041db0 msgr2=0x7fb7440426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74bfff700 1 --2- 172.21.2.111:0/1925962660 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744041db0 0x7fb7440426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74bfff700 1 -- 172.21.2.111:0/1925962660 >> v1:172.21.2.111:6789/0 conn(0x7fb744047580 legacy=0x7fb7440d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74bfff700 1 -- 172.21.2.111:0/1925962660 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440d1a60 con 0x7fb7440d1610 2021-04-03T03:12:05.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74bfff700 1 --2- 172.21.2.111:0/1925962660 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7440d1610 0x7fb744039e20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fb73c002f30 tx=0x7fb73c006dd0).ready entity=mon.0 client_cookie=773d2b0344b2fc11 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74a7fc700 1 -- 172.21.2.111:0/1925962660 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73c0097f0 con 0x7fb7440d1610 2021-04-03T03:12:05.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74a7fc700 1 -- 172.21.2.111:0/1925962660 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c009950 con 0x7fb7440d1610 2021-04-03T03:12:05.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb74a7fc700 1 -- 172.21.2.111:0/1925962660 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73c009cb0 con 0x7fb7440d1610 2021-04-03T03:12:05.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- 172.21.2.111:0/1925962660 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7440d1610 msgr2=0x7fb744039e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 --2- 172.21.2.111:0/1925962660 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7440d1610 0x7fb744039e20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fb73c002f30 tx=0x7fb73c006dd0).stop 2021-04-03T03:12:05.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- 172.21.2.111:0/1925962660 shutdown_connections 2021-04-03T03:12:05.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 --2- 172.21.2.111:0/1925962660 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb7440d1610 0x7fb744039e20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 --2- 172.21.2.111:0/1925962660 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744041db0 0x7fb7440426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- 172.21.2.111:0/1925962660 >> 172.21.2.111:0/1925962660 conn(0x7fb74400d9e0 msgr2=0x7fb744046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- 172.21.2.111:0/1925962660 shutdown_connections 2021-04-03T03:12:05.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- 172.21.2.111:0/1925962660 wait complete. 2021-04-03T03:12:05.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 Processor -- start 2021-04-03T03:12:05.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.151+0000 7fb750c47700 1 -- start start 2021-04-03T03:12:05.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb744041db0 0x7fb7440d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744047580 0x7fb7440cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb7440d1610 0x7fb7440d4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440ca930 con 0x7fb744041db0 2021-04-03T03:12:05.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440e31b0 con 0x7fb7440d1610 2021-04-03T03:12:05.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440e3450 con 0x7fb744047580 2021-04-03T03:12:05.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb744041db0 0x7fb7440d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb744041db0 0x7fb7440d3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52282/0 (socket says 172.21.2.111:52282) 2021-04-03T03:12:05.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 -- 172.21.2.111:0/2100426424 learned_addr learned my addr 172.21.2.111:0/2100426424 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:05.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74affd700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744047580 0x7fb7440cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 -- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744047580 msgr2=0x7fb7440cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744047580 0x7fb7440cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 -- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb7440d1610 msgr2=0x7fb7440d4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74bfff700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb7440d1610 0x7fb7440d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb7440d1610 0x7fb7440d4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 -- 172.21.2.111:0/2100426424 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440e36f0 con 0x7fb744041db0 2021-04-03T03:12:05.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74affd700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744047580 0x7fb7440cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:12:05.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74b7fe700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb744041db0 0x7fb7440d3b30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fb74400c6b0 tx=0x7fb740007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb748ff9700 1 -- 172.21.2.111:0/2100426424 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb740007320 con 0x7fb744041db0 2021-04-03T03:12:05.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb748ff9700 1 -- 172.21.2.111:0/2100426424 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740007480 con 0x7fb744041db0 2021-04-03T03:12:05.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7440cdbe0 con 0x7fb744041db0 2021-04-03T03:12:05.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb748ff9700 1 -- 172.21.2.111:0/2100426424 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7400057e0 con 0x7fb744041db0 2021-04-03T03:12:05.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7440e2d90 con 0x7fb744041db0 2021-04-03T03:12:05.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7440e3da0 con 0x7fb744041db0 2021-04-03T03:12:05.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb748ff9700 1 -- 172.21.2.111:0/2100426424 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb740010030 con 0x7fb744041db0 2021-04-03T03:12:05.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb748ff9700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb72c057050 0x7fb72c059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb748ff9700 1 -- 172.21.2.111:0/2100426424 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb74006feb0 con 0x7fb744041db0 2021-04-03T03:12:05.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74affd700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb72c057050 0x7fb72c059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.155+0000 7fb74affd700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb72c057050 0x7fb72c059510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb74c067000 tx=0x7fb74c05ad70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.159+0000 7fb748ff9700 1 -- 172.21.2.111:0/2100426424 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb7400452a0 con 0x7fb744041db0 2021-04-03T03:12:05.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.279+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb744004da0 con 0x7fb744041db0 2021-04-03T03:12:05.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb748ff9700 1 -- 172.21.2.111:0/2100426424 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fb7400488c0 con 0x7fb744041db0 2021-04-03T03:12:05.295 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:05.295 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:12:05.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb72c057050 msgr2=0x7fb72c059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb72c057050 0x7fb72c059510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb74c067000 tx=0x7fb74c05ad70).stop 2021-04-03T03:12:05.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb744041db0 msgr2=0x7fb7440d3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb744041db0 0x7fb7440d3b30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fb74400c6b0 tx=0x7fb740007c20).stop 2021-04-03T03:12:05.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 shutdown_connections 2021-04-03T03:12:05.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb72c057050 0x7fb72c059510 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb7440d1610 0x7fb7440d4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb744041db0 0x7fb7440d3b30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 --2- 172.21.2.111:0/2100426424 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb744047580 0x7fb7440cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 >> 172.21.2.111:0/2100426424 conn(0x7fb74400d9e0 msgr2=0x7fb744005dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 shutdown_connections 2021-04-03T03:12:05.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.283+0000 7fb750c47700 1 -- 172.21.2.111:0/2100426424 wait complete. 2021-04-03T03:12:05.299 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:12:05.303 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:12:05.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 Processor -- start 2021-04-03T03:12:05.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- start start 2021-04-03T03:12:05.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 0x7fedf8042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fedf8006bc0 0x7fedf80dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf80dc6e0 con 0x7fedf803d360 2021-04-03T03:12:05.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf80d9560 con 0x7fedf8006bc0 2021-04-03T03:12:05.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf803dcb0 con 0x7fedf80dbb90 2021-04-03T03:12:05.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee05e14700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fedf8006bc0 0x7fedf80dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee04e12700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fedf803d360 0x7fedf80064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53084/0 (socket says 172.21.2.111:53084) 2021-04-03T03:12:05.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee04e12700 1 -- 172.21.2.111:0/63031636 learned_addr learned my addr 172.21.2.111:0/63031636 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:05.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee05e14700 1 -- 172.21.2.111:0/63031636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 msgr2=0x7fedf8042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee05e14700 1 --2- 172.21.2.111:0/63031636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 0x7fedf8042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee05e14700 1 -- 172.21.2.111:0/63031636 >> v1:172.21.2.111:6789/0 conn(0x7fedf803d360 legacy=0x7fedf80064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fedeffff700 1 -- 172.21.2.111:0/63031636 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155069949 0 0) 0x7fedf80dc6e0 con 0x7fedf803d360 2021-04-03T03:12:05.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee05e14700 1 -- 172.21.2.111:0/63031636 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80dc460 con 0x7fedf8006bc0 2021-04-03T03:12:05.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee05e14700 1 --2- 172.21.2.111:0/63031636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf8006bc0 0x7fedf80dbf70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fedf803e820 tx=0x7fee0005ad70).ready entity=mon.0 client_cookie=9bf74ac1a0bf4d9b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fedeffff700 1 -- 172.21.2.111:0/63031636 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0005bda0 con 0x7fedf8006bc0 2021-04-03T03:12:05.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fedeffff700 1 -- 172.21.2.111:0/63031636 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0005df60 con 0x7fedf8006bc0 2021-04-03T03:12:05.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fedeffff700 1 -- 172.21.2.111:0/63031636 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee000699f0 con 0x7fedf8006bc0 2021-04-03T03:12:05.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- 172.21.2.111:0/63031636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf8006bc0 msgr2=0x7fedf80dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 --2- 172.21.2.111:0/63031636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf8006bc0 0x7fedf80dbf70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fedf803e820 tx=0x7fee0005ad70).stop 2021-04-03T03:12:05.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- 172.21.2.111:0/63031636 shutdown_connections 2021-04-03T03:12:05.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 --2- 172.21.2.111:0/63031636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf8006bc0 0x7fedf80dbf70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 --2- 172.21.2.111:0/63031636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 0x7fedf8042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- 172.21.2.111:0/63031636 >> 172.21.2.111:0/63031636 conn(0x7fedf80462c0 msgr2=0x7fedf80466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- 172.21.2.111:0/63031636 shutdown_connections 2021-04-03T03:12:05.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- 172.21.2.111:0/63031636 wait complete. 2021-04-03T03:12:05.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 Processor -- start 2021-04-03T03:12:05.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.347+0000 7fee06615700 1 -- start start 2021-04-03T03:12:05.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf803d360 0x7fedf803a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 0x7fedf80dd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fedf80a3070 0x7fedf80ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf80de260 con 0x7fedf803d360 2021-04-03T03:12:05.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf8039e60 con 0x7fedf80a3070 2021-04-03T03:12:05.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf80e7e90 con 0x7fedf80dbb90 2021-04-03T03:12:05.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf803d360 0x7fedf803a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf803d360 0x7fedf803a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52296/0 (socket says 172.21.2.111:52296) 2021-04-03T03:12:05.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 -- 172.21.2.111:0/1729563726 learned_addr learned my addr 172.21.2.111:0/1729563726 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:05.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 -- 172.21.2.111:0/1729563726 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 msgr2=0x7fedf80dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 0x7fedf80dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 -- 172.21.2.111:0/1729563726 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fedf80a3070 msgr2=0x7fedf80ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fedf80a3070 0x7fedf80ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 -- 172.21.2.111:0/1729563726 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80e8130 con 0x7fedf803d360 2021-04-03T03:12:05.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee05613700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf803d360 0x7fedf803a240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fedf80de3a0 tx=0x7fedf0006e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fedee7fc700 1 -- 172.21.2.111:0/1729563726 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedf000d570 con 0x7fedf803d360 2021-04-03T03:12:05.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fedee7fc700 1 -- 172.21.2.111:0/1729563726 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf0000e90 con 0x7fedf803d360 2021-04-03T03:12:05.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf80de610 con 0x7fedf803d360 2021-04-03T03:12:05.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf80deb30 con 0x7fedf803d360 2021-04-03T03:12:05.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fedee7fc700 1 -- 172.21.2.111:0/1729563726 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedf000d8c0 con 0x7fedf803d360 2021-04-03T03:12:05.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc0033f0 con 0x7fedf803d360 2021-04-03T03:12:05.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fedee7fc700 1 -- 172.21.2.111:0/1729563726 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fedf000f070 con 0x7fedf803d360 2021-04-03T03:12:05.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fedee7fc700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fede0056ef0 0x7fede00593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fedee7fc700 1 -- 172.21.2.111:0/1729563726 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fedf006fc80 con 0x7fedf803d360 2021-04-03T03:12:05.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee04e12700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fede0056ef0 0x7fede00593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.351+0000 7fee04e12700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fede0056ef0 0x7fede00593b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fedfc000b80 tx=0x7fedfc001160).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.355+0000 7fedee7fc700 1 -- 172.21.2.111:0/1729563726 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fedf0076080 con 0x7fedf803d360 2021-04-03T03:12:05.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feddc0036a0 con 0x7fedf803d360 2021-04-03T03:12:05.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fedee7fc700 1 -- 172.21.2.111:0/1729563726 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16286 (secure 0 0 0) 0x7fedf0044f60 con 0x7fedf803d360 2021-04-03T03:12:05.460 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:05.461 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":53,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:12:05.023618+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:12:02.184726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-04-03T03:12:02.990138+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:12:05.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fede0056ef0 msgr2=0x7fede00593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fede0056ef0 0x7fede00593b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fedfc000b80 tx=0x7fedfc001160).stop 2021-04-03T03:12:05.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf803d360 msgr2=0x7fedf803a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf803d360 0x7fedf803a240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fedf80de3a0 tx=0x7fedf0006e00).stop 2021-04-03T03:12:05.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 shutdown_connections 2021-04-03T03:12:05.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fede0056ef0 0x7fede00593b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fedf80a3070 0x7fedf80ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fedf803d360 0x7fedf803a240 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 --2- 172.21.2.111:0/1729563726 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fedf80dbb90 0x7fedf80dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 >> 172.21.2.111:0/1729563726 conn(0x7fedf80462c0 msgr2=0x7fedf80d7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 shutdown_connections 2021-04-03T03:12:05.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.447+0000 7fee06615700 1 -- 172.21.2.111:0/1729563726 wait complete. 2021-04-03T03:12:05.468 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:12:05.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 Processor -- start 2021-04-03T03:12:05.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- start start 2021-04-03T03:12:05.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3724153440 0x7f3724080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3724025480 0x7f37240af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240b3ca0 con 0x7f37240b38c0 2021-04-03T03:12:05.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3724025b90 con 0x7f3724025480 2021-04-03T03:12:05.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3724025e30 con 0x7f3724153440 2021-04-03T03:12:05.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372a458700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f37240b38c0 0x7f3724080550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53096/0 (socket says 172.21.2.111:53096) 2021-04-03T03:12:05.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372a458700 1 -- 172.21.2.111:0/1287150351 learned_addr learned my addr 172.21.2.111:0/1287150351 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:05.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372c0d3700 1 --2- 172.21.2.111:0/1287150351 >> v2:172.21.2.111:3300/0 conn(0x7f3724025480 0x7f37240af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372c0d3700 1 -- 172.21.2.111:0/1287150351 >> v2:172.21.2.115:3300/0 conn(0x7f3724153440 msgr2=0x7f3724080c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:12:05.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372c0d3700 1 --2- 172.21.2.111:0/1287150351 >> v2:172.21.2.115:3300/0 conn(0x7f3724153440 0x7f3724080c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372c0d3700 1 -- 172.21.2.111:0/1287150351 >> v1:172.21.2.111:6789/0 conn(0x7f37240b38c0 legacy=0x7f3724080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372c0d3700 1 -- 172.21.2.111:0/1287150351 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240260d0 con 0x7f3724025480 2021-04-03T03:12:05.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372c0d3700 1 --2- 172.21.2.111:0/1287150351 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240af2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f371c0032b0 tx=0x7f371c0091d0).ready entity=mon.0 client_cookie=fe6c275ae36a86d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f3729456700 1 -- 172.21.2.111:0/1287150351 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c00ba50 con 0x7f3724025480 2021-04-03T03:12:05.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f3729456700 1 -- 172.21.2.111:0/1287150351 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c00bbb0 con 0x7f3724025480 2021-04-03T03:12:05.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f3729456700 1 -- 172.21.2.111:0/1287150351 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c00bd20 con 0x7f3724025480 2021-04-03T03:12:05.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- 172.21.2.111:0/1287150351 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 msgr2=0x7f37240af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 --2- 172.21.2.111:0/1287150351 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240af2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f371c0032b0 tx=0x7f371c0091d0).stop 2021-04-03T03:12:05.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- 172.21.2.111:0/1287150351 shutdown_connections 2021-04-03T03:12:05.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 --2- 172.21.2.111:0/1287150351 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240af2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 --2- 172.21.2.111:0/1287150351 >> v2:172.21.2.115:3300/0 conn(0x7f3724153440 0x7f3724080c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- 172.21.2.111:0/1287150351 >> 172.21.2.111:0/1287150351 conn(0x7f3724061610 msgr2=0x7f3724061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- 172.21.2.111:0/1287150351 shutdown_connections 2021-04-03T03:12:05.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.515+0000 7f372ac59700 1 -- 172.21.2.111:0/1287150351 wait complete. 2021-04-03T03:12:05.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 Processor -- start 2021-04-03T03:12:05.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 -- start start 2021-04-03T03:12:05.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f37240b38c0 0x7f3724063f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3724153440 0x7f37240668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724065010 con 0x7f3724025480 2021-04-03T03:12:05.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372407d950 con 0x7f37240b38c0 2021-04-03T03:12:05.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37240b5800 con 0x7f3724153440 2021-04-03T03:12:05.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52310/0 (socket says 172.21.2.111:52310) 2021-04-03T03:12:05.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 -- 172.21.2.111:0/4086185720 learned_addr learned my addr 172.21.2.111:0/4086185720 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:05.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 -- 172.21.2.111:0/4086185720 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3724153440 msgr2=0x7f37240668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3724153440 0x7f37240668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 -- 172.21.2.111:0/4086185720 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f37240b38c0 msgr2=0x7f3724063f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f37240b38c0 0x7f3724063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 -- 172.21.2.111:0/4086185720 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724066430 con 0x7f3724025480 2021-04-03T03:12:05.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372a458700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240acf80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f37240af0b0 tx=0x7f3718007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3716ffd700 1 -- 172.21.2.111:0/4086185720 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3718007320 con 0x7f3724025480 2021-04-03T03:12:05.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3716ffd700 1 -- 172.21.2.111:0/4086185720 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3718007480 con 0x7f3724025480 2021-04-03T03:12:05.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3724067ec0 con 0x7f3724025480 2021-04-03T03:12:05.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37240b52f0 con 0x7f3724025480 2021-04-03T03:12:05.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3716ffd700 1 -- 172.21.2.111:0/4086185720 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37180057e0 con 0x7f3724025480 2021-04-03T03:12:05.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37240b5650 con 0x7f3724025480 2021-04-03T03:12:05.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3716ffd700 1 -- 172.21.2.111:0/4086185720 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3718010030 con 0x7f3724025480 2021-04-03T03:12:05.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3716ffd700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f370c057000 0x7f370c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3716ffd700 1 -- 172.21.2.111:0/4086185720 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f371806ff10 con 0x7f3724025480 2021-04-03T03:12:05.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3729c57700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f370c057000 0x7f370c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.519+0000 7f3729c57700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f370c057000 0x7f370c0594c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f3720000e90 tx=0x7f3720006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.523+0000 7f3716ffd700 1 -- 172.21.2.111:0/4086185720 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3718045190 con 0x7f3724025480 2021-04-03T03:12:05.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.647+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f37240665c0 con 0x7f3724025480 2021-04-03T03:12:05.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.651+0000 7f3716ffd700 1 -- 172.21.2.111:0/4086185720 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f37180487b0 con 0x7f3724025480 2021-04-03T03:12:05.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f370c057000 msgr2=0x7f370c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f370c057000 0x7f370c0594c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f3720000e90 tx=0x7f3720006910).stop 2021-04-03T03:12:05.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 msgr2=0x7f37240acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240acf80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f37240af0b0 tx=0x7f3718007c40).stop 2021-04-03T03:12:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 shutdown_connections 2021-04-03T03:12:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3724153440 0x7f37240668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3724025480 0x7f37240acf80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f370c057000 0x7f370c0594c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 --2- 172.21.2.111:0/4086185720 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f37240b38c0 0x7f3724063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 >> 172.21.2.111:0/4086185720 conn(0x7f3724061610 msgr2=0x7f3724080550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 shutdown_connections 2021-04-03T03:12:05.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.655+0000 7f372ac59700 1 -- 172.21.2.111:0/4086185720 wait complete. 2021-04-03T03:12:05.671 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:12:05.675 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:12:05.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 Processor -- start 2021-04-03T03:12:05.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- start start 2021-04-03T03:12:05.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff2f8080c60 0x7ff2f8025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f8081090 con 0x7ff2f8153440 2021-04-03T03:12:05.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f80b3ca0 con 0x7ff2f80b38c0 2021-04-03T03:12:05.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f8025970 con 0x7ff2f8080c60 2021-04-03T03:12:05.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff2f8153440 0x7ff2f80af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53112/0 (socket says 172.21.2.111:53112) 2021-04-03T03:12:05.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 -- 172.21.2.111:0/839544307 learned_addr learned my addr 172.21.2.111:0/839544307 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:05.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59006498 0 0) 0x7ff2e80010d0 con 0x7ff2f8153440 2021-04-03T03:12:05.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e4003240 con 0x7ff2f8153440 2021-04-03T03:12:05.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1157472276 0 0) 0x7ff2e4003240 con 0x7ff2f8153440 2021-04-03T03:12:05.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 >> v2:172.21.2.115:3300/0 conn(0x7ff2f8080c60 msgr2=0x7ff2f8025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 --2- 172.21.2.111:0/839544307 >> v2:172.21.2.115:3300/0 conn(0x7ff2f8080c60 0x7ff2f8025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 >> v1:172.21.2.115:6789/0 conn(0x7ff2f80b38c0 legacy=0x7ff2f8080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f8025b90 con 0x7ff2f8153440 2021-04-03T03:12:05.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ff2e8001860 con 0x7ff2f8153440 2021-04-03T03:12:05.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 >> v1:172.21.2.111:6789/0 conn(0x7ff2f8153440 legacy=0x7ff2f80af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2e40037b0 0x7ff2e4003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2e4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2e4004e50 0x7ff2e400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8025970 con 0x7ff2e40044e0 2021-04-03T03:12:05.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e4005300 con 0x7ff2e4004e50 2021-04-03T03:12:05.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e400bc20 con 0x7ff2e40037b0 2021-04-03T03:12:05.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2e4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 -- 172.21.2.111:0/839544307 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2e40037b0 msgr2=0x7ff2e4003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2e40037b0 0x7ff2e4003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 -- 172.21.2.111:0/839544307 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2e4004e50 msgr2=0x7ff2e400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2e4004e50 0x7ff2e400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 -- 172.21.2.111:0/839544307 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f8025b90 con 0x7ff2e40044e0 2021-04-03T03:12:05.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fdf77700 1 -- 172.21.2.111:0/839544307 reap_dead start 2021-04-03T03:12:05.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f77fe700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2e4004960 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7ff2e8000fe0 tx=0x7ff2e8007990).ready entity=mon.0 client_cookie=e47c9d50d0896746 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e80055f0 con 0x7ff2e40044e0 2021-04-03T03:12:05.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e8005750 con 0x7ff2e40044e0 2021-04-03T03:12:05.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2f6ffd700 1 -- 172.21.2.111:0/839544307 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e8005ab0 con 0x7ff2e40044e0 2021-04-03T03:12:05.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- 172.21.2.111:0/839544307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 msgr2=0x7ff2e4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2e4004960 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7ff2e8000fe0 tx=0x7ff2e8007990).stop 2021-04-03T03:12:05.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- 172.21.2.111:0/839544307 shutdown_connections 2021-04-03T03:12:05.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/839544307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2e4004960 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- 172.21.2.111:0/839544307 >> 172.21.2.111:0/839544307 conn(0x7ff2f8061610 msgr2=0x7ff2f8061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- 172.21.2.111:0/839544307 shutdown_connections 2021-04-03T03:12:05.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- 172.21.2.111:0/839544307 wait complete. 2021-04-03T03:12:05.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 Processor -- start 2021-04-03T03:12:05.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.763+0000 7ff2fcafd700 1 -- start start 2021-04-03T03:12:05.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2f80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2f80ad3a0 0x7ff2f8150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2f8150720 0x7ff2f8141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8065e70 con 0x7ff2e40044e0 2021-04-03T03:12:05.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8067b80 con 0x7ff2f8150720 2021-04-03T03:12:05.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8066280 con 0x7ff2f80ad3a0 2021-04-03T03:12:05.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2f80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2f80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52328/0 (socket says 172.21.2.111:52328) 2021-04-03T03:12:05.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 -- 172.21.2.111:0/164347460 learned_addr learned my addr 172.21.2.111:0/164347460 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:05.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 -- 172.21.2.111:0/164347460 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2f80ad3a0 msgr2=0x7ff2f8150230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:12:05.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2f80ad3a0 0x7ff2f8150230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 -- 172.21.2.111:0/164347460 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2f8150720 msgr2=0x7ff2f8141e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:12:05.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2f8150720 0x7ff2f8141e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 -- 172.21.2.111:0/164347460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80acf10 con 0x7ff2e40044e0 2021-04-03T03:12:05.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f7fff700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2f80ac8d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7ff2e0000f30 tx=0x7ff2e0006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f4ff9700 1 -- 172.21.2.111:0/164347460 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e000d570 con 0x7ff2e40044e0 2021-04-03T03:12:05.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f4ff9700 1 -- 172.21.2.111:0/164347460 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e000d6d0 con 0x7ff2e40044e0 2021-04-03T03:12:05.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f4ff9700 1 -- 172.21.2.111:0/164347460 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e000da30 con 0x7ff2e40044e0 2021-04-03T03:12:05.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f80aabc0 con 0x7ff2e40044e0 2021-04-03T03:12:05.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f80ab710 con 0x7ff2e40044e0 2021-04-03T03:12:05.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f80679e0 con 0x7ff2e40044e0 2021-04-03T03:12:05.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f4ff9700 1 -- 172.21.2.111:0/164347460 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff2e000f070 con 0x7ff2e40044e0 2021-04-03T03:12:05.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f4ff9700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2dc056f10 0x7ff2dc0593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f4ff9700 1 -- 172.21.2.111:0/164347460 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff2e006fdf0 con 0x7ff2e40044e0 2021-04-03T03:12:05.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f77fe700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2dc056f10 0x7ff2dc0593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.767+0000 7ff2f77fe700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2dc056f10 0x7ff2dc0593d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7ff2e8000fe0 tx=0x7ff2e8007990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:05.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.771+0000 7ff2f4ff9700 1 -- 172.21.2.111:0/164347460 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff2f80679e0 con 0x7ff2e40044e0 2021-04-03T03:12:05.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.899+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff2f80679e0 con 0x7ff2e40044e0 2021-04-03T03:12:05.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2f4ff9700 1 -- 172.21.2.111:0/164347460 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff2f80679e0 con 0x7ff2e40044e0 2021-04-03T03:12:05.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2dc056f10 msgr2=0x7ff2dc0593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2dc056f10 0x7ff2dc0593d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7ff2e8000fe0 tx=0x7ff2e8007990).stop 2021-04-03T03:12:05.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 msgr2=0x7ff2f80ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2f80ac8d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7ff2e0000f30 tx=0x7ff2e0006dd0).stop 2021-04-03T03:12:05.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 shutdown_connections 2021-04-03T03:12:05.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2dc056f10 0x7ff2dc0593d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2f8150720 0x7ff2f8141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2e40044e0 0x7ff2f80ac8d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 --2- 172.21.2.111:0/164347460 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2f80ad3a0 0x7ff2f8150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:05.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 >> 172.21.2.111:0/164347460 conn(0x7ff2f8061610 msgr2=0x7ff2f80b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:05.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 shutdown_connections 2021-04-03T03:12:05.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.903+0000 7ff2fcafd700 1 -- 172.21.2.111:0/164347460 wait complete. 2021-04-03T03:12:05.919 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:12:05.923 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:12:05.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 Processor -- start 2021-04-03T03:12:05.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- start start 2021-04-03T03:12:05.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f15f80b38c0 0x7f15f8025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 0x7f15f8080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f80b3ca0 con 0x7f15f8080a40 2021-04-03T03:12:05.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f80b0300 con 0x7f15f80b38c0 2021-04-03T03:12:05.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8025970 con 0x7f15f8153440 2021-04-03T03:12:05.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fe351700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f15f8080a40 0x7f15f814feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53130/0 (socket says 172.21.2.111:53130) 2021-04-03T03:12:05.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fe351700 1 -- 172.21.2.111:0/2916732088 learned_addr learned my addr 172.21.2.111:0/2916732088 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:05.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364734772 0 0) 0x7f15f80b3ca0 con 0x7f15f8080a40 2021-04-03T03:12:05.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e4003240 con 0x7f15f8080a40 2021-04-03T03:12:05.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f77fe700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 0x7f15f8080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 --2- 172.21.2.111:0/2916732088 >> v2:172.21.2.115:3300/0 conn(0x7f15f80b38c0 0x7f15f8025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3166284915 0 0) 0x7f15e4003240 con 0x7f15f8080a40 2021-04-03T03:12:05.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 msgr2=0x7f15f8080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 0x7f15f8080550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 >> v2:172.21.2.115:3300/0 conn(0x7f15f80b38c0 msgr2=0x7f15f8025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 --2- 172.21.2.111:0/2916732088 >> v2:172.21.2.115:3300/0 conn(0x7f15f80b38c0 0x7f15f8025480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:05.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8025b90 con 0x7f15f8080a40 2021-04-03T03:12:05.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f15f0001a50 con 0x7f15f8080a40 2021-04-03T03:12:05.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 >> v1:172.21.2.111:6789/0 conn(0x7f15f8080a40 legacy=0x7f15f814feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:05.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15e40037b0 0x7f15e4003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 0x7f15e40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15e40049d0 0x7f15e4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:05.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80b0300 con 0x7f15e40037b0 2021-04-03T03:12:05.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8025970 con 0x7f15e40049d0 2021-04-03T03:12:05.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40042a0 con 0x7f15f8153440 2021-04-03T03:12:05.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15e40037b0 0x7f15e4003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f77fe700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 0x7f15e40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fe351700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15e40049d0 0x7f15e4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 -- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 msgr2=0x7f15e40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f8153440 0x7f15e40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 -- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15e40049d0 msgr2=0x7f15e4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15e40049d0 0x7f15e4004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 -- 172.21.2.111:0/2916732088 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4003240 con 0x7f15e40037b0 2021-04-03T03:12:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fe351700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15e40049d0 0x7f15e4004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:12:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f7fff700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15e40037b0 0x7f15e4003bd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f15e40054b0 tx=0x7f15e0008040).ready entity=mon.0 client_cookie=4742d8acce34370e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15e000ebc0 con 0x7f15e40037b0 2021-04-03T03:12:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e000ed20 con 0x7f15e40037b0 2021-04-03T03:12:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15f6ffd700 1 -- 172.21.2.111:0/2916732088 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15e00125e0 con 0x7f15e40037b0 2021-04-03T03:12:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- 172.21.2.111:0/2916732088 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15e40037b0 msgr2=0x7f15e4003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 --2- 172.21.2.111:0/2916732088 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15e40037b0 0x7f15e4003bd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f15e40054b0 tx=0x7f15e0008040).stop 2021-04-03T03:12:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fe351700 1 -- 172.21.2.111:0/2916732088 reap_dead start 2021-04-03T03:12:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- 172.21.2.111:0/2916732088 shutdown_connections 2021-04-03T03:12:06.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- 172.21.2.111:0/2916732088 >> 172.21.2.111:0/2916732088 conn(0x7f15f8061610 msgr2=0x7f15f8061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- 172.21.2.111:0/2916732088 shutdown_connections 2021-04-03T03:12:06.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.971+0000 7f15fced7700 1 -- 172.21.2.111:0/2916732088 wait complete. 2021-04-03T03:12:06.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 Processor -- start 2021-04-03T03:12:06.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 -- start start 2021-04-03T03:12:06.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15f8064260 0x7f15f8065ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15f80aa980 0x7f15f813f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f813f800 0x7f15f8064a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80ad430 con 0x7f15f80aa980 2021-04-03T03:12:06.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8064640 con 0x7f15f8064260 2021-04-03T03:12:06.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80ad690 con 0x7f15f813f800 2021-04-03T03:12:06.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15f80aa980 0x7f15f813f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15f80aa980 0x7f15f813f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52350/0 (socket says 172.21.2.111:52350) 2021-04-03T03:12:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 -- 172.21.2.111:0/3432934344 learned_addr learned my addr 172.21.2.111:0/3432934344 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f7fff700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15f8064260 0x7f15f8065ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fe351700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f813f800 0x7f15f8064a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 -- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f813f800 msgr2=0x7f15f8064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f813f800 0x7f15f8064a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 -- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15f8064260 msgr2=0x7f15f8065ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15f8064260 0x7f15f8065ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 -- 172.21.2.111:0/3432934344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80ac8f0 con 0x7f15f80aa980 2021-04-03T03:12:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f77fe700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15f80aa980 0x7f15f813f310 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f15e8008000 tx=0x7f15e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f4ff9700 1 -- 172.21.2.111:0/3432934344 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15e800f960 con 0x7f15f80aa980 2021-04-03T03:12:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f4ff9700 1 -- 172.21.2.111:0/3432934344 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e800fac0 con 0x7f15f80aa980 2021-04-03T03:12:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f813fc80 con 0x7f15f80aa980 2021-04-03T03:12:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f8067e00 con 0x7f15f80aa980 2021-04-03T03:12:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f4ff9700 1 -- 172.21.2.111:0/3432934344 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15e800fc30 con 0x7f15f80aa980 2021-04-03T03:12:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d00033f0 con 0x7f15f80aa980 2021-04-03T03:12:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f4ff9700 1 -- 172.21.2.111:0/3432934344 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f15e8024070 con 0x7f15f80aa980 2021-04-03T03:12:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f4ff9700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f15dc057000 0x7f15dc0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f4ff9700 1 -- 172.21.2.111:0/3432934344 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f15e8070af0 con 0x7f15f80aa980 2021-04-03T03:12:06.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f7fff700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f15dc057000 0x7f15dc0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.975+0000 7f15f7fff700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f15dc057000 0x7f15dc0594c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f15e001b000 tx=0x7f15e0019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:05.979+0000 7f15f4ff9700 1 -- 172.21.2.111:0/3432934344 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f15e8045dd0 con 0x7f15f80aa980 2021-04-03T03:12:06.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.111+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f15d00036a0 con 0x7f15f80aa980 2021-04-03T03:12:06.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.111+0000 7f15f4ff9700 1 -- 172.21.2.111:0/3432934344 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f15e80493f0 con 0x7f15f80aa980 2021-04-03T03:12:06.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f15dc057000 msgr2=0x7f15dc0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f15dc057000 0x7f15dc0594c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f15e001b000 tx=0x7f15e0019000).stop 2021-04-03T03:12:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15f80aa980 msgr2=0x7f15f813f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15f80aa980 0x7f15f813f310 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f15e8008000 tx=0x7f15e8008040).stop 2021-04-03T03:12:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 shutdown_connections 2021-04-03T03:12:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15f813f800 0x7f15f8064a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f15dc057000 0x7f15dc0594c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15f8064260 0x7f15f8065ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 --2- 172.21.2.111:0/3432934344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15f80aa980 0x7f15f813f310 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 >> 172.21.2.111:0/3432934344 conn(0x7f15f8061610 msgr2=0x7f15f8132050 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 shutdown_connections 2021-04-03T03:12:06.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.115+0000 7f15fced7700 1 -- 172.21.2.111:0/3432934344 wait complete. 2021-04-03T03:12:06.132 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:12:06.135 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:12:06.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 Processor -- start 2021-04-03T03:12:06.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- start start 2021-04-03T03:12:06.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 0x7f766c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c0607d0 con 0x7f766c0b0c90 2021-04-03T03:12:06.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c0af400 con 0x7f766c14b700 2021-04-03T03:12:06.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0af6a0 con 0x7f766c0603f0 2021-04-03T03:12:06.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f766c0b0c90 0x7f766c147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53152/0 (socket says 172.21.2.111:53152) 2021-04-03T03:12:06.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 -- 172.21.2.111:0/1006602138 learned_addr learned my addr 172.21.2.111:0/1006602138 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:06.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468292026 0 0) 0x7f766c0607d0 con 0x7f766c0b0c90 2021-04-03T03:12:06.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7654003240 con 0x7f766c0b0c90 2021-04-03T03:12:06.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7671a87700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 0x7f766c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4009931639 0 0) 0x7f7654003240 con 0x7f766c0b0c90 2021-04-03T03:12:06.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 msgr2=0x7f766c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:06.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 0x7f766c0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 >> v1:172.21.2.115:6789/0 conn(0x7f766c14b700 legacy=0x7f766c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0afb40 con 0x7f766c0b0c90 2021-04-03T03:12:06.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f7664001a50 con 0x7f766c0b0c90 2021-04-03T03:12:06.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 >> v1:172.21.2.111:6789/0 conn(0x7f766c0b0c90 legacy=0x7f766c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7654003760 0x7f7654003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 0x7f76540044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76540049d0 0x7f7654004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0af6a0 con 0x7f76540049d0 2021-04-03T03:12:06.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654004170 con 0x7f7654003760 2021-04-03T03:12:06.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7654004330 con 0x7f766c0603f0 2021-04-03T03:12:06.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76540049d0 0x7f7654004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7671286700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 0x7f76540044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7671a87700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7654003760 0x7f7654003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 -- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 msgr2=0x7f76540044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:06.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c0603f0 0x7f76540044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 -- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7654003760 msgr2=0x7f7654003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:06.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7654003760 0x7f7654003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 -- 172.21.2.111:0/1006602138 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7654003240 con 0x7f76540049d0 2021-04-03T03:12:06.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76540049d0 0x7f7654004e50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f7664001760 tx=0x7f766c0ae670).ready entity=mon.0 client_cookie=d3ab7595ecc1b98 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7664005b80 con 0x7f76540049d0 2021-04-03T03:12:06.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7664005ce0 con 0x7f76540049d0 2021-04-03T03:12:06.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7670a85700 1 -- 172.21.2.111:0/1006602138 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76640072c0 con 0x7f76540049d0 2021-04-03T03:12:06.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- 172.21.2.111:0/1006602138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76540049d0 msgr2=0x7f7654004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 --2- 172.21.2.111:0/1006602138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76540049d0 0x7f7654004e50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f7664001760 tx=0x7f766c0ae670).stop 2021-04-03T03:12:06.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672288700 1 -- 172.21.2.111:0/1006602138 reap_dead start 2021-04-03T03:12:06.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- 172.21.2.111:0/1006602138 shutdown_connections 2021-04-03T03:12:06.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- 172.21.2.111:0/1006602138 >> 172.21.2.111:0/1006602138 conn(0x7f766c0b61c0 msgr2=0x7f766c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- 172.21.2.111:0/1006602138 shutdown_connections 2021-04-03T03:12:06.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.179+0000 7f7672a89700 1 -- 172.21.2.111:0/1006602138 wait complete. 2021-04-03T03:12:06.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 Processor -- start 2021-04-03T03:12:06.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 -- start start 2021-04-03T03:12:06.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f766c1599c0 0x7f766c062780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f766c0656f0 0x7f766c0665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c066a90 0x7f766c146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c14ea50 con 0x7f766c0656f0 2021-04-03T03:12:06.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c064c80 con 0x7f766c1599c0 2021-04-03T03:12:06.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c15a1c0 con 0x7f766c066a90 2021-04-03T03:12:06.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f766c0656f0 0x7f766c0665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f766c0656f0 0x7f766c0665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52370/0 (socket says 172.21.2.111:52370) 2021-04-03T03:12:06.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 -- 172.21.2.111:0/2812063349 learned_addr learned my addr 172.21.2.111:0/2812063349 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:06.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672288700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c066a90 0x7f766c146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671a87700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f766c1599c0 0x7f766c062780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 -- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c066a90 msgr2=0x7f766c146460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c066a90 0x7f766c146460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 -- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f766c1599c0 msgr2=0x7f766c062780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f766c1599c0 0x7f766c062780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 -- 172.21.2.111:0/2812063349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c064ee0 con 0x7f766c0656f0 2021-04-03T03:12:06.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671a87700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f766c1599c0 0x7f766c062780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:12:06.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671286700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f766c0656f0 0x7f766c0665a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f766000b000 tx=0x7f7660009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f765effd700 1 -- 172.21.2.111:0/2812063349 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766000a6d0 con 0x7f766c0656f0 2021-04-03T03:12:06.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f765effd700 1 -- 172.21.2.111:0/2812063349 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660010040 con 0x7f766c0656f0 2021-04-03T03:12:06.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c159da0 con 0x7f766c0656f0 2021-04-03T03:12:06.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c064a10 con 0x7f766c0656f0 2021-04-03T03:12:06.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f765effd700 1 -- 172.21.2.111:0/2812063349 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766000aa20 con 0x7f766c0656f0 2021-04-03T03:12:06.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c066ea0 con 0x7f766c0656f0 2021-04-03T03:12:06.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f765effd700 1 -- 172.21.2.111:0/2812063349 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f766001b030 con 0x7f766c0656f0 2021-04-03T03:12:06.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f765effd700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f764c057000 0x7f764c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f765effd700 1 -- 172.21.2.111:0/2812063349 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f766006fc00 con 0x7f766c0656f0 2021-04-03T03:12:06.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671a87700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f764c057000 0x7f764c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.183+0000 7f7671a87700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f764c057000 0x7f764c0594c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7668008000 tx=0x7f7668008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.187+0000 7f765effd700 1 -- 172.21.2.111:0/2812063349 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7660044ee0 con 0x7f766c0656f0 2021-04-03T03:12:06.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f766c0652a0 con 0x7f766c0656f0 2021-04-03T03:12:06.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f765effd700 1 -- 172.21.2.111:0/2812063349 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f7660048500 con 0x7f766c0656f0 2021-04-03T03:12:06.325 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:06.325 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:12:06.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f764c057000 msgr2=0x7f764c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f764c057000 0x7f764c0594c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7668008000 tx=0x7f7668008040).stop 2021-04-03T03:12:06.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f766c0656f0 msgr2=0x7f766c0665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f766c0656f0 0x7f766c0665a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f766000b000 tx=0x7f7660009040).stop 2021-04-03T03:12:06.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 shutdown_connections 2021-04-03T03:12:06.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f766c066a90 0x7f766c146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f764c057000 0x7f764c0594c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f766c1599c0 0x7f766c062780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 --2- 172.21.2.111:0/2812063349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f766c0656f0 0x7f766c0665a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 >> 172.21.2.111:0/2812063349 conn(0x7f766c0b61c0 msgr2=0x7f766c067870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 shutdown_connections 2021-04-03T03:12:06.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.311+0000 7f7672a89700 1 -- 172.21.2.111:0/2812063349 wait complete. 2021-04-03T03:12:06.329 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:12:06.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:12:04.020364+0000', 'modified': '2021-04-03T03:12:05.022670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5580': {'gid': 5580, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/2278897120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 2278897120}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 2278897120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:12:06.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:12:06.535 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:12:06.619 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5580,"want_state":"up:active","state":"up:active","rank_uptime":2.5835939419999998,"mdsmap_epoch":21,"osdmap_epoch":54,"osdmap_epoch_barrier":52,"uptime":6.7200992099999999} 2021-04-03T03:12:06.620 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5580, "mdsmap_epoch": 21, "osdmap_epoch": 54, "osdmap_epoch_barrier": 52, "rank_uptime": 2.583593942, "state": "up:active", "uptime": 6.72009921, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:12:06.621 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:12:06.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 Processor -- start 2021-04-03T03:12:06.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 -- start start 2021-04-03T03:12:06.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d014b700 0x7f95d00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f95d00b0c90 0x7f95d0147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d00607d0 con 0x7f95d00603f0 2021-04-03T03:12:06.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00af400 con 0x7f95d00b0c90 2021-04-03T03:12:06.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00af6a0 con 0x7f95d014b700 2021-04-03T03:12:06.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d541f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f95d00603f0 0x7f95d00b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53168/0 (socket says 172.21.2.111:53168) 2021-04-03T03:12:06.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d541f700 1 -- 172.21.2.111:0/657345703 learned_addr learned my addr 172.21.2.111:0/657345703 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:06.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d5c20700 1 --2- 172.21.2.111:0/657345703 >> v2:172.21.2.111:3300/0 conn(0x7f95d00b0c90 0x7f95d0147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d4c1e700 1 --2- 172.21.2.111:0/657345703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d014b700 0x7f95d00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d5c20700 1 -- 172.21.2.111:0/657345703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d014b700 msgr2=0x7f95d00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:06.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d5c20700 1 --2- 172.21.2.111:0/657345703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d014b700 0x7f95d00b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d5c20700 1 -- 172.21.2.111:0/657345703 >> v1:172.21.2.111:6789/0 conn(0x7f95d00603f0 legacy=0x7f95d00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d5c20700 1 -- 172.21.2.111:0/657345703 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00afb40 con 0x7f95d00b0c90 2021-04-03T03:12:06.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d5c20700 1 --2- 172.21.2.111:0/657345703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00b0c90 0x7f95d0147680 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f95c80032b0 tx=0x7f95c80091d0).ready entity=mon.0 client_cookie=1b60a6385eff5391 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95c3fff700 1 -- 172.21.2.111:0/657345703 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c800b9a0 con 0x7f95d00b0c90 2021-04-03T03:12:06.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95c3fff700 1 -- 172.21.2.111:0/657345703 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800bb00 con 0x7f95d00b0c90 2021-04-03T03:12:06.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95c3fff700 1 -- 172.21.2.111:0/657345703 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c800bc70 con 0x7f95d00b0c90 2021-04-03T03:12:06.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 -- 172.21.2.111:0/657345703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00b0c90 msgr2=0x7f95d0147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.667+0000 7f95d6421700 1 --2- 172.21.2.111:0/657345703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00b0c90 0x7f95d0147680 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f95c80032b0 tx=0x7f95c80091d0).stop 2021-04-03T03:12:06.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- 172.21.2.111:0/657345703 shutdown_connections 2021-04-03T03:12:06.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 --2- 172.21.2.111:0/657345703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00b0c90 0x7f95d0147680 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 --2- 172.21.2.111:0/657345703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d014b700 0x7f95d00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- 172.21.2.111:0/657345703 >> 172.21.2.111:0/657345703 conn(0x7f95d00b61c0 msgr2=0x7f95d005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- 172.21.2.111:0/657345703 shutdown_connections 2021-04-03T03:12:06.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- 172.21.2.111:0/657345703 wait complete. 2021-04-03T03:12:06.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 Processor -- start 2021-04-03T03:12:06.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- start start 2021-04-03T03:12:06.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00603f0 0x7f95d00aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d00b0c90 0x7f95d0156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95d014b700 0x7f95d0157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0062d00 con 0x7f95d00603f0 2021-04-03T03:12:06.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0062fa0 con 0x7f95d014b700 2021-04-03T03:12:06.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00ac0d0 con 0x7f95d00b0c90 2021-04-03T03:12:06.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00603f0 0x7f95d00aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00603f0 0x7f95d00aafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52384/0 (socket says 172.21.2.111:52384) 2021-04-03T03:12:06.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 -- 172.21.2.111:0/376247062 learned_addr learned my addr 172.21.2.111:0/376247062 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:06.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d4c1e700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d00b0c90 0x7f95d0156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d5c20700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95d014b700 0x7f95d0157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 -- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d00b0c90 msgr2=0x7f95d0156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d00b0c90 0x7f95d0156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 -- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95d014b700 msgr2=0x7f95d0157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95d014b700 0x7f95d0157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 -- 172.21.2.111:0/376247062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0156700 con 0x7f95d00603f0 2021-04-03T03:12:06.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d4c1e700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d00b0c90 0x7f95d0156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:12:06.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d5c20700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95d014b700 0x7f95d0157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:12:06.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d541f700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00603f0 0x7f95d00aafc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f95d00af200 tx=0x7f95c4007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95c27fc700 1 -- 172.21.2.111:0/376247062 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c4007320 con 0x7f95d00603f0 2021-04-03T03:12:06.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95c27fc700 1 -- 172.21.2.111:0/376247062 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4007480 con 0x7f95d00603f0 2021-04-03T03:12:06.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95c27fc700 1 -- 172.21.2.111:0/376247062 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c40057e0 con 0x7f95d00603f0 2021-04-03T03:12:06.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d00a7eb0 con 0x7f95d00603f0 2021-04-03T03:12:06.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00aaa30 con 0x7f95d00603f0 2021-04-03T03:12:06.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d0057660 con 0x7f95d00603f0 2021-04-03T03:12:06.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95c27fc700 1 -- 172.21.2.111:0/376247062 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f95c4010030 con 0x7f95d00603f0 2021-04-03T03:12:06.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95c27fc700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95b0056fb0 0x7f95b0059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95c27fc700 1 -- 172.21.2.111:0/376247062 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f95c406fee0 con 0x7f95d00603f0 2021-04-03T03:12:06.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.671+0000 7f95d4c1e700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95b0056fb0 0x7f95b0059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.675+0000 7f95d4c1e700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95b0056fb0 0x7f95b0059470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f95cc00b000 tx=0x7f95cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.675+0000 7f95c27fc700 1 -- 172.21.2.111:0/376247062 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f95c40451c0 con 0x7f95d00603f0 2021-04-03T03:12:06.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95d00686d0 con 0x7f95d00603f0 2021-04-03T03:12:06.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95c27fc700 1 -- 172.21.2.111:0/376247062 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f95c40487e0 con 0x7f95d00603f0 2021-04-03T03:12:06.781 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:06.782 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":54,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:12:06.034712+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:12:02.184726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-04-03T03:12:02.990138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:12:06.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95b0056fb0 msgr2=0x7f95b0059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95b0056fb0 0x7f95b0059470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f95cc00b000 tx=0x7f95cc008040).stop 2021-04-03T03:12:06.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00603f0 msgr2=0x7f95d00aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00603f0 0x7f95d00aafc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f95d00af200 tx=0x7f95c4007be0).stop 2021-04-03T03:12:06.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 shutdown_connections 2021-04-03T03:12:06.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95b0056fb0 0x7f95b0059470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95d014b700 0x7f95d0157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95d00603f0 0x7f95d00aafc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 --2- 172.21.2.111:0/376247062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95d00b0c90 0x7f95d0156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 >> 172.21.2.111:0/376247062 conn(0x7f95d00b61c0 msgr2=0x7f95d014bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 shutdown_connections 2021-04-03T03:12:06.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.767+0000 7f95d6421700 1 -- 172.21.2.111:0/376247062 wait complete. 2021-04-03T03:12:06.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:12:02.184726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:12:02.990138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:12:06.791 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:12:06.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 Processor -- start 2021-04-03T03:12:06.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 -- start start 2021-04-03T03:12:06.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f60400603f0 0x7f60401447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 0x7f6040144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60400607d0 con 0x7f60400add50 2021-04-03T03:12:06.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60401451c0 con 0x7f60400603f0 2021-04-03T03:12:06.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604014b090 con 0x7f60400b17b0 2021-04-03T03:12:06.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f60400603f0 0x7f60401447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f60400603f0 0x7f60401447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52392/0 (socket says 172.21.2.111:52392) 2021-04-03T03:12:06.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 -- 172.21.2.111:0/1476227683 learned_addr learned my addr 172.21.2.111:0/1476227683 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:06.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603ed9d700 1 --2- 172.21.2.111:0/1476227683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 0x7f6040144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 -- 172.21.2.111:0/1476227683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 msgr2=0x7f6040144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:06.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 --2- 172.21.2.111:0/1476227683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 0x7f6040144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 -- 172.21.2.111:0/1476227683 >> v1:172.21.2.115:6789/0 conn(0x7f60400add50 legacy=0x7f6040147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:06.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 -- 172.21.2.111:0/1476227683 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604014b330 con 0x7f60400603f0 2021-04-03T03:12:06.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603e59c700 1 --2- 172.21.2.111:0/1476227683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400603f0 0x7f60401447e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f60400b5600 tx=0x7f6028006e90).ready entity=mon.0 client_cookie=1c86bb1830affd7f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603d59a700 1 -- 172.21.2.111:0/1476227683 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60280098e0 con 0x7f60400603f0 2021-04-03T03:12:06.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603d59a700 1 -- 172.21.2.111:0/1476227683 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6028009a40 con 0x7f60400603f0 2021-04-03T03:12:06.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603d59a700 1 -- 172.21.2.111:0/1476227683 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6028009de0 con 0x7f60400603f0 2021-04-03T03:12:06.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 -- 172.21.2.111:0/1476227683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400603f0 msgr2=0x7f60401447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.839+0000 7f603f59e700 1 --2- 172.21.2.111:0/1476227683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400603f0 0x7f60401447e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f60400b5600 tx=0x7f6028006e90).stop 2021-04-03T03:12:06.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- 172.21.2.111:0/1476227683 shutdown_connections 2021-04-03T03:12:06.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 --2- 172.21.2.111:0/1476227683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 0x7f6040144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 --2- 172.21.2.111:0/1476227683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400603f0 0x7f60401447e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- 172.21.2.111:0/1476227683 >> 172.21.2.111:0/1476227683 conn(0x7f60400b61c0 msgr2=0x7f604005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- 172.21.2.111:0/1476227683 shutdown_connections 2021-04-03T03:12:06.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- 172.21.2.111:0/1476227683 wait complete. 2021-04-03T03:12:06.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 Processor -- start 2021-04-03T03:12:06.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- start start 2021-04-03T03:12:06.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60400603f0 0x7f60401443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400add50 0x7f60400a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 0x7f60400aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6040147ee0 con 0x7f60400add50 2021-04-03T03:12:06.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400aa160 con 0x7f60400603f0 2021-04-03T03:12:06.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6040144bb0 con 0x7f60400b17b0 2021-04-03T03:12:06.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400add50 0x7f60400a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400add50 0x7f60400a73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52400/0 (socket says 172.21.2.111:52400) 2021-04-03T03:12:06.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 -- 172.21.2.111:0/1529786912 learned_addr learned my addr 172.21.2.111:0/1529786912 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:06.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 -- 172.21.2.111:0/1529786912 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 msgr2=0x7f60400aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 0x7f60400aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:06.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 -- 172.21.2.111:0/1529786912 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60400603f0 msgr2=0x7f60401443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60400603f0 0x7f60401443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:06.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 -- 172.21.2.111:0/1529786912 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400a78e0 con 0x7f60400add50 2021-04-03T03:12:06.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603dd9b700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400add50 0x7f60400a73f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f6030002f50 tx=0x7f6030006db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f60377fe700 1 -- 172.21.2.111:0/1529786912 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f603000d4a0 con 0x7f60400add50 2021-04-03T03:12:06.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f60377fe700 1 -- 172.21.2.111:0/1529786912 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603000d600 con 0x7f60400add50 2021-04-03T03:12:06.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f60377fe700 1 -- 172.21.2.111:0/1529786912 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f603000d960 con 0x7f60400add50 2021-04-03T03:12:06.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60400a6f20 con 0x7f60400add50 2021-04-03T03:12:06.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6040147b60 con 0x7f60400add50 2021-04-03T03:12:06.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60400aae10 con 0x7f60400add50 2021-04-03T03:12:06.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f60377fe700 1 -- 172.21.2.111:0/1529786912 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f603000f070 con 0x7f60400add50 2021-04-03T03:12:06.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f60377fe700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6024057050 0x7f6024059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:06.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f60377fe700 1 -- 172.21.2.111:0/1529786912 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f603006f950 con 0x7f60400add50 2021-04-03T03:12:06.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603e59c700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6024057050 0x7f6024059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:06.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.843+0000 7f603e59c700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6024057050 0x7f6024059510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f60400b5600 tx=0x7f6028015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:06.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.847+0000 7f60377fe700 1 -- 172.21.2.111:0/1529786912 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6030044d40 con 0x7f60400add50 2021-04-03T03:12:06.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6040057660 con 0x7f60400add50 2021-04-03T03:12:06.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f60377fe700 1 -- 172.21.2.111:0/1529786912 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v54) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6030048360 con 0x7f60400add50 2021-04-03T03:12:06.953 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:12:06.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6024057050 msgr2=0x7f6024059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6024057050 0x7f6024059510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f60400b5600 tx=0x7f6028015000).stop 2021-04-03T03:12:06.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400add50 msgr2=0x7f60400a73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:06.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400add50 0x7f60400a73f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f6030002f50 tx=0x7f6030006db0).stop 2021-04-03T03:12:06.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 shutdown_connections 2021-04-03T03:12:06.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60400b17b0 0x7f60400aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:06.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6024057050 0x7f6024059510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60400603f0 0x7f60401443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:06.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 --2- 172.21.2.111:0/1529786912 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60400add50 0x7f60400a73f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:06.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 >> 172.21.2.111:0/1529786912 conn(0x7f60400b61c0 msgr2=0x7f604014a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:06.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 shutdown_connections 2021-04-03T03:12:06.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:06.939+0000 7f603f59e700 1 -- 172.21.2.111:0/1529786912 wait complete. 2021-04-03T03:12:06.961 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:12:07.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.003+0000 7fddd617f700 1 Processor -- start 2021-04-03T03:12:07.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.003+0000 7fddd617f700 1 -- start start 2021-04-03T03:12:07.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fddd00add50 0x7fddd0147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fddd0144ef0 0x7fddd0147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd00607d0 con 0x7fddd00603f0 2021-04-03T03:12:07.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd014b090 con 0x7fddd00add50 2021-04-03T03:12:07.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd014b330 con 0x7fddd0144ef0 2021-04-03T03:12:07.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fddd00add50 0x7fddd0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd517d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fddd00603f0 0x7fddd01447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53196/0 (socket says 172.21.2.111:53196) 2021-04-03T03:12:07.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd517d700 1 -- 172.21.2.111:0/4115385031 learned_addr learned my addr 172.21.2.111:0/4115385031 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:07.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd597e700 1 --2- 172.21.2.111:0/4115385031 >> v2:172.21.2.115:3300/0 conn(0x7fddd0144ef0 0x7fddd0147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 -- 172.21.2.111:0/4115385031 >> v2:172.21.2.115:3300/0 conn(0x7fddd0144ef0 msgr2=0x7fddd0147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:07.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- 172.21.2.111:0/4115385031 >> v2:172.21.2.115:3300/0 conn(0x7fddd0144ef0 0x7fddd0147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 -- 172.21.2.111:0/4115385031 >> v1:172.21.2.111:6789/0 conn(0x7fddd00603f0 legacy=0x7fddd01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 -- 172.21.2.111:0/4115385031 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd014b5d0 con 0x7fddd00add50 2021-04-03T03:12:07.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- 172.21.2.111:0/4115385031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd0147740 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc006dd0).ready entity=mon.0 client_cookie=98847ab7e13d2c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddc3fff700 1 -- 172.21.2.111:0/4115385031 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc0097f0 con 0x7fddd00add50 2021-04-03T03:12:07.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddc3fff700 1 -- 172.21.2.111:0/4115385031 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc009950 con 0x7fddd00add50 2021-04-03T03:12:07.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddc3fff700 1 -- 172.21.2.111:0/4115385031 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc009cb0 con 0x7fddd00add50 2021-04-03T03:12:07.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- 172.21.2.111:0/4115385031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 msgr2=0x7fddd0147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- 172.21.2.111:0/4115385031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd0147740 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc006dd0).stop 2021-04-03T03:12:07.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- 172.21.2.111:0/4115385031 shutdown_connections 2021-04-03T03:12:07.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- 172.21.2.111:0/4115385031 >> v2:172.21.2.115:3300/0 conn(0x7fddd0144ef0 0x7fddd0147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- 172.21.2.111:0/4115385031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd0147740 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- 172.21.2.111:0/4115385031 >> 172.21.2.111:0/4115385031 conn(0x7fddd00b61c0 msgr2=0x7fddd005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- 172.21.2.111:0/4115385031 shutdown_connections 2021-04-03T03:12:07.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- 172.21.2.111:0/4115385031 wait complete. 2021-04-03T03:12:07.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 Processor -- start 2021-04-03T03:12:07.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- start start 2021-04-03T03:12:07.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fddd00603f0 0x7fddd00aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd00a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fddd0144ef0 0x7fddd00a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0063110 con 0x7fddd00add50 2021-04-03T03:12:07.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00a9bf0 con 0x7fddd00603f0 2021-04-03T03:12:07.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00a9e90 con 0x7fddd0144ef0 2021-04-03T03:12:07.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd00a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd00a7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52410/0 (socket says 172.21.2.111:52410) 2021-04-03T03:12:07.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 -- 172.21.2.111:0/706497628 learned_addr learned my addr 172.21.2.111:0/706497628 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:07.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd517d700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fddd00603f0 0x7fddd00aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd597e700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fddd0144ef0 0x7fddd00a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 -- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fddd0144ef0 msgr2=0x7fddd00a83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fddd0144ef0 0x7fddd00a83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 -- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fddd00603f0 msgr2=0x7fddd00aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fddd00603f0 0x7fddd00aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 -- 172.21.2.111:0/706497628 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd00aa130 con 0x7fddd00add50 2021-04-03T03:12:07.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd497c700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd00a7eb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddc27fc700 1 -- 172.21.2.111:0/706497628 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc011ca0 con 0x7fddd00add50 2021-04-03T03:12:07.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddc27fc700 1 -- 172.21.2.111:0/706497628 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc011e00 con 0x7fddd00add50 2021-04-03T03:12:07.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.007+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd00abf50 con 0x7fddd00add50 2021-04-03T03:12:07.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddc27fc700 1 -- 172.21.2.111:0/706497628 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc01e650 con 0x7fddd00add50 2021-04-03T03:12:07.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd015a4b0 con 0x7fddd00add50 2021-04-03T03:12:07.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd00aa2c0 con 0x7fddd00add50 2021-04-03T03:12:07.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddc27fc700 1 -- 172.21.2.111:0/706497628 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fddcc025030 con 0x7fddd00add50 2021-04-03T03:12:07.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddc27fc700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fddb0057050 0x7fddb0059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddc27fc700 1 -- 172.21.2.111:0/706497628 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fddcc071500 con 0x7fddd00add50 2021-04-03T03:12:07.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddd517d700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fddb0057050 0x7fddb0059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddd517d700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fddb0057050 0x7fddb0059510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fddc400b000 tx=0x7fddc4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.011+0000 7fddc27fc700 1 -- 172.21.2.111:0/706497628 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fddcc0468f0 con 0x7fddd00add50 2021-04-03T03:12:07.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fddd0057660 con 0x7fddd00add50 2021-04-03T03:12:07.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddc27fc700 1 -- 172.21.2.111:0/706497628 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7fddcc0230a0 con 0x7fddd00add50 2021-04-03T03:12:07.121 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:12:07.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fddb0057050 msgr2=0x7fddb0059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fddb0057050 0x7fddb0059510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fddc400b000 tx=0x7fddc4009040).stop 2021-04-03T03:12:07.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 msgr2=0x7fddd00a7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd00a7eb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fddcc002f30 tx=0x7fddcc015000).stop 2021-04-03T03:12:07.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 shutdown_connections 2021-04-03T03:12:07.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fddd0144ef0 0x7fddd00a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fddb0057050 0x7fddb0059510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fddd00603f0 0x7fddd00aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 --2- 172.21.2.111:0/706497628 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fddd00add50 0x7fddd00a7eb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 >> 172.21.2.111:0/706497628 conn(0x7fddd00b61c0 msgr2=0x7fddd0149270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 shutdown_connections 2021-04-03T03:12:07.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.107+0000 7fddd617f700 1 -- 172.21.2.111:0/706497628 wait complete. 2021-04-03T03:12:07.128 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:12:07.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 Processor -- start 2021-04-03T03:12:07.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- start start 2021-04-03T03:12:07.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f17a00b3ae0 0x7f17a0063c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b2880 0x7f17a01435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f17a0066250 0x7f17a00666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a00b3ec0 con 0x7f17a0066250 2021-04-03T03:12:07.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a00b2d00 con 0x7f17a00b3ae0 2021-04-03T03:12:07.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a0064250 con 0x7f17a00b2880 2021-04-03T03:12:07.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f17a0066250 0x7f17a00666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f17a0066250 0x7f17a00666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52422/0 (socket says 172.21.2.111:52422) 2021-04-03T03:12:07.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 -- 172.21.2.111:0/3396680627 learned_addr learned my addr 172.21.2.111:0/3396680627 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:07.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 -- 172.21.2.111:0/3396680627 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b2880 msgr2=0x7f17a01435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:07.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 --2- 172.21.2.111:0/3396680627 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b2880 0x7f17a01435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 -- 172.21.2.111:0/3396680627 >> v2:172.21.2.115:3300/0 conn(0x7f17a00b3ae0 msgr2=0x7f17a0063c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:07.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 --2- 172.21.2.111:0/3396680627 >> v2:172.21.2.115:3300/0 conn(0x7f17a00b3ae0 0x7f17a0063c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 -- 172.21.2.111:0/3396680627 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0065db0 con 0x7f17a0066250 2021-04-03T03:12:07.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179ffff700 1 --2- 172.21.2.111:0/3396680627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a00666b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1790002f30 tx=0x7f1790006e90).ready entity=mon.0 client_cookie=5e3d9de6459dff65 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179e7fc700 1 -- 172.21.2.111:0/3396680627 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1790009920 con 0x7f17a0066250 2021-04-03T03:12:07.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179e7fc700 1 -- 172.21.2.111:0/3396680627 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1790009a80 con 0x7f17a0066250 2021-04-03T03:12:07.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179e7fc700 1 -- 172.21.2.111:0/3396680627 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1790009de0 con 0x7f17a0066250 2021-04-03T03:12:07.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- 172.21.2.111:0/3396680627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 msgr2=0x7f17a00666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3396680627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a00666b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1790002f30 tx=0x7f1790006e90).stop 2021-04-03T03:12:07.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- 172.21.2.111:0/3396680627 shutdown_connections 2021-04-03T03:12:07.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3396680627 >> v2:172.21.2.115:3300/0 conn(0x7f17a00b3ae0 0x7f17a0063c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3396680627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a00666b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3396680627 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b2880 0x7f17a01435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- 172.21.2.111:0/3396680627 >> 172.21.2.111:0/3396680627 conn(0x7f17a00622a0 msgr2=0x7f17a0060ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- 172.21.2.111:0/3396680627 shutdown_connections 2021-04-03T03:12:07.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- 172.21.2.111:0/3396680627 wait complete. 2021-04-03T03:12:07.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 Processor -- start 2021-04-03T03:12:07.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- start start 2021-04-03T03:12:07.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a013ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f17a00b2880 0x7f17a013d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b3ae0 0x7f17a01527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a00b6e00 con 0x7f17a0066250 2021-04-03T03:12:07.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a0142250 con 0x7f17a00b2880 2021-04-03T03:12:07.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f17a4f17700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a01424f0 con 0x7f17a00b3ae0 2021-04-03T03:12:07.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a013ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a013ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52426/0 (socket says 172.21.2.111:52426) 2021-04-03T03:12:07.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.175+0000 7f179f7fe700 1 -- 172.21.2.111:0/3049261634 learned_addr learned my addr 172.21.2.111:0/3049261634 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:07.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179f7fe700 1 -- 172.21.2.111:0/3049261634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b3ae0 msgr2=0x7f17a01527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179f7fe700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b3ae0 0x7f17a01527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179f7fe700 1 -- 172.21.2.111:0/3049261634 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f17a00b2880 msgr2=0x7f17a013d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179f7fe700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f17a00b2880 0x7f17a013d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179f7fe700 1 -- 172.21.2.111:0/3049261634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a01520a0 con 0x7f17a0066250 2021-04-03T03:12:07.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179f7fe700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a013ebf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f17980032b0 tx=0x7f17980091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179cff9700 1 -- 172.21.2.111:0/3049261634 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f179800c9e0 con 0x7f17a0066250 2021-04-03T03:12:07.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179cff9700 1 -- 172.21.2.111:0/3049261634 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179800cb40 con 0x7f17a0066250 2021-04-03T03:12:07.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a0152340 con 0x7f17a0066250 2021-04-03T03:12:07.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a013de10 con 0x7f17a0066250 2021-04-03T03:12:07.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179cff9700 1 -- 172.21.2.111:0/3049261634 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f179800cea0 con 0x7f17a0066250 2021-04-03T03:12:07.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177c0033f0 con 0x7f17a0066250 2021-04-03T03:12:07.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179cff9700 1 -- 172.21.2.111:0/3049261634 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1798015470 con 0x7f17a0066250 2021-04-03T03:12:07.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179cff9700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1780057050 0x7f1780059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179cff9700 1 -- 172.21.2.111:0/3049261634 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1798070220 con 0x7f17a0066250 2021-04-03T03:12:07.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179effd700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1780057050 0x7f1780059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.179+0000 7f179effd700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1780057050 0x7f1780059510 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f1794002f30 tx=0x7f1794006a70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.183+0000 7f179cff9700 1 -- 172.21.2.111:0/3049261634 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f17980454a0 con 0x7f17a0066250 2021-04-03T03:12:07.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f177c003c20 con 0x7f17a0066250 2021-04-03T03:12:07.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f179cff9700 1 -- 172.21.2.111:0/3049261634 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1798048ac0 con 0x7f17a0066250 2021-04-03T03:12:07.288 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:12:07.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1780057050 msgr2=0x7f1780059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1780057050 0x7f1780059510 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f1794002f30 tx=0x7f1794006a70).stop 2021-04-03T03:12:07.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 msgr2=0x7f17a013ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a013ebf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f17980032b0 tx=0x7f17980091a0).stop 2021-04-03T03:12:07.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 shutdown_connections 2021-04-03T03:12:07.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f17a00b3ae0 0x7f17a01527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f17a0066250 0x7f17a013ebf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1780057050 0x7f1780059510 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 --2- 172.21.2.111:0/3049261634 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f17a00b2880 0x7f17a013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 >> 172.21.2.111:0/3049261634 conn(0x7f17a00622a0 msgr2=0x7f17a00609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 shutdown_connections 2021-04-03T03:12:07.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.275+0000 7f17a4f17700 1 -- 172.21.2.111:0/3049261634 wait complete. 2021-04-03T03:12:07.296 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:12:07.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea9564b700 1 Processor -- start 2021-04-03T03:12:07.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea9564b700 1 -- start start 2021-04-03T03:12:07.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea9564b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fea900603f0 0x7fea901447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea9564b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea9564b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900607d0 con 0x7fea900add50 2021-04-03T03:12:07.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea9564b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea901451c0 con 0x7fea900603f0 2021-04-03T03:12:07.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea9564b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9014b090 con 0x7fea900b17b0 2021-04-03T03:12:07.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea94e4a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea8f7fe700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fea900add50 0x7fea90147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:37532/0 (socket says 172.21.2.111:37532) 2021-04-03T03:12:07.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.339+0000 7fea8f7fe700 1 -- 172.21.2.111:0/671174009 learned_addr learned my addr 172.21.2.111:0/671174009 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:07.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8ffff700 1 --2- 172.21.2.111:0/671174009 >> v2:172.21.2.115:3300/0 conn(0x7fea900603f0 0x7fea901447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea94e4a700 1 -- 172.21.2.111:0/671174009 >> v2:172.21.2.115:3300/0 conn(0x7fea900603f0 msgr2=0x7fea901447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:07.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea94e4a700 1 --2- 172.21.2.111:0/671174009 >> v2:172.21.2.115:3300/0 conn(0x7fea900603f0 0x7fea901447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea94e4a700 1 -- 172.21.2.111:0/671174009 >> v1:172.21.2.115:6789/0 conn(0x7fea900add50 legacy=0x7fea90147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea94e4a700 1 -- 172.21.2.111:0/671174009 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9014b330 con 0x7fea900b17b0 2021-04-03T03:12:07.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea94e4a700 1 --2- 172.21.2.111:0/671174009 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90144cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fea880032b0 tx=0x7fea880091d0).ready entity=mon.2 client_cookie=136ac886417b1ff3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8effd700 1 -- 172.21.2.111:0/671174009 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea8800b9a0 con 0x7fea900b17b0 2021-04-03T03:12:07.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8effd700 1 -- 172.21.2.111:0/671174009 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800bb00 con 0x7fea900b17b0 2021-04-03T03:12:07.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- 172.21.2.111:0/671174009 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 msgr2=0x7fea90144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8effd700 1 -- 172.21.2.111:0/671174009 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea8800bc70 con 0x7fea900b17b0 2021-04-03T03:12:07.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 --2- 172.21.2.111:0/671174009 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90144cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fea880032b0 tx=0x7fea880091d0).stop 2021-04-03T03:12:07.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- 172.21.2.111:0/671174009 shutdown_connections 2021-04-03T03:12:07.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 --2- 172.21.2.111:0/671174009 >> v2:172.21.2.115:3300/0 conn(0x7fea900603f0 0x7fea901447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 --2- 172.21.2.111:0/671174009 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90144cd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- 172.21.2.111:0/671174009 >> 172.21.2.111:0/671174009 conn(0x7fea900b61c0 msgr2=0x7fea9005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- 172.21.2.111:0/671174009 shutdown_connections 2021-04-03T03:12:07.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- 172.21.2.111:0/671174009 wait complete. 2021-04-03T03:12:07.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 Processor -- start 2021-04-03T03:12:07.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- start start 2021-04-03T03:12:07.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea900603f0 0x7fea900abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea900add50 0x7fea90062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900aa4a0 con 0x7fea900add50 2021-04-03T03:12:07.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900aa690 con 0x7fea900603f0 2021-04-03T03:12:07.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea9564b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900aa930 con 0x7fea900b17b0 2021-04-03T03:12:07.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea900add50 0x7fea90062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea900add50 0x7fea90062d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52440/0 (socket says 172.21.2.111:52440) 2021-04-03T03:12:07.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 -- 172.21.2.111:0/3009004651 learned_addr learned my addr 172.21.2.111:0/3009004651 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:07.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 -- 172.21.2.111:0/3009004651 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 msgr2=0x7fea90143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 -- 172.21.2.111:0/3009004651 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea900603f0 msgr2=0x7fea900abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea900603f0 0x7fea900abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.343+0000 7fea8f7fe700 1 -- 172.21.2.111:0/3009004651 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900aabd0 con 0x7fea900add50 2021-04-03T03:12:07.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8f7fe700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea900add50 0x7fea90062d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fea900b5600 tx=0x7fea84007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8d7fa700 1 -- 172.21.2.111:0/3009004651 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea840072e0 con 0x7fea900add50 2021-04-03T03:12:07.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8d7fa700 1 -- 172.21.2.111:0/3009004651 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea84007440 con 0x7fea900add50 2021-04-03T03:12:07.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8d7fa700 1 -- 172.21.2.111:0/3009004651 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea840057e0 con 0x7fea900add50 2021-04-03T03:12:07.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900a7eb0 con 0x7fea900add50 2021-04-03T03:12:07.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea90067f00 con 0x7fea900add50 2021-04-03T03:12:07.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea90057660 con 0x7fea900add50 2021-04-03T03:12:07.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8d7fa700 1 -- 172.21.2.111:0/3009004651 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fea84010070 con 0x7fea900add50 2021-04-03T03:12:07.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8d7fa700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea70056f20 0x7fea700593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8d7fa700 1 -- 172.21.2.111:0/3009004651 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fea8406f8f0 con 0x7fea900add50 2021-04-03T03:12:07.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8ffff700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea70056f20 0x7fea700593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8ffff700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea70056f20 0x7fea700593e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fea8000c000 tx=0x7fea80006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.347+0000 7fea8d7fa700 1 -- 172.21.2.111:0/3009004651 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fea84044b50 con 0x7fea900add50 2021-04-03T03:12:07.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea900ac5f0 con 0x7fea900add50 2021-04-03T03:12:07.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea8d7fa700 1 -- 172.21.2.111:0/3009004651 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+10 (secure 0 0 0) 0x7fea84018070 con 0x7fea900add50 2021-04-03T03:12:07.458 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:12:07.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea70056f20 msgr2=0x7fea700593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea70056f20 0x7fea700593e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fea8000c000 tx=0x7fea80006dd0).stop 2021-04-03T03:12:07.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea900add50 msgr2=0x7fea90062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea900add50 0x7fea90062d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fea900b5600 tx=0x7fea84007bc0).stop 2021-04-03T03:12:07.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 shutdown_connections 2021-04-03T03:12:07.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fea900b17b0 0x7fea90143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fea70056f20 0x7fea700593e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fea900603f0 0x7fea900abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 --2- 172.21.2.111:0/3009004651 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fea900add50 0x7fea90062d90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 >> 172.21.2.111:0/3009004651 conn(0x7fea900b61c0 msgr2=0x7fea900b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 shutdown_connections 2021-04-03T03:12:07.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.443+0000 7fea9564b700 1 -- 172.21.2.111:0/3009004651 wait complete. 2021-04-03T03:12:07.466 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:12:07.466 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:12:07.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.507+0000 7fdd2eefd700 1 Processor -- start 2021-04-03T03:12:07.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.507+0000 7fdd2eefd700 1 -- start start 2021-04-03T03:12:07.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.507+0000 7fdd2eefd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280603f0 0x7fdd280b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.507+0000 7fdd2eefd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdd280b0c90 0x7fdd28147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.507+0000 7fdd2eefd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd280607d0 con 0x7fdd2814b700 2021-04-03T03:12:07.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.507+0000 7fdd2eefd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd280af400 con 0x7fdd280b0c90 2021-04-03T03:12:07.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.507+0000 7fdd2eefd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd280af6a0 con 0x7fdd280603f0 2021-04-03T03:12:07.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2defb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280603f0 0x7fdd280b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2e6fc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdd280b0c90 0x7fdd28147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2d6fa700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fdd2814b700 0x7fdd280b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:37544/0 (socket says 172.21.2.111:37544) 2021-04-03T03:12:07.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2d6fa700 1 -- 172.21.2.111:0/128983536 learned_addr learned my addr 172.21.2.111:0/128983536 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:07.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2defb700 1 -- 172.21.2.111:0/128983536 >> v2:172.21.2.115:3300/0 conn(0x7fdd280b0c90 msgr2=0x7fdd28147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:07.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2defb700 1 --2- 172.21.2.111:0/128983536 >> v2:172.21.2.115:3300/0 conn(0x7fdd280b0c90 0x7fdd28147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2defb700 1 -- 172.21.2.111:0/128983536 >> v1:172.21.2.115:6789/0 conn(0x7fdd2814b700 legacy=0x7fdd280b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2defb700 1 -- 172.21.2.111:0/128983536 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280afb40 con 0x7fdd280603f0 2021-04-03T03:12:07.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2defb700 1 --2- 172.21.2.111:0/128983536 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280603f0 0x7fdd280b0090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fdd1c0010d0 tx=0x7fdd24006dd0).ready entity=mon.2 client_cookie=17a7a5cf204f34c0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2cef9700 1 -- 172.21.2.111:0/128983536 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd240095d0 con 0x7fdd280603f0 2021-04-03T03:12:07.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2cef9700 1 -- 172.21.2.111:0/128983536 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd24009730 con 0x7fdd280603f0 2021-04-03T03:12:07.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2cef9700 1 -- 172.21.2.111:0/128983536 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd24009a90 con 0x7fdd280603f0 2021-04-03T03:12:07.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 -- 172.21.2.111:0/128983536 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280603f0 msgr2=0x7fdd280b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/128983536 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280603f0 0x7fdd280b0090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fdd1c0010d0 tx=0x7fdd24006dd0).stop 2021-04-03T03:12:07.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 -- 172.21.2.111:0/128983536 shutdown_connections 2021-04-03T03:12:07.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/128983536 >> v2:172.21.2.115:3300/0 conn(0x7fdd280b0c90 0x7fdd28147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/128983536 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280603f0 0x7fdd280b0090 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 -- 172.21.2.111:0/128983536 >> 172.21.2.111:0/128983536 conn(0x7fdd280b61c0 msgr2=0x7fdd2805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 -- 172.21.2.111:0/128983536 shutdown_connections 2021-04-03T03:12:07.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 -- 172.21.2.111:0/128983536 wait complete. 2021-04-03T03:12:07.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 Processor -- start 2021-04-03T03:12:07.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.511+0000 7fdd2eefd700 1 -- start start 2021-04-03T03:12:07.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd280603f0 0x7fdd280a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280b0c90 0x7fdd28067450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd2814b700 0x7fdd28067940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28062b00 con 0x7fdd2814b700 2021-04-03T03:12:07.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28062da0 con 0x7fdd280603f0 2021-04-03T03:12:07.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28063040 con 0x7fdd280b0c90 2021-04-03T03:12:07.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd2814b700 0x7fdd28067940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd2814b700 0x7fdd28067940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52456/0 (socket says 172.21.2.111:52456) 2021-04-03T03:12:07.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 -- 172.21.2.111:0/1548510625 learned_addr learned my addr 172.21.2.111:0/1548510625 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:07.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2d6fa700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280b0c90 0x7fdd28067450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 -- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280b0c90 msgr2=0x7fdd28067450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280b0c90 0x7fdd28067450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 -- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd280603f0 msgr2=0x7fdd280a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd280603f0 0x7fdd280a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 -- 172.21.2.111:0/1548510625 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280a7b20 con 0x7fdd2814b700 2021-04-03T03:12:07.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2d6fa700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280b0c90 0x7fdd28067450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:12:07.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2e6fc700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd2814b700 0x7fdd28067940 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fdd20006de0 tx=0x7fdd200091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd1affd700 1 -- 172.21.2.111:0/1548510625 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd2000cbb0 con 0x7fdd2814b700 2021-04-03T03:12:07.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd1affd700 1 -- 172.21.2.111:0/1548510625 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2000cd10 con 0x7fdd2814b700 2021-04-03T03:12:07.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd28067e30 con 0x7fdd2814b700 2021-04-03T03:12:07.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd1affd700 1 -- 172.21.2.111:0/1548510625 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd200125e0 con 0x7fdd2814b700 2021-04-03T03:12:07.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd280a9f90 con 0x7fdd2814b700 2021-04-03T03:12:07.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd28062910 con 0x7fdd2814b700 2021-04-03T03:12:07.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd1affd700 1 -- 172.21.2.111:0/1548510625 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdd20022070 con 0x7fdd2814b700 2021-04-03T03:12:07.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd1affd700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd08056f20 0x7fdd080593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:07.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd1affd700 1 -- 172.21.2.111:0/1548510625 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fdd200709b0 con 0x7fdd2814b700 2021-04-03T03:12:07.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2defb700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd08056f20 0x7fdd080593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:07.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.515+0000 7fdd2defb700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd08056f20 0x7fdd080593e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fdd1c0010d0 tx=0x7fdd24015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:07.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.519+0000 7fdd1affd700 1 -- 172.21.2.111:0/1548510625 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdd20045da0 con 0x7fdd2814b700 2021-04-03T03:12:07.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.639+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd28068020 con 0x7fdd2814b700 2021-04-03T03:12:07.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.639+0000 7fdd1affd700 1 -- 172.21.2.111:0/1548510625 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fdd20016070 con 0x7fdd2814b700 2021-04-03T03:12:07.655 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:07.655 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:12:07.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd08056f20 msgr2=0x7fdd080593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd08056f20 0x7fdd080593e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fdd1c0010d0 tx=0x7fdd24015000).stop 2021-04-03T03:12:07.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd2814b700 msgr2=0x7fdd28067940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:07.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd2814b700 0x7fdd28067940 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fdd20006de0 tx=0x7fdd200091d0).stop 2021-04-03T03:12:07.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 shutdown_connections 2021-04-03T03:12:07.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd2814b700 0x7fdd28067940 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd08056f20 0x7fdd080593e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd280603f0 0x7fdd280a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:07.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 --2- 172.21.2.111:0/1548510625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd280b0c90 0x7fdd28067450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:07.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 >> 172.21.2.111:0/1548510625 conn(0x7fdd280b61c0 msgr2=0x7fdd28147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:07.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 shutdown_connections 2021-04-03T03:12:07.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:07.643+0000 7fdd2eefd700 1 -- 172.21.2.111:0/1548510625 wait complete. 2021-04-03T03:12:07.659 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:12:07.865 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:12:04.020364+0000', 'modified': '2021-04-03T03:12:05.022670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5580': {'gid': 5580, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/2278897120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 2278897120}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 2278897120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:12:07.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:12:07.866 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:12:07.938 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5580,"want_state":"up:active","state":"up:active","rank_uptime":3.902171686,"mdsmap_epoch":21,"osdmap_epoch":54,"osdmap_epoch_barrier":52,"uptime":8.0386760329999998} 2021-04-03T03:12:07.938 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5580, "mdsmap_epoch": 21, "osdmap_epoch": 54, "osdmap_epoch_barrier": 52, "rank_uptime": 3.902171686, "state": "up:active", "uptime": 8.038676033, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:12:07.939 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:12:07.939 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:12:07.940 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:12:07.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:07.941 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:07.951 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:12:08.005 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:12:08.005 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:12:08.052 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:12:08.053 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:12:08.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:08.055 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:12:08.102 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:12:08.103 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:08.148 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:12:08.155 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:12:08.156 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:12:08.187 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:12:08.179+0000 7f83e1c6d0c0 -1 init, newargv = 0x55f0cb928990 newargc=9 2021-04-03T03:12:08.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15203]: starting ceph client 2021-04-03T03:12:08.192 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:12:08.192 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:12:08.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:08.194 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:12:08.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15203]: starting fuse 2021-04-03T03:12:08.207 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:12:08.207 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:12:08.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:08.208 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:12:08.209 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:12:08.209 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:12:08.210 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:12:08.210 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:12:08.211 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:08.211 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:12:08.211 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:12:08.212 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:12:08.212 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:08.213 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:12:08.213 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:12:08.213 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:12:08.214 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:12:08.214 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:08.215 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:12:08.215 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:12:08.215 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:12:08.216 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:12:08.216 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:12:08.217 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:08.217 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:12:08.218 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:12:08.278 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15203.asok 2021-04-03T03:12:08.281 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15203.asok 2021-04-03T03:12:08.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:08.282 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15203.asok status 2021-04-03T03:12:08.379 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:12:08.380 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:12:08.380 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:12:08.380 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:12:08.380 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:12:08.381 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:12:08.381 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:12:08.381 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15203", 2021-04-03T03:12:08.381 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:12:08.382 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:08.382 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:12:08.382 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:12:08.382 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4559, 2021-04-03T03:12:08.382 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:12:08.383 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:12:08.383 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:12:08.383 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4559 2021-04-03T03:12:08.383 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:08.384 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:12:08.384 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:12:08.384 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:12:08.384 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 1957177840 2021-04-03T03:12:08.384 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:12:08.385 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:08.385 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:12:08.385 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:12:08.385 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:12:08.385 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 1957177840 2021-04-03T03:12:08.386 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:08.386 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4559 172.21.2.126:0/1957177840", 2021-04-03T03:12:08.386 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/1957177840", 2021-04-03T03:12:08.386 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:12:08.387 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 21, 2021-04-03T03:12:08.387 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 54, 2021-04-03T03:12:08.387 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:12:08.387 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:12:08.387 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:12:08.388 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:12:08.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:08.389 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:12:08.390 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:08.394 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:12:08.395 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:12:08.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:08.396 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:08.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:08.449 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:12:08.449 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:08.498 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:12:08.499 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:12:08.499 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:12:08.500 DEBUG:teuthology.orchestra.run.gibba026:> import sys 2021-04-03T03:12:08.500 DEBUG:teuthology.orchestra.run.gibba026:> import time 2021-04-03T03:12:08.501 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:12:08.501 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:08.501 DEBUG:teuthology.orchestra.run.gibba026:> n = 1000 2021-04-03T03:12:08.502 DEBUG:teuthology.orchestra.run.gibba026:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-04-03T03:12:08.502 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:08.502 DEBUG:teuthology.orchestra.run.gibba026:> if not os.path.exists(os.path.dirname(abs_path)): 2021-04-03T03:12:08.503 DEBUG:teuthology.orchestra.run.gibba026:> os.makedirs(os.path.dirname(abs_path)) 2021-04-03T03:12:08.503 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:08.504 DEBUG:teuthology.orchestra.run.gibba026:> for i in range(0, n): 2021-04-03T03:12:08.504 DEBUG:teuthology.orchestra.run.gibba026:> fname = "{0}_{1}".format(abs_path, i) 2021-04-03T03:12:08.504 DEBUG:teuthology.orchestra.run.gibba026:> with open(fname, '"'"'w'"'"') as f: 2021-04-03T03:12:08.505 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content'"'"') 2021-04-03T03:12:08.505 DEBUG:teuthology.orchestra.run.gibba026:> if True: 2021-04-03T03:12:08.505 DEBUG:teuthology.orchestra.run.gibba026:> f.flush() 2021-04-03T03:12:08.506 DEBUG:teuthology.orchestra.run.gibba026:> os.fsync(f.fileno()) 2021-04-03T03:12:08.506 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:12:24.807 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:12:24.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.855+0000 7f123c804700 1 Processor -- start 2021-04-03T03:12:24.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.855+0000 7f123c804700 1 -- start start 2021-04-03T03:12:24.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.855+0000 7f123c804700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12380b5d60 0x7f12380b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:24.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.855+0000 7f123c804700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f12380b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:24.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.855+0000 7f123c804700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380b4bc0 con 0x7f12380603f0 2021-04-03T03:12:24.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.855+0000 7f123c804700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12380607d0 con 0x7f12380b5d60 2021-04-03T03:12:24.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.855+0000 7f123c804700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b4df0 con 0x7f12380b44a0 2021-04-03T03:12:24.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12380b5d60 0x7f12380b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:24.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12380b5d60 0x7f12380b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52464/0 (socket says 172.21.2.111:52464) 2021-04-03T03:12:24.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 -- 172.21.2.111:0/3033338041 learned_addr learned my addr 172.21.2.111:0/3033338041 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:24.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 --2- 172.21.2.111:0/3033338041 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f12380b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:24.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 -- 172.21.2.111:0/3033338041 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 msgr2=0x7f12380b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:24.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 --2- 172.21.2.111:0/3033338041 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f12380b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:24.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 -- 172.21.2.111:0/3033338041 >> v1:172.21.2.115:6789/0 conn(0x7f12380603f0 legacy=0x7f12380b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:24.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 -- 172.21.2.111:0/3033338041 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380b5090 con 0x7f12380b5d60 2021-04-03T03:12:24.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 --2- 172.21.2.111:0/3033338041 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380b5d60 0x7f12380b2b90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f122c002f30 tx=0x7f122c006e90).ready entity=mon.0 client_cookie=d2d9acd656612246 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:24.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f12367fc700 1 -- 172.21.2.111:0/3033338041 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f122c009850 con 0x7f12380b5d60 2021-04-03T03:12:24.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f12367fc700 1 -- 172.21.2.111:0/3033338041 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122c0099b0 con 0x7f12380b5d60 2021-04-03T03:12:24.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f12367fc700 1 -- 172.21.2.111:0/3033338041 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f122c009d10 con 0x7f12380b5d60 2021-04-03T03:12:24.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- 172.21.2.111:0/3033338041 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380b5d60 msgr2=0x7f12380b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:24.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 --2- 172.21.2.111:0/3033338041 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380b5d60 0x7f12380b2b90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f122c002f30 tx=0x7f122c006e90).stop 2021-04-03T03:12:24.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- 172.21.2.111:0/3033338041 shutdown_connections 2021-04-03T03:12:24.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 --2- 172.21.2.111:0/3033338041 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f12380b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:24.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 --2- 172.21.2.111:0/3033338041 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380b5d60 0x7f12380b2b90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:24.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- 172.21.2.111:0/3033338041 >> 172.21.2.111:0/3033338041 conn(0x7f123805c7d0 msgr2=0x7f123805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:24.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- 172.21.2.111:0/3033338041 shutdown_connections 2021-04-03T03:12:24.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- 172.21.2.111:0/3033338041 wait complete. 2021-04-03T03:12:24.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 Processor -- start 2021-04-03T03:12:24.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- start start 2021-04-03T03:12:24.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380603f0 0x7f1238146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:24.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f1238145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:24.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12380b5d60 0x7f12380b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:24.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b0460 con 0x7f12380603f0 2021-04-03T03:12:24.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380aff20 con 0x7f12380b5d60 2021-04-03T03:12:24.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f123c804700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b01c0 con 0x7f12380b44a0 2021-04-03T03:12:24.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f12377fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380603f0 0x7f1238146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:24.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f12377fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380603f0 0x7f1238146810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52468/0 (socket says 172.21.2.111:52468) 2021-04-03T03:12:24.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12380b5d60 0x7f12380b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:24.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f12377fe700 1 -- 172.21.2.111:0/3284447268 learned_addr learned my addr 172.21.2.111:0/3284447268 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:24.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1236ffd700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f1238145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:24.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 -- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 msgr2=0x7f1238145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:24.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f1238145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:24.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 -- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380603f0 msgr2=0x7f1238146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:24.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380603f0 0x7f1238146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:24.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 -- 172.21.2.111:0/3284447268 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380aead0 con 0x7f12380b5d60 2021-04-03T03:12:24.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f12377fe700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380603f0 0x7f1238146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:12:24.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.859+0000 7f1237fff700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12380b5d60 0x7f12380b7d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f123000c000 tx=0x7f12300091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:24.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f1234ff9700 1 -- 172.21.2.111:0/3284447268 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123000c8d0 con 0x7f12380b5d60 2021-04-03T03:12:24.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f1234ff9700 1 -- 172.21.2.111:0/3284447268 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123000ca30 con 0x7f12380b5d60 2021-04-03T03:12:24.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12380aed80 con 0x7f12380b5d60 2021-04-03T03:12:24.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12380af1f0 con 0x7f12380b5d60 2021-04-03T03:12:24.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f1234ff9700 1 -- 172.21.2.111:0/3284447268 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123000cd90 con 0x7f12380b5d60 2021-04-03T03:12:24.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12380afd80 con 0x7f12380b5d60 2021-04-03T03:12:24.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f1234ff9700 1 -- 172.21.2.111:0/3284447268 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1230022070 con 0x7f12380b5d60 2021-04-03T03:12:24.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f1234ff9700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12180570a0 0x7f1218059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:24.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f1234ff9700 1 -- 172.21.2.111:0/3284447268 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1230070690 con 0x7f12380b5d60 2021-04-03T03:12:24.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f12377fe700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12180570a0 0x7f1218059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:24.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.863+0000 7f12377fe700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12180570a0 0x7f1218059560 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f122800d000 tx=0x7f1228009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:24.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.867+0000 7f1234ff9700 1 -- 172.21.2.111:0/3284447268 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f12380afd80 con 0x7f12380b5d60 2021-04-03T03:12:25.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.991+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12380af1f0 con 0x7f12380b5d60 2021-04-03T03:12:25.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.991+0000 7f1234ff9700 1 -- 172.21.2.111:0/3284447268 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f12380af1f0 con 0x7f12380b5d60 2021-04-03T03:12:25.006 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:25.007 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:12:25.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12180570a0 msgr2=0x7f1218059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12180570a0 0x7f1218059560 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f122800d000 tx=0x7f1228009040).stop 2021-04-03T03:12:25.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12380b5d60 msgr2=0x7f12380b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12380b5d60 0x7f12380b7d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f123000c000 tx=0x7f12300091a0).stop 2021-04-03T03:12:25.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 shutdown_connections 2021-04-03T03:12:25.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f12180570a0 0x7f1218059560 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12380b5d60 0x7f12380b7d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12380603f0 0x7f1238146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 --2- 172.21.2.111:0/3284447268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12380b44a0 0x7f1238145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 >> 172.21.2.111:0/3284447268 conn(0x7f123805c7d0 msgr2=0x7f12380b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 shutdown_connections 2021-04-03T03:12:25.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:24.995+0000 7f123c804700 1 -- 172.21.2.111:0/3284447268 wait complete. 2021-04-03T03:12:25.011 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:12:25.015 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2021-04-03T03:12:25.079 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-04-03T03:12:25.079 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-04-03T03:12:25.080 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:12:25.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 Processor -- start 2021-04-03T03:12:25.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- start start 2021-04-03T03:12:25.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b8350 0x7f56880b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f56880b30c0 0x7f5688144e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880b24d0 con 0x7f56880b3a90 2021-04-03T03:12:25.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5688142090 con 0x7f56880b30c0 2021-04-03T03:12:25.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b8730 con 0x7f56880b8350 2021-04-03T03:12:25.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f56880b30c0 0x7f5688144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f56880b30c0 0x7f5688144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52474/0 (socket says 172.21.2.111:52474) 2021-04-03T03:12:25.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 -- 172.21.2.111:0/755893796 learned_addr learned my addr 172.21.2.111:0/755893796 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 -- 172.21.2.111:0/755893796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b8350 msgr2=0x7f56880b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 --2- 172.21.2.111:0/755893796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b8350 0x7f56880b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 -- 172.21.2.111:0/755893796 >> v1:172.21.2.115:6789/0 conn(0x7f56880b3a90 legacy=0x7f56881446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 -- 172.21.2.111:0/755893796 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56881413c0 con 0x7f56880b30c0 2021-04-03T03:12:25.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5686d9d700 1 --2- 172.21.2.111:0/755893796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b30c0 0x7f5688144e00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f5688141660 tx=0x7f5680009270).ready entity=mon.0 client_cookie=8ba1fe2bc1e615d8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568559a700 1 -- 172.21.2.111:0/755893796 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f568000bbc0 con 0x7f56880b30c0 2021-04-03T03:12:25.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568559a700 1 -- 172.21.2.111:0/755893796 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568000bd20 con 0x7f56880b30c0 2021-04-03T03:12:25.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568559a700 1 -- 172.21.2.111:0/755893796 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5680011400 con 0x7f56880b30c0 2021-04-03T03:12:25.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- 172.21.2.111:0/755893796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b30c0 msgr2=0x7f5688144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- 172.21.2.111:0/755893796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b30c0 0x7f5688144e00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f5688141660 tx=0x7f5680009270).stop 2021-04-03T03:12:25.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- 172.21.2.111:0/755893796 shutdown_connections 2021-04-03T03:12:25.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- 172.21.2.111:0/755893796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b30c0 0x7f5688144e00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- 172.21.2.111:0/755893796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b8350 0x7f56880b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- 172.21.2.111:0/755893796 >> 172.21.2.111:0/755893796 conn(0x7f5688060490 msgr2=0x7f56880b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- 172.21.2.111:0/755893796 shutdown_connections 2021-04-03T03:12:25.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- 172.21.2.111:0/755893796 wait complete. 2021-04-03T03:12:25.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 Processor -- start 2021-04-03T03:12:25.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- start start 2021-04-03T03:12:25.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b30c0 0x7f5688159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b3a90 0x7f56880aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56880b8350 0x7f5688143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568815ab00 con 0x7f56880b3a90 2021-04-03T03:12:25.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568815a360 con 0x7f56880b8350 2021-04-03T03:12:25.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568815a5c0 con 0x7f56880b30c0 2021-04-03T03:12:25.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b3a90 0x7f56880aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b3a90 0x7f56880aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52482/0 (socket says 172.21.2.111:52482) 2021-04-03T03:12:25.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 -- 172.21.2.111:0/1488919580 learned_addr learned my addr 172.21.2.111:0/1488919580 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 -- 172.21.2.111:0/1488919580 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b30c0 msgr2=0x7f5688159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b30c0 0x7f5688159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 -- 172.21.2.111:0/1488919580 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56880b8350 msgr2=0x7f5688143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56880b8350 0x7f5688143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 -- 172.21.2.111:0/1488919580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568815a860 con 0x7f56880b3a90 2021-04-03T03:12:25.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f5685d9b700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b3a90 0x7f56880aed60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f5678000f30 tx=0x7f5678007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f567f7fe700 1 -- 172.21.2.111:0/1488919580 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f567800e4a0 con 0x7f56880b3a90 2021-04-03T03:12:25.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f567f7fe700 1 -- 172.21.2.111:0/1488919580 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567800e600 con 0x7f56880b3a90 2021-04-03T03:12:25.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56880b1b30 con 0x7f56880b3a90 2021-04-03T03:12:25.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f567f7fe700 1 -- 172.21.2.111:0/1488919580 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f567800e960 con 0x7f56880b3a90 2021-04-03T03:12:25.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.123+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881425f0 con 0x7f56880b3a90 2021-04-03T03:12:25.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.127+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56880b1860 con 0x7f56880b3a90 2021-04-03T03:12:25.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.127+0000 7f567f7fe700 1 -- 172.21.2.111:0/1488919580 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f567801a060 con 0x7f56880b3a90 2021-04-03T03:12:25.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.127+0000 7f567f7fe700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f566c056fb0 0x7f566c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.127+0000 7f567f7fe700 1 -- 172.21.2.111:0/1488919580 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f567806fb00 con 0x7f56880b3a90 2021-04-03T03:12:25.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.127+0000 7f568659c700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f566c056fb0 0x7f566c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.127+0000 7f568659c700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f566c056fb0 0x7f566c059470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f5670002f30 tx=0x7f56700069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.127+0000 7f567f7fe700 1 -- 172.21.2.111:0/1488919580 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5678044ef0 con 0x7f56880b3a90 2021-04-03T03:12:25.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5688057660 con 0x7f56880b3a90 2021-04-03T03:12:25.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f567f7fe700 1 -- 172.21.2.111:0/1488919580 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f5678018070 con 0x7f56880b3a90 2021-04-03T03:12:25.274 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:25.275 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:12:25.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f566c056fb0 msgr2=0x7f566c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f566c056fb0 0x7f566c059470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f5670002f30 tx=0x7f56700069f0).stop 2021-04-03T03:12:25.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b3a90 msgr2=0x7f56880aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b3a90 0x7f56880aed60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f5678000f30 tx=0x7f5678007be0).stop 2021-04-03T03:12:25.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 shutdown_connections 2021-04-03T03:12:25.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f566c056fb0 0x7f566c059470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f56880b8350 0x7f5688143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f56880b30c0 0x7f5688159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 --2- 172.21.2.111:0/1488919580 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f56880b3a90 0x7f56880aed60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 >> 172.21.2.111:0/1488919580 conn(0x7f5688060490 msgr2=0x7f5688061610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 shutdown_connections 2021-04-03T03:12:25.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.259+0000 7f568759e700 1 -- 172.21.2.111:0/1488919580 wait complete. 2021-04-03T03:12:25.278 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:12:25.282 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2021-04-03T03:12:25.346 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-04-03T03:12:25.346 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-04-03T03:12:25.347 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:12:25.357 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:12:25.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 Processor -- start 2021-04-03T03:12:25.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- start start 2021-04-03T03:12:25.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc09c0603f0 0x7fc09c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c0af400 con 0x7fc09c0b0c90 2021-04-03T03:12:25.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c0af6a0 con 0x7fc09c14b700 2021-04-03T03:12:25.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c060860 con 0x7fc09c0603f0 2021-04-03T03:12:25.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1b4a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc09c0603f0 0x7fc09c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a234b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc09c0b0c90 0x7fc09c147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53284/0 (socket says 172.21.2.111:53284) 2021-04-03T03:12:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1b4a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc09c0603f0 0x7fc09c0b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52488/0 (socket says 172.21.2.111:52488) 2021-04-03T03:12:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a234b700 1 -- 172.21.2.111:0/1568222189 learned_addr learned my addr 172.21.2.111:0/1568222189 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1b4a700 1 -- 172.21.2.111:0/1568222189 >> v1:172.21.2.115:6789/0 conn(0x7fc09c14b700 legacy=0x7fc09c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1b4a700 1 -- 172.21.2.111:0/1568222189 >> v1:172.21.2.111:6789/0 conn(0x7fc09c0b0c90 legacy=0x7fc09c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1b4a700 1 -- 172.21.2.111:0/1568222189 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c0afb40 con 0x7fc09c0603f0 2021-04-03T03:12:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1b4a700 1 --2- 172.21.2.111:0/1568222189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0603f0 0x7fc09c0b0090 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fc09c0620f0 tx=0x7fc098006dd0).ready entity=mon.0 client_cookie=865ee1f0e50db664 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a0b48700 1 -- 172.21.2.111:0/1568222189 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0980097b0 con 0x7fc09c0603f0 2021-04-03T03:12:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a0b48700 1 -- 172.21.2.111:0/1568222189 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc098009910 con 0x7fc09c0603f0 2021-04-03T03:12:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a0b48700 1 -- 172.21.2.111:0/1568222189 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc098009c70 con 0x7fc09c0603f0 2021-04-03T03:12:25.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/1568222189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0603f0 msgr2=0x7fc09c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/1568222189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0603f0 0x7fc09c0b0090 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fc09c0620f0 tx=0x7fc098006dd0).stop 2021-04-03T03:12:25.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/1568222189 shutdown_connections 2021-04-03T03:12:25.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/1568222189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0603f0 0x7fc09c0b0090 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/1568222189 >> 172.21.2.111:0/1568222189 conn(0x7fc09c0b61c0 msgr2=0x7fc09c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/1568222189 shutdown_connections 2021-04-03T03:12:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/1568222189 wait complete. 2021-04-03T03:12:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 Processor -- start 2021-04-03T03:12:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- start start 2021-04-03T03:12:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc09c0603f0 0x7fc09c062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0b0c90 0x7fc09c14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc09c14b700 0x7fc09c064190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0a9360 con 0x7fc09c0b0c90 2021-04-03T03:12:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0a9e00 con 0x7fc09c14b700 2021-04-03T03:12:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a2b4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0aa0a0 con 0x7fc09c0603f0 2021-04-03T03:12:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1349700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0b0c90 0x7fc09c14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1349700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0b0c90 0x7fc09c14ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52496/0 (socket says 172.21.2.111:52496) 2021-04-03T03:12:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1349700 1 -- 172.21.2.111:0/3419432623 learned_addr learned my addr 172.21.2.111:0/3419432623 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.403+0000 7fc0a1b4a700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc09c0603f0 0x7fc09c062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1349700 1 -- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc09c0603f0 msgr2=0x7fc09c062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1349700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc09c0603f0 0x7fc09c062bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1349700 1 -- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc09c14b700 msgr2=0x7fc09c064190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1349700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc09c14b700 0x7fc09c064190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1349700 1 -- 172.21.2.111:0/3419432623 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c0aa340 con 0x7fc09c0b0c90 2021-04-03T03:12:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1b4a700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc09c0603f0 0x7fc09c062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:12:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1349700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0b0c90 0x7fc09c14ceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fc08c002eb0 tx=0x7fc08c006d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc08affd700 1 -- 172.21.2.111:0/3419432623 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc08c00d4a0 con 0x7fc09c0b0c90 2021-04-03T03:12:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc08affd700 1 -- 172.21.2.111:0/3419432623 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c000e90 con 0x7fc09c0b0c90 2021-04-03T03:12:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c14d3a0 con 0x7fc09c0b0c90 2021-04-03T03:12:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc08affd700 1 -- 172.21.2.111:0/3419432623 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc08c00d7f0 con 0x7fc09c0b0c90 2021-04-03T03:12:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09c064890 con 0x7fc09c0b0c90 2021-04-03T03:12:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09c0ab7f0 con 0x7fc09c0b0c90 2021-04-03T03:12:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc08affd700 1 -- 172.21.2.111:0/3419432623 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc08c00f030 con 0x7fc09c0b0c90 2021-04-03T03:12:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc08affd700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc07c056ea0 0x7fc07c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc08affd700 1 -- 172.21.2.111:0/3419432623 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc08c06fdc0 con 0x7fc09c0b0c90 2021-04-03T03:12:25.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1b4a700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc07c056ea0 0x7fc07c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.407+0000 7fc0a1b4a700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc07c056ea0 0x7fc07c059360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fc098017000 tx=0x7fc098015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.411+0000 7fc08affd700 1 -- 172.21.2.111:0/3419432623 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc08c0451b0 con 0x7fc09c0b0c90 2021-04-03T03:12:25.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc09c057660 con 0x7fc09c0b0c90 2021-04-03T03:12:25.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc08affd700 1 -- 172.21.2.111:0/3419432623 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fc08c0487d0 con 0x7fc09c0b0c90 2021-04-03T03:12:25.554 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:25.554 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:12:25.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc07c056ea0 msgr2=0x7fc07c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc07c056ea0 0x7fc07c059360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fc098017000 tx=0x7fc098015000).stop 2021-04-03T03:12:25.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0b0c90 msgr2=0x7fc09c14ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0b0c90 0x7fc09c14ceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fc08c002eb0 tx=0x7fc08c006d10).stop 2021-04-03T03:12:25.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 shutdown_connections 2021-04-03T03:12:25.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc07c056ea0 0x7fc07c059360 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc09c14b700 0x7fc09c064190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc09c0603f0 0x7fc09c062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 --2- 172.21.2.111:0/3419432623 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc09c0b0c90 0x7fc09c14ceb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 >> 172.21.2.111:0/3419432623 conn(0x7fc09c0b61c0 msgr2=0x7fc09c1480a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 shutdown_connections 2021-04-03T03:12:25.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.539+0000 7fc0a2b4c700 1 -- 172.21.2.111:0/3419432623 wait complete. 2021-04-03T03:12:25.558 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:12:25.561 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 5 2021-04-03T03:12:25.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.603+0000 7f8168eb6700 1 Processor -- start 2021-04-03T03:12:25.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- start start 2021-04-03T03:12:25.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0cdcf0 0x7f815c0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c039580 con 0x7f815c042850 2021-04-03T03:12:25.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c03da60 con 0x7f815c042080 2021-04-03T03:12:25.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0406d0 con 0x7f815c0cdcf0 2021-04-03T03:12:25.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8162ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f815c042850 0x7f815c03d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53294/0 (socket says 172.21.2.111:53294) 2021-04-03T03:12:25.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8162ffd700 1 -- 172.21.2.111:0/3844942168 learned_addr learned my addr 172.21.2.111:0/3844942168 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:25.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8163fff700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0cdcf0 0x7f815c0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762768240 0 0) 0x7f815c039580 con 0x7f815c042850 2021-04-03T03:12:25.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8148003240 con 0x7f815c042850 2021-04-03T03:12:25.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 722659264 0 0) 0x7f8148003240 con 0x7f815c042850 2021-04-03T03:12:25.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0cdcf0 msgr2=0x7f815c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0cdcf0 0x7f815c0ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 >> v1:172.21.2.115:6789/0 conn(0x7f815c042080 legacy=0x7f815c03ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c042460 con 0x7f815c042850 2021-04-03T03:12:25.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f81580016d0 con 0x7f815c042850 2021-04-03T03:12:25.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 >> v1:172.21.2.111:6789/0 conn(0x7f815c042850 legacy=0x7f815c03d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8148003760 0x7f8148003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0cdcf0 0x7f81480044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81480049d0 0x7f8148004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0406d0 con 0x7f8148003760 2021-04-03T03:12:25.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148004250 con 0x7f81480049d0 2021-04-03T03:12:25.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814800d940 con 0x7f815c0cdcf0 2021-04-03T03:12:25.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81637fe700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8148003760 0x7f8148003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81637fe700 1 -- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0cdcf0 msgr2=0x7f81480044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81637fe700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0cdcf0 0x7f81480044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81637fe700 1 -- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81480049d0 msgr2=0x7f8148004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81637fe700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f81480049d0 0x7f8148004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81637fe700 1 -- 172.21.2.111:0/3844942168 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8148003240 con 0x7f8148003760 2021-04-03T03:12:25.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81637fe700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8148003760 0x7f8148003b80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f815c040ad0 tx=0x7f8150007b70).ready entity=mon.0 client_cookie=8dfe6c0dbcd88198 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f815000ca60 con 0x7f8148003760 2021-04-03T03:12:25.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150015070 con 0x7f8148003760 2021-04-03T03:12:25.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f81627fc700 1 -- 172.21.2.111:0/3844942168 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f815000cdb0 con 0x7f8148003760 2021-04-03T03:12:25.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- 172.21.2.111:0/3844942168 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8148003760 msgr2=0x7f8148003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 --2- 172.21.2.111:0/3844942168 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8148003760 0x7f8148003b80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f815c040ad0 tx=0x7f8150007b70).stop 2021-04-03T03:12:25.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8163fff700 1 -- 172.21.2.111:0/3844942168 reap_dead start 2021-04-03T03:12:25.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- 172.21.2.111:0/3844942168 shutdown_connections 2021-04-03T03:12:25.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- 172.21.2.111:0/3844942168 >> 172.21.2.111:0/3844942168 conn(0x7f815c0462b0 msgr2=0x7f815c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- 172.21.2.111:0/3844942168 shutdown_connections 2021-04-03T03:12:25.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- 172.21.2.111:0/3844942168 wait complete. 2021-04-03T03:12:25.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 Processor -- start 2021-04-03T03:12:25.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- start start 2021-04-03T03:12:25.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0d4b50 0x7f815c0b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f815c0cd1e0 0x7f815c0b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f815c0ba860 0x7f815c0e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0b7ad0 con 0x7f815c0cd1e0 2021-04-03T03:12:25.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0d4f30 con 0x7f815c0ba860 2021-04-03T03:12:25.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8168eb6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0cd680 con 0x7f815c0d4b50 2021-04-03T03:12:25.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8162ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f815c0cd1e0 0x7f815c0b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8162ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f815c0cd1e0 0x7f815c0b75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52514/0 (socket says 172.21.2.111:52514) 2021-04-03T03:12:25.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.607+0000 7f8162ffd700 1 -- 172.21.2.111:0/4283106799 learned_addr learned my addr 172.21.2.111:0/4283106799 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8162ffd700 1 -- 172.21.2.111:0/4283106799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0d4b50 msgr2=0x7f815c0b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8162ffd700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0d4b50 0x7f815c0b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8162ffd700 1 -- 172.21.2.111:0/4283106799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f815c0ba860 msgr2=0x7f815c0e1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8162ffd700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f815c0ba860 0x7f815c0e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8162ffd700 1 -- 172.21.2.111:0/4283106799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c005840 con 0x7f815c0cd1e0 2021-04-03T03:12:25.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8162ffd700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f815c0cd1e0 0x7f815c0b75e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f81580013e0 tx=0x7f8158001a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8160ff9700 1 -- 172.21.2.111:0/4283106799 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8158007170 con 0x7f815c0cd1e0 2021-04-03T03:12:25.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8160ff9700 1 -- 172.21.2.111:0/4283106799 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81580072d0 con 0x7f815c0cd1e0 2021-04-03T03:12:25.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c03ca10 con 0x7f815c0cd1e0 2021-04-03T03:12:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8160ff9700 1 -- 172.21.2.111:0/4283106799 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81580049c0 con 0x7f815c0cd1e0 2021-04-03T03:12:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c0e2130 con 0x7f815c0cd1e0 2021-04-03T03:12:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8160ff9700 1 -- 172.21.2.111:0/4283106799 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8158010070 con 0x7f815c0cd1e0 2021-04-03T03:12:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8160ff9700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f814c056fe0 0x7f814c0594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8160ff9700 1 -- 172.21.2.111:0/4283106799 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f815806fb10 con 0x7f815c0cd1e0 2021-04-03T03:12:25.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f814c056fe0 0x7f814c0594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f814c056fe0 0x7f814c0594a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f815c040ad0 tx=0x7f8150018000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 Processor -- start 2021-04-03T03:12:25.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 -- start start 2021-04-03T03:12:25.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x2914880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x26fd820 0x28dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x2827f70 con 0x28953b0 2021-04-03T03:12:25.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x28280b0 con 0x277deb0 2021-04-03T03:12:25.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x28294e0 con 0x26fd820 2021-04-03T03:12:25.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x2914880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x2914880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52524/0 (socket says 172.21.2.111:52524) 2021-04-03T03:12:25.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 -- 172.21.2.111:0/2440478221 learned_addr learned my addr 172.21.2.111:0/2440478221 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 -- 172.21.2.111:0/2440478221 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x26fd820 msgr2=0x28dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 --2- 172.21.2.111:0/2440478221 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x26fd820 0x28dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 -- 172.21.2.111:0/2440478221 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 msgr2=0x27f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 --2- 172.21.2.111:0/2440478221 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 -- 172.21.2.111:0/2440478221 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2818c90 con 0x28953b0 2021-04-03T03:12:25.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f81637fe700 1 --2- 172.21.2.111:0/2440478221 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x2914880 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f8150001720 tx=0x7f8150020000).ready entity=mon.0 client_cookie=15d34054e8fc6f9c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8168eb6700 1 -- 172.21.2.111:0/2440478221 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8150020070 con 0x28953b0 2021-04-03T03:12:25.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8168eb6700 1 -- 172.21.2.111:0/2440478221 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150011d90 con 0x28953b0 2021-04-03T03:12:25.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8168eb6700 1 -- 172.21.2.111:0/2440478221 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8150024610 con 0x28953b0 2021-04-03T03:12:25.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 -- 172.21.2.111:0/2440478221 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 msgr2=0x2914880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.611+0000 7f8178934740 1 --2- 172.21.2.111:0/2440478221 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x2914880 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f8150001720 tx=0x7f8150020000).stop 2021-04-03T03:12:25.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/2440478221 shutdown_connections 2021-04-03T03:12:25.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/2440478221 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x26fd820 0x28dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/2440478221 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x2914880 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/2440478221 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/2440478221 >> 172.21.2.111:0/2440478221 conn(0x28c4c10 msgr2=0x28db610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/2440478221 shutdown_connections 2021-04-03T03:12:25.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/2440478221 wait complete. 2021-04-03T03:12:25.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- start start 2021-04-03T03:12:25.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 Processor -- start 2021-04-03T03:12:25.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x26fd820 0x29226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x277deb0 0x28c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x29263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x280a430 con 0x28953b0 2021-04-03T03:12:25.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x2923980 con 0x26fd820 2021-04-03T03:12:25.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x2925cd0 con 0x277deb0 2021-04-03T03:12:25.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 10 client.0 ll_register_callbacks cb 0x28c4b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:12:25.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x29263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x29263a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52530/0 (socket says 172.21.2.111:52530) 2021-04-03T03:12:25.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 -- 172.21.2.111:0/380121705 learned_addr learned my addr 172.21.2.111:0/380121705 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 -- 172.21.2.111:0/380121705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x277deb0 msgr2=0x28c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x277deb0 0x28c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 -- 172.21.2.111:0/380121705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x26fd820 msgr2=0x29226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x26fd820 0x29226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 -- 172.21.2.111:0/380121705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x28214c0 con 0x28953b0 2021-04-03T03:12:25.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8163fff700 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x29263a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x28e81f0 tx=0x7f816405ad70).ready entity=mon.0 client_cookie=8dcd8453a82157a2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:12:25.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 1 -- 172.21.2.111:0/380121705 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816405df60 con 0x28953b0 2021-04-03T03:12:25.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 1 -- 172.21.2.111:0/380121705 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f81640699f0 con 0x28953b0 2021-04-03T03:12:25.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27228d0 con 0x28953b0 2021-04-03T03:12:25.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 1 -- 172.21.2.111:0/380121705 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f816405b040 con 0x28953b0 2021-04-03T03:12:25.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 10 client.5790 fetch_fsmap learned FSMap version 21 2021-04-03T03:12:25.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x26ad790 con 0x28953b0 2021-04-03T03:12:25.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 1 -- 172.21.2.111:0/380121705 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27228d0 con 0x28953b0 2021-04-03T03:12:25.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 1 -- 172.21.2.111:0/380121705 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 21) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f816406adb0 con 0x28953b0 2021-04-03T03:12:25.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 10 client.5790 fetch_fsmap finished waiting for FSMap version 21 2021-04-03T03:12:25.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 10 client.5790 resolve_mds: resolved ID 'a' to GID 5580 2021-04-03T03:12:25.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 20 client.5790 populate_metadata read hostname 'gibba011' 2021-04-03T03:12:25.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x268fc20 0x28e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 4 client.5790 mds_command: new command op to 5580 tid=0[{"prefix": "get_command_descriptions"}] 2021-04-03T03:12:25.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 --> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x268f170 con 0x268fc20 2021-04-03T03:12:25.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f81637fe700 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x268fc20 0x28e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 10 client.5790 ms_handle_connect on v2:172.21.2.111:6834/2278897120 2021-04-03T03:12:25.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f81637fe700 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x268fc20 0x28e20f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ce73c380ea85f13 server_cookie=b94751c90d7a4a57 in_seq=0 out_seq=0 2021-04-03T03:12:25.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 1 -- 172.21.2.111:0/380121705 <== mds.0 v2:172.21.2.111:6834/2278897120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x268f170 con 0x268fc20 2021-04-03T03:12:25.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 10 client.5790 handle_command_reply: tid=0 2021-04-03T03:12:25.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 client.5790 shutdown 2021-04-03T03:12:25.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 msgr2=0x29263a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x29263a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x28e81f0 tx=0x7f816405ad70).stop 2021-04-03T03:12:25.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 shutdown_connections 2021-04-03T03:12:25.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x28953b0 0x29263a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x26fd820 0x29226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x268fc20 0x28e20f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 --2- 172.21.2.111:0/380121705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x277deb0 0x28c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 >> 172.21.2.111:0/380121705 conn(0x28c4c10 msgr2=0x2919570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f813e7fc700 0 client.5790 ms_handle_reset on v2:172.21.2.111:6834/2278897120 2021-04-03T03:12:25.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 shutdown_connections 2021-04-03T03:12:25.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 1 -- 172.21.2.111:0/380121705 wait complete. 2021-04-03T03:12:25.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.615+0000 7f8178934740 20 client.5790 trim_cache size 0 max 16384 2021-04-03T03:12:25.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 Processor -- start 2021-04-03T03:12:25.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- start start 2021-04-03T03:12:25.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x28e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 0x2932400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x29164e0 con 0x26fd820 2021-04-03T03:12:25.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x28e63a0 con 0x277deb0 2021-04-03T03:12:25.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x28e6b30 con 0x28953b0 2021-04-03T03:12:25.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x28e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x28e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52538/0 (socket says 172.21.2.111:52538) 2021-04-03T03:12:25.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 -- 172.21.2.111:0/242739545 learned_addr learned my addr 172.21.2.111:0/242739545 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 -- 172.21.2.111:0/242739545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 msgr2=0x2932400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 --2- 172.21.2.111:0/242739545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 0x2932400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 -- 172.21.2.111:0/242739545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 msgr2=0x27f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 --2- 172.21.2.111:0/242739545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 -- 172.21.2.111:0/242739545 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0e98b0 con 0x26fd820 2021-04-03T03:12:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f81637fe700 1 --2- 172.21.2.111:0/242739545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x28e3ac0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f815003c000 tx=0x7f8150033040).ready entity=mon.0 client_cookie=d8156f2e6e1a5d14 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8144ff9700 1 -- 172.21.2.111:0/242739545 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8150036040 con 0x26fd820 2021-04-03T03:12:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8144ff9700 1 -- 172.21.2.111:0/242739545 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150011390 con 0x26fd820 2021-04-03T03:12:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8144ff9700 1 -- 172.21.2.111:0/242739545 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8150020040 con 0x26fd820 2021-04-03T03:12:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- 172.21.2.111:0/242739545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 msgr2=0x28e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 --2- 172.21.2.111:0/242739545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x28e3ac0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f815003c000 tx=0x7f8150033040).stop 2021-04-03T03:12:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- 172.21.2.111:0/242739545 shutdown_connections 2021-04-03T03:12:25.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 --2- 172.21.2.111:0/242739545 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 0x2932400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 --2- 172.21.2.111:0/242739545 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x28e3ac0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 --2- 172.21.2.111:0/242739545 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- 172.21.2.111:0/242739545 >> 172.21.2.111:0/242739545 conn(0x28c4c10 msgr2=0x29222c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:25.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- 172.21.2.111:0/242739545 shutdown_connections 2021-04-03T03:12:25.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.623+0000 7f8178934740 1 -- 172.21.2.111:0/242739545 wait complete. 2021-04-03T03:12:25.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 -- start start 2021-04-03T03:12:25.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 Processor -- start 2021-04-03T03:12:25.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x2914da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 0x297d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x28e4690 con 0x26fd820 2021-04-03T03:12:25.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x28e4930 con 0x277deb0 2021-04-03T03:12:25.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x297d1c0 con 0x28953b0 2021-04-03T03:12:25.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 10 client.0 ll_register_callbacks cb 0x28df970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:12:25.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x2914da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x2914da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52542/0 (socket says 172.21.2.111:52542) 2021-04-03T03:12:25.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 -- 172.21.2.111:0/1296507842 learned_addr learned my addr 172.21.2.111:0/1296507842 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:25.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8163fff700 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 0x297d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 -- 172.21.2.111:0/1296507842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 msgr2=0x297d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 0x297d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:25.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 -- 172.21.2.111:0/1296507842 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 msgr2=0x27f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:25.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:25.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 -- 172.21.2.111:0/1296507842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26b8e70 con 0x26fd820 2021-04-03T03:12:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f81637fe700 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x2914da0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f815003c000 tx=0x7f8150029040).ready entity=mon.0 client_cookie=beefb28259a53244 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f813f7fe700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:12:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f813f7fe700 1 -- 172.21.2.111:0/1296507842 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8150020070 con 0x26fd820 2021-04-03T03:12:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f813f7fe700 1 -- 172.21.2.111:0/1296507842 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f815003bd30 con 0x26fd820 2021-04-03T03:12:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2741730 con 0x26fd820 2021-04-03T03:12:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f813f7fe700 1 -- 172.21.2.111:0/1296507842 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8150036030 con 0x26fd820 2021-04-03T03:12:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 10 client.5796 fetch_fsmap learned FSMap version 21 2021-04-03T03:12:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x297cd40 con 0x26fd820 2021-04-03T03:12:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f813f7fe700 1 -- 172.21.2.111:0/1296507842 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2741730 con 0x26fd820 2021-04-03T03:12:25.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f813f7fe700 1 -- 172.21.2.111:0/1296507842 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 21) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f8150011d10 con 0x26fd820 2021-04-03T03:12:25.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 10 client.5796 fetch_fsmap finished waiting for FSMap version 21 2021-04-03T03:12:25.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 10 client.5796 resolve_mds: resolved ID 'a' to GID 5580 2021-04-03T03:12:25.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 20 client.5796 populate_metadata read hostname 'gibba011' 2021-04-03T03:12:25.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x27f9d30 0x297db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:25.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 4 client.5796 mds_command: new command op to 5580 tid=0[{"prefix": "cache drop", "timeout": 5}] 2021-04-03T03:12:25.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 --> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0x2718440 con 0x27f9d30 2021-04-03T03:12:25.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8162ffd700 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x27f9d30 0x297db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:25.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f813f7fe700 10 client.5796 ms_handle_connect on v2:172.21.2.111:6834/2278897120 2021-04-03T03:12:25.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:25.627+0000 7f8162ffd700 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x27f9d30 0x297db50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7ef10a9a0a1bea9e server_cookie=9df46e43bd171939 in_seq=0 out_seq=0 2021-04-03T03:12:28.796 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:28.799 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:28.802 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:28.806 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:28.808 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:30.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f813f7fe700 1 -- 172.21.2.111:0/1296507842 <== mds.0 v2:172.21.2.111:6834/2278897120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0x2718440 con 0x27f9d30 2021-04-03T03:12:30.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f813f7fe700 10 client.5796 handle_command_reply: tid=0 2021-04-03T03:12:30.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 client.5796 shutdown 2021-04-03T03:12:30.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 msgr2=0x2914da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:30.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x2914da0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f815003c000 tx=0x7f8150029040).stop 2021-04-03T03:12:30.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 shutdown_connections 2021-04-03T03:12:30.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:6834/2278897120,v1:172.21.2.111:6835/2278897120] conn(0x27f9d30 0x297db50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:30.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x28953b0 0x297d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:30.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x26fd820 0x2914da0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:30.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 --2- 172.21.2.111:0/1296507842 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x277deb0 0x27f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:30.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 >> 172.21.2.111:0/1296507842 conn(0x28c4c10 msgr2=0x28e6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:30.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f813f7fe700 0 client.5796 ms_handle_reset on v2:172.21.2.111:6834/2278897120 2021-04-03T03:12:30.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 shutdown_connections 2021-04-03T03:12:30.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 1 -- 172.21.2.111:0/1296507842 wait complete. 2021-04-03T03:12:30.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8178934740 20 client.5796 trim_cache size 0 max 16384 2021-04-03T03:12:30.726 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:12:30.726 INFO:teuthology.orchestra.run.gibba011.stdout: "client_recall": { 2021-04-03T03:12:30.727 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": -110, 2021-04-03T03:12:30.727 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "(110) Connection timed out", 2021-04-03T03:12:30.727 INFO:teuthology.orchestra.run.gibba011.stdout: "recalled": 120 2021-04-03T03:12:30.727 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:30.728 INFO:teuthology.orchestra.run.gibba011.stdout: "flush_journal": { 2021-04-03T03:12:30.728 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": 0, 2021-04-03T03:12:30.728 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "" 2021-04-03T03:12:30.728 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:30.728 INFO:teuthology.orchestra.run.gibba011.stdout: "trim_cache": { 2021-04-03T03:12:30.729 INFO:teuthology.orchestra.run.gibba011.stdout: "trimmed": 0 2021-04-03T03:12:30.729 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:30.729 INFO:teuthology.orchestra.run.gibba011.stdout: "cache": { 2021-04-03T03:12:30.729 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": { 2021-04-03T03:12:30.729 INFO:teuthology.orchestra.run.gibba011.stdout: "items": 19877, 2021-04-03T03:12:30.730 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes": 3091664 2021-04-03T03:12:30.730 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:12:30.730 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:12:30.730 INFO:teuthology.orchestra.run.gibba011.stdout: "duration": 5.0802227740000001 2021-04-03T03:12:30.730 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:12:30.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f814c056fe0 msgr2=0x7f814c0594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:30.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f814c056fe0 0x7f814c0594a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f815c040ad0 tx=0x7f8150018000).stop 2021-04-03T03:12:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f815c0cd1e0 msgr2=0x7f815c0b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f815c0cd1e0 0x7f815c0b75e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f81580013e0 tx=0x7f8158001a20).stop 2021-04-03T03:12:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 shutdown_connections 2021-04-03T03:12:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f814c056fe0 0x7f814c0594a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f815c0ba860 0x7f815c0e1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f815c0d4b50 0x7f815c0b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 --2- 172.21.2.111:0/4283106799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f815c0cd1e0 0x7f815c0b75e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 >> 172.21.2.111:0/4283106799 conn(0x7f815c0462b0 msgr2=0x7f815c03d330 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 shutdown_connections 2021-04-03T03:12:30.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.707+0000 7f8168eb6700 1 -- 172.21.2.111:0/4283106799 wait complete. 2021-04-03T03:12:30.735 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-04-03T03:12:30.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:30.736 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:12:30.736 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:30.745 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-04-03T03:12:30.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:12:30.747 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-04-03T03:12:30.747 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:12:30.748 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:12:30.748 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:30.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:30.842 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:12:30.842 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:30.846 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:12:30.847 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:12:30.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:12:30.847 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:30.898 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:12:31.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 Processor -- start 2021-04-03T03:12:31.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- start start 2021-04-03T03:12:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc7540603f0 0x7fc7540b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc75414b700 0x7fc7540b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc7540b12b0 0x7fc7540b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540b0f60 con 0x7fc75414b700 2021-04-03T03:12:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540607d0 con 0x7fc7540603f0 2021-04-03T03:12:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540af400 con 0x7fc7540b12b0 2021-04-03T03:12:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc75414b700 0x7fc7540b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc75414b700 0x7fc7540b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52558/0 (socket says 172.21.2.111:52558) 2021-04-03T03:12:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 -- 172.21.2.111:0/2567897161 learned_addr learned my addr 172.21.2.111:0/2567897161 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 -- 172.21.2.111:0/2567897161 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc7540b12b0 msgr2=0x7fc7540b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 --2- 172.21.2.111:0/2567897161 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc7540b12b0 0x7fc7540b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 -- 172.21.2.111:0/2567897161 >> v2:172.21.2.115:3300/0 conn(0x7fc7540603f0 msgr2=0x7fc7540b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 --2- 172.21.2.111:0/2567897161 >> v2:172.21.2.115:3300/0 conn(0x7fc7540603f0 0x7fc7540b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 -- 172.21.2.111:0/2567897161 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540af6a0 con 0x7fc75414b700 2021-04-03T03:12:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc7525e4700 1 --2- 172.21.2.111:0/2567897161 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc75414b700 0x7fc7540b0580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fc744002f30 tx=0x7fc744006e90).ready entity=mon.0 client_cookie=7f2fe80447952982 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc751de3700 1 -- 172.21.2.111:0/2567897161 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc744009920 con 0x7fc75414b700 2021-04-03T03:12:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc751de3700 1 -- 172.21.2.111:0/2567897161 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744009a80 con 0x7fc75414b700 2021-04-03T03:12:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc751de3700 1 -- 172.21.2.111:0/2567897161 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc744009de0 con 0x7fc75414b700 2021-04-03T03:12:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- 172.21.2.111:0/2567897161 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc75414b700 msgr2=0x7fc7540b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 --2- 172.21.2.111:0/2567897161 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc75414b700 0x7fc7540b0580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fc744002f30 tx=0x7fc744006e90).stop 2021-04-03T03:12:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- 172.21.2.111:0/2567897161 shutdown_connections 2021-04-03T03:12:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 --2- 172.21.2.111:0/2567897161 >> v2:172.21.2.115:3300/0 conn(0x7fc7540603f0 0x7fc7540b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 --2- 172.21.2.111:0/2567897161 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc7540b12b0 0x7fc7540b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 --2- 172.21.2.111:0/2567897161 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc75414b700 0x7fc7540b0580 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- 172.21.2.111:0/2567897161 >> 172.21.2.111:0/2567897161 conn(0x7fc7540b61c0 msgr2=0x7fc75405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- 172.21.2.111:0/2567897161 shutdown_connections 2021-04-03T03:12:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.995+0000 7fc753de7700 1 -- 172.21.2.111:0/2567897161 wait complete. 2021-04-03T03:12:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 Processor -- start 2021-04-03T03:12:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 -- start start 2021-04-03T03:12:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc7540603f0 0x7fc7540a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc7540b12b0 0x7fc754064270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc75414b700 0x7fc754064760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540628d0 con 0x7fc7540b12b0 2021-04-03T03:12:31.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc754062b30 con 0x7fc7540603f0 2021-04-03T03:12:31.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc754062dd0 con 0x7fc75414b700 2021-04-03T03:12:31.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc7540b12b0 0x7fc754064270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc7540b12b0 0x7fc754064270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52560/0 (socket says 172.21.2.111:52560) 2021-04-03T03:12:31.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 -- 172.21.2.111:0/929889995 learned_addr learned my addr 172.21.2.111:0/929889995 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 -- 172.21.2.111:0/929889995 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc75414b700 msgr2=0x7fc754064760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc75414b700 0x7fc754064760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 -- 172.21.2.111:0/929889995 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc7540603f0 msgr2=0x7fc7540a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc7540603f0 0x7fc7540a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 -- 172.21.2.111:0/929889995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc754063070 con 0x7fc7540b12b0 2021-04-03T03:12:31.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc7525e4700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc7540b12b0 0x7fc754064270 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fc744002f30 tx=0x7fc744015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc74bfff700 1 -- 172.21.2.111:0/929889995 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc744011e80 con 0x7fc7540b12b0 2021-04-03T03:12:31.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc74bfff700 1 -- 172.21.2.111:0/929889995 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74401e460 con 0x7fc7540b12b0 2021-04-03T03:12:31.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc74bfff700 1 -- 172.21.2.111:0/929889995 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc74401e7c0 con 0x7fc7540b12b0 2021-04-03T03:12:31.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7540abb70 con 0x7fc7540b12b0 2021-04-03T03:12:31.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7540a9f00 con 0x7fc7540b12b0 2021-04-03T03:12:31.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc754057660 con 0x7fc7540b12b0 2021-04-03T03:12:31.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc74bfff700 1 -- 172.21.2.111:0/929889995 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc74401a040 con 0x7fc7540b12b0 2021-04-03T03:12:31.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc74bfff700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc738056fb0 0x7fc738059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc74bfff700 1 -- 172.21.2.111:0/929889995 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc744071450 con 0x7fc7540b12b0 2021-04-03T03:12:31.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc752de5700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc738056fb0 0x7fc738059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:30.999+0000 7fc752de5700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc738056fb0 0x7fc738059470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fc73c002f30 tx=0x7fc73c006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.003+0000 7fc74bfff700 1 -- 172.21.2.111:0/929889995 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc744046840 con 0x7fc7540b12b0 2021-04-03T03:12:31.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.091+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7540686d0 con 0x7fc7540b12b0 2021-04-03T03:12:31.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.091+0000 7fc74bfff700 1 -- 172.21.2.111:0/929889995 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16273 (secure 0 0 0) 0x7fc744023020 con 0x7fc7540b12b0 2021-04-03T03:12:31.108 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:31.108 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":54,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:12:06.034712+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:12:02.184726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-04-03T03:12:02.990138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:12:31.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc738056fb0 msgr2=0x7fc738059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc738056fb0 0x7fc738059470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fc73c002f30 tx=0x7fc73c006a60).stop 2021-04-03T03:12:31.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc7540b12b0 msgr2=0x7fc754064270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc7540b12b0 0x7fc754064270 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fc744002f30 tx=0x7fc744015000).stop 2021-04-03T03:12:31.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 shutdown_connections 2021-04-03T03:12:31.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc75414b700 0x7fc754064760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc738056fb0 0x7fc738059470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc7540603f0 0x7fc7540a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 --2- 172.21.2.111:0/929889995 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc7540b12b0 0x7fc754064270 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 >> 172.21.2.111:0/929889995 conn(0x7fc7540b61c0 msgr2=0x7fc75405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 shutdown_connections 2021-04-03T03:12:31.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.095+0000 7fc753de7700 1 -- 172.21.2.111:0/929889995 wait complete. 2021-04-03T03:12:31.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:12:02.184726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:12:02.990138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:12:31.116 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:12:31.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 Processor -- start 2021-04-03T03:12:31.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 -- start start 2021-04-03T03:12:31.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb06c144ef0 0x7fb06c147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06c0607d0 con 0x7fb06c0603f0 2021-04-03T03:12:31.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06c14b090 con 0x7fb06c0add50 2021-04-03T03:12:31.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06c14b330 con 0x7fb06c144ef0 2021-04-03T03:12:31.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072602700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb06c144ef0 0x7fb06c147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072602700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb06c144ef0 0x7fb06c147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52570/0 (socket says 172.21.2.111:52570) 2021-04-03T03:12:31.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072602700 1 -- 172.21.2.111:0/1140415886 learned_addr learned my addr 172.21.2.111:0/1140415886 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072602700 1 -- 172.21.2.111:0/1140415886 >> v1:172.21.2.115:6789/0 conn(0x7fb06c0add50 legacy=0x7fb06c147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:12:31.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072602700 1 -- 172.21.2.111:0/1140415886 >> v1:172.21.2.111:6789/0 conn(0x7fb06c0603f0 legacy=0x7fb06c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072602700 1 -- 172.21.2.111:0/1140415886 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c14b5d0 con 0x7fb06c144ef0 2021-04-03T03:12:31.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072602700 1 --2- 172.21.2.111:0/1140415886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c147e50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fb06c0b5600 tx=0x7fb064009270).ready entity=mon.0 client_cookie=985e9791f2e9ee0a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb070dff700 1 -- 172.21.2.111:0/1140415886 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06400bbc0 con 0x7fb06c144ef0 2021-04-03T03:12:31.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb070dff700 1 -- 172.21.2.111:0/1140415886 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06400bd20 con 0x7fb06c144ef0 2021-04-03T03:12:31.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb070dff700 1 -- 172.21.2.111:0/1140415886 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06400be90 con 0x7fb06c144ef0 2021-04-03T03:12:31.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 -- 172.21.2.111:0/1140415886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 msgr2=0x7fb06c147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.159+0000 7fb072e03700 1 --2- 172.21.2.111:0/1140415886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c147e50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fb06c0b5600 tx=0x7fb064009270).stop 2021-04-03T03:12:31.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- 172.21.2.111:0/1140415886 shutdown_connections 2021-04-03T03:12:31.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 --2- 172.21.2.111:0/1140415886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c147e50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- 172.21.2.111:0/1140415886 >> 172.21.2.111:0/1140415886 conn(0x7fb06c0b61c0 msgr2=0x7fb06c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- 172.21.2.111:0/1140415886 shutdown_connections 2021-04-03T03:12:31.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- 172.21.2.111:0/1140415886 wait complete. 2021-04-03T03:12:31.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 Processor -- start 2021-04-03T03:12:31.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- start start 2021-04-03T03:12:31.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb06c0603f0 0x7fb06c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb06c0add50 0x7fb06c147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c0a8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c064550 con 0x7fb06c144ef0 2021-04-03T03:12:31.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0647f0 con 0x7fb06c0add50 2021-04-03T03:12:31.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0a7eb0 con 0x7fb06c0603f0 2021-04-03T03:12:31.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c0a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c0a8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52578/0 (socket says 172.21.2.111:52578) 2021-04-03T03:12:31.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 -- 172.21.2.111:0/2623707390 learned_addr learned my addr 172.21.2.111:0/2623707390 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 -- 172.21.2.111:0/2623707390 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb06c0603f0 msgr2=0x7fb06c15a2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:12:31.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb06c0603f0 0x7fb06c15a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 -- 172.21.2.111:0/2623707390 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb06c0add50 msgr2=0x7fb06c147dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:12:31.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb06c0add50 0x7fb06c147dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 -- 172.21.2.111:0/2623707390 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c14a920 con 0x7fb06c144ef0 2021-04-03T03:12:31.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072602700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c0a8350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fb06c0b5600 tx=0x7fb064016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb05effd700 1 -- 172.21.2.111:0/2623707390 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06401f420 con 0x7fb06c144ef0 2021-04-03T03:12:31.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb05effd700 1 -- 172.21.2.111:0/2623707390 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06401f580 con 0x7fb06c144ef0 2021-04-03T03:12:31.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb05effd700 1 -- 172.21.2.111:0/2623707390 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06401f8e0 con 0x7fb06c144ef0 2021-04-03T03:12:31.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c0a9500 con 0x7fb06c144ef0 2021-04-03T03:12:31.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c0aaa30 con 0x7fb06c144ef0 2021-04-03T03:12:31.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06c057660 con 0x7fb06c144ef0 2021-04-03T03:12:31.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb05effd700 1 -- 172.21.2.111:0/2623707390 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb064026030 con 0x7fb06c144ef0 2021-04-03T03:12:31.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb05effd700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb04c056ef0 0x7fb04c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb05effd700 1 -- 172.21.2.111:0/2623707390 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb064070f40 con 0x7fb06c144ef0 2021-04-03T03:12:31.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.163+0000 7fb071e01700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb04c056ef0 0x7fb04c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.167+0000 7fb071e01700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb04c056ef0 0x7fb04c0593b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb06c0ae520 tx=0x7fb068007790).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.167+0000 7fb05effd700 1 -- 172.21.2.111:0/2623707390 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb064046220 con 0x7fb06c144ef0 2021-04-03T03:12:31.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb06c148850 con 0x7fb06c144ef0 2021-04-03T03:12:31.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb05effd700 1 -- 172.21.2.111:0/2623707390 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v54) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb0640150b0 con 0x7fb06c144ef0 2021-04-03T03:12:31.275 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:12:31.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb04c056ef0 msgr2=0x7fb04c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb04c056ef0 0x7fb04c0593b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb06c0ae520 tx=0x7fb068007790).stop 2021-04-03T03:12:31.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 msgr2=0x7fb06c0a8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c0a8350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fb06c0b5600 tx=0x7fb064016000).stop 2021-04-03T03:12:31.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 shutdown_connections 2021-04-03T03:12:31.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb06c144ef0 0x7fb06c0a8350 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb06c0603f0 0x7fb06c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb04c056ef0 0x7fb04c0593b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 --2- 172.21.2.111:0/2623707390 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb06c0add50 0x7fb06c147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 >> 172.21.2.111:0/2623707390 conn(0x7fb06c0b61c0 msgr2=0x7fb06c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 shutdown_connections 2021-04-03T03:12:31.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.259+0000 7fb072e03700 1 -- 172.21.2.111:0/2623707390 wait complete. 2021-04-03T03:12:31.283 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:12:31.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 Processor -- start 2021-04-03T03:12:31.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- start start 2021-04-03T03:12:31.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7d24041060 0x7f7d240dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d240dc560 con 0x7f7d240dbba0 2021-04-03T03:12:31.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d240dc800 con 0x7f7d2403d560 2021-04-03T03:12:31.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2403de60 con 0x7f7d24041060 2021-04-03T03:12:31.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d30e00700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7d240dbba0 0x7f7d2403fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53374/0 (socket says 172.21.2.111:53374) 2021-04-03T03:12:31.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d30e00700 1 -- 172.21.2.111:0/2361505831 learned_addr learned my addr 172.21.2.111:0/2361505831 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:31.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240490037 0 0) 0x7f7d240dc560 con 0x7f7d240dbba0 2021-04-03T03:12:31.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d10003240 con 0x7f7d240dbba0 2021-04-03T03:12:31.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2208036879 0 0) 0x7f7d10003240 con 0x7f7d240dbba0 2021-04-03T03:12:31.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 >> v2:172.21.2.115:3300/0 conn(0x7f7d24041060 msgr2=0x7f7d240dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 --2- 172.21.2.111:0/2361505831 >> v2:172.21.2.115:3300/0 conn(0x7f7d24041060 0x7f7d240dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 >> v1:172.21.2.115:6789/0 conn(0x7f7d2403d560 legacy=0x7f7d24041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2403da30 con 0x7f7d240dbba0 2021-04-03T03:12:31.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f7d180016d0 con 0x7f7d240dbba0 2021-04-03T03:12:31.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 >> v1:172.21.2.111:6789/0 conn(0x7f7d240dbba0 legacy=0x7f7d2403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d10003800 0x7f7d10003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d100044e0 0x7f7d10004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d10004e50 0x7f7d1000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2403de60 con 0x7f7d10004e50 2021-04-03T03:12:31.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10005300 con 0x7f7d10003800 2021-04-03T03:12:31.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1000bc20 con 0x7f7d100044e0 2021-04-03T03:12:31.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d10004e50 0x7f7d1000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d30e00700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d10003800 0x7f7d10003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 -- 172.21.2.111:0/2361505831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d100044e0 msgr2=0x7f7d10004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d100044e0 0x7f7d10004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 -- 172.21.2.111:0/2361505831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d10003800 msgr2=0x7f7d10003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d10003800 0x7f7d10003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 -- 172.21.2.111:0/2361505831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10003240 con 0x7f7d10004e50 2021-04-03T03:12:31.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 -- 172.21.2.111:0/2361505831 reap_dead start 2021-04-03T03:12:31.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31601700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d10004e50 0x7f7d1000b730 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f7d2c06b5d0 tx=0x7f7d2c05ad70).ready entity=mon.0 client_cookie=16359ddf5b49598a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2c05bda0 con 0x7f7d10004e50 2021-04-03T03:12:31.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c05df60 con 0x7f7d10004e50 2021-04-03T03:12:31.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d2b7fe700 1 -- 172.21.2.111:0/2361505831 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2c0699f0 con 0x7f7d10004e50 2021-04-03T03:12:31.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- 172.21.2.111:0/2361505831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d10004e50 msgr2=0x7f7d1000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d10004e50 0x7f7d1000b730 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f7d2c06b5d0 tx=0x7f7d2c05ad70).stop 2021-04-03T03:12:31.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- 172.21.2.111:0/2361505831 shutdown_connections 2021-04-03T03:12:31.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2361505831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d10004e50 0x7f7d1000b730 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- 172.21.2.111:0/2361505831 >> 172.21.2.111:0/2361505831 conn(0x7f7d240462b0 msgr2=0x7f7d240466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- 172.21.2.111:0/2361505831 shutdown_connections 2021-04-03T03:12:31.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.327+0000 7f7d31e02700 1 -- 172.21.2.111:0/2361505831 wait complete. 2021-04-03T03:12:31.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 Processor -- start 2021-04-03T03:12:31.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 -- start start 2021-04-03T03:12:31.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d10004e50 0x7f7d240a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d240ace10 0x7f7d2403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d240e6cb0 0x7f7d240e7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240a19e0 con 0x7f7d240ace10 2021-04-03T03:12:31.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2403c660 con 0x7f7d240e6cb0 2021-04-03T03:12:31.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240a1c10 con 0x7f7d10004e50 2021-04-03T03:12:31.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d240ace10 0x7f7d2403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d240ace10 0x7f7d2403ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52598/0 (socket says 172.21.2.111:52598) 2021-04-03T03:12:31.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 -- 172.21.2.111:0/2070442360 learned_addr learned my addr 172.21.2.111:0/2070442360 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 -- 172.21.2.111:0/2070442360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d10004e50 msgr2=0x7f7d240a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d10004e50 0x7f7d240a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 -- 172.21.2.111:0/2070442360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d240e6cb0 msgr2=0x7f7d240e7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d240e6cb0 0x7f7d240e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 -- 172.21.2.111:0/2070442360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d240a1eb0 con 0x7f7d240ace10 2021-04-03T03:12:31.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d2bfff700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d240ace10 0x7f7d2403ac90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f7d20002f30 tx=0x7f7d20006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d29ffb700 1 -- 172.21.2.111:0/2070442360 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2000d7c0 con 0x7f7d240ace10 2021-04-03T03:12:31.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d29ffb700 1 -- 172.21.2.111:0/2070442360 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20000e90 con 0x7f7d240ace10 2021-04-03T03:12:31.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d240a2350 con 0x7f7d240ace10 2021-04-03T03:12:31.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d29ffb700 1 -- 172.21.2.111:0/2070442360 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2000db10 con 0x7f7d240ace10 2021-04-03T03:12:31.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d240e7860 con 0x7f7d240ace10 2021-04-03T03:12:31.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d240a2040 con 0x7f7d240ace10 2021-04-03T03:12:31.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d29ffb700 1 -- 172.21.2.111:0/2070442360 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7d2000dc70 con 0x7f7d240ace10 2021-04-03T03:12:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d29ffb700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d14056fb0 0x7f7d14059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d29ffb700 1 -- 172.21.2.111:0/2070442360 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7d20070370 con 0x7f7d240ace10 2021-04-03T03:12:31.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d30e00700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d14056fb0 0x7f7d14059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.331+0000 7f7d30e00700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d14056fb0 0x7f7d14059470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f7d1800b000 tx=0x7f7d18000ed0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.335+0000 7f7d29ffb700 1 -- 172.21.2.111:0/2070442360 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7d20045650 con 0x7f7d240ace10 2021-04-03T03:12:31.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.427+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7d24004db0 con 0x7f7d240ace10 2021-04-03T03:12:31.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.427+0000 7f7d29ffb700 1 -- 172.21.2.111:0/2070442360 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7d20048c70 con 0x7f7d240ace10 2021-04-03T03:12:31.444 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:12:31.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d14056fb0 msgr2=0x7f7d14059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d14056fb0 0x7f7d14059470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f7d1800b000 tx=0x7f7d18000ed0).stop 2021-04-03T03:12:31.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d240ace10 msgr2=0x7f7d2403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d240ace10 0x7f7d2403ac90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f7d20002f30 tx=0x7f7d20006e20).stop 2021-04-03T03:12:31.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 shutdown_connections 2021-04-03T03:12:31.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d14056fb0 0x7f7d14059470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d240e6cb0 0x7f7d240e7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d10004e50 0x7f7d240a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 --2- 172.21.2.111:0/2070442360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d240ace10 0x7f7d2403ac90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 >> 172.21.2.111:0/2070442360 conn(0x7f7d240462b0 msgr2=0x7f7d2403f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 shutdown_connections 2021-04-03T03:12:31.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.431+0000 7f7d31e02700 1 -- 172.21.2.111:0/2070442360 wait complete. 2021-04-03T03:12:31.451 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:12:31.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 Processor -- start 2021-04-03T03:12:31.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- start start 2021-04-03T03:12:31.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7e5002cbe0 0x7f7e500469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500d7a90 0x7f7e500d7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7e500059b0 0x7f7e50005e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e50044e00 con 0x7f7e5002cbe0 2021-04-03T03:12:31.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5002d8c0 con 0x7f7e500059b0 2021-04-03T03:12:31.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5002cfc0 con 0x7f7e500d7a90 2021-04-03T03:12:31.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7e5002cbe0 0x7f7e500469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7e5002cbe0 0x7f7e500469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52604/0 (socket says 172.21.2.111:52604) 2021-04-03T03:12:31.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 -- 172.21.2.111:0/1293623586 learned_addr learned my addr 172.21.2.111:0/1293623586 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 -- 172.21.2.111:0/1293623586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500d7a90 msgr2=0x7f7e500d7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 --2- 172.21.2.111:0/1293623586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500d7a90 0x7f7e500d7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 -- 172.21.2.111:0/1293623586 >> v2:172.21.2.115:3300/0 conn(0x7f7e500059b0 msgr2=0x7f7e50005e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:12:31.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 --2- 172.21.2.111:0/1293623586 >> v2:172.21.2.115:3300/0 conn(0x7f7e500059b0 0x7f7e50005e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 -- 172.21.2.111:0/1293623586 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e50046e90 con 0x7f7e5002cbe0 2021-04-03T03:12:31.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5ce94700 1 --2- 172.21.2.111:0/1293623586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e500469a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f7e48002f30 tx=0x7f7e48006e90).ready entity=mon.0 client_cookie=47c97ade5a1e6e67 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e577fe700 1 -- 172.21.2.111:0/1293623586 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e48009920 con 0x7f7e5002cbe0 2021-04-03T03:12:31.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e577fe700 1 -- 172.21.2.111:0/1293623586 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e48009a80 con 0x7f7e5002cbe0 2021-04-03T03:12:31.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e577fe700 1 -- 172.21.2.111:0/1293623586 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e48009de0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- 172.21.2.111:0/1293623586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 msgr2=0x7f7e500469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 --2- 172.21.2.111:0/1293623586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e500469a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f7e48002f30 tx=0x7f7e48006e90).stop 2021-04-03T03:12:31.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- 172.21.2.111:0/1293623586 shutdown_connections 2021-04-03T03:12:31.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 --2- 172.21.2.111:0/1293623586 >> v2:172.21.2.115:3300/0 conn(0x7f7e500059b0 0x7f7e50005e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 --2- 172.21.2.111:0/1293623586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e500469a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 --2- 172.21.2.111:0/1293623586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500d7a90 0x7f7e500d7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- 172.21.2.111:0/1293623586 >> 172.21.2.111:0/1293623586 conn(0x7f7e50047160 msgr2=0x7f7e50047570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- 172.21.2.111:0/1293623586 shutdown_connections 2021-04-03T03:12:31.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.499+0000 7f7e5de96700 1 -- 172.21.2.111:0/1293623586 wait complete. 2021-04-03T03:12:31.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 Processor -- start 2021-04-03T03:12:31.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 -- start start 2021-04-03T03:12:31.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500059b0 0x7f7e50020540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e50041ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e500d7a90 0x7f7e50043090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e500d5700 con 0x7f7e5002cbe0 2021-04-03T03:12:31.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e50021860 con 0x7f7e500d7a90 2021-04-03T03:12:31.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5001f580 con 0x7f7e500059b0 2021-04-03T03:12:31.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e50041ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e50041ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52610/0 (socket says 172.21.2.111:52610) 2021-04-03T03:12:31.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 -- 172.21.2.111:0/3743104705 learned_addr learned my addr 172.21.2.111:0/3743104705 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 -- 172.21.2.111:0/3743104705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500059b0 msgr2=0x7f7e50020540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5d695700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e500d7a90 0x7f7e50043090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500059b0 0x7f7e50020540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 -- 172.21.2.111:0/3743104705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e500d7a90 msgr2=0x7f7e50043090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e500d7a90 0x7f7e50043090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 -- 172.21.2.111:0/3743104705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5001f820 con 0x7f7e5002cbe0 2021-04-03T03:12:31.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e57fff700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e50041ac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f7e4c002f30 tx=0x7f7e4c006d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e55ffb700 1 -- 172.21.2.111:0/3743104705 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e4c00d4a0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e55ffb700 1 -- 172.21.2.111:0/3743104705 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c000e90 con 0x7f7e5002cbe0 2021-04-03T03:12:31.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e500443a0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e55ffb700 1 -- 172.21.2.111:0/3743104705 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e4c00d7f0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e50021630 con 0x7f7e5002cbe0 2021-04-03T03:12:31.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e50004db0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e55ffb700 1 -- 172.21.2.111:0/3743104705 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7e4c00f030 con 0x7f7e5002cbe0 2021-04-03T03:12:31.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e55ffb700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e38056ea0 0x7f7e38059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e55ffb700 1 -- 172.21.2.111:0/3743104705 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7e4c06fde0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5ce94700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e38056ea0 0x7f7e38059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.503+0000 7f7e5ce94700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e38056ea0 0x7f7e38059360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f7e48002f30 tx=0x7f7e48015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.507+0000 7f7e55ffb700 1 -- 172.21.2.111:0/3743104705 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7e4c0451d0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.595+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e500083e0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e55ffb700 1 -- 172.21.2.111:0/3743104705 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7e4c0487f0 con 0x7f7e5002cbe0 2021-04-03T03:12:31.612 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:12:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e38056ea0 msgr2=0x7f7e38059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e38056ea0 0x7f7e38059360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f7e48002f30 tx=0x7f7e48015000).stop 2021-04-03T03:12:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 msgr2=0x7f7e50041ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e50041ac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f7e4c002f30 tx=0x7f7e4c006d80).stop 2021-04-03T03:12:31.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 shutdown_connections 2021-04-03T03:12:31.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e38056ea0 0x7f7e38059360 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e500d7a90 0x7f7e50043090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e500059b0 0x7f7e50020540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 --2- 172.21.2.111:0/3743104705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e5002cbe0 0x7f7e50041ac0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 >> 172.21.2.111:0/3743104705 conn(0x7f7e50047160 msgr2=0x7f7e500d60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 shutdown_connections 2021-04-03T03:12:31.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.599+0000 7f7e5de96700 1 -- 172.21.2.111:0/3743104705 wait complete. 2021-04-03T03:12:31.619 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:12:31.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 Processor -- start 2021-04-03T03:12:31.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- start start 2021-04-03T03:12:31.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa00c0603f0 0x7fa00c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00c0607d0 con 0x7fa00c0b17b0 2021-04-03T03:12:31.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00c14b090 con 0x7fa00c0add50 2021-04-03T03:12:31.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c14b330 con 0x7fa00c0603f0 2021-04-03T03:12:31.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa010a4d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa00c0603f0 0x7fa00c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa01124e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa00c0b17b0 0x7fa00c147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53410/0 (socket says 172.21.2.111:53410) 2021-04-03T03:12:31.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa01124e700 1 -- 172.21.2.111:0/4035945120 learned_addr learned my addr 172.21.2.111:0/4035945120 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:31.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa010a4d700 1 -- 172.21.2.111:0/4035945120 >> v1:172.21.2.115:6789/0 conn(0x7fa00c0add50 legacy=0x7fa00c147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:12:31.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa00b7fe700 1 -- 172.21.2.111:0/4035945120 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635159185 0 0) 0x7fa00c0607d0 con 0x7fa00c0b17b0 2021-04-03T03:12:31.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa010a4d700 1 -- 172.21.2.111:0/4035945120 >> v1:172.21.2.111:6789/0 conn(0x7fa00c0b17b0 legacy=0x7fa00c147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa010a4d700 1 -- 172.21.2.111:0/4035945120 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c14b5d0 con 0x7fa00c0603f0 2021-04-03T03:12:31.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa010a4d700 1 --2- 172.21.2.111:0/4035945120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1447e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fa00c0b5600 tx=0x7f9ffc006dd0).ready entity=mon.0 client_cookie=5bca5d91966226f5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa00b7fe700 1 -- 172.21.2.111:0/4035945120 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ffc0097b0 con 0x7fa00c0603f0 2021-04-03T03:12:31.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa00b7fe700 1 -- 172.21.2.111:0/4035945120 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc009910 con 0x7fa00c0603f0 2021-04-03T03:12:31.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa00b7fe700 1 -- 172.21.2.111:0/4035945120 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ffc009c70 con 0x7fa00c0603f0 2021-04-03T03:12:31.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- 172.21.2.111:0/4035945120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 msgr2=0x7fa00c1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 --2- 172.21.2.111:0/4035945120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1447e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fa00c0b5600 tx=0x7f9ffc006dd0).stop 2021-04-03T03:12:31.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- 172.21.2.111:0/4035945120 shutdown_connections 2021-04-03T03:12:31.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 --2- 172.21.2.111:0/4035945120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1447e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- 172.21.2.111:0/4035945120 >> 172.21.2.111:0/4035945120 conn(0x7fa00c0b61c0 msgr2=0x7fa00c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- 172.21.2.111:0/4035945120 shutdown_connections 2021-04-03T03:12:31.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- 172.21.2.111:0/4035945120 wait complete. 2021-04-03T03:12:31.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 Processor -- start 2021-04-03T03:12:31.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 -- start start 2021-04-03T03:12:31.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.663+0000 7fa011a4f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa00c0add50 0x7fa00c0a9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa00c0b17b0 0x7fa00c0aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c156dd0 con 0x7fa00c0603f0 2021-04-03T03:12:31.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c157070 con 0x7fa00c0b17b0 2021-04-03T03:12:31.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c157310 con 0x7fa00c0add50 2021-04-03T03:12:31.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52620/0 (socket says 172.21.2.111:52620) 2021-04-03T03:12:31.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 -- 172.21.2.111:0/148361453 learned_addr learned my addr 172.21.2.111:0/148361453 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 -- 172.21.2.111:0/148361453 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa00c0add50 msgr2=0x7fa00c0a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa00c0add50 0x7fa00c0a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 -- 172.21.2.111:0/148361453 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa00c0b17b0 msgr2=0x7fa00c0aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa00c0b17b0 0x7fa00c0aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 -- 172.21.2.111:0/148361453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0a8d30 con 0x7fa00c0603f0 2021-04-03T03:12:31.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa010a4d700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1455c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fa00c0b5600 tx=0x7f9ffc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa009ffb700 1 -- 172.21.2.111:0/148361453 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ffc011db0 con 0x7fa00c0603f0 2021-04-03T03:12:31.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa009ffb700 1 -- 172.21.2.111:0/148361453 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc01e460 con 0x7fa00c0603f0 2021-04-03T03:12:31.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c1577b0 con 0x7fa00c0603f0 2021-04-03T03:12:31.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa009ffb700 1 -- 172.21.2.111:0/148361453 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ffc01e7c0 con 0x7fa00c0603f0 2021-04-03T03:12:31.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c0a9d00 con 0x7fa00c0603f0 2021-04-03T03:12:31.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c1574d0 con 0x7fa00c0603f0 2021-04-03T03:12:31.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa009ffb700 1 -- 172.21.2.111:0/148361453 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9ffc025030 con 0x7fa00c0603f0 2021-04-03T03:12:31.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa009ffb700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ff0056ea0 0x7f9ff0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa009ffb700 1 -- 172.21.2.111:0/148361453 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9ffc071750 con 0x7fa00c0603f0 2021-04-03T03:12:31.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa00bfff700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ff0056ea0 0x7f9ff0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.667+0000 7fa00bfff700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ff0056ea0 0x7f9ff0059360 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f9ff4000f30 tx=0x7f9ff40077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.671+0000 7fa009ffb700 1 -- 172.21.2.111:0/148361453 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9ffc046a30 con 0x7fa00c0603f0 2021-04-03T03:12:31.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa00c057660 con 0x7fa00c0603f0 2021-04-03T03:12:31.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa009ffb700 1 -- 172.21.2.111:0/148361453 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9ffc0230a0 con 0x7fa00c0603f0 2021-04-03T03:12:31.778 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:12:31.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ff0056ea0 msgr2=0x7f9ff0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ff0056ea0 0x7f9ff0059360 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f9ff4000f30 tx=0x7f9ff40077d0).stop 2021-04-03T03:12:31.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 msgr2=0x7fa00c1455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1455c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fa00c0b5600 tx=0x7f9ffc015000).stop 2021-04-03T03:12:31.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 shutdown_connections 2021-04-03T03:12:31.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9ff0056ea0 0x7f9ff0059360 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa00c0b17b0 0x7fa00c0aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa00c0603f0 0x7fa00c1455c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 --2- 172.21.2.111:0/148361453 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa00c0add50 0x7fa00c0a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.763+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 >> 172.21.2.111:0/148361453 conn(0x7fa00c0b61c0 msgr2=0x7fa00c0b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.767+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 shutdown_connections 2021-04-03T03:12:31.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.767+0000 7fa011a4f700 1 -- 172.21.2.111:0/148361453 wait complete. 2021-04-03T03:12:31.786 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:12:31.787 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:12:31.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.827+0000 7fa3fd982700 1 Processor -- start 2021-04-03T03:12:31.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.827+0000 7fa3fd982700 1 -- start start 2021-04-03T03:12:31.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3f80603f0 0x7fa3f80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80af400 con 0x7fa3f814b700 2021-04-03T03:12:31.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80af6a0 con 0x7fa3f80b0c90 2021-04-03T03:12:31.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f8060860 con 0x7fa3f80603f0 2021-04-03T03:12:31.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fc980700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3f80603f0 0x7fa3f80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa3f814b700 0x7fa3f80b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53422/0 (socket says 172.21.2.111:53422) 2021-04-03T03:12:31.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 -- 172.21.2.111:0/409253229 learned_addr learned my addr 172.21.2.111:0/409253229 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:12:31.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fc980700 1 -- 172.21.2.111:0/409253229 >> v1:172.21.2.115:6789/0 conn(0x7fa3f80b0c90 legacy=0x7fa3f8147680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:12:31.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fc980700 1 -- 172.21.2.111:0/409253229 >> v1:172.21.2.111:6789/0 conn(0x7fa3f814b700 legacy=0x7fa3f80b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f77fe700 1 -- 172.21.2.111:0/409253229 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708516372 0 0) 0x7fa3f80af400 con 0x7fa3f814b700 2021-04-03T03:12:31.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fc980700 1 -- 172.21.2.111:0/409253229 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80afb40 con 0x7fa3f80603f0 2021-04-03T03:12:31.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fc980700 1 --2- 172.21.2.111:0/409253229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80603f0 0x7fa3f80b0090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fa3f80620f0 tx=0x7fa3e0006dd0).ready entity=mon.0 client_cookie=b091b88e1b92c55a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f77fe700 1 -- 172.21.2.111:0/409253229 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e00097b0 con 0x7fa3f80603f0 2021-04-03T03:12:31.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f77fe700 1 -- 172.21.2.111:0/409253229 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0009910 con 0x7fa3f80603f0 2021-04-03T03:12:31.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f77fe700 1 -- 172.21.2.111:0/409253229 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e0009c70 con 0x7fa3f80603f0 2021-04-03T03:12:31.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- 172.21.2.111:0/409253229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80603f0 msgr2=0x7fa3f80b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 --2- 172.21.2.111:0/409253229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80603f0 0x7fa3f80b0090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fa3f80620f0 tx=0x7fa3e0006dd0).stop 2021-04-03T03:12:31.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- 172.21.2.111:0/409253229 shutdown_connections 2021-04-03T03:12:31.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 --2- 172.21.2.111:0/409253229 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80603f0 0x7fa3f80b0090 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- 172.21.2.111:0/409253229 >> 172.21.2.111:0/409253229 conn(0x7fa3f80b61c0 msgr2=0x7fa3f805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- 172.21.2.111:0/409253229 shutdown_connections 2021-04-03T03:12:31.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- 172.21.2.111:0/409253229 wait complete. 2021-04-03T03:12:31.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 Processor -- start 2021-04-03T03:12:31.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- start start 2021-04-03T03:12:31.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3f80603f0 0x7fa3f814a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80b0c90 0x7fa3f8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3f814b700 0x7fa3f80645b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80ab860 con 0x7fa3f80b0c90 2021-04-03T03:12:31.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f8159f60 con 0x7fa3f814b700 2021-04-03T03:12:31.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f8062e50 con 0x7fa3f80603f0 2021-04-03T03:12:31.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80b0c90 0x7fa3f8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80b0c90 0x7fa3f8159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52636/0 (socket says 172.21.2.111:52636) 2021-04-03T03:12:31.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 -- 172.21.2.111:0/2581914003 learned_addr learned my addr 172.21.2.111:0/2581914003 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:12:31.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 -- 172.21.2.111:0/2581914003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3f80603f0 msgr2=0x7fa3f814a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3f80603f0 0x7fa3f814a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 -- 172.21.2.111:0/2581914003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3f814b700 msgr2=0x7fa3f80645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3f814b700 0x7fa3f80645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 -- 172.21.2.111:0/2581914003 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f8064110 con 0x7fa3f80b0c90 2021-04-03T03:12:31.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f7fff700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80b0c90 0x7fa3f8159a70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fa3e8000f00 tx=0x7fa3e8007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f5ffb700 1 -- 172.21.2.111:0/2581914003 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e8005b80 con 0x7fa3f80b0c90 2021-04-03T03:12:31.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f5ffb700 1 -- 172.21.2.111:0/2581914003 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8005ce0 con 0x7fa3f80b0c90 2021-04-03T03:12:31.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f80630f0 con 0x7fa3f80b0c90 2021-04-03T03:12:31.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3f5ffb700 1 -- 172.21.2.111:0/2581914003 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e8007300 con 0x7fa3f80b0c90 2021-04-03T03:12:31.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.831+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f80ab020 con 0x7fa3f80b0c90 2021-04-03T03:12:31.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.835+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f80642a0 con 0x7fa3f80b0c90 2021-04-03T03:12:31.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.835+0000 7fa3f5ffb700 1 -- 172.21.2.111:0/2581914003 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa3e8010030 con 0x7fa3f80b0c90 2021-04-03T03:12:31.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.835+0000 7fa3f5ffb700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3dc056fb0 0x7fa3dc059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:12:31.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.835+0000 7fa3f5ffb700 1 -- 172.21.2.111:0/2581914003 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa3e80701c0 con 0x7fa3f80b0c90 2021-04-03T03:12:31.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.835+0000 7fa3fc980700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3dc056fb0 0x7fa3dc059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:12:31.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.835+0000 7fa3fc980700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3dc056fb0 0x7fa3dc059470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa3f80620f0 tx=0x7fa3e0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:12:31.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.835+0000 7fa3f5ffb700 1 -- 172.21.2.111:0/2581914003 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa3e80455b0 con 0x7fa3f80b0c90 2021-04-03T03:12:31.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3f8057660 con 0x7fa3f80b0c90 2021-04-03T03:12:31.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3f5ffb700 1 -- 172.21.2.111:0/2581914003 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7fa3e8048bd0 con 0x7fa3f80b0c90 2021-04-03T03:12:31.977 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:12:31.977 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:12:31.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3dc056fb0 msgr2=0x7fa3dc059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3dc056fb0 0x7fa3dc059470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa3f80620f0 tx=0x7fa3e0015000).stop 2021-04-03T03:12:31.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80b0c90 msgr2=0x7fa3f8159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:12:31.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80b0c90 0x7fa3f8159a70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fa3e8000f00 tx=0x7fa3e8007f90).stop 2021-04-03T03:12:31.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 shutdown_connections 2021-04-03T03:12:31.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3dc056fb0 0x7fa3dc059470 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3f814b700 0x7fa3f80645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3f80603f0 0x7fa3f814a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:12:31.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 --2- 172.21.2.111:0/2581914003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3f80b0c90 0x7fa3f8159a70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:12:31.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 >> 172.21.2.111:0/2581914003 conn(0x7fa3f80b61c0 msgr2=0x7fa3f814ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:12:31.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 shutdown_connections 2021-04-03T03:12:31.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:12:31.963+0000 7fa3fd982700 1 -- 172.21.2.111:0/2581914003 wait complete. 2021-04-03T03:12:31.982 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:12:32.186 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:12:04.020364+0000', 'modified': '2021-04-03T03:12:05.022670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5580': {'gid': 5580, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/2278897120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 2278897120}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 2278897120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:12:32.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:12:32.469 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:12:32.540 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5580,"want_state":"up:active","state":"up:active","rank_uptime":28.503772811000001,"mdsmap_epoch":21,"osdmap_epoch":54,"osdmap_epoch_barrier":52,"uptime":32.640278791999997} 2021-04-03T03:12:32.541 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5580, "mdsmap_epoch": 21, "osdmap_epoch": 54, "osdmap_epoch_barrier": 52, "rank_uptime": 28.503772811, "state": "up:active", "uptime": 32.640278792, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:12:32.541 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:12:32.542 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:12:32.542 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:12:32.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:32.543 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:32.553 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:12:32.605 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:12:32.605 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:12:32.653 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:12:32.654 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:12:32.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:32.655 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:12:32.707 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:12:32.707 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:32.753 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:12:32.760 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:12:32.760 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:12:32.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:12:32.779+0000 7fe676b610c0 -1 init, newargv = 0x55a153e9c2b0 newargc=9 2021-04-03T03:12:32.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15281]: starting ceph client 2021-04-03T03:12:32.796 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:12:32.796 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:12:32.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:32.798 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:12:32.801 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15281]: starting fuse 2021-04-03T03:12:32.803 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:12:32.804 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:12:32.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:32.805 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:12:32.805 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:12:32.805 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:12:32.806 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:12:32.806 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:12:32.807 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:32.807 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:12:32.808 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:12:32.808 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:12:32.808 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:32.809 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:12:32.809 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:12:32.810 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:12:32.810 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:12:32.810 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:32.811 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:12:32.811 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:12:32.812 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:12:32.812 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:12:32.812 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:12:32.813 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:12:32.813 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:12:32.814 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:12:32.898 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15281.asok 2021-04-03T03:12:32.902 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15281.asok 2021-04-03T03:12:32.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:32.903 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15281.asok status 2021-04-03T03:12:32.973 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:12:32.974 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:12:32.974 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:12:32.974 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:12:32.974 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:12:32.975 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:12:32.975 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:12:32.975 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15281", 2021-04-03T03:12:32.975 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:12:32.976 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:32.976 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:12:32.976 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:12:32.976 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4765, 2021-04-03T03:12:32.977 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:12:32.977 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:12:32.977 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:12:32.977 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4765 2021-04-03T03:12:32.977 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:32.978 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:12:32.978 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:12:32.978 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:12:32.978 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3790590227 2021-04-03T03:12:32.979 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:12:32.979 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:32.979 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:12:32.979 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:12:32.980 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:12:32.980 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3790590227 2021-04-03T03:12:32.980 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:12:32.980 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4765 172.21.2.126:0/3790590227", 2021-04-03T03:12:32.981 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/3790590227", 2021-04-03T03:12:32.981 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:12:32.981 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 21, 2021-04-03T03:12:32.981 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 54, 2021-04-03T03:12:32.981 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:12:32.982 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:12:32.982 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:12:32.982 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:12:32.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:32.984 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:12:32.984 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:32.990 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:12:32.990 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:12:32.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:12:32.991 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:12:58.879 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:58.882 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:58.885 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:58.888 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:12:58.891 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:28.927 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:28.929 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:28.932 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:28.935 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:28.938 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:29.907 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:13:29.908 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:29.920 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:29.927 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:29.938 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:29.951 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:29.963 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:13:29.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:29.964 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:29.964 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:29.984 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:13:29.984 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:13:29.985 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:13:29.985 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:13:29.986 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:30.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15281]: fuse finished with error 0 and tester_r 0 2021-04-03T03:13:30.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:30.070 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:30.070 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:30.119 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:13:30.120 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:13:30.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:13:30.154 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:30.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:30.200 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:30.200 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:13:30.210 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:13:30.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:13:30.211 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:13:30.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:13:30.212 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:13:30.261 INFO:teuthology.orchestra.run.gibba028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:13:30.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:13:30.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:30.264 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:30.264 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:13:30.273 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:13:30.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:13:30.275 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:13:30.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:13:30.276 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:13:30.323 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:13:30.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:13:30.325 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:13:30.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1876700 1 Processor -- start 2021-04-03T03:13:30.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1876700 1 -- start start 2021-04-03T03:13:30.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1876700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f71dc0add50 0x7f71dc147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1876700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc0607d0 con 0x7f71dc0603f0 2021-04-03T03:13:30.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1876700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc14b090 con 0x7f71dc144ef0 2021-04-03T03:13:30.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1876700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc14b330 con 0x7f71dc0add50 2021-04-03T03:13:30.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f71dc0603f0 0x7f71dc1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53434/0 (socket says 172.21.2.111:53434) 2021-04-03T03:13:30.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 -- 172.21.2.111:0/3815100039 learned_addr learned my addr 172.21.2.111:0/3815100039 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:30.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015047128 0 0) 0x7f71dc0607d0 con 0x7f71dc0603f0 2021-04-03T03:13:30.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c4003240 con 0x7f71dc0603f0 2021-04-03T03:13:30.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2553723855 0 0) 0x7f71c4003240 con 0x7f71dc0603f0 2021-04-03T03:13:30.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 >> v2:172.21.2.115:3300/0 conn(0x7f71dc0add50 msgr2=0x7f71dc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:30.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71dbfff700 1 --2- 172.21.2.111:0/3815100039 >> v2:172.21.2.115:3300/0 conn(0x7f71dc0add50 0x7f71dc147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 --2- 172.21.2.111:0/3815100039 >> v2:172.21.2.115:3300/0 conn(0x7f71dc0add50 0x7f71dc147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 >> v1:172.21.2.115:6789/0 conn(0x7f71dc144ef0 legacy=0x7f71dc147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:30.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc14b5d0 con 0x7f71dc0603f0 2021-04-03T03:13:30.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f71cc0016d0 con 0x7f71dc0603f0 2021-04-03T03:13:30.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 >> v1:172.21.2.111:6789/0 conn(0x7f71dc0603f0 legacy=0x7f71dc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71c4003800 0x7f71c4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c40044e0 0x7f71c4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71c4004e50 0x7f71c400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc14b330 con 0x7f71c4003800 2021-04-03T03:13:30.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4005300 con 0x7f71c40044e0 2021-04-03T03:13:30.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c400bc20 con 0x7f71c4004e50 2021-04-03T03:13:30.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71dbfff700 1 --2- 172.21.2.111:0/3815100039 >> v2:172.21.2.115:3300/0 conn(0x7f71dc0add50 0x7f71dc147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:13:30.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71c4003800 0x7f71c4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1075700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71c4004e50 0x7f71c400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 -- 172.21.2.111:0/3815100039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71c4004e50 msgr2=0x7f71c400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:30.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71c4004e50 0x7f71c400b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 -- 172.21.2.111:0/3815100039 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c40044e0 msgr2=0x7f71c4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:30.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c40044e0 0x7f71c4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:30.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e0874700 1 -- 172.21.2.111:0/3815100039 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c4003240 con 0x7f71c4003800 2021-04-03T03:13:30.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.371+0000 7f71e1075700 1 -- 172.21.2.111:0/3815100039 reap_dead start 2021-04-03T03:13:30.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e0874700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71c4003800 0x7f71c4003c00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f71dc0b5600 tx=0x7f71dc14b220).ready entity=mon.0 client_cookie=816888c2e6356001 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:30.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71cc0070d0 con 0x7f71c4003800 2021-04-03T03:13:30.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc007230 con 0x7f71c4003800 2021-04-03T03:13:30.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71db7fe700 1 -- 172.21.2.111:0/3815100039 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71cc0055f0 con 0x7f71c4003800 2021-04-03T03:13:30.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3815100039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71c4003800 msgr2=0x7f71c4003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71c4003800 0x7f71c4003c00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f71dc0b5600 tx=0x7f71dc14b220).stop 2021-04-03T03:13:30.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3815100039 shutdown_connections 2021-04-03T03:13:30.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 --2- 172.21.2.111:0/3815100039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71c4003800 0x7f71c4003c00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3815100039 >> 172.21.2.111:0/3815100039 conn(0x7f71dc0b61c0 msgr2=0x7f71dc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:30.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3815100039 shutdown_connections 2021-04-03T03:13:30.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3815100039 wait complete. 2021-04-03T03:13:30.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 Processor -- start 2021-04-03T03:13:30.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- start start 2021-04-03T03:13:30.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c4003800 0x7f71dc146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71dc062a50 0x7f71dc140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71dc062ed0 0x7f71dc1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0a9200 con 0x7f71dc062ed0 2021-04-03T03:13:30.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc14aed0 con 0x7f71c4003800 2021-04-03T03:13:30.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0aa5b0 con 0x7f71dc062a50 2021-04-03T03:13:30.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71dc062ed0 0x7f71dc1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71dc062ed0 0x7f71dc1410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52658/0 (socket says 172.21.2.111:52658) 2021-04-03T03:13:30.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 -- 172.21.2.111:0/3661058587 learned_addr learned my addr 172.21.2.111:0/3661058587 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:30.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71dbfff700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71dc062a50 0x7f71dc140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 -- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71dc062a50 msgr2=0x7f71dc140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71dc062a50 0x7f71dc140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e0874700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c4003800 0x7f71dc146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 -- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c4003800 msgr2=0x7f71dc146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c4003800 0x7f71dc146940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 -- 172.21.2.111:0/3661058587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc0ab8b0 con 0x7f71dc062ed0 2021-04-03T03:13:30.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1075700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71dc062ed0 0x7f71dc1410b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f71cc0013e0 tx=0x7f71d4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:30.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71d9ffb700 1 -- 172.21.2.111:0/3661058587 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d4006a80 con 0x7f71dc062ed0 2021-04-03T03:13:30.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71d9ffb700 1 -- 172.21.2.111:0/3661058587 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4006be0 con 0x7f71dc062ed0 2021-04-03T03:13:30.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc0a9430 con 0x7f71dc062ed0 2021-04-03T03:13:30.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71d9ffb700 1 -- 172.21.2.111:0/3661058587 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d4007030 con 0x7f71dc062ed0 2021-04-03T03:13:30.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc147600 con 0x7f71dc062ed0 2021-04-03T03:13:30.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc057660 con 0x7f71dc062ed0 2021-04-03T03:13:30.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71d9ffb700 1 -- 172.21.2.111:0/3661058587 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f71d4007190 con 0x7f71dc062ed0 2021-04-03T03:13:30.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71d9ffb700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71b8056ea0 0x7f71b8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.375+0000 7f71d9ffb700 1 -- 172.21.2.111:0/3661058587 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f71d406fd90 con 0x7f71dc062ed0 2021-04-03T03:13:30.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.379+0000 7f71e0874700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71b8056ea0 0x7f71b8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.379+0000 7f71e0874700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71b8056ea0 0x7f71b8059360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f71cc014000 tx=0x7f71cc013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:30.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.379+0000 7f71d9ffb700 1 -- 172.21.2.111:0/3661058587 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f71d4042c50 con 0x7f71dc062ed0 2021-04-03T03:13:30.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.471+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f71dc148850 con 0x7f71dc062ed0 2021-04-03T03:13:30.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.475+0000 7f71d9ffb700 1 -- 172.21.2.111:0/3661058587 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71d4044ff0 con 0x7f71dc062ed0 2021-04-03T03:13:30.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.475+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71b8056ea0 msgr2=0x7f71b8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.475+0000 7f71e1876700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71b8056ea0 0x7f71b8059360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f71cc014000 tx=0x7f71cc013000).stop 2021-04-03T03:13:30.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71dc062ed0 msgr2=0x7f71dc1410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71dc062ed0 0x7f71dc1410b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f71cc0013e0 tx=0x7f71d4009040).stop 2021-04-03T03:13:30.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 shutdown_connections 2021-04-03T03:13:30.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71dc062ed0 0x7f71dc1410b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71b8056ea0 0x7f71b8059360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71c4003800 0x7f71dc146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 --2- 172.21.2.111:0/3661058587 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71dc062a50 0x7f71dc140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 >> 172.21.2.111:0/3661058587 conn(0x7f71dc0b61c0 msgr2=0x7f71dc1423f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 shutdown_connections 2021-04-03T03:13:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.479+0000 7f71e1876700 1 -- 172.21.2.111:0/3661058587 wait complete. 2021-04-03T03:13:30.500 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2021-04-03T03:13:30.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.539+0000 7f5d30c3c700 1 Processor -- start 2021-04-03T03:13:30.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.539+0000 7f5d30c3c700 1 -- start start 2021-04-03T03:13:30.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 0x7f5d24042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5d2403d360 0x7f5d240dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d240dc6e0 con 0x7f5d240064b0 2021-04-03T03:13:30.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d240d9560 con 0x7f5d2403d360 2021-04-03T03:13:30.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2403dcb0 con 0x7f5d240dbb90 2021-04-03T03:13:30.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2bfff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5d240064b0 0x7f5d240056b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53458/0 (socket says 172.21.2.111:53458) 2021-04-03T03:13:30.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2bfff700 1 -- 172.21.2.111:0/428701764 learned_addr learned my addr 172.21.2.111:0/428701764 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:30.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- 172.21.2.111:0/428701764 >> v2:172.21.2.111:3300/0 conn(0x7f5d2403d360 0x7f5d240dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 -- 172.21.2.111:0/428701764 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 msgr2=0x7f5d24042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- 172.21.2.111:0/428701764 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 0x7f5d24042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 -- 172.21.2.111:0/428701764 >> v1:172.21.2.111:6789/0 conn(0x7f5d240064b0 legacy=0x7f5d240056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 -- 172.21.2.111:0/428701764 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d240dc460 con 0x7f5d2403d360 2021-04-03T03:13:30.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- 172.21.2.111:0/428701764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240dbf70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f5d20002f30 tx=0x7f5d20006dd0).ready entity=mon.0 client_cookie=3bd229aa56130b72 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:30.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2a7fc700 1 -- 172.21.2.111:0/428701764 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d200097f0 con 0x7f5d2403d360 2021-04-03T03:13:30.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2a7fc700 1 -- 172.21.2.111:0/428701764 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d20009950 con 0x7f5d2403d360 2021-04-03T03:13:30.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2a7fc700 1 -- 172.21.2.111:0/428701764 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d20009cb0 con 0x7f5d2403d360 2021-04-03T03:13:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/428701764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 msgr2=0x7f5d240dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/428701764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240dbf70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f5d20002f30 tx=0x7f5d20006dd0).stop 2021-04-03T03:13:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/428701764 shutdown_connections 2021-04-03T03:13:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/428701764 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 0x7f5d24042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/428701764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240dbf70 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/428701764 >> 172.21.2.111:0/428701764 conn(0x7f5d240462c0 msgr2=0x7f5d240466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/428701764 shutdown_connections 2021-04-03T03:13:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/428701764 wait complete. 2021-04-03T03:13:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 Processor -- start 2021-04-03T03:13:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- start start 2021-04-03T03:13:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d240064b0 0x7f5d2403c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 0x7f5d240d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240db0d0 con 0x7f5d2403d360 2021-04-03T03:13:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24039a70 con 0x7f5d240064b0 2021-04-03T03:13:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240dae30 con 0x7f5d240dbb90 2021-04-03T03:13:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240d57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52668/0 (socket says 172.21.2.111:52668) 2021-04-03T03:13:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 -- 172.21.2.111:0/2150894832 learned_addr learned my addr 172.21.2.111:0/2150894832 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 -- 172.21.2.111:0/2150894832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 msgr2=0x7f5d240d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 0x7f5d240d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 -- 172.21.2.111:0/2150894832 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d240064b0 msgr2=0x7f5d2403c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d240064b0 0x7f5d2403c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 -- 172.21.2.111:0/2150894832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d240e7400 con 0x7f5d2403d360 2021-04-03T03:13:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d2affd700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240d57c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f5d2403b890 tx=0x7f5d20015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d28ff9700 1 -- 172.21.2.111:0/2150894832 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d20011d30 con 0x7f5d2403d360 2021-04-03T03:13:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d28ff9700 1 -- 172.21.2.111:0/2150894832 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d20011e90 con 0x7f5d2403d360 2021-04-03T03:13:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d240e8460 con 0x7f5d2403d360 2021-04-03T03:13:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2403c950 con 0x7f5d2403d360 2021-04-03T03:13:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d28ff9700 1 -- 172.21.2.111:0/2150894832 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d2001e650 con 0x7f5d2403d360 2021-04-03T03:13:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.543+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d240e7230 con 0x7f5d2403d360 2021-04-03T03:13:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.547+0000 7f5d28ff9700 1 -- 172.21.2.111:0/2150894832 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5d20025030 con 0x7f5d2403d360 2021-04-03T03:13:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.547+0000 7f5d28ff9700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d0c056f70 0x7f5d0c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.547+0000 7f5d28ff9700 1 -- 172.21.2.111:0/2150894832 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5d200717a0 con 0x7f5d2403d360 2021-04-03T03:13:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.547+0000 7f5d2b7fe700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d0c056f70 0x7f5d0c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.547+0000 7f5d2b7fe700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d0c056f70 0x7f5d0c059430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f5d1c002f30 tx=0x7f5d1c0069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:30.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.547+0000 7f5d28ff9700 1 -- 172.21.2.111:0/2150894832 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5d20046a80 con 0x7f5d2403d360 2021-04-03T03:13:30.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.675+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f5d240e7590 con 0x7f5d2403d360 2021-04-03T03:13:30.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d28ff9700 1 -- 172.21.2.111:0/2150894832 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v200) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5d2004a0a0 con 0x7f5d2403d360 2021-04-03T03:13:30.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d0c056f70 msgr2=0x7f5d0c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d0c056f70 0x7f5d0c059430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f5d1c002f30 tx=0x7f5d1c0069f0).stop 2021-04-03T03:13:30.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 msgr2=0x7f5d240d57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:30.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240d57c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f5d2403b890 tx=0x7f5d20015000).stop 2021-04-03T03:13:30.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 shutdown_connections 2021-04-03T03:13:30.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d240dbb90 0x7f5d240d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:30.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d0c056f70 0x7f5d0c059430 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d240064b0 0x7f5d2403c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:30.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 --2- 172.21.2.111:0/2150894832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d2403d360 0x7f5d240d57c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:30.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 >> 172.21.2.111:0/2150894832 conn(0x7f5d240462c0 msgr2=0x7f5d240d7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:30.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 shutdown_connections 2021-04-03T03:13:30.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.915+0000 7f5d30c3c700 1 -- 172.21.2.111:0/2150894832 wait complete. 2021-04-03T03:13:30.939 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-04-03T03:13:30.939 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2021-04-03T03:13:30.940 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2021-04-03T03:13:31.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f864d05e700 1 Processor -- start 2021-04-03T03:13:31.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f864d05e700 1 -- start start 2021-04-03T03:13:31.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f864d05e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f864814b700 0x7f86480b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f864d05e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86480607d0 con 0x7f86480603f0 2021-04-03T03:13:31.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f864d05e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86480af400 con 0x7f86480b0c90 2021-04-03T03:13:31.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f864d05e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86480af6a0 con 0x7f864814b700 2021-04-03T03:13:31.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f8647fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f86480603f0 0x7f86480b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53468/0 (socket says 172.21.2.111:53468) 2021-04-03T03:13:31.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.983+0000 7f8647fff700 1 -- 172.21.2.111:0/450323616 learned_addr learned my addr 172.21.2.111:0/450323616 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:31.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647253091 0 0) 0x7f86480607d0 con 0x7f86480603f0 2021-04-03T03:13:31.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8628003240 con 0x7f86480603f0 2021-04-03T03:13:31.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 624763429 0 0) 0x7f8628003240 con 0x7f86480603f0 2021-04-03T03:13:31.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f864814b700 msgr2=0x7f86480b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:31.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f864814b700 0x7f86480b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:31.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 >> v1:172.21.2.115:6789/0 conn(0x7f86480b0c90 legacy=0x7f8648147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:31.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480afb40 con 0x7f86480603f0 2021-04-03T03:13:31.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f86300016d0 con 0x7f86480603f0 2021-04-03T03:13:31.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 >> v1:172.21.2.111:6789/0 conn(0x7f86480603f0 legacy=0x7f86480b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:31.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8628003760 0x7f8628003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86280044e0 0x7f8628004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f864814b700 0x7f8628004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86480af6a0 con 0x7f86280044e0 2021-04-03T03:13:31.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8628004280 con 0x7f8628003760 2021-04-03T03:13:31.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862800b730 con 0x7f864814b700 2021-04-03T03:13:31.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86280044e0 0x7f8628004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:31.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8647fff700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8628003760 0x7f8628003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:31.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f864814b700 0x7f8628004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:31.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 -- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f864814b700 msgr2=0x7f8628004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:31.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f864814b700 0x7f8628004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 -- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8628003760 msgr2=0x7f8628003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:31.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8628003760 0x7f8628003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 -- 172.21.2.111:0/450323616 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628003240 con 0x7f86280044e0 2021-04-03T03:13:31.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86280044e0 0x7f8628004960 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f8638002f30 tx=0x7f8638006e10).ready entity=mon.0 client_cookie=7973c20157c0ab0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:31.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8638009650 con 0x7f86280044e0 2021-04-03T03:13:31.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8638000e90 con 0x7f86280044e0 2021-04-03T03:13:31.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8646ffd700 1 -- 172.21.2.111:0/450323616 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86380099a0 con 0x7f86280044e0 2021-04-03T03:13:31.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- 172.21.2.111:0/450323616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86280044e0 msgr2=0x7f8628004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:31.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 --2- 172.21.2.111:0/450323616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86280044e0 0x7f8628004960 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f8638002f30 tx=0x7f8638006e10).stop 2021-04-03T03:13:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 -- 172.21.2.111:0/450323616 reap_dead start 2021-04-03T03:13:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- 172.21.2.111:0/450323616 shutdown_connections 2021-04-03T03:13:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- 172.21.2.111:0/450323616 >> 172.21.2.111:0/450323616 conn(0x7f86480b61c0 msgr2=0x7f864805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- 172.21.2.111:0/450323616 shutdown_connections 2021-04-03T03:13:31.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- 172.21.2.111:0/450323616 wait complete. 2021-04-03T03:13:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 Processor -- start 2021-04-03T03:13:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- start start 2021-04-03T03:13:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86480aaae0 0x7f8648065850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86480ab460 0x7f86481561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86480629f0 0x7f86481566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648064eb0 con 0x7f86480629f0 2021-04-03T03:13:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86480aaec0 con 0x7f86480aaae0 2021-04-03T03:13:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8648065110 con 0x7f86480ab460 2021-04-03T03:13:31.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86480629f0 0x7f86481566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86480629f0 0x7f86481566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52692/0 (socket says 172.21.2.111:52692) 2021-04-03T03:13:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 -- 172.21.2.111:0/3491019413 learned_addr learned my addr 172.21.2.111:0/3491019413 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f8647fff700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86480aaae0 0x7f8648065850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86477fe700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86480ab460 0x7f86481561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:31.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 -- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86480ab460 msgr2=0x7f86481561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86480ab460 0x7f86481561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 -- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86480aaae0 msgr2=0x7f8648065850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86480aaae0 0x7f8648065850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 -- 172.21.2.111:0/3491019413 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480653b0 con 0x7f86480629f0 2021-04-03T03:13:31.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864c85d700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86480629f0 0x7f86481566e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f86400011d0 tx=0x7f8640007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86457fa700 1 -- 172.21.2.111:0/3491019413 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864000c9a0 con 0x7f86480629f0 2021-04-03T03:13:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86457fa700 1 -- 172.21.2.111:0/3491019413 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864000cb00 con 0x7f86480629f0 2021-04-03T03:13:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8648156bd0 con 0x7f86480629f0 2021-04-03T03:13:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f86457fa700 1 -- 172.21.2.111:0/3491019413 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864000ce60 con 0x7f86480629f0 2021-04-03T03:13:31.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.987+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8648067260 con 0x7f86480629f0 2021-04-03T03:13:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.991+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8648065540 con 0x7f86480629f0 2021-04-03T03:13:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.991+0000 7f86457fa700 1 -- 172.21.2.111:0/3491019413 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f864001d460 con 0x7f86480629f0 2021-04-03T03:13:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.991+0000 7f86457fa700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f862c056f20 0x7f862c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.991+0000 7f86457fa700 1 -- 172.21.2.111:0/3491019413 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f86400705d0 con 0x7f86480629f0 2021-04-03T03:13:31.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.991+0000 7f8647fff700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f862c056f20 0x7f862c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:31.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.991+0000 7f8647fff700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f862c056f20 0x7f862c0593e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8630005000 tx=0x7f86480af590).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:31.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:30.991+0000 7f86457fa700 1 -- 172.21.2.111:0/3491019413 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8640045960 con 0x7f86480629f0 2021-04-03T03:13:31.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.123+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f8648057660 con 0x7f86480629f0 2021-04-03T03:13:31.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.919+0000 7f86457fa700 1 -- 172.21.2.111:0/3491019413 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v201) v1 ==== 150+0+0 (secure 0 0 0) 0x7f8640048f80 con 0x7f86480629f0 2021-04-03T03:13:31.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.919+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f862c056f20 msgr2=0x7f862c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:31.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.919+0000 7f864d05e700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f862c056f20 0x7f862c0593e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8630005000 tx=0x7f86480af590).stop 2021-04-03T03:13:31.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.919+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86480629f0 msgr2=0x7f86481566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:31.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.919+0000 7f864d05e700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86480629f0 0x7f86481566e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f86400011d0 tx=0x7f8640007ed0).stop 2021-04-03T03:13:31.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 shutdown_connections 2021-04-03T03:13:31.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86480629f0 0x7f86481566e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f862c056f20 0x7f862c0593e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86480aaae0 0x7f8648065850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 --2- 172.21.2.111:0/3491019413 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86480ab460 0x7f86481561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:31.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 >> 172.21.2.111:0/3491019413 conn(0x7f86480b61c0 msgr2=0x7f86480b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:31.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 shutdown_connections 2021-04-03T03:13:31.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.923+0000 7f864d05e700 1 -- 172.21.2.111:0/3491019413 wait complete. 2021-04-03T03:13:31.944 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:13:32.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 Processor -- start 2021-04-03T03:13:32.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- start start 2021-04-03T03:13:32.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f210c0dbba0 0x7f210c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c03d560 0x7f210c041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c0dc560 con 0x7f210c041060 2021-04-03T03:13:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c03de60 con 0x7f210c0dbba0 2021-04-03T03:13:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0dc800 con 0x7f210c03d560 2021-04-03T03:13:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f210c0dbba0 0x7f210c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f2111d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f210c041060 0x7f210c03fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53488/0 (socket says 172.21.2.111:53488) 2021-04-03T03:13:32.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f2111d9b700 1 -- 172.21.2.111:0/1463998334 learned_addr learned my addr 172.21.2.111:0/1463998334 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:32.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211259c700 1 -- 172.21.2.111:0/1463998334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c03d560 msgr2=0x7f210c041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211259c700 1 --2- 172.21.2.111:0/1463998334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c03d560 0x7f210c041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211259c700 1 -- 172.21.2.111:0/1463998334 >> v1:172.21.2.111:6789/0 conn(0x7f210c041060 legacy=0x7f210c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211259c700 1 -- 172.21.2.111:0/1463998334 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c03da30 con 0x7f210c0dbba0 2021-04-03T03:13:32.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211259c700 1 --2- 172.21.2.111:0/1463998334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c0dbba0 0x7f210c0dbf80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f2104002f30 tx=0x7f2104006dd0).ready entity=mon.0 client_cookie=c7b81295ecbb7eb6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211159a700 1 -- 172.21.2.111:0/1463998334 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21040097f0 con 0x7f210c0dbba0 2021-04-03T03:13:32.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211159a700 1 -- 172.21.2.111:0/1463998334 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2104009950 con 0x7f210c0dbba0 2021-04-03T03:13:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211159a700 1 -- 172.21.2.111:0/1463998334 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2104009cb0 con 0x7f210c0dbba0 2021-04-03T03:13:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- 172.21.2.111:0/1463998334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c0dbba0 msgr2=0x7f210c0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 --2- 172.21.2.111:0/1463998334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c0dbba0 0x7f210c0dbf80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f2104002f30 tx=0x7f2104006dd0).stop 2021-04-03T03:13:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- 172.21.2.111:0/1463998334 shutdown_connections 2021-04-03T03:13:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 --2- 172.21.2.111:0/1463998334 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c03d560 0x7f210c041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 --2- 172.21.2.111:0/1463998334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c0dbba0 0x7f210c0dbf80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- 172.21.2.111:0/1463998334 >> 172.21.2.111:0/1463998334 conn(0x7f210c0462b0 msgr2=0x7f210c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- 172.21.2.111:0/1463998334 shutdown_connections 2021-04-03T03:13:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.987+0000 7f211359e700 1 -- 172.21.2.111:0/1463998334 wait complete. 2021-04-03T03:13:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 Processor -- start 2021-04-03T03:13:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 -- start start 2021-04-03T03:13:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c03d560 0x7f210c0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c041060 0x7f210c0a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f210c0dbba0 0x7f210c03a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0d7a60 con 0x7f210c03d560 2021-04-03T03:13:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c09f800 con 0x7f210c0dbba0 2021-04-03T03:13:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0a0bd0 con 0x7f210c041060 2021-04-03T03:13:32.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c03d560 0x7f210c0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c03d560 0x7f210c0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52702/0 (socket says 172.21.2.111:52702) 2021-04-03T03:13:32.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 -- 172.21.2.111:0/738414888 learned_addr learned my addr 172.21.2.111:0/738414888 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:32.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 -- 172.21.2.111:0/738414888 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c041060 msgr2=0x7f210c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c041060 0x7f210c0a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 -- 172.21.2.111:0/738414888 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f210c0dbba0 msgr2=0x7f210c03a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f210c0dbba0 0x7f210c03a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 -- 172.21.2.111:0/738414888 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c09fa30 con 0x7f210c03d560 2021-04-03T03:13:32.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211259c700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c03d560 0x7f210c0a12d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f2104002f30 tx=0x7f2104015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f210b7fe700 1 -- 172.21.2.111:0/738414888 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2104011c80 con 0x7f210c03d560 2021-04-03T03:13:32.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f210b7fe700 1 -- 172.21.2.111:0/738414888 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2104011de0 con 0x7f210c03d560 2021-04-03T03:13:32.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c0a0e30 con 0x7f210c03d560 2021-04-03T03:13:32.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f210b7fe700 1 -- 172.21.2.111:0/738414888 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f210401e650 con 0x7f210c03d560 2021-04-03T03:13:32.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c0a17c0 con 0x7f210c03d560 2021-04-03T03:13:32.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c0a0fc0 con 0x7f210c03d560 2021-04-03T03:13:32.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f210b7fe700 1 -- 172.21.2.111:0/738414888 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2104025030 con 0x7f210c03d560 2021-04-03T03:13:32.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f210b7fe700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20f8057000 0x7f20f80594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f210b7fe700 1 -- 172.21.2.111:0/738414888 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f2104071730 con 0x7f210c03d560 2021-04-03T03:13:32.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f2111d9b700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20f8057000 0x7f20f80594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.991+0000 7f2111d9b700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20f8057000 0x7f20f80594c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f210c0087d0 tx=0x7f20fc0077b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:31.995+0000 7f210b7fe700 1 -- 172.21.2.111:0/738414888 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2104046b20 con 0x7f210c03d560 2021-04-03T03:13:32.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.087+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f210c004db0 con 0x7f210c03d560 2021-04-03T03:13:32.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.091+0000 7f210b7fe700 1 -- 172.21.2.111:0/738414888 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f210404a140 con 0x7f210c03d560 2021-04-03T03:13:32.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.091+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20f8057000 msgr2=0x7f20f80594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.091+0000 7f211359e700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20f8057000 0x7f20f80594c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f210c0087d0 tx=0x7f20fc0077b0).stop 2021-04-03T03:13:32.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.091+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c03d560 msgr2=0x7f210c0a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.091+0000 7f211359e700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c03d560 0x7f210c0a12d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f2104002f30 tx=0x7f2104015000).stop 2021-04-03T03:13:32.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 shutdown_connections 2021-04-03T03:13:32.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20f8057000 0x7f20f80594c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f210c0dbba0 0x7f210c03a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f210c03d560 0x7f210c0a12d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 --2- 172.21.2.111:0/738414888 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f210c041060 0x7f210c0a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 >> 172.21.2.111:0/738414888 conn(0x7f210c0462b0 msgr2=0x7f210c0459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:32.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 shutdown_connections 2021-04-03T03:13:32.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.095+0000 7f211359e700 1 -- 172.21.2.111:0/738414888 wait complete. 2021-04-03T03:13:32.116 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:13:32.116 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:32.123 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:32.131 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:32.143 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:32.155 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:13:32.167 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:13:32.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:32.168 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:32.168 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:32.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:13:32.188 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:13:32.188 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:13:32.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:32.189 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:32.189 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:13:32.201 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:13:32.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:13:32.203 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:13:32.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:32.204 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:32.204 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:13:32.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:13:32.212 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:13:32.212 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:13:32.213 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:13:32.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 Processor -- start 2021-04-03T03:13:32.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 -- start start 2021-04-03T03:13:32.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f72100b12b0 0x7f72101522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72100607d0 con 0x7f721014b700 2021-04-03T03:13:32.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72100b0eb0 con 0x7f72100603f0 2021-04-03T03:13:32.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72100af400 con 0x7f72100b12b0 2021-04-03T03:13:32.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720dd9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f721014b700 0x7f72100b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53504/0 (socket says 172.21.2.111:53504) 2021-04-03T03:13:32.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720dd9b700 1 -- 172.21.2.111:0/1862345555 learned_addr learned my addr 172.21.2.111:0/1862345555 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:32.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780094765 0 0) 0x7f72100607d0 con 0x7f721014b700 2021-04-03T03:13:32.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f0003240 con 0x7f721014b700 2021-04-03T03:13:32.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4246398863 0 0) 0x7f71f0003240 con 0x7f721014b700 2021-04-03T03:13:32.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 --2- 172.21.2.111:0/1862345555 >> v2:172.21.2.115:3300/0 conn(0x7f72100b12b0 0x7f72101522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 >> v2:172.21.2.115:3300/0 conn(0x7f72100b12b0 msgr2=0x7f72101522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 --2- 172.21.2.111:0/1862345555 >> v2:172.21.2.115:3300/0 conn(0x7f72100b12b0 0x7f72101522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 >> v1:172.21.2.115:6789/0 conn(0x7f72100603f0 legacy=0x7f72100b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100af6a0 con 0x7f721014b700 2021-04-03T03:13:32.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f71f80016d0 con 0x7f721014b700 2021-04-03T03:13:32.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 >> v1:172.21.2.111:6789/0 conn(0x7f721014b700 legacy=0x7f72100b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f0003830 0x7f71f0003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00044e0 0x7f71f0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f71f000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72100af400 con 0x7f71f0004e50 2021-04-03T03:13:32.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0005330 con 0x7f71f0003830 2021-04-03T03:13:32.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f000bc20 con 0x7f71f00044e0 2021-04-03T03:13:32.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f71f000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720e59c700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f0003830 0x7f71f0003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720dd9b700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00044e0 0x7f71f0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 -- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00044e0 msgr2=0x7f71f0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00044e0 0x7f71f0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 -- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f0003830 msgr2=0x7f71f0003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f0003830 0x7f71f0003c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 -- 172.21.2.111:0/1862345555 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0003240 con 0x7f71f0004e50 2021-04-03T03:13:32.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720dd9b700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00044e0 0x7f71f0004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:13:32.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 -- 172.21.2.111:0/1862345555 reap_dead start 2021-04-03T03:13:32.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720ed9d700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f71f000b730 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f7208007000 tx=0x7f7208008040).ready entity=mon.0 client_cookie=df90dfc04263a4f5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f720800cc30 con 0x7f71f0004e50 2021-04-03T03:13:32.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720800cd90 con 0x7f71f0004e50 2021-04-03T03:13:32.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720d59a700 1 -- 172.21.2.111:0/1862345555 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7208011410 con 0x7f71f0004e50 2021-04-03T03:13:32.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 -- 172.21.2.111:0/1862345555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 msgr2=0x7f71f000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f71f000b730 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f7208007000 tx=0x7f7208008040).stop 2021-04-03T03:13:32.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 -- 172.21.2.111:0/1862345555 shutdown_connections 2021-04-03T03:13:32.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 --2- 172.21.2.111:0/1862345555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f71f000b730 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.275+0000 7f720f59e700 1 -- 172.21.2.111:0/1862345555 >> 172.21.2.111:0/1862345555 conn(0x7f72100b61c0 msgr2=0x7f721005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:32.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- 172.21.2.111:0/1862345555 shutdown_connections 2021-04-03T03:13:32.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- 172.21.2.111:0/1862345555 wait complete. 2021-04-03T03:13:32.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 Processor -- start 2021-04-03T03:13:32.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- start start 2021-04-03T03:13:32.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f72100657e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7210065cd0 0x7f72100674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72100679d0 0x7f7210062db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210066150 con 0x7f71f0004e50 2021-04-03T03:13:32.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72100aae10 con 0x7f7210065cd0 2021-04-03T03:13:32.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72100a9fb0 con 0x7f72100679d0 2021-04-03T03:13:32.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f72100657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f72100657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52722/0 (socket says 172.21.2.111:52722) 2021-04-03T03:13:32.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 -- 172.21.2.111:0/3940687730 learned_addr learned my addr 172.21.2.111:0/3940687730 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:32.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720dd9b700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7210065cd0 0x7f72100674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720ed9d700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72100679d0 0x7f7210062db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 -- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72100679d0 msgr2=0x7f7210062db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72100679d0 0x7f7210062db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 -- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7210065cd0 msgr2=0x7f72100674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7210065cd0 0x7f72100674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 -- 172.21.2.111:0/3940687730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210156e00 con 0x7f71f0004e50 2021-04-03T03:13:32.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720e59c700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f72100657e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7210066450 tx=0x7f7200009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f72077fe700 1 -- 172.21.2.111:0/3940687730 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7200010710 con 0x7f71f0004e50 2021-04-03T03:13:32.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f72077fe700 1 -- 172.21.2.111:0/3940687730 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7200010870 con 0x7f71f0004e50 2021-04-03T03:13:32.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210157090 con 0x7f71f0004e50 2021-04-03T03:13:32.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f72077fe700 1 -- 172.21.2.111:0/3940687730 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7200010bd0 con 0x7f71f0004e50 2021-04-03T03:13:32.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72100aa9b0 con 0x7f71f0004e50 2021-04-03T03:13:32.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210057660 con 0x7f71f0004e50 2021-04-03T03:13:32.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f72077fe700 1 -- 172.21.2.111:0/3940687730 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7200010d30 con 0x7f71f0004e50 2021-04-03T03:13:32.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f72077fe700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71f4056f20 0x7f71f40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f72077fe700 1 -- 172.21.2.111:0/3940687730 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f72000700e0 con 0x7f71f0004e50 2021-04-03T03:13:32.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720dd9b700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71f4056f20 0x7f71f40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.279+0000 7f720dd9b700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71f4056f20 0x7f71f40593e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f71f800c000 tx=0x7f71f8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.283+0000 7f72077fe700 1 -- 172.21.2.111:0/3940687730 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7200045470 con 0x7f71f0004e50 2021-04-03T03:13:32.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.375+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f72100b0e20 con 0x7f71f0004e50 2021-04-03T03:13:32.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.375+0000 7f72077fe700 1 -- 172.21.2.111:0/3940687730 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+34039 (secure 0 0 0) 0x7f7200048a90 con 0x7f71f0004e50 2021-04-03T03:13:32.393 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:32.393 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:13:32.394 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 54, 2021-04-03T03:13:32.394 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:13:32.394 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:13:32.394 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:12:06.034712+0000", 2021-04-03T03:13:32.394 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:13:32.395 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:13:32.395 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:13:32.395 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:13:32.395 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:13:32.396 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:13:32.396 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:13:32.396 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:13:32.396 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:13:32.397 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:13:32.397 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:13:32.397 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:13:32.397 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:13:32.397 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:13:32.398 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:13:32.398 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 10, 2021-04-03T03:13:32.398 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:13:32.398 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:13:32.398 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:13:32.399 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:13:32.399 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:13:32.399 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.399 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:13:32.400 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:13:32.400 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:13:32.400 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:13:32.400 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:13:32.400 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:13:32.401 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:13:32.401 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:13:32.401 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:13:32.401 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:13:32.402 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:13:32.402 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:13:32.402 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:13:32.402 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:13:32.402 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:13:32.403 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:13:32.403 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:13:32.403 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:13:32.403 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:13:32.403 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:13:32.404 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:13:32.404 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:13:32.404 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:13:32.404 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.405 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:13:32.405 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:13:32.405 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:13:32.405 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:13:32.405 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:13:32.406 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:13:32.406 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:13:32.406 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:13:32.406 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:13:32.407 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:13:32.407 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:13:32.407 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:13:32.407 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:13:32.407 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:13:32.408 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:13:32.408 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:13:32.408 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:13:32.408 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:13:32.409 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:13:32.409 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:13:32.409 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:13:32.409 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:13:32.409 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:13:32.410 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:13:32.410 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:13:32.410 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:13:32.410 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:13:32.410 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.411 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:13:32.411 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:13:32.411 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:13:32.411 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:13:32.412 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:13:32.412 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:13:32.412 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:13:32.412 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:13:32.412 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:13:32.413 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:13:32.413 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:13:32.413 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:13:32.413 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:13:32.414 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.414 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:13:32.414 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:13:32.414 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.414 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.415 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.415 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:13:32.415 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:13:32.415 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:13:32.416 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:13:32.416 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:13:32.416 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:13:32.416 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:13:32.417 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:13:32.417 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:13:32.417 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:13:32.417 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:13:32.417 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:13:32.418 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:13:32.418 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:13:32.418 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:13:32.418 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:13:32.419 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:13:32.419 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:13:32.419 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:13:32.419 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:13:32.419 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:13:32.420 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:13:32.420 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:13:32.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.420 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:13:32.421 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:13:32.421 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:13:32.421 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:13:32.421 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:13:32.421 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:13:32.422 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:13:32.422 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:13:32.422 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:13:32.422 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:13:32.423 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:13:32.423 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:13:32.423 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:13:32.423 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:13:32.423 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:13:32.424 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:13:32.424 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:13:32.424 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:13:32.424 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:13:32.425 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:13:32.425 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:13:32.425 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:13:32.425 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:13:32.425 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:13:32.426 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:13:32.426 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:13:32.426 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:13:32.426 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.426 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:13:32.427 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:13:32.427 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:13:32.427 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:13:32.427 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:13:32.427 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:13:32.428 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:13:32.428 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:13:32.428 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:13:32.428 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:13:32.429 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:13:32.429 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:13:32.429 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:13:32.429 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:13:32.429 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.430 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.430 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.430 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 9, 2021-04-03T03:13:32.430 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:13:32.431 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:12:02.184726+0000", 2021-04-03T03:13:32.431 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:13:32.431 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:13:32.431 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:13:32.431 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:13:32.432 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:13:32.432 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:13:32.432 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:13:32.432 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:13:32.432 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:13:32.433 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:13:32.433 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:13:32.433 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:13:32.433 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:13:32.434 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:13:32.434 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:13:32.434 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:13:32.434 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:13:32.434 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:13:32.435 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:13:32.435 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:13:32.435 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.435 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "53", 2021-04-03T03:13:32.436 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:13:32.436 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:13:32.436 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:13:32.436 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:13:32.436 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:13:32.437 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:13:32.437 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:13:32.437 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:13:32.437 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:13:32.437 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:13:32.438 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:13:32.438 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:13:32.438 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:13:32.438 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:13:32.438 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:13:32.439 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:13:32.439 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:13:32.439 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:13:32.439 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:13:32.440 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:13:32.440 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:13:32.440 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:13:32.440 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:13:32.440 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:13:32.441 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:13:32.441 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:13:32.441 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.441 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:13:32.441 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:13:32.442 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:13:32.442 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:13:32.442 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:13:32.442 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:13:32.442 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:13:32.443 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:13:32.443 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:13:32.443 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:13:32.443 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:13:32.444 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:13:32.444 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:13:32.444 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:13:32.444 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:13:32.444 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.445 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:13:32.445 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:13:32.445 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:13:32.445 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.445 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.446 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.446 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.446 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 10, 2021-04-03T03:13:32.446 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:13:32.446 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:12:02.990138+0000", 2021-04-03T03:13:32.447 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:13:32.447 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:13:32.447 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:13:32.447 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:13:32.448 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:13:32.448 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:13:32.448 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:13:32.448 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:13:32.448 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:13:32.449 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:13:32.449 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:13:32.449 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:13:32.449 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:13:32.449 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:13:32.450 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:13:32.450 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:13:32.450 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:13:32.450 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:13:32.451 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:13:32.451 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:13:32.451 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.451 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "54", 2021-04-03T03:13:32.451 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:13:32.452 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:13:32.452 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:13:32.452 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:13:32.452 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:13:32.453 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:13:32.453 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:13:32.453 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:13:32.453 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:13:32.453 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:13:32.454 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:13:32.454 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:13:32.454 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:13:32.454 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:13:32.454 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:13:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:13:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:13:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:13:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:13:32.455 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:13:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:13:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:13:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:13:32.456 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:13:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:13:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:13:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:13:32.457 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:13:32.458 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:13:32.458 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:13:32.458 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:13:32.458 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:13:32.458 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:13:32.459 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:13:32.459 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:13:32.459 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:13:32.459 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:13:32.459 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:13:32.460 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:13:32.460 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:13:32.460 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:13:32.460 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.461 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.461 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.461 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:13:32.461 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:13:32.461 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.462 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:13:32.462 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:13:32.462 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.462 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.462 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.463 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.463 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.463 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.463 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.464 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 50, 2021-04-03T03:13:32.464 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.464 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.464 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.464 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.465 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.465 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.465 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:13:32.465 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.465 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.466 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.466 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.466 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:13:32.466 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.467 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.467 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.467 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.467 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.467 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.468 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.468 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.468 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:13:32.468 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.468 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.469 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.469 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.469 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:13:32.469 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.470 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.470 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.470 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.470 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.470 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.471 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.471 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.471 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:13:32.471 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.471 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.472 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.472 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.472 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:13:32.472 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.472 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.473 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.473 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.473 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.474 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.474 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.474 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:13:32.474 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.474 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.475 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.475 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.475 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:13:32.475 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:13:32.475 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.476 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.476 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.476 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:13:32.476 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:13:32.476 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:13:32.477 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:13:32.477 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.477 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.477 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.477 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.478 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.478 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.478 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:13:32.478 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:13:32.479 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.479 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.479 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.479 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.479 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.480 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.480 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.480 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 50, 2021-04-03T03:13:32.480 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.480 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.481 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.481 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.481 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.481 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.481 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:13:32.482 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.482 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.482 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.482 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.483 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:13:32.483 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.483 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.483 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.483 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.484 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.484 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.484 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.484 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.484 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:13:32.485 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.485 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.485 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.485 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.485 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:13:32.486 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.486 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.486 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.486 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.487 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.487 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.487 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.487 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.487 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:13:32.488 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.488 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.488 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.488 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.488 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:13:32.489 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.489 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.489 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.489 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.489 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.490 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.490 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.490 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.490 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:13:32.491 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.491 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.491 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.491 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:13:32.492 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:13:32.492 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.492 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.492 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.492 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:13:32.493 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:13:32.493 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:13:32.493 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:13:32.493 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.493 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.494 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.494 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.494 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.494 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.494 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:13:32.495 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:13:32.495 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.495 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.495 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.496 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.496 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.496 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.496 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.496 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 51, 2021-04-03T03:13:32.497 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.497 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.497 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.497 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.497 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.498 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.498 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:13:32.498 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.498 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.498 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.499 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.499 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:13:32.499 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.499 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.499 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.500 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.500 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.500 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.500 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.501 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.501 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:13:32.501 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.501 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.501 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.502 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.502 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:13:32.502 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.502 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.502 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.503 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.503 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.503 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.503 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.504 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.504 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:13:32.504 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.504 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.504 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.505 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.505 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:13:32.505 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.505 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.505 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.506 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.506 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.506 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.506 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.506 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.507 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:13:32.507 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.507 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.507 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.508 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.508 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:13:32.508 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:13:32.508 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.508 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.509 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.509 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:13:32.509 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:13:32.509 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:13:32.509 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:13:32.510 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.510 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.510 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.510 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.510 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.511 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.511 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:13:32.511 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:13:32.511 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.512 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.512 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.512 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.512 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.512 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.513 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.513 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 49, 2021-04-03T03:13:32.513 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.513 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.513 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.514 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.514 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.514 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.514 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:13:32.515 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.515 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.515 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.515 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.515 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:13:32.516 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.516 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.516 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.516 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.516 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.517 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.517 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.517 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.517 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:13:32.518 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.518 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.518 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.518 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.518 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:13:32.519 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.519 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.519 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.519 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.519 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.520 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.520 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.520 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.520 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:13:32.521 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.521 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.521 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.521 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.521 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:13:32.522 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.522 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.522 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.522 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.522 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.523 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.523 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.523 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.523 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:13:32.524 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.524 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.524 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.524 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.524 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:13:32.525 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:13:32.525 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.525 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.525 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.525 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:13:32.526 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:13:32.526 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:13:32.526 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:13:32.526 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.526 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.527 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.527 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.527 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.527 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.528 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:13:32.528 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:13:32.528 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.528 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.528 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.529 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.529 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.529 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.529 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.529 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 50, 2021-04-03T03:13:32.530 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.530 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.530 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.530 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.530 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.531 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.531 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:13:32.531 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.531 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.532 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.532 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.532 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:13:32.532 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.532 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.533 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.533 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.533 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.533 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.533 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.534 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.534 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:13:32.534 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.534 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.534 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.535 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.535 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:13:32.535 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.535 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.535 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.536 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.536 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.536 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.536 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.537 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.537 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:13:32.537 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.537 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.537 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:13:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.538 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.539 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.540 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.540 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:13:32.540 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.540 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.540 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.541 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.541 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:13:32.541 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:13:32.541 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.541 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.542 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.542 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:13:32.542 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:13:32.542 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:13:32.542 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:13:32.543 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.543 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.543 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.543 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.544 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.544 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.544 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:13:32.544 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:13:32.544 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.545 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.545 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.545 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.545 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.545 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.546 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.546 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 50, 2021-04-03T03:13:32.546 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.546 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.546 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.547 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.547 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.547 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.547 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:13:32.548 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.548 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.548 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.548 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.548 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:13:32.549 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.549 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.549 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.549 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.549 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.550 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.550 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.550 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.550 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:13:32.551 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.551 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.551 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.551 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.551 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:13:32.552 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.552 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.552 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.552 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.552 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.553 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.553 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.553 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.553 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:13:32.553 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.554 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.554 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.554 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.554 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:13:32.555 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.555 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.555 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.555 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.555 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.556 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.556 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.556 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.556 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:13:32.557 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.557 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.557 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.557 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.557 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:13:32.558 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:13:32.558 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.558 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.558 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.558 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:13:32.559 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:13:32.559 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:13:32.559 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:13:32.559 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.559 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.560 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.560 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.560 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.560 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.561 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:13:32.561 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:13:32.561 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.561 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.561 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.562 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.562 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.562 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.562 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.562 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 52, 2021-04-03T03:13:32.563 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.563 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.563 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.563 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.563 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.564 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.564 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:13:32.564 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.564 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.565 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.565 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.565 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:13:32.565 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.565 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.566 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.566 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.566 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.566 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.566 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.567 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.567 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:13:32.567 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.567 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.567 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.568 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.568 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:13:32.568 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.568 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.568 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.569 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.569 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.569 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.569 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.569 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.570 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:13:32.570 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.570 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.570 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.571 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.571 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:13:32.571 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.571 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.571 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.572 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.572 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.572 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.572 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.572 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.573 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:13:32.573 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.573 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.573 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.573 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.574 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:13:32.574 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:13:32.574 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.574 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.575 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.575 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:13:32.575 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:13:32.575 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:13:32.575 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:13:32.576 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.576 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.576 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.576 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.576 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.577 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.577 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:13:32.577 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:13:32.577 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:13:32.577 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:13:32.578 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:13:32.578 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:13:32.578 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:13:32.578 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:13:32.578 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:13:32.579 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 49, 2021-04-03T03:13:32.579 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:13:32.579 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:13:32.579 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:13:32.580 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.580 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.580 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.580 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:13:32.580 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.581 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.581 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.581 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.581 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:13:32.581 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.582 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.582 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.582 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.582 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:13:32.582 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.583 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.583 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.583 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:13:32.583 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.584 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.584 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.584 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.584 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:13:32.584 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.585 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.585 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.585 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.585 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:13:32.585 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.586 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.586 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.586 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:13:32.586 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.586 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.587 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.587 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.587 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:13:32.587 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.588 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.588 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.588 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.588 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:13:32.588 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:13:32.589 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.589 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:13:32.589 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:13:32.589 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.589 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.590 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.590 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:13:32.590 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:13:32.590 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:13:32.590 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.591 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.591 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.591 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:13:32.591 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:13:32.592 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:13:32.592 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:13:32.592 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:13:32.592 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:13:32.592 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:13:32.593 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:32.593 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.593 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:13:32.593 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:13:32.593 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.594 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:13:32.594 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.594 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.594 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.594 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.595 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.595 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:13:32.595 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.595 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.595 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.596 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:13:32.596 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.596 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.596 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.597 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.597 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.597 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:13:32.597 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.597 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.598 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.598 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:13:32.598 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.598 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.598 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.599 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.599 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.599 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:13:32.599 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.600 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.600 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:13:32.600 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.600 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.600 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.601 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.601 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.601 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:13:32.601 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.602 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.602 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.602 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:13:32.602 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.602 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.603 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.603 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.603 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.603 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:13:32.603 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.604 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.604 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.604 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:13:32.604 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.604 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.605 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.605 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.605 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.605 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:13:32.605 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.606 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.606 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.606 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:13:32.606 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.606 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.607 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.607 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.607 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.607 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:13:32.608 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.608 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.608 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:32.608 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:13:32.608 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:13:32.609 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:13:32.609 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:13:32.609 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:13:32.609 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:13:32.609 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:13:32.610 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:13:32.610 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.610 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:13:32.610 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:13:32.610 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:13:32.611 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:13:32.611 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:13:32.611 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:13:32.611 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:13:32.612 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:13:32.612 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:13:32.612 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:13:32.612 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:13:32.612 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:13:32.613 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:13:32.613 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:13:32.613 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:13:32.613 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:13:32.613 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:13:32.614 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:13:32.614 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.614 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:13:32.614 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:13:32.614 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:13:32.615 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:13:32.615 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:13:32.615 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:13:32.615 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:13:32.615 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:32.616 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:32.616 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:13:32.616 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:13:32.616 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:13:32.617 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:13:32.617 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:13:32.617 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:13:32.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71f4056f20 msgr2=0x7f71f40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71f4056f20 0x7f71f40593e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f71f800c000 tx=0x7f71f8009040).stop 2021-04-03T03:13:32.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 msgr2=0x7f72100657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f72100657e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7210066450 tx=0x7f7200009040).stop 2021-04-03T03:13:32.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 shutdown_connections 2021-04-03T03:13:32.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72100679d0 0x7f7210062db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f0004e50 0x7f72100657e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71f4056f20 0x7f71f40593e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 --2- 172.21.2.111:0/3940687730 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7210065cd0 0x7f72100674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 >> 172.21.2.111:0/3940687730 conn(0x7f72100b61c0 msgr2=0x7f721005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:32.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 shutdown_connections 2021-04-03T03:13:32.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.379+0000 7f720f59e700 1 -- 172.21.2.111:0/3940687730 wait complete. 2021-04-03T03:13:32.622 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:13:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 Processor -- start 2021-04-03T03:13:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 -- start start 2021-04-03T03:13:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f60800603f0 0x7f60800b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60800af400 con 0x7f608014b700 2021-04-03T03:13:32.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60800af6a0 con 0x7f60800b0c90 2021-04-03T03:13:32.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6080060860 con 0x7f60800603f0 2021-04-03T03:13:32.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607dd9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f608014b700 0x7f60800b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53526/0 (socket says 172.21.2.111:53526) 2021-04-03T03:13:32.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607dd9b700 1 -- 172.21.2.111:0/2252091166 learned_addr learned my addr 172.21.2.111:0/2252091166 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:32.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607e59c700 1 --2- 172.21.2.111:0/2252091166 >> v2:172.21.2.111:3300/0 conn(0x7f60800603f0 0x7f60800b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607e59c700 1 -- 172.21.2.111:0/2252091166 >> v1:172.21.2.115:6789/0 conn(0x7f60800b0c90 legacy=0x7f6080147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607e59c700 1 -- 172.21.2.111:0/2252091166 >> v1:172.21.2.111:6789/0 conn(0x7f608014b700 legacy=0x7f60800b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607e59c700 1 -- 172.21.2.111:0/2252091166 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60800afb40 con 0x7f60800603f0 2021-04-03T03:13:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607e59c700 1 --2- 172.21.2.111:0/2252091166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60800603f0 0x7f60800b0090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f60800620f0 tx=0x7f6070006dd0).ready entity=mon.0 client_cookie=d3585c644ed58bd4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607d59a700 1 -- 172.21.2.111:0/2252091166 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60700097b0 con 0x7f60800603f0 2021-04-03T03:13:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607d59a700 1 -- 172.21.2.111:0/2252091166 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6070009910 con 0x7f60800603f0 2021-04-03T03:13:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607d59a700 1 -- 172.21.2.111:0/2252091166 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6070009c70 con 0x7f60800603f0 2021-04-03T03:13:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 -- 172.21.2.111:0/2252091166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60800603f0 msgr2=0x7f60800b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.667+0000 7f607f59e700 1 --2- 172.21.2.111:0/2252091166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60800603f0 0x7f60800b0090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f60800620f0 tx=0x7f6070006dd0).stop 2021-04-03T03:13:32.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- 172.21.2.111:0/2252091166 shutdown_connections 2021-04-03T03:13:32.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 --2- 172.21.2.111:0/2252091166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60800603f0 0x7f60800b0090 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- 172.21.2.111:0/2252091166 >> 172.21.2.111:0/2252091166 conn(0x7f60800b61c0 msgr2=0x7f608005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:32.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- 172.21.2.111:0/2252091166 shutdown_connections 2021-04-03T03:13:32.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- 172.21.2.111:0/2252091166 wait complete. 2021-04-03T03:13:32.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 Processor -- start 2021-04-03T03:13:32.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- start start 2021-04-03T03:13:32.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60800603f0 0x7f6080156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60800b0c90 0x7f6080062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f608014b700 0x7f6080156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800aa7f0 con 0x7f608014b700 2021-04-03T03:13:32.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800a9a00 con 0x7f60800b0c90 2021-04-03T03:13:32.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60800aaa20 con 0x7f60800603f0 2021-04-03T03:13:32.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f608014b700 0x7f6080156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f608014b700 0x7f6080156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52740/0 (socket says 172.21.2.111:52740) 2021-04-03T03:13:32.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 -- 172.21.2.111:0/3788028512 learned_addr learned my addr 172.21.2.111:0/3788028512 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:32.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607e59c700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60800603f0 0x7f6080156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 -- 172.21.2.111:0/3788028512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60800603f0 msgr2=0x7f6080156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60800603f0 0x7f6080156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 -- 172.21.2.111:0/3788028512 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60800b0c90 msgr2=0x7f6080062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60800b0c90 0x7f6080062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 -- 172.21.2.111:0/3788028512 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080065380 con 0x7f608014b700 2021-04-03T03:13:32.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607ed9d700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f608014b700 0x7f6080156dc0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f60780032d0 tx=0x7f6078006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f60777fe700 1 -- 172.21.2.111:0/3788028512 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f607800c960 con 0x7f608014b700 2021-04-03T03:13:32.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f60777fe700 1 -- 172.21.2.111:0/3788028512 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607800cac0 con 0x7f608014b700 2021-04-03T03:13:32.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60800650a0 con 0x7f608014b700 2021-04-03T03:13:32.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60800aa5c0 con 0x7f608014b700 2021-04-03T03:13:32.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f60777fe700 1 -- 172.21.2.111:0/3788028512 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f607800ce20 con 0x7f608014b700 2021-04-03T03:13:32.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60580033f0 con 0x7f608014b700 2021-04-03T03:13:32.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.671+0000 7f60777fe700 1 -- 172.21.2.111:0/3788028512 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f607801c070 con 0x7f608014b700 2021-04-03T03:13:32.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.675+0000 7f60777fe700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6064056f70 0x7f6064059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.675+0000 7f60777fe700 1 -- 172.21.2.111:0/3788028512 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f607806fd40 con 0x7f608014b700 2021-04-03T03:13:32.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.675+0000 7f607e59c700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6064056f70 0x7f6064059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.675+0000 7f607e59c700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6064056f70 0x7f6064059430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f6070017000 tx=0x7f6070015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.675+0000 7f60777fe700 1 -- 172.21.2.111:0/3788028512 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f60780450d0 con 0x7f608014b700 2021-04-03T03:13:32.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6058003c20 con 0x7f608014b700 2021-04-03T03:13:32.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f60777fe700 1 -- 172.21.2.111:0/3788028512 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f607801a030 con 0x7f608014b700 2021-04-03T03:13:32.815 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:32.815 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:12:05.022670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/2278897120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":2278897120},{"type":"v1","addr":"172.21.2.111:6835","nonce":2278897120}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:13:32.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6064056f70 msgr2=0x7f6064059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6064056f70 0x7f6064059430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f6070017000 tx=0x7f6070015000).stop 2021-04-03T03:13:32.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f608014b700 msgr2=0x7f6080156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f608014b700 0x7f6080156dc0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f60780032d0 tx=0x7f6078006f90).stop 2021-04-03T03:13:32.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 shutdown_connections 2021-04-03T03:13:32.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f608014b700 0x7f6080156dc0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60800603f0 0x7f6080156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6064056f70 0x7f6064059430 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 --2- 172.21.2.111:0/3788028512 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60800b0c90 0x7f6080062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 >> 172.21.2.111:0/3788028512 conn(0x7f60800b61c0 msgr2=0x7f6080147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:32.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 shutdown_connections 2021-04-03T03:13:32.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.799+0000 7f607f59e700 1 -- 172.21.2.111:0/3788028512 wait complete. 2021-04-03T03:13:32.819 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 21 2021-04-03T03:13:32.822 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:13:32.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca5135700 1 Processor -- start 2021-04-03T03:13:32.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca5135700 1 -- start start 2021-04-03T03:13:32.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca5135700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 0x7f7ca01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca5135700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7ca00b17b0 0x7f7ca0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca5135700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca00607d0 con 0x7f7ca00add50 2021-04-03T03:13:32.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca5135700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca01451c0 con 0x7f7ca00b17b0 2021-04-03T03:13:32.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca5135700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca014b090 con 0x7f7ca00603f0 2021-04-03T03:13:32.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7ca00add50 0x7f7ca0147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53538/0 (socket says 172.21.2.111:53538) 2021-04-03T03:13:32.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/2699065033 learned_addr learned my addr 172.21.2.111:0/2699065033 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:32.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986600324 0 0) 0x7f7ca00607d0 con 0x7f7ca00add50 2021-04-03T03:13:32.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c80003240 con 0x7f7ca00add50 2021-04-03T03:13:32.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9ffff700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 0x7f7ca01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca4934700 1 --2- 172.21.2.111:0/2699065033 >> v2:172.21.2.115:3300/0 conn(0x7f7ca00b17b0 0x7f7ca0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 564268458 0 0) 0x7f7c80003240 con 0x7f7ca00add50 2021-04-03T03:13:32.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 msgr2=0x7f7ca01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 0x7f7ca01447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 >> v2:172.21.2.115:3300/0 conn(0x7f7ca00b17b0 msgr2=0x7f7ca0144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 --2- 172.21.2.111:0/2699065033 >> v2:172.21.2.115:3300/0 conn(0x7f7ca00b17b0 0x7f7ca0144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca014b330 con 0x7f7ca00add50 2021-04-03T03:13:32.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f7c880016d0 con 0x7f7ca00add50 2021-04-03T03:13:32.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 >> v1:172.21.2.111:6789/0 conn(0x7f7ca00add50 legacy=0x7f7ca0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c800037b0 0x7f7c80003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c800044e0 0x7f7c80004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 0x7f7c80004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca014b090 con 0x7f7c800044e0 2021-04-03T03:13:32.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c80005340 con 0x7f7c800037b0 2021-04-03T03:13:32.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8000b730 con 0x7f7ca00603f0 2021-04-03T03:13:32.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c800044e0 0x7f7c80004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7ca4934700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 0x7f7c80004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9ffff700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c800037b0 0x7f7c80003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 msgr2=0x7f7c80004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca00603f0 0x7f7c80004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c800037b0 msgr2=0x7f7c80003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:32.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c800037b0 0x7f7c80003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:32.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/2699065033 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80003240 con 0x7f7c800044e0 2021-04-03T03:13:32.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9f7fe700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c800044e0 0x7f7c80004960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7c880013e0 tx=0x7f7ca014b4c0).ready entity=mon.0 client_cookie=cdc2d84eaf0615bd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c880070d0 con 0x7f7c800044e0 2021-04-03T03:13:32.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c88007230 con 0x7f7c800044e0 2021-04-03T03:13:32.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.867+0000 7f7c9effd700 1 -- 172.21.2.111:0/2699065033 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c880055f0 con 0x7f7c800044e0 2021-04-03T03:13:32.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/2699065033 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c800044e0 msgr2=0x7f7c80004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 --2- 172.21.2.111:0/2699065033 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c800044e0 0x7f7c80004960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7c880013e0 tx=0x7f7ca014b4c0).stop 2021-04-03T03:13:32.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca4934700 1 -- 172.21.2.111:0/2699065033 reap_dead start 2021-04-03T03:13:32.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/2699065033 shutdown_connections 2021-04-03T03:13:32.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/2699065033 >> 172.21.2.111:0/2699065033 conn(0x7f7ca00b61c0 msgr2=0x7f7ca005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:32.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/2699065033 shutdown_connections 2021-04-03T03:13:32.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/2699065033 wait complete. 2021-04-03T03:13:32.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 Processor -- start 2021-04-03T03:13:32.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- start start 2021-04-03T03:13:32.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ca00a98d0 0x7f7ca0159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ca0157450 0x7f7ca015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca0063720 0x7f7ca015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca015abc0 con 0x7f7ca0157450 2021-04-03T03:13:32.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00a8990 con 0x7f7ca00a98d0 2021-04-03T03:13:32.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00a9cb0 con 0x7f7ca0063720 2021-04-03T03:13:32.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ca0157450 0x7f7ca015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ca0157450 0x7f7ca015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52760/0 (socket says 172.21.2.111:52760) 2021-04-03T03:13:32.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/4202195371 learned_addr learned my addr 172.21.2.111:0/4202195371 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:32.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/4202195371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca0063720 msgr2=0x7f7ca015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca0063720 0x7f7ca015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/4202195371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ca00a98d0 msgr2=0x7f7ca0159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:32.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ca00a98d0 0x7f7ca0159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:32.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 -- 172.21.2.111:0/4202195371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0156a70 con 0x7f7ca0157450 2021-04-03T03:13:32.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9f7fe700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ca0157450 0x7f7ca015a1e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f7c880013e0 tx=0x7f7c88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9d7fa700 1 -- 172.21.2.111:0/4202195371 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c88004900 con 0x7f7ca0157450 2021-04-03T03:13:32.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9d7fa700 1 -- 172.21.2.111:0/4202195371 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c88004a60 con 0x7f7ca0157450 2021-04-03T03:13:32.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9d7fa700 1 -- 172.21.2.111:0/4202195371 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c88004dc0 con 0x7f7ca0157450 2021-04-03T03:13:32.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca0156d10 con 0x7f7ca0157450 2021-04-03T03:13:32.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca01567b0 con 0x7f7ca0157450 2021-04-03T03:13:32.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca0057660 con 0x7f7ca0157450 2021-04-03T03:13:32.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9d7fa700 1 -- 172.21.2.111:0/4202195371 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7c88010070 con 0x7f7ca0157450 2021-04-03T03:13:32.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9d7fa700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c84057000 0x7f7c840594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:32.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9d7fa700 1 -- 172.21.2.111:0/4202195371 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7c88070cc0 con 0x7f7ca0157450 2021-04-03T03:13:32.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.871+0000 7f7c9ffff700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c84057000 0x7f7c840594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:32.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.875+0000 7f7c9ffff700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c84057000 0x7f7c840594c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f7c9000c000 tx=0x7f7c90008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:32.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:32.875+0000 7f7c9d7fa700 1 -- 172.21.2.111:0/4202195371 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7c88045fa0 con 0x7f7ca0157450 2021-04-03T03:13:33.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.003+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7ca0148850 con 0x7f7ca0157450 2021-04-03T03:13:33.934 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:13:33.919+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:13:33.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7c9d7fa700 1 -- 172.21.2.111:0/4202195371 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7c880495c0 con 0x7f7ca0157450 2021-04-03T03:13:33.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c84057000 msgr2=0x7f7c840594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:33.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c84057000 0x7f7c840594c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f7c9000c000 tx=0x7f7c90008040).stop 2021-04-03T03:13:33.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ca0157450 msgr2=0x7f7ca015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:33.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ca0157450 0x7f7ca015a1e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f7c880013e0 tx=0x7f7c88015000).stop 2021-04-03T03:13:33.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 shutdown_connections 2021-04-03T03:13:33.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ca0063720 0x7f7ca015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:33.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c84057000 0x7f7c840594c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:33.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ca00a98d0 0x7f7ca0159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:33.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 --2- 172.21.2.111:0/4202195371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ca0157450 0x7f7ca015a1e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:33.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 >> 172.21.2.111:0/4202195371 conn(0x7f7ca00b61c0 msgr2=0x7f7ca005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:33.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 shutdown_connections 2021-04-03T03:13:33.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.931+0000 7f7ca5135700 1 -- 172.21.2.111:0/4202195371 wait complete. 2021-04-03T03:13:33.951 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:13:33.956 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:13:33.968 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:13:34.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4aacd700 1 Processor -- start 2021-04-03T03:13:34.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4aacd700 1 -- start start 2021-04-03T03:13:34.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4aacd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7efc440603f0 0x7efc440b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4aacd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efc440b0c90 0x7efc44147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4aacd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc440af400 con 0x7efc4414b700 2021-04-03T03:13:34.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4aacd700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc44060860 con 0x7efc440b0c90 2021-04-03T03:13:34.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4aacd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc440af6a0 con 0x7efc440603f0 2021-04-03T03:13:34.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4a2cc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efc440b0c90 0x7efc44147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:34.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4a2cc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efc440b0c90 0x7efc44147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52780/0 (socket says 172.21.2.111:52780) 2021-04-03T03:13:34.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4a2cc700 1 -- 172.21.2.111:0/2196044542 learned_addr learned my addr 172.21.2.111:0/2196044542 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:34.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4a2cc700 1 -- 172.21.2.111:0/2196044542 >> v2:172.21.2.115:3300/0 conn(0x7efc440603f0 msgr2=0x7efc440b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4a2cc700 1 --2- 172.21.2.111:0/2196044542 >> v2:172.21.2.115:3300/0 conn(0x7efc440603f0 0x7efc440b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4a2cc700 1 -- 172.21.2.111:0/2196044542 >> v1:172.21.2.115:6789/0 conn(0x7efc4414b700 legacy=0x7efc440b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.995+0000 7efc4a2cc700 1 -- 172.21.2.111:0/2196044542 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440afb40 con 0x7efc440b0c90 2021-04-03T03:13:34.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4a2cc700 1 --2- 172.21.2.111:0/2196044542 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc44147680 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7efc4406a3b0 tx=0x7efc3c009270).ready entity=mon.0 client_cookie=a8719825d1b3e54f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc48ac9700 1 -- 172.21.2.111:0/2196044542 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc3c00bbc0 con 0x7efc440b0c90 2021-04-03T03:13:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc48ac9700 1 -- 172.21.2.111:0/2196044542 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c00bd20 con 0x7efc440b0c90 2021-04-03T03:13:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc48ac9700 1 -- 172.21.2.111:0/2196044542 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc3c00be90 con 0x7efc440b0c90 2021-04-03T03:13:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/2196044542 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 msgr2=0x7efc44147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 --2- 172.21.2.111:0/2196044542 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc44147680 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7efc4406a3b0 tx=0x7efc3c009270).stop 2021-04-03T03:13:34.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/2196044542 shutdown_connections 2021-04-03T03:13:34.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 --2- 172.21.2.111:0/2196044542 >> v2:172.21.2.115:3300/0 conn(0x7efc440603f0 0x7efc440b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 --2- 172.21.2.111:0/2196044542 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc44147680 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:34.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/2196044542 >> 172.21.2.111:0/2196044542 conn(0x7efc440b61c0 msgr2=0x7efc4405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:34.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/2196044542 shutdown_connections 2021-04-03T03:13:34.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/2196044542 wait complete. 2021-04-03T03:13:34.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 Processor -- start 2021-04-03T03:13:34.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- start start 2021-04-03T03:13:34.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc440603f0 0x7efc440a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc440a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc4414b700 0x7efc440a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc44062d00 con 0x7efc440b0c90 2021-04-03T03:13:34.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc44062fa0 con 0x7efc4414b700 2021-04-03T03:13:34.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc440abb70 con 0x7efc440603f0 2021-04-03T03:13:34.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc440a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:34.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc440a8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52782/0 (socket says 172.21.2.111:52782) 2021-04-03T03:13:34.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 -- 172.21.2.111:0/3056549831 learned_addr learned my addr 172.21.2.111:0/3056549831 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:34.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 -- 172.21.2.111:0/3056549831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc440603f0 msgr2=0x7efc440a81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc440603f0 0x7efc440a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 -- 172.21.2.111:0/3056549831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc4414b700 msgr2=0x7efc440a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc4414b700 0x7efc440a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 -- 172.21.2.111:0/3056549831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440a9590 con 0x7efc440b0c90 2021-04-03T03:13:34.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc492ca700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc440a8bb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7efc34002ed0 tx=0x7efc34006d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:34.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc32ffd700 1 -- 172.21.2.111:0/3056549831 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc3400f510 con 0x7efc440b0c90 2021-04-03T03:13:34.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc32ffd700 1 -- 172.21.2.111:0/3056549831 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3400f670 con 0x7efc440b0c90 2021-04-03T03:13:34.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc32ffd700 1 -- 172.21.2.111:0/3056549831 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc3400f7e0 con 0x7efc440b0c90 2021-04-03T03:13:34.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc440628d0 con 0x7efc440b0c90 2021-04-03T03:13:34.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc44156670 con 0x7efc440b0c90 2021-04-03T03:13:34.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc440ab520 con 0x7efc440b0c90 2021-04-03T03:13:34.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:33.999+0000 7efc32ffd700 1 -- 172.21.2.111:0/3056549831 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7efc3400f070 con 0x7efc440b0c90 2021-04-03T03:13:34.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.003+0000 7efc32ffd700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efc24056ef0 0x7efc240593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.003+0000 7efc32ffd700 1 -- 172.21.2.111:0/3056549831 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6502+0+0 (secure 0 0 0) 0x7efc3406fb50 con 0x7efc440b0c90 2021-04-03T03:13:34.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.003+0000 7efc49acb700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efc24056ef0 0x7efc240593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:34.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.003+0000 7efc49acb700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efc24056ef0 0x7efc240593b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7efc40002f30 tx=0x7efc40006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:34.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.003+0000 7efc32ffd700 1 -- 172.21.2.111:0/3056549831 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7efc34044dd0 con 0x7efc440b0c90 2021-04-03T03:13:34.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc44057660 con 0x7efc440b0c90 2021-04-03T03:13:34.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc32ffd700 1 -- 172.21.2.111:0/3056549831 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+1775 (secure 0 0 0) 0x7efc340483f0 con 0x7efc440b0c90 2021-04-03T03:13:34.145 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:34.145 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4514,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/491320125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":491320125},{"type":"v1","addr":"172.21.2.115:6833","nonce":491320125}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:12:04.020364+0000","modified":"2021-04-03T03:13:33.923327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-04-03T03:13:34.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efc24056ef0 msgr2=0x7efc240593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efc24056ef0 0x7efc240593b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7efc40002f30 tx=0x7efc40006a60).stop 2021-04-03T03:13:34.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 msgr2=0x7efc440a8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc440a8bb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7efc34002ed0 tx=0x7efc34006d30).stop 2021-04-03T03:13:34.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 shutdown_connections 2021-04-03T03:13:34.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efc24056ef0 0x7efc240593b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:34.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc4414b700 0x7efc440a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc440603f0 0x7efc440a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 --2- 172.21.2.111:0/3056549831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc440b0c90 0x7efc440a8bb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:34.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.127+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 >> 172.21.2.111:0/3056549831 conn(0x7efc440b61c0 msgr2=0x7efc44147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:34.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.131+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 shutdown_connections 2021-04-03T03:13:34.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.131+0000 7efc4aacd700 1 -- 172.21.2.111:0/3056549831 wait complete. 2021-04-03T03:13:34.149 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 22 2021-04-03T03:13:34.152 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:13:34.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 Processor -- start 2021-04-03T03:13:34.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- start start 2021-04-03T03:13:34.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200603f0 0x7f62201447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f62200add50 0x7f6220144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62200607d0 con 0x7f62200b17b0 2021-04-03T03:13:34.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62201451c0 con 0x7f62200add50 2021-04-03T03:13:34.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622014b090 con 0x7f62200603f0 2021-04-03T03:13:34.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f62200add50 0x7f6220144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:34.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f62200add50 0x7f6220144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52792/0 (socket says 172.21.2.111:52792) 2021-04-03T03:13:34.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 -- 172.21.2.111:0/501573625 learned_addr learned my addr 172.21.2.111:0/501573625 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:34.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 -- 172.21.2.111:0/501573625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200603f0 msgr2=0x7f62201447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:34.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 --2- 172.21.2.111:0/501573625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200603f0 0x7f62201447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 -- 172.21.2.111:0/501573625 >> v1:172.21.2.115:6789/0 conn(0x7f62200b17b0 legacy=0x7f6220147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:34.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 -- 172.21.2.111:0/501573625 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622014b330 con 0x7f62200add50 2021-04-03T03:13:34.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621e7fc700 1 --2- 172.21.2.111:0/501573625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200add50 0x7f6220144cd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f62200b5600 tx=0x7f6208006e90).ready entity=mon.0 client_cookie=162b8efeb2c46cd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:34.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621dffb700 1 -- 172.21.2.111:0/501573625 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62080098e0 con 0x7f62200add50 2021-04-03T03:13:34.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621dffb700 1 -- 172.21.2.111:0/501573625 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6208009a40 con 0x7f62200add50 2021-04-03T03:13:34.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621dffb700 1 -- 172.21.2.111:0/501573625 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6208009de0 con 0x7f62200add50 2021-04-03T03:13:34.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- 172.21.2.111:0/501573625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200add50 msgr2=0x7f6220144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 --2- 172.21.2.111:0/501573625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200add50 0x7f6220144cd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f62200b5600 tx=0x7f6208006e90).stop 2021-04-03T03:13:34.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- 172.21.2.111:0/501573625 shutdown_connections 2021-04-03T03:13:34.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 --2- 172.21.2.111:0/501573625 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200603f0 0x7f62201447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 --2- 172.21.2.111:0/501573625 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200add50 0x7f6220144cd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:34.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- 172.21.2.111:0/501573625 >> 172.21.2.111:0/501573625 conn(0x7f62200b61c0 msgr2=0x7f622005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:34.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- 172.21.2.111:0/501573625 shutdown_connections 2021-04-03T03:13:34.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.235+0000 7f621ffff700 1 -- 172.21.2.111:0/501573625 wait complete. 2021-04-03T03:13:34.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 Processor -- start 2021-04-03T03:13:34.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 -- start start 2021-04-03T03:13:34.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f62200603f0 0x7f62200a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200add50 0x7f62200abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200b17b0 0x7f62200ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62200a95b0 con 0x7f62200b17b0 2021-04-03T03:13:34.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62200a9850 con 0x7f62200603f0 2021-04-03T03:13:34.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6220063a60 con 0x7f62200add50 2021-04-03T03:13:34.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200b17b0 0x7f62200ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:34.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200b17b0 0x7f62200ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52798/0 (socket says 172.21.2.111:52798) 2021-04-03T03:13:34.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 -- 172.21.2.111:0/2027613822 learned_addr learned my addr 172.21.2.111:0/2027613822 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:34.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 -- 172.21.2.111:0/2027613822 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200add50 msgr2=0x7f62200abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200add50 0x7f62200abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 -- 172.21.2.111:0/2027613822 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f62200603f0 msgr2=0x7f62200a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f62200603f0 0x7f62200a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 -- 172.21.2.111:0/2027613822 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62200ab570 con 0x7f62200b17b0 2021-04-03T03:13:34.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621f7fe700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200b17b0 0x7f62200ab080 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f6218003250 tx=0x7f62180090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:34.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f6217fff700 1 -- 172.21.2.111:0/2027613822 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f621800c900 con 0x7f62200b17b0 2021-04-03T03:13:34.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f6217fff700 1 -- 172.21.2.111:0/2027613822 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621800ca60 con 0x7f62200b17b0 2021-04-03T03:13:34.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f6217fff700 1 -- 172.21.2.111:0/2027613822 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f621800cdc0 con 0x7f62200b17b0 2021-04-03T03:13:34.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6220063f00 con 0x7f62200b17b0 2021-04-03T03:13:34.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62200643c0 con 0x7f62200b17b0 2021-04-03T03:13:34.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62200ac090 con 0x7f62200b17b0 2021-04-03T03:13:34.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f6217fff700 1 -- 172.21.2.111:0/2027613822 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6218015470 con 0x7f62200b17b0 2021-04-03T03:13:34.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f6217fff700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6204056f70 0x7f6204059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:34.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f6217fff700 1 -- 172.21.2.111:0/2027613822 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f62180701b0 con 0x7f62200b17b0 2021-04-03T03:13:34.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621effd700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6204056f70 0x7f6204059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:34.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.239+0000 7f621effd700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6204056f70 0x7f6204059430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f6210002f30 tx=0x7f6210006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:34.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.243+0000 7f6217fff700 1 -- 172.21.2.111:0/2027613822 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f62180453d0 con 0x7f62200b17b0 2021-04-03T03:13:34.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.367+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6220063c20 con 0x7f62200b17b0 2021-04-03T03:13:34.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.935+0000 7f6217fff700 1 -- 172.21.2.111:0/2027613822 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f62180489f0 con 0x7f62200b17b0 2021-04-03T03:13:34.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.935+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6204056f70 msgr2=0x7f6204059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.935+0000 7f621ffff700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6204056f70 0x7f6204059430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f6210002f30 tx=0x7f6210006a60).stop 2021-04-03T03:13:34.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.935+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200b17b0 msgr2=0x7f62200ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:34.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.935+0000 7f621ffff700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200b17b0 0x7f62200ab080 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f6218003250 tx=0x7f62180090f0).stop 2021-04-03T03:13:34.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 shutdown_connections 2021-04-03T03:13:34.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f62200b17b0 0x7f62200ab080 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:34.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6204056f70 0x7f6204059430 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:34.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f62200603f0 0x7f62200a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 --2- 172.21.2.111:0/2027613822 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f62200add50 0x7f62200abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:34.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 >> 172.21.2.111:0/2027613822 conn(0x7f62200b61c0 msgr2=0x7f62200b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:34.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 shutdown_connections 2021-04-03T03:13:34.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:34.939+0000 7f621ffff700 1 -- 172.21.2.111:0/2027613822 wait complete. 2021-04-03T03:13:34.961 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:13:35.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 Processor -- start 2021-04-03T03:13:35.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 -- start start 2021-04-03T03:13:35.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f666000ad10 0x7f66600073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:35.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6660007fb0 0x7f66600288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:35.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666000b0f0 con 0x7f666002a450 2021-04-03T03:13:35.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666002d490 con 0x7f6660007fb0 2021-04-03T03:13:35.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66600417b0 con 0x7f666000ad10 2021-04-03T03:13:35.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f66677fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f666002a450 0x7f66600078a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53602/0 (socket says 172.21.2.111:53602) 2021-04-03T03:13:35.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f66677fe700 1 -- 172.21.2.111:0/4064058588 learned_addr learned my addr 172.21.2.111:0/4064058588 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:35.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666cce3700 1 --2- 172.21.2.111:0/4064058588 >> v2:172.21.2.111:3300/0 conn(0x7f6660007fb0 0x7f66600288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:35.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666cce3700 1 -- 172.21.2.111:0/4064058588 >> v2:172.21.2.115:3300/0 conn(0x7f666000ad10 msgr2=0x7f66600073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:35.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666cce3700 1 --2- 172.21.2.111:0/4064058588 >> v2:172.21.2.115:3300/0 conn(0x7f666000ad10 0x7f66600073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:35.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666cce3700 1 -- 172.21.2.111:0/4064058588 >> v1:172.21.2.111:6789/0 conn(0x7f666002a450 legacy=0x7f66600078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:35.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666cce3700 1 -- 172.21.2.111:0/4064058588 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6660041a50 con 0x7f6660007fb0 2021-04-03T03:13:35.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666cce3700 1 --2- 172.21.2.111:0/4064058588 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6660007fb0 0x7f66600288b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f666806b5d0 tx=0x7f666805ad70).ready entity=mon.0 client_cookie=680d0077bb00fc04 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:35.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f6666ffd700 1 -- 172.21.2.111:0/4064058588 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f666805bda0 con 0x7f6660007fb0 2021-04-03T03:13:35.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f6666ffd700 1 -- 172.21.2.111:0/4064058588 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666805df60 con 0x7f6660007fb0 2021-04-03T03:13:35.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f6666ffd700 1 -- 172.21.2.111:0/4064058588 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66680699f0 con 0x7f6660007fb0 2021-04-03T03:13:35.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 -- 172.21.2.111:0/4064058588 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6660007fb0 msgr2=0x7f66600288b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:35.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.003+0000 7f666d4e4700 1 --2- 172.21.2.111:0/4064058588 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6660007fb0 0x7f66600288b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f666806b5d0 tx=0x7f666805ad70).stop 2021-04-03T03:13:35.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- 172.21.2.111:0/4064058588 shutdown_connections 2021-04-03T03:13:35.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 --2- 172.21.2.111:0/4064058588 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6660007fb0 0x7f66600288b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:35.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 --2- 172.21.2.111:0/4064058588 >> v2:172.21.2.115:3300/0 conn(0x7f666000ad10 0x7f66600073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:35.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- 172.21.2.111:0/4064058588 >> 172.21.2.111:0/4064058588 conn(0x7f66600473d0 msgr2=0x7f66600477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:35.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- 172.21.2.111:0/4064058588 shutdown_connections 2021-04-03T03:13:35.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- 172.21.2.111:0/4064058588 wait complete. 2021-04-03T03:13:35.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 Processor -- start 2021-04-03T03:13:35.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- start start 2021-04-03T03:13:35.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6660007fb0 0x7f666003d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:35.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f666000ad10 0x7f6660040120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:35.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f666002a450 0x7f6660040cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:35.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600411c0 con 0x7f666002a450 2021-04-03T03:13:35.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666002c6f0 con 0x7f666000ad10 2021-04-03T03:13:35.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6660040790 con 0x7f6660007fb0 2021-04-03T03:13:35.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f666002a450 0x7f6660040cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:35.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f666002a450 0x7f6660040cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52812/0 (socket says 172.21.2.111:52812) 2021-04-03T03:13:35.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 -- 172.21.2.111:0/1681072774 learned_addr learned my addr 172.21.2.111:0/1681072774 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:35.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 -- 172.21.2.111:0/1681072774 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6660007fb0 msgr2=0x7f666003d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:35.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6660007fb0 0x7f666003d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:35.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 -- 172.21.2.111:0/1681072774 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f666000ad10 msgr2=0x7f6660040120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:35.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f666000ad10 0x7f6660040120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:35.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 -- 172.21.2.111:0/1681072774 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6660040a30 con 0x7f666002a450 2021-04-03T03:13:35.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666cce3700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f666002a450 0x7f6660040cd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f666806b5d0 tx=0x7f6668064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:35.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f66657fa700 1 -- 172.21.2.111:0/1681072774 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66680441b0 con 0x7f666002a450 2021-04-03T03:13:35.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f66657fa700 1 -- 172.21.2.111:0/1681072774 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666805f2c0 con 0x7f666002a450 2021-04-03T03:13:35.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666003c730 con 0x7f666002a450 2021-04-03T03:13:35.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f66657fa700 1 -- 172.21.2.111:0/1681072774 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f666805f620 con 0x7f666002a450 2021-04-03T03:13:35.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666002b580 con 0x7f666002a450 2021-04-03T03:13:35.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66600051e0 con 0x7f666002a450 2021-04-03T03:13:35.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f66657fa700 1 -- 172.21.2.111:0/1681072774 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f666808c020 con 0x7f666002a450 2021-04-03T03:13:35.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f66657fa700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648057050 0x7f6648059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:35.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f66657fa700 1 -- 172.21.2.111:0/1681072774 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f66680e0a20 con 0x7f666002a450 2021-04-03T03:13:35.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f6667fff700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648057050 0x7f6648059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:35.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.007+0000 7f6667fff700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648057050 0x7f6648059510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f6658002f30 tx=0x7f6658006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:35.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.011+0000 7f66657fa700 1 -- 172.21.2.111:0/1681072774 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f66680b5db0 con 0x7f666002a450 2021-04-03T03:13:35.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.107+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66600008d0 con 0x7f666002a450 2021-04-03T03:13:35.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.939+0000 7f66657fa700 1 -- 172.21.2.111:0/1681072774 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 185+0+0 (secure 0 0 0) 0x7f66680b93d0 con 0x7f666002a450 2021-04-03T03:13:35.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648057050 msgr2=0x7f6648059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:35.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648057050 0x7f6648059510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f6658002f30 tx=0x7f6658006a60).stop 2021-04-03T03:13:35.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f666002a450 msgr2=0x7f6660040cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:35.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f666002a450 0x7f6660040cd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f666806b5d0 tx=0x7f6668064030).stop 2021-04-03T03:13:35.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 shutdown_connections 2021-04-03T03:13:35.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f666002a450 0x7f6660040cd0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:35.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6660007fb0 0x7f666003d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:35.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648057050 0x7f6648059510 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:35.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 --2- 172.21.2.111:0/1681072774 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f666000ad10 0x7f6660040120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:35.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 >> 172.21.2.111:0/1681072774 conn(0x7f66600473d0 msgr2=0x7f66600d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:35.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 shutdown_connections 2021-04-03T03:13:35.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:35.943+0000 7f666d4e4700 1 -- 172.21.2.111:0/1681072774 wait complete. 2021-04-03T03:13:35.961 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:13:35.965 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:13:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f662169b700 1 Processor -- start 2021-04-03T03:13:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f662169b700 1 -- start start 2021-04-03T03:13:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f662169b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6614041060 0x7f66140dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f662169b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f661403d560 0x7f6614041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f662169b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66140dc560 con 0x7f66140dbba0 2021-04-03T03:13:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f662169b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661403de60 con 0x7f6614041060 2021-04-03T03:13:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f662169b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140dc800 con 0x7f661403d560 2021-04-03T03:13:36.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6614041060 0x7f66140dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:36.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6614041060 0x7f66140dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52824/0 (socket says 172.21.2.111:52824) 2021-04-03T03:13:36.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 -- 172.21.2.111:0/3486432832 learned_addr learned my addr 172.21.2.111:0/3486432832 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:36.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f6620e9a700 1 --2- 172.21.2.111:0/3486432832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f661403d560 0x7f6614041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:36.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 -- 172.21.2.111:0/3486432832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f661403d560 msgr2=0x7f6614041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:36.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 --2- 172.21.2.111:0/3486432832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f661403d560 0x7f6614041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:36.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 -- 172.21.2.111:0/3486432832 >> v1:172.21.2.115:6789/0 conn(0x7f66140dbba0 legacy=0x7f661403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:36.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 -- 172.21.2.111:0/3486432832 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661403da30 con 0x7f6614041060 2021-04-03T03:13:36.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661b7fe700 1 --2- 172.21.2.111:0/3486432832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140dbf80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f66140087d0 tx=0x7f6610006e90).ready entity=mon.0 client_cookie=aefb58c53317389f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:36.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661affd700 1 -- 172.21.2.111:0/3486432832 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66100098e0 con 0x7f6614041060 2021-04-03T03:13:36.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.007+0000 7f661affd700 1 -- 172.21.2.111:0/3486432832 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610009a40 con 0x7f6614041060 2021-04-03T03:13:36.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661affd700 1 -- 172.21.2.111:0/3486432832 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6610009de0 con 0x7f6614041060 2021-04-03T03:13:36.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/3486432832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 msgr2=0x7f66140dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:36.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 --2- 172.21.2.111:0/3486432832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140dbf80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f66140087d0 tx=0x7f6610006e90).stop 2021-04-03T03:13:36.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/3486432832 shutdown_connections 2021-04-03T03:13:36.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 --2- 172.21.2.111:0/3486432832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f661403d560 0x7f6614041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:36.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 --2- 172.21.2.111:0/3486432832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140dbf80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:36.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/3486432832 >> 172.21.2.111:0/3486432832 conn(0x7f66140462b0 msgr2=0x7f66140466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:36.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/3486432832 shutdown_connections 2021-04-03T03:13:36.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/3486432832 wait complete. 2021-04-03T03:13:36.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 Processor -- start 2021-04-03T03:13:36.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- start start 2021-04-03T03:13:36.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f661403d560 0x7f661403b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:36.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140d9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:36.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66140dbba0 0x7f66140e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:36.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661403b840 con 0x7f6614041060 2021-04-03T03:13:36.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614039700 con 0x7f661403d560 2021-04-03T03:13:36.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661403b530 con 0x7f66140dbba0 2021-04-03T03:13:36.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:36.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140d9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52828/0 (socket says 172.21.2.111:52828) 2021-04-03T03:13:36.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 -- 172.21.2.111:0/145784483 learned_addr learned my addr 172.21.2.111:0/145784483 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:36.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 -- 172.21.2.111:0/145784483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66140dbba0 msgr2=0x7f66140e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:36.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66140dbba0 0x7f66140e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:36.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 -- 172.21.2.111:0/145784483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f661403d560 msgr2=0x7f661403b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:36.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f661403d560 0x7f661403b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:36.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 -- 172.21.2.111:0/145784483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140d9e00 con 0x7f6614041060 2021-04-03T03:13:36.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661b7fe700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140d9910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f66140087d0 tx=0x7f6610015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:36.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f66197fa700 1 -- 172.21.2.111:0/145784483 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6610011d90 con 0x7f6614041060 2021-04-03T03:13:36.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f66197fa700 1 -- 172.21.2.111:0/145784483 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661001e460 con 0x7f6614041060 2021-04-03T03:13:36.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f66197fa700 1 -- 172.21.2.111:0/145784483 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f661001e7c0 con 0x7f6614041060 2021-04-03T03:13:36.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661403a8b0 con 0x7f6614041060 2021-04-03T03:13:36.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66140e8290 con 0x7f6614041060 2021-04-03T03:13:36.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661403a5c0 con 0x7f6614041060 2021-04-03T03:13:36.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f66197fa700 1 -- 172.21.2.111:0/145784483 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6610025030 con 0x7f6614041060 2021-04-03T03:13:36.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f66197fa700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65fc056f20 0x7f65fc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:36.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f66197fa700 1 -- 172.21.2.111:0/145784483 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6064+0+0 (secure 0 0 0) 0x7f6610071690 con 0x7f6614041060 2021-04-03T03:13:36.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661bfff700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65fc056f20 0x7f65fc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:36.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.011+0000 7f661bfff700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65fc056f20 0x7f65fc0593e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f660c002ef0 tx=0x7f660c006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:36.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.015+0000 7f66197fa700 1 -- 172.21.2.111:0/145784483 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6610046ae0 con 0x7f6614041060 2021-04-03T03:13:36.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.107+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f661403ba40 con 0x7f6614041060 2021-04-03T03:13:36.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.951+0000 7f66197fa700 1 -- 172.21.2.111:0/145784483 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 173+0+0 (secure 0 0 0) 0x7f66100230d0 con 0x7f6614041060 2021-04-03T03:13:36.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65fc056f20 msgr2=0x7f65fc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:36.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65fc056f20 0x7f65fc0593e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f660c002ef0 tx=0x7f660c006910).stop 2021-04-03T03:13:36.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 msgr2=0x7f66140d9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:36.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140d9910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f66140087d0 tx=0x7f6610015000).stop 2021-04-03T03:13:36.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 shutdown_connections 2021-04-03T03:13:36.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66140dbba0 0x7f66140e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:36.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65fc056f20 0x7f65fc0593e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:36.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f661403d560 0x7f661403b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:36.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 --2- 172.21.2.111:0/145784483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6614041060 0x7f66140d9910 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:36.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 >> 172.21.2.111:0/145784483 conn(0x7f66140462b0 msgr2=0x7f6614045700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:36.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 shutdown_connections 2021-04-03T03:13:36.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:36.955+0000 7f662169b700 1 -- 172.21.2.111:0/145784483 wait complete. 2021-04-03T03:13:36.974 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:13:36.978 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:13:36.979 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:13:36.979 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:13:36.979 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:13:36.980 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:13:36.980 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:13:36.980 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:13:36.980 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:13:36.984 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:13:36.984 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:13:36.986 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:13:36.986 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:13:37.027 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:13:37.028 DEBUG:teuthology.parallel:result is None 2021-04-03T03:13:37.030 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:13:37.030 DEBUG:teuthology.parallel:result is None 2021-04-03T03:13:37.031 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:13:37.065 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:13:37.067 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:13:37.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 Processor -- start 2021-04-03T03:13:37.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- start start 2021-04-03T03:13:37.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb9d40603f0 0x7fb9d41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:37.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb9d40add50 0x7fb9d4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:37.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d40607d0 con 0x7fb9d40b17b0 2021-04-03T03:13:37.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d41451c0 con 0x7fb9d40add50 2021-04-03T03:13:37.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d414b090 con 0x7fb9d40603f0 2021-04-03T03:13:37.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb9d40add50 0x7fb9d4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:37.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb9d40add50 0x7fb9d4144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52848/0 (socket says 172.21.2.111:52848) 2021-04-03T03:13:37.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1764300507 learned_addr learned my addr 172.21.2.111:0/1764300507 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:37.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1764300507 >> v2:172.21.2.115:3300/0 conn(0x7fb9d40603f0 msgr2=0x7fb9d41447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:13:37.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 --2- 172.21.2.111:0/1764300507 >> v2:172.21.2.115:3300/0 conn(0x7fb9d40603f0 0x7fb9d41447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:37.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1764300507 >> v1:172.21.2.111:6789/0 conn(0x7fb9d40b17b0 legacy=0x7fb9d4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:37.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1764300507 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d414b330 con 0x7fb9d40add50 2021-04-03T03:13:37.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d1d9b700 1 --2- 172.21.2.111:0/1764300507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d4144cd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb9d40ae520 tx=0x7fb9bc006e90).ready entity=mon.0 client_cookie=a207570717aa706a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:37.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d159a700 1 -- 172.21.2.111:0/1764300507 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9bc009930 con 0x7fb9d40add50 2021-04-03T03:13:37.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d159a700 1 -- 172.21.2.111:0/1764300507 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc009a90 con 0x7fb9d40add50 2021-04-03T03:13:37.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d159a700 1 -- 172.21.2.111:0/1764300507 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9bc009e30 con 0x7fb9d40add50 2021-04-03T03:13:37.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- 172.21.2.111:0/1764300507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 msgr2=0x7fb9d4144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:37.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1764300507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d4144cd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb9d40ae520 tx=0x7fb9bc006e90).stop 2021-04-03T03:13:37.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- 172.21.2.111:0/1764300507 shutdown_connections 2021-04-03T03:13:37.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1764300507 >> v2:172.21.2.115:3300/0 conn(0x7fb9d40603f0 0x7fb9d41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:37.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1764300507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d4144cd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:37.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- 172.21.2.111:0/1764300507 >> 172.21.2.111:0/1764300507 conn(0x7fb9d40b61c0 msgr2=0x7fb9d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:37.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- 172.21.2.111:0/1764300507 shutdown_connections 2021-04-03T03:13:37.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.071+0000 7fb9d359e700 1 -- 172.21.2.111:0/1764300507 wait complete. 2021-04-03T03:13:37.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 Processor -- start 2021-04-03T03:13:37.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 -- start start 2021-04-03T03:13:37.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9d40603f0 0x7fb9d40aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:37.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d40a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:37.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9d40b17b0 0x7fb9d4157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:37.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40aadb0 con 0x7fb9d40add50 2021-04-03T03:13:37.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40ab010 con 0x7fb9d40603f0 2021-04-03T03:13:37.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40abad0 con 0x7fb9d40b17b0 2021-04-03T03:13:37.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d40a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:37.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d40a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52854/0 (socket says 172.21.2.111:52854) 2021-04-03T03:13:37.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1088695922 learned_addr learned my addr 172.21.2.111:0/1088695922 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:37.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1088695922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9d40b17b0 msgr2=0x7fb9d4157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:37.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9d40b17b0 0x7fb9d4157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:37.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1088695922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9d40603f0 msgr2=0x7fb9d40aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:37.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9d40603f0 0x7fb9d40aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:37.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 -- 172.21.2.111:0/1088695922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40a7ea0 con 0x7fb9d40add50 2021-04-03T03:13:37.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d1d9b700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d40a79b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb9d40ae520 tx=0x7fb9bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:37.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9cb7fe700 1 -- 172.21.2.111:0/1088695922 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9bc01e460 con 0x7fb9d40add50 2021-04-03T03:13:37.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9cb7fe700 1 -- 172.21.2.111:0/1088695922 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc01e5c0 con 0x7fb9d40add50 2021-04-03T03:13:37.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9cb7fe700 1 -- 172.21.2.111:0/1088695922 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9bc01e920 con 0x7fb9d40add50 2021-04-03T03:13:37.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d4147570 con 0x7fb9d40add50 2021-04-03T03:13:37.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d41481e0 con 0x7fb9d40add50 2021-04-03T03:13:37.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d40abc90 con 0x7fb9d40add50 2021-04-03T03:13:37.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9cb7fe700 1 -- 172.21.2.111:0/1088695922 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb9bc025030 con 0x7fb9d40add50 2021-04-03T03:13:37.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9cb7fe700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b8057000 0x7fb9b80594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:37.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9cb7fe700 1 -- 172.21.2.111:0/1088695922 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fb9bc071600 con 0x7fb9d40add50 2021-04-03T03:13:37.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d259c700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b8057000 0x7fb9b80594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:37.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.075+0000 7fb9d259c700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b8057000 0x7fb9b80594c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fb9c4002f70 tx=0x7fb9c4006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:37.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.079+0000 7fb9cb7fe700 1 -- 172.21.2.111:0/1088695922 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb9bc046bb0 con 0x7fb9d40add50 2021-04-03T03:13:37.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.183+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb9d40abe90 con 0x7fb9d40add50 2021-04-03T03:13:37.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9cb7fe700 1 -- 172.21.2.111:0/1088695922 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb9bc023080 con 0x7fb9d40add50 2021-04-03T03:13:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b8057000 msgr2=0x7fb9b80594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b8057000 0x7fb9b80594c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fb9c4002f70 tx=0x7fb9c4006990).stop 2021-04-03T03:13:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 msgr2=0x7fb9d40a79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d40a79b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb9d40ae520 tx=0x7fb9bc015000).stop 2021-04-03T03:13:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 shutdown_connections 2021-04-03T03:13:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb9d40b17b0 0x7fb9d4157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb9b8057000 0x7fb9b80594c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb9d40603f0 0x7fb9d40aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:37.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 --2- 172.21.2.111:0/1088695922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb9d40add50 0x7fb9d40a79b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:37.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 >> 172.21.2.111:0/1088695922 conn(0x7fb9d40b61c0 msgr2=0x7fb9d41447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:37.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 shutdown_connections 2021-04-03T03:13:37.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:37.967+0000 7fb9d359e700 1 -- 172.21.2.111:0/1088695922 wait complete. 2021-04-03T03:13:37.987 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:13:37.991 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:13:37.991 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:13:38.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 Processor -- start 2021-04-03T03:13:38.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- start start 2021-04-03T03:13:38.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2ccc006bc0 0x7f2ccc042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc03d360 0x7f2ccc0dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ccc0dc6e0 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ccc0d9560 con 0x7f2ccc006bc0 2021-04-03T03:13:38.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc03dcb0 con 0x7f2ccc03d360 2021-04-03T03:13:38.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2ccc006bc0 0x7f2ccc042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2ccc006bc0 0x7f2ccc042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52862/0 (socket says 172.21.2.111:52862) 2021-04-03T03:13:38.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/796554793 learned_addr learned my addr 172.21.2.111:0/796554793 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:38.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/796554793 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc03d360 msgr2=0x7f2ccc0dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 --2- 172.21.2.111:0/796554793 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc03d360 0x7f2ccc0dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/796554793 >> v1:172.21.2.115:6789/0 conn(0x7f2ccc0dbb90 legacy=0x7f2ccc0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/796554793 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc0dc460 con 0x7f2ccc006bc0 2021-04-03T03:13:38.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cd8cf8700 1 --2- 172.21.2.111:0/796554793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc006bc0 0x7f2ccc042750 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f2ccc046210 tx=0x7f2cc8006e90).ready entity=mon.0 client_cookie=e74096f8e5f0ce52 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cc7fff700 1 -- 172.21.2.111:0/796554793 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cc80098a0 con 0x7f2ccc006bc0 2021-04-03T03:13:38.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cc7fff700 1 -- 172.21.2.111:0/796554793 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8009a00 con 0x7f2ccc006bc0 2021-04-03T03:13:38.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cc7fff700 1 -- 172.21.2.111:0/796554793 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cc8009da0 con 0x7f2ccc006bc0 2021-04-03T03:13:38.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- 172.21.2.111:0/796554793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc006bc0 msgr2=0x7f2ccc042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/796554793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc006bc0 0x7f2ccc042750 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f2ccc046210 tx=0x7f2cc8006e90).stop 2021-04-03T03:13:38.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- 172.21.2.111:0/796554793 shutdown_connections 2021-04-03T03:13:38.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/796554793 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc03d360 0x7f2ccc0dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/796554793 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc006bc0 0x7f2ccc042750 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- 172.21.2.111:0/796554793 >> 172.21.2.111:0/796554793 conn(0x7f2ccc0462c0 msgr2=0x7f2ccc0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- 172.21.2.111:0/796554793 shutdown_connections 2021-04-03T03:13:38.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- 172.21.2.111:0/796554793 wait complete. 2021-04-03T03:13:38.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 Processor -- start 2021-04-03T03:13:38.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.079+0000 7f2cda4fb700 1 -- start start 2021-04-03T03:13:38.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2ccc03d360 0x7f2ccc0e7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc0dbb90 0x7f2ccc0d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc0da2d0 0x7f2ccc0e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0db290 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0db4c0 con 0x7f2ccc03d360 2021-04-03T03:13:38.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0d4690 con 0x7f2ccc0da2d0 2021-04-03T03:13:38.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc0dbb90 0x7f2ccc0d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc0dbb90 0x7f2ccc0d41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52868/0 (socket says 172.21.2.111:52868) 2021-04-03T03:13:38.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/3172114812 learned_addr learned my addr 172.21.2.111:0/3172114812 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:38.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/3172114812 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc0da2d0 msgr2=0x7f2ccc0e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc0da2d0 0x7f2ccc0e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/3172114812 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2ccc03d360 msgr2=0x7f2ccc0e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2ccc03d360 0x7f2ccc0e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 -- 172.21.2.111:0/3172114812 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc0d6170 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd8cf8700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc0dbb90 0x7f2ccc0d41a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f2ccc046210 tx=0x7f2cc8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cc67fc700 1 -- 172.21.2.111:0/3172114812 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cc8011ce0 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cc67fc700 1 -- 172.21.2.111:0/3172114812 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8011e40 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ccc0d9770 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cc67fc700 1 -- 172.21.2.111:0/3172114812 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cc801d640 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc0e8190 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc0db680 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cc67fc700 1 -- 172.21.2.111:0/3172114812 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2cc800f070 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cc67fc700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2cb4056f20 0x7f2cb40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cc67fc700 1 -- 172.21.2.111:0/3172114812 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f2cc8071c20 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd94f9700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2cb4056f20 0x7f2cb40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.083+0000 7f2cd94f9700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2cb4056f20 0x7f2cb40593e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f2cd0002eb0 tx=0x7f2cd00068f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.087+0000 7f2cc67fc700 1 -- 172.21.2.111:0/3172114812 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2cc8047230 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2ccc004da0 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cc67fc700 1 -- 172.21.2.111:0/3172114812 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f2cc8025360 con 0x7f2ccc0dbb90 2021-04-03T03:13:38.231 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:38.231 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:13:38.231 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:13:38.232 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.232 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:13:38.232 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:13:38.232 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.232 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:13:38.233 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.233 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.233 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.233 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.234 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.234 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.234 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:13:38.234 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:13:38.234 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.235 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:13:38.235 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.235 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.235 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.236 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.236 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.236 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.236 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:13:38.236 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:13:38.237 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.237 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.237 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.237 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.237 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.238 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.238 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.238 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:13:38.238 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:13:38.239 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.239 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.239 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.239 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.240 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.240 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.240 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.240 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:13:38.240 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:13:38.241 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.241 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.241 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.241 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.241 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.242 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.242 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.242 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:13:38.242 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:13:38.242 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.243 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.243 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.243 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.243 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.244 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.244 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.244 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:13:38.244 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:13:38.244 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.245 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.245 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.245 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.245 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.245 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.246 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.246 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:13:38.246 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:13:38.246 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.247 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.247 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.247 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.247 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.248 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.248 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:13:38.248 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:13:38.248 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.248 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.249 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.249 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.249 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.249 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.250 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.250 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:13:38.250 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:13:38.250 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.250 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.251 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.251 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.251 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.251 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:13:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:13:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:13:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:13:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:13:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.253 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:13:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:13:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:13:38.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:13:38.255 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:13:38.255 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.255 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.255 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.255 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:13:38.256 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:13:38.256 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:13:38.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:13:38.257 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:13:38.257 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.257 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.257 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.257 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:13:38.258 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:13:38.258 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:13:38.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:13:38.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:13:38.259 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.259 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.259 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.259 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.259 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:13:38.260 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:13:38.260 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:13:38.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:13:38.261 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.261 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.261 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.261 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.261 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:13:38.262 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:13:38.262 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:13:38.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:13:38.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:13:38.262 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:13:38.263 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.263 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.263 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:13:38.263 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:13:38.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2cb4056f20 msgr2=0x7f2cb40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2cb4056f20 0x7f2cb40593e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f2cd0002eb0 tx=0x7f2cd00068f0).stop 2021-04-03T03:13:38.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc0dbb90 msgr2=0x7f2ccc0d41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc0dbb90 0x7f2ccc0d41a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f2ccc046210 tx=0x7f2cc8015000).stop 2021-04-03T03:13:38.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 shutdown_connections 2021-04-03T03:13:38.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ccc0da2d0 0x7f2ccc0e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2cb4056f20 0x7f2cb40593e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2ccc03d360 0x7f2ccc0e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 --2- 172.21.2.111:0/3172114812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ccc0dbb90 0x7f2ccc0d41a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 >> 172.21.2.111:0/3172114812 conn(0x7f2ccc0462c0 msgr2=0x7f2ccc0441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 shutdown_connections 2021-04-03T03:13:38.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.215+0000 7f2cda4fb700 1 -- 172.21.2.111:0/3172114812 wait complete. 2021-04-03T03:13:38.268 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:13:38.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.319+0000 7f6ffe53f700 1 Processor -- start 2021-04-03T03:13:38.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.319+0000 7f6ffe53f700 1 -- start start 2021-04-03T03:13:38.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6ff00db4a0 0x7f6ff0042410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6ff0040860 0x7f6ff003edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff0047520 con 0x7f6ff0047140 2021-04-03T03:13:38.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff003d540 con 0x7f6ff00db4a0 2021-04-03T03:13:38.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff00410b0 con 0x7f6ff0040860 2021-04-03T03:13:38.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6ff00db4a0 0x7f6ff0042410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffd53d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6ff0047140 0x7f6ff00dad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53664/0 (socket says 172.21.2.111:53664) 2021-04-03T03:13:38.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6ff00db4a0 0x7f6ff0042410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52872/0 (socket says 172.21.2.111:52872) 2021-04-03T03:13:38.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffd53d700 1 -- 172.21.2.111:0/3018340287 learned_addr learned my addr 172.21.2.111:0/3018340287 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:38.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffdd3e700 1 --2- 172.21.2.111:0/3018340287 >> v2:172.21.2.115:3300/0 conn(0x7f6ff0040860 0x7f6ff003edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 -- 172.21.2.111:0/3018340287 >> v2:172.21.2.115:3300/0 conn(0x7f6ff0040860 msgr2=0x7f6ff003edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- 172.21.2.111:0/3018340287 >> v2:172.21.2.115:3300/0 conn(0x7f6ff0040860 0x7f6ff003edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 -- 172.21.2.111:0/3018340287 >> v1:172.21.2.111:6789/0 conn(0x7f6ff0047140 legacy=0x7f6ff00dad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 -- 172.21.2.111:0/3018340287 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff003da30 con 0x7f6ff00db4a0 2021-04-03T03:13:38.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- 172.21.2.111:0/3018340287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff00db4a0 0x7f6ff0042410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f6ff4002f30 tx=0x7f6ff4006dd0).ready entity=mon.0 client_cookie=8e5ba6b88cd0f866 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6febfff700 1 -- 172.21.2.111:0/3018340287 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff40097f0 con 0x7f6ff00db4a0 2021-04-03T03:13:38.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6febfff700 1 -- 172.21.2.111:0/3018340287 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4009950 con 0x7f6ff00db4a0 2021-04-03T03:13:38.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6febfff700 1 -- 172.21.2.111:0/3018340287 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff4009cb0 con 0x7f6ff00db4a0 2021-04-03T03:13:38.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- 172.21.2.111:0/3018340287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff00db4a0 msgr2=0x7f6ff0042410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/3018340287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff00db4a0 0x7f6ff0042410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f6ff4002f30 tx=0x7f6ff4006dd0).stop 2021-04-03T03:13:38.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- 172.21.2.111:0/3018340287 shutdown_connections 2021-04-03T03:13:38.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/3018340287 >> v2:172.21.2.115:3300/0 conn(0x7f6ff0040860 0x7f6ff003edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/3018340287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff00db4a0 0x7f6ff0042410 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- 172.21.2.111:0/3018340287 >> 172.21.2.111:0/3018340287 conn(0x7f6ff000d520 msgr2=0x7f6ff0046710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- 172.21.2.111:0/3018340287 shutdown_connections 2021-04-03T03:13:38.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- 172.21.2.111:0/3018340287 wait complete. 2021-04-03T03:13:38.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 Processor -- start 2021-04-03T03:13:38.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- start start 2021-04-03T03:13:38.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ff0040860 0x7f6ff00a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff0047140 0x7f6ff0039fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ff00db4a0 0x7f6ff00a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00e7f40 con 0x7f6ff0047140 2021-04-03T03:13:38.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00e81a0 con 0x7f6ff0040860 2021-04-03T03:13:38.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffe53f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00e8440 con 0x7f6ff00db4a0 2021-04-03T03:13:38.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff0047140 0x7f6ff0039fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff0047140 0x7f6ff0039fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52880/0 (socket says 172.21.2.111:52880) 2021-04-03T03:13:38.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 -- 172.21.2.111:0/2212901955 learned_addr learned my addr 172.21.2.111:0/2212901955 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:38.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 -- 172.21.2.111:0/2212901955 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ff00db4a0 msgr2=0x7f6ff00a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ff00db4a0 0x7f6ff00a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 -- 172.21.2.111:0/2212901955 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ff0040860 msgr2=0x7f6ff00a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ff0040860 0x7f6ff00a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 -- 172.21.2.111:0/2212901955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff00a1400 con 0x7f6ff0047140 2021-04-03T03:13:38.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.323+0000 7f6ffcd3c700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff0047140 0x7f6ff0039fe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f6ff4002f30 tx=0x7f6ff4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6fea7fc700 1 -- 172.21.2.111:0/2212901955 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff4011ce0 con 0x7f6ff0047140 2021-04-03T03:13:38.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6fea7fc700 1 -- 172.21.2.111:0/2212901955 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4011e40 con 0x7f6ff0047140 2021-04-03T03:13:38.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff00d9930 con 0x7f6ff0047140 2021-04-03T03:13:38.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6fea7fc700 1 -- 172.21.2.111:0/2212901955 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff401e650 con 0x7f6ff0047140 2021-04-03T03:13:38.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff00d9df0 con 0x7f6ff0047140 2021-04-03T03:13:38.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff00e8600 con 0x7f6ff0047140 2021-04-03T03:13:38.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6fea7fc700 1 -- 172.21.2.111:0/2212901955 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6ff401a040 con 0x7f6ff0047140 2021-04-03T03:13:38.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6fea7fc700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6fe0057070 0x7f6fe0059530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6fea7fc700 1 -- 172.21.2.111:0/2212901955 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f6ff4070f40 con 0x7f6ff0047140 2021-04-03T03:13:38.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6ffd53d700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6fe0057070 0x7f6fe0059530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6ffd53d700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6fe0057070 0x7f6fe0059530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f6ff0040dd0 tx=0x7f6fec0077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.327+0000 7f6fea7fc700 1 -- 172.21.2.111:0/2212901955 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6ff4046660 con 0x7f6ff0047140 2021-04-03T03:13:38.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.423+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ff00da3f0 con 0x7f6ff0047140 2021-04-03T03:13:38.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.423+0000 7f6fea7fc700 1 -- 172.21.2.111:0/2212901955 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f6ff4049c80 con 0x7f6ff0047140 2021-04-03T03:13:38.440 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:38.441 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":58,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:13:37.958227+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:13:38.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6fe0057070 msgr2=0x7f6fe0059530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6fe0057070 0x7f6fe0059530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f6ff0040dd0 tx=0x7f6fec0077e0).stop 2021-04-03T03:13:38.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff0047140 msgr2=0x7f6ff0039fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff0047140 0x7f6ff0039fe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f6ff4002f30 tx=0x7f6ff4015000).stop 2021-04-03T03:13:38.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 shutdown_connections 2021-04-03T03:13:38.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ff00db4a0 0x7f6ff00a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6fe0057070 0x7f6fe0059530 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ff0040860 0x7f6ff00a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 --2- 172.21.2.111:0/2212901955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ff0047140 0x7f6ff0039fe0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 >> 172.21.2.111:0/2212901955 conn(0x7f6ff000d520 msgr2=0x7f6ff00431e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 shutdown_connections 2021-04-03T03:13:38.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.427+0000 7f6ffe53f700 1 -- 172.21.2.111:0/2212901955 wait complete. 2021-04-03T03:13:38.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:13:38.450 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:13:38.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6996119700 1 Processor -- start 2021-04-03T03:13:38.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6996119700 1 -- start start 2021-04-03T03:13:38.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6996119700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f69880dbba0 0x7f69880dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6996119700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f698803d560 0x7f6988041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6996119700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69880dc560 con 0x7f6988041060 2021-04-03T03:13:38.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6996119700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698803de60 con 0x7f69880dbba0 2021-04-03T03:13:38.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6996119700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69880dc800 con 0x7f698803d560 2021-04-03T03:13:38.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f69880dbba0 0x7f69880dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f69880dbba0 0x7f69880dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52886/0 (socket says 172.21.2.111:52886) 2021-04-03T03:13:38.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 -- 172.21.2.111:0/4163254066 learned_addr learned my addr 172.21.2.111:0/4163254066 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:38.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 -- 172.21.2.111:0/4163254066 >> v2:172.21.2.115:3300/0 conn(0x7f698803d560 msgr2=0x7f6988041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 --2- 172.21.2.111:0/4163254066 >> v2:172.21.2.115:3300/0 conn(0x7f698803d560 0x7f6988041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 -- 172.21.2.111:0/4163254066 >> v1:172.21.2.115:6789/0 conn(0x7f6988041060 legacy=0x7f698803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 -- 172.21.2.111:0/4163254066 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698803da30 con 0x7f69880dbba0 2021-04-03T03:13:38.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.491+0000 7f6995117700 1 --2- 172.21.2.111:0/4163254066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69880dbba0 0x7f69880dbf80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f69880087d0 tx=0x7f698c006e90).ready entity=mon.0 client_cookie=6be40d94e2e0e2fc server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697ffff700 1 -- 172.21.2.111:0/4163254066 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f698c009930 con 0x7f69880dbba0 2021-04-03T03:13:38.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697ffff700 1 -- 172.21.2.111:0/4163254066 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c009a90 con 0x7f69880dbba0 2021-04-03T03:13:38.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697ffff700 1 -- 172.21.2.111:0/4163254066 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f698c009e30 con 0x7f69880dbba0 2021-04-03T03:13:38.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/4163254066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69880dbba0 msgr2=0x7f69880dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 --2- 172.21.2.111:0/4163254066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69880dbba0 0x7f69880dbf80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f69880087d0 tx=0x7f698c006e90).stop 2021-04-03T03:13:38.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/4163254066 shutdown_connections 2021-04-03T03:13:38.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 --2- 172.21.2.111:0/4163254066 >> v2:172.21.2.115:3300/0 conn(0x7f698803d560 0x7f6988041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 --2- 172.21.2.111:0/4163254066 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69880dbba0 0x7f69880dbf80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/4163254066 >> 172.21.2.111:0/4163254066 conn(0x7f69880462b0 msgr2=0x7f69880466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/4163254066 shutdown_connections 2021-04-03T03:13:38.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/4163254066 wait complete. 2021-04-03T03:13:38.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 Processor -- start 2021-04-03T03:13:38.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- start start 2021-04-03T03:13:38.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f698803d560 0x7f69880db000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6988041060 0x7f69880d5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69880dbba0 0x7f69880ea990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698803c450 con 0x7f698803d560 2021-04-03T03:13:38.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988039580 con 0x7f69880dbba0 2021-04-03T03:13:38.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880e7e00 con 0x7f6988041060 2021-04-03T03:13:38.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f698803d560 0x7f69880db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f698803d560 0x7f69880db000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52892/0 (socket says 172.21.2.111:52892) 2021-04-03T03:13:38.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 -- 172.21.2.111:0/1926917418 learned_addr learned my addr 172.21.2.111:0/1926917418 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:38.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 -- 172.21.2.111:0/1926917418 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6988041060 msgr2=0x7f69880d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6988041060 0x7f69880d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 -- 172.21.2.111:0/1926917418 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69880dbba0 msgr2=0x7f69880ea990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69880dbba0 0x7f69880ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 -- 172.21.2.111:0/1926917418 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880e80a0 con 0x7f698803d560 2021-04-03T03:13:38.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6995117700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f698803d560 0x7f69880db000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f69880087d0 tx=0x7f698c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697e7fc700 1 -- 172.21.2.111:0/1926917418 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f698c011ce0 con 0x7f698803d560 2021-04-03T03:13:38.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697e7fc700 1 -- 172.21.2.111:0/1926917418 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c011e40 con 0x7f698803d560 2021-04-03T03:13:38.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697e7fc700 1 -- 172.21.2.111:0/1926917418 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f698c01e650 con 0x7f698803d560 2021-04-03T03:13:38.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698803c0a0 con 0x7f698803d560 2021-04-03T03:13:38.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69880ded70 con 0x7f698803d560 2021-04-03T03:13:38.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6988004db0 con 0x7f698803d560 2021-04-03T03:13:38.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697e7fc700 1 -- 172.21.2.111:0/1926917418 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f698c01a040 con 0x7f698803d560 2021-04-03T03:13:38.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697e7fc700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6970057070 0x7f6970059530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f697e7fc700 1 -- 172.21.2.111:0/1926917418 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f698c071020 con 0x7f698803d560 2021-04-03T03:13:38.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6994916700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6970057070 0x7f6970059530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.495+0000 7f6994916700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6970057070 0x7f6970059530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f6980002f50 tx=0x7f6980006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.499+0000 7f697e7fc700 1 -- 172.21.2.111:0/1926917418 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f698c046740 con 0x7f698803d560 2021-04-03T03:13:38.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.591+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f69880083c0 con 0x7f698803d560 2021-04-03T03:13:38.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f697e7fc700 1 -- 172.21.2.111:0/1926917418 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v58) v1 ==== 110+0+10 (secure 0 0 0) 0x7f698c023020 con 0x7f698803d560 2021-04-03T03:13:38.610 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:13:38.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6970057070 msgr2=0x7f6970059530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6970057070 0x7f6970059530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f6980002f50 tx=0x7f6980006990).stop 2021-04-03T03:13:38.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f698803d560 msgr2=0x7f69880db000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f698803d560 0x7f69880db000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f69880087d0 tx=0x7f698c015000).stop 2021-04-03T03:13:38.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 shutdown_connections 2021-04-03T03:13:38.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6970057070 0x7f6970059530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69880dbba0 0x7f69880ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f698803d560 0x7f69880db000 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 --2- 172.21.2.111:0/1926917418 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6988041060 0x7f69880d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 >> 172.21.2.111:0/1926917418 conn(0x7f69880462b0 msgr2=0x7f69880055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 shutdown_connections 2021-04-03T03:13:38.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.595+0000 7f6996119700 1 -- 172.21.2.111:0/1926917418 wait complete. 2021-04-03T03:13:38.618 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:13:38.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7ae87700 1 Processor -- start 2021-04-03T03:13:38.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7ae87700 1 -- start start 2021-04-03T03:13:38.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7ae87700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c740add50 0x7f2c74147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7ae87700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c740607d0 con 0x7f2c74144ef0 2021-04-03T03:13:38.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7ae87700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7414b090 con 0x7f2c740603f0 2021-04-03T03:13:38.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7ae87700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7414b330 con 0x7f2c740add50 2021-04-03T03:13:38.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7a686700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f2c74144ef0 0x7f2c74147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53696/0 (socket says 172.21.2.111:53696) 2021-04-03T03:13:38.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c7a686700 1 -- 172.21.2.111:0/2654587190 learned_addr learned my addr 172.21.2.111:0/2654587190 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:38.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035668979 0 0) 0x7f2c740607d0 con 0x7f2c74144ef0 2021-04-03T03:13:38.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c5c003240 con 0x7f2c74144ef0 2021-04-03T03:13:38.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 134403130 0 0) 0x7f2c5c003240 con 0x7f2c74144ef0 2021-04-03T03:13:38.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c740add50 msgr2=0x7f2c74147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c78e83700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c740add50 0x7f2c74147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.659+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 >> v1:172.21.2.115:6789/0 conn(0x7f2c740603f0 legacy=0x7f2c741447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7414b5d0 con 0x7f2c74144ef0 2021-04-03T03:13:38.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f2c6c001a50 con 0x7f2c74144ef0 2021-04-03T03:13:38.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 >> v1:172.21.2.111:6789/0 conn(0x7f2c74144ef0 legacy=0x7f2c74147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c740add50 0x7f2c5c003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c5c003c50 0x7f2c5c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c5c0049d0 0x7f2c5c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7414b330 con 0x7f2c5c0049d0 2021-04-03T03:13:38.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0041e0 con 0x7f2c5c003c50 2021-04-03T03:13:38.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c00b730 con 0x7f2c740add50 2021-04-03T03:13:38.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c5c0049d0 0x7f2c5c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c5c003c50 0x7f2c5c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 -- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c740add50 msgr2=0x7f2c5c003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c740add50 0x7f2c5c003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 -- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c5c003c50 msgr2=0x7f2c5c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c5c003c50 0x7f2c5c0044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 -- 172.21.2.111:0/2654587190 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c003240 con 0x7f2c5c0049d0 2021-04-03T03:13:38.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c5c0049d0 0x7f2c5c004e50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f2c740b5640 tx=0x7f2c7414b760).ready entity=mon.0 client_cookie=c32a67411d1b7478 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c6c005b80 con 0x7f2c5c0049d0 2021-04-03T03:13:38.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c005ce0 con 0x7f2c5c0049d0 2021-04-03T03:13:38.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c78e83700 1 -- 172.21.2.111:0/2654587190 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c6c0072c0 con 0x7f2c5c0049d0 2021-04-03T03:13:38.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- 172.21.2.111:0/2654587190 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c5c0049d0 msgr2=0x7f2c5c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 --2- 172.21.2.111:0/2654587190 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c5c0049d0 0x7f2c5c004e50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f2c740b5640 tx=0x7f2c7414b760).stop 2021-04-03T03:13:38.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 -- 172.21.2.111:0/2654587190 reap_dead start 2021-04-03T03:13:38.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- 172.21.2.111:0/2654587190 shutdown_connections 2021-04-03T03:13:38.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- 172.21.2.111:0/2654587190 >> 172.21.2.111:0/2654587190 conn(0x7f2c740b61c0 msgr2=0x7f2c7405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- 172.21.2.111:0/2654587190 shutdown_connections 2021-04-03T03:13:38.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- 172.21.2.111:0/2654587190 wait complete. 2021-04-03T03:13:38.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 Processor -- start 2021-04-03T03:13:38.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- start start 2021-04-03T03:13:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c741469c0 0x7f2c74147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c740a82e0 0x7f2c740a8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c74159e50 0x7f2c7415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740a8c50 con 0x7f2c740a82e0 2021-04-03T03:13:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74146da0 con 0x7f2c741469c0 2021-04-03T03:13:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c741478f0 con 0x7f2c74159e50 2021-04-03T03:13:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c740a82e0 0x7f2c740a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c740a82e0 0x7f2c740a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52914/0 (socket says 172.21.2.111:52914) 2021-04-03T03:13:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 -- 172.21.2.111:0/3710727031 learned_addr learned my addr 172.21.2.111:0/3710727031 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79e85700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c741469c0 0x7f2c74147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7a686700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c74159e50 0x7f2c7415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 -- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c74159e50 msgr2=0x7f2c7415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c74159e50 0x7f2c7415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 -- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c741469c0 msgr2=0x7f2c74147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c741469c0 0x7f2c74147250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 -- 172.21.2.111:0/3710727031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c74147b90 con 0x7f2c740a82e0 2021-04-03T03:13:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79e85700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c741469c0 0x7f2c74147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:13:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c79684700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c740a82e0 0x7f2c740a8760 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f2c70007000 tx=0x7f2c70008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c66ffd700 1 -- 172.21.2.111:0/3710727031 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c7000e5e0 con 0x7f2c740a82e0 2021-04-03T03:13:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c66ffd700 1 -- 172.21.2.111:0/3710727031 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7000e740 con 0x7f2c740a82e0 2021-04-03T03:13:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c740a7e40 con 0x7f2c740a82e0 2021-04-03T03:13:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c66ffd700 1 -- 172.21.2.111:0/3710727031 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c7000eae0 con 0x7f2c740a82e0 2021-04-03T03:13:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.663+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c74146150 con 0x7f2c740a82e0 2021-04-03T03:13:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.667+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c74147740 con 0x7f2c740a82e0 2021-04-03T03:13:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.667+0000 7f2c66ffd700 1 -- 172.21.2.111:0/3710727031 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2c7000ec40 con 0x7f2c740a82e0 2021-04-03T03:13:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.667+0000 7f2c66ffd700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2c540570c0 0x7f2c54059580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.667+0000 7f2c66ffd700 1 -- 172.21.2.111:0/3710727031 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f2c700701a0 con 0x7f2c740a82e0 2021-04-03T03:13:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.667+0000 7f2c79e85700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2c540570c0 0x7f2c54059580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.667+0000 7f2c79e85700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2c540570c0 0x7f2c54059580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2c68008000 tx=0x7f2c68009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.667+0000 7f2c66ffd700 1 -- 172.21.2.111:0/3710727031 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2c700457b0 con 0x7f2c740a82e0 2021-04-03T03:13:38.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.759+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c740a7fd0 con 0x7f2c740a82e0 2021-04-03T03:13:38.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.759+0000 7f2c66ffd700 1 -- 172.21.2.111:0/3710727031 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c70048dd0 con 0x7f2c740a82e0 2021-04-03T03:13:38.777 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:13:38.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2c540570c0 msgr2=0x7f2c54059580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2c540570c0 0x7f2c54059580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2c68008000 tx=0x7f2c68009040).stop 2021-04-03T03:13:38.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c740a82e0 msgr2=0x7f2c740a8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c740a82e0 0x7f2c740a8760 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f2c70007000 tx=0x7f2c70008040).stop 2021-04-03T03:13:38.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 shutdown_connections 2021-04-03T03:13:38.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2c74159e50 0x7f2c7415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2c540570c0 0x7f2c54059580 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2c741469c0 0x7f2c74147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 --2- 172.21.2.111:0/3710727031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2c740a82e0 0x7f2c740a8760 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 >> 172.21.2.111:0/3710727031 conn(0x7f2c740b61c0 msgr2=0x7f2c74142740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 shutdown_connections 2021-04-03T03:13:38.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.763+0000 7f2c7ae87700 1 -- 172.21.2.111:0/3710727031 wait complete. 2021-04-03T03:13:38.785 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:13:38.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 Processor -- start 2021-04-03T03:13:38.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- start start 2021-04-03T03:13:38.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8a880603f0 0x7f8a880b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a880af400 con 0x7f8a880b0c90 2021-04-03T03:13:38.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a880af6a0 con 0x7f8a8814b700 2021-04-03T03:13:38.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a88060860 con 0x7f8a880603f0 2021-04-03T03:13:38.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8d452700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8a880b0c90 0x7f8a88147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53716/0 (socket says 172.21.2.111:53716) 2021-04-03T03:13:38.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8d452700 1 -- 172.21.2.111:0/2048641699 learned_addr learned my addr 172.21.2.111:0/2048641699 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:38.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105056779 0 0) 0x7f8a880af400 con 0x7f8a880b0c90 2021-04-03T03:13:38.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a68003240 con 0x7f8a880b0c90 2021-04-03T03:13:38.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2970531051 0 0) 0x7f8a68003240 con 0x7f8a880b0c90 2021-04-03T03:13:38.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 >> v2:172.21.2.115:3300/0 conn(0x7f8a880603f0 msgr2=0x7f8a880b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 --2- 172.21.2.111:0/2048641699 >> v2:172.21.2.115:3300/0 conn(0x7f8a880603f0 0x7f8a880b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 >> v1:172.21.2.115:6789/0 conn(0x7f8a8814b700 legacy=0x7f8a880b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a880afb40 con 0x7f8a880b0c90 2021-04-03T03:13:38.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8a80001a50 con 0x7f8a880b0c90 2021-04-03T03:13:38.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 >> v1:172.21.2.111:6789/0 conn(0x7f8a880b0c90 legacy=0x7f8a88147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a68003730 0x7f8a68003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a68004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a68004e50 0x7f8a6800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a88060860 con 0x7f8a680044e0 2021-04-03T03:13:38.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68004240 con 0x7f8a68004e50 2021-04-03T03:13:38.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6800bc20 con 0x7f8a68003730 2021-04-03T03:13:38.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a87fff700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a68004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a87fff700 1 -- 172.21.2.111:0/2048641699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a68003730 msgr2=0x7f8a68003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a87fff700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a68003730 0x7f8a68003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a87fff700 1 -- 172.21.2.111:0/2048641699 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a68004e50 msgr2=0x7f8a6800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:38.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a87fff700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a68004e50 0x7f8a6800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a87fff700 1 -- 172.21.2.111:0/2048641699 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68003240 con 0x7f8a680044e0 2021-04-03T03:13:38.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8d452700 1 -- 172.21.2.111:0/2048641699 reap_dead start 2021-04-03T03:13:38.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a87fff700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a68004960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f8a78000f30 tx=0x7f8a78007b50).ready entity=mon.0 client_cookie=567b6504245d9e76 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7800c660 con 0x7f8a680044e0 2021-04-03T03:13:38.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800c7c0 con 0x7f8a680044e0 2021-04-03T03:13:38.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a877fe700 1 -- 172.21.2.111:0/2048641699 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7800cb20 con 0x7f8a680044e0 2021-04-03T03:13:38.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- 172.21.2.111:0/2048641699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 msgr2=0x7f8a68004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a68004960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f8a78000f30 tx=0x7f8a78007b50).stop 2021-04-03T03:13:38.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- 172.21.2.111:0/2048641699 shutdown_connections 2021-04-03T03:13:38.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/2048641699 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a68004960 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- 172.21.2.111:0/2048641699 >> 172.21.2.111:0/2048641699 conn(0x7f8a880b61c0 msgr2=0x7f8a8805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- 172.21.2.111:0/2048641699 shutdown_connections 2021-04-03T03:13:38.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- 172.21.2.111:0/2048641699 wait complete. 2021-04-03T03:13:38.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 Processor -- start 2021-04-03T03:13:38.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 -- start start 2021-04-03T03:13:38.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a880abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.827+0000 7f8a8dc53700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a880a7a10 0x7f8a88148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8dc53700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a880a8280 0x7f8a881455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8dc53700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880ab020 con 0x7f8a680044e0 2021-04-03T03:13:38.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8dc53700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880a8700 con 0x7f8a880a7a10 2021-04-03T03:13:38.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8dc53700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880a7030 con 0x7f8a880a8280 2021-04-03T03:13:38.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a880abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a880abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52934/0 (socket says 172.21.2.111:52934) 2021-04-03T03:13:38.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 -- 172.21.2.111:0/1076841393 learned_addr learned my addr 172.21.2.111:0/1076841393 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:38.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 -- 172.21.2.111:0/1076841393 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a880a8280 msgr2=0x7f8a881455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a880a8280 0x7f8a881455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 -- 172.21.2.111:0/1076841393 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a880a7a10 msgr2=0x7f8a88148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a880a7a10 0x7f8a88148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 -- 172.21.2.111:0/1076841393 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a880a72d0 con 0x7f8a680044e0 2021-04-03T03:13:38.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8cc51700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a880abab0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f8a70002f30 tx=0x7f8a70006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a85ffb700 1 -- 172.21.2.111:0/1076841393 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7000d570 con 0x7f8a680044e0 2021-04-03T03:13:38.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a85ffb700 1 -- 172.21.2.111:0/1076841393 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a70000e90 con 0x7f8a680044e0 2021-04-03T03:13:38.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a880a7570 con 0x7f8a680044e0 2021-04-03T03:13:38.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a85ffb700 1 -- 172.21.2.111:0/1076841393 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7000d8c0 con 0x7f8a680044e0 2021-04-03T03:13:38.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a88145cc0 con 0x7f8a680044e0 2021-04-03T03:13:38.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a880a7700 con 0x7f8a680044e0 2021-04-03T03:13:38.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a85ffb700 1 -- 172.21.2.111:0/1076841393 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8a7001a060 con 0x7f8a680044e0 2021-04-03T03:13:38.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a85ffb700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a6c056ef0 0x7f8a6c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:38.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a85ffb700 1 -- 172.21.2.111:0/1076841393 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f8a7006f7d0 con 0x7f8a680044e0 2021-04-03T03:13:38.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a87fff700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a6c056ef0 0x7f8a6c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:38.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.831+0000 7f8a87fff700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a6c056ef0 0x7f8a6c0593b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f8a78000f30 tx=0x7f8a78015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:38.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.835+0000 7f8a85ffb700 1 -- 172.21.2.111:0/1076841393 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8a70044de0 con 0x7f8a680044e0 2021-04-03T03:13:38.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8a88057660 con 0x7f8a680044e0 2021-04-03T03:13:38.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a85ffb700 1 -- 172.21.2.111:0/1076841393 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+143 (secure 0 0 0) 0x7f8a70048400 con 0x7f8a680044e0 2021-04-03T03:13:38.947 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:38.947 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:13:38.947 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.947 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:13:38.948 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:13:38.948 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:13:38.948 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:13:38.948 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:13:38.949 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:13:38.949 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:13:38.949 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:13:38.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a6c056ef0 msgr2=0x7f8a6c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a6c056ef0 0x7f8a6c0593b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f8a78000f30 tx=0x7f8a78015000).stop 2021-04-03T03:13:38.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 msgr2=0x7f8a880abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:38.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a880abab0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f8a70002f30 tx=0x7f8a70006dc0).stop 2021-04-03T03:13:38.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 shutdown_connections 2021-04-03T03:13:38.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a880a8280 0x7f8a881455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a680044e0 0x7f8a880abab0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a6c056ef0 0x7f8a6c0593b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:38.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 --2- 172.21.2.111:0/1076841393 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a880a7a10 0x7f8a88148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:38.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 >> 172.21.2.111:0/1076841393 conn(0x7f8a880b61c0 msgr2=0x7f8a880b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:38.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 shutdown_connections 2021-04-03T03:13:38.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.931+0000 7f8a8dc53700 1 -- 172.21.2.111:0/1076841393 wait complete. 2021-04-03T03:13:38.954 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:13:38.955 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:13:39.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.999+0000 7f4ff3fff700 1 Processor -- start 2021-04-03T03:13:39.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:38.999+0000 7f4ff3fff700 1 -- start start 2021-04-03T03:13:39.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4ff40603f0 0x7f4ff4144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:39.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4ff40b17f0 0x7f4ff4144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:39.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40b0be0 con 0x7f4ff40b0290 2021-04-03T03:13:39.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff40607d0 con 0x7f4ff40603f0 2021-04-03T03:13:39.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff40b0e10 con 0x7f4ff40b17f0 2021-04-03T03:13:39.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4ff40b0290 0x7f4ff40aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53734/0 (socket says 172.21.2.111:53734) 2021-04-03T03:13:39.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 -- 172.21.2.111:0/3933786151 learned_addr learned my addr 172.21.2.111:0/3933786151 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:39.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff2ffd700 1 --2- 172.21.2.111:0/3933786151 >> v2:172.21.2.111:3300/0 conn(0x7f4ff40603f0 0x7f4ff4144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff27fc700 1 --2- 172.21.2.111:0/3933786151 >> v2:172.21.2.115:3300/0 conn(0x7f4ff40b17f0 0x7f4ff4144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff1ffb700 1 -- 172.21.2.111:0/3933786151 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636172103 0 0) 0x7f4ff40b0be0 con 0x7f4ff40b0290 2021-04-03T03:13:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff1ffb700 1 -- 172.21.2.111:0/3933786151 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd4003240 con 0x7f4ff40b0290 2021-04-03T03:13:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff2ffd700 1 -- 172.21.2.111:0/3933786151 >> v2:172.21.2.115:3300/0 conn(0x7f4ff40b17f0 msgr2=0x7f4ff4144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff2ffd700 1 --2- 172.21.2.111:0/3933786151 >> v2:172.21.2.115:3300/0 conn(0x7f4ff40b17f0 0x7f4ff4144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff2ffd700 1 -- 172.21.2.111:0/3933786151 >> v1:172.21.2.111:6789/0 conn(0x7f4ff40b0290 legacy=0x7f4ff40aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff2ffd700 1 -- 172.21.2.111:0/3933786151 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40b10b0 con 0x7f4ff40603f0 2021-04-03T03:13:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff2ffd700 1 --2- 172.21.2.111:0/3933786151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40603f0 0x7f4ff4144700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f4fdc002f30 tx=0x7f4fdc006dd0).ready entity=mon.0 client_cookie=eb5480aec44a4177 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff1ffb700 1 -- 172.21.2.111:0/3933786151 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fdc0096f0 con 0x7f4ff40603f0 2021-04-03T03:13:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff1ffb700 1 -- 172.21.2.111:0/3933786151 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc009850 con 0x7f4ff40603f0 2021-04-03T03:13:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff1ffb700 1 -- 172.21.2.111:0/3933786151 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fdc009bb0 con 0x7f4ff40603f0 2021-04-03T03:13:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- 172.21.2.111:0/3933786151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40603f0 msgr2=0x7f4ff4144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/3933786151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40603f0 0x7f4ff4144700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f4fdc002f30 tx=0x7f4fdc006dd0).stop 2021-04-03T03:13:39.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- 172.21.2.111:0/3933786151 shutdown_connections 2021-04-03T03:13:39.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/3933786151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40603f0 0x7f4ff4144700 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:39.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/3933786151 >> v2:172.21.2.115:3300/0 conn(0x7f4ff40b17f0 0x7f4ff4144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:39.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- 172.21.2.111:0/3933786151 >> 172.21.2.111:0/3933786151 conn(0x7f4ff40b61c0 msgr2=0x7f4ff405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:39.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- 172.21.2.111:0/3933786151 shutdown_connections 2021-04-03T03:13:39.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- 172.21.2.111:0/3933786151 wait complete. 2021-04-03T03:13:39.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 Processor -- start 2021-04-03T03:13:39.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- start start 2021-04-03T03:13:39.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ff40603f0 0x7f4ff4065ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:39.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ff40b0290 0x7f4ff40666b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:39.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40b17f0 0x7f4ff4146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:39.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4141fc0 con 0x7f4ff40b17f0 2021-04-03T03:13:39.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4142260 con 0x7f4ff40b0290 2021-04-03T03:13:39.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4134a80 con 0x7f4ff40603f0 2021-04-03T03:13:39.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40b17f0 0x7f4ff4146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:39.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40b17f0 0x7f4ff4146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52946/0 (socket says 172.21.2.111:52946) 2021-04-03T03:13:39.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 -- 172.21.2.111:0/1586206151 learned_addr learned my addr 172.21.2.111:0/1586206151 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:39.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff2ffd700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ff40603f0 0x7f4ff4065ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:39.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff27fc700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ff40b0290 0x7f4ff40666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:39.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 -- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ff40603f0 msgr2=0x7f4ff4065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:39.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ff40603f0 0x7f4ff4065ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:39.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 -- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ff40b0290 msgr2=0x7f4ff40666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:39.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ff40b0290 0x7f4ff40666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:39.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 -- 172.21.2.111:0/1586206151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff414c7d0 con 0x7f4ff40b17f0 2021-04-03T03:13:39.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff27fc700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ff40b0290 0x7f4ff40666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:13:39.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff37fe700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40b17f0 0x7f4ff4146a20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f4fec001450 tx=0x7f4fec007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:39.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4febfff700 1 -- 172.21.2.111:0/1586206151 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fec00ce20 con 0x7f4ff40b17f0 2021-04-03T03:13:39.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4febfff700 1 -- 172.21.2.111:0/1586206151 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec005b80 con 0x7f4ff40b17f0 2021-04-03T03:13:39.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4febfff700 1 -- 172.21.2.111:0/1586206151 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fec007110 con 0x7f4ff40b17f0 2021-04-03T03:13:39.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.003+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff4066ba0 con 0x7f4ff40b17f0 2021-04-03T03:13:39.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff4065560 con 0x7f4ff40b17f0 2021-04-03T03:13:39.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff4057660 con 0x7f4ff40b17f0 2021-04-03T03:13:39.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4febfff700 1 -- 172.21.2.111:0/1586206151 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4fec010030 con 0x7f4ff40b17f0 2021-04-03T03:13:39.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4febfff700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fd8056ea0 0x7f4fd8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:39.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4febfff700 1 -- 172.21.2.111:0/1586206151 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f4fec06fe10 con 0x7f4ff40b17f0 2021-04-03T03:13:39.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4ff2ffd700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fd8056ea0 0x7f4fd8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:39.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4ff2ffd700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fd8056ea0 0x7f4fd8059360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f4fdc01c000 tx=0x7f4fdc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:39.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.007+0000 7f4febfff700 1 -- 172.21.2.111:0/1586206151 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4fec045530 con 0x7f4ff40b17f0 2021-04-03T03:13:39.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.103+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4ff40686d0 con 0x7f4ff40b17f0 2021-04-03T03:13:39.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.979+0000 7f4febfff700 1 -- 172.21.2.111:0/1586206151 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v59) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4fec048b50 con 0x7f4ff40b17f0 2021-04-03T03:13:40.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fd8056ea0 msgr2=0x7f4fd8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:40.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fd8056ea0 0x7f4fd8059360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f4fdc01c000 tx=0x7f4fdc015000).stop 2021-04-03T03:13:40.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40b17f0 msgr2=0x7f4ff4146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:40.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40b17f0 0x7f4ff4146a20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f4fec001450 tx=0x7f4fec007d20).stop 2021-04-03T03:13:40.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 shutdown_connections 2021-04-03T03:13:40.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ff40b17f0 0x7f4ff4146a20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ff40603f0 0x7f4ff4065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fd8056ea0 0x7f4fd8059360 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 --2- 172.21.2.111:0/1586206151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ff40b0290 0x7f4ff40666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 >> 172.21.2.111:0/1586206151 conn(0x7f4ff40b61c0 msgr2=0x7f4ff4143010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:40.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 shutdown_connections 2021-04-03T03:13:40.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:39.983+0000 7f4ff3fff700 1 -- 172.21.2.111:0/1586206151 wait complete. 2021-04-03T03:13:40.005 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:13:40.007 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:13:40.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 Processor -- start 2021-04-03T03:13:40.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- start start 2021-04-03T03:13:40.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:40.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f506c03d560 0x7f506c041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:40.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506c0dc560 con 0x7f506c041060 2021-04-03T03:13:40.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c03de60 con 0x7f506c03d560 2021-04-03T03:13:40.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0dc800 con 0x7f506c0dbba0 2021-04-03T03:13:40.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507994d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f506c03d560 0x7f506c041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:40.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507894b700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f506c041060 0x7f506c03fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:38054/0 (socket says 172.21.2.111:38054) 2021-04-03T03:13:40.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507894b700 1 -- 172.21.2.111:0/3859994990 learned_addr learned my addr 172.21.2.111:0/3859994990 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:40.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507914c700 1 --2- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:40.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507994d700 1 -- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 msgr2=0x7f506c0dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:40.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507994d700 1 --2- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507994d700 1 -- 172.21.2.111:0/3859994990 >> v1:172.21.2.115:6789/0 conn(0x7f506c041060 legacy=0x7f506c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:40.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507994d700 1 -- 172.21.2.111:0/3859994990 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c03da30 con 0x7f506c03d560 2021-04-03T03:13:40.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507994d700 1 --2- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c041650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f507406b5d0 tx=0x7f507405ad70).ready entity=mon.1 client_cookie=bd976a0a490ad633 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:40.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f5063fff700 1 -- 172.21.2.111:0/3859994990 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f507405bda0 con 0x7f506c03d560 2021-04-03T03:13:40.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f5063fff700 1 -- 172.21.2.111:0/3859994990 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507405df60 con 0x7f506c03d560 2021-04-03T03:13:40.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f5063fff700 1 -- 172.21.2.111:0/3859994990 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50740699f0 con 0x7f506c03d560 2021-04-03T03:13:40.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 msgr2=0x7f506c041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:40.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 --2- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c041650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f507406b5d0 tx=0x7f507405ad70).stop 2021-04-03T03:13:40.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- 172.21.2.111:0/3859994990 shutdown_connections 2021-04-03T03:13:40.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 --2- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c041650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 --2- 172.21.2.111:0/3859994990 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- 172.21.2.111:0/3859994990 >> 172.21.2.111:0/3859994990 conn(0x7f506c0462b0 msgr2=0x7f506c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:40.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- 172.21.2.111:0/3859994990 shutdown_connections 2021-04-03T03:13:40.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.055+0000 7f507a14e700 1 -- 172.21.2.111:0/3859994990 wait complete. 2021-04-03T03:13:40.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 Processor -- start 2021-04-03T03:13:40.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 -- start start 2021-04-03T03:13:40.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c045a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:40.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f506c041060 0x7f506c0d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:40.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:40.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0deff0 con 0x7f506c041060 2021-04-03T03:13:40.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c039620 con 0x7f506c03d560 2021-04-03T03:13:40.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0398c0 con 0x7f506c0dbba0 2021-04-03T03:13:40.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c045a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:40.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c045a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:42668/0 (socket says 172.21.2.111:42668) 2021-04-03T03:13:40.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507894b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f506c041060 0x7f506c0d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:40.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 -- 172.21.2.111:0/145476498 learned_addr learned my addr 172.21.2.111:0/145476498 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:40.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507994d700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:40.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 -- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 msgr2=0x7f506c0d64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:40.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0d64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 -- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f506c041060 msgr2=0x7f506c0d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:40.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f506c041060 0x7f506c0d5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:40.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 -- 172.21.2.111:0/145476498 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c039b60 con 0x7f506c03d560 2021-04-03T03:13:40.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507914c700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c045a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f507000c000 tx=0x7f5070006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:40.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f50627fc700 1 -- 172.21.2.111:0/145476498 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f507000ca10 con 0x7f506c03d560 2021-04-03T03:13:40.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f50627fc700 1 -- 172.21.2.111:0/145476498 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000cb70 con 0x7f506c03d560 2021-04-03T03:13:40.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c0d69c0 con 0x7f506c03d560 2021-04-03T03:13:40.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c03a870 con 0x7f506c03d560 2021-04-03T03:13:40.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f50627fc700 1 -- 172.21.2.111:0/145476498 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f507000be70 con 0x7f506c03d560 2021-04-03T03:13:40.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f50627fc700 1 -- 172.21.2.111:0/145476498 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f507001c460 con 0x7f506c03d560 2021-04-03T03:13:40.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c004db0 con 0x7f506c03d560 2021-04-03T03:13:40.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f50627fc700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5054057000 0x7f50540594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:40.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f50627fc700 1 -- 172.21.2.111:0/145476498 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5950+0+0 (secure 0 0 0) 0x7f5070070490 con 0x7f506c03d560 2021-04-03T03:13:40.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507894b700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5054057000 0x7f50540594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:40.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.059+0000 7f507894b700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5054057000 0x7f50540594c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f506400d000 tx=0x7f5064009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:40.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.063+0000 7f50627fc700 1 -- 172.21.2.111:0/145476498 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f50700458e0 con 0x7f506c03d560 2021-04-03T03:13:40.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.155+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f506c040e40 con 0x7f506c03d560 2021-04-03T03:13:41.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f50627fc700 1 -- 172.21.2.111:0/145476498 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v60) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5070048f00 con 0x7f506c03d560 2021-04-03T03:13:41.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5054057000 msgr2=0x7f50540594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5054057000 0x7f50540594c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f506400d000 tx=0x7f5064009040).stop 2021-04-03T03:13:41.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 msgr2=0x7f506c045a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c045a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f507000c000 tx=0x7f5070006dd0).stop 2021-04-03T03:13:41.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 shutdown_connections 2021-04-03T03:13:41.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f506c0dbba0 0x7f506c0d64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5054057000 0x7f50540594c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f506c03d560 0x7f506c045a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 --2- 172.21.2.111:0/145476498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f506c041060 0x7f506c0d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 >> 172.21.2.111:0/145476498 conn(0x7f506c0462b0 msgr2=0x7f506c041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 shutdown_connections 2021-04-03T03:13:41.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:40.991+0000 7f507a14e700 1 -- 172.21.2.111:0/145476498 wait complete. 2021-04-03T03:13:41.011 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:13:41.016 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:13:41.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 Processor -- start 2021-04-03T03:13:41.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 -- start start 2021-04-03T03:13:41.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c00d54e0 0x7f73c00440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f73c003bd70 0x7f73c00445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c00d5e30 con 0x7f73c00d58c0 2021-04-03T03:13:41.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c0006550 con 0x7f73c003bd70 2021-04-03T03:13:41.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00067f0 con 0x7f73c00d54e0 2021-04-03T03:13:41.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f73c003bd70 0x7f73c00445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f73c003bd70 0x7f73c00445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52968/0 (socket says 172.21.2.111:52968) 2021-04-03T03:13:41.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 -- 172.21.2.111:0/2002565177 learned_addr learned my addr 172.21.2.111:0/2002565177 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:41.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 -- 172.21.2.111:0/2002565177 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c00d54e0 msgr2=0x7f73c00440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:41.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 --2- 172.21.2.111:0/2002565177 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c00d54e0 0x7f73c00440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 -- 172.21.2.111:0/2002565177 >> v1:172.21.2.115:6789/0 conn(0x7f73c00d58c0 legacy=0x7f73c0044ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:41.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 -- 172.21.2.111:0/2002565177 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c0043370 con 0x7f73c003bd70 2021-04-03T03:13:41.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c67fc700 1 --2- 172.21.2.111:0/2002565177 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c003bd70 0x7f73c00445d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f73c0045b10 tx=0x7f73bc006e90).ready entity=mon.0 client_cookie=c6d95607b1dc1120 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c5ffb700 1 -- 172.21.2.111:0/2002565177 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73bc0098e0 con 0x7f73c003bd70 2021-04-03T03:13:41.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c5ffb700 1 -- 172.21.2.111:0/2002565177 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc009a40 con 0x7f73c003bd70 2021-04-03T03:13:41.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c5ffb700 1 -- 172.21.2.111:0/2002565177 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73bc009de0 con 0x7f73c003bd70 2021-04-03T03:13:41.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 -- 172.21.2.111:0/2002565177 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c003bd70 msgr2=0x7f73c00445d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.063+0000 7f73c7fff700 1 --2- 172.21.2.111:0/2002565177 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c003bd70 0x7f73c00445d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f73c0045b10 tx=0x7f73bc006e90).stop 2021-04-03T03:13:41.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- 172.21.2.111:0/2002565177 shutdown_connections 2021-04-03T03:13:41.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 --2- 172.21.2.111:0/2002565177 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c00d54e0 0x7f73c00440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 --2- 172.21.2.111:0/2002565177 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c003bd70 0x7f73c00445d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- 172.21.2.111:0/2002565177 >> 172.21.2.111:0/2002565177 conn(0x7f73c000e820 msgr2=0x7f73c000ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- 172.21.2.111:0/2002565177 shutdown_connections 2021-04-03T03:13:41.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- 172.21.2.111:0/2002565177 wait complete. 2021-04-03T03:13:41.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 Processor -- start 2021-04-03T03:13:41.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- start start 2021-04-03T03:13:41.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c003bd70 0x7f73c00d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73c00d54e0 0x7f73c00a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c00d58c0 0x7f73c00cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0042480 con 0x7f73c00d58c0 2021-04-03T03:13:41.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00e1ed0 con 0x7f73c00d54e0 2021-04-03T03:13:41.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0040a40 con 0x7f73c003bd70 2021-04-03T03:13:41.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c00d58c0 0x7f73c00cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c00d58c0 0x7f73c00cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52978/0 (socket says 172.21.2.111:52978) 2021-04-03T03:13:41.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 -- 172.21.2.111:0/3257352114 learned_addr learned my addr 172.21.2.111:0/3257352114 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:41.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c6ffd700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c003bd70 0x7f73c00d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c67fc700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73c00d54e0 0x7f73c00a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 -- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c003bd70 msgr2=0x7f73c00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c003bd70 0x7f73c00d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 -- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73c00d54e0 msgr2=0x7f73c00a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73c00d54e0 0x7f73c00a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 -- 172.21.2.111:0/3257352114 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c0040ce0 con 0x7f73c00d58c0 2021-04-03T03:13:41.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c67fc700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73c00d54e0 0x7f73c00a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:13:41.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c77fe700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c00d58c0 0x7f73c00cd2c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f73b8002eb0 tx=0x7f73b8006cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73b7fff700 1 -- 172.21.2.111:0/3257352114 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73b800d4a0 con 0x7f73c00d58c0 2021-04-03T03:13:41.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73b7fff700 1 -- 172.21.2.111:0/3257352114 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b8000e90 con 0x7f73c00d58c0 2021-04-03T03:13:41.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c0042100 con 0x7f73c00d58c0 2021-04-03T03:13:41.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73b7fff700 1 -- 172.21.2.111:0/3257352114 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73b800d7f0 con 0x7f73c00d58c0 2021-04-03T03:13:41.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c003e800 con 0x7f73c00d58c0 2021-04-03T03:13:41.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c003eac0 con 0x7f73c00d58c0 2021-04-03T03:13:41.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73b7fff700 1 -- 172.21.2.111:0/3257352114 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f73b800f030 con 0x7f73c00d58c0 2021-04-03T03:13:41.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73b7fff700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f73a4057000 0x7f73a40594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73b7fff700 1 -- 172.21.2.111:0/3257352114 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f73b806fd00 con 0x7f73c00d58c0 2021-04-03T03:13:41.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c6ffd700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f73a4057000 0x7f73a40594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.067+0000 7f73c6ffd700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f73a4057000 0x7f73a40594c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f73c8063000 tx=0x7f73c8064040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.071+0000 7f73b7fff700 1 -- 172.21.2.111:0/3257352114 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f73c003eac0 con 0x7f73c00d58c0 2021-04-03T03:13:41.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.199+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f73c003eac0 con 0x7f73c00d58c0 2021-04-03T03:13:41.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.203+0000 7f73b7fff700 1 -- 172.21.2.111:0/3257352114 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v24) v1 ==== 189+0+0 (secure 0 0 0) 0x7f73c003eac0 con 0x7f73c00d58c0 2021-04-03T03:13:41.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f73a4057000 msgr2=0x7f73a40594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f73a4057000 0x7f73a40594c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f73c8063000 tx=0x7f73c8064040).stop 2021-04-03T03:13:41.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c00d58c0 msgr2=0x7f73c00cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c00d58c0 0x7f73c00cd2c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f73b8002eb0 tx=0x7f73b8006cb0).stop 2021-04-03T03:13:41.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 shutdown_connections 2021-04-03T03:13:41.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f73c00d58c0 0x7f73c00cd2c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f73c003bd70 0x7f73c00d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f73a4057000 0x7f73a40594c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 --2- 172.21.2.111:0/3257352114 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f73c00d54e0 0x7f73c00a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 >> 172.21.2.111:0/3257352114 conn(0x7f73c000e820 msgr2=0x7f73c000ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 shutdown_connections 2021-04-03T03:13:41.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.207+0000 7f73c7fff700 1 -- 172.21.2.111:0/3257352114 wait complete. 2021-04-03T03:13:41.225 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 11 and data pool 12 2021-04-03T03:13:41.230 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:13:41.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 Processor -- start 2021-04-03T03:13:41.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- start start 2021-04-03T03:13:41.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5d380603f0 0x7f5d380b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5d380b0c90 0x7f5d38147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d380af400 con 0x7f5d3814b700 2021-04-03T03:13:41.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d38060860 con 0x7f5d380b0c90 2021-04-03T03:13:41.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d380af6a0 con 0x7f5d380603f0 2021-04-03T03:13:41.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5d380b0c90 0x7f5d38147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5d380b0c90 0x7f5d38147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:52998/0 (socket says 172.21.2.111:52998) 2021-04-03T03:13:41.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 -- 172.21.2.111:0/1578839636 learned_addr learned my addr 172.21.2.111:0/1578839636 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:41.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d37fff700 1 --2- 172.21.2.111:0/1578839636 >> v2:172.21.2.115:3300/0 conn(0x7f5d380603f0 0x7f5d380b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 -- 172.21.2.111:0/1578839636 >> v2:172.21.2.115:3300/0 conn(0x7f5d380603f0 msgr2=0x7f5d380b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:41.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 --2- 172.21.2.111:0/1578839636 >> v2:172.21.2.115:3300/0 conn(0x7f5d380603f0 0x7f5d380b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 -- 172.21.2.111:0/1578839636 >> v1:172.21.2.115:6789/0 conn(0x7f5d3814b700 legacy=0x7f5d380b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 -- 172.21.2.111:0/1578839636 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380afb40 con 0x7f5d380b0c90 2021-04-03T03:13:41.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 --2- 172.21.2.111:0/1578839636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d380b0c90 0x7f5d38147680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f5d300032b0 tx=0x7f5d30009270).ready entity=mon.0 client_cookie=b037bb92751af1cb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d36ffd700 1 -- 172.21.2.111:0/1578839636 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d3000bc00 con 0x7f5d380b0c90 2021-04-03T03:13:41.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d36ffd700 1 -- 172.21.2.111:0/1578839636 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3000bd60 con 0x7f5d380b0c90 2021-04-03T03:13:41.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d36ffd700 1 -- 172.21.2.111:0/1578839636 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d30011400 con 0x7f5d380b0c90 2021-04-03T03:13:41.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- 172.21.2.111:0/1578839636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d380b0c90 msgr2=0x7f5d38147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- 172.21.2.111:0/1578839636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d380b0c90 0x7f5d38147680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f5d300032b0 tx=0x7f5d30009270).stop 2021-04-03T03:13:41.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- 172.21.2.111:0/1578839636 shutdown_connections 2021-04-03T03:13:41.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- 172.21.2.111:0/1578839636 >> v2:172.21.2.115:3300/0 conn(0x7f5d380603f0 0x7f5d380b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- 172.21.2.111:0/1578839636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d380b0c90 0x7f5d38147680 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- 172.21.2.111:0/1578839636 >> 172.21.2.111:0/1578839636 conn(0x7f5d380b61c0 msgr2=0x7f5d3805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- 172.21.2.111:0/1578839636 shutdown_connections 2021-04-03T03:13:41.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- 172.21.2.111:0/1578839636 wait complete. 2021-04-03T03:13:41.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 Processor -- start 2021-04-03T03:13:41.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- start start 2021-04-03T03:13:41.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d380603f0 0x7f5d380a7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d380b0c90 0x7f5d380a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d3814b700 0x7f5d380628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d38063650 con 0x7f5d3814b700 2021-04-03T03:13:41.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d380a8280 con 0x7f5d380b0c90 2021-04-03T03:13:41.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3d327700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d38064f20 con 0x7f5d380603f0 2021-04-03T03:13:41.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.271+0000 7f5d3cb26700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d3814b700 0x7f5d380628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d3814b700 0x7f5d380628d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53004/0 (socket says 172.21.2.111:53004) 2021-04-03T03:13:41.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 -- 172.21.2.111:0/736282679 learned_addr learned my addr 172.21.2.111:0/736282679 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:41.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d377fe700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d380b0c90 0x7f5d380a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 -- 172.21.2.111:0/736282679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d380603f0 msgr2=0x7f5d380a7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d380603f0 0x7f5d380a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 -- 172.21.2.111:0/736282679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d380b0c90 msgr2=0x7f5d380a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d380b0c90 0x7f5d380a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 -- 172.21.2.111:0/736282679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380a9eb0 con 0x7f5d3814b700 2021-04-03T03:13:41.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3cb26700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d3814b700 0x7f5d380628d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f5d380a83c0 tx=0x7f5d30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d357fa700 1 -- 172.21.2.111:0/736282679 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d30011e90 con 0x7f5d3814b700 2021-04-03T03:13:41.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d357fa700 1 -- 172.21.2.111:0/736282679 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3001f420 con 0x7f5d3814b700 2021-04-03T03:13:41.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d357fa700 1 -- 172.21.2.111:0/736282679 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d3001f780 con 0x7f5d3814b700 2021-04-03T03:13:41.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d380aa150 con 0x7f5d3814b700 2021-04-03T03:13:41.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d380ab6e0 con 0x7f5d3814b700 2021-04-03T03:13:41.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d38064d90 con 0x7f5d3814b700 2021-04-03T03:13:41.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d357fa700 1 -- 172.21.2.111:0/736282679 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5d30026030 con 0x7f5d3814b700 2021-04-03T03:13:41.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d357fa700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d1c056ea0 0x7f5d1c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d357fa700 1 -- 172.21.2.111:0/736282679 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5d30070d80 con 0x7f5d3814b700 2021-04-03T03:13:41.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d37fff700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d1c056ea0 0x7f5d1c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d37fff700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d1c056ea0 0x7f5d1c059360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f5d2800a000 tx=0x7f5d28008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.275+0000 7f5d357fa700 1 -- 172.21.2.111:0/736282679 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5d30046170 con 0x7f5d3814b700 2021-04-03T03:13:41.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d380aa510 con 0x7f5d3814b700 2021-04-03T03:13:41.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d357fa700 1 -- 172.21.2.111:0/736282679 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f5d30072090 con 0x7f5d3814b700 2021-04-03T03:13:41.388 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:41.388 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":63,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:13:41.205509+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:13:39.120055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-04-03T03:13:40.173978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:13:41.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d1c056ea0 msgr2=0x7f5d1c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d1c056ea0 0x7f5d1c059360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f5d2800a000 tx=0x7f5d28008040).stop 2021-04-03T03:13:41.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d3814b700 msgr2=0x7f5d380628d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d3814b700 0x7f5d380628d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f5d380a83c0 tx=0x7f5d30016000).stop 2021-04-03T03:13:41.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 shutdown_connections 2021-04-03T03:13:41.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d3814b700 0x7f5d380628d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d380603f0 0x7f5d380a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d1c056ea0 0x7f5d1c059360 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 --2- 172.21.2.111:0/736282679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d380b0c90 0x7f5d380a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 >> 172.21.2.111:0/736282679 conn(0x7f5d380b61c0 msgr2=0x7f5d38148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 shutdown_connections 2021-04-03T03:13:41.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.371+0000 7f5d3d327700 1 -- 172.21.2.111:0/736282679 wait complete. 2021-04-03T03:13:41.395 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:13:41.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 Processor -- start 2021-04-03T03:13:41.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- start start 2021-04-03T03:13:41.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83000dbb90 0x7f8300042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000dc6e0 con 0x7f830003d360 2021-04-03T03:13:41.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8300042010 con 0x7f8300006bc0 2021-04-03T03:13:41.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000d9560 con 0x7f83000dbb90 2021-04-03T03:13:41.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830d19f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83000dbb90 0x7f8300042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830c99e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f830003d360 0x7f83000064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53804/0 (socket says 172.21.2.111:53804) 2021-04-03T03:13:41.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830c99e700 1 -- 172.21.2.111:0/2592123439 learned_addr learned my addr 172.21.2.111:0/2592123439 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:41.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f82fbfff700 1 -- 172.21.2.111:0/2592123439 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743632875 0 0) 0x7f83000dc6e0 con 0x7f830003d360 2021-04-03T03:13:41.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f82fbfff700 1 -- 172.21.2.111:0/2592123439 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f0003240 con 0x7f830003d360 2021-04-03T03:13:41.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830d19f700 1 -- 172.21.2.111:0/2592123439 >> v1:172.21.2.115:6789/0 conn(0x7f8300006bc0 legacy=0x7f83000056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:41.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830d19f700 1 -- 172.21.2.111:0/2592123439 >> v1:172.21.2.111:6789/0 conn(0x7f830003d360 legacy=0x7f83000064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830d19f700 1 -- 172.21.2.111:0/2592123439 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000422b0 con 0x7f83000dbb90 2021-04-03T03:13:41.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830d19f700 1 --2- 172.21.2.111:0/2592123439 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83000dbb90 0x7f8300042750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f83000087e0 tx=0x7f82fc006dd0).ready entity=mon.0 client_cookie=351e940600969585 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f82fbfff700 1 -- 172.21.2.111:0/2592123439 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82fc009770 con 0x7f83000dbb90 2021-04-03T03:13:41.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f82fbfff700 1 -- 172.21.2.111:0/2592123439 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc0098d0 con 0x7f83000dbb90 2021-04-03T03:13:41.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f82fbfff700 1 -- 172.21.2.111:0/2592123439 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82fc009c70 con 0x7f83000dbb90 2021-04-03T03:13:41.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- 172.21.2.111:0/2592123439 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83000dbb90 msgr2=0x7f8300042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 --2- 172.21.2.111:0/2592123439 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83000dbb90 0x7f8300042750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f83000087e0 tx=0x7f82fc006dd0).stop 2021-04-03T03:13:41.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- 172.21.2.111:0/2592123439 shutdown_connections 2021-04-03T03:13:41.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 --2- 172.21.2.111:0/2592123439 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83000dbb90 0x7f8300042750 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- 172.21.2.111:0/2592123439 >> 172.21.2.111:0/2592123439 conn(0x7f83000462c0 msgr2=0x7f83000466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- 172.21.2.111:0/2592123439 shutdown_connections 2021-04-03T03:13:41.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.435+0000 7f830e1a1700 1 -- 172.21.2.111:0/2592123439 wait complete. 2021-04-03T03:13:41.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 Processor -- start 2021-04-03T03:13:41.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 -- start start 2021-04-03T03:13:41.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f830003d360 0x7f83000a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83000dbb90 0x7f83000d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f830009f800 0x7f830003b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000a20e0 con 0x7f830009f800 2021-04-03T03:13:41.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830009f600 con 0x7f830003d360 2021-04-03T03:13:41.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000a1760 con 0x7f83000dbb90 2021-04-03T03:13:41.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f830009f800 0x7f830003b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f830009f800 0x7f830003b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53018/0 (socket says 172.21.2.111:53018) 2021-04-03T03:13:41.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 -- 172.21.2.111:0/3072529661 learned_addr learned my addr 172.21.2.111:0/3072529661 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:41.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 -- 172.21.2.111:0/3072529661 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83000dbb90 msgr2=0x7f83000d70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83000dbb90 0x7f83000d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 -- 172.21.2.111:0/3072529661 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f830003d360 msgr2=0x7f83000a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f830003d360 0x7f83000a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 -- 172.21.2.111:0/3072529661 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000a2f00 con 0x7f830009f800 2021-04-03T03:13:41.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d9a0700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f830009f800 0x7f830003b5a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f830806b5d0 tx=0x7f830805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f82fa7fc700 1 -- 172.21.2.111:0/3072529661 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83080699f0 con 0x7f830009f800 2021-04-03T03:13:41.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f82fa7fc700 1 -- 172.21.2.111:0/3072529661 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830805f2c0 con 0x7f830009f800 2021-04-03T03:13:41.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f82fa7fc700 1 -- 172.21.2.111:0/3072529661 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f830805f620 con 0x7f830009f800 2021-04-03T03:13:41.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000a1a00 con 0x7f830009f800 2021-04-03T03:13:41.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83000a09c0 con 0x7f830009f800 2021-04-03T03:13:41.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83000a3090 con 0x7f830009f800 2021-04-03T03:13:41.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f82fa7fc700 1 -- 172.21.2.111:0/3072529661 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f830808b020 con 0x7f830009f800 2021-04-03T03:13:41.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f82fa7fc700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82e8056fb0 0x7f82e8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f82fa7fc700 1 -- 172.21.2.111:0/3072529661 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f83080df220 con 0x7f830009f800 2021-04-03T03:13:41.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d19f700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82e8056fb0 0x7f82e8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.439+0000 7f830d19f700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82e8056fb0 0x7f82e8059470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f83000087e0 tx=0x7f82fc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.443+0000 7f82fa7fc700 1 -- 172.21.2.111:0/3072529661 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f83080b4610 con 0x7f830009f800 2021-04-03T03:13:41.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8300004da0 con 0x7f830009f800 2021-04-03T03:13:41.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f82fa7fc700 1 -- 172.21.2.111:0/3072529661 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f83080b7c30 con 0x7f830009f800 2021-04-03T03:13:41.551 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:41.551 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":63,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:13:41.205509+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:13:39.120055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-04-03T03:13:40.173978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:13:41.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82e8056fb0 msgr2=0x7f82e8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82e8056fb0 0x7f82e8059470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f83000087e0 tx=0x7f82fc015000).stop 2021-04-03T03:13:41.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f830009f800 msgr2=0x7f830003b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f830009f800 0x7f830003b5a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f830806b5d0 tx=0x7f830805ad70).stop 2021-04-03T03:13:41.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 shutdown_connections 2021-04-03T03:13:41.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f830009f800 0x7f830003b5a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82e8056fb0 0x7f82e8059470 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f830003d360 0x7f83000a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 --2- 172.21.2.111:0/3072529661 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83000dbb90 0x7f83000d70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 >> 172.21.2.111:0/3072529661 conn(0x7f83000462c0 msgr2=0x7f83000441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 shutdown_connections 2021-04-03T03:13:41.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.535+0000 7f830e1a1700 1 -- 172.21.2.111:0/3072529661 wait complete. 2021-04-03T03:13:41.559 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:13:41.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94fde7700 1 Processor -- start 2021-04-03T03:13:41.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94fde7700 1 -- start start 2021-04-03T03:13:41.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94fde7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd9500603f0 0x7fd9501447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94fde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9500607d0 con 0x7fd9500add50 2021-04-03T03:13:41.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94fde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95014b090 con 0x7fd9500b17b0 2021-04-03T03:13:41.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94fde7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95014b330 con 0x7fd9500603f0 2021-04-03T03:13:41.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94e5e4700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd9500add50 0x7fd950147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53820/0 (socket says 172.21.2.111:53820) 2021-04-03T03:13:41.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94e5e4700 1 -- 172.21.2.111:0/2691802349 learned_addr learned my addr 172.21.2.111:0/2691802349 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:41.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94ede5700 1 --2- 172.21.2.111:0/2691802349 >> v2:172.21.2.115:3300/0 conn(0x7fd9500603f0 0x7fd9501447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756033383 0 0) 0x7fd9500607d0 con 0x7fd9500add50 2021-04-03T03:13:41.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd930003240 con 0x7fd9500add50 2021-04-03T03:13:41.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1741228705 0 0) 0x7fd930003240 con 0x7fd9500add50 2021-04-03T03:13:41.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 >> v2:172.21.2.115:3300/0 conn(0x7fd9500603f0 msgr2=0x7fd9501447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:41.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 --2- 172.21.2.111:0/2691802349 >> v2:172.21.2.115:3300/0 conn(0x7fd9500603f0 0x7fd9501447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 >> v1:172.21.2.115:6789/0 conn(0x7fd9500b17b0 legacy=0x7fd950147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95014b5d0 con 0x7fd9500add50 2021-04-03T03:13:41.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fd9380016d0 con 0x7fd9500add50 2021-04-03T03:13:41.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 >> v1:172.21.2.111:6789/0 conn(0x7fd9500add50 legacy=0x7fd950147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd930003730 0x7fd930003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.599+0000 7fd94dde3700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd9300044e0 0x7fd930004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94dde3700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd930004e50 0x7fd93000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95014b330 con 0x7fd930004e50 2021-04-03T03:13:41.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930004260 con 0x7fd9300044e0 2021-04-03T03:13:41.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93000bc20 con 0x7fd930003730 2021-04-03T03:13:41.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd930004e50 0x7fd93000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/2691802349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd930003730 msgr2=0x7fd930003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:41.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd930003730 0x7fd930003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/2691802349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd9300044e0 msgr2=0x7fd930004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:41.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd9300044e0 0x7fd930004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/2691802349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930003240 con 0x7fd930004e50 2021-04-03T03:13:41.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/2691802349 reap_dead start 2021-04-03T03:13:41.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd930004e50 0x7fd93000b730 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fd9300054b0 tx=0x7fd948007ef0).ready entity=mon.0 client_cookie=5f640eb146727c41 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd948001540 con 0x7fd930004e50 2021-04-03T03:13:41.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94800c9d0 con 0x7fd930004e50 2021-04-03T03:13:41.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94dde3700 1 -- 172.21.2.111:0/2691802349 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94800cd30 con 0x7fd930004e50 2021-04-03T03:13:41.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- 172.21.2.111:0/2691802349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd930004e50 msgr2=0x7fd93000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd930004e50 0x7fd93000b730 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fd9300054b0 tx=0x7fd948007ef0).stop 2021-04-03T03:13:41.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- 172.21.2.111:0/2691802349 shutdown_connections 2021-04-03T03:13:41.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 --2- 172.21.2.111:0/2691802349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd930004e50 0x7fd93000b730 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- 172.21.2.111:0/2691802349 >> 172.21.2.111:0/2691802349 conn(0x7fd9500b61c0 msgr2=0x7fd95005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:41.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- 172.21.2.111:0/2691802349 shutdown_connections 2021-04-03T03:13:41.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- 172.21.2.111:0/2691802349 wait complete. 2021-04-03T03:13:41.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 Processor -- start 2021-04-03T03:13:41.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- start start 2021-04-03T03:13:41.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd930004e50 0x7fd9501499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9500a8f30 0x7fd9500a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9500a98a0 0x7fd9501449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500a9d50 con 0x7fd9500a98a0 2021-04-03T03:13:41.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500ab180 con 0x7fd930004e50 2021-04-03T03:13:41.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500ab420 con 0x7fd9500a8f30 2021-04-03T03:13:41.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9500a98a0 0x7fd9501449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9500a98a0 0x7fd9501449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53040/0 (socket says 172.21.2.111:53040) 2021-04-03T03:13:41.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/4106536403 learned_addr learned my addr 172.21.2.111:0/4106536403 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:41.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94ede5700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd930004e50 0x7fd9501499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/4106536403 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9500a8f30 msgr2=0x7fd9500a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9500a8f30 0x7fd9500a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:41.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/4106536403 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd930004e50 msgr2=0x7fd9501499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:41.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd930004e50 0x7fd9501499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:41.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 -- 172.21.2.111:0/4106536403 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd950149ea0 con 0x7fd9500a98a0 2021-04-03T03:13:41.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94f5e6700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9500a98a0 0x7fd9501449d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fd9300054b0 tx=0x7fd948017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd947fff700 1 -- 172.21.2.111:0/4106536403 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd948001b70 con 0x7fd9500a98a0 2021-04-03T03:13:41.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd947fff700 1 -- 172.21.2.111:0/4106536403 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd948006870 con 0x7fd9500a98a0 2021-04-03T03:13:41.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd950063480 con 0x7fd9500a98a0 2021-04-03T03:13:41.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd947fff700 1 -- 172.21.2.111:0/4106536403 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd948006bd0 con 0x7fd9500a98a0 2021-04-03T03:13:41.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.603+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9500ab8e0 con 0x7fd9500a98a0 2021-04-03T03:13:41.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.607+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd950057660 con 0x7fd9500a98a0 2021-04-03T03:13:41.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.607+0000 7fd947fff700 1 -- 172.21.2.111:0/4106536403 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd948010070 con 0x7fd9500a98a0 2021-04-03T03:13:41.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.607+0000 7fd947fff700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd934056f20 0x7fd9340593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:41.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.607+0000 7fd947fff700 1 -- 172.21.2.111:0/4106536403 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd948071520 con 0x7fd9500a98a0 2021-04-03T03:13:41.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.607+0000 7fd94ede5700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd934056f20 0x7fd9340593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:41.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.607+0000 7fd94ede5700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd934056f20 0x7fd9340593e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fd94000b000 tx=0x7fd940008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:41.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.607+0000 7fd947fff700 1 -- 172.21.2.111:0/4106536403 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd948046840 con 0x7fd9500a98a0 2021-04-03T03:13:41.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:41.735+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9500ab590 con 0x7fd9500a98a0 2021-04-03T03:13:42.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.211+0000 7fd947fff700 1 -- 172.21.2.111:0/4106536403 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd948049e20 con 0x7fd9500a98a0 2021-04-03T03:13:42.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd934056f20 msgr2=0x7fd9340593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd934056f20 0x7fd9340593e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fd94000b000 tx=0x7fd940008040).stop 2021-04-03T03:13:42.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9500a98a0 msgr2=0x7fd9501449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9500a98a0 0x7fd9501449d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fd9300054b0 tx=0x7fd948017000).stop 2021-04-03T03:13:42.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 shutdown_connections 2021-04-03T03:13:42.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9500a98a0 0x7fd9501449d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd934056f20 0x7fd9340593e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd930004e50 0x7fd9501499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 --2- 172.21.2.111:0/4106536403 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9500a8f30 0x7fd9500a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 >> 172.21.2.111:0/4106536403 conn(0x7fd9500b61c0 msgr2=0x7fd95014b990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 shutdown_connections 2021-04-03T03:13:42.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.215+0000 7fd94fde7700 1 -- 172.21.2.111:0/4106536403 wait complete. 2021-04-03T03:13:42.238 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:13:42.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 Processor -- start 2021-04-03T03:13:42.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 -- start start 2021-04-03T03:13:42.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700dbb90 0x7f9970042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f997003d360 0x7f99700dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f99700064b0 0x7f9970006940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99700d9560 con 0x7f997003d360 2021-04-03T03:13:42.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997003dcb0 con 0x7f99700064b0 2021-04-03T03:13:42.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f9977fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970006e30 con 0x7f99700dbb90 2021-04-03T03:13:42.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f997003d360 0x7f99700dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f997003d360 0x7f99700dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53046/0 (socket says 172.21.2.111:53046) 2021-04-03T03:13:42.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 -- 172.21.2.111:0/2767620409 learned_addr learned my addr 172.21.2.111:0/2767620409 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:42.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 -- 172.21.2.111:0/2767620409 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700dbb90 msgr2=0x7f9970042750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:13:42.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 --2- 172.21.2.111:0/2767620409 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700dbb90 0x7f9970042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 -- 172.21.2.111:0/2767620409 >> v2:172.21.2.115:3300/0 conn(0x7f99700064b0 msgr2=0x7f9970006940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 --2- 172.21.2.111:0/2767620409 >> v2:172.21.2.115:3300/0 conn(0x7f99700064b0 0x7f9970006940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.279+0000 7f99767fc700 1 -- 172.21.2.111:0/2767620409 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700dc6e0 con 0x7f997003d360 2021-04-03T03:13:42.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 --2- 172.21.2.111:0/2767620409 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700dbf70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f9970006f70 tx=0x7f9960006e90).ready entity=mon.0 client_cookie=b3124da0a10656c4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9975ffb700 1 -- 172.21.2.111:0/2767620409 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99600098e0 con 0x7f997003d360 2021-04-03T03:13:42.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9975ffb700 1 -- 172.21.2.111:0/2767620409 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9960009a40 con 0x7f997003d360 2021-04-03T03:13:42.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9975ffb700 1 -- 172.21.2.111:0/2767620409 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9960009da0 con 0x7f997003d360 2021-04-03T03:13:42.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/2767620409 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 msgr2=0x7f99700dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 --2- 172.21.2.111:0/2767620409 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700dbf70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f9970006f70 tx=0x7f9960006e90).stop 2021-04-03T03:13:42.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/2767620409 shutdown_connections 2021-04-03T03:13:42.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 --2- 172.21.2.111:0/2767620409 >> v2:172.21.2.115:3300/0 conn(0x7f99700064b0 0x7f9970006940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 --2- 172.21.2.111:0/2767620409 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700dbb90 0x7f9970042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 --2- 172.21.2.111:0/2767620409 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700dbf70 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/2767620409 >> 172.21.2.111:0/2767620409 conn(0x7f99700462c0 msgr2=0x7f99700466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/2767620409 shutdown_connections 2021-04-03T03:13:42.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/2767620409 wait complete. 2021-04-03T03:13:42.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 Processor -- start 2021-04-03T03:13:42.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- start start 2021-04-03T03:13:42.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700064b0 0x7f99700a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99700dbb90 0x7f9970042670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99700eab80 con 0x7f997003d360 2021-04-03T03:13:42.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99700a1680 con 0x7f99700dbb90 2021-04-03T03:13:42.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997009f7f0 con 0x7f99700064b0 2021-04-03T03:13:42.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53054/0 (socket says 172.21.2.111:53054) 2021-04-03T03:13:42.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 -- 172.21.2.111:0/1120803523 learned_addr learned my addr 172.21.2.111:0/1120803523 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:42.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 -- 172.21.2.111:0/1120803523 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700064b0 msgr2=0x7f99700a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700064b0 0x7f99700a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 -- 172.21.2.111:0/1120803523 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99700dbb90 msgr2=0x7f9970042670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99700dbb90 0x7f9970042670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 -- 172.21.2.111:0/1120803523 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997009f550 con 0x7f997003d360 2021-04-03T03:13:42.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f99767fc700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700ac0d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f9970006f70 tx=0x7f9960015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f996ffff700 1 -- 172.21.2.111:0/1120803523 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9960011db0 con 0x7f997003d360 2021-04-03T03:13:42.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99700a3270 con 0x7f997003d360 2021-04-03T03:13:42.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f996ffff700 1 -- 172.21.2.111:0/1120803523 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996001e460 con 0x7f997003d360 2021-04-03T03:13:42.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f996ffff700 1 -- 172.21.2.111:0/1120803523 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996001e7c0 con 0x7f997003d360 2021-04-03T03:13:42.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997009fb50 con 0x7f997003d360 2021-04-03T03:13:42.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99700a17f0 con 0x7f997003d360 2021-04-03T03:13:42.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f996ffff700 1 -- 172.21.2.111:0/1120803523 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9960025030 con 0x7f997003d360 2021-04-03T03:13:42.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f996ffff700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f995c056ef0 0x7f995c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f996ffff700 1 -- 172.21.2.111:0/1120803523 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9960071850 con 0x7f997003d360 2021-04-03T03:13:42.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9976ffd700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f995c056ef0 0x7f995c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.283+0000 7f9976ffd700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f995c056ef0 0x7f995c0593b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f9968002f30 tx=0x7f9968006940).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.287+0000 7f996ffff700 1 -- 172.21.2.111:0/1120803523 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9960046b30 con 0x7f997003d360 2021-04-03T03:13:42.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99700ac750 con 0x7f997003d360 2021-04-03T03:13:42.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f996ffff700 1 -- 172.21.2.111:0/1120803523 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f9960023020 con 0x7f997003d360 2021-04-03T03:13:42.430 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:42.431 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":25,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:13:42.208611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/167825619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":167825619},{"type":"v1","addr":"172.21.2.111:6835","nonce":167825619}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:13:42.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f995c056ef0 msgr2=0x7f995c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f995c056ef0 0x7f995c0593b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f9968002f30 tx=0x7f9968006940).stop 2021-04-03T03:13:42.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 msgr2=0x7f99700ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700ac0d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f9970006f70 tx=0x7f9960015000).stop 2021-04-03T03:13:42.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 shutdown_connections 2021-04-03T03:13:42.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f995c056ef0 0x7f995c0593b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99700dbb90 0x7f9970042670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99700064b0 0x7f99700a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 --2- 172.21.2.111:0/1120803523 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f997003d360 0x7f99700ac0d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 >> 172.21.2.111:0/1120803523 conn(0x7f99700462c0 msgr2=0x7f99700dacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 shutdown_connections 2021-04-03T03:13:42.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.415+0000 7f9977fff700 1 -- 172.21.2.111:0/1120803523 wait complete. 2021-04-03T03:13:42.435 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 25 2021-04-03T03:13:42.438 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:13:42.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 Processor -- start 2021-04-03T03:13:42.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- start start 2021-04-03T03:13:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fad1c14b700 0x7fad1c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0b0c90 0x7fad1c147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c0607d0 con 0x7fad1c0603f0 2021-04-03T03:13:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c0af400 con 0x7fad1c14b700 2021-04-03T03:13:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0af6a0 con 0x7fad1c0b0c90 2021-04-03T03:13:42.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fad1c14b700 0x7fad1c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fad1c14b700 0x7fad1c0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53060/0 (socket says 172.21.2.111:53060) 2021-04-03T03:13:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 -- 172.21.2.111:0/3132234018 learned_addr learned my addr 172.21.2.111:0/3132234018 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 -- 172.21.2.111:0/3132234018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0b0c90 msgr2=0x7fad1c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 --2- 172.21.2.111:0/3132234018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0b0c90 0x7fad1c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 -- 172.21.2.111:0/3132234018 >> v1:172.21.2.115:6789/0 conn(0x7fad1c0603f0 legacy=0x7fad1c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 -- 172.21.2.111:0/3132234018 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0afb40 con 0x7fad1c14b700 2021-04-03T03:13:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21a62700 1 --2- 172.21.2.111:0/3132234018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c14b700 0x7fad1c0b07a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fad1c06a3b0 tx=0x7fad18006e90).ready entity=mon.0 client_cookie=b636ae0d26e1beda server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21261700 1 -- 172.21.2.111:0/3132234018 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad180098e0 con 0x7fad1c14b700 2021-04-03T03:13:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21261700 1 -- 172.21.2.111:0/3132234018 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18009a40 con 0x7fad1c14b700 2021-04-03T03:13:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad21261700 1 -- 172.21.2.111:0/3132234018 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad18009de0 con 0x7fad1c14b700 2021-04-03T03:13:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- 172.21.2.111:0/3132234018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c14b700 msgr2=0x7fad1c0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 --2- 172.21.2.111:0/3132234018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c14b700 0x7fad1c0b07a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fad1c06a3b0 tx=0x7fad18006e90).stop 2021-04-03T03:13:42.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- 172.21.2.111:0/3132234018 shutdown_connections 2021-04-03T03:13:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 --2- 172.21.2.111:0/3132234018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0b0c90 0x7fad1c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 --2- 172.21.2.111:0/3132234018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c14b700 0x7fad1c0b07a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- 172.21.2.111:0/3132234018 >> 172.21.2.111:0/3132234018 conn(0x7fad1c0b61c0 msgr2=0x7fad1c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- 172.21.2.111:0/3132234018 shutdown_connections 2021-04-03T03:13:42.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.479+0000 7fad23265700 1 -- 172.21.2.111:0/3132234018 wait complete. 2021-04-03T03:13:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 Processor -- start 2021-04-03T03:13:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 -- start start 2021-04-03T03:13:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0603f0 0x7fad1c0abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c0b0c90 0x7fad1c062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad1c14b700 0x7fad1c0673f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0aa300 con 0x7fad1c0b0c90 2021-04-03T03:13:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0ab8a0 con 0x7fad1c14b700 2021-04-03T03:13:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0ab590 con 0x7fad1c0603f0 2021-04-03T03:13:42.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c0b0c90 0x7fad1c062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c0b0c90 0x7fad1c062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53064/0 (socket says 172.21.2.111:53064) 2021-04-03T03:13:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 -- 172.21.2.111:0/2521152090 learned_addr learned my addr 172.21.2.111:0/2521152090 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad22a64700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad1c14b700 0x7fad1c0673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad22263700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0603f0 0x7fad1c0abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 -- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0603f0 msgr2=0x7fad1c0abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0603f0 0x7fad1c0abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 -- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad1c14b700 msgr2=0x7fad1c0673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad1c14b700 0x7fad1c0673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 -- 172.21.2.111:0/2521152090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c064f90 con 0x7fad1c0b0c90 2021-04-03T03:13:42.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad21a62700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c0b0c90 0x7fad1c062bf0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fad1c06a3b0 tx=0x7fad18015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad0f7fe700 1 -- 172.21.2.111:0/2521152090 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad18011cc0 con 0x7fad1c0b0c90 2021-04-03T03:13:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad0f7fe700 1 -- 172.21.2.111:0/2521152090 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18011e20 con 0x7fad1c0b0c90 2021-04-03T03:13:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c0678e0 con 0x7fad1c0b0c90 2021-04-03T03:13:42.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad0f7fe700 1 -- 172.21.2.111:0/2521152090 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1801e650 con 0x7fad1c0b0c90 2021-04-03T03:13:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c067d20 con 0x7fad1c0b0c90 2021-04-03T03:13:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c057660 con 0x7fad1c0b0c90 2021-04-03T03:13:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad0f7fe700 1 -- 172.21.2.111:0/2521152090 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fad1801a040 con 0x7fad1c0b0c90 2021-04-03T03:13:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad0f7fe700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facfc056f20 0x7facfc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad0f7fe700 1 -- 172.21.2.111:0/2521152090 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fad18071400 con 0x7fad1c0b0c90 2021-04-03T03:13:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad22263700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facfc056f20 0x7facfc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.483+0000 7fad22263700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facfc056f20 0x7facfc0593e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fad1000a000 tx=0x7fad10008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.487+0000 7fad0f7fe700 1 -- 172.21.2.111:0/2521152090 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fad180466e0 con 0x7fad1c0b0c90 2021-04-03T03:13:42.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad1c065120 con 0x7fad1c0b0c90 2021-04-03T03:13:42.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad0f7fe700 1 -- 172.21.2.111:0/2521152090 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fad18023020 con 0x7fad1c0b0c90 2021-04-03T03:13:42.596 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:42.597 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":64,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:13:42.209640+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:13:39.120055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-04-03T03:13:40.173978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:13:42.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facfc056f20 msgr2=0x7facfc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facfc056f20 0x7facfc0593e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fad1000a000 tx=0x7fad10008040).stop 2021-04-03T03:13:42.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c0b0c90 msgr2=0x7fad1c062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c0b0c90 0x7fad1c062bf0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fad1c06a3b0 tx=0x7fad18015000).stop 2021-04-03T03:13:42.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 shutdown_connections 2021-04-03T03:13:42.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facfc056f20 0x7facfc0593e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad1c14b700 0x7fad1c0673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad1c0603f0 0x7fad1c0abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 --2- 172.21.2.111:0/2521152090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad1c0b0c90 0x7fad1c062bf0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 >> 172.21.2.111:0/2521152090 conn(0x7fad1c0b61c0 msgr2=0x7fad1c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 shutdown_connections 2021-04-03T03:13:42.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.579+0000 7fad23265700 1 -- 172.21.2.111:0/2521152090 wait complete. 2021-04-03T03:13:42.604 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:13:42.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 Processor -- start 2021-04-03T03:13:42.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 -- start start 2021-04-03T03:13:42.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f33d4046ce0 0x7f33d403f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d4041240 con 0x7f33d403d350 2021-04-03T03:13:42.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d40f1850 con 0x7f33d4042380 2021-04-03T03:13:42.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d40470c0 con 0x7f33d4046ce0 2021-04-03T03:13:42.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33db7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f33d403d350 0x7f33d4041c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53864/0 (socket says 172.21.2.111:53864) 2021-04-03T03:13:42.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33db7fe700 1 -- 172.21.2.111:0/3052241833 learned_addr learned my addr 172.21.2.111:0/3052241833 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:42.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871286186 0 0) 0x7f33d4041240 con 0x7f33d403d350 2021-04-03T03:13:42.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c4003240 con 0x7f33d403d350 2021-04-03T03:13:42.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 --2- 172.21.2.111:0/3052241833 >> v2:172.21.2.115:3300/0 conn(0x7f33d4046ce0 0x7f33d403f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1764546965 0 0) 0x7f33c4003240 con 0x7f33d403d350 2021-04-03T03:13:42.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 >> v2:172.21.2.115:3300/0 conn(0x7f33d4046ce0 msgr2=0x7f33d403f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 --2- 172.21.2.111:0/3052241833 >> v2:172.21.2.115:3300/0 conn(0x7f33d4046ce0 0x7f33d403f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 >> v1:172.21.2.115:6789/0 conn(0x7f33d4042380 legacy=0x7f33d40f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d40400d0 con 0x7f33d403d350 2021-04-03T03:13:42.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f33cc0016d0 con 0x7f33d403d350 2021-04-03T03:13:42.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 >> v1:172.21.2.111:6789/0 conn(0x7f33d403d350 legacy=0x7f33d4041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33c4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33c40044e0 0x7f33c4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33c4004e50 0x7f33c400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40470c0 con 0x7f33c4003800 2021-04-03T03:13:42.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4005300 con 0x7f33c40044e0 2021-04-03T03:13:42.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c404fc30 con 0x7f33c4004e50 2021-04-03T03:13:42.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33c4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 -- 172.21.2.111:0/3052241833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33c4004e50 msgr2=0x7f33c400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33c4004e50 0x7f33c400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 -- 172.21.2.111:0/3052241833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33c40044e0 msgr2=0x7f33c4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33c40044e0 0x7f33c4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:42.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 -- 172.21.2.111:0/3052241833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4003240 con 0x7f33c4003800 2021-04-03T03:13:42.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e1eb5700 1 -- 172.21.2.111:0/3052241833 reap_dead start 2021-04-03T03:13:42.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33dbfff700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33c4003c00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f33d000a000 tx=0x7f33d0008040).ready entity=mon.0 client_cookie=cc4b64eb2272cef9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d000da60 con 0x7f33c4003800 2021-04-03T03:13:42.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d000dbc0 con 0x7f33c4003800 2021-04-03T03:13:42.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33daffd700 1 -- 172.21.2.111:0/3052241833 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d0013480 con 0x7f33c4003800 2021-04-03T03:13:42.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 -- 172.21.2.111:0/3052241833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 msgr2=0x7f33c4003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.647+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33c4003c00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f33d000a000 tx=0x7f33d0008040).stop 2021-04-03T03:13:42.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- 172.21.2.111:0/3052241833 shutdown_connections 2021-04-03T03:13:42.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/3052241833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33c4003c00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- 172.21.2.111:0/3052241833 >> 172.21.2.111:0/3052241833 conn(0x7f33d400d0c0 msgr2=0x7f33d40462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- 172.21.2.111:0/3052241833 shutdown_connections 2021-04-03T03:13:42.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- 172.21.2.111:0/3052241833 wait complete. 2021-04-03T03:13:42.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 Processor -- start 2021-04-03T03:13:42.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- start start 2021-04-03T03:13:42.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33d403c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33d40d2610 0x7f33d40d43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33d4039a90 0x7f33d4039f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d403c200 con 0x7f33c4003800 2021-04-03T03:13:42.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40d2b20 con 0x7f33d40d2610 2021-04-03T03:13:42.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d403c3f0 con 0x7f33d4039a90 2021-04-03T03:13:42.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33d403c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33d403c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53084/0 (socket says 172.21.2.111:53084) 2021-04-03T03:13:42.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 -- 172.21.2.111:0/134538331 learned_addr learned my addr 172.21.2.111:0/134538331 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:42.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33db7fe700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33d40d2610 0x7f33d40d43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 -- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33d4039a90 msgr2=0x7f33d4039f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e1eb5700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33d4039a90 0x7f33d4039f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33d4039a90 0x7f33d4039f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 -- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33d40d2610 msgr2=0x7f33d40d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33d40d2610 0x7f33d40d43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 -- 172.21.2.111:0/134538331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d403a430 con 0x7f33c4003800 2021-04-03T03:13:42.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33dbfff700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33d403c890 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f33d000a000 tx=0x7f33d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33d8ff9700 1 -- 172.21.2.111:0/134538331 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d0021070 con 0x7f33c4003800 2021-04-03T03:13:42.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33d8ff9700 1 -- 172.21.2.111:0/134538331 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d001d440 con 0x7f33c4003800 2021-04-03T03:13:42.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d40458e0 con 0x7f33c4003800 2021-04-03T03:13:42.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d40440e0 con 0x7f33c4003800 2021-04-03T03:13:42.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33d8ff9700 1 -- 172.21.2.111:0/134538331 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d001d7a0 con 0x7f33c4003800 2021-04-03T03:13:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d40d2ce0 con 0x7f33c4003800 2021-04-03T03:13:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33d8ff9700 1 -- 172.21.2.111:0/134538331 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f33d001d900 con 0x7f33c4003800 2021-04-03T03:13:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33d8ff9700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33bc056f70 0x7f33bc059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33d8ff9700 1 -- 172.21.2.111:0/134538331 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f33d0071ca0 con 0x7f33c4003800 2021-04-03T03:13:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.651+0000 7f33db7fe700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33bc056f70 0x7f33bc059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.655+0000 7f33db7fe700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33bc056f70 0x7f33bc059430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f33cc00b000 tx=0x7f33d4040260).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.655+0000 7f33d8ff9700 1 -- 172.21.2.111:0/134538331 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f33d0046f80 con 0x7f33c4003800 2021-04-03T03:13:42.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.779+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f33d403c5b0 con 0x7f33c4003800 2021-04-03T03:13:42.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.783+0000 7f33d8ff9700 1 -- 172.21.2.111:0/134538331 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f33d004a5a0 con 0x7f33c4003800 2021-04-03T03:13:42.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33bc056f70 msgr2=0x7f33bc059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33bc056f70 0x7f33bc059430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f33cc00b000 tx=0x7f33d4040260).stop 2021-04-03T03:13:42.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 msgr2=0x7f33d403c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33d403c890 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f33d000a000 tx=0x7f33d0008040).stop 2021-04-03T03:13:42.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 shutdown_connections 2021-04-03T03:13:42.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f33d4039a90 0x7f33d4039f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f33c4003800 0x7f33d403c890 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f33bc056f70 0x7f33bc059430 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 --2- 172.21.2.111:0/134538331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f33d40d2610 0x7f33d40d43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 >> 172.21.2.111:0/134538331 conn(0x7f33d400d0c0 msgr2=0x7f33d40f3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 shutdown_connections 2021-04-03T03:13:42.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.787+0000 7f33e0a3b700 1 -- 172.21.2.111:0/134538331 wait complete. 2021-04-03T03:13:42.805 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:13:42.808 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:13:42.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.851+0000 7f3699144700 1 Processor -- start 2021-04-03T03:13:42.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.851+0000 7f3699144700 1 -- start start 2021-04-03T03:13:42.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.851+0000 7f3699144700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3694080c60 0x7f3694025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.851+0000 7f3699144700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3694081090 con 0x7f3694153440 2021-04-03T03:13:42.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36940b3ca0 con 0x7f36940b38c0 2021-04-03T03:13:42.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3694025970 con 0x7f3694080c60 2021-04-03T03:13:42.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3693fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3694153440 0x7f36940af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53884/0 (socket says 172.21.2.111:53884) 2021-04-03T03:13:42.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3693fff700 1 -- 172.21.2.111:0/3613753228 learned_addr learned my addr 172.21.2.111:0/3613753228 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:42.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 --2- 172.21.2.111:0/3613753228 >> v2:172.21.2.111:3300/0 conn(0x7f3694080c60 0x7f3694025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36937fe700 1 -- 172.21.2.111:0/3613753228 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259432592 0 0) 0x7f36840010d0 con 0x7f3694153440 2021-04-03T03:13:42.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36937fe700 1 -- 172.21.2.111:0/3613753228 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3680003240 con 0x7f3694153440 2021-04-03T03:13:42.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 -- 172.21.2.111:0/3613753228 >> v1:172.21.2.115:6789/0 conn(0x7f36940b38c0 legacy=0x7f3694080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:42.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 -- 172.21.2.111:0/3613753228 >> v1:172.21.2.111:6789/0 conn(0x7f3694153440 legacy=0x7f36940af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 -- 172.21.2.111:0/3613753228 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694025b90 con 0x7f3694080c60 2021-04-03T03:13:42.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 --2- 172.21.2.111:0/3613753228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694080c60 0x7f3694025480 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f3694063a70 tx=0x7f368c0091d0).ready entity=mon.0 client_cookie=657b7e47075285dd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36937fe700 1 -- 172.21.2.111:0/3613753228 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368c00b960 con 0x7f3694080c60 2021-04-03T03:13:42.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36937fe700 1 -- 172.21.2.111:0/3613753228 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c00bac0 con 0x7f3694080c60 2021-04-03T03:13:42.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36937fe700 1 -- 172.21.2.111:0/3613753228 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368c00bc30 con 0x7f3694080c60 2021-04-03T03:13:42.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- 172.21.2.111:0/3613753228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694080c60 msgr2=0x7f3694025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 --2- 172.21.2.111:0/3613753228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694080c60 0x7f3694025480 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f3694063a70 tx=0x7f368c0091d0).stop 2021-04-03T03:13:42.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- 172.21.2.111:0/3613753228 shutdown_connections 2021-04-03T03:13:42.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 --2- 172.21.2.111:0/3613753228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694080c60 0x7f3694025480 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- 172.21.2.111:0/3613753228 >> 172.21.2.111:0/3613753228 conn(0x7f3694061610 msgr2=0x7f3694061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:42.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- 172.21.2.111:0/3613753228 shutdown_connections 2021-04-03T03:13:42.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- 172.21.2.111:0/3613753228 wait complete. 2021-04-03T03:13:42.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 Processor -- start 2021-04-03T03:13:42.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- start start 2021-04-03T03:13:42.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3694080c60 0x7f3694067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36940b38c0 0x7f3694064b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694153440 0x7f369413a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940660d0 con 0x7f3694153440 2021-04-03T03:13:42.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940aa4a0 con 0x7f36940b38c0 2021-04-03T03:13:42.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940640c0 con 0x7f3694080c60 2021-04-03T03:13:42.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694153440 0x7f369413a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694153440 0x7f369413a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53100/0 (socket says 172.21.2.111:53100) 2021-04-03T03:13:42.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 -- 172.21.2.111:0/4052396382 learned_addr learned my addr 172.21.2.111:0/4052396382 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:42.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3698943700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3694080c60 0x7f3694067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3693fff700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36940b38c0 0x7f3694064b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 -- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3694080c60 msgr2=0x7f3694067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3694080c60 0x7f3694067ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 -- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36940b38c0 msgr2=0x7f3694064b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:42.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36940b38c0 0x7f3694064b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:42.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 -- 172.21.2.111:0/4052396382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940645e0 con 0x7f3694153440 2021-04-03T03:13:42.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3693fff700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36940b38c0 0x7f3694064b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:13:42.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f369a5be700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694153440 0x7f369413a0b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f3694063a70 tx=0x7f368c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36917fa700 1 -- 172.21.2.111:0/4052396382 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368c01f3f0 con 0x7f3694153440 2021-04-03T03:13:42.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36917fa700 1 -- 172.21.2.111:0/4052396382 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c01f550 con 0x7f3694153440 2021-04-03T03:13:42.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369415b080 con 0x7f3694153440 2021-04-03T03:13:42.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f36917fa700 1 -- 172.21.2.111:0/4052396382 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368c01f8b0 con 0x7f3694153440 2021-04-03T03:13:42.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.855+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3694150bd0 con 0x7f3694153440 2021-04-03T03:13:42.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.859+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36940aa6a0 con 0x7f3694153440 2021-04-03T03:13:42.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.859+0000 7f36917fa700 1 -- 172.21.2.111:0/4052396382 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f368c026030 con 0x7f3694153440 2021-04-03T03:13:42.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.859+0000 7f36917fa700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3678056f20 0x7f36780593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:42.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.859+0000 7f36917fa700 1 -- 172.21.2.111:0/4052396382 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f368c070e00 con 0x7f3694153440 2021-04-03T03:13:42.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.859+0000 7f3698943700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3678056f20 0x7f36780593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:42.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.859+0000 7f3698943700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3678056f20 0x7f36780593e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f367c008000 tx=0x7f367c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:42.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.859+0000 7f36917fa700 1 -- 172.21.2.111:0/4052396382 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f368c046190 con 0x7f3694153440 2021-04-03T03:13:43.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.987+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3694057660 con 0x7f3694153440 2021-04-03T03:13:43.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.991+0000 7f36917fa700 1 -- 172.21.2.111:0/4052396382 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f368c0150e0 con 0x7f3694153440 2021-04-03T03:13:43.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3678056f20 msgr2=0x7f36780593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3678056f20 0x7f36780593e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f367c008000 tx=0x7f367c009040).stop 2021-04-03T03:13:43.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694153440 msgr2=0x7f369413a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694153440 0x7f369413a0b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f3694063a70 tx=0x7f368c016000).stop 2021-04-03T03:13:43.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 shutdown_connections 2021-04-03T03:13:43.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3694153440 0x7f369413a0b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3694080c60 0x7f3694067ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3678056f20 0x7f36780593e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 --2- 172.21.2.111:0/4052396382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36940b38c0 0x7f3694064b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 >> 172.21.2.111:0/4052396382 conn(0x7f3694061610 msgr2=0x7f36940af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 shutdown_connections 2021-04-03T03:13:43.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:42.995+0000 7f3699144700 1 -- 172.21.2.111:0/4052396382 wait complete. 2021-04-03T03:13:43.013 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:13:43.018 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:13:43.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 Processor -- start 2021-04-03T03:13:43.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- start start 2021-04-03T03:13:43.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 0x7fa578042380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa578041240 con 0x7fa57803d350 2021-04-03T03:13:43.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5780f1850 con 0x7fa578046ce0 2021-04-03T03:13:43.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa578042870 con 0x7fa57803f650 2021-04-03T03:13:43.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57effd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa57803d350 0x7fa5780f51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53900/0 (socket says 172.21.2.111:53900) 2021-04-03T03:13:43.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57effd700 1 -- 172.21.2.111:0/4031332117 learned_addr learned my addr 172.21.2.111:0/4031332117 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:43.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862393112 0 0) 0x7fa578041240 con 0x7fa57803d350 2021-04-03T03:13:43.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa56c003240 con 0x7fa57803d350 2021-04-03T03:13:43.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 0x7fa578042380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4223650108 0 0) 0x7fa56c003240 con 0x7fa57803d350 2021-04-03T03:13:43.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 msgr2=0x7fa578042380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 0x7fa578042380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 >> v1:172.21.2.115:6789/0 conn(0x7fa578046ce0 legacy=0x7fa578041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780470c0 con 0x7fa57803d350 2021-04-03T03:13:43.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa5680016d0 con 0x7fa57803d350 2021-04-03T03:13:43.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 >> v1:172.21.2.111:6789/0 conn(0x7fa57803d350 legacy=0x7fa5780f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa56c0037e0 0x7fa56c003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 0x7fa56c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa56c0049d0 0x7fa56c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa578042870 con 0x7fa56c0049d0 2021-04-03T03:13:43.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c004280 con 0x7fa56c0037e0 2021-04-03T03:13:43.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c04f740 con 0x7fa57803f650 2021-04-03T03:13:43.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa56c0049d0 0x7fa56c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57effd700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 0x7fa56c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57f7fe700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa56c0037e0 0x7fa56c003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 -- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 msgr2=0x7fa56c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa57803f650 0x7fa56c0044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 -- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa56c0037e0 msgr2=0x7fa56c003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa56c0037e0 0x7fa56c003c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 -- 172.21.2.111:0/4031332117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780470c0 con 0x7fa56c0049d0 2021-04-03T03:13:43.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa56c0049d0 0x7fa56c004e50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fa580066000 tx=0x7fa58005ad70).ready entity=mon.0 client_cookie=aaada414312a660b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58005df60 con 0x7fa56c0049d0 2021-04-03T03:13:43.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5800699f0 con 0x7fa56c0049d0 2021-04-03T03:13:43.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57e7fc700 1 -- 172.21.2.111:0/4031332117 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58005b040 con 0x7fa56c0049d0 2021-04-03T03:13:43.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- 172.21.2.111:0/4031332117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa56c0049d0 msgr2=0x7fa56c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 --2- 172.21.2.111:0/4031332117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa56c0049d0 0x7fa56c004e50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fa580066000 tx=0x7fa58005ad70).stop 2021-04-03T03:13:43.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa585ac8700 1 -- 172.21.2.111:0/4031332117 reap_dead start 2021-04-03T03:13:43.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- 172.21.2.111:0/4031332117 shutdown_connections 2021-04-03T03:13:43.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- 172.21.2.111:0/4031332117 >> 172.21.2.111:0/4031332117 conn(0x7fa57800d0c0 msgr2=0x7fa5780462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- 172.21.2.111:0/4031332117 shutdown_connections 2021-04-03T03:13:43.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.063+0000 7fa57ffff700 1 -- 172.21.2.111:0/4031332117 wait complete. 2021-04-03T03:13:43.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 Processor -- start 2021-04-03T03:13:43.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 -- start start 2021-04-03T03:13:43.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5780459e0 0x7fa5780d8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa578039a30 0x7fa5780c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5780cdfa0 0x7fa5780ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780ce940 con 0x7fa578039a30 2021-04-03T03:13:43.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780bfad0 con 0x7fa5780cdfa0 2021-04-03T03:13:43.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780bfd70 con 0x7fa5780459e0 2021-04-03T03:13:43.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa578039a30 0x7fa5780c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa578039a30 0x7fa5780c04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53116/0 (socket says 172.21.2.111:53116) 2021-04-03T03:13:43.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 -- 172.21.2.111:0/2402065898 learned_addr learned my addr 172.21.2.111:0/2402065898 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:43.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa585ac8700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5780cdfa0 0x7fa5780ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57f7fe700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5780459e0 0x7fa5780d8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 -- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5780459e0 msgr2=0x7fa5780d8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5780459e0 0x7fa5780d8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 -- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5780cdfa0 msgr2=0x7fa5780ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5780cdfa0 0x7fa5780ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 -- 172.21.2.111:0/2402065898 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780c0010 con 0x7fa578039a30 2021-04-03T03:13:43.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57effd700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa578039a30 0x7fa5780c04b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fa568008000 tx=0x7fa568009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa577fff700 1 -- 172.21.2.111:0/2402065898 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa568007170 con 0x7fa578039a30 2021-04-03T03:13:43.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa577fff700 1 -- 172.21.2.111:0/2402065898 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680072d0 con 0x7fa578039a30 2021-04-03T03:13:43.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa577fff700 1 -- 172.21.2.111:0/2402065898 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa56800ba10 con 0x7fa578039a30 2021-04-03T03:13:43.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa578045dc0 con 0x7fa578039a30 2021-04-03T03:13:43.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5780bc4f0 con 0x7fa578039a30 2021-04-03T03:13:43.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5780c01a0 con 0x7fa578039a30 2021-04-03T03:13:43.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa577fff700 1 -- 172.21.2.111:0/2402065898 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa56800bb70 con 0x7fa578039a30 2021-04-03T03:13:43.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa577fff700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa564056f70 0x7fa564059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa577fff700 1 -- 172.21.2.111:0/2402065898 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa5680705c0 con 0x7fa578039a30 2021-04-03T03:13:43.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57f7fe700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa564056f70 0x7fa564059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.067+0000 7fa57f7fe700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa564056f70 0x7fa564059430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa570009000 tx=0x7fa570009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.071+0000 7fa577fff700 1 -- 172.21.2.111:0/2402065898 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa5680458a0 con 0x7fa578039a30 2021-04-03T03:13:43.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.199+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa578004d90 con 0x7fa578039a30 2021-04-03T03:13:43.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa577fff700 1 -- 172.21.2.111:0/2402065898 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa56800be00 con 0x7fa578039a30 2021-04-03T03:13:43.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa564056f70 msgr2=0x7fa564059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa564056f70 0x7fa564059430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa570009000 tx=0x7fa570009040).stop 2021-04-03T03:13:43.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa578039a30 msgr2=0x7fa5780c04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa578039a30 0x7fa5780c04b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fa568008000 tx=0x7fa568009040).stop 2021-04-03T03:13:43.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 shutdown_connections 2021-04-03T03:13:43.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa564056f70 0x7fa564059430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5780cdfa0 0x7fa5780ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5780459e0 0x7fa5780d8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 --2- 172.21.2.111:0/2402065898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa578039a30 0x7fa5780c04b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 >> 172.21.2.111:0/2402065898 conn(0x7fa57800d0c0 msgr2=0x7fa5780f3600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 shutdown_connections 2021-04-03T03:13:43.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.207+0000 7fa57ffff700 1 -- 172.21.2.111:0/2402065898 wait complete. 2021-04-03T03:13:43.226 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:13:43.230 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:13:43.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 Processor -- start 2021-04-03T03:13:43.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- start start 2021-04-03T03:13:43.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 0x7f83400b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400607d0 con 0x7f83400b0c90 2021-04-03T03:13:43.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400af400 con 0x7f834014b700 2021-04-03T03:13:43.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400af6a0 con 0x7f83400603f0 2021-04-03T03:13:43.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f83400b0c90 0x7f8340147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53918/0 (socket says 172.21.2.111:53918) 2021-04-03T03:13:43.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 -- 172.21.2.111:0/2934050234 learned_addr learned my addr 172.21.2.111:0/2934050234 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:43.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702199638 0 0) 0x7f83400607d0 con 0x7f83400b0c90 2021-04-03T03:13:43.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8320003240 con 0x7f83400b0c90 2021-04-03T03:13:43.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83456d9700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 0x7f83400b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2390612631 0 0) 0x7f8320003240 con 0x7f83400b0c90 2021-04-03T03:13:43.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 msgr2=0x7f83400b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 0x7f83400b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 >> v1:172.21.2.115:6789/0 conn(0x7f834014b700 legacy=0x7f83400b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400afb40 con 0x7f83400b0c90 2021-04-03T03:13:43.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f83280016d0 con 0x7f83400b0c90 2021-04-03T03:13:43.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 >> v1:172.21.2.111:6789/0 conn(0x7f83400b0c90 legacy=0x7f8340147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8320003760 0x7f8320003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 0x7f83200044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83200049d0 0x7f8320004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400af6a0 con 0x7f83200049d0 2021-04-03T03:13:43.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320004200 con 0x7f8320003760 2021-04-03T03:13:43.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832000b730 con 0x7f83400603f0 2021-04-03T03:13:43.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83200049d0 0x7f8320004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8344ed8700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 0x7f83200044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83456d9700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8320003760 0x7f8320003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 -- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 msgr2=0x7f83200044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83400603f0 0x7f83200044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 -- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8320003760 msgr2=0x7f8320003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8320003760 0x7f8320003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 -- 172.21.2.111:0/2934050234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320003240 con 0x7f83200049d0 2021-04-03T03:13:43.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83200049d0 0x7f8320004e50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f83280013e0 tx=0x7f83400ae670).ready entity=mon.0 client_cookie=3a7dd419cbe95cd0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83280070d0 con 0x7f83200049d0 2021-04-03T03:13:43.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328007230 con 0x7f83200049d0 2021-04-03T03:13:43.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f833ffff700 1 -- 172.21.2.111:0/2934050234 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83280055f0 con 0x7f83200049d0 2021-04-03T03:13:43.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- 172.21.2.111:0/2934050234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83200049d0 msgr2=0x7f8320004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 --2- 172.21.2.111:0/2934050234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83200049d0 0x7f8320004e50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f83280013e0 tx=0x7f83400ae670).stop 2021-04-03T03:13:43.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f8345eda700 1 -- 172.21.2.111:0/2934050234 reap_dead start 2021-04-03T03:13:43.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- 172.21.2.111:0/2934050234 shutdown_connections 2021-04-03T03:13:43.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- 172.21.2.111:0/2934050234 >> 172.21.2.111:0/2934050234 conn(0x7f83400b61c0 msgr2=0x7f834005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- 172.21.2.111:0/2934050234 shutdown_connections 2021-04-03T03:13:43.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.271+0000 7f83466db700 1 -- 172.21.2.111:0/2934050234 wait complete. 2021-04-03T03:13:43.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 Processor -- start 2021-04-03T03:13:43.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 -- start start 2021-04-03T03:13:43.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f834014e3f0 0x7f8340146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83400aa120 0x7f8340146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8340064120 0x7f83400645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834014e7d0 con 0x7f834014e3f0 2021-04-03T03:13:43.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400ab200 con 0x7f83400aa120 2021-04-03T03:13:43.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400ab4a0 con 0x7f8340064120 2021-04-03T03:13:43.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f834014e3f0 0x7f8340146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f834014e3f0 0x7f8340146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53136/0 (socket says 172.21.2.111:53136) 2021-04-03T03:13:43.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 -- 172.21.2.111:0/3447140956 learned_addr learned my addr 172.21.2.111:0/3447140956 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:43.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f8344ed8700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83400aa120 0x7f8340146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f8345eda700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8340064120 0x7f83400645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 -- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8340064120 msgr2=0x7f83400645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8340064120 0x7f83400645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 -- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83400aa120 msgr2=0x7f8340146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83400aa120 0x7f8340146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 -- 172.21.2.111:0/3447140956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400ab740 con 0x7f834014e3f0 2021-04-03T03:13:43.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f8345eda700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8340064120 0x7f83400645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:13:43.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83456d9700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f834014e3f0 0x7f8340146930 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f833000d000 tx=0x7f8330008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f833e7fc700 1 -- 172.21.2.111:0/3447140956 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f833000dc20 con 0x7f834014e3f0 2021-04-03T03:13:43.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f833e7fc700 1 -- 172.21.2.111:0/3447140956 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833000c040 con 0x7f834014e3f0 2021-04-03T03:13:43.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f833e7fc700 1 -- 172.21.2.111:0/3447140956 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8330013460 con 0x7f834014e3f0 2021-04-03T03:13:43.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8340147310 con 0x7f834014e3f0 2021-04-03T03:13:43.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400a9db0 con 0x7f834014e3f0 2021-04-03T03:13:43.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83400ab010 con 0x7f834014e3f0 2021-04-03T03:13:43.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f833e7fc700 1 -- 172.21.2.111:0/3447140956 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8330019070 con 0x7f834014e3f0 2021-04-03T03:13:43.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f833e7fc700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8324056ec0 0x7f8324059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f833e7fc700 1 -- 172.21.2.111:0/3447140956 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f83300709f0 con 0x7f834014e3f0 2021-04-03T03:13:43.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f8344ed8700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8324056ec0 0x7f8324059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.275+0000 7f8344ed8700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8324056ec0 0x7f8324059380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8338003000 tx=0x7f8338007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.279+0000 7f833e7fc700 1 -- 172.21.2.111:0/3447140956 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8330072e60 con 0x7f834014e3f0 2021-04-03T03:13:43.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83400ab8d0 con 0x7f834014e3f0 2021-04-03T03:13:43.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f833e7fc700 1 -- 172.21.2.111:0/3447140956 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f8330045de0 con 0x7f834014e3f0 2021-04-03T03:13:43.420 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:43.420 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":25,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:13:42.208611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/167825619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":167825619},{"type":"v1","addr":"172.21.2.111:6835","nonce":167825619}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:13:43.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8324056ec0 msgr2=0x7f8324059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8324056ec0 0x7f8324059380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8338003000 tx=0x7f8338007b70).stop 2021-04-03T03:13:43.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f834014e3f0 msgr2=0x7f8340146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f834014e3f0 0x7f8340146930 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f833000d000 tx=0x7f8330008040).stop 2021-04-03T03:13:43.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 shutdown_connections 2021-04-03T03:13:43.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8340064120 0x7f83400645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f834014e3f0 0x7f8340146930 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8324056ec0 0x7f8324059380 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 --2- 172.21.2.111:0/3447140956 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83400aa120 0x7f8340146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 >> 172.21.2.111:0/3447140956 conn(0x7f83400b61c0 msgr2=0x7f8340148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 shutdown_connections 2021-04-03T03:13:43.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.403+0000 7f83466db700 1 -- 172.21.2.111:0/3447140956 wait complete. 2021-04-03T03:13:43.424 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 25 2021-04-03T03:13:43.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:13:41.205623+0000', 'modified': '2021-04-03T03:13:42.208611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5934}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5934': {'gid': 5934, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/167825619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 167825619}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 167825619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:13:43.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:13:43.630 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:13:43.716 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5934,"want_state":"up:active","state":"up:active","rank_uptime":2.4912586220000001,"mdsmap_epoch":25,"osdmap_epoch":64,"osdmap_epoch_barrier":63,"uptime":6.6442847199999999} 2021-04-03T03:13:43.716 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5934, "mdsmap_epoch": 25, "osdmap_epoch": 64, "osdmap_epoch_barrier": 63, "rank_uptime": 2.491258622, "state": "up:active", "uptime": 6.64428472, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:13:43.717 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:13:43.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f757df9d700 1 Processor -- start 2021-04-03T03:13:43.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f757df9d700 1 -- start start 2021-04-03T03:13:43.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f757df9d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f75700d07c0 0x7f75700d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f757df9d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7570006640 con 0x7f7570006a30 2021-04-03T03:13:43.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f757df9d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75700d4700 con 0x7f7570006260 2021-04-03T03:13:43.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f757df9d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757003fb20 con 0x7f75700d07c0 2021-04-03T03:13:43.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f7577fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7570006a30 0x7f75700d3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53938/0 (socket says 172.21.2.111:53938) 2021-04-03T03:13:43.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.755+0000 7f7577fff700 1 -- 172.21.2.111:0/1343290549 learned_addr learned my addr 172.21.2.111:0/1343290549 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:43.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636193904 0 0) 0x7f7570006640 con 0x7f7570006a30 2021-04-03T03:13:43.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7564003240 con 0x7f7570006a30 2021-04-03T03:13:43.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3096735612 0 0) 0x7f7564003240 con 0x7f7570006a30 2021-04-03T03:13:43.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 >> v2:172.21.2.115:3300/0 conn(0x7f75700d07c0 msgr2=0x7f75700d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 --2- 172.21.2.111:0/1343290549 >> v2:172.21.2.115:3300/0 conn(0x7f75700d07c0 0x7f75700d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 >> v1:172.21.2.115:6789/0 conn(0x7f7570006260 legacy=0x7f75700d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757003fdc0 con 0x7f7570006a30 2021-04-03T03:13:43.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f75680016d0 con 0x7f7570006a30 2021-04-03T03:13:43.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 >> v1:172.21.2.111:6789/0 conn(0x7f7570006a30 legacy=0x7f75700d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7564003800 0x7f7564003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75640044e0 0x7f7564004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7564004e50 0x7f756400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757003fb20 con 0x7f7564003800 2021-04-03T03:13:43.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564005300 con 0x7f75640044e0 2021-04-03T03:13:43.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756400bc20 con 0x7f7564004e50 2021-04-03T03:13:43.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757cf9b700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7564003800 0x7f7564003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75640044e0 0x7f7564004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757cf9b700 1 -- 172.21.2.111:0/1343290549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7564004e50 msgr2=0x7f756400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757cf9b700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7564004e50 0x7f756400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757cf9b700 1 -- 172.21.2.111:0/1343290549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75640044e0 msgr2=0x7f7564004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757cf9b700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f75640044e0 0x7f7564004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757cf9b700 1 -- 172.21.2.111:0/1343290549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564003240 con 0x7f7564003800 2021-04-03T03:13:43.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757d79c700 1 -- 172.21.2.111:0/1343290549 reap_dead start 2021-04-03T03:13:43.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757cf9b700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7564003800 0x7f7564003c00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f756c000f30 tx=0x7f756c007b50).ready entity=mon.0 client_cookie=e8f6de11ab02e6a8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f756c00c870 con 0x7f7564003800 2021-04-03T03:13:43.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c00c9d0 con 0x7f7564003800 2021-04-03T03:13:43.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f75777fe700 1 -- 172.21.2.111:0/1343290549 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f756c00cd30 con 0x7f7564003800 2021-04-03T03:13:43.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- 172.21.2.111:0/1343290549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7564003800 msgr2=0x7f7564003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7564003800 0x7f7564003c00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f756c000f30 tx=0x7f756c007b50).stop 2021-04-03T03:13:43.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- 172.21.2.111:0/1343290549 shutdown_connections 2021-04-03T03:13:43.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 --2- 172.21.2.111:0/1343290549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7564003800 0x7f7564003c00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- 172.21.2.111:0/1343290549 >> 172.21.2.111:0/1343290549 conn(0x7f75700462c0 msgr2=0x7f75700466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- 172.21.2.111:0/1343290549 shutdown_connections 2021-04-03T03:13:43.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- 172.21.2.111:0/1343290549 wait complete. 2021-04-03T03:13:43.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 Processor -- start 2021-04-03T03:13:43.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- start start 2021-04-03T03:13:43.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7564003800 0x7f75700d2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75700b8110 0x7f757003ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7570039ab0 0x7f75700b9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757003c1c0 con 0x7f75700b8110 2021-04-03T03:13:43.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757003a010 con 0x7f7564003800 2021-04-03T03:13:43.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757003be00 con 0x7f7570039ab0 2021-04-03T03:13:43.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75700b8110 0x7f757003ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75700b8110 0x7f757003ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53158/0 (socket says 172.21.2.111:53158) 2021-04-03T03:13:43.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 -- 172.21.2.111:0/3794667016 learned_addr learned my addr 172.21.2.111:0/3794667016 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:43.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 -- 172.21.2.111:0/3794667016 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7570039ab0 msgr2=0x7f75700b9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7570039ab0 0x7f75700b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 -- 172.21.2.111:0/3794667016 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7564003800 msgr2=0x7f75700d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7564003800 0x7f75700d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 -- 172.21.2.111:0/3794667016 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757003bf70 con 0x7f75700b8110 2021-04-03T03:13:43.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7577fff700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75700b8110 0x7f757003ac70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f756800b000 tx=0x7f7568001670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7575ffb700 1 -- 172.21.2.111:0/3794667016 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f756800c670 con 0x7f75700b8110 2021-04-03T03:13:43.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7575ffb700 1 -- 172.21.2.111:0/3794667016 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800c7d0 con 0x7f75700b8110 2021-04-03T03:13:43.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75700b9e00 con 0x7f75700b8110 2021-04-03T03:13:43.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f7575ffb700 1 -- 172.21.2.111:0/3794667016 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f756800cb30 con 0x7f75700b8110 2021-04-03T03:13:43.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.759+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75700ba2a0 con 0x7f75700b8110 2021-04-03T03:13:43.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.763+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757003a180 con 0x7f75700b8110 2021-04-03T03:13:43.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.763+0000 7f7575ffb700 1 -- 172.21.2.111:0/3794667016 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f756800cc90 con 0x7f75700b8110 2021-04-03T03:13:43.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.763+0000 7f7575ffb700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7558056fb0 0x7f7558059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.763+0000 7f7575ffb700 1 -- 172.21.2.111:0/3794667016 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7568070860 con 0x7f75700b8110 2021-04-03T03:13:43.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.763+0000 7f757cf9b700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7558056fb0 0x7f7558059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.763+0000 7f757cf9b700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7558056fb0 0x7f7558059470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f756c000f30 tx=0x7f756c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.763+0000 7f7575ffb700 1 -- 172.21.2.111:0/3794667016 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f757003a180 con 0x7f75700b8110 2021-04-03T03:13:43.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.855+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f757003a180 con 0x7f75700b8110 2021-04-03T03:13:43.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.855+0000 7f7575ffb700 1 -- 172.21.2.111:0/3794667016 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f757003a180 con 0x7f75700b8110 2021-04-03T03:13:43.873 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:43.874 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":64,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:13:42.209640+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:13:39.120055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-04-03T03:13:40.173978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:13:43.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7558056fb0 msgr2=0x7f7558059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7558056fb0 0x7f7558059470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f756c000f30 tx=0x7f756c015000).stop 2021-04-03T03:13:43.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75700b8110 msgr2=0x7f757003ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75700b8110 0x7f757003ac70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f756800b000 tx=0x7f7568001670).stop 2021-04-03T03:13:43.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 shutdown_connections 2021-04-03T03:13:43.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7570039ab0 0x7f75700b9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7558056fb0 0x7f7558059470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7564003800 0x7f75700d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 --2- 172.21.2.111:0/3794667016 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f75700b8110 0x7f757003ac70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 >> 172.21.2.111:0/3794667016 conn(0x7f75700462c0 msgr2=0x7f75700c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 shutdown_connections 2021-04-03T03:13:43.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.859+0000 7f757df9d700 1 -- 172.21.2.111:0/3794667016 wait complete. 2021-04-03T03:13:43.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:13:39.120055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:13:40.173978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:13:43.882 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:13:43.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.923+0000 7fba6b59e700 1 Processor -- start 2021-04-03T03:13:43.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.923+0000 7fba6b59e700 1 -- start start 2021-04-03T03:13:43.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.923+0000 7fba6b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fba64041060 0x7fba640dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.923+0000 7fba6b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba6403d560 0x7fba64041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.923+0000 7fba6b59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba640dc560 con 0x7fba640dbba0 2021-04-03T03:13:43.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.923+0000 7fba6b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6403de60 con 0x7fba64041060 2021-04-03T03:13:43.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.923+0000 7fba6b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640dc800 con 0x7fba6403d560 2021-04-03T03:13:43.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6ad9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba6403d560 0x7fba64041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba69d9b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fba64041060 0x7fba640dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba69d9b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fba64041060 0x7fba640dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:42876/0 (socket says 172.21.2.111:42876) 2021-04-03T03:13:43.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6ad9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba6403d560 0x7fba64041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:36514/0 (socket says 172.21.2.111:36514) 2021-04-03T03:13:43.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba69d9b700 1 -- 172.21.2.111:0/1729185737 learned_addr learned my addr 172.21.2.111:0/1729185737 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:43.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6ad9d700 1 -- 172.21.2.111:0/1729185737 >> v2:172.21.2.115:3300/0 conn(0x7fba64041060 msgr2=0x7fba640dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:43.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6ad9d700 1 --2- 172.21.2.111:0/1729185737 >> v2:172.21.2.115:3300/0 conn(0x7fba64041060 0x7fba640dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6ad9d700 1 -- 172.21.2.111:0/1729185737 >> v1:172.21.2.115:6789/0 conn(0x7fba640dbba0 legacy=0x7fba6403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6ad9d700 1 -- 172.21.2.111:0/1729185737 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6403da30 con 0x7fba6403d560 2021-04-03T03:13:43.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba69d9b700 1 --2- 172.21.2.111:0/1729185737 >> v2:172.21.2.115:3300/0 conn(0x7fba64041060 0x7fba640dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:13:43.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6ad9d700 1 --2- 172.21.2.111:0/1729185737 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba6403d560 0x7fba64041650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fba6c06b5d0 tx=0x7fba6c05ad70).ready entity=mon.2 client_cookie=f1d5f9bc66a97030 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6959a700 1 -- 172.21.2.111:0/1729185737 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba6c05bda0 con 0x7fba6403d560 2021-04-03T03:13:43.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6959a700 1 -- 172.21.2.111:0/1729185737 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c05df60 con 0x7fba6403d560 2021-04-03T03:13:43.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6959a700 1 -- 172.21.2.111:0/1729185737 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba6c0699f0 con 0x7fba6403d560 2021-04-03T03:13:43.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 -- 172.21.2.111:0/1729185737 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba6403d560 msgr2=0x7fba64041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 --2- 172.21.2.111:0/1729185737 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba6403d560 0x7fba64041650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fba6c06b5d0 tx=0x7fba6c05ad70).stop 2021-04-03T03:13:43.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 -- 172.21.2.111:0/1729185737 shutdown_connections 2021-04-03T03:13:43.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 --2- 172.21.2.111:0/1729185737 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba6403d560 0x7fba64041650 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 --2- 172.21.2.111:0/1729185737 >> v2:172.21.2.115:3300/0 conn(0x7fba64041060 0x7fba640dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:43.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 -- 172.21.2.111:0/1729185737 >> 172.21.2.111:0/1729185737 conn(0x7fba640462b0 msgr2=0x7fba640466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:43.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 -- 172.21.2.111:0/1729185737 shutdown_connections 2021-04-03T03:13:43.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.927+0000 7fba6b59e700 1 -- 172.21.2.111:0/1729185737 wait complete. 2021-04-03T03:13:43.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 Processor -- start 2021-04-03T03:13:43.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 -- start start 2021-04-03T03:13:43.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fba6403d560 0x7fba64039b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fba64041060 0x7fba640d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba640dbba0 0x7fba640d6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640e7190 con 0x7fba6403d560 2021-04-03T03:13:43.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640df480 con 0x7fba64041060 2021-04-03T03:13:43.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640eb470 con 0x7fba640dbba0 2021-04-03T03:13:43.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fba6403d560 0x7fba64039b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fba6403d560 0x7fba64039b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53170/0 (socket says 172.21.2.111:53170) 2021-04-03T03:13:43.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 -- 172.21.2.111:0/2887603462 learned_addr learned my addr 172.21.2.111:0/2887603462 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:43.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 -- 172.21.2.111:0/2887603462 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba640dbba0 msgr2=0x7fba640d6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba640dbba0 0x7fba640d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 -- 172.21.2.111:0/2887603462 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fba64041060 msgr2=0x7fba640d5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:43.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fba64041060 0x7fba640d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:43.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 -- 172.21.2.111:0/2887603462 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640e7380 con 0x7fba6403d560 2021-04-03T03:13:43.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6a59c700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fba6403d560 0x7fba64039b30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fba640087d0 tx=0x7fba5c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba637fe700 1 -- 172.21.2.111:0/2887603462 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5c007030 con 0x7fba6403d560 2021-04-03T03:13:43.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba637fe700 1 -- 172.21.2.111:0/2887603462 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c007190 con 0x7fba6403d560 2021-04-03T03:13:43.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba640d6950 con 0x7fba6403d560 2021-04-03T03:13:43.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba640eb0f0 con 0x7fba6403d560 2021-04-03T03:13:43.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba637fe700 1 -- 172.21.2.111:0/2887603462 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5c0064f0 con 0x7fba6403d560 2021-04-03T03:13:43.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba640eb5b0 con 0x7fba6403d560 2021-04-03T03:13:43.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba637fe700 1 -- 172.21.2.111:0/2887603462 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fba5c01a060 con 0x7fba6403d560 2021-04-03T03:13:43.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba637fe700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fba50057000 0x7fba500594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:43.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba637fe700 1 -- 172.21.2.111:0/2887603462 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fba5c06fa40 con 0x7fba6403d560 2021-04-03T03:13:43.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba69d9b700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fba50057000 0x7fba500594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:43.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.931+0000 7fba69d9b700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fba50057000 0x7fba500594c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fba5400c000 tx=0x7fba54006e90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:43.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:43.935+0000 7fba637fe700 1 -- 172.21.2.111:0/2887603462 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fba5c044d20 con 0x7fba6403d560 2021-04-03T03:13:44.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.027+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fba640e7510 con 0x7fba6403d560 2021-04-03T03:13:44.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.027+0000 7fba637fe700 1 -- 172.21.2.111:0/2887603462 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v64) v1 ==== 110+0+10 (secure 0 0 0) 0x7fba5c018020 con 0x7fba6403d560 2021-04-03T03:13:44.045 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:13:44.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fba50057000 msgr2=0x7fba500594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fba50057000 0x7fba500594c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fba5400c000 tx=0x7fba54006e90).stop 2021-04-03T03:13:44.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fba6403d560 msgr2=0x7fba64039b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fba6403d560 0x7fba64039b30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fba640087d0 tx=0x7fba5c007be0).stop 2021-04-03T03:13:44.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 shutdown_connections 2021-04-03T03:13:44.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fba640dbba0 0x7fba640d6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fba6403d560 0x7fba64039b30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fba50057000 0x7fba500594c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 --2- 172.21.2.111:0/2887603462 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fba64041060 0x7fba640d5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 >> 172.21.2.111:0/2887603462 conn(0x7fba640462b0 msgr2=0x7fba64041550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 shutdown_connections 2021-04-03T03:13:44.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.031+0000 7fba6b59e700 1 -- 172.21.2.111:0/2887603462 wait complete. 2021-04-03T03:13:44.053 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:13:44.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 Processor -- start 2021-04-03T03:13:44.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- start start 2021-04-03T03:13:44.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fda00048f00 0x7fda0003b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fda00043440 0x7fda0003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda000492e0 con 0x7fda00043b40 2021-04-03T03:13:44.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda00007e50 con 0x7fda00043440 2021-04-03T03:13:44.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda00009390 con 0x7fda00048f00 2021-04-03T03:13:44.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fda00043440 0x7fda0003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fda00043440 0x7fda0003bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53180/0 (socket says 172.21.2.111:53180) 2021-04-03T03:13:44.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 -- 172.21.2.111:0/3230577133 learned_addr learned my addr 172.21.2.111:0/3230577133 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:44.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 -- 172.21.2.111:0/3230577133 >> v2:172.21.2.115:3300/0 conn(0x7fda00048f00 msgr2=0x7fda0003b7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:13:44.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 --2- 172.21.2.111:0/3230577133 >> v2:172.21.2.115:3300/0 conn(0x7fda00048f00 0x7fda0003b7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 -- 172.21.2.111:0/3230577133 >> v1:172.21.2.115:6789/0 conn(0x7fda00043b40 legacy=0x7fda000d7590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:13:44.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 -- 172.21.2.111:0/3230577133 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda00044540 con 0x7fda00043440 2021-04-03T03:13:44.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0cd1b700 1 --2- 172.21.2.111:0/3230577133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00043440 0x7fda0003bc90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fda00009520 tx=0x7fd9fc006e90).ready entity=mon.0 client_cookie=ba7a61ed877fa39f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fd9fbfff700 1 -- 172.21.2.111:0/3230577133 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9fc0098f0 con 0x7fda00043440 2021-04-03T03:13:44.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fd9fbfff700 1 -- 172.21.2.111:0/3230577133 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc009a50 con 0x7fda00043440 2021-04-03T03:13:44.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fd9fbfff700 1 -- 172.21.2.111:0/3230577133 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9fc009df0 con 0x7fda00043440 2021-04-03T03:13:44.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- 172.21.2.111:0/3230577133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00043440 msgr2=0x7fda0003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3230577133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00043440 0x7fda0003bc90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fda00009520 tx=0x7fd9fc006e90).stop 2021-04-03T03:13:44.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- 172.21.2.111:0/3230577133 shutdown_connections 2021-04-03T03:13:44.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3230577133 >> v2:172.21.2.115:3300/0 conn(0x7fda00048f00 0x7fda0003b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3230577133 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00043440 0x7fda0003bc90 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- 172.21.2.111:0/3230577133 >> 172.21.2.111:0/3230577133 conn(0x7fda0000f310 msgr2=0x7fda000484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- 172.21.2.111:0/3230577133 shutdown_connections 2021-04-03T03:13:44.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- 172.21.2.111:0/3230577133 wait complete. 2021-04-03T03:13:44.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 Processor -- start 2021-04-03T03:13:44.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- start start 2021-04-03T03:13:44.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda00043440 0x7fda0003f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda00043b40 0x7fda0003f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00048f00 0x7fda00041d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0003df90 con 0x7fda00048f00 2021-04-03T03:13:44.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000eb6a0 con 0x7fda00043b40 2021-04-03T03:13:44.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0e51e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0003b8b0 con 0x7fda00043440 2021-04-03T03:13:44.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00048f00 0x7fda00041d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00048f00 0x7fda00041d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53186/0 (socket says 172.21.2.111:53186) 2021-04-03T03:13:44.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 -- 172.21.2.111:0/3859101706 learned_addr learned my addr 172.21.2.111:0/3859101706 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:44.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 -- 172.21.2.111:0/3859101706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda00043440 msgr2=0x7fda0003f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda00043440 0x7fda0003f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 -- 172.21.2.111:0/3859101706 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda00043b40 msgr2=0x7fda0003f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda00043b40 0x7fda0003f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.095+0000 7fda0dd1d700 1 -- 172.21.2.111:0/3859101706 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0003bb50 con 0x7fda00048f00 2021-04-03T03:13:44.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fda0dd1d700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00048f00 0x7fda00041d10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fda0806b5d0 tx=0x7fda0805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fd9fa7fc700 1 -- 172.21.2.111:0/3859101706 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0805df60 con 0x7fda00048f00 2021-04-03T03:13:44.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fd9fa7fc700 1 -- 172.21.2.111:0/3859101706 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda080699f0 con 0x7fda00048f00 2021-04-03T03:13:44.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda000db250 con 0x7fda00048f00 2021-04-03T03:13:44.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fd9fa7fc700 1 -- 172.21.2.111:0/3859101706 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0805f4b0 con 0x7fda00048f00 2021-04-03T03:13:44.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda000eb3d0 con 0x7fda00048f00 2021-04-03T03:13:44.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda00004db0 con 0x7fda00048f00 2021-04-03T03:13:44.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fd9fa7fc700 1 -- 172.21.2.111:0/3859101706 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fda0805f610 con 0x7fda00048f00 2021-04-03T03:13:44.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fd9fa7fc700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd9e8056f20 0x7fd9e80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fd9fa7fc700 1 -- 172.21.2.111:0/3859101706 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fda080df1a0 con 0x7fda00048f00 2021-04-03T03:13:44.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fda0d51c700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd9e8056f20 0x7fd9e80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fda0d51c700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd9e8056f20 0x7fd9e80593e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fda04002eb0 tx=0x7fda040068f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.099+0000 7fd9fa7fc700 1 -- 172.21.2.111:0/3859101706 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fda080b4590 con 0x7fda00048f00 2021-04-03T03:13:44.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fda0000a600 con 0x7fda00048f00 2021-04-03T03:13:44.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fd9fa7fc700 1 -- 172.21.2.111:0/3859101706 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7fda080b7bb0 con 0x7fda00048f00 2021-04-03T03:13:44.211 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:13:44.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd9e8056f20 msgr2=0x7fd9e80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd9e8056f20 0x7fd9e80593e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fda04002eb0 tx=0x7fda040068f0).stop 2021-04-03T03:13:44.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00048f00 msgr2=0x7fda00041d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00048f00 0x7fda00041d10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fda0806b5d0 tx=0x7fda0805ad70).stop 2021-04-03T03:13:44.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 shutdown_connections 2021-04-03T03:13:44.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda00048f00 0x7fda00041d10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda00043440 0x7fda0003f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd9e8056f20 0x7fd9e80593e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 --2- 172.21.2.111:0/3859101706 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda00043b40 0x7fda0003f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 >> 172.21.2.111:0/3859101706 conn(0x7fda0000f310 msgr2=0x7fda000d7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 shutdown_connections 2021-04-03T03:13:44.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.195+0000 7fda0e51e700 1 -- 172.21.2.111:0/3859101706 wait complete. 2021-04-03T03:13:44.218 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:13:44.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1927fff700 1 Processor -- start 2021-04-03T03:13:44.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1927fff700 1 -- start start 2021-04-03T03:13:44.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1927fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f19280603f0 0x7f19281447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1927fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280607d0 con 0x7f19280b17b0 2021-04-03T03:13:44.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1927fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192814b090 con 0x7f19280add50 2021-04-03T03:13:44.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1927fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192814b330 con 0x7f19280603f0 2021-04-03T03:13:44.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f19277fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f19280b17b0 0x7f1928147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:53982/0 (socket says 172.21.2.111:53982) 2021-04-03T03:13:44.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f19277fe700 1 -- 172.21.2.111:0/210781349 learned_addr learned my addr 172.21.2.111:0/210781349 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:13:44.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910356952 0 0) 0x7f19280607d0 con 0x7f19280b17b0 2021-04-03T03:13:44.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1908003240 con 0x7f19280b17b0 2021-04-03T03:13:44.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.259+0000 7f1926ffd700 1 --2- 172.21.2.111:0/210781349 >> v2:172.21.2.115:3300/0 conn(0x7f19280603f0 0x7f19281447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2682265180 0 0) 0x7f1908003240 con 0x7f19280b17b0 2021-04-03T03:13:44.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 >> v2:172.21.2.115:3300/0 conn(0x7f19280603f0 msgr2=0x7f19281447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:44.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 --2- 172.21.2.111:0/210781349 >> v2:172.21.2.115:3300/0 conn(0x7f19280603f0 0x7f19281447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 >> v1:172.21.2.115:6789/0 conn(0x7f19280add50 legacy=0x7f1928147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192814b5d0 con 0x7f19280b17b0 2021-04-03T03:13:44.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f1920001a50 con 0x7f19280b17b0 2021-04-03T03:13:44.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 >> v1:172.21.2.111:6789/0 conn(0x7f19280b17b0 legacy=0x7f1928147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1908003800 0x7f1908003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19080044e0 0x7f1908004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1908004e50 0x7f190800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192814b330 con 0x7f1908003800 2021-04-03T03:13:44.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908005360 con 0x7f19080044e0 2021-04-03T03:13:44.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190800bc50 con 0x7f1908004e50 2021-04-03T03:13:44.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1908003800 0x7f1908003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f19277fe700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1908004e50 0x7f190800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f19267fc700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19080044e0 0x7f1908004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 -- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1908004e50 msgr2=0x7f190800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:44.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1908004e50 0x7f190800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 -- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19080044e0 msgr2=0x7f1908004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:44.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19080044e0 0x7f1908004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 -- 172.21.2.111:0/210781349 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1908003240 con 0x7f1908003800 2021-04-03T03:13:44.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f19277fe700 1 -- 172.21.2.111:0/210781349 reap_dead start 2021-04-03T03:13:44.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1908003800 0x7f1908003c20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f191800a000 tx=0x7f1918008040).ready entity=mon.0 client_cookie=ea1815bc7e19803f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191800da60 con 0x7f1908003800 2021-04-03T03:13:44.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191800dbc0 con 0x7f1908003800 2021-04-03T03:13:44.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1925ffb700 1 -- 172.21.2.111:0/210781349 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1918013480 con 0x7f1908003800 2021-04-03T03:13:44.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- 172.21.2.111:0/210781349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1908003800 msgr2=0x7f1908003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1908003800 0x7f1908003c20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f191800a000 tx=0x7f1918008040).stop 2021-04-03T03:13:44.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- 172.21.2.111:0/210781349 shutdown_connections 2021-04-03T03:13:44.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 --2- 172.21.2.111:0/210781349 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1908003800 0x7f1908003c20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- 172.21.2.111:0/210781349 >> 172.21.2.111:0/210781349 conn(0x7f19280b61c0 msgr2=0x7f192805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- 172.21.2.111:0/210781349 shutdown_connections 2021-04-03T03:13:44.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- 172.21.2.111:0/210781349 wait complete. 2021-04-03T03:13:44.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 Processor -- start 2021-04-03T03:13:44.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- start start 2021-04-03T03:13:44.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1908003800 0x7f1928065f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19280663f0 0x7f19281567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1928156c90 0x7f1928159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19281576c0 con 0x7f1928156c90 2021-04-03T03:13:44.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1928062b10 con 0x7f1908003800 2021-04-03T03:13:44.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19280aae00 con 0x7f19280663f0 2021-04-03T03:13:44.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1908003800 0x7f1928065f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1908003800 0x7f1928065f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:42910/0 (socket says 172.21.2.111:42910) 2021-04-03T03:13:44.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 -- 172.21.2.111:0/3366815249 learned_addr learned my addr 172.21.2.111:0/3366815249 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:44.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f19277fe700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1928156c90 0x7f1928159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f19267fc700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19280663f0 0x7f19281567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 -- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19280663f0 msgr2=0x7f19281567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19280663f0 0x7f19281567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 -- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1928156c90 msgr2=0x7f1928159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1928156c90 0x7f1928159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 -- 172.21.2.111:0/3366815249 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1928066870 con 0x7f1908003800 2021-04-03T03:13:44.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1926ffd700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1908003800 0x7f1928065f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f191800a000 tx=0x7f1918008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f191ffff700 1 -- 172.21.2.111:0/3366815249 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1918013da0 con 0x7f1908003800 2021-04-03T03:13:44.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f191ffff700 1 -- 172.21.2.111:0/3366815249 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918006760 con 0x7f1908003800 2021-04-03T03:13:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f191ffff700 1 -- 172.21.2.111:0/3366815249 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191801d600 con 0x7f1908003800 2021-04-03T03:13:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1928157140 con 0x7f1908003800 2021-04-03T03:13:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.263+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192815a260 con 0x7f1908003800 2021-04-03T03:13:44.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.267+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1928057660 con 0x7f1908003800 2021-04-03T03:13:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.267+0000 7f191ffff700 1 -- 172.21.2.111:0/3366815249 <== mon.1 v2:172.21.2.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f191801d760 con 0x7f1908003800 2021-04-03T03:13:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.267+0000 7f191ffff700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f190c056fb0 0x7f190c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.267+0000 7f191ffff700 1 -- 172.21.2.111:0/3366815249 <== mon.1 v2:172.21.2.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1918071fe0 con 0x7f1908003800 2021-04-03T03:13:44.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.267+0000 7f19267fc700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f190c056fb0 0x7f190c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.267+0000 7f19267fc700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f190c056fb0 0x7f190c059470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f191000d000 tx=0x7f1910009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.271+0000 7f191ffff700 1 -- 172.21.2.111:0/3366815249 <== mon.1 v2:172.21.2.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f19180472c0 con 0x7f1908003800 2021-04-03T03:13:44.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1928148850 con 0x7f1908003800 2021-04-03T03:13:44.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f191ffff700 1 -- 172.21.2.111:0/3366815249 <== mon.1 v2:172.21.2.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+10 (secure 0 0 0) 0x7f191804a8e0 con 0x7f1908003800 2021-04-03T03:13:44.384 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:13:44.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f190c056fb0 msgr2=0x7f190c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f190c056fb0 0x7f190c059470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f191000d000 tx=0x7f1910009040).stop 2021-04-03T03:13:44.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1908003800 msgr2=0x7f1928065f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1908003800 0x7f1928065f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f191800a000 tx=0x7f1918008040).stop 2021-04-03T03:13:44.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 shutdown_connections 2021-04-03T03:13:44.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1928156c90 0x7f1928159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f190c056fb0 0x7f190c059470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1908003800 0x7f1928065f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 --2- 172.21.2.111:0/3366815249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19280663f0 0x7f19281567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 >> 172.21.2.111:0/3366815249 conn(0x7f19280b61c0 msgr2=0x7f1928145b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 shutdown_connections 2021-04-03T03:13:44.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.367+0000 7f1927fff700 1 -- 172.21.2.111:0/3366815249 wait complete. 2021-04-03T03:13:44.392 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:13:44.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 Processor -- start 2021-04-03T03:13:44.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- start start 2021-04-03T03:13:44.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07540603f0 0x7f07541447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 0x7f0754144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07540607d0 con 0x7f07540add50 2021-04-03T03:13:44.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07541451c0 con 0x7f07540603f0 2021-04-03T03:13:44.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075414b090 con 0x7f07540b17b0 2021-04-03T03:13:44.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07540603f0 0x7f07541447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f07540603f0 0x7f07541447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53210/0 (socket says 172.21.2.111:53210) 2021-04-03T03:13:44.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 -- 172.21.2.111:0/2463964299 learned_addr learned my addr 172.21.2.111:0/2463964299 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:44.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0758866700 1 --1- 172.21.2.111:0/2463964299 >> v1:172.21.2.111:6789/0 conn(0x7f07540add50 0x7f0754147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54006/0 (socket says 172.21.2.111:54006) 2021-04-03T03:13:44.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 -- 172.21.2.111:0/2463964299 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 msgr2=0x7f0754144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:13:44.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 --2- 172.21.2.111:0/2463964299 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 0x7f0754144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 -- 172.21.2.111:0/2463964299 >> v1:172.21.2.111:6789/0 conn(0x7f07540add50 legacy=0x7f0754147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 -- 172.21.2.111:0/2463964299 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075414b330 con 0x7f07540603f0 2021-04-03T03:13:44.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0759067700 1 --2- 172.21.2.111:0/2463964299 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540603f0 0x7f07541447e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f0750002f30 tx=0x7f0750006e90).ready entity=mon.0 client_cookie=1a2c38243183448d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0747fff700 1 -- 172.21.2.111:0/2463964299 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0750009970 con 0x7f07540603f0 2021-04-03T03:13:44.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0747fff700 1 -- 172.21.2.111:0/2463964299 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0750009ad0 con 0x7f07540603f0 2021-04-03T03:13:44.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0747fff700 1 -- 172.21.2.111:0/2463964299 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0750009e30 con 0x7f07540603f0 2021-04-03T03:13:44.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- 172.21.2.111:0/2463964299 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540603f0 msgr2=0x7f07541447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- 172.21.2.111:0/2463964299 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540603f0 0x7f07541447e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f0750002f30 tx=0x7f0750006e90).stop 2021-04-03T03:13:44.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- 172.21.2.111:0/2463964299 shutdown_connections 2021-04-03T03:13:44.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- 172.21.2.111:0/2463964299 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 0x7f0754144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- 172.21.2.111:0/2463964299 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540603f0 0x7f07541447e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- 172.21.2.111:0/2463964299 >> 172.21.2.111:0/2463964299 conn(0x7f07540b61c0 msgr2=0x7f075405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- 172.21.2.111:0/2463964299 shutdown_connections 2021-04-03T03:13:44.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- 172.21.2.111:0/2463964299 wait complete. 2021-04-03T03:13:44.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 Processor -- start 2021-04-03T03:13:44.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- start start 2021-04-03T03:13:44.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07540603f0 0x7f075415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540add50 0x7f0754145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 0x7f07540a8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07540a7930 con 0x7f07540add50 2021-04-03T03:13:44.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0754157cd0 con 0x7f07540603f0 2021-04-03T03:13:44.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f075a069700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075415abb0 con 0x7f07540b17b0 2021-04-03T03:13:44.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0758866700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540add50 0x7f0754145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0758866700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540add50 0x7f0754145310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53218/0 (socket says 172.21.2.111:53218) 2021-04-03T03:13:44.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.435+0000 7f0758866700 1 -- 172.21.2.111:0/3101996486 learned_addr learned my addr 172.21.2.111:0/3101996486 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:44.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0758866700 1 -- 172.21.2.111:0/3101996486 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 msgr2=0x7f07540a8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0758866700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 0x7f07540a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0758866700 1 -- 172.21.2.111:0/3101996486 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07540603f0 msgr2=0x7f075415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0758866700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07540603f0 0x7f075415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0758866700 1 -- 172.21.2.111:0/3101996486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07540a7b90 con 0x7f07540add50 2021-04-03T03:13:44.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0758866700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540add50 0x7f0754145310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f07540b5600 tx=0x7f0748007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f07467fc700 1 -- 172.21.2.111:0/3101996486 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07480072e0 con 0x7f07540add50 2021-04-03T03:13:44.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f07467fc700 1 -- 172.21.2.111:0/3101996486 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748005530 con 0x7f07540add50 2021-04-03T03:13:44.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075415a8a0 con 0x7f07540add50 2021-04-03T03:13:44.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0754147680 con 0x7f07540add50 2021-04-03T03:13:44.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f07467fc700 1 -- 172.21.2.111:0/3101996486 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0748005890 con 0x7f07540add50 2021-04-03T03:13:44.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07540a7d20 con 0x7f07540add50 2021-04-03T03:13:44.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f07467fc700 1 -- 172.21.2.111:0/3101996486 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0748010070 con 0x7f07540add50 2021-04-03T03:13:44.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f07467fc700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0734056ef0 0x7f07340593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f07467fc700 1 -- 172.21.2.111:0/3101996486 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f074806f960 con 0x7f07540add50 2021-04-03T03:13:44.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0759067700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0734056ef0 0x7f07340593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f0759067700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0734056ef0 0x7f07340593b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f0750002f30 tx=0x7f0750015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.439+0000 7f07467fc700 1 -- 172.21.2.111:0/3101996486 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0748044c40 con 0x7f07540add50 2021-04-03T03:13:44.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0754057660 con 0x7f07540add50 2021-04-03T03:13:44.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f07467fc700 1 -- 172.21.2.111:0/3101996486 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0748048260 con 0x7f07540add50 2021-04-03T03:13:44.553 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:13:44.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0734056ef0 msgr2=0x7f07340593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0734056ef0 0x7f07340593b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f0750002f30 tx=0x7f0750015000).stop 2021-04-03T03:13:44.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540add50 msgr2=0x7f0754145310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540add50 0x7f0754145310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f07540b5600 tx=0x7f0748007bc0).stop 2021-04-03T03:13:44.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 shutdown_connections 2021-04-03T03:13:44.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07540b17b0 0x7f07540a8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0734056ef0 0x7f07340593b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07540603f0 0x7f075415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 --2- 172.21.2.111:0/3101996486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07540add50 0x7f0754145310 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 >> 172.21.2.111:0/3101996486 conn(0x7f07540b61c0 msgr2=0x7f075405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 shutdown_connections 2021-04-03T03:13:44.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.535+0000 7f075a069700 1 -- 172.21.2.111:0/3101996486 wait complete. 2021-04-03T03:13:44.560 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:13:44.561 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:13:44.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 Processor -- start 2021-04-03T03:13:44.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- start start 2021-04-03T03:13:44.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8bec14b700 0x7f8bec0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0b0c90 0x7f8bec147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bec0607d0 con 0x7f8bec0603f0 2021-04-03T03:13:44.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bec0af400 con 0x7f8bec14b700 2021-04-03T03:13:44.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0af6a0 con 0x7f8bec0b0c90 2021-04-03T03:13:44.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8bec14b700 0x7f8bec0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8bec14b700 0x7f8bec0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53228/0 (socket says 172.21.2.111:53228) 2021-04-03T03:13:44.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 -- 172.21.2.111:0/3800138312 learned_addr learned my addr 172.21.2.111:0/3800138312 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:44.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 -- 172.21.2.111:0/3800138312 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0b0c90 msgr2=0x7f8bec147680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:13:44.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 --2- 172.21.2.111:0/3800138312 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0b0c90 0x7f8bec147680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 -- 172.21.2.111:0/3800138312 >> v1:172.21.2.115:6789/0 conn(0x7f8bec0603f0 legacy=0x7f8bec0b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:13:44.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 -- 172.21.2.111:0/3800138312 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec0afb40 con 0x7f8bec14b700 2021-04-03T03:13:44.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf0eba700 1 --2- 172.21.2.111:0/3800138312 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec0b07a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f8bec06a3b0 tx=0x7f8be0006e90).ready entity=mon.0 client_cookie=a895c4e45f0e91ab server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bdffff700 1 -- 172.21.2.111:0/3800138312 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be00098e0 con 0x7f8bec14b700 2021-04-03T03:13:44.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bdffff700 1 -- 172.21.2.111:0/3800138312 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0009a40 con 0x7f8bec14b700 2021-04-03T03:13:44.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bdffff700 1 -- 172.21.2.111:0/3800138312 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be0009de0 con 0x7f8bec14b700 2021-04-03T03:13:44.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- 172.21.2.111:0/3800138312 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 msgr2=0x7f8bec0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/3800138312 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec0b07a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f8bec06a3b0 tx=0x7f8be0006e90).stop 2021-04-03T03:13:44.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- 172.21.2.111:0/3800138312 shutdown_connections 2021-04-03T03:13:44.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/3800138312 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0b0c90 0x7f8bec147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/3800138312 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec0b07a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- 172.21.2.111:0/3800138312 >> 172.21.2.111:0/3800138312 conn(0x7f8bec0b61c0 msgr2=0x7f8bec05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- 172.21.2.111:0/3800138312 shutdown_connections 2021-04-03T03:13:44.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.599+0000 7f8bf26bd700 1 -- 172.21.2.111:0/3800138312 wait complete. 2021-04-03T03:13:44.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 Processor -- start 2021-04-03T03:13:44.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 -- start start 2021-04-03T03:13:44.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0603f0 0x7f8bec063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bec0b0c90 0x7f8bec063cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0a97c0 con 0x7f8bec14b700 2021-04-03T03:13:44.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0abe40 con 0x7f8bec0b0c90 2021-04-03T03:13:44.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec157110 con 0x7f8bec0603f0 2021-04-03T03:13:44.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec067520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53234/0 (socket says 172.21.2.111:53234) 2021-04-03T03:13:44.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 -- 172.21.2.111:0/719863255 learned_addr learned my addr 172.21.2.111:0/719863255 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:13:44.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf0eba700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bec0b0c90 0x7f8bec063cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 -- 172.21.2.111:0/719863255 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0603f0 msgr2=0x7f8bec063800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0603f0 0x7f8bec063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 -- 172.21.2.111:0/719863255 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bec0b0c90 msgr2=0x7f8bec063cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bec0b0c90 0x7f8bec063cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 -- 172.21.2.111:0/719863255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec1573b0 con 0x7f8bec14b700 2021-04-03T03:13:44.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf1ebc700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec067520 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f8be40032f0 tx=0x7f8be40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bde7fc700 1 -- 172.21.2.111:0/719863255 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be400c9c0 con 0x7f8bec14b700 2021-04-03T03:13:44.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bde7fc700 1 -- 172.21.2.111:0/719863255 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be400cb20 con 0x7f8bec14b700 2021-04-03T03:13:44.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bde7fc700 1 -- 172.21.2.111:0/719863255 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be4015420 con 0x7f8bec14b700 2021-04-03T03:13:44.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec0a9510 con 0x7f8bec14b700 2021-04-03T03:13:44.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec0a88b0 con 0x7f8bec14b700 2021-04-03T03:13:44.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec156f60 con 0x7f8bec14b700 2021-04-03T03:13:44.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bde7fc700 1 -- 172.21.2.111:0/719863255 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8be401c070 con 0x7f8bec14b700 2021-04-03T03:13:44.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bde7fc700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bcc056ea0 0x7f8bcc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:13:44.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bde7fc700 1 -- 172.21.2.111:0/719863255 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8be406ff40 con 0x7f8bec14b700 2021-04-03T03:13:44.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf16bb700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bcc056ea0 0x7f8bcc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:13:44.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.603+0000 7f8bf16bb700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bcc056ea0 0x7f8bcc059360 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8be8002f70 tx=0x7f8be8006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:13:44.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.607+0000 7f8bde7fc700 1 -- 172.21.2.111:0/719863255 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8be40451a0 con 0x7f8bec14b700 2021-04-03T03:13:44.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bec157540 con 0x7f8bec14b700 2021-04-03T03:13:44.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bde7fc700 1 -- 172.21.2.111:0/719863255 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f8be40487c0 con 0x7f8bec14b700 2021-04-03T03:13:44.746 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:13:44.747 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":25,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:13:42.208611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/167825619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":167825619},{"type":"v1","addr":"172.21.2.111:6835","nonce":167825619}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:13:44.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bcc056ea0 msgr2=0x7f8bcc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bcc056ea0 0x7f8bcc059360 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8be8002f70 tx=0x7f8be8006990).stop 2021-04-03T03:13:44.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 msgr2=0x7f8bec067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:13:44.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec067520 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f8be40032f0 tx=0x7f8be40091a0).stop 2021-04-03T03:13:44.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 shutdown_connections 2021-04-03T03:13:44.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8bec14b700 0x7f8bec067520 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8bec0603f0 0x7f8bec063800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:13:44.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bcc056ea0 0x7f8bcc059360 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 --2- 172.21.2.111:0/719863255 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8bec0b0c90 0x7f8bec063cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:13:44.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 >> 172.21.2.111:0/719863255 conn(0x7f8bec0b61c0 msgr2=0x7f8bec147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:13:44.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 shutdown_connections 2021-04-03T03:13:44.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:13:44.731+0000 7f8bf26bd700 1 -- 172.21.2.111:0/719863255 wait complete. 2021-04-03T03:13:44.750 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 25 2021-04-03T03:13:44.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:13:41.205623+0000', 'modified': '2021-04-03T03:13:42.208611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5934}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5934': {'gid': 5934, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/167825619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 167825619}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 167825619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:13:44.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:13:44.957 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:13:45.044 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":5934,"want_state":"up:active","state":"up:active","rank_uptime":3.8194507870000001,"mdsmap_epoch":25,"osdmap_epoch":64,"osdmap_epoch_barrier":63,"uptime":7.9724751830000002} 2021-04-03T03:13:45.045 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 5934, "mdsmap_epoch": 25, "osdmap_epoch": 64, "osdmap_epoch_barrier": 63, "rank_uptime": 3.819450787, "state": "up:active", "uptime": 7.972475183, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:13:45.045 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:13:45.046 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:13:45.046 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:13:45.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.047 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:45.060 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:13:45.114 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:13:45.114 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:13:45.163 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:13:45.163 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:13:45.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.165 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:13:45.215 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:13:45.216 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:45.261 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:13:45.268 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:13:45.269 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:13:45.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:13:45.291+0000 7f2c6b9340c0 -1 init, newargv = 0x5595531104d0 newargc=9 2021-04-03T03:13:45.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15366]: starting ceph client 2021-04-03T03:13:45.307 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:13:45.307 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:13:45.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.309 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:13:45.314 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15366]: starting fuse 2021-04-03T03:13:45.317 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:13:45.317 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:13:45.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.318 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:13:45.319 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:13:45.319 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:13:45.320 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:13:45.320 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:13:45.320 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:13:45.321 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:13:45.321 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:13:45.322 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:13:45.322 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:13:45.322 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:13:45.323 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:13:45.323 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:13:45.324 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:13:45.324 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:13:45.324 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:13:45.325 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:13:45.326 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:13:45.326 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:13:45.327 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:13:45.327 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:13:45.327 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:13:45.328 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:13:45.411 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15366.asok 2021-04-03T03:13:45.415 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15366.asok 2021-04-03T03:13:45.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.416 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15366.asok status 2021-04-03T03:13:45.486 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:13:45.487 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:13:45.487 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:13:45.487 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:13:45.488 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:13:45.488 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:13:45.488 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:13:45.488 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15366", 2021-04-03T03:13:45.488 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:13:45.489 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:13:45.489 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:13:45.489 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:13:45.489 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4861, 2021-04-03T03:13:45.490 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:13:45.490 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:13:45.490 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:13:45.490 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4861 2021-04-03T03:13:45.490 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:13:45.491 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:13:45.491 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:13:45.491 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:13:45.491 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 4132889834 2021-04-03T03:13:45.492 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:13:45.492 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:13:45.492 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:13:45.492 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:13:45.492 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:13:45.493 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 4132889834 2021-04-03T03:13:45.493 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:13:45.493 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4861 172.21.2.126:0/4132889834", 2021-04-03T03:13:45.493 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/4132889834", 2021-04-03T03:13:45.494 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:13:45.494 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 25, 2021-04-03T03:13:45.494 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 64, 2021-04-03T03:13:45.494 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:13:45.494 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:13:45.495 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:13:45.495 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:13:45.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.497 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:45.497 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:45.502 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:13:45.503 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:13:45.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.503 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:45.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:13:45.554 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:13:45.555 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:13:45.602 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:13:45.603 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:13:45.604 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:13:45.604 DEBUG:teuthology.orchestra.run.gibba026:> import sys 2021-04-03T03:13:45.605 DEBUG:teuthology.orchestra.run.gibba026:> import time 2021-04-03T03:13:45.605 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:13:45.605 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:13:45.606 DEBUG:teuthology.orchestra.run.gibba026:> n = 1000 2021-04-03T03:13:45.606 DEBUG:teuthology.orchestra.run.gibba026:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-04-03T03:13:45.606 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:13:45.607 DEBUG:teuthology.orchestra.run.gibba026:> if not os.path.exists(os.path.dirname(abs_path)): 2021-04-03T03:13:45.607 DEBUG:teuthology.orchestra.run.gibba026:> os.makedirs(os.path.dirname(abs_path)) 2021-04-03T03:13:45.608 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:13:45.608 DEBUG:teuthology.orchestra.run.gibba026:> for i in range(0, n): 2021-04-03T03:13:45.608 DEBUG:teuthology.orchestra.run.gibba026:> fname = "{0}_{1}".format(abs_path, i) 2021-04-03T03:13:45.609 DEBUG:teuthology.orchestra.run.gibba026:> with open(fname, '"'"'w'"'"') as f: 2021-04-03T03:13:45.609 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content'"'"') 2021-04-03T03:13:45.610 DEBUG:teuthology.orchestra.run.gibba026:> if True: 2021-04-03T03:13:45.610 DEBUG:teuthology.orchestra.run.gibba026:> f.flush() 2021-04-03T03:13:45.610 DEBUG:teuthology.orchestra.run.gibba026:> os.fsync(f.fileno()) 2021-04-03T03:13:45.611 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:13:58.966 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:58.969 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:58.972 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:58.975 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:13:58.978 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:01.919 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:01.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 Processor -- start 2021-04-03T03:14:01.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 -- start start 2021-04-03T03:14:01.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52f40456f0 0x7f52f40dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:01.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f52f40d7660 0x7f52f40d7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:01.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f40db880 con 0x7f52f404b130 2021-04-03T03:14:01.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f404b510 con 0x7f52f40456f0 2021-04-03T03:14:01.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f4009890 con 0x7f52f40d7660 2021-04-03T03:14:01.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301235700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52f40456f0 0x7f52f40dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:01.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301a36700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f52f404b130 0x7f52f40dbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54030/0 (socket says 172.21.2.111:54030) 2021-04-03T03:14:01.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301a36700 1 -- 172.21.2.111:0/1602431981 learned_addr learned my addr 172.21.2.111:0/1602431981 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:01.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301235700 1 -- 172.21.2.111:0/1602431981 >> v2:172.21.2.115:3300/0 conn(0x7f52f40d7660 msgr2=0x7f52f40d7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:01.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301235700 1 --2- 172.21.2.111:0/1602431981 >> v2:172.21.2.115:3300/0 conn(0x7f52f40d7660 0x7f52f40d7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:01.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301235700 1 -- 172.21.2.111:0/1602431981 >> v1:172.21.2.111:6789/0 conn(0x7f52f404b130 legacy=0x7f52f40dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:01.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301235700 1 -- 172.21.2.111:0/1602431981 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009b30 con 0x7f52f40456f0 2021-04-03T03:14:01.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5301235700 1 --2- 172.21.2.111:0/1602431981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40dc3c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f52ec002f30 tx=0x7f52ec006dd0).ready entity=mon.0 client_cookie=cf632533aeb5809a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:01.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5300a34700 1 -- 172.21.2.111:0/1602431981 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec0097f0 con 0x7f52f40456f0 2021-04-03T03:14:01.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5300a34700 1 -- 172.21.2.111:0/1602431981 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec009950 con 0x7f52f40456f0 2021-04-03T03:14:01.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5300a34700 1 -- 172.21.2.111:0/1602431981 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ec009cb0 con 0x7f52f40456f0 2021-04-03T03:14:01.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 -- 172.21.2.111:0/1602431981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 msgr2=0x7f52f40dc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:01.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.963+0000 7f5302a38700 1 --2- 172.21.2.111:0/1602431981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40dc3c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f52ec002f30 tx=0x7f52ec006dd0).stop 2021-04-03T03:14:01.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- 172.21.2.111:0/1602431981 shutdown_connections 2021-04-03T03:14:01.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 --2- 172.21.2.111:0/1602431981 >> v2:172.21.2.115:3300/0 conn(0x7f52f40d7660 0x7f52f40d7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:01.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 --2- 172.21.2.111:0/1602431981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40dc3c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:01.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- 172.21.2.111:0/1602431981 >> 172.21.2.111:0/1602431981 conn(0x7f52f4011530 msgr2=0x7f52f404a700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:01.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- 172.21.2.111:0/1602431981 shutdown_connections 2021-04-03T03:14:01.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- 172.21.2.111:0/1602431981 wait complete. 2021-04-03T03:14:01.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 Processor -- start 2021-04-03T03:14:01.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- start start 2021-04-03T03:14:01.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:01.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f404b130 0x7f52f40d9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:01.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f40d7660 0x7f52f40d9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:01.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400c450 con 0x7f52f40456f0 2021-04-03T03:14:01.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400ba90 con 0x7f52f40d7660 2021-04-03T03:14:01.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400bd30 con 0x7f52f404b130 2021-04-03T03:14:01.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:01.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40e7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53246/0 (socket says 172.21.2.111:53246) 2021-04-03T03:14:01.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 -- 172.21.2.111:0/709973697 learned_addr learned my addr 172.21.2.111:0/709973697 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:01.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302237700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f40d7660 0x7f52f40d9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:01.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 -- 172.21.2.111:0/709973697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f404b130 msgr2=0x7f52f40d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:01.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f404b130 0x7f52f40d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:01.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 -- 172.21.2.111:0/709973697 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f40d7660 msgr2=0x7f52f40d9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:01.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f40d7660 0x7f52f40d9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:01.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 -- 172.21.2.111:0/709973697 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f400bfd0 con 0x7f52f40456f0 2021-04-03T03:14:01.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301a36700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40e7fb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f52f400a580 tx=0x7f52f8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:01.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f52eaffd700 1 -- 172.21.2.111:0/709973697 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52f8007320 con 0x7f52f40456f0 2021-04-03T03:14:01.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f52eaffd700 1 -- 172.21.2.111:0/709973697 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f8007480 con 0x7f52f40456f0 2021-04-03T03:14:01.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f40e84a0 con 0x7f52f40456f0 2021-04-03T03:14:01.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f400cfa0 con 0x7f52f40456f0 2021-04-03T03:14:01.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f52eaffd700 1 -- 172.21.2.111:0/709973697 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52f80057e0 con 0x7f52f40456f0 2021-04-03T03:14:01.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f4004da0 con 0x7f52f40456f0 2021-04-03T03:14:01.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f52eaffd700 1 -- 172.21.2.111:0/709973697 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f52f8010030 con 0x7f52f40456f0 2021-04-03T03:14:01.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f52eaffd700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52dc056f20 0x7f52dc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:01.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f52eaffd700 1 -- 172.21.2.111:0/709973697 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f52f806ff80 con 0x7f52f40456f0 2021-04-03T03:14:01.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.967+0000 7f5301235700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52dc056f20 0x7f52dc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:01.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.971+0000 7f5301235700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52dc056f20 0x7f52dc0593e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f52ec002f30 tx=0x7f52ec015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:01.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:01.971+0000 7f52eaffd700 1 -- 172.21.2.111:0/709973697 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f52f8045260 con 0x7f52f40456f0 2021-04-03T03:14:02.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52f40e7d50 con 0x7f52f40456f0 2021-04-03T03:14:02.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f52eaffd700 1 -- 172.21.2.111:0/709973697 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f52f8048880 con 0x7f52f40456f0 2021-04-03T03:14:02.112 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:02.112 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":25,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:13:42.208611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/167825619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":167825619},{"type":"v1","addr":"172.21.2.111:6835","nonce":167825619}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:14:02.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52dc056f20 msgr2=0x7f52dc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52dc056f20 0x7f52dc0593e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f52ec002f30 tx=0x7f52ec015000).stop 2021-04-03T03:14:02.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 msgr2=0x7f52f40e7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40e7fb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f52f400a580 tx=0x7f52f8007c20).stop 2021-04-03T03:14:02.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 shutdown_connections 2021-04-03T03:14:02.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52dc056f20 0x7f52dc0593e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52f40d7660 0x7f52f40d9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52f40456f0 0x7f52f40e7fb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 --2- 172.21.2.111:0/709973697 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52f404b130 0x7f52f40d9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 >> 172.21.2.111:0/709973697 conn(0x7f52f4011530 msgr2=0x7f52f404a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 shutdown_connections 2021-04-03T03:14:02.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.095+0000 7f5302a38700 1 -- 172.21.2.111:0/709973697 wait complete. 2021-04-03T03:14:02.117 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 25 2021-04-03T03:14:02.119 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2021-04-03T03:14:02.183 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-04-03T03:14:02.184 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-04-03T03:14:02.184 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:02.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 Processor -- start 2021-04-03T03:14:02.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- start start 2021-04-03T03:14:02.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fab5c005dc0 0x7fab5c042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c0dc6e0 con 0x7fab5c0dbb90 2021-04-03T03:14:02.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c042010 con 0x7fab5c006bc0 2021-04-03T03:14:02.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c0d9560 con 0x7fab5c005dc0 2021-04-03T03:14:02.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab63fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fab5c0dbb90 0x7fab5c0064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54044/0 (socket says 172.21.2.111:54044) 2021-04-03T03:14:02.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab63fff700 1 -- 172.21.2.111:0/3150058908 learned_addr learned my addr 172.21.2.111:0/3150058908 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:02.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab68d3a700 1 --2- 172.21.2.111:0/3150058908 >> v2:172.21.2.111:3300/0 conn(0x7fab5c005dc0 0x7fab5c042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab62ffd700 1 -- 172.21.2.111:0/3150058908 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821378048 0 0) 0x7fab5c0dc6e0 con 0x7fab5c0dbb90 2021-04-03T03:14:02.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab68d3a700 1 -- 172.21.2.111:0/3150058908 >> v1:172.21.2.115:6789/0 conn(0x7fab5c006bc0 legacy=0x7fab5c0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab68d3a700 1 -- 172.21.2.111:0/3150058908 >> v1:172.21.2.111:6789/0 conn(0x7fab5c0dbb90 legacy=0x7fab5c0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab68d3a700 1 -- 172.21.2.111:0/3150058908 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0422b0 con 0x7fab5c005dc0 2021-04-03T03:14:02.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab68d3a700 1 --2- 172.21.2.111:0/3150058908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c005dc0 0x7fab5c042750 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fab5c0087e0 tx=0x7fab6405ad70).ready entity=mon.0 client_cookie=16d82504356c59e3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab62ffd700 1 -- 172.21.2.111:0/3150058908 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6405bda0 con 0x7fab5c005dc0 2021-04-03T03:14:02.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab62ffd700 1 -- 172.21.2.111:0/3150058908 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6405df60 con 0x7fab5c005dc0 2021-04-03T03:14:02.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab62ffd700 1 -- 172.21.2.111:0/3150058908 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab640699f0 con 0x7fab5c005dc0 2021-04-03T03:14:02.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- 172.21.2.111:0/3150058908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c005dc0 msgr2=0x7fab5c042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 --2- 172.21.2.111:0/3150058908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c005dc0 0x7fab5c042750 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fab5c0087e0 tx=0x7fab6405ad70).stop 2021-04-03T03:14:02.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- 172.21.2.111:0/3150058908 shutdown_connections 2021-04-03T03:14:02.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 --2- 172.21.2.111:0/3150058908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c005dc0 0x7fab5c042750 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- 172.21.2.111:0/3150058908 >> 172.21.2.111:0/3150058908 conn(0x7fab5c0462c0 msgr2=0x7fab5c0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- 172.21.2.111:0/3150058908 shutdown_connections 2021-04-03T03:14:02.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.227+0000 7fab6953b700 1 -- 172.21.2.111:0/3150058908 wait complete. 2021-04-03T03:14:02.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 Processor -- start 2021-04-03T03:14:02.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 -- start start 2021-04-03T03:14:02.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab5c005dc0 0x7fab5c0d5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab5c0dbb90 0x7fab5c0dc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c09f4b0 0x7fab5c0e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0d6260 con 0x7fab5c09f4b0 2021-04-03T03:14:02.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0de4e0 con 0x7fab5c005dc0 2021-04-03T03:14:02.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0e7520 con 0x7fab5c0dbb90 2021-04-03T03:14:02.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c09f4b0 0x7fab5c0e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c09f4b0 0x7fab5c0e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53260/0 (socket says 172.21.2.111:53260) 2021-04-03T03:14:02.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 -- 172.21.2.111:0/108600293 learned_addr learned my addr 172.21.2.111:0/108600293 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:02.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab63fff700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab5c005dc0 0x7fab5c0d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab637fe700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab5c0dbb90 0x7fab5c0dc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 -- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab5c0dbb90 msgr2=0x7fab5c0dc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab5c0dbb90 0x7fab5c0dc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 -- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab5c005dc0 msgr2=0x7fab5c0d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab5c005dc0 0x7fab5c0d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 -- 172.21.2.111:0/108600293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0e77c0 con 0x7fab5c09f4b0 2021-04-03T03:14:02.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab63fff700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab5c005dc0 0x7fab5c0d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:02.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab637fe700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab5c0dbb90 0x7fab5c0dc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:02.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab68d3a700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c09f4b0 0x7fab5c0e7c60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fab5c0087e0 tx=0x7fab64064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab617fa700 1 -- 172.21.2.111:0/108600293 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab640441b0 con 0x7fab5c09f4b0 2021-04-03T03:14:02.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab617fa700 1 -- 172.21.2.111:0/108600293 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6405f2c0 con 0x7fab5c09f4b0 2021-04-03T03:14:02.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab617fa700 1 -- 172.21.2.111:0/108600293 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6405f620 con 0x7fab5c09f4b0 2021-04-03T03:14:02.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c0de1b0 con 0x7fab5c09f4b0 2021-04-03T03:14:02.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c0dce60 con 0x7fab5c09f4b0 2021-04-03T03:14:02.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c0de340 con 0x7fab5c09f4b0 2021-04-03T03:14:02.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab617fa700 1 -- 172.21.2.111:0/108600293 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fab6408c020 con 0x7fab5c09f4b0 2021-04-03T03:14:02.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab617fa700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab44056fb0 0x7fab44059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab617fa700 1 -- 172.21.2.111:0/108600293 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fab640e0ad0 con 0x7fab5c09f4b0 2021-04-03T03:14:02.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab63fff700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab44056fb0 0x7fab44059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.231+0000 7fab63fff700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab44056fb0 0x7fab44059470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fab54008000 tx=0x7fab54009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.235+0000 7fab617fa700 1 -- 172.21.2.111:0/108600293 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fab5c0de340 con 0x7fab5c09f4b0 2021-04-03T03:14:02.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab5c0de340 con 0x7fab5c09f4b0 2021-04-03T03:14:02.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab617fa700 1 -- 172.21.2.111:0/108600293 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fab5c0de340 con 0x7fab5c09f4b0 2021-04-03T03:14:02.375 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:02.376 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":25,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:13:42.208611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/167825619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":167825619},{"type":"v1","addr":"172.21.2.111:6835","nonce":167825619}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:14:02.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab44056fb0 msgr2=0x7fab44059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab44056fb0 0x7fab44059470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fab54008000 tx=0x7fab54009040).stop 2021-04-03T03:14:02.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c09f4b0 msgr2=0x7fab5c0e7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c09f4b0 0x7fab5c0e7c60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fab5c0087e0 tx=0x7fab64064030).stop 2021-04-03T03:14:02.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 shutdown_connections 2021-04-03T03:14:02.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab5c09f4b0 0x7fab5c0e7c60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab44056fb0 0x7fab44059470 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab5c005dc0 0x7fab5c0d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 --2- 172.21.2.111:0/108600293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab5c0dbb90 0x7fab5c0dc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 >> 172.21.2.111:0/108600293 conn(0x7fab5c0462c0 msgr2=0x7fab5c0d7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 shutdown_connections 2021-04-03T03:14:02.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.359+0000 7fab6953b700 1 -- 172.21.2.111:0/108600293 wait complete. 2021-04-03T03:14:02.380 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 25 2021-04-03T03:14:02.382 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2021-04-03T03:14:02.446 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-04-03T03:14:02.447 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-04-03T03:14:02.448 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:02.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 Processor -- start 2021-04-03T03:14:02.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- start start 2021-04-03T03:14:02.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa3dc005dc0 0x7fa3dc042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0dc6e0 con 0x7fa3dc006bc0 2021-04-03T03:14:02.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc042010 con 0x7fa3dc0dbb90 2021-04-03T03:14:02.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0d9560 con 0x7fa3dc005dc0 2021-04-03T03:14:02.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e3fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa3dc006bc0 0x7fa3dc0056b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54058/0 (socket says 172.21.2.111:54058) 2021-04-03T03:14:02.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e3fff700 1 -- 172.21.2.111:0/399812666 learned_addr learned my addr 172.21.2.111:0/399812666 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:02.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171114626 0 0) 0x7fa3dc0dc6e0 con 0x7fa3dc006bc0 2021-04-03T03:14:02.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c8003240 con 0x7fa3dc006bc0 2021-04-03T03:14:02.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9616700 1 --2- 172.21.2.111:0/399812666 >> v2:172.21.2.115:3300/0 conn(0x7fa3dc005dc0 0x7fa3dc042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 813086362 0 0) 0x7fa3c8003240 con 0x7fa3dc006bc0 2021-04-03T03:14:02.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 >> v2:172.21.2.115:3300/0 conn(0x7fa3dc005dc0 msgr2=0x7fa3dc042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 --2- 172.21.2.111:0/399812666 >> v2:172.21.2.115:3300/0 conn(0x7fa3dc005dc0 0x7fa3dc042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 >> v1:172.21.2.115:6789/0 conn(0x7fa3dc0dbb90 legacy=0x7fa3dc0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0422b0 con 0x7fa3dc006bc0 2021-04-03T03:14:02.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa3e40450f0 con 0x7fa3dc006bc0 2021-04-03T03:14:02.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 >> v1:172.21.2.111:6789/0 conn(0x7fa3dc006bc0 legacy=0x7fa3dc0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c80037b0 0x7fa3c8003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3c80044e0 0x7fa3c8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c8004e50 0x7fa3c800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0d9560 con 0x7fa3c80037b0 2021-04-03T03:14:02.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80041d0 con 0x7fa3c8004e50 2021-04-03T03:14:02.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c800bc20 con 0x7fa3c80044e0 2021-04-03T03:14:02.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c80037b0 0x7fa3c8003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e8e15700 1 -- 172.21.2.111:0/399812666 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3c80044e0 msgr2=0x7fa3c8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3c80044e0 0x7fa3c8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e8e15700 1 -- 172.21.2.111:0/399812666 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c8004e50 msgr2=0x7fa3c800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c8004e50 0x7fa3c800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e8e15700 1 -- 172.21.2.111:0/399812666 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c8003240 con 0x7fa3c80037b0 2021-04-03T03:14:02.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9616700 1 -- 172.21.2.111:0/399812666 reap_dead start 2021-04-03T03:14:02.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c80037b0 0x7fa3c8003bb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fa3d0001f60 tx=0x7fa3d0007b70).ready entity=mon.0 client_cookie=727b0d3b8f59035f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d000c870 con 0x7fa3c80037b0 2021-04-03T03:14:02.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d000c9d0 con 0x7fa3c80037b0 2021-04-03T03:14:02.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e37fe700 1 -- 172.21.2.111:0/399812666 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d000cd30 con 0x7fa3c80037b0 2021-04-03T03:14:02.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- 172.21.2.111:0/399812666 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c80037b0 msgr2=0x7fa3c8003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c80037b0 0x7fa3c8003bb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fa3d0001f60 tx=0x7fa3d0007b70).stop 2021-04-03T03:14:02.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- 172.21.2.111:0/399812666 shutdown_connections 2021-04-03T03:14:02.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/399812666 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c80037b0 0x7fa3c8003bb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- 172.21.2.111:0/399812666 >> 172.21.2.111:0/399812666 conn(0x7fa3dc0462c0 msgr2=0x7fa3dc0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- 172.21.2.111:0/399812666 shutdown_connections 2021-04-03T03:14:02.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.487+0000 7fa3e9e17700 1 -- 172.21.2.111:0/399812666 wait complete. 2021-04-03T03:14:02.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 Processor -- start 2021-04-03T03:14:02.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 -- start start 2021-04-03T03:14:02.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c80037b0 0x7fa3dc0dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3dc0e7270 0x7fa3dc0e7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3dc0a33e0 0x7fa3dc0e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0d6530 con 0x7fa3dc0e7270 2021-04-03T03:14:02.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc09ff60 con 0x7fa3c80037b0 2021-04-03T03:14:02.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc039760 con 0x7fa3dc0a33e0 2021-04-03T03:14:02.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3dc0e7270 0x7fa3dc0e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3dc0e7270 0x7fa3dc0e7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53282/0 (socket says 172.21.2.111:53282) 2021-04-03T03:14:02.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 -- 172.21.2.111:0/2193328029 learned_addr learned my addr 172.21.2.111:0/2193328029 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:02.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c80037b0 0x7fa3dc0dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9616700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3dc0a33e0 0x7fa3dc0e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 -- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3dc0a33e0 msgr2=0x7fa3dc0e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3dc0a33e0 0x7fa3dc0e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 -- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c80037b0 msgr2=0x7fa3dc0dcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c80037b0 0x7fa3dc0dcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 -- 172.21.2.111:0/2193328029 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc039a00 con 0x7fa3dc0e7270 2021-04-03T03:14:02.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9616700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3dc0a33e0 0x7fa3dc0e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:02.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c80037b0 0x7fa3dc0dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:02.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e3fff700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3dc0e7270 0x7fa3dc0e7960 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fa3dc0a00f0 tx=0x7fa3dc040b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e1ffb700 1 -- 172.21.2.111:0/2193328029 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e40699f0 con 0x7fa3dc0e7270 2021-04-03T03:14:02.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e1ffb700 1 -- 172.21.2.111:0/2193328029 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e405b040 con 0x7fa3dc0e7270 2021-04-03T03:14:02.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e1ffb700 1 -- 172.21.2.111:0/2193328029 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e4061300 con 0x7fa3dc0e7270 2021-04-03T03:14:02.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc09fcc0 con 0x7fa3dc0e7270 2021-04-03T03:14:02.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc039ca0 con 0x7fa3dc0e7270 2021-04-03T03:14:02.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc004da0 con 0x7fa3dc0e7270 2021-04-03T03:14:02.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e1ffb700 1 -- 172.21.2.111:0/2193328029 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa3e405f9c0 con 0x7fa3dc0e7270 2021-04-03T03:14:02.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e1ffb700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3cc056f20 0x7fa3cc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e1ffb700 1 -- 172.21.2.111:0/2193328029 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa3e40df2b0 con 0x7fa3dc0e7270 2021-04-03T03:14:02.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.491+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3cc056f20 0x7fa3cc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.495+0000 7fa3e8e15700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3cc056f20 0x7fa3cc0593e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fa3d0008000 tx=0x7fa3d0016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.495+0000 7fa3e1ffb700 1 -- 172.21.2.111:0/2193328029 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa3e40b4640 con 0x7fa3dc0e7270 2021-04-03T03:14:02.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3dc046ff0 con 0x7fa3dc0e7270 2021-04-03T03:14:02.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e1ffb700 1 -- 172.21.2.111:0/2193328029 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fa3e408a070 con 0x7fa3dc0e7270 2021-04-03T03:14:02.643 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:02.644 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":25,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:13:42.208611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/167825619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":167825619},{"type":"v1","addr":"172.21.2.111:6835","nonce":167825619}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:14:02.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3cc056f20 msgr2=0x7fa3cc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3cc056f20 0x7fa3cc0593e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fa3d0008000 tx=0x7fa3d0016000).stop 2021-04-03T03:14:02.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3dc0e7270 msgr2=0x7fa3dc0e7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3dc0e7270 0x7fa3dc0e7960 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fa3dc0a00f0 tx=0x7fa3dc040b00).stop 2021-04-03T03:14:02.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 shutdown_connections 2021-04-03T03:14:02.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3dc0a33e0 0x7fa3dc0e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3cc056f20 0x7fa3cc0593e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c80037b0 0x7fa3dc0dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 --2- 172.21.2.111:0/2193328029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3dc0e7270 0x7fa3dc0e7960 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 >> 172.21.2.111:0/2193328029 conn(0x7fa3dc0462c0 msgr2=0x7fa3dc0d7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 shutdown_connections 2021-04-03T03:14:02.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.627+0000 7fa3e9e17700 1 -- 172.21.2.111:0/2193328029 wait complete. 2021-04-03T03:14:02.648 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 25 2021-04-03T03:14:02.651 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 10 2021-04-03T03:14:02.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.691+0000 7f9a620e8700 1 Processor -- start 2021-04-03T03:14:02.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.691+0000 7f9a620e8700 1 -- start start 2021-04-03T03:14:02.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.691+0000 7f9a620e8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0603f0 0x7f9a5c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.691+0000 7f9a620e8700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9a5c0b17b0 0x7f9a5c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.691+0000 7f9a620e8700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5c0607d0 con 0x7f9a5c0add50 2021-04-03T03:14:02.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.691+0000 7f9a620e8700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c1451c0 con 0x7f9a5c0b17b0 2021-04-03T03:14:02.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.691+0000 7f9a620e8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c14b090 con 0x7f9a5c0603f0 2021-04-03T03:14:02.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a610e6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0603f0 0x7f9a5c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a618e7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9a5c0b17b0 0x7f9a5c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a608e5700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f9a5c0add50 0x7f9a5c147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:38384/0 (socket says 172.21.2.111:38384) 2021-04-03T03:14:02.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a608e5700 1 -- 172.21.2.111:0/1217703196 learned_addr learned my addr 172.21.2.111:0/1217703196 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:02.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a618e7700 1 -- 172.21.2.111:0/1217703196 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0603f0 msgr2=0x7f9a5c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a618e7700 1 --2- 172.21.2.111:0/1217703196 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0603f0 0x7f9a5c1447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a618e7700 1 -- 172.21.2.111:0/1217703196 >> v1:172.21.2.115:6789/0 conn(0x7f9a5c0add50 legacy=0x7f9a5c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a618e7700 1 -- 172.21.2.111:0/1217703196 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c14b330 con 0x7f9a5c0b17b0 2021-04-03T03:14:02.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a618e7700 1 --2- 172.21.2.111:0/1217703196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 0x7f9a5c144cd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f9a540032b0 tx=0x7f9a540091d0).ready entity=mon.1 client_cookie=2fa562966742372d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a4ffff700 1 -- 172.21.2.111:0/1217703196 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5400b9a0 con 0x7f9a5c0b17b0 2021-04-03T03:14:02.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a4ffff700 1 -- 172.21.2.111:0/1217703196 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5400bb00 con 0x7f9a5c0b17b0 2021-04-03T03:14:02.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a4ffff700 1 -- 172.21.2.111:0/1217703196 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5400bc70 con 0x7f9a5c0b17b0 2021-04-03T03:14:02.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 -- 172.21.2.111:0/1217703196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 msgr2=0x7f9a5c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 --2- 172.21.2.111:0/1217703196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 0x7f9a5c144cd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f9a540032b0 tx=0x7f9a540091d0).stop 2021-04-03T03:14:02.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 -- 172.21.2.111:0/1217703196 shutdown_connections 2021-04-03T03:14:02.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 --2- 172.21.2.111:0/1217703196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 0x7f9a5c144cd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 --2- 172.21.2.111:0/1217703196 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0603f0 0x7f9a5c1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 -- 172.21.2.111:0/1217703196 >> 172.21.2.111:0/1217703196 conn(0x7f9a5c0b61c0 msgr2=0x7f9a5c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 -- 172.21.2.111:0/1217703196 shutdown_connections 2021-04-03T03:14:02.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 -- 172.21.2.111:0/1217703196 wait complete. 2021-04-03T03:14:02.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 Processor -- start 2021-04-03T03:14:02.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.695+0000 7f9a620e8700 1 -- start start 2021-04-03T03:14:02.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a5c0603f0 0x7f9a5c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0add50 0x7f9a5c063b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 0x7f9a5c066230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c064000 con 0x7f9a5c0603f0 2021-04-03T03:14:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c065e00 con 0x7f9a5c0b17b0 2021-04-03T03:14:02.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c062a70 con 0x7f9a5c0add50 2021-04-03T03:14:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a5c0603f0 0x7f9a5c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a5c0603f0 0x7f9a5c0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53292/0 (socket says 172.21.2.111:53292) 2021-04-03T03:14:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 -- 172.21.2.111:0/419722415 learned_addr learned my addr 172.21.2.111:0/419722415 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 -- 172.21.2.111:0/419722415 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0add50 msgr2=0x7f9a5c063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0add50 0x7f9a5c063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 -- 172.21.2.111:0/419722415 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 msgr2=0x7f9a5c066230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 0x7f9a5c066230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 -- 172.21.2.111:0/419722415 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c062d10 con 0x7f9a5c0603f0 2021-04-03T03:14:02.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a610e6700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a5c0603f0 0x7f9a5c0b4cf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f9a5800c000 tx=0x7f9a58006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a4e7fc700 1 -- 172.21.2.111:0/419722415 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5800cae0 con 0x7f9a5c0603f0 2021-04-03T03:14:02.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a4e7fc700 1 -- 172.21.2.111:0/419722415 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5800cc40 con 0x7f9a5c0603f0 2021-04-03T03:14:02.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c062fb0 con 0x7f9a5c0603f0 2021-04-03T03:14:02.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a4e7fc700 1 -- 172.21.2.111:0/419722415 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a580123f0 con 0x7f9a5c0603f0 2021-04-03T03:14:02.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c0635c0 con 0x7f9a5c0603f0 2021-04-03T03:14:02.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a4e7fc700 1 -- 172.21.2.111:0/419722415 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9a58012550 con 0x7f9a5c0603f0 2021-04-03T03:14:02.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a4e7fc700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a3c056ea0 0x7f9a3c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a4e7fc700 1 -- 172.21.2.111:0/419722415 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9a58070790 con 0x7f9a5c0603f0 2021-04-03T03:14:02.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a608e5700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a3c056ea0 0x7f9a3c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a608e5700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a3c056ea0 0x7f9a3c059360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f9a5c0b5600 tx=0x7f9a500077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a71b66740 1 Processor -- start 2021-04-03T03:14:02.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.699+0000 7f9a71b66740 1 -- start start 2021-04-03T03:14:02.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x271a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 0x25ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2503820 0x26e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x262df70 con 0x269b3b0 2021-04-03T03:14:02.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x262e0b0 con 0x2503820 2021-04-03T03:14:02.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x262f4e0 con 0x2583eb0 2021-04-03T03:14:02.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x271a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x271a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53302/0 (socket says 172.21.2.111:53302) 2021-04-03T03:14:02.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 -- 172.21.2.111:0/3213554766 learned_addr learned my addr 172.21.2.111:0/3213554766 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:02.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 -- 172.21.2.111:0/3213554766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 msgr2=0x25ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 --2- 172.21.2.111:0/3213554766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 0x25ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 -- 172.21.2.111:0/3213554766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2503820 msgr2=0x26e35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 --2- 172.21.2.111:0/3213554766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2503820 0x26e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 -- 172.21.2.111:0/3213554766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x261ec90 con 0x269b3b0 2021-04-03T03:14:02.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 --2- 172.21.2.111:0/3213554766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x271a880 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9a580709a0 tx=0x7f9a58014000).ready entity=mon.0 client_cookie=e11d9e1eba7d095d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a620e8700 1 -- 172.21.2.111:0/3213554766 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a58048ec0 con 0x269b3b0 2021-04-03T03:14:02.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a620e8700 1 -- 172.21.2.111:0/3213554766 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a580455d0 con 0x269b3b0 2021-04-03T03:14:02.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a620e8700 1 -- 172.21.2.111:0/3213554766 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a58045930 con 0x269b3b0 2021-04-03T03:14:02.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/3213554766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 msgr2=0x271a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- 172.21.2.111:0/3213554766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x271a880 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9a580709a0 tx=0x7f9a58014000).stop 2021-04-03T03:14:02.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/3213554766 shutdown_connections 2021-04-03T03:14:02.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- 172.21.2.111:0/3213554766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2503820 0x26e35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- 172.21.2.111:0/3213554766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x271a880 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- 172.21.2.111:0/3213554766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 0x25ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/3213554766 >> 172.21.2.111:0/3213554766 conn(0x26cac10 msgr2=0x26e1610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/3213554766 shutdown_connections 2021-04-03T03:14:02.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/3213554766 wait complete. 2021-04-03T03:14:02.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- start start 2021-04-03T03:14:02.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 Processor -- start 2021-04-03T03:14:02.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 0x26e8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2583eb0 0x26ea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x26ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x2610430 con 0x269b3b0 2021-04-03T03:14:02.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x2728d80 con 0x2583eb0 2021-04-03T03:14:02.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x2729020 con 0x2503820 2021-04-03T03:14:02.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 10 client.0 ll_register_callbacks cb 0x26cab70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:14:02.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x26ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x26ea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53310/0 (socket says 172.21.2.111:53310) 2021-04-03T03:14:02.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 -- 172.21.2.111:0/2192011249 learned_addr learned my addr 172.21.2.111:0/2192011249 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:02.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 -- 172.21.2.111:0/2192011249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 msgr2=0x26e8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 0x26e8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 -- 172.21.2.111:0/2192011249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2583eb0 msgr2=0x26ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2583eb0 0x26ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 -- 172.21.2.111:0/2192011249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26274c0 con 0x269b3b0 2021-04-03T03:14:02.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a618e7700 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x26ea830 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9a540032b0 tx=0x7f9a54016000).ready entity=mon.0 client_cookie=cd6d05a32b696d52 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:14:02.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 1 -- 172.21.2.111:0/2192011249 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a54006e80 con 0x269b3b0 2021-04-03T03:14:02.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 1 -- 172.21.2.111:0/2192011249 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9a5400be70 con 0x269b3b0 2021-04-03T03:14:02.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25288d0 con 0x269b3b0 2021-04-03T03:14:02.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 1 -- 172.21.2.111:0/2192011249 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a54006ca0 con 0x269b3b0 2021-04-03T03:14:02.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 10 client.6129 fetch_fsmap learned FSMap version 25 2021-04-03T03:14:02.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x24b3790 con 0x269b3b0 2021-04-03T03:14:02.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 1 -- 172.21.2.111:0/2192011249 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25288d0 con 0x269b3b0 2021-04-03T03:14:02.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 1 -- 172.21.2.111:0/2192011249 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 25) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f9a54020930 con 0x269b3b0 2021-04-03T03:14:02.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 10 client.6129 fetch_fsmap finished waiting for FSMap version 25 2021-04-03T03:14:02.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 10 client.6129 resolve_mds: resolved ID 'a' to GID 5934 2021-04-03T03:14:02.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 20 client.6129 populate_metadata read hostname 'gibba011' 2021-04-03T03:14:02.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2495c20 0x272bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 4 client.6129 mds_command: new command op to 5934 tid=0[{"prefix": "get_command_descriptions"}] 2021-04-03T03:14:02.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 --> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2495170 con 0x2495c20 2021-04-03T03:14:02.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2495c20 0x272bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a610e6700 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2495c20 0x272bf60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ee307b6ea930ffe6 server_cookie=42b06eb60fa86d9d in_seq=0 out_seq=0 2021-04-03T03:14:02.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 10 client.6129 ms_handle_connect on v2:172.21.2.111:6834/167825619 2021-04-03T03:14:02.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 1 -- 172.21.2.111:0/2192011249 <== mds.0 v2:172.21.2.111:6834/167825619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2495170 con 0x2495c20 2021-04-03T03:14:02.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a37fff700 10 client.6129 handle_command_reply: tid=0 2021-04-03T03:14:02.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 client.6129 shutdown 2021-04-03T03:14:02.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 msgr2=0x26ea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.703+0000 7f9a71b66740 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x26ea830 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9a540032b0 tx=0x7f9a54016000).stop 2021-04-03T03:14:02.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 shutdown_connections 2021-04-03T03:14:02.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2495c20 0x272bf60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x269b3b0 0x26ea830 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 0x26e8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 --2- 172.21.2.111:0/2192011249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x2583eb0 0x26ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a37fff700 0 client.6129 ms_handle_reset on v2:172.21.2.111:6834/167825619 2021-04-03T03:14:02.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 >> 172.21.2.111:0/2192011249 conn(0x26cac10 msgr2=0x2719690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 shutdown_connections 2021-04-03T03:14:02.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 1 -- 172.21.2.111:0/2192011249 wait complete. 2021-04-03T03:14:02.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.707+0000 7f9a71b66740 20 client.6129 trim_cache size 0 max 16384 2021-04-03T03:14:02.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 Processor -- start 2021-04-03T03:14:02.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 -- start start 2021-04-03T03:14:02.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 0x26e4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2583eb0 0x26e1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 0x26e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x25ff6c0 con 0x2583eb0 2021-04-03T03:14:02.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x271a0e0 con 0x269b3b0 2021-04-03T03:14:02.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.711+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x26eb6b0 con 0x2503820 2021-04-03T03:14:02.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2583eb0 0x26e1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2583eb0 0x26e1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53318/0 (socket says 172.21.2.111:53318) 2021-04-03T03:14:02.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 -- 172.21.2.111:0/1073159835 learned_addr learned my addr 172.21.2.111:0/1073159835 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:02.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 0x26e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 -- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 msgr2=0x26e4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 0x26e4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 -- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 msgr2=0x26e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a618e7700 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 0x26e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 0x26e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 -- 172.21.2.111:0/1073159835 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2729c00 con 0x2583eb0 2021-04-03T03:14:02.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2583eb0 0x26e1610 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f9a50001b50 tx=0x7f9a50007000).ready entity=mon.0 client_cookie=f24a16d0f31b5614 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a427fc700 1 -- 172.21.2.111:0/1073159835 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a50014710 con 0x2583eb0 2021-04-03T03:14:02.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a427fc700 1 -- 172.21.2.111:0/1073159835 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5001c030 con 0x2583eb0 2021-04-03T03:14:02.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a427fc700 1 -- 172.21.2.111:0/1073159835 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a50014a60 con 0x2583eb0 2021-04-03T03:14:02.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/1073159835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2583eb0 msgr2=0x26e1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:02.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2583eb0 0x26e1610 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f9a50001b50 tx=0x7f9a50007000).stop 2021-04-03T03:14:02.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/1073159835 shutdown_connections 2021-04-03T03:14:02.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 0x26e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2503820 0x26e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- 172.21.2.111:0/1073159835 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2583eb0 0x26e1610 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:02.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/1073159835 >> 172.21.2.111:0/1073159835 conn(0x26cac10 msgr2=0x26ead80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:02.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/1073159835 shutdown_connections 2021-04-03T03:14:02.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/1073159835 wait complete. 2021-04-03T03:14:02.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- start start 2021-04-03T03:14:02.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 Processor -- start 2021-04-03T03:14:02.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2503820 0x271ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 0x277ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 0x277f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x271f8c0 con 0x2503820 2021-04-03T03:14:02.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x26ea4d0 con 0x269b3b0 2021-04-03T03:14:02.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x271fb00 con 0x2583eb0 2021-04-03T03:14:02.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 10 client.0 ll_register_callbacks cb 0x26ec8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-04-03T03:14:02.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2503820 0x271ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2503820 0x271ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53324/0 (socket says 172.21.2.111:53324) 2021-04-03T03:14:02.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 -- 172.21.2.111:0/673169951 learned_addr learned my addr 172.21.2.111:0/673169951 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:02.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 -- 172.21.2.111:0/673169951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 msgr2=0x277ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 0x277ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 -- 172.21.2.111:0/673169951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 msgr2=0x277f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:02.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 0x277f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:02.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 -- 172.21.2.111:0/673169951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c14a540 con 0x2503820 2021-04-03T03:14:02.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a610e6700 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2503820 0x271ffa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f9a5806f000 tx=0x7f9a58035280).ready entity=mon.0 client_cookie=67e09702f433e2aa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:02.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a417fa700 10 client.0 ms_handle_connect on v2:172.21.2.111:3300/0 2021-04-03T03:14:02.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a417fa700 1 -- 172.21.2.111:0/673169951 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a58049320 con 0x2503820 2021-04-03T03:14:02.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a417fa700 1 -- 172.21.2.111:0/673169951 <== mon.0 v2:172.21.2.111:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9a58038d80 con 0x2503820 2021-04-03T03:14:02.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26e2c80 con 0x2503820 2021-04-03T03:14:02.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a417fa700 1 -- 172.21.2.111:0/673169951 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a58048740 con 0x2503820 2021-04-03T03:14:02.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 10 client.6135 fetch_fsmap learned FSMap version 25 2021-04-03T03:14:02.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x24bee70 con 0x2503820 2021-04-03T03:14:02.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a417fa700 1 -- 172.21.2.111:0/673169951 <== mon.0 v2:172.21.2.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26e2c80 con 0x2503820 2021-04-03T03:14:02.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a417fa700 1 -- 172.21.2.111:0/673169951 <== mon.0 v2:172.21.2.111:3300/0 5 ==== fsmap(e 25) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f9a580488a0 con 0x2503820 2021-04-03T03:14:02.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 10 client.6135 fetch_fsmap finished waiting for FSMap version 25 2021-04-03T03:14:02.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 10 client.6135 resolve_mds: resolved ID 'a' to GID 5934 2021-04-03T03:14:02.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 20 client.6135 populate_metadata read hostname 'gibba011' 2021-04-03T03:14:02.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2720490 0x277f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:02.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 4 client.6135 mds_command: new command op to 5934 tid=0[{"prefix": "cache drop", "timeout": 10}] 2021-04-03T03:14:02.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 --> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x2547730 con 0x2720490 2021-04-03T03:14:02.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2720490 0x277f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:02.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a417fa700 10 client.6135 ms_handle_connect on v2:172.21.2.111:6834/167825619 2021-04-03T03:14:02.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:02.715+0000 7f9a608e5700 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2720490 0x277f840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=228e380ec3cc466f server_cookie=69dd7f4b722c2329 in_seq=0 out_seq=0 2021-04-03T03:14:12.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a417fa700 1 -- 172.21.2.111:0/673169951 <== mds.0 v2:172.21.2.111:6834/167825619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+400 (crc 0 0 0) 0x2547730 con 0x2720490 2021-04-03T03:14:12.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a417fa700 10 client.6135 handle_command_reply: tid=0 2021-04-03T03:14:12.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 client.6135 shutdown 2021-04-03T03:14:12.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2503820 msgr2=0x271ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:12.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2503820 0x271ffa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f9a5806f000 tx=0x7f9a58035280).stop 2021-04-03T03:14:12.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 shutdown_connections 2021-04-03T03:14:12.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x269b3b0 0x277f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:12.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:6834/167825619,v1:172.21.2.111:6835/167825619] conn(0x2720490 0x277f840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:12.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x2503820 0x271ffa0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:12.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 --2- 172.21.2.111:0/673169951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x2583eb0 0x277ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:12.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a417fa700 0 client.6135 ms_handle_reset on v2:172.21.2.111:6834/167825619 2021-04-03T03:14:12.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 >> 172.21.2.111:0/673169951 conn(0x26cac10 msgr2=0x26e1610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:12.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 shutdown_connections 2021-04-03T03:14:12.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 1 -- 172.21.2.111:0/673169951 wait complete. 2021-04-03T03:14:12.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a71b66740 20 client.6135 trim_cache size 0 max 16384 2021-04-03T03:14:12.872 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:14:12.873 INFO:teuthology.orchestra.run.gibba011.stdout: "client_recall": { 2021-04-03T03:14:12.873 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": -110, 2021-04-03T03:14:12.873 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "(110) Connection timed out", 2021-04-03T03:14:12.873 INFO:teuthology.orchestra.run.gibba011.stdout: "recalled": 240 2021-04-03T03:14:12.874 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:12.874 INFO:teuthology.orchestra.run.gibba011.stdout: "flush_journal": { 2021-04-03T03:14:12.874 INFO:teuthology.orchestra.run.gibba011.stdout: "return_code": 0, 2021-04-03T03:14:12.874 INFO:teuthology.orchestra.run.gibba011.stdout: "message": "" 2021-04-03T03:14:12.874 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:12.875 INFO:teuthology.orchestra.run.gibba011.stdout: "trim_cache": { 2021-04-03T03:14:12.875 INFO:teuthology.orchestra.run.gibba011.stdout: "trimmed": 220 2021-04-03T03:14:12.875 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:12.875 INFO:teuthology.orchestra.run.gibba011.stdout: "cache": { 2021-04-03T03:14:12.876 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": { 2021-04-03T03:14:12.876 INFO:teuthology.orchestra.run.gibba011.stdout: "items": 15444, 2021-04-03T03:14:12.876 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes": 2428432 2021-04-03T03:14:12.876 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:12.876 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:12.877 INFO:teuthology.orchestra.run.gibba011.stdout: "duration": 10.135330905 2021-04-03T03:14:12.877 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:14:12.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a3c056ea0 msgr2=0x7f9a3c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:12.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a620e8700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a3c056ea0 0x7f9a3c059360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f9a5c0b5600 tx=0x7f9a500077e0).stop 2021-04-03T03:14:12.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a5c0603f0 msgr2=0x7f9a5c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:12.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a620e8700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a5c0603f0 0x7f9a5c0b4cf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f9a5800c000 tx=0x7f9a58006dd0).stop 2021-04-03T03:14:12.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 shutdown_connections 2021-04-03T03:14:12.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.853+0000 7f9a620e8700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a3c056ea0 0x7f9a3c059360 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:12.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.857+0000 7f9a620e8700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a5c0b17b0 0x7f9a5c066230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:12.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.857+0000 7f9a620e8700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a5c0603f0 0x7f9a5c0b4cf0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:12.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.857+0000 7f9a620e8700 1 --2- 172.21.2.111:0/419722415 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a5c0add50 0x7f9a5c063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:12.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.857+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 >> 172.21.2.111:0/419722415 conn(0x7f9a5c0b61c0 msgr2=0x7f9a5c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:12.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.857+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 shutdown_connections 2021-04-03T03:14:12.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:12.857+0000 7f9a620e8700 1 -- 172.21.2.111:0/419722415 wait complete. 2021-04-03T03:14:12.881 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:14:12.883 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:12.890 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:12.897 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:12.909 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:12.921 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:12.934 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:14:12.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:12.935 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:12.936 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:12.957 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:14:12.958 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:14:12.958 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:14:12.959 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:14:12.959 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:13.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15366]: fuse finished with error 0 and tester_r 0 2021-04-03T03:14:13.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:13.050 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:13.050 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:13.054 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:14:13.054 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:14:13.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:14:13.144 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:13.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:13.154 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:13.154 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:13.163 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:14:13.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:14:13.165 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:14:13.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:14:13.166 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:13.214 INFO:teuthology.orchestra.run.gibba028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:14:13.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:14:13.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:13.216 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:13.217 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:14:13.226 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:14:13.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:14:13.228 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:14:13.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:14:13.229 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:14:13.279 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:14:13.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:14:13.281 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:14:13.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 Processor -- start 2021-04-03T03:14:13.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 -- start start 2021-04-03T03:14:13.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc0add50 0x7f5efc147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5efc144ef0 0x7f5efc147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0607d0 con 0x7f5efc0603f0 2021-04-03T03:14:13.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc14b090 con 0x7f5efc144ef0 2021-04-03T03:14:13.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc14b330 con 0x7f5efc0add50 2021-04-03T03:14:13.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efad9d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5efc144ef0 0x7f5efc147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef9d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc0add50 0x7f5efc147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efa59c700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f5efc0603f0 0x7f5efc1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:38426/0 (socket says 172.21.2.111:38426) 2021-04-03T03:14:13.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efa59c700 1 -- 172.21.2.111:0/2947064017 learned_addr learned my addr 172.21.2.111:0/2947064017 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:13.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef9d9b700 1 -- 172.21.2.111:0/2947064017 >> v2:172.21.2.115:3300/0 conn(0x7f5efc144ef0 msgr2=0x7f5efc147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef9d9b700 1 --2- 172.21.2.111:0/2947064017 >> v2:172.21.2.115:3300/0 conn(0x7f5efc144ef0 0x7f5efc147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef9d9b700 1 -- 172.21.2.111:0/2947064017 >> v1:172.21.2.115:6789/0 conn(0x7f5efc0603f0 legacy=0x7f5efc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef959a700 1 -- 172.21.2.111:0/2947064017 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865387342 0 0) 0x7f5efc0607d0 con 0x7f5efc0603f0 2021-04-03T03:14:13.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef9d9b700 1 -- 172.21.2.111:0/2947064017 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc14b5d0 con 0x7f5efc0add50 2021-04-03T03:14:13.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efad9d700 1 --2- 172.21.2.111:0/2947064017 >> v2:172.21.2.115:3300/0 conn(0x7f5efc144ef0 0x7f5efc147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:14:13.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef9d9b700 1 --2- 172.21.2.111:0/2947064017 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc0add50 0x7f5efc147740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f5eec002f30 tx=0x7f5eec006dd0).ready entity=mon.2 client_cookie=1de82fc1f58e7468 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef959a700 1 -- 172.21.2.111:0/2947064017 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec009760 con 0x7f5efc0add50 2021-04-03T03:14:13.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef959a700 1 -- 172.21.2.111:0/2947064017 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec0098c0 con 0x7f5efc0add50 2021-04-03T03:14:13.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 -- 172.21.2.111:0/2947064017 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc0add50 msgr2=0x7f5efc147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5efb59e700 1 --2- 172.21.2.111:0/2947064017 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc0add50 0x7f5efc147740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f5eec002f30 tx=0x7f5eec006dd0).stop 2021-04-03T03:14:13.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.333+0000 7f5ef959a700 1 -- 172.21.2.111:0/2947064017 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec009c20 con 0x7f5efc0add50 2021-04-03T03:14:13.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- 172.21.2.111:0/2947064017 shutdown_connections 2021-04-03T03:14:13.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 --2- 172.21.2.111:0/2947064017 >> v2:172.21.2.115:3300/0 conn(0x7f5efc144ef0 0x7f5efc147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 --2- 172.21.2.111:0/2947064017 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc0add50 0x7f5efc147740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- 172.21.2.111:0/2947064017 >> 172.21.2.111:0/2947064017 conn(0x7f5efc0b61c0 msgr2=0x7f5efc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:13.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- 172.21.2.111:0/2947064017 shutdown_connections 2021-04-03T03:14:13.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- 172.21.2.111:0/2947064017 wait complete. 2021-04-03T03:14:13.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 Processor -- start 2021-04-03T03:14:13.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- start start 2021-04-03T03:14:13.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5efc0603f0 0x7f5efc1477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc0add50 0x7f5efc0a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc144ef0 0x7f5efc0a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0a88f0 con 0x7f5efc0add50 2021-04-03T03:14:13.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0aa570 con 0x7f5efc0603f0 2021-04-03T03:14:13.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc157570 con 0x7f5efc144ef0 2021-04-03T03:14:13.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc0add50 0x7f5efc0a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc0add50 0x7f5efc0a9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53338/0 (socket says 172.21.2.111:53338) 2021-04-03T03:14:13.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 -- 172.21.2.111:0/1681512643 learned_addr learned my addr 172.21.2.111:0/1681512643 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:13.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efa59c700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5efc0603f0 0x7f5efc1477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efad9d700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc144ef0 0x7f5efc0a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 -- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc144ef0 msgr2=0x7f5efc0a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc144ef0 0x7f5efc0a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 -- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5efc0603f0 msgr2=0x7f5efc1477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5efc0603f0 0x7f5efc1477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 -- 172.21.2.111:0/1681512643 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc157810 con 0x7f5efc0add50 2021-04-03T03:14:13.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efa59c700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5efc0603f0 0x7f5efc1477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:14:13.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef9d9b700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc0add50 0x7f5efc0a9a90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f5eec002f30 tx=0x7f5eec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef37fe700 1 -- 172.21.2.111:0/1681512643 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec01d450 con 0x7f5efc0add50 2021-04-03T03:14:13.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef37fe700 1 -- 172.21.2.111:0/1681512643 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec01d5b0 con 0x7f5efc0add50 2021-04-03T03:14:13.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc147c90 con 0x7f5efc0add50 2021-04-03T03:14:13.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef37fe700 1 -- 172.21.2.111:0/1681512643 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec01d910 con 0x7f5efc0add50 2021-04-03T03:14:13.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc0a6f20 con 0x7f5efc0add50 2021-04-03T03:14:13.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc057660 con 0x7f5efc0add50 2021-04-03T03:14:13.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef37fe700 1 -- 172.21.2.111:0/1681512643 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5eec00f070 con 0x7f5efc0add50 2021-04-03T03:14:13.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.337+0000 7f5ef37fe700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ee0056fb0 0x7f5ee0059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.341+0000 7f5ef37fe700 1 -- 172.21.2.111:0/1681512643 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5eec071d90 con 0x7f5efc0add50 2021-04-03T03:14:13.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.341+0000 7f5efa59c700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ee0056fb0 0x7f5ee0059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.341+0000 7f5efa59c700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ee0056fb0 0x7f5ee0059470 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f5ee400b000 tx=0x7f5ee4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.341+0000 7f5ef37fe700 1 -- 172.21.2.111:0/1681512643 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5eec047180 con 0x7f5efc0add50 2021-04-03T03:14:13.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.433+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5efc148850 con 0x7f5efc0add50 2021-04-03T03:14:13.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5ef37fe700 1 -- 172.21.2.111:0/1681512643 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5eec04a7a0 con 0x7f5efc0add50 2021-04-03T03:14:13.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ee0056fb0 msgr2=0x7f5ee0059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ee0056fb0 0x7f5ee0059470 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f5ee400b000 tx=0x7f5ee4009040).stop 2021-04-03T03:14:13.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc0add50 msgr2=0x7f5efc0a9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc0add50 0x7f5efc0a9a90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f5eec002f30 tx=0x7f5eec015000).stop 2021-04-03T03:14:13.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 shutdown_connections 2021-04-03T03:14:13.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5efc144ef0 0x7f5efc0a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5ee0056fb0 0x7f5ee0059470 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5efc0603f0 0x7f5efc1477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 --2- 172.21.2.111:0/1681512643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5efc0add50 0x7f5efc0a9a90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 >> 172.21.2.111:0/1681512643 conn(0x7f5efc0b61c0 msgr2=0x7f5efc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:13.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 shutdown_connections 2021-04-03T03:14:13.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.437+0000 7f5efb59e700 1 -- 172.21.2.111:0/1681512643 wait complete. 2021-04-03T03:14:13.459 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2021-04-03T03:14:13.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fdbad700 1 Processor -- start 2021-04-03T03:14:13.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fdbad700 1 -- start start 2021-04-03T03:14:13.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fdbad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f0048f00 0x7f71f00d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fdbad700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f71f0044040 0x7f71f0040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fdbad700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f00d52e0 con 0x7f71f0041ff0 2021-04-03T03:14:13.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fdbad700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f00492e0 con 0x7f71f0044040 2021-04-03T03:14:13.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fdbad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0044590 con 0x7f71f0048f00 2021-04-03T03:14:13.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f7fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f71f0041ff0 0x7f71f00d59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54140/0 (socket says 172.21.2.111:54140) 2021-04-03T03:14:13.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f7fff700 1 -- 172.21.2.111:0/1825112920 learned_addr learned my addr 172.21.2.111:0/1825112920 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:13.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fd3ac700 1 --2- 172.21.2.111:0/1825112920 >> v2:172.21.2.115:3300/0 conn(0x7f71f0044040 0x7f71f0040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71fcbab700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f0048f00 0x7f71f00d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739958600 0 0) 0x7f71f00d52e0 con 0x7f71f0041ff0 2021-04-03T03:14:13.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e4003240 con 0x7f71f0041ff0 2021-04-03T03:14:13.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2577058200 0 0) 0x7f71e4003240 con 0x7f71f0041ff0 2021-04-03T03:14:13.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f0048f00 msgr2=0x7f71f00d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f0048f00 0x7f71f00d54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 >> v2:172.21.2.115:3300/0 conn(0x7f71f0044040 msgr2=0x7f71f0040fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 --2- 172.21.2.111:0/1825112920 >> v2:172.21.2.115:3300/0 conn(0x7f71f0044040 0x7f71f0040fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0044830 con 0x7f71f0041ff0 2021-04-03T03:14:13.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f71e80016d0 con 0x7f71f0041ff0 2021-04-03T03:14:13.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 >> v1:172.21.2.111:6789/0 conn(0x7f71f0041ff0 legacy=0x7f71f00d59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f0048f00 0x7f71e40037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71e4003d10 0x7f71e40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71e40049d0 0x7f71e4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00492e0 con 0x7f71e40049d0 2021-04-03T03:14:13.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0044590 con 0x7f71e4003d10 2021-04-03T03:14:13.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.501+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e40042d0 con 0x7f71f0048f00 2021-04-03T03:14:13.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71e40049d0 0x7f71e4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 -- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f0048f00 msgr2=0x7f71e40037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f0048f00 0x7f71e40037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:13.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 -- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71e4003d10 msgr2=0x7f71e40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71e4003d10 0x7f71e40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:13.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 -- 172.21.2.111:0/1825112920 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e4003240 con 0x7f71e40049d0 2021-04-03T03:14:13.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71e40049d0 0x7f71e4004e50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f71f8082000 tx=0x7f71f805ad70).ready entity=mon.0 client_cookie=62a7a2da4ccc4e74 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f805df60 con 0x7f71e40049d0 2021-04-03T03:14:13.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f80699f0 con 0x7f71e40049d0 2021-04-03T03:14:13.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f77fe700 1 -- 172.21.2.111:0/1825112920 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f806b610 con 0x7f71e40049d0 2021-04-03T03:14:13.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/1825112920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71e40049d0 msgr2=0x7f71e4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 --2- 172.21.2.111:0/1825112920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71e40049d0 0x7f71e4004e50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f71f8082000 tx=0x7f71f805ad70).stop 2021-04-03T03:14:13.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 -- 172.21.2.111:0/1825112920 reap_dead start 2021-04-03T03:14:13.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/1825112920 shutdown_connections 2021-04-03T03:14:13.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/1825112920 >> 172.21.2.111:0/1825112920 conn(0x7f71f000f2e0 msgr2=0x7f71f00484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:13.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/1825112920 shutdown_connections 2021-04-03T03:14:13.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/1825112920 wait complete. 2021-04-03T03:14:13.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 Processor -- start 2021-04-03T03:14:13.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- start start 2021-04-03T03:14:13.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f00a29f0 0x7f71f00a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00d5eb0 0x7f71f00e4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f00e4500 0x7f71f00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00a35e0 con 0x7f71f00a29f0 2021-04-03T03:14:13.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00d6b20 con 0x7f71f00e4500 2021-04-03T03:14:13.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00e14e0 con 0x7f71f00d5eb0 2021-04-03T03:14:13.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f00a29f0 0x7f71f00a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f00a29f0 0x7f71f00a30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53360/0 (socket says 172.21.2.111:53360) 2021-04-03T03:14:13.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 -- 172.21.2.111:0/3060778327 learned_addr learned my addr 172.21.2.111:0/3060778327 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:13.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f7fff700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00d5eb0 0x7f71f00e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fd3ac700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f00e4500 0x7f71f00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 -- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00d5eb0 msgr2=0x7f71f00e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00d5eb0 0x7f71f00e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 -- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f00e4500 msgr2=0x7f71f00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f00e4500 0x7f71f00cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 -- 172.21.2.111:0/3060778327 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00e4990 con 0x7f71f00a29f0 2021-04-03T03:14:13.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fcbab700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f00a29f0 0x7f71f00a30f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f71f00a25c0 tx=0x7f71ec006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f5ffb700 1 -- 172.21.2.111:0/3060778327 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71ec016070 con 0x7f71f00a29f0 2021-04-03T03:14:13.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f5ffb700 1 -- 172.21.2.111:0/3060778327 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec00c860 con 0x7f71f00a29f0 2021-04-03T03:14:13.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f00d6d50 con 0x7f71f00a29f0 2021-04-03T03:14:13.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f5ffb700 1 -- 172.21.2.111:0/3060778327 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71ec00cbc0 con 0x7f71f00a29f0 2021-04-03T03:14:13.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f00d5c10 con 0x7f71f00a29f0 2021-04-03T03:14:13.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f0004da0 con 0x7f71f00a29f0 2021-04-03T03:14:13.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.505+0000 7f71f5ffb700 1 -- 172.21.2.111:0/3060778327 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f71ec00cd20 con 0x7f71f00a29f0 2021-04-03T03:14:13.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.509+0000 7f71f5ffb700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71d8056f20 0x7f71d80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.509+0000 7f71f5ffb700 1 -- 172.21.2.111:0/3060778327 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f71ec06fc70 con 0x7f71f00a29f0 2021-04-03T03:14:13.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.509+0000 7f71f7fff700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71d8056f20 0x7f71d80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.509+0000 7f71f7fff700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71d8056f20 0x7f71d80593e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f71e800d000 tx=0x7f71f00449c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.509+0000 7f71f5ffb700 1 -- 172.21.2.111:0/3060778327 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f71ec042c50 con 0x7f71f00a29f0 2021-04-03T03:14:13.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.641+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f71f00481a0 con 0x7f71f00a29f0 2021-04-03T03:14:13.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71f5ffb700 1 -- 172.21.2.111:0/3060778327 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v229) v1 ==== 147+0+0 (secure 0 0 0) 0x7f71ec045060 con 0x7f71f00a29f0 2021-04-03T03:14:13.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71d8056f20 msgr2=0x7f71d80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71d8056f20 0x7f71d80593e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f71e800d000 tx=0x7f71f00449c0).stop 2021-04-03T03:14:13.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f00a29f0 msgr2=0x7f71f00a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f00a29f0 0x7f71f00a30f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f71f00a25c0 tx=0x7f71ec006dd0).stop 2021-04-03T03:14:13.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 shutdown_connections 2021-04-03T03:14:13.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f71d8056f20 0x7f71d80593e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71f00e4500 0x7f71f00cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71f00a29f0 0x7f71f00a30f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 --2- 172.21.2.111:0/3060778327 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71f00d5eb0 0x7f71f00e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 >> 172.21.2.111:0/3060778327 conn(0x7f71f000f2e0 msgr2=0x7f71f00057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:13.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 shutdown_connections 2021-04-03T03:14:13.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.785+0000 7f71fdbad700 1 -- 172.21.2.111:0/3060778327 wait complete. 2021-04-03T03:14:13.806 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-04-03T03:14:13.806 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:14:13.807 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2021-04-03T03:14:13.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d125d4700 1 Processor -- start 2021-04-03T03:14:13.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d125d4700 1 -- start start 2021-04-03T03:14:13.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d125d4700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1d04006bc0 0x7f1d04042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d125d4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 0x7f1d040dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d125d4700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d040dc6e0 con 0x7f1d040dbb90 2021-04-03T03:14:13.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d125d4700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d040d9560 con 0x7f1d04006bc0 2021-04-03T03:14:13.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d125d4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0403dcb0 con 0x7f1d0403d360 2021-04-03T03:14:13.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d115d2700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1d040dbb90 0x7f1d040064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54158/0 (socket says 172.21.2.111:54158) 2021-04-03T03:14:13.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.853+0000 7f1d115d2700 1 -- 172.21.2.111:0/4192152032 learned_addr learned my addr 172.21.2.111:0/4192152032 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:13.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d10dd1700 1 --2- 172.21.2.111:0/4192152032 >> v2:172.21.2.115:3300/0 conn(0x7f1d04006bc0 0x7f1d04042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 0x7f1d040dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655427390 0 0) 0x7f1d040dc6e0 con 0x7f1d040dbb90 2021-04-03T03:14:13.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf4003240 con 0x7f1d040dbb90 2021-04-03T03:14:13.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3895441442 0 0) 0x7f1cf4003240 con 0x7f1d040dbb90 2021-04-03T03:14:13.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 msgr2=0x7f1d040dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 0x7f1d040dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 >> v2:172.21.2.115:3300/0 conn(0x7f1d04006bc0 msgr2=0x7f1d04042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 --2- 172.21.2.111:0/4192152032 >> v2:172.21.2.115:3300/0 conn(0x7f1d04006bc0 0x7f1d04042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040dc460 con 0x7f1d040dbb90 2021-04-03T03:14:13.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f1d000016d0 con 0x7f1d040dbb90 2021-04-03T03:14:13.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 >> v1:172.21.2.111:6789/0 conn(0x7f1d040dbb90 legacy=0x7f1d040064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1cf40037e0 0x7f1cf4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 0x7f1cf40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1cf40049d0 0x7f1cf4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040d9560 con 0x7f1cf40037e0 2021-04-03T03:14:13.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40042d0 con 0x7f1cf40049d0 2021-04-03T03:14:13.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf400b730 con 0x7f1d0403d360 2021-04-03T03:14:13.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1cf40037e0 0x7f1cf4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 -- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 msgr2=0x7f1cf40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d10dd1700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 0x7f1cf40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1cf40049d0 0x7f1cf4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d0403d360 0x7f1cf40044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 -- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1cf40049d0 msgr2=0x7f1cf4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:13.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1cf40049d0 0x7f1cf4004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 -- 172.21.2.111:0/4192152032 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4003240 con 0x7f1cf40037e0 2021-04-03T03:14:13.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1cf40037e0 0x7f1cf4003c00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f1d0403e5f0 tx=0x7f1d0403e7f0).ready entity=mon.0 client_cookie=7320b3858352f36d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0000ce90 con 0x7f1cf40037e0 2021-04-03T03:14:13.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000070d0 con 0x7f1cf40037e0 2021-04-03T03:14:13.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfffff700 1 -- 172.21.2.111:0/4192152032 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d00007430 con 0x7f1cf40037e0 2021-04-03T03:14:13.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/4192152032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1cf40037e0 msgr2=0x7f1cf4003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 --2- 172.21.2.111:0/4192152032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1cf40037e0 0x7f1cf4003c00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f1d0403e5f0 tx=0x7f1d0403e7f0).stop 2021-04-03T03:14:13.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 -- 172.21.2.111:0/4192152032 reap_dead start 2021-04-03T03:14:13.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/4192152032 shutdown_connections 2021-04-03T03:14:13.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/4192152032 >> 172.21.2.111:0/4192152032 conn(0x7f1d040462c0 msgr2=0x7f1d040466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:13.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/4192152032 shutdown_connections 2021-04-03T03:14:13.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/4192152032 wait complete. 2021-04-03T03:14:13.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 Processor -- start 2021-04-03T03:14:13.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- start start 2021-04-03T03:14:13.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d0403aa70 0x7f1d0403ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d040a0010 0x7f1d040d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d040d5bb0 0x7f1d040e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04039e90 con 0x7f1d040d5bb0 2021-04-03T03:14:13.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040df3a0 con 0x7f1d0403aa70 2021-04-03T03:14:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0403b340 con 0x7f1d040a0010 2021-04-03T03:14:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d040d5bb0 0x7f1d040e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d040d5bb0 0x7f1d040e6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53380/0 (socket says 172.21.2.111:53380) 2021-04-03T03:14:13.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d10dd1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d040a0010 0x7f1d040d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 -- 172.21.2.111:0/1112964977 learned_addr learned my addr 172.21.2.111:0/1112964977 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d115d2700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d0403aa70 0x7f1d0403ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 -- 172.21.2.111:0/1112964977 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d040a0010 msgr2=0x7f1d040d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d040a0010 0x7f1d040d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 -- 172.21.2.111:0/1112964977 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d0403aa70 msgr2=0x7f1d0403ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d0403aa70 0x7f1d0403ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 -- 172.21.2.111:0/1112964977 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0403a0c0 con 0x7f1d040d5bb0 2021-04-03T03:14:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d11dd3700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d040d5bb0 0x7f1d040e6e20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f1d0c082000 tx=0x7f1d0c064040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfe7fc700 1 -- 172.21.2.111:0/1112964977 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c0699f0 con 0x7f1d040d5bb0 2021-04-03T03:14:13.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfe7fc700 1 -- 172.21.2.111:0/1112964977 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c0685b0 con 0x7f1d040d5bb0 2021-04-03T03:14:13.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d040df5d0 con 0x7f1d040d5bb0 2021-04-03T03:14:13.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1cfe7fc700 1 -- 172.21.2.111:0/1112964977 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c068910 con 0x7f1d040d5bb0 2021-04-03T03:14:13.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d040d6030 con 0x7f1d040d5bb0 2021-04-03T03:14:13.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.857+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d04004da0 con 0x7f1d040d5bb0 2021-04-03T03:14:13.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.861+0000 7f1cfe7fc700 1 -- 172.21.2.111:0/1112964977 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1d0c05f2c0 con 0x7f1d040d5bb0 2021-04-03T03:14:13.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.861+0000 7f1cfe7fc700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cec056ea0 0x7f1cec059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:13.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.861+0000 7f1cfe7fc700 1 -- 172.21.2.111:0/1112964977 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1d0c0df120 con 0x7f1d040d5bb0 2021-04-03T03:14:13.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.861+0000 7f1d115d2700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cec056ea0 0x7f1cec059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:13.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.861+0000 7f1d115d2700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cec056ea0 0x7f1cec059360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f1d0001b000 tx=0x7f1d00015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:13.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.861+0000 7f1cfe7fc700 1 -- 172.21.2.111:0/1112964977 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1d0c0b4450 con 0x7f1d040d5bb0 2021-04-03T03:14:14.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:13.989+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f1d04046ff0 con 0x7f1d040d5bb0 2021-04-03T03:14:14.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1cfe7fc700 1 -- 172.21.2.111:0/1112964977 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v230) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1d0c0b7a70 con 0x7f1d040d5bb0 2021-04-03T03:14:14.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cec056ea0 msgr2=0x7f1cec059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cec056ea0 0x7f1cec059360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f1d0001b000 tx=0x7f1d00015000).stop 2021-04-03T03:14:14.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d040d5bb0 msgr2=0x7f1d040e6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d040d5bb0 0x7f1d040e6e20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f1d0c082000 tx=0x7f1d0c064040).stop 2021-04-03T03:14:14.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 shutdown_connections 2021-04-03T03:14:14.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1d040d5bb0 0x7f1d040e6e20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1cec056ea0 0x7f1cec059360 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1d0403aa70 0x7f1d0403ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 --2- 172.21.2.111:0/1112964977 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1d040a0010 0x7f1d040d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 >> 172.21.2.111:0/1112964977 conn(0x7f1d040462c0 msgr2=0x7f1d04045880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:14.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 shutdown_connections 2021-04-03T03:14:14.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.793+0000 7f1d125d4700 1 -- 172.21.2.111:0/1112964977 wait complete. 2021-04-03T03:14:14.813 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:14:14.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bffff700 1 Processor -- start 2021-04-03T03:14:14.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bffff700 1 -- start start 2021-04-03T03:14:14.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b8041060 0x7fe7b80dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b80dc560 con 0x7fe7b803d560 2021-04-03T03:14:14.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b80dc800 con 0x7fe7b80dbba0 2021-04-03T03:14:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b803de60 con 0x7fe7b8041060 2021-04-03T03:14:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bf7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe7b803d560 0x7fe7b8041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54178/0 (socket says 172.21.2.111:54178) 2021-04-03T03:14:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bf7fe700 1 -- 172.21.2.111:0/510179553 learned_addr learned my addr 172.21.2.111:0/510179553 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43006174 0 0) 0x7fe7b80dc560 con 0x7fe7b803d560 2021-04-03T03:14:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a0003240 con 0x7fe7b803d560 2021-04-03T03:14:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 174985139 0 0) 0x7fe7a0003240 con 0x7fe7b803d560 2021-04-03T03:14:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b8041060 msgr2=0x7fe7b80dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bdffb700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b8041060 0x7fe7b80dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.857+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 >> v1:172.21.2.115:6789/0 conn(0x7fe7b80dbba0 legacy=0x7fe7b803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b803da30 con 0x7fe7b803d560 2021-04-03T03:14:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe7c00450f0 con 0x7fe7b803d560 2021-04-03T03:14:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 >> v1:172.21.2.111:6789/0 conn(0x7fe7b803d560 legacy=0x7fe7b8041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b8041060 0x7fe7a0003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7a0003c50 0x7fe7a00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7a00049d0 0x7fe7a0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b803de60 con 0x7fe7a0003c50 2021-04-03T03:14:14.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0004140 con 0x7fe7a00049d0 2021-04-03T03:14:14.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0004300 con 0x7fe7b8041060 2021-04-03T03:14:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7a0003c50 0x7fe7a00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bf7fe700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7a00049d0 0x7fe7a0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b8041060 0x7fe7a0003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 -- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b8041060 msgr2=0x7fe7a0003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b8041060 0x7fe7a0003760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 -- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7a00049d0 msgr2=0x7fe7a0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7a00049d0 0x7fe7a0004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 -- 172.21.2.111:0/510179553 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0003240 con 0x7fe7a0003c50 2021-04-03T03:14:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7a0003c50 0x7fe7a00044e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fe7b0002f30 tx=0x7fe7b0006e10).ready entity=mon.0 client_cookie=619d104916bdbc79 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7b00097f0 con 0x7fe7a0003c50 2021-04-03T03:14:14.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0000e90 con 0x7fe7a0003c50 2021-04-03T03:14:14.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bdffb700 1 -- 172.21.2.111:0/510179553 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7b0009b40 con 0x7fe7a0003c50 2021-04-03T03:14:14.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- 172.21.2.111:0/510179553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7a0003c50 msgr2=0x7fe7a00044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 --2- 172.21.2.111:0/510179553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7a0003c50 0x7fe7a00044e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fe7b0002f30 tx=0x7fe7b0006e10).stop 2021-04-03T03:14:14.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bf7fe700 1 -- 172.21.2.111:0/510179553 reap_dead start 2021-04-03T03:14:14.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- 172.21.2.111:0/510179553 shutdown_connections 2021-04-03T03:14:14.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- 172.21.2.111:0/510179553 >> 172.21.2.111:0/510179553 conn(0x7fe7b80462b0 msgr2=0x7fe7b80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:14.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- 172.21.2.111:0/510179553 shutdown_connections 2021-04-03T03:14:14.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- 172.21.2.111:0/510179553 wait complete. 2021-04-03T03:14:14.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 Processor -- start 2021-04-03T03:14:14.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- start start 2021-04-03T03:14:14.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7b80d7810 0x7fe7b809fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b80a31e0 0x7fe7b80a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7b8039580 0x7fe7b80a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80d7bf0 con 0x7fe7b80d7810 2021-04-03T03:14:14.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b803ad30 con 0x7fe7b8039580 2021-04-03T03:14:14.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b803afd0 con 0x7fe7b80a31e0 2021-04-03T03:14:14.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7b80d7810 0x7fe7b809fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:14.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7b80d7810 0x7fe7b809fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53396/0 (socket says 172.21.2.111:53396) 2021-04-03T03:14:14.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 -- 172.21.2.111:0/1549270160 learned_addr learned my addr 172.21.2.111:0/1549270160 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:14.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bf7fe700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7b8039580 0x7fe7b80a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:14.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7be7fc700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b80a31e0 0x7fe7b80a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:14.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 -- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b80a31e0 msgr2=0x7fe7b80a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b80a31e0 0x7fe7b80a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 -- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7b8039580 msgr2=0x7fe7b80a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7b8039580 0x7fe7b80a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 -- 172.21.2.111:0/1549270160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b803b2a0 con 0x7fe7b80d7810 2021-04-03T03:14:14.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7beffd700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7b80d7810 0x7fe7b809fb40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fe7a8008000 tx=0x7fe7a8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:14.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7b7fff700 1 -- 172.21.2.111:0/1549270160 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7a800f510 con 0x7fe7b80d7810 2021-04-03T03:14:14.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7b7fff700 1 -- 172.21.2.111:0/1549270160 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a800f670 con 0x7fe7b80d7810 2021-04-03T03:14:14.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7b7fff700 1 -- 172.21.2.111:0/1549270160 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7a800f9d0 con 0x7fe7b80d7810 2021-04-03T03:14:14.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b803b540 con 0x7fe7b80d7810 2021-04-03T03:14:14.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.861+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b80dd970 con 0x7fe7b80d7810 2021-04-03T03:14:14.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.865+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b8004db0 con 0x7fe7b80d7810 2021-04-03T03:14:14.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.865+0000 7fe7b7fff700 1 -- 172.21.2.111:0/1549270160 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe7a801a030 con 0x7fe7b80d7810 2021-04-03T03:14:14.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.865+0000 7fe7b7fff700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7a4057000 0x7fe7a40594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:14.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.865+0000 7fe7b7fff700 1 -- 172.21.2.111:0/1549270160 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe7a806fa70 con 0x7fe7b80d7810 2021-04-03T03:14:14.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.865+0000 7fe7be7fc700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7a4057000 0x7fe7a40594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:14.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.865+0000 7fe7be7fc700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7a4057000 0x7fe7a40594c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fe7b0016000 tx=0x7fe7b0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:14.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.865+0000 7fe7b7fff700 1 -- 172.21.2.111:0/1549270160 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe7a8043c50 con 0x7fe7b80d7810 2021-04-03T03:14:14.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.957+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe7b803b6d0 con 0x7fe7b80d7810 2021-04-03T03:14:14.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.961+0000 7fe7b7fff700 1 -- 172.21.2.111:0/1549270160 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7a806fe60 con 0x7fe7b80d7810 2021-04-03T03:14:14.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7a4057000 msgr2=0x7fe7a40594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7a4057000 0x7fe7a40594c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fe7b0016000 tx=0x7fe7b0015000).stop 2021-04-03T03:14:14.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7b80d7810 msgr2=0x7fe7b809fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:14.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7b80d7810 0x7fe7b809fb40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fe7a8008000 tx=0x7fe7a8009040).stop 2021-04-03T03:14:14.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 shutdown_connections 2021-04-03T03:14:14.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7a4057000 0x7fe7a40594c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7b8039580 0x7fe7b80a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7b80d7810 0x7fe7b809fb40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 --2- 172.21.2.111:0/1549270160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7b80a31e0 0x7fe7b80a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 >> 172.21.2.111:0/1549270160 conn(0x7fe7b80462b0 msgr2=0x7fe7b80055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 shutdown_connections 2021-04-03T03:14:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:14.965+0000 7fe7bffff700 1 -- 172.21.2.111:0/1549270160 wait complete. 2021-04-03T03:14:14.983 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:14:14.984 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:14.990 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:14.997 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:15.010 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:15.022 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:14:15.034 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:14:15.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:15.034 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:15.035 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:15.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:14:15.056 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:14:15.056 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:14:15.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:15.057 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:15.058 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:15.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:14:15.067 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:14:15.067 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:14:15.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:15.068 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:15.069 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:14:15.079 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:14:15.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:14:15.081 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:14:15.081 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:14:15.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 Processor -- start 2021-04-03T03:14:15.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 -- start start 2021-04-03T03:14:15.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcdf40473f0 0x7fcdf40d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fcdf403dea0 0x7fcdf40420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40d06e0 con 0x7fcdf40d0e20 2021-04-03T03:14:15.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40d0980 con 0x7fcdf40473f0 2021-04-03T03:14:15.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40d0410 con 0x7fcdf403dea0 2021-04-03T03:14:15.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcdf40473f0 0x7fcdf40d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcdf40473f0 0x7fcdf40d4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53404/0 (socket says 172.21.2.111:53404) 2021-04-03T03:14:15.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 -- 172.21.2.111:0/1537513775 learned_addr learned my addr 172.21.2.111:0/1537513775 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:15.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 -- 172.21.2.111:0/1537513775 >> v2:172.21.2.115:3300/0 conn(0x7fcdf403dea0 msgr2=0x7fcdf40420c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:14:15.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 --2- 172.21.2.111:0/1537513775 >> v2:172.21.2.115:3300/0 conn(0x7fcdf403dea0 0x7fcdf40420c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 -- 172.21.2.111:0/1537513775 >> v1:172.21.2.115:6789/0 conn(0x7fcdf40d0e20 legacy=0x7fcdf403ce50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:14:15.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 -- 172.21.2.111:0/1537513775 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40425b0 con 0x7fcdf40473f0 2021-04-03T03:14:15.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfaffd700 1 --2- 172.21.2.111:0/1537513775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf40d4330 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fcdf403eaf0 tx=0x7fcdec006e90).ready entity=mon.0 client_cookie=92dd932fd6607e11 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdf9ffb700 1 -- 172.21.2.111:0/1537513775 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdec009930 con 0x7fcdf40473f0 2021-04-03T03:14:15.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdf9ffb700 1 -- 172.21.2.111:0/1537513775 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec009a90 con 0x7fcdf40473f0 2021-04-03T03:14:15.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdf9ffb700 1 -- 172.21.2.111:0/1537513775 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdec009e30 con 0x7fcdf40473f0 2021-04-03T03:14:15.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 -- 172.21.2.111:0/1537513775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 msgr2=0x7fcdf40d4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.129+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/1537513775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf40d4330 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fcdf403eaf0 tx=0x7fcdec006e90).stop 2021-04-03T03:14:15.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- 172.21.2.111:0/1537513775 shutdown_connections 2021-04-03T03:14:15.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/1537513775 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf40d4330 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:15.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/1537513775 >> v2:172.21.2.115:3300/0 conn(0x7fcdf403dea0 0x7fcdf40420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- 172.21.2.111:0/1537513775 >> 172.21.2.111:0/1537513775 conn(0x7fcdf400d9e0 msgr2=0x7fcdf4046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:15.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- 172.21.2.111:0/1537513775 shutdown_connections 2021-04-03T03:14:15.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- 172.21.2.111:0/1537513775 wait complete. 2021-04-03T03:14:15.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 Processor -- start 2021-04-03T03:14:15.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- start start 2021-04-03T03:14:15.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcdf403dea0 0x7fcdf4006f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf403bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcdf40d0e20 0x7fcdf403c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40ccb40 con 0x7fcdf40473f0 2021-04-03T03:14:15.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40c83a0 con 0x7fcdf403dea0 2021-04-03T03:14:15.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40c8640 con 0x7fcdf40d0e20 2021-04-03T03:14:15.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf403bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf403bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53414/0 (socket says 172.21.2.111:53414) 2021-04-03T03:14:15.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 -- 172.21.2.111:0/2844065189 learned_addr learned my addr 172.21.2.111:0/2844065189 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:15.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 -- 172.21.2.111:0/2844065189 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcdf40d0e20 msgr2=0x7fcdf403c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcdf40d0e20 0x7fcdf403c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 -- 172.21.2.111:0/2844065189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcdf403dea0 msgr2=0x7fcdf4006f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcdf403dea0 0x7fcdf4006f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 -- 172.21.2.111:0/2844065189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40cc7c0 con 0x7fcdf40473f0 2021-04-03T03:14:15.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfa7fc700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf403bf30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7fcdf0002f70 tx=0x7fcdf0006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdebfff700 1 -- 172.21.2.111:0/2844065189 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf000d570 con 0x7fcdf40473f0 2021-04-03T03:14:15.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdebfff700 1 -- 172.21.2.111:0/2844065189 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf000d6d0 con 0x7fcdf40473f0 2021-04-03T03:14:15.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdebfff700 1 -- 172.21.2.111:0/2844065189 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf000da30 con 0x7fcdf40473f0 2021-04-03T03:14:15.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf40c8c10 con 0x7fcdf40473f0 2021-04-03T03:14:15.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf40c9420 con 0x7fcdf40473f0 2021-04-03T03:14:15.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf40c8a00 con 0x7fcdf40473f0 2021-04-03T03:14:15.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdebfff700 1 -- 172.21.2.111:0/2844065189 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fcdf001a060 con 0x7fcdf40473f0 2021-04-03T03:14:15.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdebfff700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcdd8056ef0 0x7fcdd80593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdebfff700 1 -- 172.21.2.111:0/2844065189 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fcdf006fc10 con 0x7fcdf40473f0 2021-04-03T03:14:15.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfaffd700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcdd8056ef0 0x7fcdd80593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.133+0000 7fcdfaffd700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcdd8056ef0 0x7fcdd80593b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fcdf403eaf0 tx=0x7fcdec015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.137+0000 7fcdebfff700 1 -- 172.21.2.111:0/2844065189 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fcdf0044ef0 con 0x7fcdf40473f0 2021-04-03T03:14:15.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.229+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcdf40c8800 con 0x7fcdf40473f0 2021-04-03T03:14:15.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.229+0000 7fcdebfff700 1 -- 172.21.2.111:0/2844065189 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v64) v1 ==== 81+0+34040 (secure 0 0 0) 0x7fcdf0018020 con 0x7fcdf40473f0 2021-04-03T03:14:15.244 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:15.244 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:14:15.245 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 64, 2021-04-03T03:14:15.245 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:14:15.245 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:14:15.245 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:13:42.209640+0000", 2021-04-03T03:14:15.245 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:14:15.246 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:14:15.246 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:14:15.246 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:14:15.246 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:14:15.247 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:14:15.247 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:14:15.247 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:14:15.247 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:14:15.247 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:14:15.248 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:14:15.248 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:14:15.248 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:14:15.248 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:14:15.248 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:14:15.249 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 12, 2021-04-03T03:14:15.249 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:14:15.249 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:14:15.249 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:14:15.249 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:14:15.250 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:14:15.250 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.250 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:14:15.250 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:14:15.250 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:14:15.251 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:14:15.251 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:14:15.251 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:14:15.251 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:14:15.252 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:14:15.252 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:14:15.252 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:14:15.252 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:14:15.252 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:14:15.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:14:15.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:14:15.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:14:15.253 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:14:15.253 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:14:15.254 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:14:15.254 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:14:15.254 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:14:15.254 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:14:15.254 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:14:15.255 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:14:15.255 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.255 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:14:15.255 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:14:15.255 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:14:15.256 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:14:15.256 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:14:15.256 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:14:15.256 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:14:15.256 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:14:15.257 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:14:15.257 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:14:15.257 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:14:15.257 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:14:15.258 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:14:15.258 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:14:15.258 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:14:15.258 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:14:15.258 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:14:15.259 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:14:15.259 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:14:15.259 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:14:15.259 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:14:15.259 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:14:15.260 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:14:15.260 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:14:15.260 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:14:15.260 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:14:15.260 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:14:15.261 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.261 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:14:15.261 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:14:15.261 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:14:15.261 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:14:15.262 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:14:15.262 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:14:15.262 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:14:15.262 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:14:15.262 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:14:15.263 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:14:15.263 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:14:15.263 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:14:15.263 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:14:15.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.264 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:14:15.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:14:15.264 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.265 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.265 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:14:15.265 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:14:15.265 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:14:15.265 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:14:15.266 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:14:15.266 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:14:15.266 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:14:15.266 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:14:15.266 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:14:15.267 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:14:15.267 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:14:15.267 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:14:15.267 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:14:15.267 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:14:15.268 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:14:15.268 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:14:15.268 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:14:15.268 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:14:15.268 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:14:15.269 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:14:15.269 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:14:15.269 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:14:15.269 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:14:15.270 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.270 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:14:15.270 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:14:15.270 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:14:15.270 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:14:15.271 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:14:15.271 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:14:15.271 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:14:15.271 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:14:15.271 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:14:15.272 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:14:15.272 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:14:15.272 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:14:15.272 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:14:15.272 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:14:15.273 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:14:15.273 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:14:15.273 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:14:15.273 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:14:15.273 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:14:15.274 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:14:15.274 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:14:15.274 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:14:15.274 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:14:15.274 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:14:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:14:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:14:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:14:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.275 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:14:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:14:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:14:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:14:15.276 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:14:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:14:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:14:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:14:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:14:15.277 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:14:15.278 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:14:15.278 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:14:15.278 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:14:15.278 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:14:15.278 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.279 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.279 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.279 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 11, 2021-04-03T03:14:15.279 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:14:15.279 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:13:39.120055+0000", 2021-04-03T03:14:15.280 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:14:15.280 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:14:15.280 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:14:15.280 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:14:15.280 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:14:15.281 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:14:15.281 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:14:15.281 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:14:15.281 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:14:15.281 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:14:15.282 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:14:15.282 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:14:15.282 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:14:15.282 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:14:15.282 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:14:15.283 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:14:15.283 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:14:15.283 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:14:15.283 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:14:15.283 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:14:15.284 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.284 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "63", 2021-04-03T03:14:15.284 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:14:15.284 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:14:15.285 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:14:15.285 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:14:15.285 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:14:15.285 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:14:15.285 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:14:15.286 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:14:15.286 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:14:15.286 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:14:15.286 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:14:15.286 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:14:15.287 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:14:15.287 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:14:15.287 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:14:15.287 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:14:15.287 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:14:15.288 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:14:15.288 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:14:15.288 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:14:15.288 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:14:15.288 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:14:15.289 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:14:15.289 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:14:15.289 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:14:15.289 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:14:15.289 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.290 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:14:15.290 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:14:15.290 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:14:15.290 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:14:15.290 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:14:15.291 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:14:15.291 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:14:15.291 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:14:15.291 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:14:15.292 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:14:15.292 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:14:15.292 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:14:15.292 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:14:15.292 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:14:15.293 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:14:15.293 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.295 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:14:15.295 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:14:15.295 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:14:15.296 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.296 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.296 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.296 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.296 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 12, 2021-04-03T03:14:15.297 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:14:15.297 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:13:40.173978+0000", 2021-04-03T03:14:15.297 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:14:15.297 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:14:15.298 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:14:15.298 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:14:15.298 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:14:15.298 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:14:15.298 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:14:15.299 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:14:15.299 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:14:15.299 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:14:15.299 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:14:15.299 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:14:15.300 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:14:15.300 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:14:15.300 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:14:15.300 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:14:15.300 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:14:15.301 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:14:15.301 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:14:15.301 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:14:15.301 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.301 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "64", 2021-04-03T03:14:15.302 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:14:15.302 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:14:15.302 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:14:15.302 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:14:15.302 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:14:15.303 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:14:15.303 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:14:15.303 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:14:15.303 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:14:15.303 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:14:15.304 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:14:15.304 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:14:15.304 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:14:15.304 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:14:15.304 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:14:15.305 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:14:15.305 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:14:15.305 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:14:15.305 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:14:15.306 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:14:15.306 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:14:15.306 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:14:15.306 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:14:15.306 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:14:15.307 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:14:15.307 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:14:15.307 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.307 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:14:15.307 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:14:15.308 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:14:15.308 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:14:15.308 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:14:15.308 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:14:15.308 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:14:15.309 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:14:15.309 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:14:15.309 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:14:15.309 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:14:15.309 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:14:15.310 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:14:15.310 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:14:15.310 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:14:15.310 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.310 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.311 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.311 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:14:15.311 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:14:15.311 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.311 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:14:15.312 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:14:15.312 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.312 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.312 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.312 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.313 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.313 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.313 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.313 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 60, 2021-04-03T03:14:15.313 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.314 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.314 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.314 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.314 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.314 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.315 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:14:15.315 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.315 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.315 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.316 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.316 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:14:15.316 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.316 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.316 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.317 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.317 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.317 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.317 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.317 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.318 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:14:15.318 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.318 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.318 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.318 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.319 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:14:15.319 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.319 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.319 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.319 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.320 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.320 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.320 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.320 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.320 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:14:15.321 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.321 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.321 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.321 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.321 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:14:15.322 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.322 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.322 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.322 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.322 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.323 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.323 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.323 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.323 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:14:15.324 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.324 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.324 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.324 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.324 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:14:15.325 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:14:15.325 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.325 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.325 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.325 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:14:15.326 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:14:15.326 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:14:15.326 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:14:15.326 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.326 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.327 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.327 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.327 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.327 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.327 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:14:15.328 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:14:15.328 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.328 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.328 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.328 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.329 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.329 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.329 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.329 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 60, 2021-04-03T03:14:15.329 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.330 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.330 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.330 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.330 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.330 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.331 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:14:15.331 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.331 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.331 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.332 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.332 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:14:15.332 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.332 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.332 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.333 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.333 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.333 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.333 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.333 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.334 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:14:15.334 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.334 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.334 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.334 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.335 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:14:15.335 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.335 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.335 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.335 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.336 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.336 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.336 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.336 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.336 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:14:15.337 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.337 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.337 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.337 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.337 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:14:15.338 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.338 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.338 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.338 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.338 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.339 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.339 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.339 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.339 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:14:15.340 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.340 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.340 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.340 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.340 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:14:15.341 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:14:15.341 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.341 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.341 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.341 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:14:15.342 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:14:15.342 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:14:15.342 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:14:15.342 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.342 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.343 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.343 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.343 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.343 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.343 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:14:15.344 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:14:15.344 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.344 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.344 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.344 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.345 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.345 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.345 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.345 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 59, 2021-04-03T03:14:15.345 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.346 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.346 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.346 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.346 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.346 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.347 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:14:15.347 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.347 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.347 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.347 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.348 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:14:15.348 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.348 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.348 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.349 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.349 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.349 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.349 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.349 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.350 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:14:15.350 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.350 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.350 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.350 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:14:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.351 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.352 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:14:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.353 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:14:15.354 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.354 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.354 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.354 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.355 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.355 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.355 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.355 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.355 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:14:15.356 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.356 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.356 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.356 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.356 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:14:15.357 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:14:15.357 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.357 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.357 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.357 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:14:15.358 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:14:15.358 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:14:15.358 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:14:15.358 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.358 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.359 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.359 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.359 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.359 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.360 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:14:15.360 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:14:15.360 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.360 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.360 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.361 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.361 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.361 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.361 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.362 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 49, 2021-04-03T03:14:15.362 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.362 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.362 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.362 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.363 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.363 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.363 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:14:15.363 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.363 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.364 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.364 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.364 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:14:15.364 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.364 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.365 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.365 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.365 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.365 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.365 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.366 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.366 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:14:15.366 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.366 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.367 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.367 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.367 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:14:15.367 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.367 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.368 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.368 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.368 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.368 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.368 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.369 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.369 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:14:15.369 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.369 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.369 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.370 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.370 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:14:15.370 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.370 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.370 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.371 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.371 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.371 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.371 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.372 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:14:15.372 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.372 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.372 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.373 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.373 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:14:15.373 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:14:15.373 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.373 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.374 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.374 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:14:15.374 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:14:15.374 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:14:15.374 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:14:15.375 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.375 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.375 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.375 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.375 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.376 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.376 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:14:15.376 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:14:15.376 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.376 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.377 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.377 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.377 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.377 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.377 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.378 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 62, 2021-04-03T03:14:15.378 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.378 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.378 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.378 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.379 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.379 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.379 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:14:15.379 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.379 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.380 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.380 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.380 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:14:15.380 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.380 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.381 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.381 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.381 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.381 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.382 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.382 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.382 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:14:15.382 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.382 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.383 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.383 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.383 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:14:15.383 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.383 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.384 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.384 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.384 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.384 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.384 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.385 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.385 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:14:15.385 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.385 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.385 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.386 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.386 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:14:15.386 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.386 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.386 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.387 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.387 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.387 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.387 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.387 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.388 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:14:15.388 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.388 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.388 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.388 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.389 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:14:15.389 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:14:15.389 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.389 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.389 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.390 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:14:15.390 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:14:15.390 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:14:15.390 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:14:15.390 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.391 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.391 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.391 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.391 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.391 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.392 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:14:15.392 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:14:15.392 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.392 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.392 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.393 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 59, 2021-04-03T03:14:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.394 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:14:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.395 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.396 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.396 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:14:15.396 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.396 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.397 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.397 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.397 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.397 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.397 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.398 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.398 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:14:15.398 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.398 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.398 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.399 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.399 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:14:15.399 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.399 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.399 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.400 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.400 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.400 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.400 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.400 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.401 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:14:15.401 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.401 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.401 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.401 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.402 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:14:15.402 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.402 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.402 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.402 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.403 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.403 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.403 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.403 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.403 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:14:15.404 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.404 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.404 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.404 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.404 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:14:15.405 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:14:15.405 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.405 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.405 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.405 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:14:15.406 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:14:15.406 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:14:15.406 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:14:15.406 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.407 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.407 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.407 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.407 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.407 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.408 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:14:15.408 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:14:15.408 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.408 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.408 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.409 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.409 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.409 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.409 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.409 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 60, 2021-04-03T03:14:15.410 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.410 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.410 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.410 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.410 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.411 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.411 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:14:15.411 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.411 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.411 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.412 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.412 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:14:15.412 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.412 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.412 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.413 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.413 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.413 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.413 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.413 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.414 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:14:15.414 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.414 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.414 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.414 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.415 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:14:15.415 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.415 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.415 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.415 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.416 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.416 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.416 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.416 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.416 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:14:15.417 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.417 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.417 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.417 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.417 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:14:15.418 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.418 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.418 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.418 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.419 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.419 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.419 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.419 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.419 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:14:15.420 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.420 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.420 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.420 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:14:15.421 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:14:15.421 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.421 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.421 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.421 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:14:15.422 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:14:15.422 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:14:15.422 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:14:15.422 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.422 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.423 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.423 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.423 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.423 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.423 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:14:15.424 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:14:15.424 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:14:15.424 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:14:15.424 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:14:15.425 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:14:15.425 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:14:15.425 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:14:15.425 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:14:15.425 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 59, 2021-04-03T03:14:15.426 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:14:15.426 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:14:15.426 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:14:15.426 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.426 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.427 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.427 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:14:15.427 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.427 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.427 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.428 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.428 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:14:15.428 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.428 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.428 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.429 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.429 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:14:15.429 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.429 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.429 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.430 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:14:15.430 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.430 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.430 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.430 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.431 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:14:15.431 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.431 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.431 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.431 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.432 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:14:15.432 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.432 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.432 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.432 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:14:15.433 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.433 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.433 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.433 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.433 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:14:15.434 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.434 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.434 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.434 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.434 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:14:15.435 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:14:15.435 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.435 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:14:15.435 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:14:15.436 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.436 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.436 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.436 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:14:15.436 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:14:15.437 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:14:15.437 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.437 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.437 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.437 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:14:15.438 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:14:15.438 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:14:15.438 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:14:15.438 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:14:15.438 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:14:15.439 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:14:15.439 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:15.439 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.439 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:14:15.439 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:14:15.440 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.440 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:14:15.440 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.440 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.440 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.441 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.441 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.441 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:14:15.441 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.441 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.442 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.442 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:14:15.442 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.442 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.442 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.443 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.443 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.443 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:14:15.443 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.443 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.444 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.444 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:14:15.444 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.444 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.445 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.445 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.445 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.445 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:14:15.445 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.446 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.446 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.446 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:14:15.446 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.446 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.447 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.447 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.447 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.447 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:14:15.447 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.448 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.448 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.448 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:14:15.448 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.448 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.449 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.449 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.449 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.449 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:14:15.449 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.450 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.450 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.450 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:14:15.450 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.450 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.451 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.451 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.451 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.451 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:14:15.451 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.452 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.452 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.452 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:14:15.452 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.452 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.453 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.453 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.453 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.453 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:14:15.453 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.454 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.454 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:15.454 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:14:15.454 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:14:15.455 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:14:15.455 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:14:15.455 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:14:15.455 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:14:15.455 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:14:15.456 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:14:15.456 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.456 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:14:15.456 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:14:15.456 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:14:15.457 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:14:15.457 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:14:15.457 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:14:15.457 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:14:15.457 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:14:15.458 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:14:15.458 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:14:15.458 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:14:15.458 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:14:15.458 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:14:15.459 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:14:15.459 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:14:15.459 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:14:15.459 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:14:15.459 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:14:15.460 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.460 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:14:15.460 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:14:15.460 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:14:15.460 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:14:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:14:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:14:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:14:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:15.461 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:14:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:14:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:14:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:14:15.462 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:14:15.463 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:14:15.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcdd8056ef0 msgr2=0x7fcdd80593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcdd8056ef0 0x7fcdd80593b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fcdf403eaf0 tx=0x7fcdec015000).stop 2021-04-03T03:14:15.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 msgr2=0x7fcdf403bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf403bf30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7fcdf0002f70 tx=0x7fcdf0006dc0).stop 2021-04-03T03:14:15.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 shutdown_connections 2021-04-03T03:14:15.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcdf40d0e20 0x7fcdf403c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcdd8056ef0 0x7fcdd80593b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:15.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcdf403dea0 0x7fcdf4006f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 --2- 172.21.2.111:0/2844065189 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcdf40473f0 0x7fcdf403bf30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:15.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 >> 172.21.2.111:0/2844065189 conn(0x7fcdf400d9e0 msgr2=0x7fcdf40cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:15.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 shutdown_connections 2021-04-03T03:14:15.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.233+0000 7fcdfbfff700 1 -- 172.21.2.111:0/2844065189 wait complete. 2021-04-03T03:14:15.467 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:15.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.517+0000 7fc69bfff700 1 Processor -- start 2021-04-03T03:14:15.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.517+0000 7fc69bfff700 1 -- start start 2021-04-03T03:14:15.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0603f0 0x7fc69c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc69c0b5d60 0x7fc69c0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc69c0b44a0 0x7fc69c0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c0b4bc0 con 0x7fc69c0b5d60 2021-04-03T03:14:15.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c0607d0 con 0x7fc69c0b44a0 2021-04-03T03:14:15.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0b4df0 con 0x7fc69c0603f0 2021-04-03T03:14:15.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc69c0b5d60 0x7fc69c0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc69c0b5d60 0x7fc69c0b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53418/0 (socket says 172.21.2.111:53418) 2021-04-03T03:14:15.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 -- 172.21.2.111:0/1801429838 learned_addr learned my addr 172.21.2.111:0/1801429838 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:15.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 -- 172.21.2.111:0/1801429838 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0603f0 msgr2=0x7fc69c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:15.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 --2- 172.21.2.111:0/1801429838 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0603f0 0x7fc69c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 -- 172.21.2.111:0/1801429838 >> v2:172.21.2.115:3300/0 conn(0x7fc69c0b44a0 msgr2=0x7fc69c0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:15.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 --2- 172.21.2.111:0/1801429838 >> v2:172.21.2.115:3300/0 conn(0x7fc69c0b44a0 0x7fc69c0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 -- 172.21.2.111:0/1801429838 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0b5090 con 0x7fc69c0b5d60 2021-04-03T03:14:15.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69a7fc700 1 --2- 172.21.2.111:0/1801429838 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c0b2480 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fc684002f30 tx=0x7fc684006e90).ready entity=mon.0 client_cookie=dad7340be1daa06f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc699ffb700 1 -- 172.21.2.111:0/1801429838 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc684009920 con 0x7fc69c0b5d60 2021-04-03T03:14:15.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc699ffb700 1 -- 172.21.2.111:0/1801429838 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684009a80 con 0x7fc69c0b5d60 2021-04-03T03:14:15.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc699ffb700 1 -- 172.21.2.111:0/1801429838 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc684009de0 con 0x7fc69c0b5d60 2021-04-03T03:14:15.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- 172.21.2.111:0/1801429838 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 msgr2=0x7fc69c0b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- 172.21.2.111:0/1801429838 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c0b2480 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fc684002f30 tx=0x7fc684006e90).stop 2021-04-03T03:14:15.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- 172.21.2.111:0/1801429838 shutdown_connections 2021-04-03T03:14:15.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- 172.21.2.111:0/1801429838 >> v2:172.21.2.115:3300/0 conn(0x7fc69c0b44a0 0x7fc69c0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- 172.21.2.111:0/1801429838 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0603f0 0x7fc69c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- 172.21.2.111:0/1801429838 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c0b2480 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:15.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- 172.21.2.111:0/1801429838 >> 172.21.2.111:0/1801429838 conn(0x7fc69c05c7d0 msgr2=0x7fc69c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- 172.21.2.111:0/1801429838 shutdown_connections 2021-04-03T03:14:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- 172.21.2.111:0/1801429838 wait complete. 2021-04-03T03:14:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 Processor -- start 2021-04-03T03:14:15.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- start start 2021-04-03T03:14:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc69c0603f0 0x7fc69c0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0b44a0 0x7fc69c0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0b0620 con 0x7fc69c0b5d60 2021-04-03T03:14:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c153060 con 0x7fc69c0603f0 2021-04-03T03:14:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c153300 con 0x7fc69c0b44a0 2021-04-03T03:14:15.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53424/0 (socket says 172.21.2.111:53424) 2021-04-03T03:14:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 -- 172.21.2.111:0/252494827 learned_addr learned my addr 172.21.2.111:0/252494827 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 -- 172.21.2.111:0/252494827 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0b44a0 msgr2=0x7fc69c0afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0b44a0 0x7fc69c0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 -- 172.21.2.111:0/252494827 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc69c0603f0 msgr2=0x7fc69c0ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc69c0603f0 0x7fc69c0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 -- 172.21.2.111:0/252494827 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0ae050 con 0x7fc69c0b5d60 2021-04-03T03:14:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69b7fe700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c149040 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fc694003300 tx=0x7fc6940091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc693fff700 1 -- 172.21.2.111:0/252494827 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc69400c960 con 0x7fc69c0b5d60 2021-04-03T03:14:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc693fff700 1 -- 172.21.2.111:0/252494827 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69400cac0 con 0x7fc69c0b5d60 2021-04-03T03:14:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c0b0110 con 0x7fc69c0b5d60 2021-04-03T03:14:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc693fff700 1 -- 172.21.2.111:0/252494827 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc69400ce20 con 0x7fc69c0b5d60 2021-04-03T03:14:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.521+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c0af5a0 con 0x7fc69c0b5d60 2021-04-03T03:14:15.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.525+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c057660 con 0x7fc69c0b5d60 2021-04-03T03:14:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.525+0000 7fc693fff700 1 -- 172.21.2.111:0/252494827 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc69401c070 con 0x7fc69c0b5d60 2021-04-03T03:14:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.525+0000 7fc693fff700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc680057050 0x7fc680059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.525+0000 7fc693fff700 1 -- 172.21.2.111:0/252494827 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc69406fca0 con 0x7fc69c0b5d60 2021-04-03T03:14:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.525+0000 7fc69affd700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc680057050 0x7fc680059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.525+0000 7fc69affd700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc680057050 0x7fc680059510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fc68c002f30 tx=0x7fc68c006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.525+0000 7fc693fff700 1 -- 172.21.2.111:0/252494827 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc694074070 con 0x7fc69c0b5d60 2021-04-03T03:14:15.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc69c0b0830 con 0x7fc69c0b5d60 2021-04-03T03:14:15.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc693fff700 1 -- 172.21.2.111:0/252494827 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fc694044ff0 con 0x7fc69c0b5d60 2021-04-03T03:14:15.663 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:15.663 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":25,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:13:42.208611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/167825619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":167825619},{"type":"v1","addr":"172.21.2.111:6835","nonce":167825619}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:14:15.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc680057050 msgr2=0x7fc680059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc680057050 0x7fc680059510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fc68c002f30 tx=0x7fc68c006a60).stop 2021-04-03T03:14:15.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 msgr2=0x7fc69c149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c149040 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fc694003300 tx=0x7fc6940091a0).stop 2021-04-03T03:14:15.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 shutdown_connections 2021-04-03T03:14:15.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc69c0b5d60 0x7fc69c149040 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:15.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc680057050 0x7fc680059510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:15.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc69c0603f0 0x7fc69c0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 --2- 172.21.2.111:0/252494827 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc69c0b44a0 0x7fc69c0afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 >> 172.21.2.111:0/252494827 conn(0x7fc69c05c7d0 msgr2=0x7fc69c143700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:15.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 shutdown_connections 2021-04-03T03:14:15.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.649+0000 7fc69bfff700 1 -- 172.21.2.111:0/252494827 wait complete. 2021-04-03T03:14:15.667 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 25 2021-04-03T03:14:15.670 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:14:15.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 Processor -- start 2021-04-03T03:14:15.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 -- start start 2021-04-03T03:14:15.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f79d40603f0 0x7f79d41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79d40add50 0x7f79d4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40607d0 con 0x7f79d40b17b0 2021-04-03T03:14:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d41451c0 con 0x7f79d40add50 2021-04-03T03:14:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d414b090 con 0x7f79d40603f0 2021-04-03T03:14:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79d40add50 0x7f79d4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79d40add50 0x7f79d4144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53432/0 (socket says 172.21.2.111:53432) 2021-04-03T03:14:15.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 -- 172.21.2.111:0/3323726113 learned_addr learned my addr 172.21.2.111:0/3323726113 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:15.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 -- 172.21.2.111:0/3323726113 >> v2:172.21.2.115:3300/0 conn(0x7f79d40603f0 msgr2=0x7f79d41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:15.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 --2- 172.21.2.111:0/3323726113 >> v2:172.21.2.115:3300/0 conn(0x7f79d40603f0 0x7f79d41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 -- 172.21.2.111:0/3323726113 >> v1:172.21.2.115:6789/0 conn(0x7f79d40b17b0 legacy=0x7f79d4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:15.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 -- 172.21.2.111:0/3323726113 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d414b330 con 0x7f79d40add50 2021-04-03T03:14:15.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d1d9b700 1 --2- 172.21.2.111:0/3323726113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40add50 0x7f79d4144cd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f79d40ae520 tx=0x7f79bc006e90).ready entity=mon.0 client_cookie=ec4f4f716497cfd7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d159a700 1 -- 172.21.2.111:0/3323726113 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc009930 con 0x7f79d40add50 2021-04-03T03:14:15.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d159a700 1 -- 172.21.2.111:0/3323726113 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc009a90 con 0x7f79d40add50 2021-04-03T03:14:15.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d159a700 1 -- 172.21.2.111:0/3323726113 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc009e30 con 0x7f79d40add50 2021-04-03T03:14:15.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 -- 172.21.2.111:0/3323726113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40add50 msgr2=0x7f79d4144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 --2- 172.21.2.111:0/3323726113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40add50 0x7f79d4144cd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f79d40ae520 tx=0x7f79bc006e90).stop 2021-04-03T03:14:15.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 -- 172.21.2.111:0/3323726113 shutdown_connections 2021-04-03T03:14:15.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 --2- 172.21.2.111:0/3323726113 >> v2:172.21.2.115:3300/0 conn(0x7f79d40603f0 0x7f79d41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 --2- 172.21.2.111:0/3323726113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40add50 0x7f79d4144cd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:15.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.713+0000 7f79d359e700 1 -- 172.21.2.111:0/3323726113 >> 172.21.2.111:0/3323726113 conn(0x7f79d40b61c0 msgr2=0x7f79d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:15.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- 172.21.2.111:0/3323726113 shutdown_connections 2021-04-03T03:14:15.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- 172.21.2.111:0/3323726113 wait complete. 2021-04-03T03:14:15.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 Processor -- start 2021-04-03T03:14:15.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- start start 2021-04-03T03:14:15.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d40603f0 0x7f79d40ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d40add50 0x7f79d40b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40b17b0 0x7f79d40a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40659b0 con 0x7f79d40b17b0 2021-04-03T03:14:15.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4065c50 con 0x7f79d40add50 2021-04-03T03:14:15.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4065ef0 con 0x7f79d40603f0 2021-04-03T03:14:15.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40b17b0 0x7f79d40a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40b17b0 0x7f79d40a85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53438/0 (socket says 172.21.2.111:53438) 2021-04-03T03:14:15.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 -- 172.21.2.111:0/3127955617 learned_addr learned my addr 172.21.2.111:0/3127955617 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:15.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 -- 172.21.2.111:0/3127955617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d40603f0 msgr2=0x7f79d40ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d40603f0 0x7f79d40ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 -- 172.21.2.111:0/3127955617 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d40add50 msgr2=0x7f79d40b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:15.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d40add50 0x7f79d40b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:15.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 -- 172.21.2.111:0/3127955617 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4065660 con 0x7f79d40b17b0 2021-04-03T03:14:15.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d2d9d700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40b17b0 0x7f79d40a85b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f79cc003250 tx=0x7f79cc006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79cb7fe700 1 -- 172.21.2.111:0/3127955617 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79cc00d8f0 con 0x7f79d40b17b0 2021-04-03T03:14:15.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79cb7fe700 1 -- 172.21.2.111:0/3127955617 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc00da50 con 0x7f79d40b17b0 2021-04-03T03:14:15.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d415a060 con 0x7f79d40b17b0 2021-04-03T03:14:15.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d40abf90 con 0x7f79d40b17b0 2021-04-03T03:14:15.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79cb7fe700 1 -- 172.21.2.111:0/3127955617 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79cc00ddb0 con 0x7f79d40b17b0 2021-04-03T03:14:15.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d415a1f0 con 0x7f79d40b17b0 2021-04-03T03:14:15.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79cb7fe700 1 -- 172.21.2.111:0/3127955617 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f79cc01c060 con 0x7f79d40b17b0 2021-04-03T03:14:15.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79cb7fe700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79b8056f10 0x7f79b80593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:15.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79cb7fe700 1 -- 172.21.2.111:0/3127955617 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f79cc0700b0 con 0x7f79d40b17b0 2021-04-03T03:14:15.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.717+0000 7f79d259c700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79b8056f10 0x7f79b80593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:15.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.721+0000 7f79d259c700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79b8056f10 0x7f79b80593d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f79c4002f30 tx=0x7f79c4006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:15.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.721+0000 7f79cb7fe700 1 -- 172.21.2.111:0/3127955617 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f79cc0454a0 con 0x7f79d40b17b0 2021-04-03T03:14:15.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:15.845+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f79d4057660 con 0x7f79d40b17b0 2021-04-03T03:14:16.807 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:14:16.794+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:14:16.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.810+0000 7f79cb7fe700 1 -- 172.21.2.111:0/3127955617 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f79cc01a020 con 0x7f79d40b17b0 2021-04-03T03:14:16.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79b8056f10 msgr2=0x7f79b80593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:16.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79b8056f10 0x7f79b80593d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f79c4002f30 tx=0x7f79c4006a60).stop 2021-04-03T03:14:16.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40b17b0 msgr2=0x7f79d40a85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:16.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40b17b0 0x7f79d40a85b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f79cc003250 tx=0x7f79cc006f90).stop 2021-04-03T03:14:16.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 shutdown_connections 2021-04-03T03:14:16.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d40b17b0 0x7f79d40a85b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:16.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d40603f0 0x7f79d40ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:16.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79b8056f10 0x7f79b80593d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:16.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 --2- 172.21.2.111:0/3127955617 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d40add50 0x7f79d40b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:16.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 >> 172.21.2.111:0/3127955617 conn(0x7f79d40b61c0 msgr2=0x7f79d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:16.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 shutdown_connections 2021-04-03T03:14:16.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.814+0000 7f79d359e700 1 -- 172.21.2.111:0/3127955617 wait complete. 2021-04-03T03:14:16.828 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:14:16.832 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:16.844 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:14:16.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.878+0000 7f003f59e700 1 Processor -- start 2021-04-03T03:14:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.878+0000 7f003f59e700 1 -- start start 2021-04-03T03:14:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.878+0000 7f003f59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0038006bc0 0x7f0038042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.878+0000 7f003f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f003803d360 0x7f00380dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:16.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.878+0000 7f003f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00380dc6e0 con 0x7f00380dbb90 2021-04-03T03:14:16.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.878+0000 7f003f59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00380d9560 con 0x7f0038006bc0 2021-04-03T03:14:16.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.878+0000 7f003f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003803dcb0 con 0x7f003803d360 2021-04-03T03:14:16.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0038006bc0 0x7f0038042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:16.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0038006bc0 0x7f0038042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:43174/0 (socket says 172.21.2.111:43174) 2021-04-03T03:14:16.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 -- 172.21.2.111:0/311396489 learned_addr learned my addr 172.21.2.111:0/311396489 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:16.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003ed9d700 1 --2- 172.21.2.111:0/311396489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f003803d360 0x7f00380dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003ed9d700 1 -- 172.21.2.111:0/311396489 >> v2:172.21.2.115:3300/0 conn(0x7f0038006bc0 msgr2=0x7f0038042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003ed9d700 1 --2- 172.21.2.111:0/311396489 >> v2:172.21.2.115:3300/0 conn(0x7f0038006bc0 0x7f0038042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003ed9d700 1 -- 172.21.2.111:0/311396489 >> v1:172.21.2.115:6789/0 conn(0x7f00380dbb90 legacy=0x7f00380064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003ed9d700 1 -- 172.21.2.111:0/311396489 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380dc460 con 0x7f003803d360 2021-04-03T03:14:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003ed9d700 1 --2- 172.21.2.111:0/311396489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f003803d360 0x7f00380dbf70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f004006b5d0 tx=0x7f004005ad70).ready entity=mon.2 client_cookie=db0df20574e43746 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003d59a700 1 -- 172.21.2.111:0/311396489 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f004005bda0 con 0x7f003803d360 2021-04-03T03:14:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003d59a700 1 -- 172.21.2.111:0/311396489 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004005df60 con 0x7f003803d360 2021-04-03T03:14:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003d59a700 1 -- 172.21.2.111:0/311396489 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00400699f0 con 0x7f003803d360 2021-04-03T03:14:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- 172.21.2.111:0/311396489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f003803d360 msgr2=0x7f00380dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:16.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 --2- 172.21.2.111:0/311396489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f003803d360 0x7f00380dbf70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f004006b5d0 tx=0x7f004005ad70).stop 2021-04-03T03:14:16.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- 172.21.2.111:0/311396489 shutdown_connections 2021-04-03T03:14:16.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 --2- 172.21.2.111:0/311396489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f003803d360 0x7f00380dbf70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:16.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 --2- 172.21.2.111:0/311396489 >> v2:172.21.2.115:3300/0 conn(0x7f0038006bc0 0x7f0038042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- 172.21.2.111:0/311396489 >> 172.21.2.111:0/311396489 conn(0x7f00380462c0 msgr2=0x7f00380466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- 172.21.2.111:0/311396489 shutdown_connections 2021-04-03T03:14:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- 172.21.2.111:0/311396489 wait complete. 2021-04-03T03:14:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 Processor -- start 2021-04-03T03:14:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- start start 2021-04-03T03:14:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f003803d360 0x7f003803c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00380dbb90 0x7f00380d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0038039ad0 0x7f00380d3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00380d5180 con 0x7f00380dbb90 2021-04-03T03:14:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003803c820 con 0x7f003803d360 2021-04-03T03:14:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00380d55e0 con 0x7f0038039ad0 2021-04-03T03:14:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00380dbb90 0x7f00380d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00380dbb90 0x7f00380d38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53472/0 (socket says 172.21.2.111:53472) 2021-04-03T03:14:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 -- 172.21.2.111:0/1399325493 learned_addr learned my addr 172.21.2.111:0/1399325493 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003ed9d700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0038039ad0 0x7f00380d3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003e59c700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f003803d360 0x7f003803c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 -- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0038039ad0 msgr2=0x7f00380d3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0038039ad0 0x7f00380d3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 -- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f003803d360 msgr2=0x7f003803c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:16.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f003803d360 0x7f003803c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:16.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 -- 172.21.2.111:0/1399325493 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003803ca80 con 0x7f00380dbb90 2021-04-03T03:14:16.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003dd9b700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00380dbb90 0x7f00380d38d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f003000c000 tx=0x7f0030006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:16.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f002f7fe700 1 -- 172.21.2.111:0/1399325493 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f003000c820 con 0x7f00380dbb90 2021-04-03T03:14:16.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f002f7fe700 1 -- 172.21.2.111:0/1399325493 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003000c980 con 0x7f00380dbb90 2021-04-03T03:14:16.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00380d42b0 con 0x7f00380dbb90 2021-04-03T03:14:16.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f002f7fe700 1 -- 172.21.2.111:0/1399325493 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f003000cce0 con 0x7f00380dbb90 2021-04-03T03:14:16.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.882+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00380d4c40 con 0x7f00380dbb90 2021-04-03T03:14:16.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.886+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00380d5340 con 0x7f00380dbb90 2021-04-03T03:14:16.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.886+0000 7f002f7fe700 1 -- 172.21.2.111:0/1399325493 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f00300123f0 con 0x7f00380dbb90 2021-04-03T03:14:16.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.886+0000 7f002f7fe700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f001c056fb0 0x7f001c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:16.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.886+0000 7f002f7fe700 1 -- 172.21.2.111:0/1399325493 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f00300703c0 con 0x7f00380dbb90 2021-04-03T03:14:16.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.886+0000 7f003e59c700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f001c056fb0 0x7f001c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:16.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.886+0000 7f003e59c700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f001c056fb0 0x7f001c059470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f003400d000 tx=0x7f0034007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:16.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:16.886+0000 7f002f7fe700 1 -- 172.21.2.111:0/1399325493 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f00380d5340 con 0x7f00380dbb90 2021-04-03T03:14:17.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00380d5340 con 0x7f00380dbb90 2021-04-03T03:14:17.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f002f7fe700 1 -- 172.21.2.111:0/1399325493 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f00380d5340 con 0x7f00380dbb90 2021-04-03T03:14:17.023 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:17.023 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2476725252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2476725252},{"type":"v1","addr":"172.21.2.115:6833","nonce":2476725252}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:13:41.205623+0000","modified":"2021-04-03T03:14:16.797767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-04-03T03:14:17.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f001c056fb0 msgr2=0x7f001c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f001c056fb0 0x7f001c059470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f003400d000 tx=0x7f0034007b70).stop 2021-04-03T03:14:17.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00380dbb90 msgr2=0x7f00380d38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00380dbb90 0x7f00380d38d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f003000c000 tx=0x7f0030006e90).stop 2021-04-03T03:14:17.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 shutdown_connections 2021-04-03T03:14:17.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0038039ad0 0x7f00380d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f001c056fb0 0x7f001c059470 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f003803d360 0x7f003803c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 --2- 172.21.2.111:0/1399325493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00380dbb90 0x7f00380d38d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 >> 172.21.2.111:0/1399325493 conn(0x7f00380462c0 msgr2=0x7f00380055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:17.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 shutdown_connections 2021-04-03T03:14:17.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.010+0000 7f003f59e700 1 -- 172.21.2.111:0/1399325493 wait complete. 2021-04-03T03:14:17.027 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 26 2021-04-03T03:14:17.030 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:14:17.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 Processor -- start 2021-04-03T03:14:17.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- start start 2021-04-03T03:14:17.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c0add50 0x7fe55c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe55c144ef0 0x7fe55c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c0607d0 con 0x7fe55c0603f0 2021-04-03T03:14:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c14b090 con 0x7fe55c144ef0 2021-04-03T03:14:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c14b330 con 0x7fe55c0add50 2021-04-03T03:14:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55ad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe55c144ef0 0x7fe55c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55a59c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe55c0603f0 0x7fe55c1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54268/0 (socket says 172.21.2.111:54268) 2021-04-03T03:14:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55a59c700 1 -- 172.21.2.111:0/2894854637 learned_addr learned my addr 172.21.2.111:0/2894854637 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55ad9d700 1 -- 172.21.2.111:0/2894854637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c0add50 msgr2=0x7fe55c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55ad9d700 1 --2- 172.21.2.111:0/2894854637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c0add50 0x7fe55c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55ad9d700 1 -- 172.21.2.111:0/2894854637 >> v1:172.21.2.111:6789/0 conn(0x7fe55c0603f0 legacy=0x7fe55c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55ad9d700 1 -- 172.21.2.111:0/2894854637 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c14b5d0 con 0x7fe55c144ef0 2021-04-03T03:14:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55ad9d700 1 --2- 172.21.2.111:0/2894854637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c144ef0 0x7fe55c147c30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fe54c0032b0 tx=0x7fe54c0091d0).ready entity=mon.0 client_cookie=d485fd2dff4fb45 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55959a700 1 -- 172.21.2.111:0/2894854637 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe54c00ba50 con 0x7fe55c144ef0 2021-04-03T03:14:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55959a700 1 -- 172.21.2.111:0/2894854637 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c00bbb0 con 0x7fe55c144ef0 2021-04-03T03:14:17.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55959a700 1 -- 172.21.2.111:0/2894854637 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe54c00bd20 con 0x7fe55c144ef0 2021-04-03T03:14:17.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- 172.21.2.111:0/2894854637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c144ef0 msgr2=0x7fe55c147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- 172.21.2.111:0/2894854637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c144ef0 0x7fe55c147c30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fe54c0032b0 tx=0x7fe54c0091d0).stop 2021-04-03T03:14:17.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- 172.21.2.111:0/2894854637 shutdown_connections 2021-04-03T03:14:17.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- 172.21.2.111:0/2894854637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c144ef0 0x7fe55c147c30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- 172.21.2.111:0/2894854637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c0add50 0x7fe55c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- 172.21.2.111:0/2894854637 >> 172.21.2.111:0/2894854637 conn(0x7fe55c0b61c0 msgr2=0x7fe55c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- 172.21.2.111:0/2894854637 shutdown_connections 2021-04-03T03:14:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- 172.21.2.111:0/2894854637 wait complete. 2021-04-03T03:14:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 Processor -- start 2021-04-03T03:14:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- start start 2021-04-03T03:14:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe55c0603f0 0x7fe55c063d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c0add50 0x7fe55c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c144ef0 0x7fe55c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c143760 con 0x7fe55c0add50 2021-04-03T03:14:17.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c1573a0 con 0x7fe55c0603f0 2021-04-03T03:14:17.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.134+0000 7fe55b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0645f0 con 0x7fe55c144ef0 2021-04-03T03:14:17.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c0add50 0x7fe55c1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c0add50 0x7fe55c1569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53484/0 (socket says 172.21.2.111:53484) 2021-04-03T03:14:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 -- 172.21.2.111:0/926327799 learned_addr learned my addr 172.21.2.111:0/926327799 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 -- 172.21.2.111:0/926327799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c144ef0 msgr2=0x7fe55c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c144ef0 0x7fe55c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 -- 172.21.2.111:0/926327799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe55c0603f0 msgr2=0x7fe55c063d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe55c0603f0 0x7fe55c063d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 -- 172.21.2.111:0/926327799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c157600 con 0x7fe55c0add50 2021-04-03T03:14:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe559d9b700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c0add50 0x7fe55c1569c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe554002f30 tx=0x7fe554006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe5537fe700 1 -- 172.21.2.111:0/926327799 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55400d710 con 0x7fe55c0add50 2021-04-03T03:14:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe5537fe700 1 -- 172.21.2.111:0/926327799 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554000e90 con 0x7fe55c0add50 2021-04-03T03:14:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe5537fe700 1 -- 172.21.2.111:0/926327799 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55400da60 con 0x7fe55c0add50 2021-04-03T03:14:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c0a9630 con 0x7fe55c0add50 2021-04-03T03:14:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c064390 con 0x7fe55c0add50 2021-04-03T03:14:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c057660 con 0x7fe55c0add50 2021-04-03T03:14:17.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe5537fe700 1 -- 172.21.2.111:0/926327799 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe55400dbc0 con 0x7fe55c0add50 2021-04-03T03:14:17.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe5537fe700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe540056f70 0x7fe540059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe5537fe700 1 -- 172.21.2.111:0/926327799 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fe55406fe70 con 0x7fe55c0add50 2021-04-03T03:14:17.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe55a59c700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe540056f70 0x7fe540059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:17.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.138+0000 7fe55a59c700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe540056f70 0x7fe540059430 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe55c0b5600 tx=0x7fe5440077b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:17.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.142+0000 7fe5537fe700 1 -- 172.21.2.111:0/926327799 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe554045090 con 0x7fe55c0add50 2021-04-03T03:14:17.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.266+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe55c148850 con 0x7fe55c0add50 2021-04-03T03:14:17.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe5537fe700 1 -- 172.21.2.111:0/926327799 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe5540486b0 con 0x7fe55c0add50 2021-04-03T03:14:17.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe540056f70 msgr2=0x7fe540059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe540056f70 0x7fe540059430 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe55c0b5600 tx=0x7fe5440077b0).stop 2021-04-03T03:14:17.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c0add50 msgr2=0x7fe55c1569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c0add50 0x7fe55c1569c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe554002f30 tx=0x7fe554006e70).stop 2021-04-03T03:14:17.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 shutdown_connections 2021-04-03T03:14:17.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe55c144ef0 0x7fe55c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe540056f70 0x7fe540059430 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe55c0603f0 0x7fe55c063d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 --2- 172.21.2.111:0/926327799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe55c0add50 0x7fe55c1569c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 >> 172.21.2.111:0/926327799 conn(0x7fe55c0b61c0 msgr2=0x7fe55c144120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:17.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 shutdown_connections 2021-04-03T03:14:17.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.814+0000 7fe55b59e700 1 -- 172.21.2.111:0/926327799 wait complete. 2021-04-03T03:14:17.833 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:14:17.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.878+0000 7f02d759e700 1 Processor -- start 2021-04-03T03:14:17.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.878+0000 7f02d759e700 1 -- start start 2021-04-03T03:14:17.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.878+0000 7f02d759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f02d80603f0 0x7f02d81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.878+0000 7f02d759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80add50 0x7f02d8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.878+0000 7f02d759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d80607d0 con 0x7f02d80b17b0 2021-04-03T03:14:17.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.878+0000 7f02d759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d81451c0 con 0x7f02d80603f0 2021-04-03T03:14:17.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.878+0000 7f02d759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d814b090 con 0x7f02d80add50 2021-04-03T03:14:17.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d659c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f02d80603f0 0x7f02d81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:17.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80add50 0x7f02d8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:17.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d659c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f02d80603f0 0x7f02d81447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:43200/0 (socket says 172.21.2.111:43200) 2021-04-03T03:14:17.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d659c700 1 -- 172.21.2.111:0/486440893 learned_addr learned my addr 172.21.2.111:0/486440893 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:17.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 -- 172.21.2.111:0/486440893 >> v2:172.21.2.115:3300/0 conn(0x7f02d80603f0 msgr2=0x7f02d81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:17.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 --2- 172.21.2.111:0/486440893 >> v2:172.21.2.115:3300/0 conn(0x7f02d80603f0 0x7f02d81447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 -- 172.21.2.111:0/486440893 >> v1:172.21.2.115:6789/0 conn(0x7f02d80b17b0 legacy=0x7f02d8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 -- 172.21.2.111:0/486440893 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d814b330 con 0x7f02d80add50 2021-04-03T03:14:17.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 --2- 172.21.2.111:0/486440893 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80add50 0x7f02d8144cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f02c0002f30 tx=0x7f02c0006dd0).ready entity=mon.2 client_cookie=4f1a1aac39331174 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:17.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d559a700 1 -- 172.21.2.111:0/486440893 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c0009760 con 0x7f02d80add50 2021-04-03T03:14:17.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d559a700 1 -- 172.21.2.111:0/486440893 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c00098c0 con 0x7f02d80add50 2021-04-03T03:14:17.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d559a700 1 -- 172.21.2.111:0/486440893 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c0009c20 con 0x7f02d80add50 2021-04-03T03:14:17.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- 172.21.2.111:0/486440893 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80add50 msgr2=0x7f02d8144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 --2- 172.21.2.111:0/486440893 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80add50 0x7f02d8144cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f02c0002f30 tx=0x7f02c0006dd0).stop 2021-04-03T03:14:17.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- 172.21.2.111:0/486440893 shutdown_connections 2021-04-03T03:14:17.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 --2- 172.21.2.111:0/486440893 >> v2:172.21.2.115:3300/0 conn(0x7f02d80603f0 0x7f02d81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 --2- 172.21.2.111:0/486440893 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80add50 0x7f02d8144cd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:17.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- 172.21.2.111:0/486440893 >> 172.21.2.111:0/486440893 conn(0x7f02d80b61c0 msgr2=0x7f02d805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:17.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- 172.21.2.111:0/486440893 shutdown_connections 2021-04-03T03:14:17.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- 172.21.2.111:0/486440893 wait complete. 2021-04-03T03:14:17.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 Processor -- start 2021-04-03T03:14:17.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- start start 2021-04-03T03:14:17.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02d80603f0 0x7f02d80abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02d80add50 0x7f02d80ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80b17b0 0x7f02d8064910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80aa2c0 con 0x7f02d80add50 2021-04-03T03:14:17.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80aa560 con 0x7f02d80603f0 2021-04-03T03:14:17.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80ab560 con 0x7f02d80b17b0 2021-04-03T03:14:17.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02d80add50 0x7f02d80ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:17.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02d80add50 0x7f02d80ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53498/0 (socket says 172.21.2.111:53498) 2021-04-03T03:14:17.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.882+0000 7f02d5d9b700 1 -- 172.21.2.111:0/1849489151 learned_addr learned my addr 172.21.2.111:0/1849489151 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:17.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d5d9b700 1 -- 172.21.2.111:0/1849489151 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80b17b0 msgr2=0x7f02d8064910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d5d9b700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80b17b0 0x7f02d8064910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d5d9b700 1 -- 172.21.2.111:0/1849489151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02d80603f0 msgr2=0x7f02d80abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:17.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d5d9b700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02d80603f0 0x7f02d80abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:17.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d5d9b700 1 -- 172.21.2.111:0/1849489151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80679f0 con 0x7f02d80add50 2021-04-03T03:14:17.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d5d9b700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02d80add50 0x7f02d80ab070 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f02c0002f30 tx=0x7f02c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:17.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02cf7fe700 1 -- 172.21.2.111:0/1849489151 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c0021070 con 0x7f02d80add50 2021-04-03T03:14:17.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02cf7fe700 1 -- 172.21.2.111:0/1849489151 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c0011e80 con 0x7f02d80add50 2021-04-03T03:14:17.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02cf7fe700 1 -- 172.21.2.111:0/1849489151 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c001c7b0 con 0x7f02d80add50 2021-04-03T03:14:17.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d8067c40 con 0x7f02d80add50 2021-04-03T03:14:17.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d8065010 con 0x7f02d80add50 2021-04-03T03:14:17.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d80aa720 con 0x7f02d80add50 2021-04-03T03:14:17.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02cf7fe700 1 -- 172.21.2.111:0/1849489151 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f02c000f070 con 0x7f02d80add50 2021-04-03T03:14:17.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02cf7fe700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f02bc056f20 0x7f02bc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:17.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02cf7fe700 1 -- 172.21.2.111:0/1849489151 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f02c0071680 con 0x7f02d80add50 2021-04-03T03:14:17.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d659c700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f02bc056f20 0x7f02bc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:17.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.886+0000 7f02d659c700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f02bc056f20 0x7f02bc0593e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f02c800c000 tx=0x7f02c8006e90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:17.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.890+0000 7f02cf7fe700 1 -- 172.21.2.111:0/1849489151 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f02c0046a10 con 0x7f02d80add50 2021-04-03T03:14:17.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:17.978+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f02d8057660 con 0x7f02d80add50 2021-04-03T03:14:18.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02cf7fe700 1 -- 172.21.2.111:0/1849489151 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 185+0+0 (secure 0 0 0) 0x7f02c004a030 con 0x7f02d80add50 2021-04-03T03:14:18.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f02bc056f20 msgr2=0x7f02bc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:18.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f02bc056f20 0x7f02bc0593e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f02c800c000 tx=0x7f02c8006e90).stop 2021-04-03T03:14:18.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02d80add50 msgr2=0x7f02d80ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:18.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02d80add50 0x7f02d80ab070 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f02c0002f30 tx=0x7f02c0015000).stop 2021-04-03T03:14:18.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 shutdown_connections 2021-04-03T03:14:18.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02d80b17b0 0x7f02d8064910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:18.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f02bc056f20 0x7f02bc0593e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:18.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02d80603f0 0x7f02d80abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:18.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 --2- 172.21.2.111:0/1849489151 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02d80add50 0x7f02d80ab070 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:18.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 >> 172.21.2.111:0/1849489151 conn(0x7f02d80b61c0 msgr2=0x7f02d81447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:18.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 shutdown_connections 2021-04-03T03:14:18.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.814+0000 7f02d759e700 1 -- 172.21.2.111:0/1849489151 wait complete. 2021-04-03T03:14:18.834 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:14:18.838 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:14:18.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d8bc700 1 Processor -- start 2021-04-03T03:14:18.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d8bc700 1 -- start start 2021-04-03T03:14:18.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d8bc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f54180603f0 0x7f5418144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d8bc700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54180b0be0 con 0x7f54180b0290 2021-04-03T03:14:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d8bc700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54180607d0 con 0x7f54180b17f0 2021-04-03T03:14:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d8bc700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5418145300 con 0x7f54180603f0 2021-04-03T03:14:18.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d0bb700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f54180b0290 0x7f54181400e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54298/0 (socket says 172.21.2.111:54298) 2021-04-03T03:14:18.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541d0bb700 1 -- 172.21.2.111:0/4109704783 learned_addr learned my addr 172.21.2.111:0/4109704783 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:18.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958547341 0 0) 0x7f54180b0be0 con 0x7f54180b0290 2021-04-03T03:14:18.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f8003240 con 0x7f54180b0290 2021-04-03T03:14:18.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f541c8ba700 1 --2- 172.21.2.111:0/4109704783 >> v2:172.21.2.115:3300/0 conn(0x7f54180603f0 0x7f5418144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:18.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1190421669 0 0) 0x7f53f8003240 con 0x7f54180b0290 2021-04-03T03:14:18.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 >> v2:172.21.2.115:3300/0 conn(0x7f54180603f0 msgr2=0x7f5418144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 --2- 172.21.2.111:0/4109704783 >> v2:172.21.2.115:3300/0 conn(0x7f54180603f0 0x7f5418144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 >> v1:172.21.2.115:6789/0 conn(0x7f54180b17f0 legacy=0x7f5418144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54180b0e10 con 0x7f54180b0290 2021-04-03T03:14:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.882+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f5410001a50 con 0x7f54180b0290 2021-04-03T03:14:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 >> v1:172.21.2.111:6789/0 conn(0x7f54180b0290 legacy=0x7f54181400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f8003800 0x7f53f8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53f80044e0 0x7f53f8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f8004e50 0x7f53f800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418145300 con 0x7f53f8004e50 2021-04-03T03:14:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8005360 con 0x7f53f80044e0 2021-04-03T03:14:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f800bc50 con 0x7f53f8003800 2021-04-03T03:14:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f8004e50 0x7f53f800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/4109704783 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f8003800 msgr2=0x7f53f8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f8003800 0x7f53f8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/4109704783 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53f80044e0 msgr2=0x7f53f8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53f80044e0 0x7f53f8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/4109704783 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8003240 con 0x7f53f8004e50 2021-04-03T03:14:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/4109704783 reap_dead start 2021-04-03T03:14:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f8004e50 0x7f53f800b730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f5410001760 tx=0x7f54180b0fa0).ready entity=mon.0 client_cookie=bd5c39dcee471d1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5410005e40 con 0x7f53f8004e50 2021-04-03T03:14:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5410007ad0 con 0x7f53f8004e50 2021-04-03T03:14:18.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f54177fe700 1 -- 172.21.2.111:0/4109704783 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54100072c0 con 0x7f53f8004e50 2021-04-03T03:14:18.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/4109704783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f8004e50 msgr2=0x7f53f800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:18.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f8004e50 0x7f53f800b730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f5410001760 tx=0x7f54180b0fa0).stop 2021-04-03T03:14:18.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/4109704783 shutdown_connections 2021-04-03T03:14:18.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 --2- 172.21.2.111:0/4109704783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f8004e50 0x7f53f800b730 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:18.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/4109704783 >> 172.21.2.111:0/4109704783 conn(0x7f54180b61c0 msgr2=0x7f541805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:18.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/4109704783 shutdown_connections 2021-04-03T03:14:18.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/4109704783 wait complete. 2021-04-03T03:14:18.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 Processor -- start 2021-04-03T03:14:18.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- start start 2021-04-03T03:14:18.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f8004e50 0x7f5418141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f541813cb00 0x7f54181511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54180ace60 0x7f5418139860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541813cf80 con 0x7f54180ace60 2021-04-03T03:14:18.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418140c20 con 0x7f541813cb00 2021-04-03T03:14:18.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541813d330 con 0x7f53f8004e50 2021-04-03T03:14:18.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54180ace60 0x7f5418139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:18.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54180ace60 0x7f5418139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53522/0 (socket says 172.21.2.111:53522) 2021-04-03T03:14:18.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/1476222014 learned_addr learned my addr 172.21.2.111:0/1476222014 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:18.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f5417fff700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f541813cb00 0x7f54181511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:18.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/1476222014 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f8004e50 msgr2=0x7f5418141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:18.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f8004e50 0x7f5418141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:18.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/1476222014 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f541813cb00 msgr2=0x7f54181511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:18.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f541813cb00 0x7f54181511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:18.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 -- 172.21.2.111:0/1476222014 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541813d5d0 con 0x7f54180ace60 2021-04-03T03:14:18.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d0bb700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54180ace60 0x7f5418139860 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f5410001760 tx=0x7f5410017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:18.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f5415ffb700 1 -- 172.21.2.111:0/1476222014 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5410007ad0 con 0x7f54180ace60 2021-04-03T03:14:18.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f5415ffb700 1 -- 172.21.2.111:0/1476222014 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5410001a70 con 0x7f54180ace60 2021-04-03T03:14:18.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f5415ffb700 1 -- 172.21.2.111:0/1476222014 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5410011960 con 0x7f54180ace60 2021-04-03T03:14:18.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54181408a0 con 0x7f54180ace60 2021-04-03T03:14:18.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54181516b0 con 0x7f54180ace60 2021-04-03T03:14:18.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.886+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54180ad600 con 0x7f54180ace60 2021-04-03T03:14:18.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.890+0000 7f5415ffb700 1 -- 172.21.2.111:0/1476222014 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f541002d070 con 0x7f54180ace60 2021-04-03T03:14:18.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.890+0000 7f5415ffb700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53fc056ef0 0x7f53fc0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:18.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.890+0000 7f541c8ba700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53fc056ef0 0x7f53fc0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:18.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.890+0000 7f5415ffb700 1 -- 172.21.2.111:0/1476222014 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6064+0+0 (secure 0 0 0) 0x7f5410075750 con 0x7f54180ace60 2021-04-03T03:14:18.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.890+0000 7f541c8ba700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53fc056ef0 0x7f53fc0593b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f540800c000 tx=0x7f5408006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:18.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.890+0000 7f5415ffb700 1 -- 172.21.2.111:0/1476222014 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f541004ab90 con 0x7f54180ace60 2021-04-03T03:14:18.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:18.982+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f541813d760 con 0x7f54180ace60 2021-04-03T03:14:19.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.827+0000 7f5415ffb700 1 -- 172.21.2.111:0/1476222014 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 173+0+0 (secure 0 0 0) 0x7f541004e1b0 con 0x7f54180ace60 2021-04-03T03:14:19.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53fc056ef0 msgr2=0x7f53fc0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:19.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53fc056ef0 0x7f53fc0593b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f540800c000 tx=0x7f5408006dd0).stop 2021-04-03T03:14:19.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54180ace60 msgr2=0x7f5418139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:19.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54180ace60 0x7f5418139860 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f5410001760 tx=0x7f5410017000).stop 2021-04-03T03:14:19.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 shutdown_connections 2021-04-03T03:14:19.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54180ace60 0x7f5418139860 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f8004e50 0x7f5418141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:19.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53fc056ef0 0x7f53fc0593b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 --2- 172.21.2.111:0/1476222014 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f541813cb00 0x7f54181511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 >> 172.21.2.111:0/1476222014 conn(0x7f54180b61c0 msgr2=0x7f541813ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:19.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 shutdown_connections 2021-04-03T03:14:19.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.831+0000 7f541d8bc700 1 -- 172.21.2.111:0/1476222014 wait complete. 2021-04-03T03:14:19.848 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:14:19.852 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:14:19.854 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:14:19.854 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:14:19.854 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:14:19.854 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:14:19.855 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:14:19.855 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:14:19.855 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:14:19.861 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:14:19.861 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:14:19.865 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:14:19.865 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:14:19.906 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:14:19.906 DEBUG:teuthology.parallel:result is None 2021-04-03T03:14:19.908 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:14:19.909 DEBUG:teuthology.parallel:result is None 2021-04-03T03:14:19.909 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:14:19.944 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:14:19.946 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:14:19.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.955+0000 7fa2c9839700 1 Processor -- start 2021-04-03T03:14:19.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.955+0000 7fa2c9839700 1 -- start start 2021-04-03T03:14:19.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.955+0000 7fa2c9839700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:19.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.955+0000 7fa2c9839700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa2bc0431a0 0x7fa2bc0d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:19.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.955+0000 7fa2c9839700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc0d5c50 con 0x7fa2bc0cb960 2021-04-03T03:14:19.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.955+0000 7fa2c9839700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc044e40 con 0x7fa2bc0431a0 2021-04-03T03:14:19.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.955+0000 7fa2c9839700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc0d5ef0 con 0x7fa2bc0d51e0 2021-04-03T03:14:19.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa2bc0431a0 0x7fa2bc0d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:19.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa2bc0431a0 0x7fa2bc0d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:43248/0 (socket says 172.21.2.111:43248) 2021-04-03T03:14:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 -- 172.21.2.111:0/805153170 learned_addr learned my addr 172.21.2.111:0/805153170 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9038700 1 --1- 172.21.2.111:0/805153170 >> v1:172.21.2.115:6789/0 conn(0x7fa2bc0cb960 0x7fa2bc0d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:38636/0 (socket says 172.21.2.111:38636) 2021-04-03T03:14:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c8837700 1 --2- 172.21.2.111:0/805153170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc006600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c8837700 1 -- 172.21.2.111:0/805153170 >> v2:172.21.2.115:3300/0 conn(0x7fa2bc0431a0 msgr2=0x7fa2bc0d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c8837700 1 --2- 172.21.2.111:0/805153170 >> v2:172.21.2.115:3300/0 conn(0x7fa2bc0431a0 0x7fa2bc0d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c8837700 1 -- 172.21.2.111:0/805153170 >> v1:172.21.2.115:6789/0 conn(0x7fa2bc0cb960 legacy=0x7fa2bc0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c8837700 1 -- 172.21.2.111:0/805153170 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0cbdd0 con 0x7fa2bc0d51e0 2021-04-03T03:14:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c8837700 1 --2- 172.21.2.111:0/805153170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc006600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa2b8002f30 tx=0x7fa2b8006d70).ready entity=mon.2 client_cookie=bed827eb46ebdd86 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c37fe700 1 -- 172.21.2.111:0/805153170 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b8009640 con 0x7fa2bc0d51e0 2021-04-03T03:14:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c37fe700 1 -- 172.21.2.111:0/805153170 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b80097a0 con 0x7fa2bc0d51e0 2021-04-03T03:14:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c37fe700 1 -- 172.21.2.111:0/805153170 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b8009910 con 0x7fa2bc0d51e0 2021-04-03T03:14:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/805153170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 msgr2=0x7fa2bc006600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 --2- 172.21.2.111:0/805153170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc006600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa2b8002f30 tx=0x7fa2b8006d70).stop 2021-04-03T03:14:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/805153170 shutdown_connections 2021-04-03T03:14:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 --2- 172.21.2.111:0/805153170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc006600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 --2- 172.21.2.111:0/805153170 >> v2:172.21.2.115:3300/0 conn(0x7fa2bc0431a0 0x7fa2bc0d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/805153170 >> 172.21.2.111:0/805153170 conn(0x7fa2bc00e820 msgr2=0x7fa2bc00ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/805153170 shutdown_connections 2021-04-03T03:14:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/805153170 wait complete. 2021-04-03T03:14:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 Processor -- start 2021-04-03T03:14:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- start start 2021-04-03T03:14:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2bc0431a0 0x7fa2bc0e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2bc0cb960 0x7fa2bc0d7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:19.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc0cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:19.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc041fc0 con 0x7fa2bc0cb960 2021-04-03T03:14:19.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc041320 con 0x7fa2bc0431a0 2021-04-03T03:14:19.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc0cd3e0 con 0x7fa2bc0d51e0 2021-04-03T03:14:19.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2bc0cb960 0x7fa2bc0d7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:19.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2bc0cb960 0x7fa2bc0d7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53546/0 (socket says 172.21.2.111:53546) 2021-04-03T03:14:19.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 -- 172.21.2.111:0/4081443922 learned_addr learned my addr 172.21.2.111:0/4081443922 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:19.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c8837700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2bc0431a0 0x7fa2bc0e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:19.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9038700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc0cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:19.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 -- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 msgr2=0x7fa2bc0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:19.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc0cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 -- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2bc0431a0 msgr2=0x7fa2bc0e41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:19.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2bc0431a0 0x7fa2bc0e41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:19.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 -- 172.21.2.111:0/4081443922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc041c40 con 0x7fa2bc0cb960 2021-04-03T03:14:19.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9038700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc0cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:14:19.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c3fff700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2bc0cb960 0x7fa2bc0d7550 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fa2b400c000 tx=0x7fa2b4006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:19.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c1ffb700 1 -- 172.21.2.111:0/4081443922 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b400c820 con 0x7fa2bc0cb960 2021-04-03T03:14:19.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c1ffb700 1 -- 172.21.2.111:0/4081443922 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b400c980 con 0x7fa2bc0cb960 2021-04-03T03:14:19.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc0cc530 con 0x7fa2bc0cb960 2021-04-03T03:14:19.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c1ffb700 1 -- 172.21.2.111:0/4081443922 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b400cce0 con 0x7fa2bc0cb960 2021-04-03T03:14:19.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc0cb490 con 0x7fa2bc0cb960 2021-04-03T03:14:19.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.959+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc0414e0 con 0x7fa2bc0cb960 2021-04-03T03:14:19.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.963+0000 7fa2c1ffb700 1 -- 172.21.2.111:0/4081443922 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa2b401b400 con 0x7fa2bc0cb960 2021-04-03T03:14:19.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.963+0000 7fa2c1ffb700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa2a4056f10 0x7fa2a40593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:19.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.963+0000 7fa2c1ffb700 1 -- 172.21.2.111:0/4081443922 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fa2b40703f0 con 0x7fa2bc0cb960 2021-04-03T03:14:19.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.963+0000 7fa2c8837700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa2a4056f10 0x7fa2a40593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:19.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.963+0000 7fa2c8837700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa2a4056f10 0x7fa2a40593d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fa2b801b000 tx=0x7fa2b8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:19.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:19.963+0000 7fa2c1ffb700 1 -- 172.21.2.111:0/4081443922 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa2b4045940 con 0x7fa2bc0cb960 2021-04-03T03:14:20.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.067+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa2bc004d30 con 0x7fa2bc0cb960 2021-04-03T03:14:20.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c1ffb700 1 -- 172.21.2.111:0/4081443922 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa2b4048f60 con 0x7fa2bc0cb960 2021-04-03T03:14:20.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa2a4056f10 msgr2=0x7fa2a40593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:20.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa2a4056f10 0x7fa2a40593d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fa2b801b000 tx=0x7fa2b8016000).stop 2021-04-03T03:14:20.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2bc0cb960 msgr2=0x7fa2bc0d7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:20.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2bc0cb960 0x7fa2bc0d7550 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fa2b400c000 tx=0x7fa2b4006e90).stop 2021-04-03T03:14:20.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 shutdown_connections 2021-04-03T03:14:20.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa2bc0d51e0 0x7fa2bc0cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:20.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa2a4056f10 0x7fa2a40593d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:20.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa2bc0431a0 0x7fa2bc0e41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:20.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 --2- 172.21.2.111:0/4081443922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa2bc0cb960 0x7fa2bc0d7550 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:20.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 >> 172.21.2.111:0/4081443922 conn(0x7fa2bc00e820 msgr2=0x7fa2bc046ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:20.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 shutdown_connections 2021-04-03T03:14:20.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.843+0000 7fa2c9839700 1 -- 172.21.2.111:0/4081443922 wait complete. 2021-04-03T03:14:20.860 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:14:20.864 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:14:20.865 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:14:20.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e99a0700 1 Processor -- start 2021-04-03T03:14:20.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e99a0700 1 -- start start 2021-04-03T03:14:20.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e99a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc006bc0 0x7f20dc042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:20.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e99a0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f20dc03d360 0x7f20dc0dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:20.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e99a0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20dc0dc6e0 con 0x7f20dc0dbb90 2021-04-03T03:14:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e99a0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20dc0d9560 con 0x7f20dc03d360 2021-04-03T03:14:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e99a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc03dcb0 con 0x7f20dc006bc0 2021-04-03T03:14:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e899e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f20dc0dbb90 0x7f20dc0064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54346/0 (socket says 172.21.2.111:54346) 2021-04-03T03:14:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e899e700 1 -- 172.21.2.111:0/2505894436 learned_addr learned my addr 172.21.2.111:0/2505894436 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e919f700 1 --2- 172.21.2.111:0/2505894436 >> v2:172.21.2.115:3300/0 conn(0x7f20dc03d360 0x7f20dc0dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e3fff700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc006bc0 0x7f20dc042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665630315 0 0) 0x7f20dc0dc6e0 con 0x7f20dc0dbb90 2021-04-03T03:14:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c8003240 con 0x7f20dc0dbb90 2021-04-03T03:14:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1961134275 0 0) 0x7f20c8003240 con 0x7f20dc0dbb90 2021-04-03T03:14:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc006bc0 msgr2=0x7f20dc042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc006bc0 0x7f20dc042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 >> v2:172.21.2.115:3300/0 conn(0x7f20dc03d360 msgr2=0x7f20dc0dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 --2- 172.21.2.111:0/2505894436 >> v2:172.21.2.115:3300/0 conn(0x7f20dc03d360 0x7f20dc0dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc0dc460 con 0x7f20dc0dbb90 2021-04-03T03:14:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f20d00016d0 con 0x7f20dc0dbb90 2021-04-03T03:14:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 >> v1:172.21.2.111:6789/0 conn(0x7f20dc0dbb90 legacy=0x7f20dc0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20c80037b0 0x7f20c8003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20c80044e0 0x7f20c8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20c8004e50 0x7f20c800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:20.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0d9560 con 0x7f20c8004e50 2021-04-03T03:14:20.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8004200 con 0x7f20c80044e0 2021-04-03T03:14:20.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c800bc20 con 0x7f20c80037b0 2021-04-03T03:14:20.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.975+0000 7f20e919f700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20c8004e50 0x7f20c800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e919f700 1 -- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20c80037b0 msgr2=0x7f20c8003b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e919f700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20c80037b0 0x7f20c8003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e919f700 1 -- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20c80044e0 msgr2=0x7f20c8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e919f700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20c80044e0 0x7f20c8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e919f700 1 -- 172.21.2.111:0/2505894436 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c8003240 con 0x7f20c8004e50 2021-04-03T03:14:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e919f700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20c8004e50 0x7f20c800b730 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f20e4082000 tx=0x7f20e405ad70).ready entity=mon.0 client_cookie=87e0b9d4c60fcbd8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20e405df60 con 0x7f20c8004e50 2021-04-03T03:14:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e40699f0 con 0x7f20c8004e50 2021-04-03T03:14:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e37fe700 1 -- 172.21.2.111:0/2505894436 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20e406b610 con 0x7f20c8004e50 2021-04-03T03:14:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/2505894436 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20c8004e50 msgr2=0x7f20c800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 --2- 172.21.2.111:0/2505894436 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20c8004e50 0x7f20c800b730 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f20e4082000 tx=0x7f20e405ad70).stop 2021-04-03T03:14:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e919f700 1 -- 172.21.2.111:0/2505894436 reap_dead start 2021-04-03T03:14:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/2505894436 shutdown_connections 2021-04-03T03:14:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/2505894436 >> 172.21.2.111:0/2505894436 conn(0x7f20dc0462c0 msgr2=0x7f20dc0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/2505894436 shutdown_connections 2021-04-03T03:14:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/2505894436 wait complete. 2021-04-03T03:14:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 Processor -- start 2021-04-03T03:14:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- start start 2021-04-03T03:14:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20dc0456a0 0x7f20dc0a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20dc03c5b0 0x7f20dc0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc0d5590 0x7f20dc0d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0a2f70 con 0x7f20dc03c5b0 2021-04-03T03:14:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0a0800 con 0x7f20dc0456a0 2021-04-03T03:14:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0a0100 con 0x7f20dc0d5590 2021-04-03T03:14:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20dc03c5b0 0x7f20dc0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20dc03c5b0 0x7f20dc0a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53566/0 (socket says 172.21.2.111:53566) 2021-04-03T03:14:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 -- 172.21.2.111:0/1408058844 learned_addr learned my addr 172.21.2.111:0/1408058844 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e899e700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20dc0456a0 0x7f20dc0a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 -- 172.21.2.111:0/1408058844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc0d5590 msgr2=0x7f20dc0d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc0d5590 0x7f20dc0d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 -- 172.21.2.111:0/1408058844 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20dc0456a0 msgr2=0x7f20dc0a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20dc0456a0 0x7f20dc0a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 -- 172.21.2.111:0/1408058844 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc045a80 con 0x7f20dc03c5b0 2021-04-03T03:14:21.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e3fff700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20dc03c5b0 0x7f20dc0a09e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f20d800c000 tx=0x7f20d8006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e1ffb700 1 -- 172.21.2.111:0/1408058844 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20d800ca10 con 0x7f20dc03c5b0 2021-04-03T03:14:21.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e1ffb700 1 -- 172.21.2.111:0/1408058844 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d800cb70 con 0x7f20dc03c5b0 2021-04-03T03:14:21.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc09fe60 con 0x7f20dc03c5b0 2021-04-03T03:14:21.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc0a0ed0 con 0x7f20dc03c5b0 2021-04-03T03:14:21.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e1ffb700 1 -- 172.21.2.111:0/1408058844 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20d8013450 con 0x7f20dc03c5b0 2021-04-03T03:14:21.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.979+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc004da0 con 0x7f20dc03c5b0 2021-04-03T03:14:21.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.983+0000 7f20e1ffb700 1 -- 172.21.2.111:0/1408058844 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f20d80135b0 con 0x7f20dc03c5b0 2021-04-03T03:14:21.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.983+0000 7f20e1ffb700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20cc056f70 0x7f20cc059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.983+0000 7f20e1ffb700 1 -- 172.21.2.111:0/1408058844 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f20d80700f0 con 0x7f20dc03c5b0 2021-04-03T03:14:21.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.983+0000 7f20e899e700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20cc056f70 0x7f20cc059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.983+0000 7f20e899e700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20cc056f70 0x7f20cc059430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f20d0008000 tx=0x7f20d0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:20.983+0000 7f20e1ffb700 1 -- 172.21.2.111:0/1408058844 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f20d80456a0 con 0x7f20dc03c5b0 2021-04-03T03:14:21.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f20dc046ff0 con 0x7f20dc03c5b0 2021-04-03T03:14:21.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e1ffb700 1 -- 172.21.2.111:0/1408058844 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f20d8048cc0 con 0x7f20dc03c5b0 2021-04-03T03:14:21.126 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:21.127 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:14:21.127 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:14:21.127 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.128 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:14:21.128 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:14:21.129 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.129 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:14:21.129 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.130 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.130 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.131 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.131 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.131 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.132 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:14:21.132 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:14:21.133 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.133 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:14:21.134 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.134 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.134 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.135 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.135 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.136 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.136 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:14:21.136 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:14:21.137 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.137 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.138 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.138 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.138 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.139 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.139 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.140 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:14:21.140 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:14:21.141 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.141 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.141 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.142 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.142 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.143 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.143 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.143 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:14:21.144 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:14:21.144 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.145 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.146 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.146 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.147 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.147 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:14:21.147 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:14:21.148 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.148 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.149 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.149 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.150 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.150 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.150 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.151 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:14:21.151 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:14:21.152 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.152 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.152 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.153 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.153 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.154 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.154 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.154 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:14:21.155 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:14:21.155 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.156 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.156 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.156 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.157 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.157 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.158 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.158 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:14:21.158 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:14:21.159 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.159 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.160 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.160 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.161 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.161 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.161 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.162 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:14:21.162 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:14:21.163 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.163 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.163 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.164 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.164 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.165 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.165 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.165 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:14:21.166 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:14:21.166 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.167 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:14:21.167 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:14:21.168 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:14:21.168 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.168 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.169 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.169 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:14:21.170 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:14:21.170 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.170 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:14:21.170 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:14:21.171 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:14:21.171 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.171 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.171 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.172 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:14:21.172 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:14:21.172 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.172 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:14:21.172 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:14:21.173 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:14:21.173 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.173 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.173 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.174 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:14:21.174 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:14:21.174 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.174 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:14:21.175 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:14:21.175 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:14:21.175 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.175 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.175 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.176 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.176 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:14:21.176 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:14:21.176 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.177 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:14:21.177 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:14:21.177 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.177 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.178 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.178 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.178 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:14:21.178 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:14:21.178 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:14:21.179 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:14:21.179 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:14:21.179 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:14:21.179 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.180 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.180 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:14:21.180 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:14:21.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20cc056f70 msgr2=0x7f20cc059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20cc056f70 0x7f20cc059430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f20d0008000 tx=0x7f20d0009040).stop 2021-04-03T03:14:21.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20dc03c5b0 msgr2=0x7f20dc0a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20dc03c5b0 0x7f20dc0a09e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f20d800c000 tx=0x7f20d8006dd0).stop 2021-04-03T03:14:21.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 shutdown_connections 2021-04-03T03:14:21.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f20dc0d5590 0x7f20dc0d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f20cc056f70 0x7f20cc059430 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f20dc0456a0 0x7f20dc0a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 --2- 172.21.2.111:0/1408058844 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f20dc03c5b0 0x7f20dc0a09e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 >> 172.21.2.111:0/1408058844 conn(0x7f20dc0462c0 msgr2=0x7f20dc03d840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 shutdown_connections 2021-04-03T03:14:21.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.111+0000 7f20e99a0700 1 -- 172.21.2.111:0/1408058844 wait complete. 2021-04-03T03:14:21.185 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:14:21.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.247+0000 7f2272e48700 1 Processor -- start 2021-04-03T03:14:21.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.247+0000 7f2272e48700 1 -- start start 2021-04-03T03:14:21.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.247+0000 7f2272e48700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 0x7f226c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.247+0000 7f2272e48700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f226c0b0c90 0x7f226c147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.247+0000 7f2272e48700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0607d0 con 0x7f226c14b700 2021-04-03T03:14:21.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.247+0000 7f2272e48700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c0af400 con 0x7f226c0b0c90 2021-04-03T03:14:21.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.247+0000 7f2272e48700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0af6a0 con 0x7f226c0603f0 2021-04-03T03:14:21.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f226c0b0c90 0x7f226c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f226c0b0c90 0x7f226c147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53572/0 (socket says 172.21.2.111:53572) 2021-04-03T03:14:21.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 -- 172.21.2.111:0/1701058834 learned_addr learned my addr 172.21.2.111:0/1701058834 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 -- 172.21.2.111:0/1701058834 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 msgr2=0x7f226c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 --2- 172.21.2.111:0/1701058834 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 0x7f226c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 -- 172.21.2.111:0/1701058834 >> v1:172.21.2.115:6789/0 conn(0x7f226c14b700 legacy=0x7f226c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 -- 172.21.2.111:0/1701058834 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0afb40 con 0x7f226c0b0c90 2021-04-03T03:14:21.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272647700 1 --2- 172.21.2.111:0/1701058834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c147680 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f226c0628d0 tx=0x7f2264009270).ready entity=mon.0 client_cookie=5c3b43d5fcd98310 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2270e44700 1 -- 172.21.2.111:0/1701058834 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f226400bbc0 con 0x7f226c0b0c90 2021-04-03T03:14:21.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2270e44700 1 -- 172.21.2.111:0/1701058834 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400bd20 con 0x7f226c0b0c90 2021-04-03T03:14:21.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2270e44700 1 -- 172.21.2.111:0/1701058834 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2264011400 con 0x7f226c0b0c90 2021-04-03T03:14:21.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- 172.21.2.111:0/1701058834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 msgr2=0x7f226c147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 --2- 172.21.2.111:0/1701058834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c147680 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f226c0628d0 tx=0x7f2264009270).stop 2021-04-03T03:14:21.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- 172.21.2.111:0/1701058834 shutdown_connections 2021-04-03T03:14:21.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 --2- 172.21.2.111:0/1701058834 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c147680 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 --2- 172.21.2.111:0/1701058834 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 0x7f226c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- 172.21.2.111:0/1701058834 >> 172.21.2.111:0/1701058834 conn(0x7f226c0b61c0 msgr2=0x7f226c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- 172.21.2.111:0/1701058834 shutdown_connections 2021-04-03T03:14:21.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- 172.21.2.111:0/1701058834 wait complete. 2021-04-03T03:14:21.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 Processor -- start 2021-04-03T03:14:21.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- start start 2021-04-03T03:14:21.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 0x7f226c0ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f226c14b700 0x7f226c156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c063470 con 0x7f226c0b0c90 2021-04-03T03:14:21.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0aa600 con 0x7f226c14b700 2021-04-03T03:14:21.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0aa8a0 con 0x7f226c0603f0 2021-04-03T03:14:21.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53576/0 (socket says 172.21.2.111:53576) 2021-04-03T03:14:21.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 -- 172.21.2.111:0/3825150102 learned_addr learned my addr 172.21.2.111:0/3825150102 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 -- 172.21.2.111:0/3825150102 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 msgr2=0x7f226c0ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 0x7f226c0ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 -- 172.21.2.111:0/3825150102 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f226c14b700 msgr2=0x7f226c156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f226c14b700 0x7f226c156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 -- 172.21.2.111:0/3825150102 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0aab40 con 0x7f226c0b0c90 2021-04-03T03:14:21.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2271645700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c156840 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f2268002ef0 tx=0x7f2268006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f225effd700 1 -- 172.21.2.111:0/3825150102 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f226800f510 con 0x7f226c0b0c90 2021-04-03T03:14:21.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f225effd700 1 -- 172.21.2.111:0/3825150102 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226800f670 con 0x7f226c0b0c90 2021-04-03T03:14:21.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f225effd700 1 -- 172.21.2.111:0/3825150102 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f226800f7e0 con 0x7f226c0b0c90 2021-04-03T03:14:21.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c157700 con 0x7f226c0b0c90 2021-04-03T03:14:21.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.251+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c0a7eb0 con 0x7f226c0b0c90 2021-04-03T03:14:21.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.255+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c057660 con 0x7f226c0b0c90 2021-04-03T03:14:21.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.255+0000 7f225effd700 1 -- 172.21.2.111:0/3825150102 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f226800f070 con 0x7f226c0b0c90 2021-04-03T03:14:21.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.255+0000 7f225effd700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f224c056ea0 0x7f224c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.255+0000 7f225effd700 1 -- 172.21.2.111:0/3825150102 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f226806f780 con 0x7f226c0b0c90 2021-04-03T03:14:21.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.255+0000 7f2271e46700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f224c056ea0 0x7f224c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.255+0000 7f2271e46700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f224c056ea0 0x7f224c059360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f2260002f30 tx=0x7f2260006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.255+0000 7f225effd700 1 -- 172.21.2.111:0/3825150102 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2268044ea0 con 0x7f226c0b0c90 2021-04-03T03:14:21.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.347+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f226c0686d0 con 0x7f226c0b0c90 2021-04-03T03:14:21.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.347+0000 7f225effd700 1 -- 172.21.2.111:0/3825150102 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f22680484c0 con 0x7f226c0b0c90 2021-04-03T03:14:21.363 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:21.363 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":68,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:14:20.834573+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:14:21.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f224c056ea0 msgr2=0x7f224c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f224c056ea0 0x7f224c059360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f2260002f30 tx=0x7f2260006a60).stop 2021-04-03T03:14:21.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 msgr2=0x7f226c156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c156840 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f2268002ef0 tx=0x7f2268006d40).stop 2021-04-03T03:14:21.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 shutdown_connections 2021-04-03T03:14:21.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f224c056ea0 0x7f224c059360 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f226c14b700 0x7f226c156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f226c0603f0 0x7f226c0ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 --2- 172.21.2.111:0/3825150102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f226c0b0c90 0x7f226c156840 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 >> 172.21.2.111:0/3825150102 conn(0x7f226c0b61c0 msgr2=0x7f226c146c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 shutdown_connections 2021-04-03T03:14:21.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.351+0000 7f2272e48700 1 -- 172.21.2.111:0/3825150102 wait complete. 2021-04-03T03:14:21.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:14:21.371 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:14:21.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 Processor -- start 2021-04-03T03:14:21.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- start start 2021-04-03T03:14:21.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ef40603f0 0x7f7ef41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7ef40add50 0x7f7ef4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40b17b0 0x7f7ef4147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef41451c0 con 0x7f7ef40603f0 2021-04-03T03:14:21.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef40607d0 con 0x7f7ef40add50 2021-04-03T03:14:21.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef414b090 con 0x7f7ef40b17b0 2021-04-03T03:14:21.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ef40603f0 0x7f7ef41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ef40603f0 0x7f7ef41447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53584/0 (socket says 172.21.2.111:53584) 2021-04-03T03:14:21.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 -- 172.21.2.111:0/2280813348 learned_addr learned my addr 172.21.2.111:0/2280813348 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef27fc700 1 --2- 172.21.2.111:0/2280813348 >> v2:172.21.2.115:3300/0 conn(0x7f7ef40add50 0x7f7ef4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef37fe700 1 --2- 172.21.2.111:0/2280813348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40b17b0 0x7f7ef4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 -- 172.21.2.111:0/2280813348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40b17b0 msgr2=0x7f7ef4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 --2- 172.21.2.111:0/2280813348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40b17b0 0x7f7ef4147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 -- 172.21.2.111:0/2280813348 >> v2:172.21.2.115:3300/0 conn(0x7f7ef40add50 msgr2=0x7f7ef4144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 --2- 172.21.2.111:0/2280813348 >> v2:172.21.2.115:3300/0 conn(0x7f7ef40add50 0x7f7ef4144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 -- 172.21.2.111:0/2280813348 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef414b330 con 0x7f7ef40603f0 2021-04-03T03:14:21.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef2ffd700 1 --2- 172.21.2.111:0/2280813348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40603f0 0x7f7ef41447e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f7edc002f30 tx=0x7f7edc006e90).ready entity=mon.0 client_cookie=7cbf8b5e93b1aeab server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef1ffb700 1 -- 172.21.2.111:0/2280813348 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7edc009730 con 0x7f7ef40603f0 2021-04-03T03:14:21.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef1ffb700 1 -- 172.21.2.111:0/2280813348 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc009890 con 0x7f7ef40603f0 2021-04-03T03:14:21.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef1ffb700 1 -- 172.21.2.111:0/2280813348 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7edc009bf0 con 0x7f7ef40603f0 2021-04-03T03:14:21.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- 172.21.2.111:0/2280813348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40603f0 msgr2=0x7f7ef41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/2280813348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40603f0 0x7f7ef41447e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f7edc002f30 tx=0x7f7edc006e90).stop 2021-04-03T03:14:21.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- 172.21.2.111:0/2280813348 shutdown_connections 2021-04-03T03:14:21.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/2280813348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40b17b0 0x7f7ef4147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/2280813348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40603f0 0x7f7ef41447e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/2280813348 >> v2:172.21.2.115:3300/0 conn(0x7f7ef40add50 0x7f7ef4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- 172.21.2.111:0/2280813348 >> 172.21.2.111:0/2280813348 conn(0x7f7ef40b61c0 msgr2=0x7f7ef405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- 172.21.2.111:0/2280813348 shutdown_connections 2021-04-03T03:14:21.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.415+0000 7f7ef3fff700 1 -- 172.21.2.111:0/2280813348 wait complete. 2021-04-03T03:14:21.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 Processor -- start 2021-04-03T03:14:21.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 -- start start 2021-04-03T03:14:21.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40603f0 0x7f7ef4156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40add50 0x7f7ef40b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ef40b17b0 0x7f7ef415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4147c70 con 0x7f7ef40add50 2021-04-03T03:14:21.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef415a160 con 0x7f7ef40b17b0 2021-04-03T03:14:21.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4156f00 con 0x7f7ef40603f0 2021-04-03T03:14:21.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40add50 0x7f7ef40b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40add50 0x7f7ef40b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53590/0 (socket says 172.21.2.111:53590) 2021-04-03T03:14:21.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 -- 172.21.2.111:0/222904365 learned_addr learned my addr 172.21.2.111:0/222904365 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef37fe700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ef40b17b0 0x7f7ef415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef2ffd700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40603f0 0x7f7ef4156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 -- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40603f0 msgr2=0x7f7ef4156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40603f0 0x7f7ef4156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 -- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ef40b17b0 msgr2=0x7f7ef415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ef40b17b0 0x7f7ef415a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 -- 172.21.2.111:0/222904365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40b4c90 con 0x7f7ef40add50 2021-04-03T03:14:21.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef37fe700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ef40b17b0 0x7f7ef415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:14:21.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef27fc700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40add50 0x7f7ef40b47a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f7ee400c000 tx=0x7f7ee4006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7eebfff700 1 -- 172.21.2.111:0/222904365 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ee400ca10 con 0x7f7ef40add50 2021-04-03T03:14:21.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7eebfff700 1 -- 172.21.2.111:0/222904365 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee400cb70 con 0x7f7ef40add50 2021-04-03T03:14:21.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7eebfff700 1 -- 172.21.2.111:0/222904365 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ee40123f0 con 0x7f7ef40add50 2021-04-03T03:14:21.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef41476f0 con 0x7f7ef40add50 2021-04-03T03:14:21.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef415aaf0 con 0x7f7ef40add50 2021-04-03T03:14:21.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef40b4ee0 con 0x7f7ef40add50 2021-04-03T03:14:21.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7eebfff700 1 -- 172.21.2.111:0/222904365 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7ee4012550 con 0x7f7ef40add50 2021-04-03T03:14:21.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7eebfff700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ed8056f20 0x7f7ed80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7eebfff700 1 -- 172.21.2.111:0/222904365 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f7ee40704a0 con 0x7f7ef40add50 2021-04-03T03:14:21.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef2ffd700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ed8056f20 0x7f7ed80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.419+0000 7f7ef2ffd700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ed8056f20 0x7f7ed80593e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7edc01c000 tx=0x7f7edc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.423+0000 7f7eebfff700 1 -- 172.21.2.111:0/222904365 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7ee4045a50 con 0x7f7ef40add50 2021-04-03T03:14:21.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.515+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7ef415a320 con 0x7f7ef40add50 2021-04-03T03:14:21.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.515+0000 7f7eebfff700 1 -- 172.21.2.111:0/222904365 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v68) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7ee4049070 con 0x7f7ef40add50 2021-04-03T03:14:21.531 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:14:21.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ed8056f20 msgr2=0x7f7ed80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ed8056f20 0x7f7ed80593e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7edc01c000 tx=0x7f7edc015000).stop 2021-04-03T03:14:21.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40add50 msgr2=0x7f7ef40b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40add50 0x7f7ef40b47a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f7ee400c000 tx=0x7f7ee4006dd0).stop 2021-04-03T03:14:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 shutdown_connections 2021-04-03T03:14:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ed8056f20 0x7f7ed80593e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ef40b17b0 0x7f7ef415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ef40603f0 0x7f7ef4156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 --2- 172.21.2.111:0/222904365 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ef40add50 0x7f7ef40b47a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 >> 172.21.2.111:0/222904365 conn(0x7f7ef40b61c0 msgr2=0x7f7ef405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 shutdown_connections 2021-04-03T03:14:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.519+0000 7f7ef3fff700 1 -- 172.21.2.111:0/222904365 wait complete. 2021-04-03T03:14:21.539 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:14:21.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f271759e700 1 Processor -- start 2021-04-03T03:14:21.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f271759e700 1 -- start start 2021-04-03T03:14:21.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f271759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f271003f9d0 0x7f271003fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f271759e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2710041360 con 0x7f27100d3ef0 2021-04-03T03:14:21.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f271759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2710042b40 con 0x7f2710047130 2021-04-03T03:14:21.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f271759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2710047510 con 0x7f271003f9d0 2021-04-03T03:14:21.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f2715d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f27100d3ef0 0x7f271003f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54390/0 (socket says 172.21.2.111:54390) 2021-04-03T03:14:21.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f2715d9b700 1 -- 172.21.2.111:0/4203543719 learned_addr learned my addr 172.21.2.111:0/4203543719 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:21.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.583+0000 7f2716d9d700 1 --2- 172.21.2.111:0/4203543719 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f271003f9d0 0x7f271003fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271559a700 1 -- 172.21.2.111:0/4203543719 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323182321 0 0) 0x7f2710041360 con 0x7f27100d3ef0 2021-04-03T03:14:21.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271559a700 1 -- 172.21.2.111:0/4203543719 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f8003240 con 0x7f27100d3ef0 2021-04-03T03:14:21.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2716d9d700 1 -- 172.21.2.111:0/4203543719 >> v1:172.21.2.115:6789/0 conn(0x7f2710047130 legacy=0x7f27100d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2716d9d700 1 -- 172.21.2.111:0/4203543719 >> v1:172.21.2.111:6789/0 conn(0x7f27100d3ef0 legacy=0x7f271003f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2716d9d700 1 -- 172.21.2.111:0/4203543719 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2710042920 con 0x7f271003f9d0 2021-04-03T03:14:21.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2716d9d700 1 --2- 172.21.2.111:0/4203543719 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f271003f9d0 0x7f271003fe20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f271806b5d0 tx=0x7f271805ad70).ready entity=mon.2 client_cookie=b73281cccd08355d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271559a700 1 -- 172.21.2.111:0/4203543719 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f271805bda0 con 0x7f271003f9d0 2021-04-03T03:14:21.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271559a700 1 -- 172.21.2.111:0/4203543719 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271805df60 con 0x7f271003f9d0 2021-04-03T03:14:21.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271559a700 1 -- 172.21.2.111:0/4203543719 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27180699f0 con 0x7f271003f9d0 2021-04-03T03:14:21.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/4203543719 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f271003f9d0 msgr2=0x7f271003fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 --2- 172.21.2.111:0/4203543719 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f271003f9d0 0x7f271003fe20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f271806b5d0 tx=0x7f271805ad70).stop 2021-04-03T03:14:21.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/4203543719 shutdown_connections 2021-04-03T03:14:21.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 --2- 172.21.2.111:0/4203543719 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f271003f9d0 0x7f271003fe20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/4203543719 >> 172.21.2.111:0/4203543719 conn(0x7f271000d510 msgr2=0x7f2710046700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/4203543719 shutdown_connections 2021-04-03T03:14:21.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/4203543719 wait complete. 2021-04-03T03:14:21.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 Processor -- start 2021-04-03T03:14:21.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- start start 2021-04-03T03:14:21.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f271003f9d0 0x7f27100d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2710047130 0x7f271003be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27100d3ef0 0x7f27100e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271003c9b0 con 0x7f2710047130 2021-04-03T03:14:21.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27100d5f60 con 0x7f271003f9d0 2021-04-03T03:14:21.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27100d95f0 con 0x7f27100d3ef0 2021-04-03T03:14:21.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2710047130 0x7f271003be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2710047130 0x7f271003be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53604/0 (socket says 172.21.2.111:53604) 2021-04-03T03:14:21.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 -- 172.21.2.111:0/142202214 learned_addr learned my addr 172.21.2.111:0/142202214 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2716d9d700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27100d3ef0 0x7f27100e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271659c700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f271003f9d0 0x7f27100d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 -- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27100d3ef0 msgr2=0x7f27100e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27100d3ef0 0x7f27100e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 -- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f271003f9d0 msgr2=0x7f27100d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f271003f9d0 0x7f27100d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 -- 172.21.2.111:0/142202214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100e6750 con 0x7f2710047130 2021-04-03T03:14:21.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271659c700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f271003f9d0 0x7f27100d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:14:21.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f2715d9b700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2710047130 0x7f271003be00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f27080010d0 tx=0x7f2708007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f270f7fe700 1 -- 172.21.2.111:0/142202214 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27080072e0 con 0x7f2710047130 2021-04-03T03:14:21.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f270f7fe700 1 -- 172.21.2.111:0/142202214 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2708007440 con 0x7f2710047130 2021-04-03T03:14:21.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27100d59e0 con 0x7f2710047130 2021-04-03T03:14:21.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100d9a90 con 0x7f2710047130 2021-04-03T03:14:21.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f270f7fe700 1 -- 172.21.2.111:0/142202214 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27080057e0 con 0x7f2710047130 2021-04-03T03:14:21.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27100d60d0 con 0x7f2710047130 2021-04-03T03:14:21.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.587+0000 7f270f7fe700 1 -- 172.21.2.111:0/142202214 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f270801a060 con 0x7f2710047130 2021-04-03T03:14:21.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.591+0000 7f270f7fe700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f26fc056fe0 0x7f26fc0594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.591+0000 7f270f7fe700 1 -- 172.21.2.111:0/142202214 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f270806f230 con 0x7f2710047130 2021-04-03T03:14:21.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.591+0000 7f271659c700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f26fc056fe0 0x7f26fc0594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.591+0000 7f271659c700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f26fc056fe0 0x7f26fc0594a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f270000b000 tx=0x7f2700009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.591+0000 7f270f7fe700 1 -- 172.21.2.111:0/142202214 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f27100d60d0 con 0x7f2710047130 2021-04-03T03:14:21.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.683+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f27100d60d0 con 0x7f2710047130 2021-04-03T03:14:21.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f270f7fe700 1 -- 172.21.2.111:0/142202214 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7f27100d60d0 con 0x7f2710047130 2021-04-03T03:14:21.699 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:14:21.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f26fc056fe0 msgr2=0x7f26fc0594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f26fc056fe0 0x7f26fc0594a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f270000b000 tx=0x7f2700009040).stop 2021-04-03T03:14:21.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2710047130 msgr2=0x7f271003be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2710047130 0x7f271003be00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f27080010d0 tx=0x7f2708007ab0).stop 2021-04-03T03:14:21.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 shutdown_connections 2021-04-03T03:14:21.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27100d3ef0 0x7f27100e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f26fc056fe0 0x7f26fc0594a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f271003f9d0 0x7f27100d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 --2- 172.21.2.111:0/142202214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2710047130 0x7f271003be00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 >> 172.21.2.111:0/142202214 conn(0x7f271000d510 msgr2=0x7f2710045da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 shutdown_connections 2021-04-03T03:14:21.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.687+0000 7f271759e700 1 -- 172.21.2.111:0/142202214 wait complete. 2021-04-03T03:14:21.706 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:14:21.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 Processor -- start 2021-04-03T03:14:21.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- start start 2021-04-03T03:14:21.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa1900dbb90 0x7fa190042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa190006bc0 0x7fa1900dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1900dc6e0 con 0x7fa19003d360 2021-04-03T03:14:21.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1900d9560 con 0x7fa1900dbb90 2021-04-03T03:14:21.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19003dcb0 con 0x7fa190006bc0 2021-04-03T03:14:21.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19cdfc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa1900dbb90 0x7fa190042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa197fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa19003d360 0x7fa1900064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54404/0 (socket says 172.21.2.111:54404) 2021-04-03T03:14:21.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa197fff700 1 -- 172.21.2.111:0/1609856633 learned_addr learned my addr 172.21.2.111:0/1609856633 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:21.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19d5fd700 1 --2- 172.21.2.111:0/1609856633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa190006bc0 0x7fa1900dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa1977fe700 1 -- 172.21.2.111:0/1609856633 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773080581 0 0) 0x7fa1900dc6e0 con 0x7fa19003d360 2021-04-03T03:14:21.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19cdfc700 1 -- 172.21.2.111:0/1609856633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa190006bc0 msgr2=0x7fa1900dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19cdfc700 1 --2- 172.21.2.111:0/1609856633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa190006bc0 0x7fa1900dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19cdfc700 1 -- 172.21.2.111:0/1609856633 >> v1:172.21.2.111:6789/0 conn(0x7fa19003d360 legacy=0x7fa1900064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19cdfc700 1 -- 172.21.2.111:0/1609856633 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1900dc460 con 0x7fa1900dbb90 2021-04-03T03:14:21.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19d5fd700 1 --2- 172.21.2.111:0/1609856633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa190006bc0 0x7fa1900dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:21.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19cdfc700 1 --2- 172.21.2.111:0/1609856633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1900dbb90 0x7fa190042750 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fa19003e5f0 tx=0x7fa18c006dd0).ready entity=mon.0 client_cookie=9df9865359dffc39 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa1977fe700 1 -- 172.21.2.111:0/1609856633 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18c0097b0 con 0x7fa1900dbb90 2021-04-03T03:14:21.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa1977fe700 1 -- 172.21.2.111:0/1609856633 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c009910 con 0x7fa1900dbb90 2021-04-03T03:14:21.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa1977fe700 1 -- 172.21.2.111:0/1609856633 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18c009c70 con 0x7fa1900dbb90 2021-04-03T03:14:21.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1609856633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1900dbb90 msgr2=0x7fa190042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1609856633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1900dbb90 0x7fa190042750 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fa19003e5f0 tx=0x7fa18c006dd0).stop 2021-04-03T03:14:21.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1609856633 shutdown_connections 2021-04-03T03:14:21.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1609856633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa190006bc0 0x7fa1900dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1609856633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1900dbb90 0x7fa190042750 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1609856633 >> 172.21.2.111:0/1609856633 conn(0x7fa1900462c0 msgr2=0x7fa1900466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1609856633 shutdown_connections 2021-04-03T03:14:21.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.751+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1609856633 wait complete. 2021-04-03T03:14:21.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 Processor -- start 2021-04-03T03:14:21.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 -- start start 2021-04-03T03:14:21.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19003d360 0x7fa1900eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1900dbb90 0x7fa1900da180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19003a2b0 0x7fa1900da670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19003cb90 con 0x7fa19003d360 2021-04-03T03:14:21.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1900eb1d0 con 0x7fa1900dbb90 2021-04-03T03:14:21.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1900a31a0 con 0x7fa19003a2b0 2021-04-03T03:14:21.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19003d360 0x7fa1900eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19003d360 0x7fa1900eace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53616/0 (socket says 172.21.2.111:53616) 2021-04-03T03:14:21.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 -- 172.21.2.111:0/1141809613 learned_addr learned my addr 172.21.2.111:0/1141809613 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 -- 172.21.2.111:0/1141809613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19003a2b0 msgr2=0x7fa1900da670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19003a2b0 0x7fa1900da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 -- 172.21.2.111:0/1141809613 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1900dbb90 msgr2=0x7fa1900da180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1900dbb90 0x7fa1900da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 -- 172.21.2.111:0/1141809613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1900eb430 con 0x7fa19003d360 2021-04-03T03:14:21.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19cdfc700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19003d360 0x7fa1900eace0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fa19003e5f0 tx=0x7fa18c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa195ffb700 1 -- 172.21.2.111:0/1141809613 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18c011db0 con 0x7fa19003d360 2021-04-03T03:14:21.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa195ffb700 1 -- 172.21.2.111:0/1141809613 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c01e460 con 0x7fa19003d360 2021-04-03T03:14:21.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa190044160 con 0x7fa19003d360 2021-04-03T03:14:21.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa195ffb700 1 -- 172.21.2.111:0/1141809613 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18c01e7c0 con 0x7fa19003d360 2021-04-03T03:14:21.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19003a040 con 0x7fa19003d360 2021-04-03T03:14:21.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1900daeb0 con 0x7fa19003d360 2021-04-03T03:14:21.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa195ffb700 1 -- 172.21.2.111:0/1141809613 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa18c025030 con 0x7fa19003d360 2021-04-03T03:14:21.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa195ffb700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa178056fb0 0x7fa178059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa195ffb700 1 -- 172.21.2.111:0/1141809613 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fa18c071470 con 0x7fa19003d360 2021-04-03T03:14:21.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa197fff700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa178056fb0 0x7fa178059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.755+0000 7fa197fff700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa178056fb0 0x7fa178059470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fa188000b80 tx=0x7fa188001160).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.759+0000 7fa195ffb700 1 -- 172.21.2.111:0/1141809613 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa18c046b90 con 0x7fa19003d360 2021-04-03T03:14:21.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa1900eb7f0 con 0x7fa19003d360 2021-04-03T03:14:21.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa195ffb700 1 -- 172.21.2.111:0/1141809613 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa18c023090 con 0x7fa19003d360 2021-04-03T03:14:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:21.868 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:14:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.868 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:14:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:14:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:14:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:14:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:14:21.869 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:14:21.870 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:14:21.870 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:14:21.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa178056fb0 msgr2=0x7fa178059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa178056fb0 0x7fa178059470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fa188000b80 tx=0x7fa188001160).stop 2021-04-03T03:14:21.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19003d360 msgr2=0x7fa1900eace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19003d360 0x7fa1900eace0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fa19003e5f0 tx=0x7fa18c015000).stop 2021-04-03T03:14:21.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 shutdown_connections 2021-04-03T03:14:21.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19003a2b0 0x7fa1900da670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19003d360 0x7fa1900eace0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa178056fb0 0x7fa178059470 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 --2- 172.21.2.111:0/1141809613 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1900dbb90 0x7fa1900da180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:21.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 >> 172.21.2.111:0/1141809613 conn(0x7fa1900462c0 msgr2=0x7fa1900055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 shutdown_connections 2021-04-03T03:14:21.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.855+0000 7fa19ddfe700 1 -- 172.21.2.111:0/1141809613 wait complete. 2021-04-03T03:14:21.876 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:14:21.876 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:14:21.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.919+0000 7fd2a72ce700 1 Processor -- start 2021-04-03T03:14:21.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.919+0000 7fd2a72ce700 1 -- start start 2021-04-03T03:14:21.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.919+0000 7fd2a72ce700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd298041060 0x7fd2980dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.919+0000 7fd2a72ce700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd29803d560 0x7fd298041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.919+0000 7fd2a72ce700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2980dc560 con 0x7fd2980dbba0 2021-04-03T03:14:21.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.919+0000 7fd2a72ce700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29803de60 con 0x7fd29803d560 2021-04-03T03:14:21.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.919+0000 7fd2a72ce700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980dc800 con 0x7fd298041060 2021-04-03T03:14:21.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a6acd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd29803d560 0x7fd298041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a5acb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd298041060 0x7fd2980dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a6acd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd29803d560 0x7fd298041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:43336/0 (socket says 172.21.2.111:43336) 2021-04-03T03:14:21.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a6acd700 1 -- 172.21.2.111:0/4080618592 learned_addr learned my addr 172.21.2.111:0/4080618592 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a5acb700 1 -- 172.21.2.111:0/4080618592 >> v2:172.21.2.115:3300/0 conn(0x7fd29803d560 msgr2=0x7fd298041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:21.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a5acb700 1 --2- 172.21.2.111:0/4080618592 >> v2:172.21.2.115:3300/0 conn(0x7fd29803d560 0x7fd298041650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a5acb700 1 -- 172.21.2.111:0/4080618592 >> v1:172.21.2.115:6789/0 conn(0x7fd2980dbba0 legacy=0x7fd29803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a5acb700 1 -- 172.21.2.111:0/4080618592 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29803da30 con 0x7fd298041060 2021-04-03T03:14:21.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a6acd700 1 --2- 172.21.2.111:0/4080618592 >> v2:172.21.2.115:3300/0 conn(0x7fd29803d560 0x7fd298041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:14:21.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a5acb700 1 --2- 172.21.2.111:0/4080618592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd298041060 0x7fd2980dbf80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd29c0010d0 tx=0x7fd294006dd0).ready entity=mon.2 client_cookie=a24fdea0ca052f93 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a52ca700 1 -- 172.21.2.111:0/4080618592 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2940095d0 con 0x7fd298041060 2021-04-03T03:14:21.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a52ca700 1 -- 172.21.2.111:0/4080618592 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd294009730 con 0x7fd298041060 2021-04-03T03:14:21.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a52ca700 1 -- 172.21.2.111:0/4080618592 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd294009a90 con 0x7fd298041060 2021-04-03T03:14:21.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 -- 172.21.2.111:0/4080618592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd298041060 msgr2=0x7fd2980dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/4080618592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd298041060 0x7fd2980dbf80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd29c0010d0 tx=0x7fd294006dd0).stop 2021-04-03T03:14:21.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 -- 172.21.2.111:0/4080618592 shutdown_connections 2021-04-03T03:14:21.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/4080618592 >> v2:172.21.2.115:3300/0 conn(0x7fd29803d560 0x7fd298041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/4080618592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd298041060 0x7fd2980dbf80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 -- 172.21.2.111:0/4080618592 >> 172.21.2.111:0/4080618592 conn(0x7fd2980462b0 msgr2=0x7fd2980466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:21.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 -- 172.21.2.111:0/4080618592 shutdown_connections 2021-04-03T03:14:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.923+0000 7fd2a72ce700 1 -- 172.21.2.111:0/4080618592 wait complete. 2021-04-03T03:14:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 Processor -- start 2021-04-03T03:14:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 -- start start 2021-04-03T03:14:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd29803d560 0x7fd2980dcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd298041060 0x7fd2980a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2980dbba0 0x7fd2980a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980df790 con 0x7fd298041060 2021-04-03T03:14:21.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980dcfe0 con 0x7fd29803d560 2021-04-03T03:14:21.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980e7dd0 con 0x7fd2980dbba0 2021-04-03T03:14:21.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd298041060 0x7fd2980a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd298041060 0x7fd2980a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53630/0 (socket says 172.21.2.111:53630) 2021-04-03T03:14:21.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 -- 172.21.2.111:0/2020312993 learned_addr learned my addr 172.21.2.111:0/2020312993 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:21.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 -- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2980dbba0 msgr2=0x7fd2980a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a6acd700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2980dbba0 0x7fd2980a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a62cc700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd29803d560 0x7fd2980dcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2980dbba0 0x7fd2980a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 -- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd29803d560 msgr2=0x7fd2980dcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd29803d560 0x7fd2980dcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:21.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 -- 172.21.2.111:0/2020312993 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980e8070 con 0x7fd298041060 2021-04-03T03:14:21.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a6acd700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2980dbba0 0x7fd2980a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:21.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a62cc700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd29803d560 0x7fd2980dcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:21.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a5acb700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd298041060 0x7fd2980a00f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fd29c0010d0 tx=0x7fd294015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2937fe700 1 -- 172.21.2.111:0/2020312993 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd294011de0 con 0x7fd298041060 2021-04-03T03:14:21.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2937fe700 1 -- 172.21.2.111:0/2020312993 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29401e460 con 0x7fd298041060 2021-04-03T03:14:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2980e7ab0 con 0x7fd298041060 2021-04-03T03:14:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2937fe700 1 -- 172.21.2.111:0/2020312993 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd29401e7c0 con 0x7fd298041060 2021-04-03T03:14:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2980d6570 con 0x7fd298041060 2021-04-03T03:14:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2980e8200 con 0x7fd298041060 2021-04-03T03:14:21.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2937fe700 1 -- 172.21.2.111:0/2020312993 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd294025030 con 0x7fd298041060 2021-04-03T03:14:21.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2937fe700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd280057000 0x7fd2800594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:21.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2937fe700 1 -- 172.21.2.111:0/2020312993 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fd294071220 con 0x7fd298041060 2021-04-03T03:14:21.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a62cc700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd280057000 0x7fd2800594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:21.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.927+0000 7fd2a62cc700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd280057000 0x7fd2800594c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fd29c008000 tx=0x7fd29c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:21.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:21.931+0000 7fd2937fe700 1 -- 172.21.2.111:0/2020312993 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd294046940 con 0x7fd298041060 2021-04-03T03:14:22.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.023+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fd298004db0 con 0x7fd298041060 2021-04-03T03:14:22.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.859+0000 7fd2937fe700 1 -- 172.21.2.111:0/2020312993 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v69) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd2940230a0 con 0x7fd298041060 2021-04-03T03:14:22.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd280057000 msgr2=0x7fd2800594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:22.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd280057000 0x7fd2800594c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fd29c008000 tx=0x7fd29c009040).stop 2021-04-03T03:14:22.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd298041060 msgr2=0x7fd2980a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:22.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd298041060 0x7fd2980a00f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fd29c0010d0 tx=0x7fd294015000).stop 2021-04-03T03:14:22.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 shutdown_connections 2021-04-03T03:14:22.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2980dbba0 0x7fd2980a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:22.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd280057000 0x7fd2800594c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:22.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd29803d560 0x7fd2980dcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:22.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 --2- 172.21.2.111:0/2020312993 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd298041060 0x7fd2980a00f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:22.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 >> 172.21.2.111:0/2020312993 conn(0x7fd2980462b0 msgr2=0x7fd298005ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:22.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 shutdown_connections 2021-04-03T03:14:22.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.863+0000 7fd2a72ce700 1 -- 172.21.2.111:0/2020312993 wait complete. 2021-04-03T03:14:22.879 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:14:22.889 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:14:22.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.931+0000 7f216df20700 1 Processor -- start 2021-04-03T03:14:22.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- start start 2021-04-03T03:14:22.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f21680603f0 0x7f21681447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:22.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f21680add50 0x7f2168144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:22.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21680607d0 con 0x7f21680b17b0 2021-04-03T03:14:22.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21681451c0 con 0x7f21680add50 2021-04-03T03:14:22.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216814b090 con 0x7f21680603f0 2021-04-03T03:14:22.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216d71f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f21680b17b0 0x7f2168147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54430/0 (socket says 172.21.2.111:54430) 2021-04-03T03:14:22.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216d71f700 1 -- 172.21.2.111:0/2051158902 learned_addr learned my addr 172.21.2.111:0/2051158902 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:22.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f2167fff700 1 --2- 172.21.2.111:0/2051158902 >> v2:172.21.2.111:3300/0 conn(0x7f21680add50 0x7f2168144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:22.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f2167fff700 1 -- 172.21.2.111:0/2051158902 >> v2:172.21.2.115:3300/0 conn(0x7f21680603f0 msgr2=0x7f21681447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:22.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f2167fff700 1 --2- 172.21.2.111:0/2051158902 >> v2:172.21.2.115:3300/0 conn(0x7f21680603f0 0x7f21681447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:22.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f2167fff700 1 -- 172.21.2.111:0/2051158902 >> v1:172.21.2.111:6789/0 conn(0x7f21680b17b0 legacy=0x7f2168147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:22.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f2167fff700 1 -- 172.21.2.111:0/2051158902 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216814b330 con 0x7f21680add50 2021-04-03T03:14:22.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f2167fff700 1 --2- 172.21.2.111:0/2051158902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680add50 0x7f2168144cd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f2158002f30 tx=0x7f2158006dd0).ready entity=mon.0 client_cookie=94c84b7d80896eef server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:22.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f21677fe700 1 -- 172.21.2.111:0/2051158902 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21580097f0 con 0x7f21680add50 2021-04-03T03:14:22.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f21677fe700 1 -- 172.21.2.111:0/2051158902 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158009950 con 0x7f21680add50 2021-04-03T03:14:22.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f21677fe700 1 -- 172.21.2.111:0/2051158902 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2158009cb0 con 0x7f21680add50 2021-04-03T03:14:22.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- 172.21.2.111:0/2051158902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680add50 msgr2=0x7f2168144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:22.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- 172.21.2.111:0/2051158902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680add50 0x7f2168144cd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f2158002f30 tx=0x7f2158006dd0).stop 2021-04-03T03:14:22.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- 172.21.2.111:0/2051158902 shutdown_connections 2021-04-03T03:14:22.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- 172.21.2.111:0/2051158902 >> v2:172.21.2.115:3300/0 conn(0x7f21680603f0 0x7f21681447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:22.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- 172.21.2.111:0/2051158902 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680add50 0x7f2168144cd0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:22.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- 172.21.2.111:0/2051158902 >> 172.21.2.111:0/2051158902 conn(0x7f21680b61c0 msgr2=0x7f216805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:22.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- 172.21.2.111:0/2051158902 shutdown_connections 2021-04-03T03:14:22.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- 172.21.2.111:0/2051158902 wait complete. 2021-04-03T03:14:22.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 Processor -- start 2021-04-03T03:14:22.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- start start 2021-04-03T03:14:22.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680603f0 0x7f21680aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:22.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21680add50 0x7f2168156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:22.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21680b17b0 0x7f21681573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:22.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168062ea0 con 0x7f21680603f0 2021-04-03T03:14:22.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168063140 con 0x7f21680add50 2021-04-03T03:14:22.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216df20700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21680633e0 con 0x7f21680b17b0 2021-04-03T03:14:22.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216cf1e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680603f0 0x7f21680aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:22.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216cf1e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680603f0 0x7f21680aab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53642/0 (socket says 172.21.2.111:53642) 2021-04-03T03:14:22.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.935+0000 7f216cf1e700 1 -- 172.21.2.111:0/1518833679 learned_addr learned my addr 172.21.2.111:0/1518833679 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:22.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216cf1e700 1 -- 172.21.2.111:0/1518833679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21680b17b0 msgr2=0x7f21681573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:22.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216cf1e700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21680b17b0 0x7f21681573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:22.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216cf1e700 1 -- 172.21.2.111:0/1518833679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21680add50 msgr2=0x7f2168156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:22.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216cf1e700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21680add50 0x7f2168156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:22.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216cf1e700 1 -- 172.21.2.111:0/1518833679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168063880 con 0x7f21680603f0 2021-04-03T03:14:22.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216cf1e700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680603f0 0x7f21680aab10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f2150002f30 tx=0x7f2150006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:22.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2165ffb700 1 -- 172.21.2.111:0/1518833679 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f215000d840 con 0x7f21680603f0 2021-04-03T03:14:22.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2165ffb700 1 -- 172.21.2.111:0/1518833679 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2150000e90 con 0x7f21680603f0 2021-04-03T03:14:22.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2168062a70 con 0x7f21680603f0 2021-04-03T03:14:22.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2165ffb700 1 -- 172.21.2.111:0/1518833679 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f215000db90 con 0x7f21680603f0 2021-04-03T03:14:22.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2168065b70 con 0x7f21680603f0 2021-04-03T03:14:22.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21680abc30 con 0x7f21680603f0 2021-04-03T03:14:22.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2165ffb700 1 -- 172.21.2.111:0/1518833679 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2150017030 con 0x7f21680603f0 2021-04-03T03:14:22.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2165ffb700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f214c056f60 0x7f214c059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:22.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2165ffb700 1 -- 172.21.2.111:0/1518833679 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5950+0+0 (secure 0 0 0) 0x7f215006f310 con 0x7f21680603f0 2021-04-03T03:14:22.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2167fff700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f214c056f60 0x7f214c059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:22.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.939+0000 7f2167fff700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f214c056f60 0x7f214c059420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f2158002f30 tx=0x7f2158015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:22.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:22.943+0000 7f2165ffb700 1 -- 172.21.2.111:0/1518833679 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2150044980 con 0x7f21680603f0 2021-04-03T03:14:23.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.035+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f21680635a0 con 0x7f21680603f0 2021-04-03T03:14:23.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f2165ffb700 1 -- 172.21.2.111:0/1518833679 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v70) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2150047fa0 con 0x7f21680603f0 2021-04-03T03:14:23.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f214c056f60 msgr2=0x7f214c059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:23.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f214c056f60 0x7f214c059420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f2158002f30 tx=0x7f2158015000).stop 2021-04-03T03:14:23.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680603f0 msgr2=0x7f21680aab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:23.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680603f0 0x7f21680aab10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f2150002f30 tx=0x7f2150006ea0).stop 2021-04-03T03:14:23.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 shutdown_connections 2021-04-03T03:14:23.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f21680b17b0 0x7f21681573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:23.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f21680603f0 0x7f21680aab10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:23.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f214c056f60 0x7f214c059420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:23.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 --2- 172.21.2.111:0/1518833679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f21680add50 0x7f2168156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:23.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 >> 172.21.2.111:0/1518833679 conn(0x7f21680b61c0 msgr2=0x7f21681449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:23.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 shutdown_connections 2021-04-03T03:14:23.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.867+0000 7f216df20700 1 -- 172.21.2.111:0/1518833679 wait complete. 2021-04-03T03:14:23.884 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:14:23.888 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:14:23.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 Processor -- start 2021-04-03T03:14:23.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- start start 2021-04-03T03:14:23.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 0x7f10600dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600dc560 con 0x7f106003d560 2021-04-03T03:14:23.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600dc800 con 0x7f1060041060 2021-04-03T03:14:23.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106003de60 con 0x7f10600dbba0 2021-04-03T03:14:23.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1067fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f106003d560 0x7f1060041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54446/0 (socket says 172.21.2.111:54446) 2021-04-03T03:14:23.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1067fff700 1 -- 172.21.2.111:0/2306172729 learned_addr learned my addr 172.21.2.111:0/2306172729 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:23.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661445997 0 0) 0x7f10600dc560 con 0x7f106003d560 2021-04-03T03:14:23.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c003240 con 0x7f106003d560 2021-04-03T03:14:23.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10677fe700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 0x7f10600dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:23.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 37487677 0 0) 0x7f104c003240 con 0x7f106003d560 2021-04-03T03:14:23.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 msgr2=0x7f10600dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:23.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 0x7f10600dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:23.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 >> v1:172.21.2.115:6789/0 conn(0x7f1060041060 legacy=0x7f106003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:23.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106003da30 con 0x7f106003d560 2021-04-03T03:14:23.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f10680450f0 con 0x7f106003d560 2021-04-03T03:14:23.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 >> v1:172.21.2.111:6789/0 conn(0x7f106003d560 legacy=0x7f1060041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:23.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f104c003760 0x7f104c003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f104c004070 0x7f104c004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 0x7f104c004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106003de60 con 0x7f104c004070 2021-04-03T03:14:23.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0050e0 con 0x7f104c003760 2021-04-03T03:14:23.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0052a0 con 0x7f10600dbba0 2021-04-03T03:14:23.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10677fe700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f104c003760 0x7f104c003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:23.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1066ffd700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f104c004070 0x7f104c004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:23.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1067fff700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 0x7f104c004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:23.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1066ffd700 1 -- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 msgr2=0x7f104c004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:23.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1066ffd700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f10600dbba0 0x7f104c004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:23.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1066ffd700 1 -- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f104c003760 msgr2=0x7f104c003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:23.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1066ffd700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f104c003760 0x7f104c003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:23.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1066ffd700 1 -- 172.21.2.111:0/2306172729 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c003240 con 0x7f104c004070 2021-04-03T03:14:23.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1066ffd700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f104c004070 0x7f104c004540 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f1054001f60 tx=0x7f1054007b70).ready entity=mon.0 client_cookie=21e38b05bbeaa626 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:23.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f105400c870 con 0x7f104c004070 2021-04-03T03:14:23.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105400c9d0 con 0x7f104c004070 2021-04-03T03:14:23.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f10667fc700 1 -- 172.21.2.111:0/2306172729 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f105400cd30 con 0x7f104c004070 2021-04-03T03:14:23.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- 172.21.2.111:0/2306172729 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f104c004070 msgr2=0x7f104c004540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:23.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 --2- 172.21.2.111:0/2306172729 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f104c004070 0x7f104c004540 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f1054001f60 tx=0x7f1054007b70).stop 2021-04-03T03:14:23.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f1067fff700 1 -- 172.21.2.111:0/2306172729 reap_dead start 2021-04-03T03:14:23.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- 172.21.2.111:0/2306172729 shutdown_connections 2021-04-03T03:14:23.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- 172.21.2.111:0/2306172729 >> 172.21.2.111:0/2306172729 conn(0x7f10600462b0 msgr2=0x7f10600466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:23.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- 172.21.2.111:0/2306172729 shutdown_connections 2021-04-03T03:14:23.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.939+0000 7f106caa7700 1 -- 172.21.2.111:0/2306172729 wait complete. 2021-04-03T03:14:23.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 Processor -- start 2021-04-03T03:14:23.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 -- start start 2021-04-03T03:14:23.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106003ad50 0x7f10600e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f10600a07a0 0x7f106009fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106009ff70 0x7f10600ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600a1c70 con 0x7f10600a07a0 2021-04-03T03:14:23.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600a19d0 con 0x7f106003ad50 2021-04-03T03:14:23.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106003bb00 con 0x7f106009ff70 2021-04-03T03:14:23.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f10600a07a0 0x7f106009fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:23.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f10600a07a0 0x7f106009fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53664/0 (socket says 172.21.2.111:53664) 2021-04-03T03:14:23.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 -- 172.21.2.111:0/2957924290 learned_addr learned my addr 172.21.2.111:0/2957924290 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:23.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 -- 172.21.2.111:0/2957924290 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106009ff70 msgr2=0x7f10600ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:23.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106009ff70 0x7f10600ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:23.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 -- 172.21.2.111:0/2957924290 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106003ad50 msgr2=0x7f10600e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:23.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106003ad50 0x7f10600e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:23.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 -- 172.21.2.111:0/2957924290 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600a0400 con 0x7f10600a07a0 2021-04-03T03:14:23.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1066ffd700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f10600a07a0 0x7f106009fa80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f1054001f60 tx=0x7f1054016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:23.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1064ff9700 1 -- 172.21.2.111:0/2957924290 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1054013dd0 con 0x7f10600a07a0 2021-04-03T03:14:23.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1064ff9700 1 -- 172.21.2.111:0/2957924290 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1054007030 con 0x7f10600a07a0 2021-04-03T03:14:23.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106003b130 con 0x7f10600a07a0 2021-04-03T03:14:23.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1064ff9700 1 -- 172.21.2.111:0/2957924290 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1054007390 con 0x7f10600a07a0 2021-04-03T03:14:23.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10600a0e70 con 0x7f10600a07a0 2021-04-03T03:14:23.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10600a0590 con 0x7f10600a07a0 2021-04-03T03:14:23.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1064ff9700 1 -- 172.21.2.111:0/2957924290 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1054024030 con 0x7f10600a07a0 2021-04-03T03:14:23.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1064ff9700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050056ec0 0x7f1050059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:23.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f1064ff9700 1 -- 172.21.2.111:0/2957924290 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f1054070da0 con 0x7f10600a07a0 2021-04-03T03:14:23.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f10677fe700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050056ec0 0x7f1050059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:23.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.943+0000 7f10677fe700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050056ec0 0x7f1050059380 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f105c00b000 tx=0x7f105c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:23.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:23.948+0000 7f1064ff9700 1 -- 172.21.2.111:0/2957924290 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f10540462b0 con 0x7f10600a07a0 2021-04-03T03:14:24.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.076+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f1060004db0 con 0x7f10600a07a0 2021-04-03T03:14:24.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f1064ff9700 1 -- 172.21.2.111:0/2957924290 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v28) v1 ==== 189+0+0 (secure 0 0 0) 0x7f1054022030 con 0x7f10600a07a0 2021-04-03T03:14:24.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050056ec0 msgr2=0x7f1050059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050056ec0 0x7f1050059380 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f105c00b000 tx=0x7f105c008040).stop 2021-04-03T03:14:24.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f10600a07a0 msgr2=0x7f106009fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f10600a07a0 0x7f106009fa80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f1054001f60 tx=0x7f1054016000).stop 2021-04-03T03:14:24.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 shutdown_connections 2021-04-03T03:14:24.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106009ff70 0x7f10600ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050056ec0 0x7f1050059380 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106003ad50 0x7f10600e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 --2- 172.21.2.111:0/2957924290 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f10600a07a0 0x7f106009fa80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 >> 172.21.2.111:0/2957924290 conn(0x7f10600462b0 msgr2=0x7f1060041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:24.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 shutdown_connections 2021-04-03T03:14:24.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.080+0000 7f106caa7700 1 -- 172.21.2.111:0/2957924290 wait complete. 2021-04-03T03:14:24.096 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 13 and data pool 14 2021-04-03T03:14:24.100 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:14:24.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 Processor -- start 2021-04-03T03:14:24.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- start start 2021-04-03T03:14:24.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9eb00b3a90 0x7f9eb00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb00b24d0 con 0x7f9eb00b8350 2021-04-03T03:14:24.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb00b8730 con 0x7f9eb00ad660 2021-04-03T03:14:24.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb0142090 con 0x7f9eb00b3a90 2021-04-03T03:14:24.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f9eb00b8350 0x7f9eb00b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54474/0 (socket says 172.21.2.111:54474) 2021-04-03T03:14:24.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/2060283445 learned_addr learned my addr 172.21.2.111:0/2060283445 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:24.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaffff700 1 --2- 172.21.2.111:0/2060283445 >> v2:172.21.2.115:3300/0 conn(0x7f9eb00b3a90 0x7f9eb00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799383201 0 0) 0x7f9eb00b24d0 con 0x7f9eb00b8350 2021-04-03T03:14:24.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e9c003240 con 0x7f9eb00b8350 2021-04-03T03:14:24.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 634122669 0 0) 0x7f9e9c003240 con 0x7f9eb00b8350 2021-04-03T03:14:24.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 >> v2:172.21.2.115:3300/0 conn(0x7f9eb00b3a90 msgr2=0x7f9eb00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:24.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 --2- 172.21.2.111:0/2060283445 >> v2:172.21.2.115:3300/0 conn(0x7f9eb00b3a90 0x7f9eb00ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 >> v1:172.21.2.115:6789/0 conn(0x7f9eb00ad660 legacy=0x7f9eb00ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb01413c0 con 0x7f9eb00b8350 2021-04-03T03:14:24.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f9ea40016d0 con 0x7f9eb00b8350 2021-04-03T03:14:24.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 >> v1:172.21.2.111:6789/0 conn(0x7f9eb00b8350 legacy=0x7f9eb00b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9e9c003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e9c0044e0 0x7f9e9c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e9c004e50 0x7f9e9c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0142090 con 0x7f9e9c003780 2021-04-03T03:14:24.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c005300 con 0x7f9e9c004e50 2021-04-03T03:14:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c00bc20 con 0x7f9e9c0044e0 2021-04-03T03:14:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9e9c003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/2060283445 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e9c0044e0 msgr2=0x7f9e9c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaffff700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e9c0044e0 0x7f9e9c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e9c0044e0 0x7f9e9c004960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/2060283445 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e9c004e50 msgr2=0x7f9e9c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e9c004e50 0x7f9e9c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/2060283445 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c003240 con 0x7f9e9c003780 2021-04-03T03:14:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5210700 1 -- 172.21.2.111:0/2060283445 reap_dead start 2021-04-03T03:14:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb4a0f700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9e9c003ba0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f9ea40013e0 tx=0x7f9eb0141550).ready entity=mon.0 client_cookie=f91a715489a6d194 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea400ce30 con 0x7f9e9c003780 2021-04-03T03:14:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea40070d0 con 0x7f9e9c003780 2021-04-03T03:14:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eaf7fe700 1 -- 172.21.2.111:0/2060283445 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea4007430 con 0x7f9e9c003780 2021-04-03T03:14:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- 172.21.2.111:0/2060283445 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 msgr2=0x7f9e9c003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9e9c003ba0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f9ea40013e0 tx=0x7f9eb0141550).stop 2021-04-03T03:14:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- 172.21.2.111:0/2060283445 shutdown_connections 2021-04-03T03:14:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/2060283445 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9e9c003ba0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- 172.21.2.111:0/2060283445 >> 172.21.2.111:0/2060283445 conn(0x7f9eb0060490 msgr2=0x7f9eb00b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- 172.21.2.111:0/2060283445 shutdown_connections 2021-04-03T03:14:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.144+0000 7f9eb5a11700 1 -- 172.21.2.111:0/2060283445 wait complete. 2021-04-03T03:14:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 Processor -- start 2021-04-03T03:14:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 -- start start 2021-04-03T03:14:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9eb013b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9eb01457b0 0x7f9eb00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9eb0147ed0 0x7f9eb013d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0146890 con 0x7f9e9c003780 2021-04-03T03:14:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0142770 con 0x7f9eb01457b0 2021-04-03T03:14:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb013d920 con 0x7f9eb0147ed0 2021-04-03T03:14:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9eb013b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9eb013b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53692/0 (socket says 172.21.2.111:53692) 2021-04-03T03:14:24.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/4210484175 learned_addr learned my addr 172.21.2.111:0/4210484175 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5210700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9eb0147ed0 0x7f9eb013d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/4210484175 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9eb0147ed0 msgr2=0x7f9eb013d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9eb0147ed0 0x7f9eb013d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/4210484175 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9eb01457b0 msgr2=0x7f9eb00b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9eb01457b0 0x7f9eb00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 -- 172.21.2.111:0/4210484175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb013dae0 con 0x7f9e9c003780 2021-04-03T03:14:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb4a0f700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9eb013b0d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f9ea40013e0 tx=0x7f9ea4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eadffb700 1 -- 172.21.2.111:0/4210484175 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea4005bc0 con 0x7f9e9c003780 2021-04-03T03:14:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eadffb700 1 -- 172.21.2.111:0/4210484175 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea4005d20 con 0x7f9e9c003780 2021-04-03T03:14:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb0148350 con 0x7f9e9c003780 2021-04-03T03:14:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eadffb700 1 -- 172.21.2.111:0/4210484175 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea4020670 con 0x7f9e9c003780 2021-04-03T03:14:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb012a2f0 con 0x7f9e9c003780 2021-04-03T03:14:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb0057660 con 0x7f9e9c003780 2021-04-03T03:14:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eadffb700 1 -- 172.21.2.111:0/4210484175 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9ea4010070 con 0x7f9e9c003780 2021-04-03T03:14:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eadffb700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e90056ea0 0x7f9e90059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eadffb700 1 -- 172.21.2.111:0/4210484175 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9ea40713d0 con 0x7f9e9c003780 2021-04-03T03:14:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.148+0000 7f9eaffff700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e90056ea0 0x7f9e90059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.152+0000 7f9eaffff700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e90056ea0 0x7f9e90059360 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f9ea800b000 tx=0x7f9ea8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.152+0000 7f9eadffb700 1 -- 172.21.2.111:0/4210484175 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9ea40467c0 con 0x7f9e9c003780 2021-04-03T03:14:24.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9eb00686d0 con 0x7f9e9c003780 2021-04-03T03:14:24.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eadffb700 1 -- 172.21.2.111:0/4210484175 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f9ea4025070 con 0x7f9e9c003780 2021-04-03T03:14:24.261 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:24.261 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":73,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:14:24.078988+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:14:22.039727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-04-03T03:14:23.053211+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:14:24.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e90056ea0 msgr2=0x7f9e90059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e90056ea0 0x7f9e90059360 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f9ea800b000 tx=0x7f9ea8008040).stop 2021-04-03T03:14:24.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 msgr2=0x7f9eb013b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9eb013b0d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f9ea40013e0 tx=0x7f9ea4015000).stop 2021-04-03T03:14:24.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 shutdown_connections 2021-04-03T03:14:24.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9eb0147ed0 0x7f9eb013d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e9c003780 0x7f9eb013b0d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e90056ea0 0x7f9e90059360 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 --2- 172.21.2.111:0/4210484175 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9eb01457b0 0x7f9eb00b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 >> 172.21.2.111:0/4210484175 conn(0x7f9eb0060490 msgr2=0x7f9eb0065b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:24.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 shutdown_connections 2021-04-03T03:14:24.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.248+0000 7f9eb5a11700 1 -- 172.21.2.111:0/4210484175 wait complete. 2021-04-03T03:14:24.269 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:14:24.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c67ba700 1 Processor -- start 2021-04-03T03:14:24.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c67ba700 1 -- start start 2021-04-03T03:14:24.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c67ba700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc4c00b12b0 0x7fc4c01522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c67ba700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c00607d0 con 0x7fc4c014b700 2021-04-03T03:14:24.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c67ba700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c00b0eb0 con 0x7fc4c00603f0 2021-04-03T03:14:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c67ba700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c00af400 con 0x7fc4c00b12b0 2021-04-03T03:14:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c4fb7700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc4c014b700 0x7fc4c00b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54492/0 (socket says 172.21.2.111:54492) 2021-04-03T03:14:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3311036268 learned_addr learned my addr 172.21.2.111:0/3311036268 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c5fb9700 1 --2- 172.21.2.111:0/3311036268 >> v2:172.21.2.115:3300/0 conn(0x7fc4c00b12b0 0x7fc4c01522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326356498 0 0) 0x7fc4c00607d0 con 0x7fc4c014b700 2021-04-03T03:14:24.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a8003240 con 0x7fc4c014b700 2021-04-03T03:14:24.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2022975688 0 0) 0x7fc4a8003240 con 0x7fc4c014b700 2021-04-03T03:14:24.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 >> v2:172.21.2.115:3300/0 conn(0x7fc4c00b12b0 msgr2=0x7fc4c01522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:24.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 --2- 172.21.2.111:0/3311036268 >> v2:172.21.2.115:3300/0 conn(0x7fc4c00b12b0 0x7fc4c01522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 >> v1:172.21.2.115:6789/0 conn(0x7fc4c00603f0 legacy=0x7fc4c00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00af6a0 con 0x7fc4c014b700 2021-04-03T03:14:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fc4bc0016d0 con 0x7fc4c014b700 2021-04-03T03:14:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 >> v1:172.21.2.111:6789/0 conn(0x7fc4c014b700 legacy=0x7fc4c00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc4a8003760 0x7fc4a8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4a80044e0 0x7fc4a8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4a8004e50 0x7fc4a800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00af400 con 0x7fc4a80044e0 2021-04-03T03:14:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8004290 con 0x7fc4a8003760 2021-04-03T03:14:24.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a800bc20 con 0x7fc4a8004e50 2021-04-03T03:14:24.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.312+0000 7fc4c4fb7700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4a80044e0 0x7fc4a8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3311036268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4a8004e50 msgr2=0x7fc4a800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:24.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4a8004e50 0x7fc4a800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3311036268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc4a8003760 msgr2=0x7fc4a8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:24.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc4a8003760 0x7fc4a8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3311036268 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8003240 con 0x7fc4a80044e0 2021-04-03T03:14:24.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c5fb9700 1 -- 172.21.2.111:0/3311036268 reap_dead start 2021-04-03T03:14:24.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4a80044e0 0x7fc4a8004960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fc4a8004450 tx=0x7fc4c00af830).ready entity=mon.0 client_cookie=bfb29d0bd0015b4b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4bc0070d0 con 0x7fc4a80044e0 2021-04-03T03:14:24.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc007230 con 0x7fc4a80044e0 2021-04-03T03:14:24.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b3fff700 1 -- 172.21.2.111:0/3311036268 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4bc0055f0 con 0x7fc4a80044e0 2021-04-03T03:14:24.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3311036268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4a80044e0 msgr2=0x7fc4a8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4a80044e0 0x7fc4a8004960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fc4a8004450 tx=0x7fc4c00af830).stop 2021-04-03T03:14:24.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3311036268 shutdown_connections 2021-04-03T03:14:24.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3311036268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4a80044e0 0x7fc4a8004960 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3311036268 >> 172.21.2.111:0/3311036268 conn(0x7fc4c00b61c0 msgr2=0x7fc4c005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:24.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3311036268 shutdown_connections 2021-04-03T03:14:24.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3311036268 wait complete. 2021-04-03T03:14:24.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 Processor -- start 2021-04-03T03:14:24.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- start start 2021-04-03T03:14:24.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc4a80044e0 0x7fc4c01597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4c00aa240 0x7fc4c00a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4c0064270 0x7fc4c00646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00ab5f0 con 0x7fc4c00aa240 2021-04-03T03:14:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0064be0 con 0x7fc4a80044e0 2021-04-03T03:14:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0064d50 con 0x7fc4c0064270 2021-04-03T03:14:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4c00aa240 0x7fc4c00a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4c00aa240 0x7fc4c00a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53714/0 (socket says 172.21.2.111:53714) 2021-04-03T03:14:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3897176913 learned_addr learned my addr 172.21.2.111:0/3897176913 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3897176913 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4c0064270 msgr2=0x7fc4c00646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4c0064270 0x7fc4c00646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3897176913 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc4a80044e0 msgr2=0x7fc4c01597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc4a80044e0 0x7fc4c01597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 -- 172.21.2.111:0/3897176913 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00aaf80 con 0x7fc4c00aa240 2021-04-03T03:14:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c4fb7700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4c00aa240 0x7fc4c00a75c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fc4a8004450 tx=0x7fc4bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b27fc700 1 -- 172.21.2.111:0/3897176913 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4bc005d00 con 0x7fc4c00aa240 2021-04-03T03:14:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b27fc700 1 -- 172.21.2.111:0/3897176913 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc005e60 con 0x7fc4c00aa240 2021-04-03T03:14:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c00a9e60 con 0x7fc4c00aa240 2021-04-03T03:14:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b27fc700 1 -- 172.21.2.111:0/3897176913 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4bc013640 con 0x7fc4c00aa240 2021-04-03T03:14:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c00a86c0 con 0x7fc4c00aa240 2021-04-03T03:14:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c00ab760 con 0x7fc4c00aa240 2021-04-03T03:14:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b27fc700 1 -- 172.21.2.111:0/3897176913 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc4bc026020 con 0x7fc4c00aa240 2021-04-03T03:14:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b27fc700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4a0056ea0 0x7fc4a0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4b27fc700 1 -- 172.21.2.111:0/3897176913 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc4bc071600 con 0x7fc4c00aa240 2021-04-03T03:14:24.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.316+0000 7fc4c57b8700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4a0056ea0 0x7fc4a0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.320+0000 7fc4c57b8700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4a0056ea0 0x7fc4a0059360 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc4c006a3b0 tx=0x7fc4b4007790).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.320+0000 7fc4b27fc700 1 -- 172.21.2.111:0/3897176913 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc4c00ab760 con 0x7fc4c00aa240 2021-04-03T03:14:24.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.412+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4c0057660 con 0x7fc4c00aa240 2021-04-03T03:14:24.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4b27fc700 1 -- 172.21.2.111:0/3897176913 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fc4bc0468e0 con 0x7fc4c00aa240 2021-04-03T03:14:24.427 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:24.427 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":73,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:14:24.078988+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:14:22.039727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-04-03T03:14:23.053211+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:14:24.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4a0056ea0 msgr2=0x7fc4a0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4a0056ea0 0x7fc4a0059360 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc4c006a3b0 tx=0x7fc4b4007790).stop 2021-04-03T03:14:24.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4c00aa240 msgr2=0x7fc4c00a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4c00aa240 0x7fc4c00a75c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fc4a8004450 tx=0x7fc4bc015000).stop 2021-04-03T03:14:24.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 shutdown_connections 2021-04-03T03:14:24.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4c0064270 0x7fc4c00646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4a0056ea0 0x7fc4a0059360 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc4a80044e0 0x7fc4c01597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 --2- 172.21.2.111:0/3897176913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4c00aa240 0x7fc4c00a75c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 >> 172.21.2.111:0/3897176913 conn(0x7fc4c00b61c0 msgr2=0x7fc4c00b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:24.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 shutdown_connections 2021-04-03T03:14:24.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.416+0000 7fc4c67ba700 1 -- 172.21.2.111:0/3897176913 wait complete. 2021-04-03T03:14:24.435 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:14:24.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.480+0000 7fad5ecd9700 1 Processor -- start 2021-04-03T03:14:24.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.480+0000 7fad5ecd9700 1 -- start start 2021-04-03T03:14:24.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.480+0000 7fad5ecd9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fad50048f00 0x7fad500cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad500492e0 con 0x7fad50043b40 2021-04-03T03:14:24.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5003bbd0 con 0x7fad500d5230 2021-04-03T03:14:24.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5003be70 con 0x7fad50048f00 2021-04-03T03:14:24.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fad50043b40 0x7fad5003d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54516/0 (socket says 172.21.2.111:54516) 2021-04-03T03:14:24.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 -- 172.21.2.111:0/2677879501 learned_addr learned my addr 172.21.2.111:0/2677879501 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5dcd7700 1 --2- 172.21.2.111:0/2677879501 >> v2:172.21.2.111:3300/0 conn(0x7fad50048f00 0x7fad500cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ccd5700 1 -- 172.21.2.111:0/2677879501 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426089498 0 0) 0x7fad500492e0 con 0x7fad50043b40 2021-04-03T03:14:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ccd5700 1 -- 172.21.2.111:0/2677879501 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad40003240 con 0x7fad50043b40 2021-04-03T03:14:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5dcd7700 1 -- 172.21.2.111:0/2677879501 >> v1:172.21.2.115:6789/0 conn(0x7fad500d5230 legacy=0x7fad500cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5dcd7700 1 -- 172.21.2.111:0/2677879501 >> v1:172.21.2.111:6789/0 conn(0x7fad50043b40 legacy=0x7fad5003d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5dcd7700 1 -- 172.21.2.111:0/2677879501 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5003c110 con 0x7fad50048f00 2021-04-03T03:14:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5dcd7700 1 --2- 172.21.2.111:0/2677879501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad50048f00 0x7fad500cebc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fad5000e570 tx=0x7fad54006dd0).ready entity=mon.0 client_cookie=4ef2adbf01104dca server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ccd5700 1 -- 172.21.2.111:0/2677879501 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad54009770 con 0x7fad50048f00 2021-04-03T03:14:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ccd5700 1 -- 172.21.2.111:0/2677879501 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad540098d0 con 0x7fad50048f00 2021-04-03T03:14:24.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ccd5700 1 -- 172.21.2.111:0/2677879501 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad54009c70 con 0x7fad50048f00 2021-04-03T03:14:24.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2677879501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad50048f00 msgr2=0x7fad500cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2677879501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad50048f00 0x7fad500cebc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fad5000e570 tx=0x7fad54006dd0).stop 2021-04-03T03:14:24.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2677879501 shutdown_connections 2021-04-03T03:14:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2677879501 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad50048f00 0x7fad500cebc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2677879501 >> 172.21.2.111:0/2677879501 conn(0x7fad5000f310 msgr2=0x7fad500484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2677879501 shutdown_connections 2021-04-03T03:14:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2677879501 wait complete. 2021-04-03T03:14:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 Processor -- start 2021-04-03T03:14:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- start start 2021-04-03T03:14:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad50043b40 0x7fad500d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad50048f00 0x7fad500a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad500d5230 0x7fad500ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500d64b0 con 0x7fad500d5230 2021-04-03T03:14:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500d8630 con 0x7fad50043b40 2021-04-03T03:14:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad500d88d0 con 0x7fad50048f00 2021-04-03T03:14:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad500d5230 0x7fad500ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad500d5230 0x7fad500ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53730/0 (socket says 172.21.2.111:53730) 2021-04-03T03:14:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 -- 172.21.2.111:0/2426996939 learned_addr learned my addr 172.21.2.111:0/2426996939 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 -- 172.21.2.111:0/2426996939 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad50048f00 msgr2=0x7fad500a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad50048f00 0x7fad500a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 -- 172.21.2.111:0/2426996939 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad50043b40 msgr2=0x7fad500d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad50043b40 0x7fad500d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 -- 172.21.2.111:0/2426996939 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad500d6790 con 0x7fad500d5230 2021-04-03T03:14:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5e4d8700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad500d5230 0x7fad500ce760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7fad58069660 tx=0x7fad5805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad4affd700 1 -- 172.21.2.111:0/2426996939 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5805df60 con 0x7fad500d5230 2021-04-03T03:14:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad4affd700 1 -- 172.21.2.111:0/2426996939 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad580699f0 con 0x7fad500d5230 2021-04-03T03:14:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad500d6c30 con 0x7fad500d5230 2021-04-03T03:14:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad4affd700 1 -- 172.21.2.111:0/2426996939 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5806b5d0 con 0x7fad500d5230 2021-04-03T03:14:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.484+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad500d78a0 con 0x7fad500d5230 2021-04-03T03:14:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.488+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad500d6920 con 0x7fad500d5230 2021-04-03T03:14:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.488+0000 7fad4affd700 1 -- 172.21.2.111:0/2426996939 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fad58061110 con 0x7fad500d5230 2021-04-03T03:14:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.488+0000 7fad4affd700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad38056f20 0x7fad380593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.488+0000 7fad4affd700 1 -- 172.21.2.111:0/2426996939 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fad580deb40 con 0x7fad500d5230 2021-04-03T03:14:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.488+0000 7fad5dcd7700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad38056f20 0x7fad380593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.488+0000 7fad5dcd7700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad38056f20 0x7fad380593e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fad5000e570 tx=0x7fad54015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.488+0000 7fad4affd700 1 -- 172.21.2.111:0/2426996939 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fad580b3e20 con 0x7fad500d5230 2021-04-03T03:14:24.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:24.616+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fad50004db0 con 0x7fad500d5230 2021-04-03T03:14:25.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad4affd700 1 -- 172.21.2.111:0/2426996939 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7fad580dee60 con 0x7fad500d5230 2021-04-03T03:14:25.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad38056f20 msgr2=0x7fad380593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad38056f20 0x7fad380593e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fad5000e570 tx=0x7fad54015000).stop 2021-04-03T03:14:25.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad500d5230 msgr2=0x7fad500ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad500d5230 0x7fad500ce760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7fad58069660 tx=0x7fad5805ad70).stop 2021-04-03T03:14:25.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 shutdown_connections 2021-04-03T03:14:25.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad500d5230 0x7fad500ce760 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fad38056f20 0x7fad380593e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad50043b40 0x7fad500d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 --2- 172.21.2.111:0/2426996939 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad50048f00 0x7fad500a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 >> 172.21.2.111:0/2426996939 conn(0x7fad5000f310 msgr2=0x7fad50041be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 shutdown_connections 2021-04-03T03:14:25.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.088+0000 7fad5ecd9700 1 -- 172.21.2.111:0/2426996939 wait complete. 2021-04-03T03:14:25.108 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:25.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 Processor -- start 2021-04-03T03:14:25.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- start start 2021-04-03T03:14:25.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3dc40dbb90 0x7f3dc4042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3dc403d360 0x7f3dc40dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc40dc6e0 con 0x7f3dc40064b0 2021-04-03T03:14:25.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc40d9560 con 0x7f3dc40dbb90 2021-04-03T03:14:25.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc403dcb0 con 0x7f3dc403d360 2021-04-03T03:14:25.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3dc40dbb90 0x7f3dc4042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3dc40dbb90 0x7f3dc4042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53734/0 (socket says 172.21.2.111:53734) 2021-04-03T03:14:25.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 -- 172.21.2.111:0/251206859 learned_addr learned my addr 172.21.2.111:0/251206859 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 -- 172.21.2.111:0/251206859 >> v2:172.21.2.115:3300/0 conn(0x7f3dc403d360 msgr2=0x7f3dc40dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 --2- 172.21.2.111:0/251206859 >> v2:172.21.2.115:3300/0 conn(0x7f3dc403d360 0x7f3dc40dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 -- 172.21.2.111:0/251206859 >> v1:172.21.2.115:6789/0 conn(0x7f3dc40064b0 legacy=0x7f3dc40056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 -- 172.21.2.111:0/251206859 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40dc460 con 0x7f3dc40dbb90 2021-04-03T03:14:25.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 --2- 172.21.2.111:0/251206859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc40dbb90 0x7f3dc4042750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f3dc4046210 tx=0x7f3dc0006e90).ready entity=mon.0 client_cookie=9163f3462c6384e9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcb7fe700 1 -- 172.21.2.111:0/251206859 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc0009930 con 0x7f3dc40dbb90 2021-04-03T03:14:25.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcb7fe700 1 -- 172.21.2.111:0/251206859 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc0009a90 con 0x7f3dc40dbb90 2021-04-03T03:14:25.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcb7fe700 1 -- 172.21.2.111:0/251206859 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc0009e30 con 0x7f3dc40dbb90 2021-04-03T03:14:25.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- 172.21.2.111:0/251206859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc40dbb90 msgr2=0x7f3dc4042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/251206859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc40dbb90 0x7f3dc4042750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f3dc4046210 tx=0x7f3dc0006e90).stop 2021-04-03T03:14:25.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- 172.21.2.111:0/251206859 shutdown_connections 2021-04-03T03:14:25.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/251206859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc40dbb90 0x7f3dc4042750 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/251206859 >> v2:172.21.2.115:3300/0 conn(0x7f3dc403d360 0x7f3dc40dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- 172.21.2.111:0/251206859 >> 172.21.2.111:0/251206859 conn(0x7f3dc40462c0 msgr2=0x7f3dc40466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- 172.21.2.111:0/251206859 shutdown_connections 2021-04-03T03:14:25.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- 172.21.2.111:0/251206859 wait complete. 2021-04-03T03:14:25.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 Processor -- start 2021-04-03T03:14:25.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- start start 2021-04-03T03:14:25.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc40064b0 0x7f3dc403c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc403d360 0x7f3dc40d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc40dbb90 0x7f3dc40eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40a2860 con 0x7f3dc403d360 2021-04-03T03:14:25.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40a2b00 con 0x7f3dc40064b0 2021-04-03T03:14:25.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40a2da0 con 0x7f3dc40dbb90 2021-04-03T03:14:25.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc403d360 0x7f3dc40d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc403d360 0x7f3dc40d7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53744/0 (socket says 172.21.2.111:53744) 2021-04-03T03:14:25.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 -- 172.21.2.111:0/3933866609 learned_addr learned my addr 172.21.2.111:0/3933866609 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd0cc8700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc40064b0 0x7f3dc403c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 -- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc40dbb90 msgr2=0x7f3dc40eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd14c9700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc40dbb90 0x7f3dc40eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc40dbb90 0x7f3dc40eadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 -- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc40064b0 msgr2=0x7f3dc403c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc40064b0 0x7f3dc403c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 -- 172.21.2.111:0/3933866609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40a1720 con 0x7f3dc403d360 2021-04-03T03:14:25.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dcbfff700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc403d360 0x7f3dc40d7a30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f3dbc002f30 tx=0x7f3dbc006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dc9ffb700 1 -- 172.21.2.111:0/3933866609 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dbc00d840 con 0x7f3dc403d360 2021-04-03T03:14:25.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dc9ffb700 1 -- 172.21.2.111:0/3933866609 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc000e90 con 0x7f3dc403d360 2021-04-03T03:14:25.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc40e7550 con 0x7f3dc403d360 2021-04-03T03:14:25.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dc9ffb700 1 -- 172.21.2.111:0/3933866609 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dbc00db90 con 0x7f3dc403d360 2021-04-03T03:14:25.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.156+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc40a25c0 con 0x7f3dc403d360 2021-04-03T03:14:25.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.160+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc4004da0 con 0x7f3dc403d360 2021-04-03T03:14:25.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.160+0000 7f3dc9ffb700 1 -- 172.21.2.111:0/3933866609 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3dbc017030 con 0x7f3dc403d360 2021-04-03T03:14:25.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.160+0000 7f3dc9ffb700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3dac056f20 0x7f3dac0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.160+0000 7f3dc9ffb700 1 -- 172.21.2.111:0/3933866609 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3dbc06f660 con 0x7f3dc403d360 2021-04-03T03:14:25.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.160+0000 7f3dd0cc8700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3dac056f20 0x7f3dac0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.160+0000 7f3dd0cc8700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3dac056f20 0x7f3dac0593e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f3dc001a000 tx=0x7f3dc0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.160+0000 7f3dc9ffb700 1 -- 172.21.2.111:0/3933866609 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3dbc0449f0 con 0x7f3dc403d360 2021-04-03T03:14:25.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.284+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dc40db650 con 0x7f3dc403d360 2021-04-03T03:14:25.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dc9ffb700 1 -- 172.21.2.111:0/3933866609 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f3dbc048010 con 0x7f3dc403d360 2021-04-03T03:14:25.298 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:25.299 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:14:25.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3dac056f20 msgr2=0x7f3dac0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3dac056f20 0x7f3dac0593e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f3dc001a000 tx=0x7f3dc0015000).stop 2021-04-03T03:14:25.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc403d360 msgr2=0x7f3dc40d7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc403d360 0x7f3dc40d7a30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f3dbc002f30 tx=0x7f3dbc006ea0).stop 2021-04-03T03:14:25.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 shutdown_connections 2021-04-03T03:14:25.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc40dbb90 0x7f3dc40eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3dac056f20 0x7f3dac0593e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc40064b0 0x7f3dc403c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 --2- 172.21.2.111:0/3933866609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc403d360 0x7f3dc40d7a30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 >> 172.21.2.111:0/3933866609 conn(0x7f3dc40462c0 msgr2=0x7f3dc40d8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 shutdown_connections 2021-04-03T03:14:25.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.288+0000 7f3dd1cca700 1 -- 172.21.2.111:0/3933866609 wait complete. 2021-04-03T03:14:25.302 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:14:25.306 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:14:25.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 Processor -- start 2021-04-03T03:14:25.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- start start 2021-04-03T03:14:25.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 0x7f78a80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80607d0 con 0x7f78a80b0c90 2021-04-03T03:14:25.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80af400 con 0x7f78a814b700 2021-04-03T03:14:25.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80af6a0 con 0x7f78a80603f0 2021-04-03T03:14:25.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78ad6ec700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f78a80b0c90 0x7f78a8147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54544/0 (socket says 172.21.2.111:54544) 2021-04-03T03:14:25.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78ad6ec700 1 -- 172.21.2.111:0/1943059534 learned_addr learned my addr 172.21.2.111:0/1943059534 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:25.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78aceeb700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 0x7f78a80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060303864 0 0) 0x7f78a80607d0 con 0x7f78a80b0c90 2021-04-03T03:14:25.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7894003240 con 0x7f78a80b0c90 2021-04-03T03:14:25.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3593204059 0 0) 0x7f7894003240 con 0x7f78a80b0c90 2021-04-03T03:14:25.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 msgr2=0x7f78a80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 0x7f78a80b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 >> v1:172.21.2.115:6789/0 conn(0x7f78a814b700 legacy=0x7f78a80b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80afb40 con 0x7f78a80b0c90 2021-04-03T03:14:25.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f78a0001a50 con 0x7f78a80b0c90 2021-04-03T03:14:25.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78aceeb700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 0x7f78a80b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:14:25.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 >> v1:172.21.2.111:6789/0 conn(0x7f78a80b0c90 legacy=0x7f78a8147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7894003760 0x7f7894003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78940044e0 0x7f7894004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 0x7f7894004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80af6a0 con 0x7f78940044e0 2021-04-03T03:14:25.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894005390 con 0x7f7894003760 2021-04-03T03:14:25.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7894004300 con 0x7f78a80603f0 2021-04-03T03:14:25.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a7fff700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78940044e0 0x7f7894004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78aceeb700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7894003760 0x7f7894003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a7fff700 1 -- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 msgr2=0x7f7894004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a7fff700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80603f0 0x7f7894004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a7fff700 1 -- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7894003760 msgr2=0x7f7894003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a7fff700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7894003760 0x7f7894003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a7fff700 1 -- 172.21.2.111:0/1943059534 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80afb40 con 0x7f78940044e0 2021-04-03T03:14:25.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a7fff700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78940044e0 0x7f7894004960 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f78a80b5640 tx=0x7f789c007b70).ready entity=mon.0 client_cookie=7417f341d068d668 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c00c660 con 0x7f78940044e0 2021-04-03T03:14:25.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c00c7c0 con 0x7f78940044e0 2021-04-03T03:14:25.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78a77fe700 1 -- 172.21.2.111:0/1943059534 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c00cb20 con 0x7f78940044e0 2021-04-03T03:14:25.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- 172.21.2.111:0/1943059534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78940044e0 msgr2=0x7f7894004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 --2- 172.21.2.111:0/1943059534 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78940044e0 0x7f7894004960 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f78a80b5640 tx=0x7f789c007b70).stop 2021-04-03T03:14:25.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78ad6ec700 1 -- 172.21.2.111:0/1943059534 reap_dead start 2021-04-03T03:14:25.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- 172.21.2.111:0/1943059534 shutdown_connections 2021-04-03T03:14:25.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- 172.21.2.111:0/1943059534 >> 172.21.2.111:0/1943059534 conn(0x7f78a80b61c0 msgr2=0x7f78a805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- 172.21.2.111:0/1943059534 shutdown_connections 2021-04-03T03:14:25.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.352+0000 7f78adeed700 1 -- 172.21.2.111:0/1943059534 wait complete. 2021-04-03T03:14:25.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 Processor -- start 2021-04-03T03:14:25.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 -- start start 2021-04-03T03:14:25.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80ad620 0x7f78a81469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78a80aba50 0x7f78a8146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78a80aaa60 0x7f78a8063b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80a9ae0 con 0x7f78a80aba50 2021-04-03T03:14:25.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80ada00 con 0x7f78a80aaa60 2021-04-03T03:14:25.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80a98c0 con 0x7f78a80ad620 2021-04-03T03:14:25.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78a80aba50 0x7f78a8146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78a80aba50 0x7f78a8146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53762/0 (socket says 172.21.2.111:53762) 2021-04-03T03:14:25.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 -- 172.21.2.111:0/1928700791 learned_addr learned my addr 172.21.2.111:0/1928700791 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 -- 172.21.2.111:0/1928700791 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80ad620 msgr2=0x7f78a81469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80ad620 0x7f78a81469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 -- 172.21.2.111:0/1928700791 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78a80aaa60 msgr2=0x7f78a8063b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78a80aaa60 0x7f78a8063b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 -- 172.21.2.111:0/1928700791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80a9d10 con 0x7f78a80aba50 2021-04-03T03:14:25.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a7fff700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78a80aba50 0x7f78a8146e90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f78a80b5640 tx=0x7f789c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a5ffb700 1 -- 172.21.2.111:0/1928700791 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c013a60 con 0x7f78a80aba50 2021-04-03T03:14:25.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a5ffb700 1 -- 172.21.2.111:0/1928700791 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c013bc0 con 0x7f78a80aba50 2021-04-03T03:14:25.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a80a9fb0 con 0x7f78a80aba50 2021-04-03T03:14:25.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a5ffb700 1 -- 172.21.2.111:0/1928700791 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c01d460 con 0x7f78a80aba50 2021-04-03T03:14:25.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a81599a0 con 0x7f78a80aba50 2021-04-03T03:14:25.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a8057660 con 0x7f78a80aba50 2021-04-03T03:14:25.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a5ffb700 1 -- 172.21.2.111:0/1928700791 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f789c010030 con 0x7f78a80aba50 2021-04-03T03:14:25.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a5ffb700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7888056ea0 0x7f7888059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78a5ffb700 1 -- 172.21.2.111:0/1928700791 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f789c070d70 con 0x7f78a80aba50 2021-04-03T03:14:25.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78aceeb700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7888056ea0 0x7f7888059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.356+0000 7f78aceeb700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7888056ea0 0x7f7888059360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f789800c000 tx=0x7f7898008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.360+0000 7f78a5ffb700 1 -- 172.21.2.111:0/1928700791 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f789c046160 con 0x7f78a80aba50 2021-04-03T03:14:25.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78a80686d0 con 0x7f78a80aba50 2021-04-03T03:14:25.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78a5ffb700 1 -- 172.21.2.111:0/1928700791 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f789c022030 con 0x7f78a80aba50 2021-04-03T03:14:25.464 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:25.464 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":74,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:14:25.083032+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:14:22.039727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-04-03T03:14:23.053211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:14:25.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7888056ea0 msgr2=0x7f7888059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7888056ea0 0x7f7888059360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f789800c000 tx=0x7f7898008040).stop 2021-04-03T03:14:25.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78a80aba50 msgr2=0x7f78a8146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78a80aba50 0x7f78a8146e90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f78a80b5640 tx=0x7f789c016000).stop 2021-04-03T03:14:25.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 shutdown_connections 2021-04-03T03:14:25.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7888056ea0 0x7f7888059360 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78a80aaa60 0x7f78a8063b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78a80ad620 0x7f78a81469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 --2- 172.21.2.111:0/1928700791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78a80aba50 0x7f78a8146e90 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 >> 172.21.2.111:0/1928700791 conn(0x7f78a80b61c0 msgr2=0x7f78a80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 shutdown_connections 2021-04-03T03:14:25.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.452+0000 7f78adeed700 1 -- 172.21.2.111:0/1928700791 wait complete. 2021-04-03T03:14:25.472 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:14:25.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 Processor -- start 2021-04-03T03:14:25.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- start start 2021-04-03T03:14:25.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fefe40b38c0 0x7fefe4025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4153440 0x7fefe4080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe40b3ca0 con 0x7fefe4080a40 2021-04-03T03:14:25.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe40b0300 con 0x7fefe40b38c0 2021-04-03T03:14:25.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4025970 con 0x7fefe4153440 2021-04-03T03:14:25.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fefe40b38c0 0x7fefe4025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fefe40b38c0 0x7fefe4025480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53770/0 (socket says 172.21.2.111:53770) 2021-04-03T03:14:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 -- 172.21.2.111:0/1691464886 learned_addr learned my addr 172.21.2.111:0/1691464886 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 -- 172.21.2.111:0/1691464886 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4153440 msgr2=0x7fefe4080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 --2- 172.21.2.111:0/1691464886 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4153440 0x7fefe4080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 -- 172.21.2.111:0/1691464886 >> v1:172.21.2.115:6789/0 conn(0x7fefe4080a40 legacy=0x7fefe414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 -- 172.21.2.111:0/1691464886 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4025b90 con 0x7fefe40b38c0 2021-04-03T03:14:25.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe2b85700 1 --2- 172.21.2.111:0/1691464886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe40b38c0 0x7fefe4025480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fefe40b48e0 tx=0x7fefd4006e90).ready entity=mon.0 client_cookie=3176e0c3a161b90c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe1b83700 1 -- 172.21.2.111:0/1691464886 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd40098e0 con 0x7fefe40b38c0 2021-04-03T03:14:25.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe1b83700 1 -- 172.21.2.111:0/1691464886 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4009a40 con 0x7fefe40b38c0 2021-04-03T03:14:25.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe1b83700 1 -- 172.21.2.111:0/1691464886 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd4009de0 con 0x7fefe40b38c0 2021-04-03T03:14:25.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- 172.21.2.111:0/1691464886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe40b38c0 msgr2=0x7fefe4025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 --2- 172.21.2.111:0/1691464886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe40b38c0 0x7fefe4025480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fefe40b48e0 tx=0x7fefd4006e90).stop 2021-04-03T03:14:25.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- 172.21.2.111:0/1691464886 shutdown_connections 2021-04-03T03:14:25.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 --2- 172.21.2.111:0/1691464886 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe40b38c0 0x7fefe4025480 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 --2- 172.21.2.111:0/1691464886 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4153440 0x7fefe4080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- 172.21.2.111:0/1691464886 >> 172.21.2.111:0/1691464886 conn(0x7fefe4061610 msgr2=0x7fefe4061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- 172.21.2.111:0/1691464886 shutdown_connections 2021-04-03T03:14:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.520+0000 7fefe3386700 1 -- 172.21.2.111:0/1691464886 wait complete. 2021-04-03T03:14:25.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 Processor -- start 2021-04-03T03:14:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 -- start start 2021-04-03T03:14:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4080a40 0x7fefe4066990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fefe40b38c0 0x7fefe40649c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe4153440 0x7fefe4152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4065f70 con 0x7fefe4153440 2021-04-03T03:14:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40661d0 con 0x7fefe40b38c0 2021-04-03T03:14:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe41393c0 con 0x7fefe4080a40 2021-04-03T03:14:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe4153440 0x7fefe4152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe4153440 0x7fefe4152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53778/0 (socket says 172.21.2.111:53778) 2021-04-03T03:14:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 -- 172.21.2.111:0/3412653538 learned_addr learned my addr 172.21.2.111:0/3412653538 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe2384700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fefe40b38c0 0x7fefe40649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 -- 172.21.2.111:0/3412653538 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4080a40 msgr2=0x7fefe4066990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4080a40 0x7fefe4066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 -- 172.21.2.111:0/3412653538 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fefe40b38c0 msgr2=0x7fefe40649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fefe40b38c0 0x7fefe40649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 -- 172.21.2.111:0/3412653538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4139860 con 0x7fefe4153440 2021-04-03T03:14:25.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe88c0700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe4153440 0x7fefe4152d60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fefdc0022e0 tx=0x7fefdc007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefdb7fe700 1 -- 172.21.2.111:0/3412653538 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefdc00a7b0 con 0x7fefe4153440 2021-04-03T03:14:25.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefdb7fe700 1 -- 172.21.2.111:0/3412653538 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc00a910 con 0x7fefe4153440 2021-04-03T03:14:25.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe4139b10 con 0x7fefe4153440 2021-04-03T03:14:25.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe4139e90 con 0x7fefe4153440 2021-04-03T03:14:25.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefdb7fe700 1 -- 172.21.2.111:0/3412653538 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefdc00ac70 con 0x7fefe4153440 2021-04-03T03:14:25.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe4057660 con 0x7fefe4153440 2021-04-03T03:14:25.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefdb7fe700 1 -- 172.21.2.111:0/3412653538 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fefdc019030 con 0x7fefe4153440 2021-04-03T03:14:25.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefdb7fe700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefc8056f70 0x7fefc8059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefdb7fe700 1 -- 172.21.2.111:0/3412653538 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fefdc06fc80 con 0x7fefe4153440 2021-04-03T03:14:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe2b85700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefc8056f70 0x7fefc8059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.524+0000 7fefe2b85700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefc8056f70 0x7fefc8059430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fefe40b48e0 tx=0x7fefd4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.528+0000 7fefdb7fe700 1 -- 172.21.2.111:0/3412653538 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fefdc076080 con 0x7fefe4153440 2021-04-03T03:14:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.656+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fefe40b0730 con 0x7fefe4153440 2021-04-03T03:14:25.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.660+0000 7fefdb7fe700 1 -- 172.21.2.111:0/3412653538 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7fefdc048990 con 0x7fefe4153440 2021-04-03T03:14:25.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefc8056f70 msgr2=0x7fefc8059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefc8056f70 0x7fefc8059430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fefe40b48e0 tx=0x7fefd4015000).stop 2021-04-03T03:14:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe4153440 msgr2=0x7fefe4152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe4153440 0x7fefe4152d60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fefdc0022e0 tx=0x7fefdc007f90).stop 2021-04-03T03:14:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 shutdown_connections 2021-04-03T03:14:25.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fefe4153440 0x7fefe4152d60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fefe4080a40 0x7fefe4066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fefc8056f70 0x7fefc8059430 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 --2- 172.21.2.111:0/3412653538 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fefe40b38c0 0x7fefe40649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 >> 172.21.2.111:0/3412653538 conn(0x7fefe4061610 msgr2=0x7fefe40b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 shutdown_connections 2021-04-03T03:14:25.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.664+0000 7fefe3386700 1 -- 172.21.2.111:0/3412653538 wait complete. 2021-04-03T03:14:25.677 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:14:25.681 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:14:25.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 Processor -- start 2021-04-03T03:14:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 -- start start 2021-04-03T03:14:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3028046ce0 0x7f302803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 0x7f30280f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3028041240 con 0x7f302803d350 2021-04-03T03:14:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3028042830 con 0x7f3028046ce0 2021-04-03T03:14:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30280f1850 con 0x7f3028042380 2021-04-03T03:14:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3028046ce0 0x7f302803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3028046ce0 0x7f302803f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53782/0 (socket says 172.21.2.111:53782) 2021-04-03T03:14:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 -- 172.21.2.111:0/3355547201 learned_addr learned my addr 172.21.2.111:0/3355547201 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 -- 172.21.2.111:0/3355547201 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 msgr2=0x7f30280f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 --2- 172.21.2.111:0/3355547201 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 0x7f30280f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 -- 172.21.2.111:0/3355547201 >> v1:172.21.2.115:6789/0 conn(0x7f302803d350 legacy=0x7f3028041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 -- 172.21.2.111:0/3355547201 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280470c0 con 0x7f3028046ce0 2021-04-03T03:14:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302f7fe700 1 --2- 172.21.2.111:0/3355547201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f302803f650 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f302800c2d0 tx=0x7f3020006e90).ready entity=mon.0 client_cookie=d71281fba3e0f3f2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302e7fc700 1 -- 172.21.2.111:0/3355547201 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30200098e0 con 0x7f3028046ce0 2021-04-03T03:14:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302e7fc700 1 -- 172.21.2.111:0/3355547201 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020009a40 con 0x7f3028046ce0 2021-04-03T03:14:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302e7fc700 1 -- 172.21.2.111:0/3355547201 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3020009de0 con 0x7f3028046ce0 2021-04-03T03:14:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 -- 172.21.2.111:0/3355547201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 msgr2=0x7f302803f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.728+0000 7f302ffff700 1 --2- 172.21.2.111:0/3355547201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f302803f650 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f302800c2d0 tx=0x7f3020006e90).stop 2021-04-03T03:14:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- 172.21.2.111:0/3355547201 shutdown_connections 2021-04-03T03:14:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 --2- 172.21.2.111:0/3355547201 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 0x7f30280f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 --2- 172.21.2.111:0/3355547201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f302803f650 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- 172.21.2.111:0/3355547201 >> 172.21.2.111:0/3355547201 conn(0x7f302800d0c0 msgr2=0x7f30280462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- 172.21.2.111:0/3355547201 shutdown_connections 2021-04-03T03:14:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- 172.21.2.111:0/3355547201 wait complete. 2021-04-03T03:14:25.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 Processor -- start 2021-04-03T03:14:25.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- start start 2021-04-03T03:14:25.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f302803d350 0x7f30280f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 0x7f30280fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f3028039ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302803c2a0 con 0x7f3028046ce0 2021-04-03T03:14:25.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3028039700 con 0x7f302803d350 2021-04-03T03:14:25.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30280bb670 con 0x7f3028042380 2021-04-03T03:14:25.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f3028039ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f3028039ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53790/0 (socket says 172.21.2.111:53790) 2021-04-03T03:14:25.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 -- 172.21.2.111:0/1327503725 learned_addr learned my addr 172.21.2.111:0/1327503725 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 -- 172.21.2.111:0/1327503725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 msgr2=0x7f30280fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 0x7f30280fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 -- 172.21.2.111:0/1327503725 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f302803d350 msgr2=0x7f30280f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f302803d350 0x7f30280f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 -- 172.21.2.111:0/1327503725 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280bb830 con 0x7f3028046ce0 2021-04-03T03:14:25.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3035981700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f3028039ba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f303006b5d0 tx=0x7f303005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3017fff700 1 -- 172.21.2.111:0/1327503725 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f303005f2c0 con 0x7f3028046ce0 2021-04-03T03:14:25.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3017fff700 1 -- 172.21.2.111:0/1327503725 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303005f420 con 0x7f3028046ce0 2021-04-03T03:14:25.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30280fb0f0 con 0x7f3028046ce0 2021-04-03T03:14:25.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302803b530 con 0x7f3028046ce0 2021-04-03T03:14:25.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3017fff700 1 -- 172.21.2.111:0/1327503725 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f303005f780 con 0x7f3028046ce0 2021-04-03T03:14:25.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302803c4a0 con 0x7f3028046ce0 2021-04-03T03:14:25.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3017fff700 1 -- 172.21.2.111:0/1327503725 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f303007a020 con 0x7f3028046ce0 2021-04-03T03:14:25.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3017fff700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30100570c0 0x7f3010059580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f3017fff700 1 -- 172.21.2.111:0/1327503725 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f30300cdf30 con 0x7f3028046ce0 2021-04-03T03:14:25.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302f7fe700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30100570c0 0x7f3010059580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.732+0000 7f302f7fe700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30100570c0 0x7f3010059580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f302800c2d0 tx=0x7f3020015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.736+0000 7f3017fff700 1 -- 172.21.2.111:0/1327503725 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f30300a33d0 con 0x7f3028046ce0 2021-04-03T03:14:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.860+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f30280bb830 con 0x7f3028046ce0 2021-04-03T03:14:25.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f3017fff700 1 -- 172.21.2.111:0/1327503725 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f30280bb830 con 0x7f3028046ce0 2021-04-03T03:14:25.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30100570c0 msgr2=0x7f3010059580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30100570c0 0x7f3010059580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f302800c2d0 tx=0x7f3020015000).stop 2021-04-03T03:14:25.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 msgr2=0x7f3028039ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f3028039ba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f303006b5d0 tx=0x7f303005ad70).stop 2021-04-03T03:14:25.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 shutdown_connections 2021-04-03T03:14:25.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3028046ce0 0x7f3028039ba0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30100570c0 0x7f3010059580 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f302803d350 0x7f30280f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 --2- 172.21.2.111:0/1327503725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3028042380 0x7f30280fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 >> 172.21.2.111:0/1327503725 conn(0x7f302800d0c0 msgr2=0x7f302803ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 shutdown_connections 2021-04-03T03:14:25.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.868+0000 7f302ffff700 1 -- 172.21.2.111:0/1327503725 wait complete. 2021-04-03T03:14:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:14:25.887 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:14:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.932+0000 7fde36b83700 1 Processor -- start 2021-04-03T03:14:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- start start 2021-04-03T03:14:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fde28046ce0 0x7fde2803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde2803d350 0x7fde28041c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde28041240 con 0x7fde28042160 2021-04-03T03:14:25.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280f1850 con 0x7fde28046ce0 2021-04-03T03:14:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280470c0 con 0x7fde2803d350 2021-04-03T03:14:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde37ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fde28042160 0x7fde280f51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54592/0 (socket says 172.21.2.111:54592) 2021-04-03T03:14:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde37ffd700 1 -- 172.21.2.111:0/3761915138 learned_addr learned my addr 172.21.2.111:0/3761915138 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:25.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36382700 1 --2- 172.21.2.111:0/3761915138 >> v2:172.21.2.115:3300/0 conn(0x7fde28046ce0 0x7fde2803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde35b81700 1 --2- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde2803d350 0x7fde28041c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde35380700 1 -- 172.21.2.111:0/3761915138 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193419075 0 0) 0x7fde28041240 con 0x7fde28042160 2021-04-03T03:14:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde35380700 1 -- 172.21.2.111:0/3761915138 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde1c003240 con 0x7fde28042160 2021-04-03T03:14:25.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36382700 1 -- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde2803d350 msgr2=0x7fde28041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde35380700 1 -- 172.21.2.111:0/3761915138 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3682058772 0 0) 0x7fde1c003240 con 0x7fde28042160 2021-04-03T03:14:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36382700 1 --2- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde2803d350 0x7fde28041c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36382700 1 -- 172.21.2.111:0/3761915138 >> v1:172.21.2.111:6789/0 conn(0x7fde28042160 legacy=0x7fde280f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36382700 1 -- 172.21.2.111:0/3761915138 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280400d0 con 0x7fde28046ce0 2021-04-03T03:14:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36382700 1 --2- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde28046ce0 0x7fde2803f650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fde2c002f30 tx=0x7fde2c006dd0).ready entity=mon.1 client_cookie=f8c5a5250ca77d02 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde35380700 1 -- 172.21.2.111:0/3761915138 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde300450f0 con 0x7fde28046ce0 2021-04-03T03:14:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde35380700 1 -- 172.21.2.111:0/3761915138 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c0095d0 con 0x7fde28046ce0 2021-04-03T03:14:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde35380700 1 -- 172.21.2.111:0/3761915138 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2c009740 con 0x7fde28046ce0 2021-04-03T03:14:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde28046ce0 msgr2=0x7fde2803f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 --2- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde28046ce0 0x7fde2803f650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fde2c002f30 tx=0x7fde2c006dd0).stop 2021-04-03T03:14:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- 172.21.2.111:0/3761915138 shutdown_connections 2021-04-03T03:14:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 --2- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde28046ce0 0x7fde2803f650 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 --2- 172.21.2.111:0/3761915138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde2803d350 0x7fde28041c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- 172.21.2.111:0/3761915138 >> 172.21.2.111:0/3761915138 conn(0x7fde2800d0c0 msgr2=0x7fde280462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- 172.21.2.111:0/3761915138 shutdown_connections 2021-04-03T03:14:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.936+0000 7fde36b83700 1 -- 172.21.2.111:0/3761915138 wait complete. 2021-04-03T03:14:25.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 Processor -- start 2021-04-03T03:14:25.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 -- start start 2021-04-03T03:14:25.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde2803d350 0x7fde280f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde28042160 0x7fde2809f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde28046ce0 0x7fde280f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280a0b30 con 0x7fde28042160 2021-04-03T03:14:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280a0dd0 con 0x7fde2803d350 2021-04-03T03:14:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280a1070 con 0x7fde28046ce0 2021-04-03T03:14:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde28042160 0x7fde2809f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde28042160 0x7fde2809f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53804/0 (socket says 172.21.2.111:53804) 2021-04-03T03:14:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 -- 172.21.2.111:0/3979239153 learned_addr learned my addr 172.21.2.111:0/3979239153 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36382700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde2803d350 0x7fde280f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 -- 172.21.2.111:0/3979239153 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde28046ce0 msgr2=0x7fde280f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde28046ce0 0x7fde280f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 -- 172.21.2.111:0/3979239153 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde2803d350 msgr2=0x7fde280f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde2803d350 0x7fde280f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 -- 172.21.2.111:0/3979239153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280a1510 con 0x7fde28042160 2021-04-03T03:14:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde35b81700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde28042160 0x7fde2809f5f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fde2400c000 tx=0x7fde24006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde22ffd700 1 -- 172.21.2.111:0/3979239153 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2400cae0 con 0x7fde28042160 2021-04-03T03:14:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde22ffd700 1 -- 172.21.2.111:0/3979239153 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2400cc40 con 0x7fde28042160 2021-04-03T03:14:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde280b11c0 con 0x7fde28042160 2021-04-03T03:14:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde280f0ff0 con 0x7fde28042160 2021-04-03T03:14:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde22ffd700 1 -- 172.21.2.111:0/3979239153 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2400b390 con 0x7fde28042160 2021-04-03T03:14:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde280a1230 con 0x7fde28042160 2021-04-03T03:14:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde22ffd700 1 -- 172.21.2.111:0/3979239153 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fde2401c460 con 0x7fde28042160 2021-04-03T03:14:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde22ffd700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde18057000 0x7fde180594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:25.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde22ffd700 1 -- 172.21.2.111:0/3979239153 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fde24070750 con 0x7fde28042160 2021-04-03T03:14:25.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36382700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde18057000 0x7fde180594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:25.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.940+0000 7fde36382700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde18057000 0x7fde180594c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fde2c017000 tx=0x7fde2c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:25.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:25.944+0000 7fde22ffd700 1 -- 172.21.2.111:0/3979239153 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fde240459d0 con 0x7fde28042160 2021-04-03T03:14:26.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.068+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fde280a1700 con 0x7fde28042160 2021-04-03T03:14:26.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.072+0000 7fde22ffd700 1 -- 172.21.2.111:0/3979239153 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7fde24048ff0 con 0x7fde28042160 2021-04-03T03:14:26.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde18057000 msgr2=0x7fde180594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde18057000 0x7fde180594c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fde2c017000 tx=0x7fde2c015000).stop 2021-04-03T03:14:26.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde28042160 msgr2=0x7fde2809f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde28042160 0x7fde2809f5f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fde2400c000 tx=0x7fde24006dd0).stop 2021-04-03T03:14:26.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 shutdown_connections 2021-04-03T03:14:26.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde28046ce0 0x7fde280f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde18057000 0x7fde180594c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde2803d350 0x7fde280f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 --2- 172.21.2.111:0/3979239153 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde28042160 0x7fde2809f5f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 >> 172.21.2.111:0/3979239153 conn(0x7fde2800d0c0 msgr2=0x7fde280419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 shutdown_connections 2021-04-03T03:14:26.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.076+0000 7fde36b83700 1 -- 172.21.2.111:0/3979239153 wait complete. 2021-04-03T03:14:26.090 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:14:26.095 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:26.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.136+0000 7fa822ad1700 1 Processor -- start 2021-04-03T03:14:26.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.136+0000 7fa822ad1700 1 -- start start 2021-04-03T03:14:26.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa81c14b700 0x7fa81c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c0af400 con 0x7fa81c0603f0 2021-04-03T03:14:26.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c0af6a0 con 0x7fa81c0b0c90 2021-04-03T03:14:26.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c060860 con 0x7fa81c14b700 2021-04-03T03:14:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa821acf700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa81c0603f0 0x7fa81c0b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54602/0 (socket says 172.21.2.111:54602) 2021-04-03T03:14:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa821acf700 1 -- 172.21.2.111:0/861095412 learned_addr learned my addr 172.21.2.111:0/861095412 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 --2- 172.21.2.111:0/861095412 >> v2:172.21.2.115:3300/0 conn(0x7fa81c14b700 0x7fa81c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481072684 0 0) 0x7fa81c0af400 con 0x7fa81c0603f0 2021-04-03T03:14:26.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa804003240 con 0x7fa81c0603f0 2021-04-03T03:14:26.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2557321658 0 0) 0x7fa804003240 con 0x7fa81c0603f0 2021-04-03T03:14:26.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 >> v2:172.21.2.115:3300/0 conn(0x7fa81c14b700 msgr2=0x7fa81c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 --2- 172.21.2.111:0/861095412 >> v2:172.21.2.115:3300/0 conn(0x7fa81c14b700 0x7fa81c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 >> v1:172.21.2.115:6789/0 conn(0x7fa81c0b0c90 legacy=0x7fa81c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0afb40 con 0x7fa81c0603f0 2021-04-03T03:14:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa8180016d0 con 0x7fa81c0603f0 2021-04-03T03:14:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 >> v1:172.21.2.111:6789/0 conn(0x7fa81c0603f0 legacy=0x7fa81c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa804003780 0x7fa804003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8040044e0 0x7fa804004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa804004e50 0x7fa80400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c060860 con 0x7fa8040044e0 2021-04-03T03:14:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804005300 con 0x7fa804004e50 2021-04-03T03:14:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80400bc20 con 0x7fa804003780 2021-04-03T03:14:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8040044e0 0x7fa804004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa821acf700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa804003780 0x7fa804003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8222d0700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa804004e50 0x7fa80400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 -- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa804003780 msgr2=0x7fa804003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa804003780 0x7fa804003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 -- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa804004e50 msgr2=0x7fa80400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa804004e50 0x7fa80400b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 -- 172.21.2.111:0/861095412 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa804003240 con 0x7fa8040044e0 2021-04-03T03:14:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8222d0700 1 -- 172.21.2.111:0/861095412 reap_dead start 2021-04-03T03:14:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa8212ce700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8040044e0 0x7fa804004960 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fa81000c000 tx=0x7fa810008040).ready entity=mon.0 client_cookie=873286ccef98347b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81000e910 con 0x7fa8040044e0 2021-04-03T03:14:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81000ea70 con 0x7fa8040044e0 2021-04-03T03:14:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa820acd700 1 -- 172.21.2.111:0/861095412 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81000edd0 con 0x7fa8040044e0 2021-04-03T03:14:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- 172.21.2.111:0/861095412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8040044e0 msgr2=0x7fa804004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8040044e0 0x7fa804004960 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fa81000c000 tx=0x7fa810008040).stop 2021-04-03T03:14:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- 172.21.2.111:0/861095412 shutdown_connections 2021-04-03T03:14:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 --2- 172.21.2.111:0/861095412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8040044e0 0x7fa804004960 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- 172.21.2.111:0/861095412 >> 172.21.2.111:0/861095412 conn(0x7fa81c0b61c0 msgr2=0x7fa81c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- 172.21.2.111:0/861095412 shutdown_connections 2021-04-03T03:14:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.140+0000 7fa822ad1700 1 -- 172.21.2.111:0/861095412 wait complete. 2021-04-03T03:14:26.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 Processor -- start 2021-04-03T03:14:26.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 -- start start 2021-04-03T03:14:26.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8040044e0 0x7fa81c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa81c0a9570 0x7fa81c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa81c156d50 0x7fa81c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0639a0 con 0x7fa81c156d50 2021-04-03T03:14:26.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c1566b0 con 0x7fa81c0a9570 2021-04-03T03:14:26.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c1568f0 con 0x7fa8040044e0 2021-04-03T03:14:26.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa81c156d50 0x7fa81c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa81c156d50 0x7fa81c1571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53826/0 (socket says 172.21.2.111:53826) 2021-04-03T03:14:26.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 -- 172.21.2.111:0/2057523228 learned_addr learned my addr 172.21.2.111:0/2057523228 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:26.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8212ce700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa81c0a9570 0x7fa81c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa821acf700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8040044e0 0x7fa81c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 -- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8040044e0 msgr2=0x7fa81c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8040044e0 0x7fa81c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 -- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa81c0a9570 msgr2=0x7fa81c14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa81c0a9570 0x7fa81c14c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 -- 172.21.2.111:0/2057523228 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c063700 con 0x7fa81c156d50 2021-04-03T03:14:26.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa8222d0700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa81c156d50 0x7fa81c1571e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7fa81c0b5600 tx=0x7fa814009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa80effd700 1 -- 172.21.2.111:0/2057523228 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81400f510 con 0x7fa81c156d50 2021-04-03T03:14:26.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa80effd700 1 -- 172.21.2.111:0/2057523228 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81400f670 con 0x7fa81c156d50 2021-04-03T03:14:26.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c15a1a0 con 0x7fa81c156d50 2021-04-03T03:14:26.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa80effd700 1 -- 172.21.2.111:0/2057523228 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81400f9d0 con 0x7fa81c156d50 2021-04-03T03:14:26.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c14c6c0 con 0x7fa81c156d50 2021-04-03T03:14:26.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c057660 con 0x7fa81c156d50 2021-04-03T03:14:26.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa80effd700 1 -- 172.21.2.111:0/2057523228 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa81400fb30 con 0x7fa81c156d50 2021-04-03T03:14:26.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa80effd700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa7fc056ef0 0x7fa7fc0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa80effd700 1 -- 172.21.2.111:0/2057523228 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa81406fff0 con 0x7fa81c156d50 2021-04-03T03:14:26.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa821acf700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa7fc056ef0 0x7fa7fc0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.144+0000 7fa821acf700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa7fc056ef0 0x7fa7fc0593b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fa81800d000 tx=0x7fa818009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.148+0000 7fa80effd700 1 -- 172.21.2.111:0/2057523228 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa814042c50 con 0x7fa81c156d50 2021-04-03T03:14:26.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.272+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa81c148810 con 0x7fa81c156d50 2021-04-03T03:14:26.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.272+0000 7fa80effd700 1 -- 172.21.2.111:0/2057523228 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fa814045330 con 0x7fa81c156d50 2021-04-03T03:14:26.285 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:26.286 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:14:26.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa7fc056ef0 msgr2=0x7fa7fc0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa7fc056ef0 0x7fa7fc0593b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fa81800d000 tx=0x7fa818009040).stop 2021-04-03T03:14:26.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa81c156d50 msgr2=0x7fa81c1571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa81c156d50 0x7fa81c1571e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7fa81c0b5600 tx=0x7fa814009040).stop 2021-04-03T03:14:26.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 shutdown_connections 2021-04-03T03:14:26.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa81c156d50 0x7fa81c1571e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8040044e0 0x7fa81c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa7fc056ef0 0x7fa7fc0593b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 --2- 172.21.2.111:0/2057523228 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa81c0a9570 0x7fa81c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 >> 172.21.2.111:0/2057523228 conn(0x7fa81c0b61c0 msgr2=0x7fa81c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 shutdown_connections 2021-04-03T03:14:26.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.276+0000 7fa822ad1700 1 -- 172.21.2.111:0/2057523228 wait complete. 2021-04-03T03:14:26.289 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:14:26.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:14:24.079081+0000', 'modified': '2021-04-03T03:14:25.081940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6210': {'gid': 6210, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/560115150', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 560115150}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 560115150}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:14:26.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:14:26.496 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:14:26.570 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6210,"want_state":"up:active","state":"up:active","rank_uptime":2.474810019,"mdsmap_epoch":29,"osdmap_epoch":74,"osdmap_epoch_barrier":73,"uptime":6.6181446900000003} 2021-04-03T03:14:26.570 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6210, "mdsmap_epoch": 29, "osdmap_epoch": 74, "osdmap_epoch_barrier": 73, "rank_uptime": 2.474810019, "state": "up:active", "uptime": 6.61814469, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:14:26.571 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:14:26.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.624+0000 7f5aa62ba700 1 Processor -- start 2021-04-03T03:14:26.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.624+0000 7f5aa62ba700 1 -- start start 2021-04-03T03:14:26.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa00603f0 0x7f5aa00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5aa014b700 0x7f5aa00b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa00607d0 con 0x7f5aa00b12b0 2021-04-03T03:14:26.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa00af400 con 0x7f5aa014b700 2021-04-03T03:14:26.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00af6a0 con 0x7f5aa00603f0 2021-04-03T03:14:26.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa5ab9700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5aa00b12b0 0x7f5aa01465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54624/0 (socket says 172.21.2.111:54624) 2021-04-03T03:14:26.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa5ab9700 1 -- 172.21.2.111:0/3436589749 learned_addr learned my addr 172.21.2.111:0/3436589749 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:26.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa4ab7700 1 --2- 172.21.2.111:0/3436589749 >> v2:172.21.2.111:3300/0 conn(0x7f5aa014b700 0x7f5aa00b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a93fff700 1 -- 172.21.2.111:0/3436589749 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124362145 0 0) 0x7f5aa00607d0 con 0x7f5aa00b12b0 2021-04-03T03:14:26.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a93fff700 1 -- 172.21.2.111:0/3436589749 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a88003240 con 0x7f5aa00b12b0 2021-04-03T03:14:26.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa4ab7700 1 -- 172.21.2.111:0/3436589749 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa00603f0 msgr2=0x7f5aa00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa4ab7700 1 --2- 172.21.2.111:0/3436589749 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa00603f0 0x7f5aa00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa4ab7700 1 -- 172.21.2.111:0/3436589749 >> v1:172.21.2.111:6789/0 conn(0x7f5aa00b12b0 legacy=0x7f5aa01465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa4ab7700 1 -- 172.21.2.111:0/3436589749 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00afb40 con 0x7f5aa014b700 2021-04-03T03:14:26.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa4ab7700 1 --2- 172.21.2.111:0/3436589749 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa014b700 0x7f5aa00b0580 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f5a94002f30 tx=0x7f5a94006dd0).ready entity=mon.0 client_cookie=2e178787dd2877e4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a93fff700 1 -- 172.21.2.111:0/3436589749 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a94009760 con 0x7f5aa014b700 2021-04-03T03:14:26.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a93fff700 1 -- 172.21.2.111:0/3436589749 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a940098c0 con 0x7f5aa014b700 2021-04-03T03:14:26.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a93fff700 1 -- 172.21.2.111:0/3436589749 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a94009c20 con 0x7f5aa014b700 2021-04-03T03:14:26.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3436589749 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa014b700 msgr2=0x7f5aa00b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3436589749 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa014b700 0x7f5aa00b0580 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f5a94002f30 tx=0x7f5a94006dd0).stop 2021-04-03T03:14:26.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3436589749 shutdown_connections 2021-04-03T03:14:26.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3436589749 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa00603f0 0x7f5aa00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3436589749 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa014b700 0x7f5aa00b0580 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3436589749 >> 172.21.2.111:0/3436589749 conn(0x7f5aa00b61c0 msgr2=0x7f5aa005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3436589749 shutdown_connections 2021-04-03T03:14:26.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3436589749 wait complete. 2021-04-03T03:14:26.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 Processor -- start 2021-04-03T03:14:26.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- start start 2021-04-03T03:14:26.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa00603f0 0x7f5aa00a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aa00b12b0 0x7f5aa0064860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa014b700 0x7f5aa0146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0062ad0 con 0x7f5aa00603f0 2021-04-03T03:14:26.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00ab500 con 0x7f5aa00b12b0 2021-04-03T03:14:26.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00ab7a0 con 0x7f5aa014b700 2021-04-03T03:14:26.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa00603f0 0x7f5aa00a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa00603f0 0x7f5aa00a9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53838/0 (socket says 172.21.2.111:53838) 2021-04-03T03:14:26.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 -- 172.21.2.111:0/3125894848 learned_addr learned my addr 172.21.2.111:0/3125894848 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:26.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 -- 172.21.2.111:0/3125894848 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa014b700 msgr2=0x7f5aa0146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa014b700 0x7f5aa0146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 -- 172.21.2.111:0/3125894848 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aa00b12b0 msgr2=0x7f5aa0064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aa00b12b0 0x7f5aa0064860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 -- 172.21.2.111:0/3125894848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00aba40 con 0x7f5aa00603f0 2021-04-03T03:14:26.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa52b8700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa00603f0 0x7f5aa00a9e20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f5a9c002f30 tx=0x7f5a9c006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a927fc700 1 -- 172.21.2.111:0/3125894848 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9c00d840 con 0x7f5aa00603f0 2021-04-03T03:14:26.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a927fc700 1 -- 172.21.2.111:0/3125894848 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c000e90 con 0x7f5aa00603f0 2021-04-03T03:14:26.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5a927fc700 1 -- 172.21.2.111:0/3125894848 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9c00db90 con 0x7f5aa00603f0 2021-04-03T03:14:26.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa00aac20 con 0x7f5aa00603f0 2021-04-03T03:14:26.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.628+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa0147260 con 0x7f5aa00603f0 2021-04-03T03:14:26.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.632+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa00abe00 con 0x7f5aa00603f0 2021-04-03T03:14:26.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.632+0000 7f5a927fc700 1 -- 172.21.2.111:0/3125894848 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5a9c017030 con 0x7f5aa00603f0 2021-04-03T03:14:26.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.632+0000 7f5a927fc700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5a80056ea0 0x7f5a80059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.632+0000 7f5a927fc700 1 -- 172.21.2.111:0/3125894848 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5a9c06f5e0 con 0x7f5aa00603f0 2021-04-03T03:14:26.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.632+0000 7f5aa4ab7700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5a80056ea0 0x7f5a80059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.632+0000 7f5aa4ab7700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5a80056ea0 0x7f5a80059360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f5a94002f30 tx=0x7f5a94015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.632+0000 7f5a927fc700 1 -- 172.21.2.111:0/3125894848 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5a9c044a80 con 0x7f5aa00603f0 2021-04-03T03:14:26.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aa00abbd0 con 0x7f5aa00603f0 2021-04-03T03:14:26.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5a927fc700 1 -- 172.21.2.111:0/3125894848 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f5a9c0480a0 con 0x7f5aa00603f0 2021-04-03T03:14:26.740 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:26.740 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":74,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:14:25.083032+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:14:22.039727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-04-03T03:14:23.053211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:14:26.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5a80056ea0 msgr2=0x7f5a80059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5a80056ea0 0x7f5a80059360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f5a94002f30 tx=0x7f5a94015000).stop 2021-04-03T03:14:26.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa00603f0 msgr2=0x7f5aa00a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa00603f0 0x7f5aa00a9e20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f5a9c002f30 tx=0x7f5a9c006ea0).stop 2021-04-03T03:14:26.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 shutdown_connections 2021-04-03T03:14:26.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aa014b700 0x7f5aa0146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aa00603f0 0x7f5aa00a9e20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5a80056ea0 0x7f5a80059360 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 --2- 172.21.2.111:0/3125894848 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aa00b12b0 0x7f5aa0064860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 >> 172.21.2.111:0/3125894848 conn(0x7f5aa00b61c0 msgr2=0x7f5aa00b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 shutdown_connections 2021-04-03T03:14:26.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.728+0000 7f5aa62ba700 1 -- 172.21.2.111:0/3125894848 wait complete. 2021-04-03T03:14:26.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:14:22.039727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:14:23.053211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:14:26.749 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:14:26.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 Processor -- start 2021-04-03T03:14:26.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- start start 2021-04-03T03:14:26.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f44200603f0 0x7f44201447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 0x7f4420144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f44200b17b0 0x7f4420147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44201451c0 con 0x7f44200603f0 2021-04-03T03:14:26.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44200607d0 con 0x7f44200b17b0 2021-04-03T03:14:26.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442014b090 con 0x7f44200add50 2021-04-03T03:14:26.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f44200603f0 0x7f44201447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f44200603f0 0x7f44201447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53844/0 (socket says 172.21.2.111:53844) 2021-04-03T03:14:26.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 -- 172.21.2.111:0/4207978102 learned_addr learned my addr 172.21.2.111:0/4207978102 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:26.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 -- 172.21.2.111:0/4207978102 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 msgr2=0x7f4420144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 --2- 172.21.2.111:0/4207978102 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 0x7f4420144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 -- 172.21.2.111:0/4207978102 >> v2:172.21.2.115:3300/0 conn(0x7f44200b17b0 msgr2=0x7f4420147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 --2- 172.21.2.111:0/4207978102 >> v2:172.21.2.115:3300/0 conn(0x7f44200b17b0 0x7f4420147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 -- 172.21.2.111:0/4207978102 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442014b330 con 0x7f44200603f0 2021-04-03T03:14:26.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441e59c700 1 --2- 172.21.2.111:0/4207978102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44201447e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f4410002f30 tx=0x7f4410006e90).ready entity=mon.0 client_cookie=4b2546b54bf662ff server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441d59a700 1 -- 172.21.2.111:0/4207978102 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4410009920 con 0x7f44200603f0 2021-04-03T03:14:26.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441d59a700 1 -- 172.21.2.111:0/4207978102 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410009a80 con 0x7f44200603f0 2021-04-03T03:14:26.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441d59a700 1 -- 172.21.2.111:0/4207978102 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4410009de0 con 0x7f44200603f0 2021-04-03T03:14:26.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- 172.21.2.111:0/4207978102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 msgr2=0x7f44201447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 --2- 172.21.2.111:0/4207978102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44201447e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f4410002f30 tx=0x7f4410006e90).stop 2021-04-03T03:14:26.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- 172.21.2.111:0/4207978102 shutdown_connections 2021-04-03T03:14:26.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 --2- 172.21.2.111:0/4207978102 >> v2:172.21.2.115:3300/0 conn(0x7f44200b17b0 0x7f4420147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 --2- 172.21.2.111:0/4207978102 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44201447e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 --2- 172.21.2.111:0/4207978102 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 0x7f4420144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- 172.21.2.111:0/4207978102 >> 172.21.2.111:0/4207978102 conn(0x7f44200b61c0 msgr2=0x7f442005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- 172.21.2.111:0/4207978102 shutdown_connections 2021-04-03T03:14:26.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- 172.21.2.111:0/4207978102 wait complete. 2021-04-03T03:14:26.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 Processor -- start 2021-04-03T03:14:26.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.796+0000 7f441f59e700 1 -- start start 2021-04-03T03:14:26.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44200b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 0x7f4420146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44200b17b0 0x7f4420159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200aa810 con 0x7f44200603f0 2021-04-03T03:14:26.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4420146340 con 0x7f44200b17b0 2021-04-03T03:14:26.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44201465e0 con 0x7f44200add50 2021-04-03T03:14:26.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44200b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44200b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53848/0 (socket says 172.21.2.111:53848) 2021-04-03T03:14:26.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 -- 172.21.2.111:0/2758525272 learned_addr learned my addr 172.21.2.111:0/2758525272 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:26.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 -- 172.21.2.111:0/2758525272 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 msgr2=0x7f4420146a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:14:26.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 0x7f4420146a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 -- 172.21.2.111:0/2758525272 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44200b17b0 msgr2=0x7f4420159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44200b17b0 0x7f4420159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 -- 172.21.2.111:0/2758525272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420067db0 con 0x7f44200603f0 2021-04-03T03:14:26.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441e59c700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44200b3840 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f4410002f30 tx=0x7f4410015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f44177fe700 1 -- 172.21.2.111:0/2758525272 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4410011e00 con 0x7f44200603f0 2021-04-03T03:14:26.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f44177fe700 1 -- 172.21.2.111:0/2758525272 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441001e460 con 0x7f44200603f0 2021-04-03T03:14:26.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4420067f40 con 0x7f44200603f0 2021-04-03T03:14:26.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f44177fe700 1 -- 172.21.2.111:0/2758525272 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f441001e7c0 con 0x7f44200603f0 2021-04-03T03:14:26.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44200aa580 con 0x7f44200603f0 2021-04-03T03:14:26.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420147290 con 0x7f44200603f0 2021-04-03T03:14:26.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f44177fe700 1 -- 172.21.2.111:0/2758525272 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4410025030 con 0x7f44200603f0 2021-04-03T03:14:26.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f44177fe700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f44177fe700 1 -- 172.21.2.111:0/2758525272 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f4410071710 con 0x7f44200603f0 2021-04-03T03:14:26.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441dd9b700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.800+0000 7f441dd9b700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f4408002f30 tx=0x7f44080069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.804+0000 7f44177fe700 1 -- 172.21.2.111:0/2758525272 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f44100469f0 con 0x7f44200603f0 2021-04-03T03:14:26.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4420057660 con 0x7f44200603f0 2021-04-03T03:14:26.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f44177fe700 1 -- 172.21.2.111:0/2758525272 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7f44100230b0 con 0x7f44200603f0 2021-04-03T03:14:26.909 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:14:26.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 msgr2=0x7f4404059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f4408002f30 tx=0x7f44080069f0).stop 2021-04-03T03:14:26.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 msgr2=0x7f44200b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44200b3840 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f4410002f30 tx=0x7f4410015000).stop 2021-04-03T03:14:26.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 shutdown_connections 2021-04-03T03:14:26.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4404056fb0 0x7f4404059470 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f44200b17b0 0x7f4420159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44200603f0 0x7f44200b3840 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 --2- 172.21.2.111:0/2758525272 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44200add50 0x7f4420146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.896+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 >> 172.21.2.111:0/2758525272 conn(0x7f44200b61c0 msgr2=0x7f4420143f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.900+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 shutdown_connections 2021-04-03T03:14:26.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.900+0000 7f441f59e700 1 -- 172.21.2.111:0/2758525272 wait complete. 2021-04-03T03:14:26.916 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:14:26.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 Processor -- start 2021-04-03T03:14:26.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- start start 2021-04-03T03:14:26.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f78b405f9c0 0x7f78b405fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f78b40adae0 0x7f78b414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b40b1cf0 con 0x7f78b4060650 2021-04-03T03:14:26.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b40ad7c0 con 0x7f78b40adae0 2021-04-03T03:14:26.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b40ae160 con 0x7f78b405f9c0 2021-04-03T03:14:26.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f78b40adae0 0x7f78b414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f78b40adae0 0x7f78b414b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53856/0 (socket says 172.21.2.111:53856) 2021-04-03T03:14:26.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 -- 172.21.2.111:0/4011257667 learned_addr learned my addr 172.21.2.111:0/4011257667 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:26.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b2ffd700 1 --2- 172.21.2.111:0/4011257667 >> v2:172.21.2.115:3300/0 conn(0x7f78b405f9c0 0x7f78b405fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 -- 172.21.2.111:0/4011257667 >> v2:172.21.2.115:3300/0 conn(0x7f78b405f9c0 msgr2=0x7f78b405fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 --2- 172.21.2.111:0/4011257667 >> v2:172.21.2.115:3300/0 conn(0x7f78b405f9c0 0x7f78b405fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 -- 172.21.2.111:0/4011257667 >> v1:172.21.2.115:6789/0 conn(0x7f78b4060650 legacy=0x7f78b4146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:26.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 -- 172.21.2.111:0/4011257667 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b41475a0 con 0x7f78b40adae0 2021-04-03T03:14:26.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b27fc700 1 --2- 172.21.2.111:0/4011257667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b414b8c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f78b4062690 tx=0x7f78a4006e90).ready entity=mon.0 client_cookie=87e253e5985d07c4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b1ffb700 1 -- 172.21.2.111:0/4011257667 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a4009930 con 0x7f78b40adae0 2021-04-03T03:14:26.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b1ffb700 1 -- 172.21.2.111:0/4011257667 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a4009a90 con 0x7f78b40adae0 2021-04-03T03:14:26.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b1ffb700 1 -- 172.21.2.111:0/4011257667 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a4009e30 con 0x7f78b40adae0 2021-04-03T03:14:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- 172.21.2.111:0/4011257667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 msgr2=0x7f78b414b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- 172.21.2.111:0/4011257667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b414b8c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f78b4062690 tx=0x7f78a4006e90).stop 2021-04-03T03:14:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- 172.21.2.111:0/4011257667 shutdown_connections 2021-04-03T03:14:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- 172.21.2.111:0/4011257667 >> v2:172.21.2.115:3300/0 conn(0x7f78b405f9c0 0x7f78b405fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- 172.21.2.111:0/4011257667 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b414b8c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- 172.21.2.111:0/4011257667 >> 172.21.2.111:0/4011257667 conn(0x7f78b40b5fb0 msgr2=0x7f78b40b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:26.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- 172.21.2.111:0/4011257667 shutdown_connections 2021-04-03T03:14:26.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- 172.21.2.111:0/4011257667 wait complete. 2021-04-03T03:14:26.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 Processor -- start 2021-04-03T03:14:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- start start 2021-04-03T03:14:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78b405f9c0 0x7f78b40a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78b4060650 0x7f78b4065ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b4143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40a8710 con 0x7f78b40adae0 2021-04-03T03:14:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4065260 con 0x7f78b405f9c0 2021-04-03T03:14:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.964+0000 7f78b3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4067280 con 0x7f78b4060650 2021-04-03T03:14:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b4143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b4143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53866/0 (socket says 172.21.2.111:53866) 2021-04-03T03:14:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 -- 172.21.2.111:0/1318590040 learned_addr learned my addr 172.21.2.111:0/1318590040 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 -- 172.21.2.111:0/1318590040 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78b4060650 msgr2=0x7f78b4065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b2ffd700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78b405f9c0 0x7f78b40a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78b4060650 0x7f78b4065ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 -- 172.21.2.111:0/1318590040 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78b405f9c0 msgr2=0x7f78b40a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78b405f9c0 0x7f78b40a76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 -- 172.21.2.111:0/1318590040 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40a9d50 con 0x7f78b40adae0 2021-04-03T03:14:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b37fe700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b4143960 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f78ac0032d0 tx=0x7f78ac006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78abfff700 1 -- 172.21.2.111:0/1318590040 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ac00c960 con 0x7f78b40adae0 2021-04-03T03:14:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78abfff700 1 -- 172.21.2.111:0/1318590040 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac00cac0 con 0x7f78b40adae0 2021-04-03T03:14:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78abfff700 1 -- 172.21.2.111:0/1318590040 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ac00ce20 con 0x7f78b40adae0 2021-04-03T03:14:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b40664e0 con 0x7f78b40adae0 2021-04-03T03:14:26.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b40a6f20 con 0x7f78b40adae0 2021-04-03T03:14:26.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b40a9b90 con 0x7f78b40adae0 2021-04-03T03:14:26.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78abfff700 1 -- 172.21.2.111:0/1318590040 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f78ac01c070 con 0x7f78b40adae0 2021-04-03T03:14:26.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78abfff700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78980570a0 0x7f7898059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:26.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78abfff700 1 -- 172.21.2.111:0/1318590040 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f78ac06fe90 con 0x7f78b40adae0 2021-04-03T03:14:26.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b2ffd700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78980570a0 0x7f7898059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:26.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.968+0000 7f78b2ffd700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78980570a0 0x7f7898059560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f789c008000 tx=0x7f789c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:26.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:26.972+0000 7f78abfff700 1 -- 172.21.2.111:0/1318590040 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f78ac045170 con 0x7f78b40adae0 2021-04-03T03:14:27.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.060+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f78b40aa0d0 con 0x7f78b40adae0 2021-04-03T03:14:27.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.060+0000 7f78abfff700 1 -- 172.21.2.111:0/1318590040 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f78ac048790 con 0x7f78b40adae0 2021-04-03T03:14:27.075 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:14:27.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78980570a0 msgr2=0x7f7898059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78980570a0 0x7f7898059560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f789c008000 tx=0x7f789c008040).stop 2021-04-03T03:14:27.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 msgr2=0x7f78b4143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b4143960 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f78ac0032d0 tx=0x7f78ac006f90).stop 2021-04-03T03:14:27.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 shutdown_connections 2021-04-03T03:14:27.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78b40adae0 0x7f78b4143960 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78980570a0 0x7f7898059560 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78b405f9c0 0x7f78b40a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 --2- 172.21.2.111:0/1318590040 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78b4060650 0x7f78b4065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 >> 172.21.2.111:0/1318590040 conn(0x7f78b40b5fb0 msgr2=0x7f78b40af300 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:27.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 shutdown_connections 2021-04-03T03:14:27.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.064+0000 7f78b3fff700 1 -- 172.21.2.111:0/1318590040 wait complete. 2021-04-03T03:14:27.082 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:14:27.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.128+0000 7fbcf9ffe700 1 Processor -- start 2021-04-03T03:14:27.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.128+0000 7fbcf9ffe700 1 -- start start 2021-04-03T03:14:27.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbcec0dbba0 0x7fbcec0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbcec041060 0x7fbcec03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0dc560 con 0x7fbcec03d560 2021-04-03T03:14:27.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec03de60 con 0x7fbcec0dbba0 2021-04-03T03:14:27.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec0dc800 con 0x7fbcec041060 2021-04-03T03:14:27.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf8ffc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbcec0dbba0 0x7fbcec0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf97fd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fbcec03d560 0x7fbcec041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54664/0 (socket says 172.21.2.111:54664) 2021-04-03T03:14:27.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf97fd700 1 -- 172.21.2.111:0/2387129746 learned_addr learned my addr 172.21.2.111:0/2387129746 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:27.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf8ffc700 1 -- 172.21.2.111:0/2387129746 >> v2:172.21.2.115:3300/0 conn(0x7fbcec041060 msgr2=0x7fbcec03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:27.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf8ffc700 1 --2- 172.21.2.111:0/2387129746 >> v2:172.21.2.115:3300/0 conn(0x7fbcec041060 0x7fbcec03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf8ffc700 1 -- 172.21.2.111:0/2387129746 >> v1:172.21.2.111:6789/0 conn(0x7fbcec03d560 legacy=0x7fbcec041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf8ffc700 1 -- 172.21.2.111:0/2387129746 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec03da30 con 0x7fbcec0dbba0 2021-04-03T03:14:27.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf8ffc700 1 --2- 172.21.2.111:0/2387129746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec0dbba0 0x7fbcec0dbf80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fbce4002f30 tx=0x7fbce4006dd0).ready entity=mon.0 client_cookie=14e96205789f7d31 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf37fe700 1 -- 172.21.2.111:0/2387129746 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce40097f0 con 0x7fbcec0dbba0 2021-04-03T03:14:27.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf37fe700 1 -- 172.21.2.111:0/2387129746 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce4009950 con 0x7fbcec0dbba0 2021-04-03T03:14:27.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf37fe700 1 -- 172.21.2.111:0/2387129746 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce4009cb0 con 0x7fbcec0dbba0 2021-04-03T03:14:27.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/2387129746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec0dbba0 msgr2=0x7fbcec0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/2387129746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec0dbba0 0x7fbcec0dbf80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fbce4002f30 tx=0x7fbce4006dd0).stop 2021-04-03T03:14:27.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/2387129746 shutdown_connections 2021-04-03T03:14:27.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/2387129746 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec0dbba0 0x7fbcec0dbf80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/2387129746 >> v2:172.21.2.115:3300/0 conn(0x7fbcec041060 0x7fbcec03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/2387129746 >> 172.21.2.111:0/2387129746 conn(0x7fbcec0462b0 msgr2=0x7fbcec0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:27.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/2387129746 shutdown_connections 2021-04-03T03:14:27.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/2387129746 wait complete. 2021-04-03T03:14:27.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 Processor -- start 2021-04-03T03:14:27.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- start start 2021-04-03T03:14:27.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbcec03d560 0x7fbcec03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec041060 0x7fbcec0de110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbcec0dbba0 0x7fbcec03a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0e7270 con 0x7fbcec041060 2021-04-03T03:14:27.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0a0280 con 0x7fbcec03d560 2021-04-03T03:14:27.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec03c090 con 0x7fbcec0dbba0 2021-04-03T03:14:27.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec041060 0x7fbcec0de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec041060 0x7fbcec0de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53878/0 (socket says 172.21.2.111:53878) 2021-04-03T03:14:27.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 -- 172.21.2.111:0/1151092653 learned_addr learned my addr 172.21.2.111:0/1151092653 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:27.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf8ffc700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbcec03d560 0x7fbcec03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 -- 172.21.2.111:0/1151092653 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbcec0dbba0 msgr2=0x7fbcec03a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbcec0dbba0 0x7fbcec03a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 -- 172.21.2.111:0/1151092653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbcec03d560 msgr2=0x7fbcec03b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbcec03d560 0x7fbcec03b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 -- 172.21.2.111:0/1151092653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0e7460 con 0x7fbcec041060 2021-04-03T03:14:27.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf3fff700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec041060 0x7fbcec0de110 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fbce8002f30 tx=0x7fbce8006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf1ffb700 1 -- 172.21.2.111:0/1151092653 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce800d770 con 0x7fbcec041060 2021-04-03T03:14:27.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf1ffb700 1 -- 172.21.2.111:0/1151092653 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8000e90 con 0x7fbcec041060 2021-04-03T03:14:27.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec0e7700 con 0x7fbcec041060 2021-04-03T03:14:27.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf1ffb700 1 -- 172.21.2.111:0/1151092653 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce800dac0 con 0x7fbcec041060 2021-04-03T03:14:27.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.132+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec03ab50 con 0x7fbcec041060 2021-04-03T03:14:27.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.136+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0e7890 con 0x7fbcec041060 2021-04-03T03:14:27.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.136+0000 7fbcf1ffb700 1 -- 172.21.2.111:0/1151092653 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbce8017030 con 0x7fbcec041060 2021-04-03T03:14:27.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.136+0000 7fbcf1ffb700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbcd4057050 0x7fbcd4059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.136+0000 7fbcf1ffb700 1 -- 172.21.2.111:0/1151092653 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fbce806f530 con 0x7fbcec041060 2021-04-03T03:14:27.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.136+0000 7fbcf8ffc700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbcd4057050 0x7fbcd4059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.136+0000 7fbcf8ffc700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbcd4057050 0x7fbcd4059510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbce4017000 tx=0x7fbce4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.136+0000 7fbcf1ffb700 1 -- 172.21.2.111:0/1151092653 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbce80449d0 con 0x7fbcec041060 2021-04-03T03:14:27.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbcec004db0 con 0x7fbcec041060 2021-04-03T03:14:27.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf1ffb700 1 -- 172.21.2.111:0/1151092653 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbce8047ff0 con 0x7fbcec041060 2021-04-03T03:14:27.244 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:14:27.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbcd4057050 msgr2=0x7fbcd4059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbcd4057050 0x7fbcd4059510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbce4017000 tx=0x7fbce4015000).stop 2021-04-03T03:14:27.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec041060 msgr2=0x7fbcec0de110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec041060 0x7fbcec0de110 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fbce8002f30 tx=0x7fbce8006ea0).stop 2021-04-03T03:14:27.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 shutdown_connections 2021-04-03T03:14:27.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbcec0dbba0 0x7fbcec03a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbcd4057050 0x7fbcd4059510 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbcec03d560 0x7fbcec03b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 --2- 172.21.2.111:0/1151092653 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbcec041060 0x7fbcec0de110 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 >> 172.21.2.111:0/1151092653 conn(0x7fbcec0462b0 msgr2=0x7fbcec044d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:27.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 shutdown_connections 2021-04-03T03:14:27.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.232+0000 7fbcf9ffe700 1 -- 172.21.2.111:0/1151092653 wait complete. 2021-04-03T03:14:27.251 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:14:27.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3093227700 1 Processor -- start 2021-04-03T03:14:27.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3093227700 1 -- start start 2021-04-03T03:14:27.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3093227700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3084047eb0 0x7f3084006cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3093227700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f30840cb860 0x7f30840cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3093227700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3084043110 con 0x7f3084047ad0 2021-04-03T03:14:27.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3093227700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30840427a0 con 0x7f30840cb860 2021-04-03T03:14:27.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3093227700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3084042a40 con 0x7f3084047eb0 2021-04-03T03:14:27.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3092a26700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f30840cb860 0x7f30840cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3092225700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3084047ad0 0x7f30840085a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54676/0 (socket says 172.21.2.111:54676) 2021-04-03T03:14:27.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.296+0000 7f3092225700 1 -- 172.21.2.111:0/3390708555 learned_addr learned my addr 172.21.2.111:0/3390708555 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:27.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092a26700 1 -- 172.21.2.111:0/3390708555 >> v2:172.21.2.115:3300/0 conn(0x7f3084047eb0 msgr2=0x7f3084006cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:27.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092a26700 1 --2- 172.21.2.111:0/3390708555 >> v2:172.21.2.115:3300/0 conn(0x7f3084047eb0 0x7f3084006cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092a26700 1 -- 172.21.2.111:0/3390708555 >> v1:172.21.2.111:6789/0 conn(0x7f3084047ad0 legacy=0x7f30840085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092a26700 1 -- 172.21.2.111:0/3390708555 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3084042d90 con 0x7f30840cb860 2021-04-03T03:14:27.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092a26700 1 --2- 172.21.2.111:0/3390708555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30840cb860 0x7f30840cbca0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f308c06b5d0 tx=0x7f308c05ad70).ready entity=mon.0 client_cookie=fe8cb30b94876a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3091223700 1 -- 172.21.2.111:0/3390708555 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f308c05bda0 con 0x7f30840cb860 2021-04-03T03:14:27.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3091223700 1 -- 172.21.2.111:0/3390708555 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c05df60 con 0x7f30840cb860 2021-04-03T03:14:27.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3091223700 1 -- 172.21.2.111:0/3390708555 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f308c0699f0 con 0x7f30840cb860 2021-04-03T03:14:27.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3390708555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30840cb860 msgr2=0x7f30840cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 --2- 172.21.2.111:0/3390708555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30840cb860 0x7f30840cbca0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f308c06b5d0 tx=0x7f308c05ad70).stop 2021-04-03T03:14:27.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3390708555 shutdown_connections 2021-04-03T03:14:27.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 --2- 172.21.2.111:0/3390708555 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30840cb860 0x7f30840cbca0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 --2- 172.21.2.111:0/3390708555 >> v2:172.21.2.115:3300/0 conn(0x7f3084047eb0 0x7f3084006cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3390708555 >> 172.21.2.111:0/3390708555 conn(0x7f308400e210 msgr2=0x7f30840473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:27.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3390708555 shutdown_connections 2021-04-03T03:14:27.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3390708555 wait complete. 2021-04-03T03:14:27.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 Processor -- start 2021-04-03T03:14:27.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- start start 2021-04-03T03:14:27.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3084047ad0 0x7f3084040340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3084047eb0 0x7f308403a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30840cb860 0x7f30840cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30840cc770 con 0x7f3084047ad0 2021-04-03T03:14:27.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308403c1e0 con 0x7f3084047eb0 2021-04-03T03:14:27.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308403ab80 con 0x7f30840cb860 2021-04-03T03:14:27.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3084047ad0 0x7f3084040340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3084047ad0 0x7f3084040340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53890/0 (socket says 172.21.2.111:53890) 2021-04-03T03:14:27.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 -- 172.21.2.111:0/3943103637 learned_addr learned my addr 172.21.2.111:0/3943103637 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:27.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 -- 172.21.2.111:0/3943103637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30840cb860 msgr2=0x7f30840cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30840cb860 0x7f30840cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 -- 172.21.2.111:0/3943103637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3084047eb0 msgr2=0x7f308403a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3084047eb0 0x7f308403a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 -- 172.21.2.111:0/3943103637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308403bf40 con 0x7f3084047ad0 2021-04-03T03:14:27.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3092225700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3084047ad0 0x7f3084040340 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f308403da40 tx=0x7f3088007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f307f7fe700 1 -- 172.21.2.111:0/3943103637 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3088007320 con 0x7f3084047ad0 2021-04-03T03:14:27.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f307f7fe700 1 -- 172.21.2.111:0/3943103637 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3088007480 con 0x7f3084047ad0 2021-04-03T03:14:27.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30840cd610 con 0x7f3084047ad0 2021-04-03T03:14:27.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3084040c50 con 0x7f3084047ad0 2021-04-03T03:14:27.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f307f7fe700 1 -- 172.21.2.111:0/3943103637 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30880057e0 con 0x7f3084047ad0 2021-04-03T03:14:27.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.300+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308403bd50 con 0x7f3084047ad0 2021-04-03T03:14:27.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.304+0000 7f307f7fe700 1 -- 172.21.2.111:0/3943103637 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3088010030 con 0x7f3084047ad0 2021-04-03T03:14:27.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.304+0000 7f307f7fe700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f306c056f70 0x7f306c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.304+0000 7f307f7fe700 1 -- 172.21.2.111:0/3943103637 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f308806fe60 con 0x7f3084047ad0 2021-04-03T03:14:27.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.304+0000 7f3091a24700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f306c056f70 0x7f306c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.304+0000 7f3091a24700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f306c056f70 0x7f306c059430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f3080002f30 tx=0x7f3080006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.304+0000 7f307f7fe700 1 -- 172.21.2.111:0/3943103637 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f30880451f0 con 0x7f3084047ad0 2021-04-03T03:14:27.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.396+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3084005200 con 0x7f3084047ad0 2021-04-03T03:14:27.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f307f7fe700 1 -- 172.21.2.111:0/3943103637 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3088048810 con 0x7f3084047ad0 2021-04-03T03:14:27.411 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:14:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f306c056f70 msgr2=0x7f306c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f306c056f70 0x7f306c059430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f3080002f30 tx=0x7f3080006a60).stop 2021-04-03T03:14:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3084047ad0 msgr2=0x7f3084040340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3084047ad0 0x7f3084040340 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f308403da40 tx=0x7f3088007c20).stop 2021-04-03T03:14:27.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 shutdown_connections 2021-04-03T03:14:27.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30840cb860 0x7f30840cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3084047ad0 0x7f3084040340 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f306c056f70 0x7f306c059430 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 --2- 172.21.2.111:0/3943103637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3084047eb0 0x7f308403a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 >> 172.21.2.111:0/3943103637 conn(0x7f308400e210 msgr2=0x7f30840d3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:27.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 shutdown_connections 2021-04-03T03:14:27.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.400+0000 7f3093227700 1 -- 172.21.2.111:0/3943103637 wait complete. 2021-04-03T03:14:27.418 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:14:27.418 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:27.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 Processor -- start 2021-04-03T03:14:27.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- start start 2021-04-03T03:14:27.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f76e814b700 0x7f76e80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f76e80b0c90 0x7f76e8147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e80af400 con 0x7f76e80603f0 2021-04-03T03:14:27.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e8060860 con 0x7f76e80b0c90 2021-04-03T03:14:27.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e80af6a0 con 0x7f76e814b700 2021-04-03T03:14:27.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f76e80b0c90 0x7f76e8147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f76e80b0c90 0x7f76e8147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53902/0 (socket says 172.21.2.111:53902) 2021-04-03T03:14:27.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 -- 172.21.2.111:0/3205448498 learned_addr learned my addr 172.21.2.111:0/3205448498 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:27.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 -- 172.21.2.111:0/3205448498 >> v2:172.21.2.115:3300/0 conn(0x7f76e814b700 msgr2=0x7f76e80b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:27.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 --2- 172.21.2.111:0/3205448498 >> v2:172.21.2.115:3300/0 conn(0x7f76e814b700 0x7f76e80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 -- 172.21.2.111:0/3205448498 >> v1:172.21.2.115:6789/0 conn(0x7f76e80603f0 legacy=0x7f76e80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:27.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 -- 172.21.2.111:0/3205448498 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e80afb40 con 0x7f76e80b0c90 2021-04-03T03:14:27.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e77fe700 1 --2- 172.21.2.111:0/3205448498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e8147680 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f76e806a3b0 tx=0x7f76e0009270).ready entity=mon.0 client_cookie=cdeaa23616d7e951 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e5ffb700 1 -- 172.21.2.111:0/3205448498 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e000bbc0 con 0x7f76e80b0c90 2021-04-03T03:14:27.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e5ffb700 1 -- 172.21.2.111:0/3205448498 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e000bd20 con 0x7f76e80b0c90 2021-04-03T03:14:27.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e5ffb700 1 -- 172.21.2.111:0/3205448498 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e000be90 con 0x7f76e80b0c90 2021-04-03T03:14:27.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- 172.21.2.111:0/3205448498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 msgr2=0x7f76e8147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 --2- 172.21.2.111:0/3205448498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e8147680 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f76e806a3b0 tx=0x7f76e0009270).stop 2021-04-03T03:14:27.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- 172.21.2.111:0/3205448498 shutdown_connections 2021-04-03T03:14:27.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 --2- 172.21.2.111:0/3205448498 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e8147680 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 --2- 172.21.2.111:0/3205448498 >> v2:172.21.2.115:3300/0 conn(0x7f76e814b700 0x7f76e80b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- 172.21.2.111:0/3205448498 >> 172.21.2.111:0/3205448498 conn(0x7f76e80b61c0 msgr2=0x7f76e805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:27.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- 172.21.2.111:0/3205448498 shutdown_connections 2021-04-03T03:14:27.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.464+0000 7f76e7fff700 1 -- 172.21.2.111:0/3205448498 wait complete. 2021-04-03T03:14:27.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 Processor -- start 2021-04-03T03:14:27.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 -- start start 2021-04-03T03:14:27.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f76e80603f0 0x7f76e80ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e80675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f76e814b700 0x7f76e8067a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8157910 con 0x7f76e80b0c90 2021-04-03T03:14:27.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80a9ac0 con 0x7f76e80603f0 2021-04-03T03:14:27.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8064340 con 0x7f76e814b700 2021-04-03T03:14:27.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e80675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e80675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53906/0 (socket says 172.21.2.111:53906) 2021-04-03T03:14:27.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 -- 172.21.2.111:0/2289287899 learned_addr learned my addr 172.21.2.111:0/2289287899 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:27.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 -- 172.21.2.111:0/2289287899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f76e814b700 msgr2=0x7f76e8067a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f76e814b700 0x7f76e8067a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 -- 172.21.2.111:0/2289287899 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f76e80603f0 msgr2=0x7f76e80ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f76e80603f0 0x7f76e80ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 -- 172.21.2.111:0/2289287899 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8064570 con 0x7f76e80b0c90 2021-04-03T03:14:27.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e67fc700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e80675a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f76d0002f30 tx=0x7f76d0006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76dffff700 1 -- 172.21.2.111:0/2289287899 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d000d570 con 0x7f76e80b0c90 2021-04-03T03:14:27.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76dffff700 1 -- 172.21.2.111:0/2289287899 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d000d6d0 con 0x7f76e80b0c90 2021-04-03T03:14:27.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76dffff700 1 -- 172.21.2.111:0/2289287899 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d000da30 con 0x7f76e80b0c90 2021-04-03T03:14:27.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e8064810 con 0x7f76e80b0c90 2021-04-03T03:14:27.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e80abed0 con 0x7f76e80b0c90 2021-04-03T03:14:27.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e8057660 con 0x7f76e80b0c90 2021-04-03T03:14:27.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76dffff700 1 -- 172.21.2.111:0/2289287899 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f76d0018030 con 0x7f76e80b0c90 2021-04-03T03:14:27.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76dffff700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f76cc056ea0 0x7f76cc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:27.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76dffff700 1 -- 172.21.2.111:0/2289287899 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f76d006f630 con 0x7f76e80b0c90 2021-04-03T03:14:27.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e6ffd700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f76cc056ea0 0x7f76cc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:27.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.468+0000 7f76e6ffd700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f76cc056ea0 0x7f76cc059360 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f76d8002f50 tx=0x7f76d8006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:27.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.472+0000 7f76dffff700 1 -- 172.21.2.111:0/2289287899 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f76d0041c50 con 0x7f76e80b0c90 2021-04-03T03:14:27.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.592+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76e80b1550 con 0x7f76e80b0c90 2021-04-03T03:14:27.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.592+0000 7f76dffff700 1 -- 172.21.2.111:0/2289287899 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f76d0044ad0 con 0x7f76e80b0c90 2021-04-03T03:14:27.606 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:27.606 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:14:27.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f76cc056ea0 msgr2=0x7f76cc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f76cc056ea0 0x7f76cc059360 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f76d8002f50 tx=0x7f76d8006990).stop 2021-04-03T03:14:27.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 msgr2=0x7f76e80675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:27.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e80675a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f76d0002f30 tx=0x7f76d0006dd0).stop 2021-04-03T03:14:27.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 shutdown_connections 2021-04-03T03:14:27.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f76e814b700 0x7f76e8067a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f76cc056ea0 0x7f76cc059360 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f76e80603f0 0x7f76e80ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:27.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 --2- 172.21.2.111:0/2289287899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f76e80b0c90 0x7f76e80675a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:27.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 >> 172.21.2.111:0/2289287899 conn(0x7f76e80b61c0 msgr2=0x7f76e80acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:27.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 shutdown_connections 2021-04-03T03:14:27.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:27.596+0000 7f76e7fff700 1 -- 172.21.2.111:0/2289287899 wait complete. 2021-04-03T03:14:27.610 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:14:27.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:14:24.079081+0000', 'modified': '2021-04-03T03:14:25.081940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6210': {'gid': 6210, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/560115150', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 560115150}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 560115150}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:14:27.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:14:27.816 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:14:27.889 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6210,"want_state":"up:active","state":"up:active","rank_uptime":3.7946507220000001,"mdsmap_epoch":29,"osdmap_epoch":74,"osdmap_epoch_barrier":73,"uptime":7.9379869430000003} 2021-04-03T03:14:27.889 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6210, "mdsmap_epoch": 29, "osdmap_epoch": 74, "osdmap_epoch_barrier": 73, "rank_uptime": 3.794650722, "state": "up:active", "uptime": 7.937986943, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:14:27.890 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:14:27.890 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:14:27.891 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:14:27.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:27.892 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:27.902 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:14:27.954 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:14:27.955 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:14:28.002 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:14:28.003 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:14:28.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:28.006 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:14:28.056 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:14:28.057 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:28.102 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:14:28.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:14:28.111 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:14:28.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:14:28.132+0000 7fde5cb5b0c0 -1 init, newargv = 0x562834f2c380 newargc=9 2021-04-03T03:14:28.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15459]: starting ceph client 2021-04-03T03:14:28.146 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:14:28.146 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:14:28.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:28.148 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:14:28.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15459]: starting fuse 2021-04-03T03:14:28.156 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:14:28.156 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:14:28.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:28.157 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:14:28.157 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:14:28.158 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:14:28.158 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:14:28.159 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:14:28.159 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:14:28.159 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:14:28.160 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:14:28.160 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:14:28.161 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:14:28.161 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:14:28.161 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:14:28.162 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:14:28.162 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:14:28.163 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:14:28.163 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:14:28.164 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:14:28.164 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:14:28.164 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:14:28.165 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:14:28.165 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:14:28.166 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:14:28.166 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:14:28.255 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15459.asok 2021-04-03T03:14:28.258 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15459.asok 2021-04-03T03:14:28.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:28.259 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15459.asok status 2021-04-03T03:14:28.315 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:14:28.316 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:14:28.316 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:14:28.316 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:14:28.316 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:14:28.316 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:14:28.317 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:14:28.317 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15459", 2021-04-03T03:14:28.317 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:14:28.317 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:14:28.317 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:14:28.318 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:14:28.318 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4993, 2021-04-03T03:14:28.318 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:14:28.318 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:14:28.319 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:14:28.319 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4993 2021-04-03T03:14:28.319 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:14:28.319 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:14:28.319 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:14:28.320 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:14:28.320 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2003837950 2021-04-03T03:14:28.320 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:14:28.320 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:14:28.320 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:14:28.321 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:14:28.321 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:14:28.321 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2003837950 2021-04-03T03:14:28.321 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:14:28.322 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4993 172.21.2.126:0/2003837950", 2021-04-03T03:14:28.322 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/2003837950", 2021-04-03T03:14:28.322 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:14:28.322 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 29, 2021-04-03T03:14:28.322 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 74, 2021-04-03T03:14:28.323 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:14:28.323 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:14:28.323 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:14:28.323 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:14:28.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:28.325 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:28.325 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:28.331 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:14:28.331 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:14:28.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:28.332 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:28.380 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:14:28.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe833386700 1 Processor -- start 2021-04-03T03:14:28.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe833386700 1 -- start start 2021-04-03T03:14:28.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe833386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe83414b700 0x7fe8340b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe833386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe8340b0c90 0x7fe834147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe833386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8340af400 con 0x7fe8340603f0 2021-04-03T03:14:28.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe833386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe834060860 con 0x7fe8340b0c90 2021-04-03T03:14:28.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe833386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8340af6a0 con 0x7fe83414b700 2021-04-03T03:14:28.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe832b85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe8340b0c90 0x7fe834147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe832384700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe8340603f0 0x7fe8340b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54704/0 (socket says 172.21.2.111:54704) 2021-04-03T03:14:28.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.428+0000 7fe832384700 1 -- 172.21.2.111:0/1017553043 learned_addr learned my addr 172.21.2.111:0/1017553043 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:28.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 -- 172.21.2.111:0/1017553043 >> v2:172.21.2.115:3300/0 conn(0x7fe83414b700 msgr2=0x7fe8340b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:28.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 --2- 172.21.2.111:0/1017553043 >> v2:172.21.2.115:3300/0 conn(0x7fe83414b700 0x7fe8340b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 -- 172.21.2.111:0/1017553043 >> v1:172.21.2.111:6789/0 conn(0x7fe8340603f0 legacy=0x7fe8340b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 -- 172.21.2.111:0/1017553043 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340afb40 con 0x7fe8340b0c90 2021-04-03T03:14:28.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 --2- 172.21.2.111:0/1017553043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8340b0c90 0x7fe834147680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fe82c0032b0 tx=0x7fe82c0091d0).ready entity=mon.0 client_cookie=dce805ea961c6a04 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe831382700 1 -- 172.21.2.111:0/1017553043 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe82c00ba50 con 0x7fe8340b0c90 2021-04-03T03:14:28.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe831382700 1 -- 172.21.2.111:0/1017553043 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c00bbb0 con 0x7fe8340b0c90 2021-04-03T03:14:28.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe831382700 1 -- 172.21.2.111:0/1017553043 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe82c00bd20 con 0x7fe8340b0c90 2021-04-03T03:14:28.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- 172.21.2.111:0/1017553043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8340b0c90 msgr2=0x7fe834147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 --2- 172.21.2.111:0/1017553043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8340b0c90 0x7fe834147680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fe82c0032b0 tx=0x7fe82c0091d0).stop 2021-04-03T03:14:28.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- 172.21.2.111:0/1017553043 shutdown_connections 2021-04-03T03:14:28.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 --2- 172.21.2.111:0/1017553043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8340b0c90 0x7fe834147680 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 --2- 172.21.2.111:0/1017553043 >> v2:172.21.2.115:3300/0 conn(0x7fe83414b700 0x7fe8340b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- 172.21.2.111:0/1017553043 >> 172.21.2.111:0/1017553043 conn(0x7fe8340b61c0 msgr2=0x7fe83405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:28.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- 172.21.2.111:0/1017553043 shutdown_connections 2021-04-03T03:14:28.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- 172.21.2.111:0/1017553043 wait complete. 2021-04-03T03:14:28.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 Processor -- start 2021-04-03T03:14:28.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- start start 2021-04-03T03:14:28.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8340603f0 0x7fe834062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8340b0c90 0x7fe834064770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe83414b700 0x7fe834156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe834064c60 con 0x7fe83414b700 2021-04-03T03:14:28.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe834064e80 con 0x7fe8340b0c90 2021-04-03T03:14:28.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe834065120 con 0x7fe8340603f0 2021-04-03T03:14:28.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe83414b700 0x7fe834156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe83414b700 0x7fe834156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53920/0 (socket says 172.21.2.111:53920) 2021-04-03T03:14:28.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 -- 172.21.2.111:0/800838676 learned_addr learned my addr 172.21.2.111:0/800838676 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:28.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 -- 172.21.2.111:0/800838676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8340603f0 msgr2=0x7fe834062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8340603f0 0x7fe834062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 -- 172.21.2.111:0/800838676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8340b0c90 msgr2=0x7fe834064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8340b0c90 0x7fe834064770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 -- 172.21.2.111:0/800838676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340653c0 con 0x7fe83414b700 2021-04-03T03:14:28.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe832b85700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe83414b700 0x7fe834156b10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fe82c0032b0 tx=0x7fe82c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe82b7fe700 1 -- 172.21.2.111:0/800838676 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe82c01f5e0 con 0x7fe83414b700 2021-04-03T03:14:28.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe82b7fe700 1 -- 172.21.2.111:0/800838676 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c01f740 con 0x7fe83414b700 2021-04-03T03:14:28.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe834156580 con 0x7fe83414b700 2021-04-03T03:14:28.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe82b7fe700 1 -- 172.21.2.111:0/800838676 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe82c01faa0 con 0x7fe83414b700 2021-04-03T03:14:28.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.432+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe834067d20 con 0x7fe83414b700 2021-04-03T03:14:28.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.436+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe834057660 con 0x7fe83414b700 2021-04-03T03:14:28.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.436+0000 7fe82b7fe700 1 -- 172.21.2.111:0/800838676 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe82c026030 con 0x7fe83414b700 2021-04-03T03:14:28.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.436+0000 7fe82b7fe700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe818056ea0 0x7fe818059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.436+0000 7fe82b7fe700 1 -- 172.21.2.111:0/800838676 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe82c0713b0 con 0x7fe83414b700 2021-04-03T03:14:28.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.436+0000 7fe832384700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe818056ea0 0x7fe818059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.436+0000 7fe832384700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe818056ea0 0x7fe818059360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fe83406a3b0 tx=0x7fe8240077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.436+0000 7fe82b7fe700 1 -- 172.21.2.111:0/800838676 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe82c046690 con 0x7fe83414b700 2021-04-03T03:14:28.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.528+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8340686d0 con 0x7fe83414b700 2021-04-03T03:14:28.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe82b7fe700 1 -- 172.21.2.111:0/800838676 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fe82c015090 con 0x7fe83414b700 2021-04-03T03:14:28.542 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:28.542 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":74,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:14:25.083032+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:14:22.039727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-04-03T03:14:23.053211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:14:28.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe818056ea0 msgr2=0x7fe818059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe818056ea0 0x7fe818059360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fe83406a3b0 tx=0x7fe8240077e0).stop 2021-04-03T03:14:28.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe83414b700 msgr2=0x7fe834156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe83414b700 0x7fe834156b10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fe82c0032b0 tx=0x7fe82c016000).stop 2021-04-03T03:14:28.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 shutdown_connections 2021-04-03T03:14:28.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe83414b700 0x7fe834156b10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8340603f0 0x7fe834062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe818056ea0 0x7fe818059360 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 --2- 172.21.2.111:0/800838676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8340b0c90 0x7fe834064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 >> 172.21.2.111:0/800838676 conn(0x7fe8340b61c0 msgr2=0x7fe834146bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:28.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 shutdown_connections 2021-04-03T03:14:28.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.532+0000 7fe833386700 1 -- 172.21.2.111:0/800838676 wait complete. 2021-04-03T03:14:28.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:14:22.039727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:14:23.053211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:14:28.551 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:14:28.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 Processor -- start 2021-04-03T03:14:28.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 -- start start 2021-04-03T03:14:28.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc6680603f0 0x7fc6681447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc6680add50 0x7fc668144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680607d0 con 0x7fc6680b17b0 2021-04-03T03:14:28.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6681451c0 con 0x7fc6680add50 2021-04-03T03:14:28.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66814b090 con 0x7fc6680603f0 2021-04-03T03:14:28.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66e828700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc6680b17b0 0x7fc668147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54722/0 (socket says 172.21.2.111:54722) 2021-04-03T03:14:28.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66e828700 1 -- 172.21.2.111:0/3956747589 learned_addr learned my addr 172.21.2.111:0/3956747589 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:28.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d826700 1 --2- 172.21.2.111:0/3956747589 >> v2:172.21.2.111:3300/0 conn(0x7fc6680add50 0x7fc668144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d826700 1 -- 172.21.2.111:0/3956747589 >> v2:172.21.2.115:3300/0 conn(0x7fc6680603f0 msgr2=0x7fc6681447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:28.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d826700 1 --2- 172.21.2.111:0/3956747589 >> v2:172.21.2.115:3300/0 conn(0x7fc6680603f0 0x7fc6681447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d826700 1 -- 172.21.2.111:0/3956747589 >> v1:172.21.2.111:6789/0 conn(0x7fc6680b17b0 legacy=0x7fc668147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d826700 1 -- 172.21.2.111:0/3956747589 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66814b330 con 0x7fc6680add50 2021-04-03T03:14:28.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d826700 1 --2- 172.21.2.111:0/3956747589 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668144cd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7fc65c002f30 tx=0x7fc65c006dd0).ready entity=mon.0 client_cookie=a72c33cff14d22cb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d025700 1 -- 172.21.2.111:0/3956747589 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc65c0097f0 con 0x7fc6680add50 2021-04-03T03:14:28.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d025700 1 -- 172.21.2.111:0/3956747589 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c009950 con 0x7fc6680add50 2021-04-03T03:14:28.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66d025700 1 -- 172.21.2.111:0/3956747589 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc65c009cb0 con 0x7fc6680add50 2021-04-03T03:14:28.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 -- 172.21.2.111:0/3956747589 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 msgr2=0x7fc668144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 --2- 172.21.2.111:0/3956747589 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668144cd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7fc65c002f30 tx=0x7fc65c006dd0).stop 2021-04-03T03:14:28.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 -- 172.21.2.111:0/3956747589 shutdown_connections 2021-04-03T03:14:28.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 --2- 172.21.2.111:0/3956747589 >> v2:172.21.2.115:3300/0 conn(0x7fc6680603f0 0x7fc6681447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 --2- 172.21.2.111:0/3956747589 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668144cd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.596+0000 7fc66f029700 1 -- 172.21.2.111:0/3956747589 >> 172.21.2.111:0/3956747589 conn(0x7fc6680b61c0 msgr2=0x7fc66805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:28.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- 172.21.2.111:0/3956747589 shutdown_connections 2021-04-03T03:14:28.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- 172.21.2.111:0/3956747589 wait complete. 2021-04-03T03:14:28.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 Processor -- start 2021-04-03T03:14:28.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- start start 2021-04-03T03:14:28.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6680603f0 0x7fc6680a8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6680b17b0 0x7fc668064ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc668159f60 con 0x7fc6680add50 2021-04-03T03:14:28.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc668065150 con 0x7fc6680603f0 2021-04-03T03:14:28.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680653f0 con 0x7fc6680b17b0 2021-04-03T03:14:28.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53932/0 (socket says 172.21.2.111:53932) 2021-04-03T03:14:28.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 -- 172.21.2.111:0/1066625233 learned_addr learned my addr 172.21.2.111:0/1066625233 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 -- 172.21.2.111:0/1066625233 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6680b17b0 msgr2=0x7fc668064ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6680b17b0 0x7fc668064ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 -- 172.21.2.111:0/1066625233 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6680603f0 msgr2=0x7fc6680a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6680603f0 0x7fc6680a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 -- 172.21.2.111:0/1066625233 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66815a390 con 0x7fc6680add50 2021-04-03T03:14:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66d826700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668148110 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fc65c002f30 tx=0x7fc65c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc65b7fe700 1 -- 172.21.2.111:0/1066625233 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc65c011d70 con 0x7fc6680add50 2021-04-03T03:14:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc65b7fe700 1 -- 172.21.2.111:0/1066625233 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c01e460 con 0x7fc6680add50 2021-04-03T03:14:28.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66815a640 con 0x7fc6680add50 2021-04-03T03:14:28.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc65b7fe700 1 -- 172.21.2.111:0/1066625233 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc65c01e7c0 con 0x7fc6680add50 2021-04-03T03:14:28.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66815aae0 con 0x7fc6680add50 2021-04-03T03:14:28.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc668064fc0 con 0x7fc6680add50 2021-04-03T03:14:28.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc65b7fe700 1 -- 172.21.2.111:0/1066625233 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc65c025030 con 0x7fc6680add50 2021-04-03T03:14:28.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc65b7fe700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc648056f70 0x7fc648059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc65b7fe700 1 -- 172.21.2.111:0/1066625233 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc65c0718c0 con 0x7fc6680add50 2021-04-03T03:14:28.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66e027700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc648056f70 0x7fc648059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.600+0000 7fc66e027700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc648056f70 0x7fc648059430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fc664002f30 tx=0x7fc664006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.604+0000 7fc65b7fe700 1 -- 172.21.2.111:0/1066625233 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc65c046ba0 con 0x7fc6680add50 2021-04-03T03:14:28.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc66815a0a0 con 0x7fc6680add50 2021-04-03T03:14:28.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc65b7fe700 1 -- 172.21.2.111:0/1066625233 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc65c04a1c0 con 0x7fc6680add50 2021-04-03T03:14:28.706 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:14:28.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc648056f70 msgr2=0x7fc648059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc648056f70 0x7fc648059430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fc664002f30 tx=0x7fc664006a60).stop 2021-04-03T03:14:28.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 msgr2=0x7fc668148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668148110 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fc65c002f30 tx=0x7fc65c015000).stop 2021-04-03T03:14:28.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 shutdown_connections 2021-04-03T03:14:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6680b17b0 0x7fc668064ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc648056f70 0x7fc648059430 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6680603f0 0x7fc6680a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 --2- 172.21.2.111:0/1066625233 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6680add50 0x7fc668148110 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 >> 172.21.2.111:0/1066625233 conn(0x7fc6680b61c0 msgr2=0x7fc66814a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 shutdown_connections 2021-04-03T03:14:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.696+0000 7fc66f029700 1 -- 172.21.2.111:0/1066625233 wait complete. 2021-04-03T03:14:28.714 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:14:28.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae7fff700 1 Processor -- start 2021-04-03T03:14:28.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae7fff700 1 -- start start 2021-04-03T03:14:28.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae7fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdae80add50 0x7fdae8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae7fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae80607d0 con 0x7fdae80b17b0 2021-04-03T03:14:28.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae7fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae81451c0 con 0x7fdae80add50 2021-04-03T03:14:28.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae814b090 con 0x7fdae80603f0 2021-04-03T03:14:28.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae67fc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdae80add50 0x7fdae8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae6ffd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae77fe700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fdae80b17b0 0x7fdae8147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:39036/0 (socket says 172.21.2.111:39036) 2021-04-03T03:14:28.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.764+0000 7fdae77fe700 1 -- 172.21.2.111:0/277863813 learned_addr learned my addr 172.21.2.111:0/277863813 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:28.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae6ffd700 1 -- 172.21.2.111:0/277863813 >> v2:172.21.2.115:3300/0 conn(0x7fdae80add50 msgr2=0x7fdae8144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:28.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae6ffd700 1 --2- 172.21.2.111:0/277863813 >> v2:172.21.2.115:3300/0 conn(0x7fdae80add50 0x7fdae8144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae6ffd700 1 -- 172.21.2.111:0/277863813 >> v1:172.21.2.115:6789/0 conn(0x7fdae80b17b0 legacy=0x7fdae8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae6ffd700 1 -- 172.21.2.111:0/277863813 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae814b330 con 0x7fdae80603f0 2021-04-03T03:14:28.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae67fc700 1 --2- 172.21.2.111:0/277863813 >> v2:172.21.2.115:3300/0 conn(0x7fdae80add50 0x7fdae8144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:14:28.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae6ffd700 1 --2- 172.21.2.111:0/277863813 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81447e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdad0002f30 tx=0x7fdad0006dd0).ready entity=mon.2 client_cookie=28ba9ead45251215 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae5ffb700 1 -- 172.21.2.111:0/277863813 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdad0009760 con 0x7fdae80603f0 2021-04-03T03:14:28.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae5ffb700 1 -- 172.21.2.111:0/277863813 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad00098c0 con 0x7fdae80603f0 2021-04-03T03:14:28.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- 172.21.2.111:0/277863813 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 msgr2=0x7fdae81447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 --2- 172.21.2.111:0/277863813 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81447e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdad0002f30 tx=0x7fdad0006dd0).stop 2021-04-03T03:14:28.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- 172.21.2.111:0/277863813 shutdown_connections 2021-04-03T03:14:28.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 --2- 172.21.2.111:0/277863813 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81447e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 --2- 172.21.2.111:0/277863813 >> v2:172.21.2.115:3300/0 conn(0x7fdae80add50 0x7fdae8144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- 172.21.2.111:0/277863813 >> 172.21.2.111:0/277863813 conn(0x7fdae80b61c0 msgr2=0x7fdae805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:28.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- 172.21.2.111:0/277863813 shutdown_connections 2021-04-03T03:14:28.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- 172.21.2.111:0/277863813 wait complete. 2021-04-03T03:14:28.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 Processor -- start 2021-04-03T03:14:28.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- start start 2021-04-03T03:14:28.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdae80add50 0x7fdae8157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdae80b17b0 0x7fdae814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae80b3dd0 con 0x7fdae80b17b0 2021-04-03T03:14:28.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae8156a80 con 0x7fdae80add50 2021-04-03T03:14:28.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae8156d20 con 0x7fdae80603f0 2021-04-03T03:14:28.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdae80b17b0 0x7fdae814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdae80b17b0 0x7fdae814bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53948/0 (socket says 172.21.2.111:53948) 2021-04-03T03:14:28.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 -- 172.21.2.111:0/938439227 learned_addr learned my addr 172.21.2.111:0/938439227 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:28.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 -- 172.21.2.111:0/938439227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 msgr2=0x7fdae81481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 -- 172.21.2.111:0/938439227 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdae80add50 msgr2=0x7fdae8157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdae80add50 0x7fdae8157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 -- 172.21.2.111:0/938439227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8156fc0 con 0x7fdae80b17b0 2021-04-03T03:14:28.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae77fe700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdae80b17b0 0x7fdae814bdd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7fdae80b5600 tx=0x7fdae0007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdadffff700 1 -- 172.21.2.111:0/938439227 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdae0002080 con 0x7fdae80b17b0 2021-04-03T03:14:28.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdadffff700 1 -- 172.21.2.111:0/938439227 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0005b80 con 0x7fdae80b17b0 2021-04-03T03:14:28.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdadffff700 1 -- 172.21.2.111:0/938439227 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdae0007110 con 0x7fdae80b17b0 2021-04-03T03:14:28.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae8146810 con 0x7fdae80b17b0 2021-04-03T03:14:28.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.768+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae80b3650 con 0x7fdae80b17b0 2021-04-03T03:14:28.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.772+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae8157150 con 0x7fdae80b17b0 2021-04-03T03:14:28.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.772+0000 7fdadffff700 1 -- 172.21.2.111:0/938439227 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdae0010030 con 0x7fdae80b17b0 2021-04-03T03:14:28.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.772+0000 7fdadffff700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdacc056f60 0x7fdacc059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.772+0000 7fdadffff700 1 -- 172.21.2.111:0/938439227 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fdae0070140 con 0x7fdae80b17b0 2021-04-03T03:14:28.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.772+0000 7fdae6ffd700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdacc056f60 0x7fdacc059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.772+0000 7fdae6ffd700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdacc056f60 0x7fdacc059420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fdad0002f30 tx=0x7fdad0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.772+0000 7fdadffff700 1 -- 172.21.2.111:0/938439227 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdae0045530 con 0x7fdae80b17b0 2021-04-03T03:14:28.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.865+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdae8057660 con 0x7fdae80b17b0 2021-04-03T03:14:28.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.865+0000 7fdadffff700 1 -- 172.21.2.111:0/938439227 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdae0048b50 con 0x7fdae80b17b0 2021-04-03T03:14:28.877 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:14:28.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.865+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdacc056f60 msgr2=0x7fdacc059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.865+0000 7fdae7fff700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdacc056f60 0x7fdacc059420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fdad0002f30 tx=0x7fdad0015000).stop 2021-04-03T03:14:28.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.865+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdae80b17b0 msgr2=0x7fdae814bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.865+0000 7fdae7fff700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdae80b17b0 0x7fdae814bdd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7fdae80b5600 tx=0x7fdae0007f90).stop 2021-04-03T03:14:28.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 shutdown_connections 2021-04-03T03:14:28.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdae80b17b0 0x7fdae814bdd0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdae80603f0 0x7fdae81481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdacc056f60 0x7fdacc059420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 --2- 172.21.2.111:0/938439227 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdae80add50 0x7fdae8157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 >> 172.21.2.111:0/938439227 conn(0x7fdae80b61c0 msgr2=0x7fdae8143df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:28.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 shutdown_connections 2021-04-03T03:14:28.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.869+0000 7fdae7fff700 1 -- 172.21.2.111:0/938439227 wait complete. 2021-04-03T03:14:28.884 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:14:28.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c77fff700 1 Processor -- start 2021-04-03T03:14:28.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c77fff700 1 -- start start 2021-04-03T03:14:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c77fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78144ef0 0x7f6c78147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c77fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780607d0 con 0x7f6c780603f0 2021-04-03T03:14:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c77fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7814b090 con 0x7f6c780add50 2021-04-03T03:14:28.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c77fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7814b330 con 0x7f6c78144ef0 2021-04-03T03:14:28.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c76ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6c780603f0 0x7f6c781447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54744/0 (socket says 172.21.2.111:54744) 2021-04-03T03:14:28.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.941+0000 7f6c76ffd700 1 -- 172.21.2.111:0/11924238 learned_addr learned my addr 172.21.2.111:0/11924238 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:28.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125879587 0 0) 0x7f6c780607d0 con 0x7f6c780603f0 2021-04-03T03:14:28.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c58003240 con 0x7f6c780603f0 2021-04-03T03:14:28.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2686320470 0 0) 0x7f6c58003240 con 0x7f6c780603f0 2021-04-03T03:14:28.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c777fe700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78144ef0 0x7f6c78147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78144ef0 msgr2=0x7f6c78147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:28.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78144ef0 0x7f6c78147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:28.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 >> v1:172.21.2.115:6789/0 conn(0x7f6c780add50 legacy=0x7f6c78147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:28.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7814b5d0 con 0x7f6c780603f0 2021-04-03T03:14:28.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f6c680016d0 con 0x7f6c780603f0 2021-04-03T03:14:28.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 >> v1:172.21.2.111:6789/0 conn(0x7f6c780603f0 legacy=0x7f6c781447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c58003760 0x7f6c58003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c580044e0 0x7f6c58004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c58004e50 0x7f6c5800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7814b330 con 0x7f6c58003760 2021-04-03T03:14:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580041d0 con 0x7f6c58004e50 2021-04-03T03:14:28.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5800bc20 con 0x7f6c580044e0 2021-04-03T03:14:28.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c777fe700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78144ef0 0x7f6c78147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:28.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c58003760 0x7f6c58003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 -- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c580044e0 msgr2=0x7f6c58004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c580044e0 0x7f6c58004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 -- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c58004e50 msgr2=0x7f6c5800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c58004e50 0x7f6c5800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 -- 172.21.2.111:0/11924238 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58003240 con 0x7f6c58003760 2021-04-03T03:14:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c58003760 0x7f6c58003b60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f6c780b5640 tx=0x7f6c7814b220).ready entity=mon.0 client_cookie=9211edafe5d22640 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c6800ce00 con 0x7f6c58003760 2021-04-03T03:14:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c680072e0 con 0x7f6c58003760 2021-04-03T03:14:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c75ffb700 1 -- 172.21.2.111:0/11924238 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c680055f0 con 0x7f6c58003760 2021-04-03T03:14:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- 172.21.2.111:0/11924238 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c58003760 msgr2=0x7f6c58003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 --2- 172.21.2.111:0/11924238 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c58003760 0x7f6c58003b60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f6c780b5640 tx=0x7f6c7814b220).stop 2021-04-03T03:14:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c777fe700 1 -- 172.21.2.111:0/11924238 reap_dead start 2021-04-03T03:14:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- 172.21.2.111:0/11924238 shutdown_connections 2021-04-03T03:14:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- 172.21.2.111:0/11924238 >> 172.21.2.111:0/11924238 conn(0x7f6c780b61c0 msgr2=0x7f6c7805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- 172.21.2.111:0/11924238 shutdown_connections 2021-04-03T03:14:28.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- 172.21.2.111:0/11924238 wait complete. 2021-04-03T03:14:28.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 Processor -- start 2021-04-03T03:14:28.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- start start 2021-04-03T03:14:28.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c78147640 0x7f6c780a7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c780a83f0 0x7f6c780a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78159d80 0x7f6c7815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780a6f20 con 0x7f6c78147640 2021-04-03T03:14:28.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78147a20 con 0x7f6c780a83f0 2021-04-03T03:14:28.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c77fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780a8a70 con 0x7f6c78159d80 2021-04-03T03:14:28.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c78147640 0x7f6c780a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c78147640 0x7f6c780a7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53964/0 (socket says 172.21.2.111:53964) 2021-04-03T03:14:28.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 -- 172.21.2.111:0/1971161600 learned_addr learned my addr 172.21.2.111:0/1971161600 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:28.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 -- 172.21.2.111:0/1971161600 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78159d80 msgr2=0x7f6c7815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78159d80 0x7f6c7815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 -- 172.21.2.111:0/1971161600 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c780a83f0 msgr2=0x7f6c780a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:28.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c780a83f0 0x7f6c780a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:28.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 -- 172.21.2.111:0/1971161600 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780a8130 con 0x7f6c78147640 2021-04-03T03:14:28.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.945+0000 7f6c76ffd700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c78147640 0x7f6c780a7310 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f6c780b5640 tx=0x7f6c68013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c6ffff700 1 -- 172.21.2.111:0/1971161600 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c6801a070 con 0x7f6c78147640 2021-04-03T03:14:28.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c6ffff700 1 -- 172.21.2.111:0/1971161600 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68013640 con 0x7f6c78147640 2021-04-03T03:14:28.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780a7cf0 con 0x7f6c78147640 2021-04-03T03:14:28.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c6ffff700 1 -- 172.21.2.111:0/1971161600 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c680139a0 con 0x7f6c78147640 2021-04-03T03:14:28.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c78067f50 con 0x7f6c78147640 2021-04-03T03:14:28.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c780abad0 con 0x7f6c78147640 2021-04-03T03:14:28.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c6ffff700 1 -- 172.21.2.111:0/1971161600 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6c68026070 con 0x7f6c78147640 2021-04-03T03:14:28.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c6ffff700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c5c056ef0 0x7f6c5c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:28.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c6ffff700 1 -- 172.21.2.111:0/1971161600 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f6c68070b90 con 0x7f6c78147640 2021-04-03T03:14:28.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c767fc700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c5c056ef0 0x7f6c5c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:28.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.949+0000 7f6c767fc700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c5c056ef0 0x7f6c5c0593b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f6c60001f60 tx=0x7f6c600077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:28.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:28.953+0000 7f6c6ffff700 1 -- 172.21.2.111:0/1971161600 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6c68045f80 con 0x7f6c78147640 2021-04-03T03:14:29.015 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:29.019 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:29.022 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:29.025 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:29.028 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:29.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c78057660 con 0x7f6c78147640 2021-04-03T03:14:29.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c6ffff700 1 -- 172.21.2.111:0/1971161600 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6c680495a0 con 0x7f6c78147640 2021-04-03T03:14:29.056 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:14:29.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c5c056ef0 msgr2=0x7f6c5c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c5c056ef0 0x7f6c5c0593b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f6c60001f60 tx=0x7f6c600077d0).stop 2021-04-03T03:14:29.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c78147640 msgr2=0x7f6c780a7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c78147640 0x7f6c780a7310 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f6c780b5640 tx=0x7f6c68013000).stop 2021-04-03T03:14:29.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 shutdown_connections 2021-04-03T03:14:29.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c78159d80 0x7f6c7815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c78147640 0x7f6c780a7310 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c5c056ef0 0x7f6c5c0593b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 --2- 172.21.2.111:0/1971161600 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c780a83f0 0x7f6c780a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 >> 172.21.2.111:0/1971161600 conn(0x7f6c780b61c0 msgr2=0x7f6c780ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:29.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 shutdown_connections 2021-04-03T03:14:29.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.045+0000 7f6c77fff700 1 -- 172.21.2.111:0/1971161600 wait complete. 2021-04-03T03:14:29.063 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:14:29.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 Processor -- start 2021-04-03T03:14:29.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- start start 2021-04-03T03:14:29.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83380603f0 0x7f83381447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f83380add50 0x7f8338144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83380607d0 con 0x7f83380b17b0 2021-04-03T03:14:29.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83381451c0 con 0x7f83380603f0 2021-04-03T03:14:29.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833814b090 con 0x7f83380add50 2021-04-03T03:14:29.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833d82f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83380603f0 0x7f83381447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:29.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e030700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f83380b17b0 0x7f8338147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54766/0 (socket says 172.21.2.111:54766) 2021-04-03T03:14:29.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e030700 1 -- 172.21.2.111:0/3911302330 learned_addr learned my addr 172.21.2.111:0/3911302330 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:29.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833d82f700 1 -- 172.21.2.111:0/3911302330 >> v2:172.21.2.115:3300/0 conn(0x7f83380add50 msgr2=0x7f8338144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:29.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833d82f700 1 --2- 172.21.2.111:0/3911302330 >> v2:172.21.2.115:3300/0 conn(0x7f83380add50 0x7f8338144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833d82f700 1 -- 172.21.2.111:0/3911302330 >> v1:172.21.2.111:6789/0 conn(0x7f83380b17b0 legacy=0x7f8338147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833d82f700 1 -- 172.21.2.111:0/3911302330 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833814b330 con 0x7f83380603f0 2021-04-03T03:14:29.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833d82f700 1 --2- 172.21.2.111:0/3911302330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f83381447e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f8334002f30 tx=0x7f8334006dd0).ready entity=mon.0 client_cookie=8ec3e10780c840a7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:29.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833c82d700 1 -- 172.21.2.111:0/3911302330 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83340097f0 con 0x7f83380603f0 2021-04-03T03:14:29.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833c82d700 1 -- 172.21.2.111:0/3911302330 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334009950 con 0x7f83380603f0 2021-04-03T03:14:29.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833c82d700 1 -- 172.21.2.111:0/3911302330 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8334009cb0 con 0x7f83380603f0 2021-04-03T03:14:29.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- 172.21.2.111:0/3911302330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 msgr2=0x7f83381447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 --2- 172.21.2.111:0/3911302330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f83381447e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f8334002f30 tx=0x7f8334006dd0).stop 2021-04-03T03:14:29.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- 172.21.2.111:0/3911302330 shutdown_connections 2021-04-03T03:14:29.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 --2- 172.21.2.111:0/3911302330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f83381447e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 --2- 172.21.2.111:0/3911302330 >> v2:172.21.2.115:3300/0 conn(0x7f83380add50 0x7f8338144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- 172.21.2.111:0/3911302330 >> 172.21.2.111:0/3911302330 conn(0x7f83380b61c0 msgr2=0x7f833805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:29.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- 172.21.2.111:0/3911302330 shutdown_connections 2021-04-03T03:14:29.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 -- 172.21.2.111:0/3911302330 wait complete. 2021-04-03T03:14:29.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.161+0000 7f833e831700 1 Processor -- start 2021-04-03T03:14:29.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 -- start start 2021-04-03T03:14:29.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f833814a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83380add50 0x7f833815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83380b17b0 0x7f833815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338143840 con 0x7f83380603f0 2021-04-03T03:14:29.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833815a010 con 0x7f83380b17b0 2021-04-03T03:14:29.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83380a7f70 con 0x7f83380add50 2021-04-03T03:14:29.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f833814a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:29.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f833814a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53978/0 (socket says 172.21.2.111:53978) 2021-04-03T03:14:29.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 -- 172.21.2.111:0/1992337295 learned_addr learned my addr 172.21.2.111:0/1992337295 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:29.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 -- 172.21.2.111:0/1992337295 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83380add50 msgr2=0x7f833815a400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:14:29.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83380add50 0x7f833815a400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 -- 172.21.2.111:0/1992337295 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83380b17b0 msgr2=0x7f833815a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:14:29.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83380b17b0 0x7f833815a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 -- 172.21.2.111:0/1992337295 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83380a8210 con 0x7f83380603f0 2021-04-03T03:14:29.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d82f700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f833814a7b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f83380aa840 tx=0x7f8334015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:29.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f832affd700 1 -- 172.21.2.111:0/1992337295 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8334011df0 con 0x7f83380603f0 2021-04-03T03:14:29.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f832affd700 1 -- 172.21.2.111:0/1992337295 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833401e460 con 0x7f83380603f0 2021-04-03T03:14:29.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83380a84b0 con 0x7f83380603f0 2021-04-03T03:14:29.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f832affd700 1 -- 172.21.2.111:0/1992337295 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f833401e7c0 con 0x7f83380603f0 2021-04-03T03:14:29.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833815ade0 con 0x7f83380603f0 2021-04-03T03:14:29.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83380a8640 con 0x7f83380603f0 2021-04-03T03:14:29.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f832affd700 1 -- 172.21.2.111:0/1992337295 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8334025030 con 0x7f83380603f0 2021-04-03T03:14:29.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f832affd700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8318056f70 0x7f8318059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f832affd700 1 -- 172.21.2.111:0/1992337295 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f83340718c0 con 0x7f83380603f0 2021-04-03T03:14:29.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d02e700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8318056f70 0x7f8318059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:29.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.165+0000 7f833d02e700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8318056f70 0x7f8318059430 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f832c002f30 tx=0x7f832c0069e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:29.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.169+0000 7f832affd700 1 -- 172.21.2.111:0/1992337295 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8334046ba0 con 0x7f83380603f0 2021-04-03T03:14:29.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8338057660 con 0x7f83380603f0 2021-04-03T03:14:29.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f832affd700 1 -- 172.21.2.111:0/1992337295 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7f833404a1c0 con 0x7f83380603f0 2021-04-03T03:14:29.272 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:14:29.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8318056f70 msgr2=0x7f8318059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8318056f70 0x7f8318059430 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f832c002f30 tx=0x7f832c0069e0).stop 2021-04-03T03:14:29.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 msgr2=0x7f833814a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f833814a7b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f83380aa840 tx=0x7f8334015000).stop 2021-04-03T03:14:29.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 shutdown_connections 2021-04-03T03:14:29.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8318056f70 0x7f8318059430 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83380b17b0 0x7f833815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83380603f0 0x7f833814a7b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 --2- 172.21.2.111:0/1992337295 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83380add50 0x7f833815a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 >> 172.21.2.111:0/1992337295 conn(0x7f83380b61c0 msgr2=0x7f83380b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:29.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 shutdown_connections 2021-04-03T03:14:29.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.261+0000 7f833e831700 1 -- 172.21.2.111:0/1992337295 wait complete. 2021-04-03T03:14:29.279 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:14:29.280 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:29.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7fff103700 1 Processor -- start 2021-04-03T03:14:29.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7fff103700 1 -- start start 2021-04-03T03:14:29.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7fff103700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7ff80603f0 0x7f7ff80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7fff103700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff80af400 con 0x7f7ff80b0c90 2021-04-03T03:14:29.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7fff103700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff80af6a0 con 0x7f7ff814b700 2021-04-03T03:14:29.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7fff103700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff8060860 con 0x7f7ff80603f0 2021-04-03T03:14:29.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7ffe902700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7ff80b0c90 0x7f7ff8147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54780/0 (socket says 172.21.2.111:54780) 2021-04-03T03:14:29.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7ffe902700 1 -- 172.21.2.111:0/1509450637 learned_addr learned my addr 172.21.2.111:0/1509450637 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:14:29.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564822612 0 0) 0x7f7ff80af400 con 0x7f7ff80b0c90 2021-04-03T03:14:29.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe0003240 con 0x7f7ff80b0c90 2021-04-03T03:14:29.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1360985084 0 0) 0x7f7fe0003240 con 0x7f7ff80b0c90 2021-04-03T03:14:29.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.325+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 >> v2:172.21.2.115:3300/0 conn(0x7f7ff80603f0 msgr2=0x7f7ff80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:29.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 --2- 172.21.2.111:0/1509450637 >> v2:172.21.2.115:3300/0 conn(0x7f7ff80603f0 0x7f7ff80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 >> v1:172.21.2.115:6789/0 conn(0x7f7ff814b700 legacy=0x7f7ff80b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:29.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80afb40 con 0x7f7ff80b0c90 2021-04-03T03:14:29.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f7ff0001a50 con 0x7f7ff80b0c90 2021-04-03T03:14:29.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 >> v1:172.21.2.111:6789/0 conn(0x7f7ff80b0c90 legacy=0x7f7ff8147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7fe0003730 0x7f7fe0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7fe00044e0 0x7f7fe0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7fe0004e50 0x7f7fe000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8060860 con 0x7f7fe00044e0 2021-04-03T03:14:29.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0004240 con 0x7f7fe0004e50 2021-04-03T03:14:29.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe000bc20 con 0x7f7fe0003730 2021-04-03T03:14:29.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd900700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7fe00044e0 0x7f7fe0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:29.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd900700 1 -- 172.21.2.111:0/1509450637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7fe0003730 msgr2=0x7f7fe0003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:29.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd900700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7fe0003730 0x7f7fe0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd900700 1 -- 172.21.2.111:0/1509450637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7fe0004e50 msgr2=0x7f7fe000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:29.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd900700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7fe0004e50 0x7f7fe000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd900700 1 -- 172.21.2.111:0/1509450637 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0003240 con 0x7f7fe00044e0 2021-04-03T03:14:29.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 -- 172.21.2.111:0/1509450637 reap_dead start 2021-04-03T03:14:29.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd900700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7fe00044e0 0x7f7fe0004960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f7fec000f30 tx=0x7f7fec007b50).ready entity=mon.0 client_cookie=23409f60423cd935 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:29.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fec00c660 con 0x7f7fe00044e0 2021-04-03T03:14:29.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec00c7c0 con 0x7f7fe00044e0 2021-04-03T03:14:29.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffd0ff700 1 -- 172.21.2.111:0/1509450637 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fec00cb20 con 0x7f7fe00044e0 2021-04-03T03:14:29.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- 172.21.2.111:0/1509450637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7fe00044e0 msgr2=0x7f7fe0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7fe00044e0 0x7f7fe0004960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f7fec000f30 tx=0x7f7fec007b50).stop 2021-04-03T03:14:29.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- 172.21.2.111:0/1509450637 shutdown_connections 2021-04-03T03:14:29.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 --2- 172.21.2.111:0/1509450637 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7fe00044e0 0x7f7fe0004960 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- 172.21.2.111:0/1509450637 >> 172.21.2.111:0/1509450637 conn(0x7f7ff80b61c0 msgr2=0x7f7ff805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:29.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- 172.21.2.111:0/1509450637 shutdown_connections 2021-04-03T03:14:29.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- 172.21.2.111:0/1509450637 wait complete. 2021-04-03T03:14:29.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 Processor -- start 2021-04-03T03:14:29.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- start start 2021-04-03T03:14:29.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7fe00044e0 0x7f7ff80a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ff8159a20 0x7f7ff80a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ff8064160 0x7f7ff80645e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8063d30 con 0x7f7ff8064160 2021-04-03T03:14:29.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8159ea0 con 0x7f7ff8159a20 2021-04-03T03:14:29.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80aae30 con 0x7f7fe00044e0 2021-04-03T03:14:29.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ff8064160 0x7f7ff80645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:29.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ff8064160 0x7f7ff80645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:53998/0 (socket says 172.21.2.111:53998) 2021-04-03T03:14:29.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 -- 172.21.2.111:0/851847718 learned_addr learned my addr 172.21.2.111:0/851847718 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:29.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 -- 172.21.2.111:0/851847718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7fe00044e0 msgr2=0x7f7ff80a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7fe00044e0 0x7f7ff80a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 -- 172.21.2.111:0/851847718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ff8159a20 msgr2=0x7f7ff80a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ff8159a20 0x7f7ff80a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 -- 172.21.2.111:0/851847718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80ab0d0 con 0x7f7ff8064160 2021-04-03T03:14:29.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7ffe902700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ff8064160 0x7f7ff80645e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f7ff806b530 tx=0x7f7ff80af830).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:29.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7feb7fe700 1 -- 172.21.2.111:0/851847718 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff0004b60 con 0x7f7ff8064160 2021-04-03T03:14:29.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7feb7fe700 1 -- 172.21.2.111:0/851847718 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff0004cc0 con 0x7f7ff8064160 2021-04-03T03:14:29.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff80ab370 con 0x7f7ff8064160 2021-04-03T03:14:29.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7feb7fe700 1 -- 172.21.2.111:0/851847718 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff0007220 con 0x7f7ff8064160 2021-04-03T03:14:29.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.329+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff80aa010 con 0x7f7ff8064160 2021-04-03T03:14:29.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.333+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff8057660 con 0x7f7ff8064160 2021-04-03T03:14:29.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.333+0000 7f7feb7fe700 1 -- 172.21.2.111:0/851847718 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7ff0010070 con 0x7f7ff8064160 2021-04-03T03:14:29.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.333+0000 7f7feb7fe700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7fd8056fb0 0x7f7fd8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:29.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.333+0000 7f7feb7fe700 1 -- 172.21.2.111:0/851847718 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7ff006fd30 con 0x7f7ff8064160 2021-04-03T03:14:29.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.333+0000 7f7ffe101700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7fd8056fb0 0x7f7fd8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:29.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.333+0000 7f7ffe101700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7fd8056fb0 0x7f7fd8059470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f7ff4002f30 tx=0x7f7ff40069e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:29.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.333+0000 7f7feb7fe700 1 -- 172.21.2.111:0/851847718 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7ff0045120 con 0x7f7ff8064160 2021-04-03T03:14:29.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.457+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ff80686d0 con 0x7f7ff8064160 2021-04-03T03:14:29.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.457+0000 7f7feb7fe700 1 -- 172.21.2.111:0/851847718 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f7ff0048740 con 0x7f7ff8064160 2021-04-03T03:14:29.469 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:29.470 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:14:29.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7fd8056fb0 msgr2=0x7f7fd8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7fd8056fb0 0x7f7fd8059470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f7ff4002f30 tx=0x7f7ff40069e0).stop 2021-04-03T03:14:29.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ff8064160 msgr2=0x7f7ff80645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:29.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ff8064160 0x7f7ff80645e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f7ff806b530 tx=0x7f7ff80af830).stop 2021-04-03T03:14:29.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 shutdown_connections 2021-04-03T03:14:29.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ff8064160 0x7f7ff80645e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7fe00044e0 0x7f7ff80a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7fd8056fb0 0x7f7fd8059470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:29.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 --2- 172.21.2.111:0/851847718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ff8159a20 0x7f7ff80a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:29.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 >> 172.21.2.111:0/851847718 conn(0x7f7ff80b61c0 msgr2=0x7f7ff80b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:29.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 shutdown_connections 2021-04-03T03:14:29.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:29.461+0000 7f7fff103700 1 -- 172.21.2.111:0/851847718 wait complete. 2021-04-03T03:14:29.477 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:14:29.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:14:24.079081+0000', 'modified': '2021-04-03T03:14:25.081940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6210': {'gid': 6210, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/560115150', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 560115150}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 560115150}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:14:29.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:14:29.682 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:14:29.753 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6210,"want_state":"up:active","state":"up:active","rank_uptime":5.6594682970000001,"mdsmap_epoch":29,"osdmap_epoch":74,"osdmap_epoch_barrier":73,"uptime":9.8028048870000006} 2021-04-03T03:14:29.754 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6210, "mdsmap_epoch": 29, "osdmap_epoch": 74, "osdmap_epoch_barrier": 73, "rank_uptime": 5.659468297, "state": "up:active", "uptime": 9.802804887, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:14:29.754 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:14:29.755 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:14:29.755 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:14:29.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:29.756 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:29.766 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:14:29.821 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:14:29.821 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:14:29.869 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:14:29.870 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:14:29.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:29.872 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:14:29.923 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:14:29.923 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:29.969 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:14:29.975 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:14:29.976 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:14:30.023 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:14:30.024 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:14:30.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:30.025 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:14:30.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:14:30.017+0000 7fefd697b0c0 -1 init, newargv = 0x55b087fe9650 newargc=9 2021-04-03T03:14:30.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15057]: starting ceph client 2021-04-03T03:14:30.031 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15057]: starting fuse 2021-04-03T03:14:30.039 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:14:30.039 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:14:30.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:30.040 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:14:30.041 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:14:30.041 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:14:30.041 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:14:30.042 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:14:30.042 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:14:30.042 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:14:30.043 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:14:30.043 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:14:30.044 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:14:30.044 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:14:30.044 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:14:30.045 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:14:30.045 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:14:30.046 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:14:30.046 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:14:30.046 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:14:30.047 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:14:30.047 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:14:30.048 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:14:30.048 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:14:30.048 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:14:30.049 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:14:30.120 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15057.asok 2021-04-03T03:14:30.122 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15057.asok 2021-04-03T03:14:30.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:30.123 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15057.asok status 2021-04-03T03:14:30.235 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:14:30.235 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:14:30.235 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:14:30.236 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:14:30.236 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:14:30.236 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:14:30.236 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:14:30.236 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15057", 2021-04-03T03:14:30.237 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:14:30.237 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:14:30.237 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:14:30.237 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:14:30.237 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 6420, 2021-04-03T03:14:30.238 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:14:30.238 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:14:30.238 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:14:30.238 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 6420 2021-04-03T03:14:30.238 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:14:30.239 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:14:30.239 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:14:30.239 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:14:30.239 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 421564197 2021-04-03T03:14:30.240 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:14:30.240 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:14:30.240 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:14:30.240 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:14:30.240 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:14:30.241 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 421564197 2021-04-03T03:14:30.241 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:14:30.241 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.6420 172.21.2.128:0/421564197", 2021-04-03T03:14:30.241 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/421564197", 2021-04-03T03:14:30.241 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:14:30.242 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 29, 2021-04-03T03:14:30.242 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 74, 2021-04-03T03:14:30.242 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:14:30.242 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:14:30.243 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:14:30.243 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:14:30.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:30.244 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:30.245 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:30.249 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:14:30.250 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:14:30.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:30.251 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:30.300 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:30.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 Processor -- start 2021-04-03T03:14:30.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- start start 2021-04-03T03:14:30.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fab600603f0 0x7fab600b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b5d60 0x7fab600b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600b4bc0 con 0x7fab600b44a0 2021-04-03T03:14:30.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600607d0 con 0x7fab600603f0 2021-04-03T03:14:30.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab600b4df0 con 0x7fab600b5d60 2021-04-03T03:14:30.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fab600603f0 0x7fab600b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fab600603f0 0x7fab600b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54010/0 (socket says 172.21.2.111:54010) 2021-04-03T03:14:30.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 -- 172.21.2.111:0/487567250 learned_addr learned my addr 172.21.2.111:0/487567250 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:30.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 -- 172.21.2.111:0/487567250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b5d60 msgr2=0x7fab600b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:30.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 --2- 172.21.2.111:0/487567250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b5d60 0x7fab600b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 -- 172.21.2.111:0/487567250 >> v1:172.21.2.115:6789/0 conn(0x7fab600b44a0 legacy=0x7fab600b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:30.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 -- 172.21.2.111:0/487567250 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600b5090 con 0x7fab600603f0 2021-04-03T03:14:30.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5ffff700 1 --2- 172.21.2.111:0/487567250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600603f0 0x7fab600b5530 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fab600b5410 tx=0x7fab50006e90).ready entity=mon.0 client_cookie=2d527e7bd092b5a2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:30.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5effd700 1 -- 172.21.2.111:0/487567250 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab500098e0 con 0x7fab600603f0 2021-04-03T03:14:30.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5effd700 1 -- 172.21.2.111:0/487567250 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50009a40 con 0x7fab600603f0 2021-04-03T03:14:30.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab5effd700 1 -- 172.21.2.111:0/487567250 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab50009de0 con 0x7fab600603f0 2021-04-03T03:14:30.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- 172.21.2.111:0/487567250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600603f0 msgr2=0x7fab600b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 --2- 172.21.2.111:0/487567250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600603f0 0x7fab600b5530 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fab600b5410 tx=0x7fab50006e90).stop 2021-04-03T03:14:30.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- 172.21.2.111:0/487567250 shutdown_connections 2021-04-03T03:14:30.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 --2- 172.21.2.111:0/487567250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600603f0 0x7fab600b5530 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 --2- 172.21.2.111:0/487567250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b5d60 0x7fab600b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- 172.21.2.111:0/487567250 >> 172.21.2.111:0/487567250 conn(0x7fab6005c7d0 msgr2=0x7fab6005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:30.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- 172.21.2.111:0/487567250 shutdown_connections 2021-04-03T03:14:30.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.353+0000 7fab65006700 1 -- 172.21.2.111:0/487567250 wait complete. 2021-04-03T03:14:30.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 Processor -- start 2021-04-03T03:14:30.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 -- start start 2021-04-03T03:14:30.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab600603f0 0x7fab60155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b44a0 0x7fab601563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600b5d60 0x7fab600ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab600b02f0 con 0x7fab600b5d60 2021-04-03T03:14:30.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab600ad250 con 0x7fab600603f0 2021-04-03T03:14:30.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab601477e0 con 0x7fab600b44a0 2021-04-03T03:14:30.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600b5d60 0x7fab600ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600b5d60 0x7fab600ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54014/0 (socket says 172.21.2.111:54014) 2021-04-03T03:14:30.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 -- 172.21.2.111:0/2636140149 learned_addr learned my addr 172.21.2.111:0/2636140149 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:30.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5ffff700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab600603f0 0x7fab60155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5f7fe700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b44a0 0x7fab601563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 -- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b44a0 msgr2=0x7fab601563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b44a0 0x7fab601563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 -- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab600603f0 msgr2=0x7fab60155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab600603f0 0x7fab60155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 -- 172.21.2.111:0/2636140149 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600b0520 con 0x7fab600b5d60 2021-04-03T03:14:30.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5ffff700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab600603f0 0x7fab60155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:30.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5f7fe700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b44a0 0x7fab601563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:14:30.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab64805700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600b5d60 0x7fab600ae5e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fab58003320 tx=0x7fab580090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:30.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5d7fa700 1 -- 172.21.2.111:0/2636140149 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab5800c960 con 0x7fab600b5d60 2021-04-03T03:14:30.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5d7fa700 1 -- 172.21.2.111:0/2636140149 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5800cac0 con 0x7fab600b5d60 2021-04-03T03:14:30.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab60153490 con 0x7fab600b5d60 2021-04-03T03:14:30.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5d7fa700 1 -- 172.21.2.111:0/2636140149 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab5800ce20 con 0x7fab600b5d60 2021-04-03T03:14:30.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab60146710 con 0x7fab600b5d60 2021-04-03T03:14:30.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab60057660 con 0x7fab600b5d60 2021-04-03T03:14:30.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5d7fa700 1 -- 172.21.2.111:0/2636140149 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fab5801c070 con 0x7fab600b5d60 2021-04-03T03:14:30.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5d7fa700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab40056fb0 0x7fab40059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5d7fa700 1 -- 172.21.2.111:0/2636140149 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fab5806fd70 con 0x7fab600b5d60 2021-04-03T03:14:30.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5ffff700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab40056fb0 0x7fab40059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.357+0000 7fab5ffff700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab40056fb0 0x7fab40059470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fab50017000 tx=0x7fab50015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:30.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.361+0000 7fab5d7fa700 1 -- 172.21.2.111:0/2636140149 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fab58045160 con 0x7fab600b5d60 2021-04-03T03:14:30.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.481+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab60069270 con 0x7fab600b5d60 2021-04-03T03:14:30.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.481+0000 7fab5d7fa700 1 -- 172.21.2.111:0/2636140149 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fab5801a030 con 0x7fab600b5d60 2021-04-03T03:14:30.492 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:30.493 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:14:30.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.481+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab40056fb0 msgr2=0x7fab40059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.481+0000 7fab65006700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab40056fb0 0x7fab40059470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fab50017000 tx=0x7fab50015000).stop 2021-04-03T03:14:30.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600b5d60 msgr2=0x7fab600ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600b5d60 0x7fab600ae5e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fab58003320 tx=0x7fab580090f0).stop 2021-04-03T03:14:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 shutdown_connections 2021-04-03T03:14:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab600b5d60 0x7fab600ae5e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fab40056fb0 0x7fab40059470 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab600603f0 0x7fab60155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 --2- 172.21.2.111:0/2636140149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab600b44a0 0x7fab601563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 >> 172.21.2.111:0/2636140149 conn(0x7fab6005c7d0 msgr2=0x7fab60142210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 shutdown_connections 2021-04-03T03:14:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.485+0000 7fab65006700 1 -- 172.21.2.111:0/2636140149 wait complete. 2021-04-03T03:14:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:14:30.500 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:14:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.549+0000 7f49b2449700 1 Processor -- start 2021-04-03T03:14:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.549+0000 7f49b2449700 1 -- start start 2021-04-03T03:14:30.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f49ac14b700 0x7f49ac0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f49ac0b0c90 0x7f49ac147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ac0af400 con 0x7f49ac0603f0 2021-04-03T03:14:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac060860 con 0x7f49ac14b700 2021-04-03T03:14:30.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0af6a0 con 0x7f49ac0b0c90 2021-04-03T03:14:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f49ac14b700 0x7f49ac0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f49ac14b700 0x7f49ac0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54022/0 (socket says 172.21.2.111:54022) 2021-04-03T03:14:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 -- 172.21.2.111:0/825765146 learned_addr learned my addr 172.21.2.111:0/825765146 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 -- 172.21.2.111:0/825765146 >> v2:172.21.2.115:3300/0 conn(0x7f49ac0b0c90 msgr2=0x7f49ac147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 --2- 172.21.2.111:0/825765146 >> v2:172.21.2.115:3300/0 conn(0x7f49ac0b0c90 0x7f49ac147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 -- 172.21.2.111:0/825765146 >> v1:172.21.2.115:6789/0 conn(0x7f49ac0603f0 legacy=0x7f49ac0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:14:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 -- 172.21.2.111:0/825765146 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0afb40 con 0x7f49ac14b700 2021-04-03T03:14:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b0c46700 1 --2- 172.21.2.111:0/825765146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac0b07a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f49ac06a3b0 tx=0x7f49a4006e90).ready entity=mon.0 client_cookie=a768d9440b7f5291 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49abfff700 1 -- 172.21.2.111:0/825765146 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a4009930 con 0x7f49ac14b700 2021-04-03T03:14:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49abfff700 1 -- 172.21.2.111:0/825765146 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a4009a90 con 0x7f49ac14b700 2021-04-03T03:14:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49abfff700 1 -- 172.21.2.111:0/825765146 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a4009e30 con 0x7f49ac14b700 2021-04-03T03:14:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- 172.21.2.111:0/825765146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 msgr2=0x7f49ac0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- 172.21.2.111:0/825765146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac0b07a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f49ac06a3b0 tx=0x7f49a4006e90).stop 2021-04-03T03:14:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- 172.21.2.111:0/825765146 shutdown_connections 2021-04-03T03:14:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- 172.21.2.111:0/825765146 >> v2:172.21.2.115:3300/0 conn(0x7f49ac0b0c90 0x7f49ac147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- 172.21.2.111:0/825765146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac0b07a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- 172.21.2.111:0/825765146 >> 172.21.2.111:0/825765146 conn(0x7f49ac0b61c0 msgr2=0x7f49ac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- 172.21.2.111:0/825765146 shutdown_connections 2021-04-03T03:14:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- 172.21.2.111:0/825765146 wait complete. 2021-04-03T03:14:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 Processor -- start 2021-04-03T03:14:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- start start 2021-04-03T03:14:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49ac0603f0 0x7f49ac062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49ac0b0c90 0x7f49ac067490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac067980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0646d0 con 0x7f49ac14b700 2021-04-03T03:14:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac064970 con 0x7f49ac0b0c90 2021-04-03T03:14:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac064c10 con 0x7f49ac0603f0 2021-04-03T03:14:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac067980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54026/0 (socket says 172.21.2.111:54026) 2021-04-03T03:14:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 -- 172.21.2.111:0/4008662249 learned_addr learned my addr 172.21.2.111:0/4008662249 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:14:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 -- 172.21.2.111:0/4008662249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49ac0603f0 msgr2=0x7f49ac062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49ac0603f0 0x7f49ac062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 -- 172.21.2.111:0/4008662249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49ac0b0c90 msgr2=0x7f49ac067490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49ac0b0c90 0x7f49ac067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 -- 172.21.2.111:0/4008662249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0ac060 con 0x7f49ac14b700 2021-04-03T03:14:30.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b1c48700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac067980 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f499c0032b0 tx=0x7f499c009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:30.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49aa7fc700 1 -- 172.21.2.111:0/4008662249 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f499c00ca60 con 0x7f49ac14b700 2021-04-03T03:14:30.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49aa7fc700 1 -- 172.21.2.111:0/4008662249 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c00cbc0 con 0x7f49ac14b700 2021-04-03T03:14:30.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac067e70 con 0x7f49ac14b700 2021-04-03T03:14:30.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49aa7fc700 1 -- 172.21.2.111:0/4008662249 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f499c015420 con 0x7f49ac14b700 2021-04-03T03:14:30.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.553+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac064440 con 0x7f49ac14b700 2021-04-03T03:14:30.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.557+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac0aa4d0 con 0x7f49ac14b700 2021-04-03T03:14:30.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.557+0000 7f49aa7fc700 1 -- 172.21.2.111:0/4008662249 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f499c01c070 con 0x7f49ac14b700 2021-04-03T03:14:30.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.557+0000 7f49aa7fc700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f498c056ea0 0x7f498c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:14:30.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.557+0000 7f49aa7fc700 1 -- 172.21.2.111:0/4008662249 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f499c06fe50 con 0x7f49ac14b700 2021-04-03T03:14:30.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.557+0000 7f49b1447700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f498c056ea0 0x7f498c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:14:30.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.557+0000 7f49b1447700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f498c056ea0 0x7f498c059360 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f49a0002f50 tx=0x7f49a0006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:14:30.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.557+0000 7f49aa7fc700 1 -- 172.21.2.111:0/4008662249 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f499c045130 con 0x7f49ac14b700 2021-04-03T03:14:30.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.685+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49ac064dd0 con 0x7f49ac14b700 2021-04-03T03:14:30.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.685+0000 7f49aa7fc700 1 -- 172.21.2.111:0/4008662249 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f499c071030 con 0x7f49ac14b700 2021-04-03T03:14:30.696 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:14:30.697 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:14:30.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f498c056ea0 msgr2=0x7f498c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f498c056ea0 0x7f498c059360 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f49a0002f50 tx=0x7f49a0006990).stop 2021-04-03T03:14:30.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 msgr2=0x7f49ac067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:14:30.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac067980 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f499c0032b0 tx=0x7f499c009220).stop 2021-04-03T03:14:30.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 shutdown_connections 2021-04-03T03:14:30.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49ac14b700 0x7f49ac067980 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49ac0603f0 0x7f49ac062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f498c056ea0 0x7f498c059360 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:14:30.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 --2- 172.21.2.111:0/4008662249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49ac0b0c90 0x7f49ac067490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:14:30.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 >> 172.21.2.111:0/4008662249 conn(0x7f49ac0b61c0 msgr2=0x7f49ac0acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:14:30.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 shutdown_connections 2021-04-03T03:14:30.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:14:30.689+0000 7f49b2449700 1 -- 172.21.2.111:0/4008662249 wait complete. 2021-04-03T03:14:30.700 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:14:30.704 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2021-04-03T03:14:30.768 INFO:teuthology.orchestra.run.gibba011.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2021-04-03T03:14:30.768 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2021-04-03T03:14:30.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:30.769 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:30.769 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:14:30.780 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:14:30.781 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:14:30.782 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:14:30.782 DEBUG:teuthology.orchestra.run.gibba026:> import time 2021-04-03T03:14:30.782 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:14:30.783 DEBUG:teuthology.orchestra.run.gibba026:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2021-04-03T03:14:30.783 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content'"'"') 2021-04-03T03:14:30.783 DEBUG:teuthology.orchestra.run.gibba026:> f.flush() 2021-04-03T03:14:30.784 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content2'"'"') 2021-04-03T03:14:30.784 DEBUG:teuthology.orchestra.run.gibba026:> while True: 2021-04-03T03:14:30.785 DEBUG:teuthology.orchestra.run.gibba026:> time.sleep(1) 2021-04-03T03:14:30.785 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:14:30.825 DEBUG:teuthology.orchestra.run.gibba026:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2021-04-03T03:14:30.838 INFO:teuthology.orchestra.run.gibba026.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2021-04-03T03:14:30.839 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-04-03T03:14:31.841 DEBUG:teuthology.orchestra.run.gibba026:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2021-04-03T03:14:31.852 INFO:teuthology.orchestra.run.gibba026.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2021-04-03T03:14:31.854 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2021-04-03T03:14:31.854 DEBUG:teuthology.orchestra.run.gibba028:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2021-04-03T03:14:31.899 INFO:teuthology.orchestra.run.gibba028.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2021-04-03T03:14:31.900 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2021-04-03T03:14:31.901 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:14:31.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:14:31.913 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:14:31.913 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:14:31.923 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:14:31.924 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:14:31.925 DEBUG:teuthology.orchestra.run.gibba028:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:14:31.925 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:14:31.926 DEBUG:teuthology.orchestra.run.gibba028:> import time 2021-04-03T03:14:31.926 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:14:31.926 DEBUG:teuthology.orchestra.run.gibba028:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2021-04-03T03:14:31.927 DEBUG:teuthology.orchestra.run.gibba028:> try: 2021-04-03T03:14:31.927 DEBUG:teuthology.orchestra.run.gibba028:> while True: 2021-04-03T03:14:31.928 DEBUG:teuthology.orchestra.run.gibba028:> os.write(fd, b'"'"'content'"'"') 2021-04-03T03:14:31.928 DEBUG:teuthology.orchestra.run.gibba028:> time.sleep(1) 2021-04-03T03:14:31.928 DEBUG:teuthology.orchestra.run.gibba028:> if not False: 2021-04-03T03:14:31.929 DEBUG:teuthology.orchestra.run.gibba028:> break 2021-04-03T03:14:31.929 DEBUG:teuthology.orchestra.run.gibba028:> except IOError as e: 2021-04-03T03:14:31.929 DEBUG:teuthology.orchestra.run.gibba028:> pass 2021-04-03T03:14:31.930 DEBUG:teuthology.orchestra.run.gibba028:> os.close(fd) 2021-04-03T03:14:31.930 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:14:59.070 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:59.074 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:59.077 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:59.080 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:14:59.084 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:11.125 INFO:tasks.cephfs.test_misc:cap_waiter waited 39.15642333030701s 2021-04-03T03:15:11.128 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist ls --format=json 2021-04-03T03:15:11.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 Processor -- start 2021-04-03T03:15:11.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- start start 2021-04-03T03:15:11.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5b8c0b25b0 0x7f5b8c13e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8c0afd00 con 0x7f5b8c1420d0 2021-04-03T03:15:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8c144a80 con 0x7f5b8c0af920 2021-04-03T03:15:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b8c141330 con 0x7f5b8c0b25b0 2021-04-03T03:15:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b91792700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5b8c1420d0 0x7f5b8c13e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54828/0 (socket says 172.21.2.111:54828) 2021-04-03T03:15:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b91792700 1 -- 172.21.2.111:0/2038363196 learned_addr learned my addr 172.21.2.111:0/2038363196 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92794700 1 --2- 172.21.2.111:0/2038363196 >> v2:172.21.2.111:3300/0 conn(0x7f5b8c0b25b0 0x7f5b8c13e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b90f91700 1 -- 172.21.2.111:0/2038363196 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562449728 0 0) 0x7f5b8c0afd00 con 0x7f5b8c1420d0 2021-04-03T03:15:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b90f91700 1 -- 172.21.2.111:0/2038363196 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b74003240 con 0x7f5b8c1420d0 2021-04-03T03:15:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92794700 1 -- 172.21.2.111:0/2038363196 >> v1:172.21.2.115:6789/0 conn(0x7f5b8c0af920 legacy=0x7f5b8c1419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92794700 1 -- 172.21.2.111:0/2038363196 >> v1:172.21.2.111:6789/0 conn(0x7f5b8c1420d0 legacy=0x7f5b8c13e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92794700 1 -- 172.21.2.111:0/2038363196 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c141520 con 0x7f5b8c0b25b0 2021-04-03T03:15:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92794700 1 --2- 172.21.2.111:0/2038363196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13e970 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f5b8c060aa0 tx=0x7f5b840091d0).ready entity=mon.0 client_cookie=9eb69b09d2215d3c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b90f91700 1 -- 172.21.2.111:0/2038363196 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8400b960 con 0x7f5b8c0b25b0 2021-04-03T03:15:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b90f91700 1 -- 172.21.2.111:0/2038363196 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8400bac0 con 0x7f5b8c0b25b0 2021-04-03T03:15:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b90f91700 1 -- 172.21.2.111:0/2038363196 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8400bc30 con 0x7f5b8c0b25b0 2021-04-03T03:15:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- 172.21.2.111:0/2038363196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c0b25b0 msgr2=0x7f5b8c13e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 --2- 172.21.2.111:0/2038363196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13e970 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f5b8c060aa0 tx=0x7f5b840091d0).stop 2021-04-03T03:15:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- 172.21.2.111:0/2038363196 shutdown_connections 2021-04-03T03:15:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 --2- 172.21.2.111:0/2038363196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13e970 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- 172.21.2.111:0/2038363196 >> 172.21.2.111:0/2038363196 conn(0x7f5b8c0b5fb0 msgr2=0x7f5b8c0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- 172.21.2.111:0/2038363196 shutdown_connections 2021-04-03T03:15:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 -- 172.21.2.111:0/2038363196 wait complete. 2021-04-03T03:15:11.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.186+0000 7f5b92f95700 1 Processor -- start 2021-04-03T03:15:11.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 -- start start 2021-04-03T03:15:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5b8c0af920 0x7f5b8c148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c1420d0 0x7f5b8c13f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0ac740 con 0x7f5b8c1420d0 2021-04-03T03:15:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c13fa00 con 0x7f5b8c0b25b0 2021-04-03T03:15:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0aecb0 con 0x7f5b8c0af920 2021-04-03T03:15:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c1420d0 0x7f5b8c13f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c1420d0 0x7f5b8c13f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54044/0 (socket says 172.21.2.111:54044) 2021-04-03T03:15:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 -- 172.21.2.111:0/99190908 learned_addr learned my addr 172.21.2.111:0/99190908 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 -- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5b8c0af920 msgr2=0x7f5b8c148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b91f93700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5b8c0af920 0x7f5b8c148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b91792700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5b8c0af920 0x7f5b8c148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 -- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5b8c0b25b0 msgr2=0x7f5b8c13f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 -- 172.21.2.111:0/99190908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c140e30 con 0x7f5b8c1420d0 2021-04-03T03:15:11.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b91792700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:11.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92794700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c1420d0 0x7f5b8c13f510 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f5b8c060aa0 tx=0x7f5b84016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b7affd700 1 -- 172.21.2.111:0/99190908 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8401f3f0 con 0x7f5b8c1420d0 2021-04-03T03:15:11.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b7affd700 1 -- 172.21.2.111:0/99190908 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8401f550 con 0x7f5b8c1420d0 2021-04-03T03:15:11.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b7affd700 1 -- 172.21.2.111:0/99190908 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8401f8b0 con 0x7f5b8c1420d0 2021-04-03T03:15:11.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8c0aeee0 con 0x7f5b8c1420d0 2021-04-03T03:15:11.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8c067a40 con 0x7f5b8c1420d0 2021-04-03T03:15:11.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8c13ea70 con 0x7f5b8c1420d0 2021-04-03T03:15:11.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b7affd700 1 -- 172.21.2.111:0/99190908 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5b8400f070 con 0x7f5b8c1420d0 2021-04-03T03:15:11.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b7affd700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5b6c056f20 0x7f5b6c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b7affd700 1 -- 172.21.2.111:0/99190908 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f5b84070f30 con 0x7f5b8c1420d0 2021-04-03T03:15:11.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b91f93700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5b6c056f20 0x7f5b6c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.190+0000 7f5b91f93700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5b6c056f20 0x7f5b6c0593e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f5b7c008000 tx=0x7f5b7c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.194+0000 7f5b7affd700 1 -- 172.21.2.111:0/99190908 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5b840462f0 con 0x7f5b8c1420d0 2021-04-03T03:15:11.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7f5b8c069270 con 0x7f5b8c1420d0 2021-04-03T03:15:11.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b7affd700 1 -- 172.21.2.111:0/99190908 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v75) v1 ==== 98+0+80 (secure 0 0 0) 0x7f5b84015070 con 0x7f5b8c1420d0 2021-04-03T03:15:11.300 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:11.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5b6c056f20 msgr2=0x7f5b6c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5b6c056f20 0x7f5b6c0593e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f5b7c008000 tx=0x7f5b7c009040).stop 2021-04-03T03:15:11.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c1420d0 msgr2=0x7f5b8c13f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c1420d0 0x7f5b8c13f510 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f5b8c060aa0 tx=0x7f5b84016000).stop 2021-04-03T03:15:11.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 shutdown_connections 2021-04-03T03:15:11.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5b8c1420d0 0x7f5b8c13f510 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5b8c0af920 0x7f5b8c148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5b6c056f20 0x7f5b6c0593e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 --2- 172.21.2.111:0/99190908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5b8c0b25b0 0x7f5b8c13f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 >> 172.21.2.111:0/99190908 conn(0x7f5b8c0b5fb0 msgr2=0x7f5b8c0b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:11.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 shutdown_connections 2021-04-03T03:15:11.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.294+0000 7f5b92f95700 1 -- 172.21.2.111:0/99190908 wait complete. 2021-04-03T03:15:11.304 INFO:teuthology.orchestra.run.gibba011.stderr:listed 1 entries 2021-04-03T03:15:11.307 INFO:teuthology.orchestra.run.gibba011.stdout:[{"addr":"172.21.2.126:0/2003837950","until":"2021-04-03T04:15:04.941896+0000"}] 2021-04-03T03:15:11.319 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-04-03T03:15:11.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:11.320 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:11.320 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:11.370 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-04-03T03:15:11.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:11.373 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:11.373 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:15:11.374 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:15:11.375 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:11.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:11.448 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:11.448 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:11.494 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:15:11.494 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:15:11.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:11.495 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:11.546 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:11.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 Processor -- start 2021-04-03T03:15:11.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- start start 2021-04-03T03:15:11.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe6500603f0 0x7fe6500b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe65014b700 0x7fe6500b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 0x7fe6500b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6500b0f60 con 0x7fe6500603f0 2021-04-03T03:15:11.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6500607d0 con 0x7fe65014b700 2021-04-03T03:15:11.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6500af400 con 0x7fe6500b12b0 2021-04-03T03:15:11.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe6500603f0 0x7fe6500b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe6500603f0 0x7fe6500b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54050/0 (socket says 172.21.2.111:54050) 2021-04-03T03:15:11.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 -- 172.21.2.111:0/2553881013 learned_addr learned my addr 172.21.2.111:0/2553881013 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:11.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 -- 172.21.2.111:0/2553881013 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 msgr2=0x7fe6500b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 --2- 172.21.2.111:0/2553881013 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 0x7fe6500b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 -- 172.21.2.111:0/2553881013 >> v2:172.21.2.115:3300/0 conn(0x7fe65014b700 msgr2=0x7fe6500b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 --2- 172.21.2.111:0/2553881013 >> v2:172.21.2.115:3300/0 conn(0x7fe65014b700 0x7fe6500b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 -- 172.21.2.111:0/2553881013 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6500af6a0 con 0x7fe6500603f0 2021-04-03T03:15:11.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64e384700 1 --2- 172.21.2.111:0/2553881013 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500b0090 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fe638002f30 tx=0x7fe638006e90).ready entity=mon.0 client_cookie=db83256e9013cf18 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64d382700 1 -- 172.21.2.111:0/2553881013 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe638009920 con 0x7fe6500603f0 2021-04-03T03:15:11.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64d382700 1 -- 172.21.2.111:0/2553881013 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe638009a80 con 0x7fe6500603f0 2021-04-03T03:15:11.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64d382700 1 -- 172.21.2.111:0/2553881013 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe638009de0 con 0x7fe6500603f0 2021-04-03T03:15:11.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- 172.21.2.111:0/2553881013 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 msgr2=0x7fe6500b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 --2- 172.21.2.111:0/2553881013 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500b0090 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fe638002f30 tx=0x7fe638006e90).stop 2021-04-03T03:15:11.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- 172.21.2.111:0/2553881013 shutdown_connections 2021-04-03T03:15:11.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 --2- 172.21.2.111:0/2553881013 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 0x7fe6500b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 --2- 172.21.2.111:0/2553881013 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500b0090 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 --2- 172.21.2.111:0/2553881013 >> v2:172.21.2.115:3300/0 conn(0x7fe65014b700 0x7fe6500b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- 172.21.2.111:0/2553881013 >> 172.21.2.111:0/2553881013 conn(0x7fe6500b61c0 msgr2=0x7fe65005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:11.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- 172.21.2.111:0/2553881013 shutdown_connections 2021-04-03T03:15:11.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.606+0000 7fe64f386700 1 -- 172.21.2.111:0/2553881013 wait complete. 2021-04-03T03:15:11.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 Processor -- start 2021-04-03T03:15:11.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 -- start start 2021-04-03T03:15:11.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500aa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 0x7fe650067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe65014b700 0x7fe650067950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6500a99c0 con 0x7fe6500603f0 2021-04-03T03:15:11.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe650064380 con 0x7fe65014b700 2021-04-03T03:15:11.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe650064620 con 0x7fe6500b12b0 2021-04-03T03:15:11.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54056/0 (socket says 172.21.2.111:54056) 2021-04-03T03:15:11.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 -- 172.21.2.111:0/3025432686 learned_addr learned my addr 172.21.2.111:0/3025432686 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:11.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 -- 172.21.2.111:0/3025432686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 msgr2=0x7fe650067460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 0x7fe650067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 -- 172.21.2.111:0/3025432686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe65014b700 msgr2=0x7fe650067950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe65014b700 0x7fe650067950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 -- 172.21.2.111:0/3025432686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6500632a0 con 0x7fe6500603f0 2021-04-03T03:15:11.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64e384700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500aa540 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fe638002f30 tx=0x7fe638015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe6477fe700 1 -- 172.21.2.111:0/3025432686 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe638009e00 con 0x7fe6500603f0 2021-04-03T03:15:11.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe6477fe700 1 -- 172.21.2.111:0/3025432686 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63801d5d0 con 0x7fe6500603f0 2021-04-03T03:15:11.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6500abcc0 con 0x7fe6500603f0 2021-04-03T03:15:11.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe6477fe700 1 -- 172.21.2.111:0/3025432686 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe63801d930 con 0x7fe6500603f0 2021-04-03T03:15:11.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe650067fd0 con 0x7fe6500603f0 2021-04-03T03:15:11.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe650065080 con 0x7fe6500603f0 2021-04-03T03:15:11.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe6477fe700 1 -- 172.21.2.111:0/3025432686 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe638025030 con 0x7fe6500603f0 2021-04-03T03:15:11.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe6477fe700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe634056ef0 0x7fe6340593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe6477fe700 1 -- 172.21.2.111:0/3025432686 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe6380719d0 con 0x7fe6500603f0 2021-04-03T03:15:11.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64db83700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe634056ef0 0x7fe6340593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.610+0000 7fe64db83700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe634056ef0 0x7fe6340593b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe640002f30 tx=0x7fe640006970).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.614+0000 7fe6477fe700 1 -- 172.21.2.111:0/3025432686 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe638046c80 con 0x7fe6500603f0 2021-04-03T03:15:11.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe650057660 con 0x7fe6500603f0 2021-04-03T03:15:11.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe6477fe700 1 -- 172.21.2.111:0/3025432686 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16335 (secure 0 0 0) 0x7fe63804a2a0 con 0x7fe6500603f0 2021-04-03T03:15:11.712 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:11.712 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":75,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:05.913226+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:14:22.039727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-04-03T03:14:23.053211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.126:0/2003837950":"2021-04-03T04:15:04.941896+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:11.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe634056ef0 msgr2=0x7fe6340593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe634056ef0 0x7fe6340593b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe640002f30 tx=0x7fe640006970).stop 2021-04-03T03:15:11.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 msgr2=0x7fe6500aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500aa540 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fe638002f30 tx=0x7fe638015000).stop 2021-04-03T03:15:11.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 shutdown_connections 2021-04-03T03:15:11.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe634056ef0 0x7fe6340593b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe65014b700 0x7fe650067950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6500603f0 0x7fe6500aa540 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 --2- 172.21.2.111:0/3025432686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6500b12b0 0x7fe650067460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 >> 172.21.2.111:0/3025432686 conn(0x7fe6500b61c0 msgr2=0x7fe65014a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:11.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 shutdown_connections 2021-04-03T03:15:11.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.706+0000 7fe64f386700 1 -- 172.21.2.111:0/3025432686 wait complete. 2021-04-03T03:15:11.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:14:22.039727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:14:23.053211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:15:11.720 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:11.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.770+0000 7f3ace9ca700 1 Processor -- start 2021-04-03T03:15:11.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.770+0000 7f3ace9ca700 1 -- start start 2021-04-03T03:15:11.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3ac80603f0 0x7f3ac81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3ac80add50 0x7f3ac8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80607d0 con 0x7f3ac80b17b0 2021-04-03T03:15:11.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac81451c0 con 0x7f3ac80add50 2021-04-03T03:15:11.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac814b090 con 0x7f3ac80603f0 2021-04-03T03:15:11.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3ac80b17b0 0x7f3ac8147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54860/0 (socket says 172.21.2.111:54860) 2021-04-03T03:15:11.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 -- 172.21.2.111:0/662668104 learned_addr learned my addr 172.21.2.111:0/662668104 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:11.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acd1c7700 1 --2- 172.21.2.111:0/662668104 >> v2:172.21.2.111:3300/0 conn(0x7f3ac80add50 0x7f3ac8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acd1c7700 1 -- 172.21.2.111:0/662668104 >> v2:172.21.2.115:3300/0 conn(0x7f3ac80603f0 msgr2=0x7f3ac81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acd1c7700 1 --2- 172.21.2.111:0/662668104 >> v2:172.21.2.115:3300/0 conn(0x7f3ac80603f0 0x7f3ac81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acd1c7700 1 -- 172.21.2.111:0/662668104 >> v1:172.21.2.111:6789/0 conn(0x7f3ac80b17b0 legacy=0x7f3ac8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acd1c7700 1 -- 172.21.2.111:0/662668104 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac814b330 con 0x7f3ac80add50 2021-04-03T03:15:11.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acd1c7700 1 --2- 172.21.2.111:0/662668104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80add50 0x7f3ac8144cd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f3ac4002f30 tx=0x7f3ac4006dd0).ready entity=mon.0 client_cookie=45ac31a2c12dce89 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acc9c6700 1 -- 172.21.2.111:0/662668104 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ac40097f0 con 0x7f3ac80add50 2021-04-03T03:15:11.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acc9c6700 1 -- 172.21.2.111:0/662668104 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4009950 con 0x7f3ac80add50 2021-04-03T03:15:11.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3acc9c6700 1 -- 172.21.2.111:0/662668104 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ac4009cb0 con 0x7f3ac80add50 2021-04-03T03:15:11.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- 172.21.2.111:0/662668104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80add50 msgr2=0x7f3ac8144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/662668104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80add50 0x7f3ac8144cd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f3ac4002f30 tx=0x7f3ac4006dd0).stop 2021-04-03T03:15:11.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- 172.21.2.111:0/662668104 shutdown_connections 2021-04-03T03:15:11.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/662668104 >> v2:172.21.2.115:3300/0 conn(0x7f3ac80603f0 0x7f3ac81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/662668104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80add50 0x7f3ac8144cd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- 172.21.2.111:0/662668104 >> 172.21.2.111:0/662668104 conn(0x7f3ac80b61c0 msgr2=0x7f3ac805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:11.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- 172.21.2.111:0/662668104 shutdown_connections 2021-04-03T03:15:11.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- 172.21.2.111:0/662668104 wait complete. 2021-04-03T03:15:11.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 Processor -- start 2021-04-03T03:15:11.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- start start 2021-04-03T03:15:11.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ac80603f0 0x7f3ac80a8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ac80add50 0x7f3ac8148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80b17b0 0x7f3ac8064ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8159f60 con 0x7f3ac80b17b0 2021-04-03T03:15:11.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8065150 con 0x7f3ac80add50 2021-04-03T03:15:11.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80653f0 con 0x7f3ac80603f0 2021-04-03T03:15:11.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80b17b0 0x7f3ac8064ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80b17b0 0x7f3ac8064ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54074/0 (socket says 172.21.2.111:54074) 2021-04-03T03:15:11.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 -- 172.21.2.111:0/3693846136 learned_addr learned my addr 172.21.2.111:0/3693846136 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:11.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 -- 172.21.2.111:0/3693846136 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ac80603f0 msgr2=0x7f3ac80a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ac80603f0 0x7f3ac80a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 -- 172.21.2.111:0/3693846136 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ac80add50 msgr2=0x7f3ac8148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ac80add50 0x7f3ac8148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 -- 172.21.2.111:0/3693846136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac815a390 con 0x7f3ac80b17b0 2021-04-03T03:15:11.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace1c9700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80b17b0 0x7f3ac8064ad0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f3ac80ae520 tx=0x7f3ac0007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3abaffd700 1 -- 172.21.2.111:0/3693846136 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ac0002060 con 0x7f3ac80b17b0 2021-04-03T03:15:11.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3abaffd700 1 -- 172.21.2.111:0/3693846136 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0005b80 con 0x7f3ac80b17b0 2021-04-03T03:15:11.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac815a640 con 0x7f3ac80b17b0 2021-04-03T03:15:11.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3abaffd700 1 -- 172.21.2.111:0/3693846136 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ac0007110 con 0x7f3ac80b17b0 2021-04-03T03:15:11.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.774+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac815aad0 con 0x7f3ac80b17b0 2021-04-03T03:15:11.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.778+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac8064fc0 con 0x7f3ac80b17b0 2021-04-03T03:15:11.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.778+0000 7f3abaffd700 1 -- 172.21.2.111:0/3693846136 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3ac0010030 con 0x7f3ac80b17b0 2021-04-03T03:15:11.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.778+0000 7f3abaffd700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3aa8056f20 0x7f3aa80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.778+0000 7f3abaffd700 1 -- 172.21.2.111:0/3693846136 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3ac0070170 con 0x7f3ac80b17b0 2021-04-03T03:15:11.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.778+0000 7f3acd9c8700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3aa8056f20 0x7f3aa80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.778+0000 7f3acd9c8700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3aa8056f20 0x7f3aa80593e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3abc002f30 tx=0x7f3abc006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.778+0000 7f3abaffd700 1 -- 172.21.2.111:0/3693846136 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3ac00454b0 con 0x7f3ac80b17b0 2021-04-03T03:15:11.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.870+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3ac815a0a0 con 0x7f3ac80b17b0 2021-04-03T03:15:11.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.870+0000 7f3abaffd700 1 -- 172.21.2.111:0/3693846136 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v75) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3ac0048ad0 con 0x7f3ac80b17b0 2021-04-03T03:15:11.877 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:11.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3aa8056f20 msgr2=0x7f3aa80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3aa8056f20 0x7f3aa80593e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3abc002f30 tx=0x7f3abc006a60).stop 2021-04-03T03:15:11.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80b17b0 msgr2=0x7f3ac8064ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80b17b0 0x7f3ac8064ad0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f3ac80ae520 tx=0x7f3ac0007f60).stop 2021-04-03T03:15:11.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 shutdown_connections 2021-04-03T03:15:11.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ac80b17b0 0x7f3ac8064ad0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ac80603f0 0x7f3ac80a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3aa8056f20 0x7f3aa80593e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 --2- 172.21.2.111:0/3693846136 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ac80add50 0x7f3ac8148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 >> 172.21.2.111:0/3693846136 conn(0x7f3ac80b61c0 msgr2=0x7f3ac814a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:11.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 shutdown_connections 2021-04-03T03:15:11.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.874+0000 7f3ace9ca700 1 -- 172.21.2.111:0/3693846136 wait complete. 2021-04-03T03:15:11.885 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:11.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 Processor -- start 2021-04-03T03:15:11.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 -- start start 2021-04-03T03:15:11.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f127c0dbba0 0x7f127c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c041060 0x7f127c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127c0dc560 con 0x7f127c03d560 2021-04-03T03:15:11.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127c03de60 con 0x7f127c0dbba0 2021-04-03T03:15:11.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0dc800 con 0x7f127c041060 2021-04-03T03:15:11.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f128899f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f127c03d560 0x7f127c041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54874/0 (socket says 172.21.2.111:54874) 2021-04-03T03:15:11.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f128899f700 1 -- 172.21.2.111:0/4943796 learned_addr learned my addr 172.21.2.111:0/4943796 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:11.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002094299 0 0) 0x7f127c0dc560 con 0x7f127c03d560 2021-04-03T03:15:11.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1270003240 con 0x7f127c03d560 2021-04-03T03:15:11.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 --2- 172.21.2.111:0/4943796 >> v2:172.21.2.115:3300/0 conn(0x7f127c0dbba0 0x7f127c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12837fe700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c041060 0x7f127c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3902664712 0 0) 0x7f1270003240 con 0x7f127c03d560 2021-04-03T03:15:11.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c041060 msgr2=0x7f127c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c041060 0x7f127c03fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 >> v2:172.21.2.115:3300/0 conn(0x7f127c0dbba0 msgr2=0x7f127c0dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 --2- 172.21.2.111:0/4943796 >> v2:172.21.2.115:3300/0 conn(0x7f127c0dbba0 0x7f127c0dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:11.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c03da30 con 0x7f127c03d560 2021-04-03T03:15:11.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f12840450f0 con 0x7f127c03d560 2021-04-03T03:15:11.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 >> v1:172.21.2.111:6789/0 conn(0x7f127c03d560 legacy=0x7f127c041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12700037b0 0x7f1270003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c041060 0x7f12700044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12700049d0 0x7f1270004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c03de60 con 0x7f12700037b0 2021-04-03T03:15:11.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0dc800 con 0x7f12700049d0 2021-04-03T03:15:11.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12700041b0 con 0x7f127c041060 2021-04-03T03:15:11.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 --2- 172.21.2.111:0/4943796 >> v2:172.21.2.115:3300/0 conn(0x7f127c0dbba0 0x7f127c0dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:11.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12700037b0 0x7f1270003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 -- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c041060 msgr2=0x7f12700044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c041060 0x7f12700044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 -- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12700049d0 msgr2=0x7f1270004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:11.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12700049d0 0x7f1270004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 -- 172.21.2.111:0/4943796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1270003240 con 0x7f12700037b0 2021-04-03T03:15:11.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1283fff700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12700037b0 0x7f1270003bd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f12700054b0 tx=0x7f1278008040).ready entity=mon.0 client_cookie=881b63e3dc0aa16a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1278015070 con 0x7f12700037b0 2021-04-03T03:15:11.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127800c930 con 0x7f12700037b0 2021-04-03T03:15:11.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f1282ffd700 1 -- 172.21.2.111:0/4943796 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f127800cc90 con 0x7f12700037b0 2021-04-03T03:15:11.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 -- 172.21.2.111:0/4943796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12700037b0 msgr2=0x7f1270003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f12891a0700 1 --2- 172.21.2.111:0/4943796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12700037b0 0x7f1270003bd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f12700054b0 tx=0x7f1278008040).stop 2021-04-03T03:15:11.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.938+0000 7f128899f700 1 -- 172.21.2.111:0/4943796 reap_dead start 2021-04-03T03:15:11.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- 172.21.2.111:0/4943796 shutdown_connections 2021-04-03T03:15:11.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- 172.21.2.111:0/4943796 >> 172.21.2.111:0/4943796 conn(0x7f127c0462b0 msgr2=0x7f127c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:11.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- 172.21.2.111:0/4943796 shutdown_connections 2021-04-03T03:15:11.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- 172.21.2.111:0/4943796 wait complete. 2021-04-03T03:15:11.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 Processor -- start 2021-04-03T03:15:11.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- start start 2021-04-03T03:15:11.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127c0a2e20 0x7f127c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f127c0a0970 0x7f127c0a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c03c150 0x7f127c0ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0e78e0 con 0x7f127c0a2e20 2021-04-03T03:15:11.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0a3200 con 0x7f127c0a0970 2021-04-03T03:15:11.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c039670 con 0x7f127c03c150 2021-04-03T03:15:11.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127c0a2e20 0x7f127c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127c0a2e20 0x7f127c0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54090/0 (socket says 172.21.2.111:54090) 2021-04-03T03:15:11.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 -- 172.21.2.111:0/1747710670 learned_addr learned my addr 172.21.2.111:0/1747710670 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:11.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 -- 172.21.2.111:0/1747710670 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c03c150 msgr2=0x7f127c0ea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c03c150 0x7f127c0ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 -- 172.21.2.111:0/1747710670 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f127c0a0970 msgr2=0x7f127c0a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:11.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f127c0a0970 0x7f127c0a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:11.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 -- 172.21.2.111:0/1747710670 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c09f620 con 0x7f127c0a2e20 2021-04-03T03:15:11.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f1283fff700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127c0a2e20 0x7f127c0a0d90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f12700054b0 tx=0x7f1278008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12817fa700 1 -- 172.21.2.111:0/1747710670 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f127801b450 con 0x7f127c0a2e20 2021-04-03T03:15:11.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12817fa700 1 -- 172.21.2.111:0/1747710670 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127801b5b0 con 0x7f127c0a2e20 2021-04-03T03:15:11.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c0a04c0 con 0x7f127c0a2e20 2021-04-03T03:15:11.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12817fa700 1 -- 172.21.2.111:0/1747710670 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f127801b950 con 0x7f127c0a2e20 2021-04-03T03:15:11.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c0e7f60 con 0x7f127c0a2e20 2021-04-03T03:15:11.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c0e7a50 con 0x7f127c0a2e20 2021-04-03T03:15:11.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12817fa700 1 -- 172.21.2.111:0/1747710670 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f127801bab0 con 0x7f127c0a2e20 2021-04-03T03:15:11.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12817fa700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1264056f20 0x7f12640593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:11.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12817fa700 1 -- 172.21.2.111:0/1747710670 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1278071430 con 0x7f127c0a2e20 2021-04-03T03:15:11.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12837fe700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1264056f20 0x7f12640593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:11.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.942+0000 7f12837fe700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1264056f20 0x7f12640593e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f1274007000 tx=0x7f1274008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:11.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:11.946+0000 7f12817fa700 1 -- 172.21.2.111:0/1747710670 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f12780466e0 con 0x7f127c0a2e20 2021-04-03T03:15:12.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.038+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f127c004db0 con 0x7f127c0a2e20 2021-04-03T03:15:12.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.038+0000 7f12817fa700 1 -- 172.21.2.111:0/1747710670 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1278049d00 con 0x7f127c0a2e20 2021-04-03T03:15:12.045 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:12.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1264056f20 msgr2=0x7f12640593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1264056f20 0x7f12640593e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f1274007000 tx=0x7f1274008040).stop 2021-04-03T03:15:12.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127c0a2e20 msgr2=0x7f127c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127c0a2e20 0x7f127c0a0d90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f12700054b0 tx=0x7f1278008040).stop 2021-04-03T03:15:12.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 shutdown_connections 2021-04-03T03:15:12.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f127c03c150 0x7f127c0ea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f127c0a2e20 0x7f127c0a0d90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1264056f20 0x7f12640593e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 --2- 172.21.2.111:0/1747710670 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f127c0a0970 0x7f127c0a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 >> 172.21.2.111:0/1747710670 conn(0x7f127c0462b0 msgr2=0x7f127c041850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:12.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 shutdown_connections 2021-04-03T03:15:12.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.042+0000 7f12891a0700 1 -- 172.21.2.111:0/1747710670 wait complete. 2021-04-03T03:15:12.053 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:15:12.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d21d8700 1 Processor -- start 2021-04-03T03:15:12.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d21d8700 1 -- start start 2021-04-03T03:15:12.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d21d8700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa9cc0603f0 0x7fa9cc0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d21d8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b5620 0x7fa9cc142310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d21d8700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc0607d0 con 0x7fa9cc0b44a0 2021-04-03T03:15:12.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d21d8700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc1433c0 con 0x7fa9cc0603f0 2021-04-03T03:15:12.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d21d8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0b3f80 con 0x7fa9cc0b5620 2021-04-03T03:15:12.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d19d7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b5620 0x7fa9cc142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa9cc0603f0 0x7fa9cc0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa9cc0603f0 0x7fa9cc0b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:43806/0 (socket says 172.21.2.111:43806) 2021-04-03T03:15:12.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 -- 172.21.2.111:0/2125354882 learned_addr learned my addr 172.21.2.111:0/2125354882 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:12.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 -- 172.21.2.111:0/2125354882 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b5620 msgr2=0x7fa9cc142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:12.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 --2- 172.21.2.111:0/2125354882 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b5620 0x7fa9cc142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 -- 172.21.2.111:0/2125354882 >> v1:172.21.2.115:6789/0 conn(0x7fa9cc0b44a0 legacy=0x7fa9cc0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 -- 172.21.2.111:0/2125354882 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0b4c60 con 0x7fa9cc0603f0 2021-04-03T03:15:12.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9bffff700 1 -- 172.21.2.111:0/2125354882 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392406405 0 0) 0x7fa9cc0607d0 con 0x7fa9cc0b44a0 2021-04-03T03:15:12.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9d11d6700 1 --2- 172.21.2.111:0/2125354882 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 0x7fa9cc0b2480 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fa9cc0b2320 tx=0x7fa9c8006e90).ready entity=mon.1 client_cookie=922da9979f007e5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9bffff700 1 -- 172.21.2.111:0/2125354882 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c8009810 con 0x7fa9cc0603f0 2021-04-03T03:15:12.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9bffff700 1 -- 172.21.2.111:0/2125354882 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8009970 con 0x7fa9cc0603f0 2021-04-03T03:15:12.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.106+0000 7fa9bffff700 1 -- 172.21.2.111:0/2125354882 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c8009cd0 con 0x7fa9cc0603f0 2021-04-03T03:15:12.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2125354882 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 msgr2=0x7fa9cc0b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2125354882 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 0x7fa9cc0b2480 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fa9cc0b2320 tx=0x7fa9c8006e90).stop 2021-04-03T03:15:12.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2125354882 shutdown_connections 2021-04-03T03:15:12.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2125354882 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 0x7fa9cc0b2480 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2125354882 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b5620 0x7fa9cc142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2125354882 >> 172.21.2.111:0/2125354882 conn(0x7fa9cc05c7d0 msgr2=0x7fa9cc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:12.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2125354882 shutdown_connections 2021-04-03T03:15:12.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2125354882 wait complete. 2021-04-03T03:15:12.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 Processor -- start 2021-04-03T03:15:12.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- start start 2021-04-03T03:15:12.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 0x7fa9cc147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b44a0 0x7fa9cc1405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9cc0b5620 0x7fa9cc140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc112cf0 con 0x7fa9cc0b5620 2021-04-03T03:15:12.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc112f90 con 0x7fa9cc0603f0 2021-04-03T03:15:12.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc113230 con 0x7fa9cc0b44a0 2021-04-03T03:15:12.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9cc0b5620 0x7fa9cc140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9cc0b5620 0x7fa9cc140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54104/0 (socket says 172.21.2.111:54104) 2021-04-03T03:15:12.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 -- 172.21.2.111:0/2369017139 learned_addr learned my addr 172.21.2.111:0/2369017139 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:12.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 -- 172.21.2.111:0/2369017139 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b44a0 msgr2=0x7fa9cc1405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b44a0 0x7fa9cc1405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 -- 172.21.2.111:0/2369017139 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 msgr2=0x7fa9cc147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 0x7fa9cc147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 -- 172.21.2.111:0/2369017139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc1134d0 con 0x7fa9cc0b5620 2021-04-03T03:15:12.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d19d7700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9cc0b5620 0x7fa9cc140aa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fa9c4006de0 tx=0x7fa9c40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9be7fc700 1 -- 172.21.2.111:0/2369017139 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c400ca00 con 0x7fa9cc0b5620 2021-04-03T03:15:12.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9be7fc700 1 -- 172.21.2.111:0/2369017139 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400cb60 con 0x7fa9cc0b5620 2021-04-03T03:15:12.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc113970 con 0x7fa9cc0b5620 2021-04-03T03:15:12.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc0ad170 con 0x7fa9cc0b5620 2021-04-03T03:15:12.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9be7fc700 1 -- 172.21.2.111:0/2369017139 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c40123f0 con 0x7fa9cc0b5620 2021-04-03T03:15:12.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc0adf60 con 0x7fa9cc0b5620 2021-04-03T03:15:12.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9be7fc700 1 -- 172.21.2.111:0/2369017139 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa9c4022070 con 0x7fa9cc0b5620 2021-04-03T03:15:12.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.110+0000 7fa9be7fc700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9ac056ef0 0x7fa9ac0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.114+0000 7fa9be7fc700 1 -- 172.21.2.111:0/2369017139 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa9c4070930 con 0x7fa9cc0b5620 2021-04-03T03:15:12.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.114+0000 7fa9d11d6700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9ac056ef0 0x7fa9ac0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.114+0000 7fa9d11d6700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9ac056ef0 0x7fa9ac0593b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fa9cc0b2320 tx=0x7fa9c8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.114+0000 7fa9be7fc700 1 -- 172.21.2.111:0/2369017139 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa9c4045c90 con 0x7fa9cc0b5620 2021-04-03T03:15:12.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9cc113660 con 0x7fa9cc0b5620 2021-04-03T03:15:12.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9be7fc700 1 -- 172.21.2.111:0/2369017139 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa9c40160a0 con 0x7fa9cc0b5620 2021-04-03T03:15:12.213 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:12.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9ac056ef0 msgr2=0x7fa9ac0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9ac056ef0 0x7fa9ac0593b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fa9cc0b2320 tx=0x7fa9c8015000).stop 2021-04-03T03:15:12.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9cc0b5620 msgr2=0x7fa9cc140aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9cc0b5620 0x7fa9cc140aa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fa9c4006de0 tx=0x7fa9c40091d0).stop 2021-04-03T03:15:12.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 shutdown_connections 2021-04-03T03:15:12.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9cc0b5620 0x7fa9cc140aa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9ac056ef0 0x7fa9ac0593b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9cc0603f0 0x7fa9cc147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 --2- 172.21.2.111:0/2369017139 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9cc0b44a0 0x7fa9cc1405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 >> 172.21.2.111:0/2369017139 conn(0x7fa9cc05c7d0 msgr2=0x7fa9cc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:12.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 shutdown_connections 2021-04-03T03:15:12.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.206+0000 7fa9d21d8700 1 -- 172.21.2.111:0/2369017139 wait complete. 2021-04-03T03:15:12.220 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:15:12.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.270+0000 7f4e7ef3f700 1 Processor -- start 2021-04-03T03:15:12.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.270+0000 7f4e7ef3f700 1 -- start start 2021-04-03T03:15:12.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4e700d0bd0 0x7f4e700d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e700405b0 con 0x7f4e70006230 2021-04-03T03:15:12.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e70006610 con 0x7f4e70040150 2021-04-03T03:15:12.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7003db40 con 0x7f4e700d0bd0 2021-04-03T03:15:12.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7df3d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4e70006230 0x7f4e700cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54900/0 (socket says 172.21.2.111:54900) 2021-04-03T03:15:12.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7df3d700 1 -- 172.21.2.111:0/1352582011 learned_addr learned my addr 172.21.2.111:0/1352582011 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:12.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192564164 0 0) 0x7f4e700405b0 con 0x7f4e70006230 2021-04-03T03:15:12.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e60003240 con 0x7f4e70006230 2021-04-03T03:15:12.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1352582011 >> v2:172.21.2.115:3300/0 conn(0x7f4e700d0bd0 0x7f4e700d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2940508159 0 0) 0x7f4e60003240 con 0x7f4e70006230 2021-04-03T03:15:12.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 >> v2:172.21.2.115:3300/0 conn(0x7f4e700d0bd0 msgr2=0x7f4e700d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:12.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 --2- 172.21.2.111:0/1352582011 >> v2:172.21.2.115:3300/0 conn(0x7f4e700d0bd0 0x7f4e700d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 >> v1:172.21.2.115:6789/0 conn(0x7f4e70040150 legacy=0x7f4e700d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:12.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7003dde0 con 0x7f4e70006230 2021-04-03T03:15:12.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f4e740016d0 con 0x7f4e70006230 2021-04-03T03:15:12.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 >> v1:172.21.2.111:6789/0 conn(0x7f4e70006230 legacy=0x7f4e700cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e60003730 0x7f4e60003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e600044e0 0x7f4e60004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e60004e50 0x7f4e6000c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7003db40 con 0x7f4e60004e50 2021-04-03T03:15:12.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e60004210 con 0x7f4e600044e0 2021-04-03T03:15:12.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6000cd30 con 0x7f4e60003730 2021-04-03T03:15:12.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e60004e50 0x7f4e6000c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7d73c700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e600044e0 0x7f4e60004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1352582011 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e60003730 msgr2=0x7f4e60003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:12.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e60003730 0x7f4e60003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1352582011 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e600044e0 msgr2=0x7f4e60004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:12.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e600044e0 0x7f4e60004960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1352582011 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7003dde0 con 0x7f4e60004e50 2021-04-03T03:15:12.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1352582011 reap_dead start 2021-04-03T03:15:12.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e60004e50 0x7f4e6000c840 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f4e78066000 tx=0x7f4e7805ad70).ready entity=mon.0 client_cookie=a2393183a8e14e1a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7805df60 con 0x7f4e60004e50 2021-04-03T03:15:12.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e780699f0 con 0x7f4e60004e50 2021-04-03T03:15:12.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7cf3b700 1 -- 172.21.2.111:0/1352582011 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7805b040 con 0x7f4e60004e50 2021-04-03T03:15:12.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1352582011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e60004e50 msgr2=0x7f4e6000c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e60004e50 0x7f4e6000c840 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f4e78066000 tx=0x7f4e7805ad70).stop 2021-04-03T03:15:12.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1352582011 shutdown_connections 2021-04-03T03:15:12.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1352582011 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e60004e50 0x7f4e6000c840 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1352582011 >> 172.21.2.111:0/1352582011 conn(0x7f4e700462b0 msgr2=0x7f4e700466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:12.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1352582011 shutdown_connections 2021-04-03T03:15:12.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1352582011 wait complete. 2021-04-03T03:15:12.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 Processor -- start 2021-04-03T03:15:12.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- start start 2021-04-03T03:15:12.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e60004e50 0x7f4e700e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e7003bd50 0x7f4e700a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e700d5f10 0x7f4e700cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700d6390 con 0x7f4e700d5f10 2021-04-03T03:15:12.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7009fa10 con 0x7f4e60004e50 2021-04-03T03:15:12.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7ef3f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700a0020 con 0x7f4e7003bd50 2021-04-03T03:15:12.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e700d5f10 0x7f4e700cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e700d5f10 0x7f4e700cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54124/0 (socket says 172.21.2.111:54124) 2021-04-03T03:15:12.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.274+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1467223200 learned_addr learned my addr 172.21.2.111:0/1467223200 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:12.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1467223200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e7003bd50 msgr2=0x7f4e700a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e7003bd50 0x7f4e700a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1467223200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e60004e50 msgr2=0x7f4e700e27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e60004e50 0x7f4e700e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7e73e700 1 -- 172.21.2.111:0/1467223200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e700a02c0 con 0x7f4e700d5f10 2021-04-03T03:15:12.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7e73e700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e700d5f10 0x7f4e700cfcd0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f4e78066000 tx=0x7f4e78082000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e6affd700 1 -- 172.21.2.111:0/1467223200 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7805f4d0 con 0x7f4e700d5f10 2021-04-03T03:15:12.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e6affd700 1 -- 172.21.2.111:0/1467223200 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7805f630 con 0x7f4e700d5f10 2021-04-03T03:15:12.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e700d01c0 con 0x7f4e700d5f10 2021-04-03T03:15:12.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e6affd700 1 -- 172.21.2.111:0/1467223200 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7805f7a0 con 0x7f4e700d5f10 2021-04-03T03:15:12.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e700d0680 con 0x7f4e700d5f10 2021-04-03T03:15:12.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e700a0450 con 0x7f4e700d5f10 2021-04-03T03:15:12.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e6affd700 1 -- 172.21.2.111:0/1467223200 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4e78096030 con 0x7f4e700d5f10 2021-04-03T03:15:12.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e6affd700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e58057050 0x7f4e58059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e6affd700 1 -- 172.21.2.111:0/1467223200 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4e780e03f0 con 0x7f4e700d5f10 2021-04-03T03:15:12.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7df3d700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e58057050 0x7f4e58059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.278+0000 7f4e7df3d700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e58057050 0x7f4e58059510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f4e740013e0 tx=0x7f4e74001300).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.282+0000 7f4e6affd700 1 -- 172.21.2.111:0/1467223200 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4e780b57b0 con 0x7f4e700d5f10 2021-04-03T03:15:12.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e70004db0 con 0x7f4e700d5f10 2021-04-03T03:15:12.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e6affd700 1 -- 172.21.2.111:0/1467223200 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4e78094070 con 0x7f4e700d5f10 2021-04-03T03:15:12.379 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:12.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e58057050 msgr2=0x7f4e58059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e58057050 0x7f4e58059510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f4e740013e0 tx=0x7f4e74001300).stop 2021-04-03T03:15:12.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e700d5f10 msgr2=0x7f4e700cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e700d5f10 0x7f4e700cfcd0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f4e78066000 tx=0x7f4e78082000).stop 2021-04-03T03:15:12.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 shutdown_connections 2021-04-03T03:15:12.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e700d5f10 0x7f4e700cfcd0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e58057050 0x7f4e58059510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e60004e50 0x7f4e700e27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 --2- 172.21.2.111:0/1467223200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e7003bd50 0x7f4e700a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 >> 172.21.2.111:0/1467223200 conn(0x7f4e700462b0 msgr2=0x7f4e7003c770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:12.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 shutdown_connections 2021-04-03T03:15:12.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.374+0000 7f4e7ef3f700 1 -- 172.21.2.111:0/1467223200 wait complete. 2021-04-03T03:15:12.386 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:15:12.387 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:12.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 Processor -- start 2021-04-03T03:15:12.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- start start 2021-04-03T03:15:12.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f606c0b12b0 0x7f606c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c0607d0 con 0x7f606c0603f0 2021-04-03T03:15:12.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c0b0eb0 con 0x7f606c14b700 2021-04-03T03:15:12.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f606c0af400 con 0x7f606c0b12b0 2021-04-03T03:15:12.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606b7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f606c0603f0 0x7f606c0b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54920/0 (socket says 172.21.2.111:54920) 2021-04-03T03:15:12.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606b7fe700 1 -- 172.21.2.111:0/3328395307 learned_addr learned my addr 172.21.2.111:0/3328395307 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:12.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606bfff700 1 --2- 172.21.2.111:0/3328395307 >> v2:172.21.2.111:3300/0 conn(0x7f606c0b12b0 0x7f606c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606bfff700 1 -- 172.21.2.111:0/3328395307 >> v1:172.21.2.115:6789/0 conn(0x7f606c14b700 legacy=0x7f606c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:12.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606bfff700 1 -- 172.21.2.111:0/3328395307 >> v1:172.21.2.111:6789/0 conn(0x7f606c0603f0 legacy=0x7f606c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606bfff700 1 -- 172.21.2.111:0/3328395307 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0af6a0 con 0x7f606c0b12b0 2021-04-03T03:15:12.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606bfff700 1 --2- 172.21.2.111:0/3328395307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c1522d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f606c1486c0 tx=0x7f60640091d0).ready entity=mon.0 client_cookie=386e94f8e6364c0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606a7fc700 1 -- 172.21.2.111:0/3328395307 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f606400bb40 con 0x7f606c0b12b0 2021-04-03T03:15:12.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606a7fc700 1 -- 172.21.2.111:0/3328395307 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400bca0 con 0x7f606c0b12b0 2021-04-03T03:15:12.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f606a7fc700 1 -- 172.21.2.111:0/3328395307 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f606400be10 con 0x7f606c0b12b0 2021-04-03T03:15:12.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- 172.21.2.111:0/3328395307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 msgr2=0x7f606c1522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 --2- 172.21.2.111:0/3328395307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c1522d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f606c1486c0 tx=0x7f60640091d0).stop 2021-04-03T03:15:12.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- 172.21.2.111:0/3328395307 shutdown_connections 2021-04-03T03:15:12.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 --2- 172.21.2.111:0/3328395307 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c1522d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- 172.21.2.111:0/3328395307 >> 172.21.2.111:0/3328395307 conn(0x7f606c0b61c0 msgr2=0x7f606c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:12.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- 172.21.2.111:0/3328395307 shutdown_connections 2021-04-03T03:15:12.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.438+0000 7f6070d7d700 1 -- 172.21.2.111:0/3328395307 wait complete. 2021-04-03T03:15:12.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 Processor -- start 2021-04-03T03:15:12.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 -- start start 2021-04-03T03:15:12.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f606c0603f0 0x7f606c0aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c064ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f606c14b700 0x7f606c0653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c146790 con 0x7f606c0b12b0 2021-04-03T03:15:12.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0ab730 con 0x7f606c0603f0 2021-04-03T03:15:12.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0ab490 con 0x7f606c14b700 2021-04-03T03:15:12.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c064ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c064ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54134/0 (socket says 172.21.2.111:54134) 2021-04-03T03:15:12.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 -- 172.21.2.111:0/2300718572 learned_addr learned my addr 172.21.2.111:0/2300718572 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:12.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 -- 172.21.2.111:0/2300718572 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f606c14b700 msgr2=0x7f606c0653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f606c14b700 0x7f606c0653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 -- 172.21.2.111:0/2300718572 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f606c0603f0 msgr2=0x7f606c0aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f606c0603f0 0x7f606c0aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 -- 172.21.2.111:0/2300718572 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c064a30 con 0x7f606c0b12b0 2021-04-03T03:15:12.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606affd700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c064ed0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f6054002eb0 tx=0x7f6054006d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6068ff9700 1 -- 172.21.2.111:0/2300718572 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605400d4a0 con 0x7f606c0b12b0 2021-04-03T03:15:12.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6068ff9700 1 -- 172.21.2.111:0/2300718572 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6054000e90 con 0x7f606c0b12b0 2021-04-03T03:15:12.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606c0633c0 con 0x7f606c0b12b0 2021-04-03T03:15:12.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6068ff9700 1 -- 172.21.2.111:0/2300718572 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605400d7f0 con 0x7f606c0b12b0 2021-04-03T03:15:12.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c0abc70 con 0x7f606c0b12b0 2021-04-03T03:15:12.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606c064bc0 con 0x7f606c0b12b0 2021-04-03T03:15:12.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6068ff9700 1 -- 172.21.2.111:0/2300718572 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f605400f030 con 0x7f606c0b12b0 2021-04-03T03:15:12.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6068ff9700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6050056ec0 0x7f6050059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:12.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f6068ff9700 1 -- 172.21.2.111:0/2300718572 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f605406fd60 con 0x7f606c0b12b0 2021-04-03T03:15:12.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606b7fe700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6050056ec0 0x7f6050059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:12.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.442+0000 7f606b7fe700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6050056ec0 0x7f6050059380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f606c0ae400 tx=0x7f605c0077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:12.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.446+0000 7f6068ff9700 1 -- 172.21.2.111:0/2300718572 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6054045120 con 0x7f606c0b12b0 2021-04-03T03:15:12.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.566+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f606c057660 con 0x7f606c0b12b0 2021-04-03T03:15:12.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.566+0000 7f6068ff9700 1 -- 172.21.2.111:0/2300718572 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f6054048740 con 0x7f606c0b12b0 2021-04-03T03:15:12.573 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:12.574 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:15:12.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6050056ec0 msgr2=0x7f6050059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6050056ec0 0x7f6050059380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f606c0ae400 tx=0x7f605c0077e0).stop 2021-04-03T03:15:12.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 msgr2=0x7f606c064ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:12.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c064ed0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f6054002eb0 tx=0x7f6054006d10).stop 2021-04-03T03:15:12.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 shutdown_connections 2021-04-03T03:15:12.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f606c14b700 0x7f606c0653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6050056ec0 0x7f6050059380 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f606c0603f0 0x7f606c0aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:12.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 --2- 172.21.2.111:0/2300718572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f606c0b12b0 0x7f606c064ed0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:12.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 >> 172.21.2.111:0/2300718572 conn(0x7f606c0b61c0 msgr2=0x7f606c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:12.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 shutdown_connections 2021-04-03T03:15:12.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:12.570+0000 7f6070d7d700 1 -- 172.21.2.111:0/2300718572 wait complete. 2021-04-03T03:15:12.578 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:15:12.782 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:14:24.079081+0000', 'modified': '2021-04-03T03:14:25.081940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6210': {'gid': 6210, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/560115150', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 560115150}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 560115150}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:15:12.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:15:12.783 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:15:12.861 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6210,"want_state":"up:active","state":"up:active","rank_uptime":48.770695932000002,"mdsmap_epoch":29,"osdmap_epoch":75,"osdmap_epoch_barrier":75,"uptime":52.914031940999998} 2021-04-03T03:15:12.861 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6210, "mdsmap_epoch": 29, "osdmap_epoch": 75, "osdmap_epoch_barrier": 75, "rank_uptime": 48.770695932, "state": "up:active", "uptime": 52.914031941, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:15:12.862 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:15:12.863 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:15:12.863 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:15:12.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:12.864 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:12.875 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:15:12.927 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:12.928 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:12.976 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:12.976 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:12.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:12.978 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:15:13.029 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:15:13.030 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:13.075 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:15:13.083 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:13.083 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:13.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:15:13.110+0000 7f4fb03140c0 -1 init, newargv = 0x55a33aa96d70 newargc=9 2021-04-03T03:15:13.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15543]: starting ceph client 2021-04-03T03:15:13.121 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:13.123 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:13.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.124 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:15:13.128 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15543]: starting fuse 2021-04-03T03:15:13.132 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:15:13.132 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:15:13.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.134 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:15:13.134 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:15:13.135 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:15:13.136 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:15:13.136 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:15:13.137 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:13.138 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:15:13.139 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:15:13.139 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:15:13.140 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:13.141 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:15:13.142 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:15:13.142 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:15:13.143 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:15:13.144 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:13.144 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:15:13.145 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:15:13.146 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:15:13.147 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:15:13.147 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:15:13.148 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:13.149 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:15:13.149 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:15:13.198 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15543.asok 2021-04-03T03:15:13.200 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15543.asok 2021-04-03T03:15:13.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.201 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15543.asok status 2021-04-03T03:15:13.314 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:15:13.315 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:15:13.315 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:15:13.315 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:15:13.315 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:15:13.316 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:15:13.316 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:15:13.316 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15543", 2021-04-03T03:15:13.316 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:15:13.316 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:13.317 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:15:13.317 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:15:13.317 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4718, 2021-04-03T03:15:13.317 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:15:13.317 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:15:13.318 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:15:13.318 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4718 2021-04-03T03:15:13.318 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:13.318 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:15:13.319 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:15:13.319 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:15:13.319 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 1288667308 2021-04-03T03:15:13.319 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:15:13.319 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:13.320 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:15:13.320 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:15:13.320 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:15:13.320 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 1288667308 2021-04-03T03:15:13.320 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:13.321 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4718 172.21.2.126:0/1288667308", 2021-04-03T03:15:13.321 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/1288667308", 2021-04-03T03:15:13.321 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:15:13.321 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 29, 2021-04-03T03:15:13.321 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 75, 2021-04-03T03:15:13.322 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:15:13.322 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:15:13.322 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:15:13.322 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:15:13.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.324 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:13.324 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:13.329 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:15:13.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:13.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.331 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:13.384 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:15:13.387 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:13.399 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:13.411 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:13.435 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:13.447 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:13.460 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:15:13.460 INFO:tasks.cephfs.mount:Terminating background process 2021-04-03T03:15:13.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.461 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:13.461 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:13.483 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:15:13.483 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:13.483 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:15:13.484 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:15:13.484 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:13.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15543]: fuse finished with error 0 and tester_r 0 2021-04-03T03:15:13.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.578 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:13.578 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:13.588 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:15:13.589 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:15:13.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:13.710 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:13.762 INFO:tasks.cephfs.mount:Terminating background process 2021-04-03T03:15:13.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.763 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:13.763 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:13.774 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:15:13.775 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:13.776 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:15:13.776 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:15:13.777 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:13.862 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15057]: fuse finished with error 0 and tester_r 0 2021-04-03T03:15:13.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:13.865 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:13.865 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:13.874 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:15:13.875 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:15:13.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:13.989 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:14.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:14.034 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:14.034 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:14.043 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:14.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:14.045 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:15:14.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:14.046 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:14.094 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:14.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:14.096 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:15:14.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206d01700 1 Processor -- start 2021-04-03T03:15:14.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206d01700 1 -- start start 2021-04-03T03:15:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206d01700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f42000603f0 0x7f42001447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206d01700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42000607d0 con 0x7f42000b17b0 2021-04-03T03:15:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206d01700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420014b090 con 0x7f42000add50 2021-04-03T03:15:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206d01700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420014b330 con 0x7f42000603f0 2021-04-03T03:15:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206500700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f42000b17b0 0x7f4200147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:54938/0 (socket says 172.21.2.111:54938) 2021-04-03T03:15:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4206500700 1 -- 172.21.2.111:0/3489359972 learned_addr learned my addr 172.21.2.111:0/3489359972 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324761207 0 0) 0x7f42000607d0 con 0x7f42000b17b0 2021-04-03T03:15:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e8003240 con 0x7f42000b17b0 2021-04-03T03:15:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4205cff700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f42000603f0 0x7f42001447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1928440642 0 0) 0x7f41e8003240 con 0x7f42000b17b0 2021-04-03T03:15:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f42000603f0 msgr2=0x7f42001447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f42000603f0 0x7f42001447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 >> v1:172.21.2.115:6789/0 conn(0x7f42000add50 legacy=0x7f4200147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420014b5d0 con 0x7f42000b17b0 2021-04-03T03:15:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f41f8001a50 con 0x7f42000b17b0 2021-04-03T03:15:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 >> v1:172.21.2.111:6789/0 conn(0x7f42000b17b0 legacy=0x7f4200147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f41e8003760 0x7f41e8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41e80044e0 0x7f41e8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f42000603f0 0x7f41e8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420014b330 con 0x7f41e80044e0 2021-04-03T03:15:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8005390 con 0x7f41e8003760 2021-04-03T03:15:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8004320 con 0x7f42000603f0 2021-04-03T03:15:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.154+0000 7f42054fe700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41e80044e0 0x7f41e8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f42054fe700 1 -- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f42000603f0 msgr2=0x7f41e8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f42054fe700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f42000603f0 0x7f41e8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f41e8003760 0x7f41e8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f42054fe700 1 -- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f41e8003760 msgr2=0x7f41e8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f42054fe700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f41e8003760 0x7f41e8003b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f42054fe700 1 -- 172.21.2.111:0/3489359972 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e8003240 con 0x7f41e80044e0 2021-04-03T03:15:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f42054fe700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41e80044e0 0x7f41e8004960 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f42000b5640 tx=0x7f41fc007b70).ready entity=mon.0 client_cookie=57a572a27f7daaf9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41fc00c660 con 0x7f41e80044e0 2021-04-03T03:15:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc00c7c0 con 0x7f41e80044e0 2021-04-03T03:15:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4204cfd700 1 -- 172.21.2.111:0/3489359972 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41fc00cb20 con 0x7f41e80044e0 2021-04-03T03:15:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/3489359972 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41e80044e0 msgr2=0x7f41e8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 --2- 172.21.2.111:0/3489359972 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f41e80044e0 0x7f41e8004960 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f42000b5640 tx=0x7f41fc007b70).stop 2021-04-03T03:15:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206500700 1 -- 172.21.2.111:0/3489359972 reap_dead start 2021-04-03T03:15:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/3489359972 shutdown_connections 2021-04-03T03:15:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/3489359972 >> 172.21.2.111:0/3489359972 conn(0x7f42000b61c0 msgr2=0x7f420005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/3489359972 shutdown_connections 2021-04-03T03:15:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/3489359972 wait complete. 2021-04-03T03:15:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 Processor -- start 2021-04-03T03:15:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- start start 2021-04-03T03:15:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f42000a94e0 0x7f4200066240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4200063c00 0x7f4200064640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4200064b30 0x7f42000aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4200062be0 con 0x7f42000a94e0 2021-04-03T03:15:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42000a98c0 con 0x7f4200064b30 2021-04-03T03:15:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4200067de0 con 0x7f4200063c00 2021-04-03T03:15:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f42000a94e0 0x7f4200066240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f42000a94e0 0x7f4200066240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54156/0 (socket says 172.21.2.111:54156) 2021-04-03T03:15:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 -- 172.21.2.111:0/1354409683 learned_addr learned my addr 172.21.2.111:0/1354409683 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 -- 172.21.2.111:0/1354409683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4200063c00 msgr2=0x7f4200064640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4200063c00 0x7f4200064640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 -- 172.21.2.111:0/1354409683 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4200064b30 msgr2=0x7f42000aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4200064b30 0x7f42000aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 -- 172.21.2.111:0/1354409683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4200062e10 con 0x7f42000a94e0 2021-04-03T03:15:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4205cff700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f42000a94e0 0x7f4200066240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f41f4008000 tx=0x7f41f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f41f2ffd700 1 -- 172.21.2.111:0/1354409683 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41f40133f0 con 0x7f42000a94e0 2021-04-03T03:15:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f41f2ffd700 1 -- 172.21.2.111:0/1354409683 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f4013550 con 0x7f42000a94e0 2021-04-03T03:15:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f41f2ffd700 1 -- 172.21.2.111:0/1354409683 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41f40138b0 con 0x7f42000a94e0 2021-04-03T03:15:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42000abcb0 con 0x7f42000a94e0 2021-04-03T03:15:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42000630f0 con 0x7f42000a94e0 2021-04-03T03:15:14.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4200057660 con 0x7f42000a94e0 2021-04-03T03:15:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f41f2ffd700 1 -- 172.21.2.111:0/1354409683 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f41f4019020 con 0x7f42000a94e0 2021-04-03T03:15:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f41f2ffd700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f41e0056fe0 0x7f41e00594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f41f2ffd700 1 -- 172.21.2.111:0/1354409683 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f41f406f600 con 0x7f42000a94e0 2021-04-03T03:15:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.158+0000 7f42054fe700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f41e0056fe0 0x7f41e00594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.162+0000 7f42054fe700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f41e0056fe0 0x7f41e00594a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f42000b5640 tx=0x7f41fc016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:14.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.162+0000 7f41f2ffd700 1 -- 172.21.2.111:0/1354409683 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f41f4043c50 con 0x7f42000a94e0 2021-04-03T03:15:14.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.254+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4200148850 con 0x7f42000a94e0 2021-04-03T03:15:14.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.258+0000 7f41f2ffd700 1 -- 172.21.2.111:0/1354409683 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f41f4017020 con 0x7f42000a94e0 2021-04-03T03:15:14.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f41e0056fe0 msgr2=0x7f41e00594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f41e0056fe0 0x7f41e00594a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f42000b5640 tx=0x7f41fc016000).stop 2021-04-03T03:15:14.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f42000a94e0 msgr2=0x7f4200066240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f42000a94e0 0x7f4200066240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f41f4008000 tx=0x7f41f4008040).stop 2021-04-03T03:15:14.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 shutdown_connections 2021-04-03T03:15:14.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f41e0056fe0 0x7f41e00594a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:14.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4200064b30 0x7f42000aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f42000a94e0 0x7f4200066240 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:14.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 --2- 172.21.2.111:0/1354409683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4200063c00 0x7f4200064640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 >> 172.21.2.111:0/1354409683 conn(0x7f42000b61c0 msgr2=0x7f420014a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:14.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 shutdown_connections 2021-04-03T03:15:14.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.262+0000 7f4206d01700 1 -- 172.21.2.111:0/1354409683 wait complete. 2021-04-03T03:15:14.273 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2021-04-03T03:15:14.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.322+0000 7fdc914cb700 1 Processor -- start 2021-04-03T03:15:14.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.322+0000 7fdc914cb700 1 -- start start 2021-04-03T03:15:14.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0603f0 0x7fdc8c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdc8c14b700 0x7fdc8c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdc8c0b12b0 0x7fdc8c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b0f60 con 0x7fdc8c0b12b0 2021-04-03T03:15:14.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8c0607d0 con 0x7fdc8c14b700 2021-04-03T03:15:14.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0af400 con 0x7fdc8c0603f0 2021-04-03T03:15:14.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdc8c0b12b0 0x7fdc8c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdc8c0b12b0 0x7fdc8c0b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54166/0 (socket says 172.21.2.111:54166) 2021-04-03T03:15:14.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 -- 172.21.2.111:0/2302722854 learned_addr learned my addr 172.21.2.111:0/2302722854 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:14.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 -- 172.21.2.111:0/2302722854 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0603f0 msgr2=0x7fdc8c0b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:14.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 --2- 172.21.2.111:0/2302722854 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0603f0 0x7fdc8c0b0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 -- 172.21.2.111:0/2302722854 >> v2:172.21.2.115:3300/0 conn(0x7fdc8c14b700 msgr2=0x7fdc8c0b0580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:14.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 --2- 172.21.2.111:0/2302722854 >> v2:172.21.2.115:3300/0 conn(0x7fdc8c14b700 0x7fdc8c0b0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 -- 172.21.2.111:0/2302722854 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0af6a0 con 0x7fdc8c0b12b0 2021-04-03T03:15:14.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc90cca700 1 --2- 172.21.2.111:0/2302722854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0b12b0 0x7fdc8c0b0a70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fdc8c062740 tx=0x7fdc84009270).ready entity=mon.0 client_cookie=6526ffd2323018b4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:14.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8affd700 1 -- 172.21.2.111:0/2302722854 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc8400bbc0 con 0x7fdc8c0b12b0 2021-04-03T03:15:14.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8affd700 1 -- 172.21.2.111:0/2302722854 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8400bd20 con 0x7fdc8c0b12b0 2021-04-03T03:15:14.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8affd700 1 -- 172.21.2.111:0/2302722854 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc84011400 con 0x7fdc8c0b12b0 2021-04-03T03:15:14.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- 172.21.2.111:0/2302722854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0b12b0 msgr2=0x7fdc8c0b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- 172.21.2.111:0/2302722854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0b12b0 0x7fdc8c0b0a70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fdc8c062740 tx=0x7fdc84009270).stop 2021-04-03T03:15:14.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- 172.21.2.111:0/2302722854 shutdown_connections 2021-04-03T03:15:14.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- 172.21.2.111:0/2302722854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0b12b0 0x7fdc8c0b0a70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:14.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- 172.21.2.111:0/2302722854 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0603f0 0x7fdc8c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- 172.21.2.111:0/2302722854 >> v2:172.21.2.115:3300/0 conn(0x7fdc8c14b700 0x7fdc8c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- 172.21.2.111:0/2302722854 >> 172.21.2.111:0/2302722854 conn(0x7fdc8c0b61c0 msgr2=0x7fdc8c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:14.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- 172.21.2.111:0/2302722854 shutdown_connections 2021-04-03T03:15:14.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- 172.21.2.111:0/2302722854 wait complete. 2021-04-03T03:15:14.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 Processor -- start 2021-04-03T03:15:14.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- start start 2021-04-03T03:15:14.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0603f0 0x7fdc8c0aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0b12b0 0x7fdc8c0a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc8c14b700 0x7fdc8c146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0ab6d0 con 0x7fdc8c0603f0 2021-04-03T03:15:14.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0ab970 con 0x7fdc8c14b700 2021-04-03T03:15:14.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0abc10 con 0x7fdc8c0b12b0 2021-04-03T03:15:14.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0603f0 0x7fdc8c0aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0603f0 0x7fdc8c0aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54168/0 (socket says 172.21.2.111:54168) 2021-04-03T03:15:14.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 -- 172.21.2.111:0/3625933005 learned_addr learned my addr 172.21.2.111:0/3625933005 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:14.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 -- 172.21.2.111:0/3625933005 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0b12b0 msgr2=0x7fdc8c0a81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0b12b0 0x7fdc8c0a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 -- 172.21.2.111:0/3625933005 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc8c14b700 msgr2=0x7fdc8c146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc8c14b700 0x7fdc8c146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 -- 172.21.2.111:0/3625933005 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0aa160 con 0x7fdc8c0603f0 2021-04-03T03:15:14.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc8bfff700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0603f0 0x7fdc8c0aa4f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fdc7c002f70 tx=0x7fdc7c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:14.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc897fa700 1 -- 172.21.2.111:0/3625933005 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc7c00d570 con 0x7fdc8c0603f0 2021-04-03T03:15:14.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc897fa700 1 -- 172.21.2.111:0/3625933005 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c00d6d0 con 0x7fdc8c0603f0 2021-04-03T03:15:14.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c1563f0 con 0x7fdc8c0603f0 2021-04-03T03:15:14.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc897fa700 1 -- 172.21.2.111:0/3625933005 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc7c00da30 con 0x7fdc8c0603f0 2021-04-03T03:15:14.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.326+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c146e20 con 0x7fdc8c0603f0 2021-04-03T03:15:14.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.330+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c0aa350 con 0x7fdc8c0603f0 2021-04-03T03:15:14.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.330+0000 7fdc897fa700 1 -- 172.21.2.111:0/3625933005 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdc7c00f070 con 0x7fdc8c0603f0 2021-04-03T03:15:14.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.330+0000 7fdc897fa700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc70056ea0 0x7fdc70059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:14.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.330+0000 7fdc897fa700 1 -- 172.21.2.111:0/3625933005 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fdc7c06fd10 con 0x7fdc8c0603f0 2021-04-03T03:15:14.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.330+0000 7fdc8b7fe700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc70056ea0 0x7fdc70059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:14.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.330+0000 7fdc8b7fe700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc70056ea0 0x7fdc70059360 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fdc74000e90 tx=0x7fdc74006930).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:14.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.330+0000 7fdc897fa700 1 -- 172.21.2.111:0/3625933005 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdc8c0aa350 con 0x7fdc8c0603f0 2021-04-03T03:15:14.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.458+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7fdc8c0aa350 con 0x7fdc8c0603f0 2021-04-03T03:15:14.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.970+0000 7fdc897fa700 1 -- 172.21.2.111:0/3625933005 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v270) v1 ==== 139+0+0 (secure 0 0 0) 0x7fdc8c0aa350 con 0x7fdc8c0603f0 2021-04-03T03:15:14.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc70056ea0 msgr2=0x7fdc70059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc70056ea0 0x7fdc70059360 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fdc74000e90 tx=0x7fdc74006930).stop 2021-04-03T03:15:14.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0603f0 msgr2=0x7fdc8c0aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:14.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0603f0 0x7fdc8c0aa4f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fdc7c002f70 tx=0x7fdc7c006dd0).stop 2021-04-03T03:15:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 shutdown_connections 2021-04-03T03:15:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc70056ea0 0x7fdc70059360 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc8c14b700 0x7fdc8c146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc8c0603f0 0x7fdc8c0aa4f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:14.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 --2- 172.21.2.111:0/3625933005 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc8c0b12b0 0x7fdc8c0a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:14.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 >> 172.21.2.111:0/3625933005 conn(0x7fdc8c0b61c0 msgr2=0x7fdc8c0b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:14.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 shutdown_connections 2021-04-03T03:15:14.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:14.974+0000 7fdc914cb700 1 -- 172.21.2.111:0/3625933005 wait complete. 2021-04-03T03:15:14.985 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2021-04-03T03:15:14.986 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:15:14.986 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2021-04-03T03:15:15.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 Processor -- start 2021-04-03T03:15:15.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- start start 2021-04-03T03:15:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe004ae10 0x7fbfe004b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbfe00d7d90 0x7fbfe00d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00d8300 con 0x7fbfe00d4330 2021-04-03T03:15:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00460a0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0046340 con 0x7fbfe004ae10 2021-04-03T03:15:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbfe00d7d90 0x7fbfe00d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbfe00d7d90 0x7fbfe00d3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:43886/0 (socket says 172.21.2.111:43886) 2021-04-03T03:15:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 -- 172.21.2.111:0/974302614 learned_addr learned my addr 172.21.2.111:0/974302614 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe7fff700 1 --2- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe004ae10 0x7fbfe004b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 -- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe004ae10 msgr2=0x7fbfe004b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 --2- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe004ae10 0x7fbfe004b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 -- 172.21.2.111:0/974302614 >> v1:172.21.2.115:6789/0 conn(0x7fbfe00d4330 legacy=0x7fbfe00467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 -- 172.21.2.111:0/974302614 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0046ef0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe77fe700 1 --2- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d7d90 0x7fbfe00d3e40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fbfd4002f30 tx=0x7fbfd4006e90).ready entity=mon.1 client_cookie=7df08a7bfe674545 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:15.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe6ffd700 1 -- 172.21.2.111:0/974302614 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd4009970 con 0x7fbfe00d7d90 2021-04-03T03:15:15.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe6ffd700 1 -- 172.21.2.111:0/974302614 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4009ad0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe6ffd700 1 -- 172.21.2.111:0/974302614 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd4009e30 con 0x7fbfe00d7d90 2021-04-03T03:15:15.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d7d90 msgr2=0x7fbfe00d3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:15.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d7d90 0x7fbfe00d3e40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fbfd4002f30 tx=0x7fbfd4006e90).stop 2021-04-03T03:15:15.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- 172.21.2.111:0/974302614 shutdown_connections 2021-04-03T03:15:15.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe004ae10 0x7fbfe004b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- 172.21.2.111:0/974302614 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d7d90 0x7fbfe00d3e40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- 172.21.2.111:0/974302614 >> 172.21.2.111:0/974302614 conn(0x7fbfe0011530 msgr2=0x7fbfe004a700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:15.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- 172.21.2.111:0/974302614 shutdown_connections 2021-04-03T03:15:15.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- 172.21.2.111:0/974302614 wait complete. 2021-04-03T03:15:15.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 Processor -- start 2021-04-03T03:15:15.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- start start 2021-04-03T03:15:15.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbfe004ae10 0x7fbfe0042f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:15.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d4330 0x7fbfe0044470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:15.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe00d7d90 0x7fbfe00e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:15.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0042af0 con 0x7fbfe004ae10 2021-04-03T03:15:15.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00420e0 con 0x7fbfe00d4330 2021-04-03T03:15:15.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfed01c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00e6a80 con 0x7fbfe00d7d90 2021-04-03T03:15:15.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.038+0000 7fbfe7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbfe004ae10 0x7fbfe0042f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:15.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbfe004ae10 0x7fbfe0042f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54182/0 (socket says 172.21.2.111:54182) 2021-04-03T03:15:15.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe7fff700 1 -- 172.21.2.111:0/2122698644 learned_addr learned my addr 172.21.2.111:0/2122698644 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:15.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfec81b700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe00d7d90 0x7fbfe00e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:15.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe77fe700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d4330 0x7fbfe0044470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:15.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfec81b700 1 -- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d4330 msgr2=0x7fbfe0044470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:15.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfec81b700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d4330 0x7fbfe0044470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfec81b700 1 -- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbfe004ae10 msgr2=0x7fbfe0042f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:15.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfec81b700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbfe004ae10 0x7fbfe0042f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfec81b700 1 -- 172.21.2.111:0/2122698644 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00e6ce0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe7fff700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbfe004ae10 0x7fbfe0042f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:15:15.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfec81b700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe00d7d90 0x7fbfe00e6590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fbfe00466c0 tx=0x7fbfe805ad70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:15.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe57fa700 1 -- 172.21.2.111:0/2122698644 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfe80699f0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe57fa700 1 -- 172.21.2.111:0/2122698644 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe806b5d0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe00e6fe0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe00cecd0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe57fa700 1 -- 172.21.2.111:0/2122698644 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfe806b740 con 0x7fbfe00d7d90 2021-04-03T03:15:15.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0044190 con 0x7fbfe00d7d90 2021-04-03T03:15:15.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe57fa700 1 -- 172.21.2.111:0/2122698644 <== mon.2 v2:172.21.2.115:3301/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbfe808b020 con 0x7fbfe00d7d90 2021-04-03T03:15:15.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe57fa700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbfd0056fb0 0x7fbfd0059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:15.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe57fa700 1 -- 172.21.2.111:0/2122698644 <== mon.2 v2:172.21.2.115:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbfe80df0e0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe7fff700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbfd0056fb0 0x7fbfd0059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:15.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.042+0000 7fbfe7fff700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbfd0056fb0 0x7fbfd0059470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fbfdc008000 tx=0x7fbfdc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:15.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.050+0000 7fbfe57fa700 1 -- 172.21.2.111:0/2122698644 <== mon.2 v2:172.21.2.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbfe80b44a0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.174+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7fbfe0004da0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfe57fa700 1 -- 172.21.2.111:0/2122698644 <== mon.2 v2:172.21.2.115:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v271) v1 ==== 129+0+0 (secure 0 0 0) 0x7fbfe80b7ac0 con 0x7fbfe00d7d90 2021-04-03T03:15:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbfd0056fb0 msgr2=0x7fbfd0059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbfd0056fb0 0x7fbfd0059470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fbfdc008000 tx=0x7fbfdc008040).stop 2021-04-03T03:15:15.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe00d7d90 msgr2=0x7fbfe00e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe00d7d90 0x7fbfe00e6590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fbfe00466c0 tx=0x7fbfe805ad70).stop 2021-04-03T03:15:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 shutdown_connections 2021-04-03T03:15:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbfe00d7d90 0x7fbfe00e6590 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbfe004ae10 0x7fbfe0042f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbfd0056fb0 0x7fbfd0059470 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 --2- 172.21.2.111:0/2122698644 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbfe00d4330 0x7fbfe0044470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 >> 172.21.2.111:0/2122698644 conn(0x7fbfe0011530 msgr2=0x7fbfe0049c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 shutdown_connections 2021-04-03T03:15:15.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:15.978+0000 7fbfed01c700 1 -- 172.21.2.111:0/2122698644 wait complete. 2021-04-03T03:15:15.993 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:15:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 Processor -- start 2021-04-03T03:15:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 -- start start 2021-04-03T03:15:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9e5802cbe0 0x7f9e580469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 0x7f9e580de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e58044e00 con 0x7f9e580d7a90 2021-04-03T03:15:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5802d8c0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5802cfc0 con 0x7f9e580069c0 2021-04-03T03:15:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9e5802cbe0 0x7f9e580469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9e5802cbe0 0x7f9e580469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54192/0 (socket says 172.21.2.111:54192) 2021-04-03T03:15:16.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 -- 172.21.2.111:0/635565418 learned_addr learned my addr 172.21.2.111:0/635565418 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:16.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 -- 172.21.2.111:0/635565418 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 msgr2=0x7f9e580de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 --2- 172.21.2.111:0/635565418 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 0x7f9e580de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 -- 172.21.2.111:0/635565418 >> v1:172.21.2.115:6789/0 conn(0x7f9e580d7a90 legacy=0x7f9e580d7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 -- 172.21.2.111:0/635565418 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58046e90 con 0x7f9e5802cbe0 2021-04-03T03:15:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6574c700 1 --2- 172.21.2.111:0/635565418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e580469a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f9e5800f010 tx=0x7f9e50006e90).ready entity=mon.0 client_cookie=849f026bb4360927 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e4ffff700 1 -- 172.21.2.111:0/635565418 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e500098e0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e4ffff700 1 -- 172.21.2.111:0/635565418 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50009a40 con 0x7f9e5802cbe0 2021-04-03T03:15:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e4ffff700 1 -- 172.21.2.111:0/635565418 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e50009de0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 -- 172.21.2.111:0/635565418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 msgr2=0x7f9e580469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.042+0000 7f9e6674e700 1 --2- 172.21.2.111:0/635565418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e580469a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f9e5800f010 tx=0x7f9e50006e90).stop 2021-04-03T03:15:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- 172.21.2.111:0/635565418 shutdown_connections 2021-04-03T03:15:16.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 --2- 172.21.2.111:0/635565418 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 0x7f9e580de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 --2- 172.21.2.111:0/635565418 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e580469a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- 172.21.2.111:0/635565418 >> 172.21.2.111:0/635565418 conn(0x7f9e58047160 msgr2=0x7f9e58047570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:16.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- 172.21.2.111:0/635565418 shutdown_connections 2021-04-03T03:15:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- 172.21.2.111:0/635565418 wait complete. 2021-04-03T03:15:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 Processor -- start 2021-04-03T03:15:16.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- start start 2021-04-03T03:15:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 0x7f9e580db240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e5800cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e580d7a90 0x7f9e5800d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580da6a0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5800a750 con 0x7f9e580d7a90 2021-04-03T03:15:16.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5800a430 con 0x7f9e580069c0 2021-04-03T03:15:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e5800cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e5800cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54198/0 (socket says 172.21.2.111:54198) 2021-04-03T03:15:16.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 -- 172.21.2.111:0/3529310072 learned_addr learned my addr 172.21.2.111:0/3529310072 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:16.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 -- 172.21.2.111:0/3529310072 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 msgr2=0x7f9e580db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 0x7f9e580db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 -- 172.21.2.111:0/3529310072 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e580d7a90 msgr2=0x7f9e5800d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e580d7a90 0x7f9e5800d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 -- 172.21.2.111:0/3529310072 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5800d6e0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e64f4b700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e5800cd00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f9e6006b5d0 tx=0x7f9e6005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e4e7fc700 1 -- 172.21.2.111:0/3529310072 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e600699f0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e4e7fc700 1 -- 172.21.2.111:0/3529310072 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6005f2c0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580d9350 con 0x7f9e5802cbe0 2021-04-03T03:15:16.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e4e7fc700 1 -- 172.21.2.111:0/3529310072 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e6005f620 con 0x7f9e5802cbe0 2021-04-03T03:15:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580dac60 con 0x7f9e5802cbe0 2021-04-03T03:15:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5800a920 con 0x7f9e5802cbe0 2021-04-03T03:15:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e4e7fc700 1 -- 172.21.2.111:0/3529310072 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9e6006a2d0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e4e7fc700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e40056ea0 0x7f9e40059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e4e7fc700 1 -- 172.21.2.111:0/3529310072 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9e600df520 con 0x7f9e5802cbe0 2021-04-03T03:15:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.046+0000 7f9e6574c700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e40056ea0 0x7f9e40059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.050+0000 7f9e6574c700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e40056ea0 0x7f9e40059360 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f9e5800f010 tx=0x7f9e50015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.050+0000 7f9e4e7fc700 1 -- 172.21.2.111:0/3529310072 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9e600b4930 con 0x7f9e5802cbe0 2021-04-03T03:15:16.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.142+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9e58004db0 con 0x7f9e5802cbe0 2021-04-03T03:15:16.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.146+0000 7f9e4e7fc700 1 -- 172.21.2.111:0/3529310072 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9e600b7f50 con 0x7f9e5802cbe0 2021-04-03T03:15:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e40056ea0 msgr2=0x7f9e40059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e40056ea0 0x7f9e40059360 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f9e5800f010 tx=0x7f9e50015000).stop 2021-04-03T03:15:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 msgr2=0x7f9e5800cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e5800cd00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f9e6006b5d0 tx=0x7f9e6005ad70).stop 2021-04-03T03:15:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 shutdown_connections 2021-04-03T03:15:16.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9e40056ea0 0x7f9e40059360 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9e580d7a90 0x7f9e5800d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9e580069c0 0x7f9e580db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 --2- 172.21.2.111:0/3529310072 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9e5802cbe0 0x7f9e5800cd00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 >> 172.21.2.111:0/3529310072 conn(0x7f9e58047160 msgr2=0x7f9e580485c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:16.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 shutdown_connections 2021-04-03T03:15:16.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.150+0000 7f9e6674e700 1 -- 172.21.2.111:0/3529310072 wait complete. 2021-04-03T03:15:16.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:15:16.165 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:16.171 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:16.179 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:16.190 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:16.202 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:16.215 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:15:16.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:16.216 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:16.216 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:16.240 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:15:16.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:16.241 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:16.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:16.243 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:16.243 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:16.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:16.250 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:15:16.250 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:16.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:16.251 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:16.252 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:16.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:16.263 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:16.263 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:15:16.264 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:15:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.314+0000 7f8d358b1700 1 Processor -- start 2021-04-03T03:15:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.314+0000 7f8d358b1700 1 -- start start 2021-04-03T03:15:16.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8d3014b700 0x7f8d300b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8d300b0c90 0x7f8d30147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d300af400 con 0x7f8d300603f0 2021-04-03T03:15:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d30060860 con 0x7f8d300b0c90 2021-04-03T03:15:16.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d300af6a0 con 0x7f8d3014b700 2021-04-03T03:15:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8d300b0c90 0x7f8d30147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8d300b0c90 0x7f8d30147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54206/0 (socket says 172.21.2.111:54206) 2021-04-03T03:15:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 -- 172.21.2.111:0/115001452 learned_addr learned my addr 172.21.2.111:0/115001452 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:16.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 -- 172.21.2.111:0/115001452 >> v2:172.21.2.115:3300/0 conn(0x7f8d3014b700 msgr2=0x7f8d300b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 --2- 172.21.2.111:0/115001452 >> v2:172.21.2.115:3300/0 conn(0x7f8d3014b700 0x7f8d300b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 -- 172.21.2.111:0/115001452 >> v1:172.21.2.111:6789/0 conn(0x7f8d300603f0 legacy=0x7f8d300b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d2f7fe700 1 -- 172.21.2.111:0/115001452 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419362916 0 0) 0x7f8d300af400 con 0x7f8d300603f0 2021-04-03T03:15:16.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 -- 172.21.2.111:0/115001452 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300afb40 con 0x7f8d300b0c90 2021-04-03T03:15:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d350b0700 1 --2- 172.21.2.111:0/115001452 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300b0c90 0x7f8d30147680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f8d280032b0 tx=0x7f8d28009270).ready entity=mon.0 client_cookie=ac8f92c5d99e7d6e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d2f7fe700 1 -- 172.21.2.111:0/115001452 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d2800bc00 con 0x7f8d300b0c90 2021-04-03T03:15:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d2f7fe700 1 -- 172.21.2.111:0/115001452 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2800bd60 con 0x7f8d300b0c90 2021-04-03T03:15:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d2f7fe700 1 -- 172.21.2.111:0/115001452 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d28011400 con 0x7f8d300b0c90 2021-04-03T03:15:16.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- 172.21.2.111:0/115001452 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300b0c90 msgr2=0x7f8d30147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- 172.21.2.111:0/115001452 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300b0c90 0x7f8d30147680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f8d280032b0 tx=0x7f8d28009270).stop 2021-04-03T03:15:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- 172.21.2.111:0/115001452 shutdown_connections 2021-04-03T03:15:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- 172.21.2.111:0/115001452 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300b0c90 0x7f8d30147680 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- 172.21.2.111:0/115001452 >> v2:172.21.2.115:3300/0 conn(0x7f8d3014b700 0x7f8d300b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- 172.21.2.111:0/115001452 >> 172.21.2.111:0/115001452 conn(0x7f8d300b61c0 msgr2=0x7f8d3005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- 172.21.2.111:0/115001452 shutdown_connections 2021-04-03T03:15:16.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- 172.21.2.111:0/115001452 wait complete. 2021-04-03T03:15:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 Processor -- start 2021-04-03T03:15:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- start start 2021-04-03T03:15:16.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300603f0 0x7f8d30062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d300b0c90 0x7f8d301469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d3014b700 0x7f8d30146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300ab630 con 0x7f8d300603f0 2021-04-03T03:15:16.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300ab8d0 con 0x7f8d300b0c90 2021-04-03T03:15:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d301473a0 con 0x7f8d3014b700 2021-04-03T03:15:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300603f0 0x7f8d30062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300603f0 0x7f8d30062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54210/0 (socket says 172.21.2.111:54210) 2021-04-03T03:15:16.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 -- 172.21.2.111:0/3463638022 learned_addr learned my addr 172.21.2.111:0/3463638022 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:16.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 -- 172.21.2.111:0/3463638022 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d3014b700 msgr2=0x7f8d30146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d3014b700 0x7f8d30146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 -- 172.21.2.111:0/3463638022 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d300b0c90 msgr2=0x7f8d301469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d300b0c90 0x7f8d301469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 -- 172.21.2.111:0/3463638022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300aa370 con 0x7f8d300603f0 2021-04-03T03:15:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d348af700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300603f0 0x7f8d30062bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f8d18000b80 tx=0x7f8d18007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d2dffb700 1 -- 172.21.2.111:0/3463638022 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d1800e4a0 con 0x7f8d300603f0 2021-04-03T03:15:16.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d2dffb700 1 -- 172.21.2.111:0/3463638022 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1800e600 con 0x7f8d300603f0 2021-04-03T03:15:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d2dffb700 1 -- 172.21.2.111:0/3463638022 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d1800e960 con 0x7f8d300603f0 2021-04-03T03:15:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d301570f0 con 0x7f8d300603f0 2021-04-03T03:15:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.318+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d300abd70 con 0x7f8d300603f0 2021-04-03T03:15:16.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.322+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d301575a0 con 0x7f8d300603f0 2021-04-03T03:15:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.322+0000 7f8d2dffb700 1 -- 172.21.2.111:0/3463638022 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8d18019030 con 0x7f8d300603f0 2021-04-03T03:15:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.322+0000 7f8d2dffb700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d14056ef0 0x7f8d140593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.322+0000 7f8d2dffb700 1 -- 172.21.2.111:0/3463638022 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8d1806f810 con 0x7f8d300603f0 2021-04-03T03:15:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.322+0000 7f8d2ffff700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d14056ef0 0x7f8d140593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.322+0000 7f8d2ffff700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d14056ef0 0x7f8d140593b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f8d20002f30 tx=0x7f8d200069d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.322+0000 7f8d2dffb700 1 -- 172.21.2.111:0/3463638022 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8d18044ac0 con 0x7f8d300603f0 2021-04-03T03:15:16.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8d30057660 con 0x7f8d300603f0 2021-04-03T03:15:16.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d2dffb700 1 -- 172.21.2.111:0/3463638022 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v75) v1 ==== 81+0+34116 (secure 0 0 0) 0x7f8d18017020 con 0x7f8d300603f0 2021-04-03T03:15:16.429 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:16.430 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:15:16.430 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 75, 2021-04-03T03:15:16.430 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:15:16.430 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:15:16.430 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:15:05.913226+0000", 2021-04-03T03:15:16.431 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:15:16.431 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:15:16.431 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:15:16.431 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:15:16.432 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:15:16.432 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:15:16.432 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:15:16.432 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:15:16.432 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:15:16.433 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:16.433 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:15:16.433 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:15:16.433 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:15:16.434 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:15:16.434 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:15:16.434 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 14, 2021-04-03T03:15:16.435 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:15:16.435 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:15:16.435 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:15:16.435 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:15:16.435 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:15:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:15:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:15:16.436 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:15:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:16.437 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:16.438 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:16.438 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:16.438 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:16.438 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:15:16.438 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:15:16.439 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:15:16.439 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:15:16.439 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:15:16.440 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:16.440 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:16.441 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:16.441 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:16.441 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:16.442 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:16.442 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:16.443 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.443 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:15:16.444 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:16.444 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:16.444 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:16.445 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:16.445 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:16.446 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:16.446 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:16.446 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:16.447 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:16.447 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:16.448 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:16.448 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:16.448 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:16.449 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:16.449 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:16.450 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:16.450 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:16.450 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:16.451 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:16.451 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:16.452 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:16.452 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:16.453 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:16.453 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:16.453 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:16.454 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:16.454 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.455 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:16.455 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:16.456 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:16.456 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:16.457 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:16.457 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:16.457 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:16.458 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:16.458 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:16.459 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:16.459 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:16.459 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:15:16.460 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:15:16.460 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.461 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:16.461 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:15:16.461 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.461 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.461 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.462 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:15:16.462 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:15:16.462 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:15:16.462 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:16.463 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:16.463 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:16.463 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:16.463 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:16.463 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:16.464 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:16.464 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:16.464 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:16.464 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:16.464 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:16.465 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:16.465 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:16.465 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:16.465 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:16.466 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:16.466 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:16.466 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:16.466 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:16.466 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:16.467 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.467 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:15:16.467 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:16.467 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:16.467 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:16.468 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:16.468 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:16.468 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:16.468 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:16.468 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:16.469 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:16.469 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:16.469 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:16.469 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:16.470 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:16.470 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:16.470 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:16.470 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:16.470 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:16.471 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:16.471 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:16.471 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:16.471 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:16.471 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:16.472 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:16.472 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:16.472 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:16.472 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:16.472 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.473 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:16.473 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:16.473 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:16.473 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:16.474 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:16.474 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:16.474 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:16.474 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:16.474 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:16.475 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:16.475 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:16.475 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:15:16.475 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:16.475 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:15:16.476 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.476 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.476 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.476 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 13, 2021-04-03T03:15:16.476 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:15:16.477 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:14:22.039727+0000", 2021-04-03T03:15:16.477 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:16.477 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:16.477 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:16.478 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:16.478 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:16.478 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:16.478 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:16.478 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:16.479 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:16.479 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:16.479 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:16.479 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:16.479 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:16.480 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:16.480 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:16.480 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:16.480 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:16.480 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:16.481 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:16.481 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:16.481 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.481 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "73", 2021-04-03T03:15:16.482 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:16.482 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:16.482 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:16.482 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:16.482 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:16.483 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:16.483 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:16.483 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:16.483 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:16.483 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:16.484 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:16.484 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:16.484 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:16.484 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:16.484 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:16.485 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:16.485 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:16.485 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:16.485 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:16.485 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:16.486 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:16.486 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:16.486 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:16.486 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:16.487 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:16.487 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:16.487 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.487 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:16.487 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:16.488 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:16.488 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:16.488 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:16.488 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:16.488 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:16.489 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:16.489 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:16.489 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:16.489 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:16.490 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:15:16.490 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:15:16.490 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:15:16.490 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:15:16.490 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.491 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:16.491 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:15:16.491 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:15:16.491 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.491 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.492 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.492 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.492 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 14, 2021-04-03T03:15:16.492 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:15:16.492 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:14:23.053211+0000", 2021-04-03T03:15:16.493 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:16.493 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:16.493 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:16.493 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:16.494 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:16.494 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:16.494 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:16.494 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:16.494 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:16.495 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:16.495 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:16.495 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:16.495 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:16.495 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:16.496 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:16.496 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:16.496 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:16.496 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:16.496 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:16.497 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:16.497 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.497 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "74", 2021-04-03T03:15:16.497 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:16.498 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:16.498 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:16.498 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:16.498 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:16.498 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:16.499 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:16.499 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:16.499 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:16.499 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:16.499 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:16.500 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:16.500 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:16.500 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:16.500 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:16.500 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:16.501 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:16.501 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:16.501 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:16.501 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:16.501 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:16.502 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:16.502 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:16.502 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:16.502 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:16.502 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:16.503 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.503 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:16.503 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:16.503 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:16.504 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:16.504 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:16.504 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:16.504 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:16.504 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:16.505 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:16.505 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:16.505 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:16.505 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:15:16.505 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:16.506 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:15:16.506 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:15:16.506 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.506 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.506 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.507 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:16.507 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:15:16.507 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.507 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:15:16.508 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:15:16.508 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.508 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.508 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.508 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.509 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.509 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.509 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.509 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 70, 2021-04-03T03:15:16.509 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.510 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.510 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.510 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.510 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.510 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.511 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:15:16.511 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.511 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.511 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.511 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.512 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:15:16.512 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.512 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.512 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.512 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.513 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.513 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.513 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.513 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.514 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:15:16.514 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.514 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.514 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.514 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.515 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:15:16.515 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.515 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.515 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.515 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.516 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.516 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.516 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.516 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.516 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:15:16.517 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.517 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.517 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.517 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.517 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:15:16.518 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.518 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.518 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.518 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.519 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.519 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.519 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.519 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.519 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:15:16.520 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.520 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.520 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.520 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.520 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:15:16.521 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:16.521 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.521 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.521 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.521 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:15:16.522 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:15:16.522 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:15:16.522 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:15:16.522 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.522 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.523 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.523 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.523 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.523 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.524 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:15:16.524 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:15:16.524 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.524 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.524 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.525 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.525 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.525 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.525 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.525 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 60, 2021-04-03T03:15:16.526 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.526 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.526 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.526 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.526 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.527 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.527 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:15:16.527 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.527 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.527 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.528 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.528 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:15:16.528 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.528 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.529 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.529 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.529 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.529 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.529 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.530 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.530 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:15:16.530 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.530 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.530 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.531 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.531 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:15:16.531 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.531 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.531 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.532 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.532 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.532 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.532 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.532 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.533 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:15:16.533 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.533 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.533 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.533 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.534 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:15:16.534 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.534 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.534 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.535 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.535 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.535 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.535 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.535 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.536 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:15:16.536 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.536 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.536 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.536 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.537 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:15:16.537 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:16.537 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.537 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.537 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.538 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:15:16.538 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:15:16.538 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:15:16.538 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:15:16.539 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.539 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.539 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.539 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.539 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.540 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.540 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:15:16.540 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:15:16.540 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.540 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.541 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.541 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.541 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.541 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.541 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.542 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 70, 2021-04-03T03:15:16.542 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.542 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.542 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.542 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.543 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.543 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.543 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:15:16.543 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.543 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.544 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.544 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.544 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:15:16.544 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.545 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.545 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.545 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.545 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.545 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.546 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.546 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.546 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:15:16.546 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.546 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.547 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.547 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.547 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:15:16.547 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.547 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.548 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.548 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.548 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.548 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.549 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.549 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.549 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:15:16.549 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.549 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.550 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.550 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.550 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:15:16.550 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.550 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.551 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.551 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.551 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.551 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.551 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.552 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.552 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:15:16.552 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.552 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.552 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.553 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.553 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:15:16.553 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:16.553 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.554 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.554 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.554 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:15:16.554 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:15:16.554 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:15:16.555 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:15:16.555 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.555 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.555 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.555 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.556 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.556 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.556 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:15:16.556 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:15:16.556 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.557 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.557 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.557 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.557 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.557 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.558 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.558 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 70, 2021-04-03T03:15:16.558 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.558 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.558 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.559 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.559 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.559 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.559 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:15:16.560 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.560 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.560 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.560 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.560 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:15:16.561 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.561 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.561 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.561 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.561 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.562 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.562 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.562 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.562 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:15:16.562 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.563 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.563 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.563 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.563 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:15:16.564 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.564 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.564 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.564 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.564 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.565 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.565 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.565 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.565 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:15:16.565 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.566 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.566 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.566 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.566 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:15:16.566 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.567 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.567 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.567 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.567 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.567 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.568 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.568 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.568 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:15:16.568 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.568 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.569 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.569 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.569 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:15:16.569 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:16.570 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.570 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.570 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.570 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:15:16.570 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:15:16.571 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:15:16.571 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:15:16.571 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.571 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.571 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.572 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.572 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.572 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.572 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:15:16.572 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:15:16.573 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.573 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.573 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.573 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.574 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.574 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.574 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.574 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 69, 2021-04-03T03:15:16.574 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.575 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.575 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.575 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.575 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.575 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.576 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:15:16.576 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.576 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.576 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.576 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.577 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:15:16.577 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.577 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.577 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.577 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.578 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.578 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.578 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.578 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.578 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:15:16.579 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.579 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.579 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.579 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.580 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:15:16.580 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.580 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.580 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.580 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.581 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.581 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.581 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.581 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.581 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:15:16.582 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.582 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.582 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.582 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.582 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:15:16.583 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.583 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.583 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.583 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.583 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.584 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.584 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.584 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.584 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:15:16.584 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.585 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.585 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.585 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.585 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:15:16.586 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:16.586 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.586 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.586 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.586 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:15:16.587 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:15:16.587 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:15:16.587 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:15:16.587 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.587 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.588 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.588 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.588 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.588 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.588 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:15:16.589 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:15:16.589 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.589 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.589 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.589 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.590 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.590 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.590 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.590 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 70, 2021-04-03T03:15:16.591 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.591 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.591 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.591 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.591 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.592 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.592 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:15:16.592 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.592 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.592 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.593 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.593 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:15:16.593 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.593 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.593 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.594 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.594 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.594 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.594 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.594 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.595 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:15:16.595 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.595 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.595 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.595 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.596 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:15:16.596 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.596 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.596 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.597 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.597 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.597 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.597 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.597 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.598 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:15:16.598 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.598 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.598 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.598 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.599 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:15:16.599 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.599 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.599 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.600 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.600 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.600 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.600 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.600 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:15:16.601 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.601 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.601 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.601 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.602 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:15:16.602 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:16.602 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.602 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.602 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.603 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:15:16.603 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:15:16.603 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:15:16.603 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:15:16.603 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.604 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.604 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.604 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.604 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.604 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.605 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:15:16.605 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:15:16.605 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.605 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.605 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.606 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.606 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.606 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.606 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.606 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 70, 2021-04-03T03:15:16.607 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.607 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.607 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.607 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.608 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.608 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.608 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:15:16.608 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.608 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.609 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.609 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.609 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:15:16.609 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.609 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.610 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.610 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.610 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.610 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.610 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.611 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.611 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:15:16.611 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.611 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.611 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.612 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.612 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:15:16.612 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.612 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.612 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.613 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.613 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.613 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.613 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.614 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.614 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:15:16.614 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.614 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.614 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.615 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.615 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:15:16.615 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.615 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.615 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.616 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.616 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.616 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.616 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.616 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.617 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:15:16.617 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.617 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.617 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.617 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.618 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:15:16.618 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:16.618 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.618 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.618 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.619 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:15:16.619 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:15:16.619 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:15:16.619 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:15:16.620 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.620 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.620 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.620 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.620 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.621 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.621 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:15:16.621 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:15:16.621 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:16.621 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:16.622 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:16.622 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:16.622 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:16.622 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:16.622 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:16.623 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 69, 2021-04-03T03:15:16.623 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:16.623 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:16.623 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:16.623 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.624 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.624 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.624 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:15:16.624 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.624 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.625 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.625 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.625 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:15:16.625 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.625 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.626 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.626 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.626 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:16.626 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.627 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.627 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.627 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:15:16.627 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.627 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.628 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.628 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.628 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:15:16.628 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.628 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.629 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.629 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.629 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:16.629 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.629 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.630 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.630 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:15:16.630 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.630 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.630 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.631 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.631 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:15:16.631 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.631 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.632 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.632 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.632 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:16.632 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:16.632 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.633 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:16.633 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:15:16.633 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.633 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.633 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.634 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:16.634 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:15:16.634 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:16.634 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.634 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.635 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.635 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:15:16.635 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:15:16.635 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:15:16.635 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:15:16.636 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:16.636 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:16.636 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:16.636 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:16.636 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.637 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:16.637 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:15:16.637 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.637 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:15:16.637 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.638 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.638 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.638 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.638 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.639 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:15:16.639 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.639 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.639 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.639 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:15:16.640 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.640 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.640 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.640 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.640 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.641 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:15:16.641 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.641 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.641 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.641 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:15:16.642 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.642 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.642 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.642 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.642 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.643 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:15:16.643 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.643 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.643 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.644 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:15:16.644 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.644 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.644 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.644 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.645 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.645 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:15:16.645 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.645 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.645 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.646 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:15:16.646 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.646 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.646 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.646 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.647 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.647 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:15:16.647 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.647 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.648 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.648 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:15:16.648 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.648 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.648 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.649 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.649 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.649 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:15:16.649 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.650 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.650 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:15:16.650 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.650 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.650 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.651 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.651 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.651 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:15:16.651 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.652 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.652 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:16.652 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:15:16.652 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:16.652 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:16.653 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:16.653 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:16.653 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:16.653 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:15:16.653 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:16.654 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.654 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:16.654 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:15:16.654 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:15:16.654 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:15:16.655 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:15:16.655 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": { 2021-04-03T03:15:16.655 INFO:teuthology.orchestra.run.gibba011.stdout: "172.21.2.126:0/2003837950": "2021-04-03T04:15:04.941896+0000" 2021-04-03T03:15:16.655 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.656 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:15:16.656 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:15:16.656 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:15:16.656 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:15:16.656 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:15:16.657 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:15:16.657 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:15:16.657 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:15:16.657 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:15:16.657 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:15:16.658 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:15:16.658 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:15:16.658 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.658 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:15:16.658 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:15:16.659 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:15:16.659 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:15:16.659 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:15:16.659 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:15:16.660 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:15:16.660 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:16.660 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:16.660 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:15:16.660 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:15:16.661 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:15:16.661 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:15:16.661 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:15:16.661 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:15:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d14056ef0 msgr2=0x7f8d140593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d14056ef0 0x7f8d140593b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f8d20002f30 tx=0x7f8d200069d0).stop 2021-04-03T03:15:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300603f0 msgr2=0x7f8d30062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300603f0 0x7f8d30062bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f8d18000b80 tx=0x7f8d18007a50).stop 2021-04-03T03:15:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 shutdown_connections 2021-04-03T03:15:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d3014b700 0x7f8d30146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d300603f0 0x7f8d30062bf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d14056ef0 0x7f8d140593b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 --2- 172.21.2.111:0/3463638022 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d300b0c90 0x7f8d301469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 >> 172.21.2.111:0/3463638022 conn(0x7f8d300b61c0 msgr2=0x7f8d300acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 shutdown_connections 2021-04-03T03:15:16.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.418+0000 7f8d358b1700 1 -- 172.21.2.111:0/3463638022 wait complete. 2021-04-03T03:15:16.666 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:16.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 Processor -- start 2021-04-03T03:15:16.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 -- start start 2021-04-03T03:15:16.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 0x7fcbd4006260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fcbd40056b0 0x7fcbd40d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcbd4005b30 0x7fcbd40d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd403fb20 con 0x7fcbd4005b30 2021-04-03T03:15:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd403fdc0 con 0x7fcbd40056b0 2021-04-03T03:15:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4040060 con 0x7fcbd4006a30 2021-04-03T03:15:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcbd4005b30 0x7fcbd40d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcbd4005b30 0x7fcbd40d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54222/0 (socket says 172.21.2.111:54222) 2021-04-03T03:15:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 -- 172.21.2.111:0/1565794246 learned_addr learned my addr 172.21.2.111:0/1565794246 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 -- 172.21.2.111:0/1565794246 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 msgr2=0x7fcbd4006260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 --2- 172.21.2.111:0/1565794246 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 0x7fcbd4006260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 -- 172.21.2.111:0/1565794246 >> v2:172.21.2.115:3300/0 conn(0x7fcbd40056b0 msgr2=0x7fcbd40d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 --2- 172.21.2.111:0/1565794246 >> v2:172.21.2.115:3300/0 conn(0x7fcbd40056b0 0x7fcbd40d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 -- 172.21.2.111:0/1565794246 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4040500 con 0x7fcbd4005b30 2021-04-03T03:15:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdad9d700 1 --2- 172.21.2.111:0/1565794246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40d0db0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fcbdc06b5d0 tx=0x7fcbdc05ad70).ready entity=mon.0 client_cookie=abb0e37d258d766f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbd959a700 1 -- 172.21.2.111:0/1565794246 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbdc05bda0 con 0x7fcbd4005b30 2021-04-03T03:15:16.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbd959a700 1 -- 172.21.2.111:0/1565794246 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc05df60 con 0x7fcbd4005b30 2021-04-03T03:15:16.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbd959a700 1 -- 172.21.2.111:0/1565794246 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbdc0699f0 con 0x7fcbd4005b30 2021-04-03T03:15:16.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 -- 172.21.2.111:0/1565794246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 msgr2=0x7fcbd40d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/1565794246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40d0db0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fcbdc06b5d0 tx=0x7fcbdc05ad70).stop 2021-04-03T03:15:16.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 -- 172.21.2.111:0/1565794246 shutdown_connections 2021-04-03T03:15:16.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/1565794246 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40d0db0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/1565794246 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 0x7fcbd4006260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/1565794246 >> v2:172.21.2.115:3300/0 conn(0x7fcbd40056b0 0x7fcbd40d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.718+0000 7fcbdb59e700 1 -- 172.21.2.111:0/1565794246 >> 172.21.2.111:0/1565794246 conn(0x7fcbd40462c0 msgr2=0x7fcbd40466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:16.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- 172.21.2.111:0/1565794246 shutdown_connections 2021-04-03T03:15:16.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- 172.21.2.111:0/1565794246 wait complete. 2021-04-03T03:15:16.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 Processor -- start 2021-04-03T03:15:16.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- start start 2021-04-03T03:15:16.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcbd40056b0 0x7fcbd40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 0x7fcbd40d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40d0680 con 0x7fcbd4005b30 2021-04-03T03:15:16.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40e42a0 con 0x7fcbd40056b0 2021-04-03T03:15:16.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cdf20 con 0x7fcbd4006a30 2021-04-03T03:15:16.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54226/0 (socket says 172.21.2.111:54226) 2021-04-03T03:15:16.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 -- 172.21.2.111:0/491233618 learned_addr learned my addr 172.21.2.111:0/491233618 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:16.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 -- 172.21.2.111:0/491233618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 msgr2=0x7fcbd40d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 0x7fcbd40d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 -- 172.21.2.111:0/491233618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcbd40056b0 msgr2=0x7fcbd40cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcbd40056b0 0x7fcbd40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 -- 172.21.2.111:0/491233618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40ce1c0 con 0x7fcbd4005b30 2021-04-03T03:15:16.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd9d9b700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40cf120 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fcbcc002f30 tx=0x7fcbcc006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd37fe700 1 -- 172.21.2.111:0/491233618 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbcc00d840 con 0x7fcbd4005b30 2021-04-03T03:15:16.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd37fe700 1 -- 172.21.2.111:0/491233618 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc000e90 con 0x7fcbd4005b30 2021-04-03T03:15:16.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd37fe700 1 -- 172.21.2.111:0/491233618 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbcc00db90 con 0x7fcbd4005b30 2021-04-03T03:15:16.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40cdc10 con 0x7fcbd4005b30 2021-04-03T03:15:16.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd40d6040 con 0x7fcbd4005b30 2021-04-03T03:15:16.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd4004da0 con 0x7fcbd4005b30 2021-04-03T03:15:16.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd37fe700 1 -- 172.21.2.111:0/491233618 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fcbcc017030 con 0x7fcbd4005b30 2021-04-03T03:15:16.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd37fe700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcbc0056ea0 0x7fcbc0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbd37fe700 1 -- 172.21.2.111:0/491233618 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fcbcc06f610 con 0x7fcbd4005b30 2021-04-03T03:15:16.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbda59c700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcbc0056ea0 0x7fcbc0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.722+0000 7fcbda59c700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcbc0056ea0 0x7fcbc0059360 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fcbc4002f30 tx=0x7fcbc4006970).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.726+0000 7fcbd37fe700 1 -- 172.21.2.111:0/491233618 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fcbcc044a80 con 0x7fcbd4005b30 2021-04-03T03:15:16.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbd40470d0 con 0x7fcbd4005b30 2021-04-03T03:15:16.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbd37fe700 1 -- 172.21.2.111:0/491233618 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fcbcc0480a0 con 0x7fcbd4005b30 2021-04-03T03:15:16.859 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:16.859 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":29,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:14:25.081940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6210":{"gid":6210,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/560115150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":560115150},{"type":"v1","addr":"172.21.2.111:6835","nonce":560115150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:15:16.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcbc0056ea0 msgr2=0x7fcbc0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcbc0056ea0 0x7fcbc0059360 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fcbc4002f30 tx=0x7fcbc4006970).stop 2021-04-03T03:15:16.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 msgr2=0x7fcbd40cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40cf120 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fcbcc002f30 tx=0x7fcbcc006ea0).stop 2021-04-03T03:15:16.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 shutdown_connections 2021-04-03T03:15:16.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcbd4006a30 0x7fcbd40d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcbc0056ea0 0x7fcbc0059360 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcbd40056b0 0x7fcbd40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 --2- 172.21.2.111:0/491233618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcbd4005b30 0x7fcbd40cf120 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 >> 172.21.2.111:0/491233618 conn(0x7fcbd40462c0 msgr2=0x7fcbd40d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:16.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 shutdown_connections 2021-04-03T03:15:16.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.850+0000 7fcbdb59e700 1 -- 172.21.2.111:0/491233618 wait complete. 2021-04-03T03:15:16.863 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 29 2021-04-03T03:15:16.866 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:15:16.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 Processor -- start 2021-04-03T03:15:16.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 -- start start 2021-04-03T03:15:16.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe6f80add50 0x7fe6f8147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f80607d0 con 0x7fe6f8144ef0 2021-04-03T03:15:16.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f814b090 con 0x7fe6f80603f0 2021-04-03T03:15:16.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f814b330 con 0x7fe6f80add50 2021-04-03T03:15:16.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f7fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe6f8144ef0 0x7fe6f8147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55028/0 (socket says 172.21.2.111:55028) 2021-04-03T03:15:16.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f7fff700 1 -- 172.21.2.111:0/2076555077 learned_addr learned my addr 172.21.2.111:0/2076555077 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:16.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732555090 0 0) 0x7fe6f80607d0 con 0x7fe6f8144ef0 2021-04-03T03:15:16.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6d8003240 con 0x7fe6f8144ef0 2021-04-03T03:15:16.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/2076555077 >> v2:172.21.2.115:3300/0 conn(0x7fe6f80add50 0x7fe6f8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2277828869 0 0) 0x7fe6d8003240 con 0x7fe6f8144ef0 2021-04-03T03:15:16.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 >> v2:172.21.2.115:3300/0 conn(0x7fe6f80add50 msgr2=0x7fe6f8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 --2- 172.21.2.111:0/2076555077 >> v2:172.21.2.115:3300/0 conn(0x7fe6f80add50 0x7fe6f8147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 >> v1:172.21.2.115:6789/0 conn(0x7fe6f80603f0 legacy=0x7fe6f81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f814b5d0 con 0x7fe6f8144ef0 2021-04-03T03:15:16.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe6f0001a50 con 0x7fe6f8144ef0 2021-04-03T03:15:16.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 >> v1:172.21.2.111:6789/0 conn(0x7fe6f8144ef0 legacy=0x7fe6f8147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d8003800 0x7fe6d8003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d80044e0 0x7fe6d8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d8004e50 0x7fe6d800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f814b330 con 0x7fe6d80044e0 2021-04-03T03:15:16.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d8005300 con 0x7fe6d8004e50 2021-04-03T03:15:16.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d800bc20 con 0x7fe6d8003800 2021-04-03T03:15:16.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d80044e0 0x7fe6d8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/2076555077 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d8003800 msgr2=0x7fe6d8003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d8003800 0x7fe6d8003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/2076555077 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d8004e50 msgr2=0x7fe6d800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:16.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d8004e50 0x7fe6d800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:16.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 -- 172.21.2.111:0/2076555077 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d8003240 con 0x7fe6d80044e0 2021-04-03T03:15:16.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f7fff700 1 -- 172.21.2.111:0/2076555077 reap_dead start 2021-04-03T03:15:16.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d80044e0 0x7fe6d8004960 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe6e800a000 tx=0x7fe6e8008040).ready entity=mon.0 client_cookie=bda5e941b85502cd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6e800da60 con 0x7fe6d80044e0 2021-04-03T03:15:16.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e800dbc0 con 0x7fe6d80044e0 2021-04-03T03:15:16.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6f67fc700 1 -- 172.21.2.111:0/2076555077 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6e8013480 con 0x7fe6d80044e0 2021-04-03T03:15:16.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/2076555077 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d80044e0 msgr2=0x7fe6d8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.915+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d80044e0 0x7fe6d8004960 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe6e800a000 tx=0x7fe6e8008040).stop 2021-04-03T03:15:16.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/2076555077 shutdown_connections 2021-04-03T03:15:16.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/2076555077 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d80044e0 0x7fe6d8004960 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/2076555077 >> 172.21.2.111:0/2076555077 conn(0x7fe6f80b61c0 msgr2=0x7fe6f805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:16.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/2076555077 shutdown_connections 2021-04-03T03:15:16.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/2076555077 wait complete. 2021-04-03T03:15:16.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 Processor -- start 2021-04-03T03:15:16.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- start start 2021-04-03T03:15:16.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d80044e0 0x7fe6f8143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8146260 0x7fe6f8146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f8067cd0 0x7fe6f8146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80aa890 con 0x7fe6f8067cd0 2021-04-03T03:15:16.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80aa5f0 con 0x7fe6d80044e0 2021-04-03T03:15:16.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f8147ff0 con 0x7fe6f8146260 2021-04-03T03:15:16.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f8067cd0 0x7fe6f8146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f8067cd0 0x7fe6f8146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54248/0 (socket says 172.21.2.111:54248) 2021-04-03T03:15:16.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1552801997 learned_addr learned my addr 172.21.2.111:0/1552801997 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:16.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f77fe700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d80044e0 0x7fe6f8143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8146260 0x7fe6f8146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8146260 msgr2=0x7fe6f8146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8146260 0x7fe6f8146960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d80044e0 msgr2=0x7fe6f8143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:16.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d80044e0 0x7fe6f8143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:16.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 -- 172.21.2.111:0/1552801997 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f8147340 con 0x7fe6f8067cd0 2021-04-03T03:15:16.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f6ffd700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8146260 0x7fe6f8146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:16.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f7fff700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f8067cd0 0x7fe6f8146e50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7fe6f80af1d0 tx=0x7fe6f814b220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f4ff9700 1 -- 172.21.2.111:0/1552801997 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f000cea0 con 0x7fe6f8067cd0 2021-04-03T03:15:16.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f4ff9700 1 -- 172.21.2.111:0/1552801997 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0004a90 con 0x7fe6f8067cd0 2021-04-03T03:15:16.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6f4ff9700 1 -- 172.21.2.111:0/1552801997 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f0004df0 con 0x7fe6f8067cd0 2021-04-03T03:15:16.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f8147920 con 0x7fe6f8067cd0 2021-04-03T03:15:16.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f815a390 con 0x7fe6f8067cd0 2021-04-03T03:15:16.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.919+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f8159e70 con 0x7fe6f8067cd0 2021-04-03T03:15:16.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.923+0000 7fe6f4ff9700 1 -- 172.21.2.111:0/1552801997 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe6f0010070 con 0x7fe6f8067cd0 2021-04-03T03:15:16.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.923+0000 7fe6f4ff9700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6dc056f20 0x7fe6dc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:16.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.923+0000 7fe6f4ff9700 1 -- 172.21.2.111:0/1552801997 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe6f006f220 con 0x7fe6f8067cd0 2021-04-03T03:15:16.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.923+0000 7fe6f77fe700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6dc056f20 0x7fe6dc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:16.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.923+0000 7fe6f77fe700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6dc056f20 0x7fe6dc0593e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fe6e000b000 tx=0x7fe6e0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:16.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:16.923+0000 7fe6f4ff9700 1 -- 172.21.2.111:0/1552801997 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe6f00444d0 con 0x7fe6f8067cd0 2021-04-03T03:15:17.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.051+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe6f8057660 con 0x7fe6f8067cd0 2021-04-03T03:15:17.987 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:15:17.983+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:15:17.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.991+0000 7fe6f4ff9700 1 -- 172.21.2.111:0/1552801997 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe6f0047af0 con 0x7fe6f8067cd0 2021-04-03T03:15:18.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6dc056f20 msgr2=0x7fe6dc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6dc056f20 0x7fe6dc0593e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fe6e000b000 tx=0x7fe6e0009040).stop 2021-04-03T03:15:18.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f8067cd0 msgr2=0x7fe6f8146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f8067cd0 0x7fe6f8146e50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7fe6f80af1d0 tx=0x7fe6f814b220).stop 2021-04-03T03:15:18.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 shutdown_connections 2021-04-03T03:15:18.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6f8067cd0 0x7fe6f8146e50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6dc056f20 0x7fe6dc0593e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d80044e0 0x7fe6f8143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 --2- 172.21.2.111:0/1552801997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6f8146260 0x7fe6f8146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 >> 172.21.2.111:0/1552801997 conn(0x7fe6f80b61c0 msgr2=0x7fe6f8143c90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:18.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 shutdown_connections 2021-04-03T03:15:18.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:17.995+0000 7fe6fcdf1700 1 -- 172.21.2.111:0/1552801997 wait complete. 2021-04-03T03:15:18.004 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:15:18.008 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:18.016 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:15:18.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 Processor -- start 2021-04-03T03:15:18.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- start start 2021-04-03T03:15:18.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa73c0603f0 0x7fa73c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c14b700 0x7fa73c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa73c0b12b0 0x7fa73c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0b0f60 con 0x7fa73c0603f0 2021-04-03T03:15:18.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0607d0 con 0x7fa73c0b12b0 2021-04-03T03:15:18.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0af400 con 0x7fa73c14b700 2021-04-03T03:15:18.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa73c0603f0 0x7fa73c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa73c0603f0 0x7fa73c0b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54264/0 (socket says 172.21.2.111:54264) 2021-04-03T03:15:18.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 -- 172.21.2.111:0/1995490842 learned_addr learned my addr 172.21.2.111:0/1995490842 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:18.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 -- 172.21.2.111:0/1995490842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c14b700 msgr2=0x7fa73c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:18.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 --2- 172.21.2.111:0/1995490842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c14b700 0x7fa73c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:18.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73bfff700 1 --2- 172.21.2.111:0/1995490842 >> v2:172.21.2.115:3300/0 conn(0x7fa73c0b12b0 0x7fa73c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 -- 172.21.2.111:0/1995490842 >> v2:172.21.2.115:3300/0 conn(0x7fa73c0b12b0 msgr2=0x7fa73c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:18.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 --2- 172.21.2.111:0/1995490842 >> v2:172.21.2.115:3300/0 conn(0x7fa73c0b12b0 0x7fa73c0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 -- 172.21.2.111:0/1995490842 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0af6a0 con 0x7fa73c0603f0 2021-04-03T03:15:18.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73bfff700 1 --2- 172.21.2.111:0/1995490842 >> v2:172.21.2.115:3300/0 conn(0x7fa73c0b12b0 0x7fa73c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:18.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73b7fe700 1 --2- 172.21.2.111:0/1995490842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c0603f0 0x7fa73c0b0090 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fa724002f30 tx=0x7fa724006e90).ready entity=mon.0 client_cookie=16a3df1b6689e953 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:18.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73a7fc700 1 -- 172.21.2.111:0/1995490842 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa724009920 con 0x7fa73c0603f0 2021-04-03T03:15:18.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73a7fc700 1 -- 172.21.2.111:0/1995490842 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724009a80 con 0x7fa73c0603f0 2021-04-03T03:15:18.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa73a7fc700 1 -- 172.21.2.111:0/1995490842 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa724009de0 con 0x7fa73c0603f0 2021-04-03T03:15:18.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- 172.21.2.111:0/1995490842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c0603f0 msgr2=0x7fa73c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 --2- 172.21.2.111:0/1995490842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c0603f0 0x7fa73c0b0090 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fa724002f30 tx=0x7fa724006e90).stop 2021-04-03T03:15:18.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- 172.21.2.111:0/1995490842 shutdown_connections 2021-04-03T03:15:18.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 --2- 172.21.2.111:0/1995490842 >> v2:172.21.2.115:3300/0 conn(0x7fa73c0b12b0 0x7fa73c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 --2- 172.21.2.111:0/1995490842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c0603f0 0x7fa73c0b0090 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 --2- 172.21.2.111:0/1995490842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c14b700 0x7fa73c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:18.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- 172.21.2.111:0/1995490842 >> 172.21.2.111:0/1995490842 conn(0x7fa73c0b61c0 msgr2=0x7fa73c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:18.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- 172.21.2.111:0/1995490842 shutdown_connections 2021-04-03T03:15:18.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- 172.21.2.111:0/1995490842 wait complete. 2021-04-03T03:15:18.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 Processor -- start 2021-04-03T03:15:18.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.059+0000 7fa74087b700 1 -- start start 2021-04-03T03:15:18.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c0603f0 0x7fa73c062db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa73c0b12b0 0x7fa73c0aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c14b700 0x7fa73c0ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c064c70 con 0x7fa73c14b700 2021-04-03T03:15:18.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c064ee0 con 0x7fa73c0b12b0 2021-04-03T03:15:18.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0aa1a0 con 0x7fa73c0603f0 2021-04-03T03:15:18.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c14b700 0x7fa73c0ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c14b700 0x7fa73c0ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54274/0 (socket says 172.21.2.111:54274) 2021-04-03T03:15:18.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 -- 172.21.2.111:0/2207986063 learned_addr learned my addr 172.21.2.111:0/2207986063 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:18.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73b7fe700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c0603f0 0x7fa73c062db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73affd700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa73c0b12b0 0x7fa73c0aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 -- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c0603f0 msgr2=0x7fa73c062db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c0603f0 0x7fa73c062db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 -- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa73c0b12b0 msgr2=0x7fa73c0aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa73c0b12b0 0x7fa73c0aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 -- 172.21.2.111:0/2207986063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0abe20 con 0x7fa73c14b700 2021-04-03T03:15:18.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73affd700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa73c0b12b0 0x7fa73c0aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:18.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73bfff700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c14b700 0x7fa73c0ab3c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fa734007000 tx=0x7fa734008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:18.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa738ff9700 1 -- 172.21.2.111:0/2207986063 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa734015070 con 0x7fa73c14b700 2021-04-03T03:15:18.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa738ff9700 1 -- 172.21.2.111:0/2207986063 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73400a8f0 con 0x7fa73c14b700 2021-04-03T03:15:18.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa738ff9700 1 -- 172.21.2.111:0/2207986063 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa73400ac50 con 0x7fa73c14b700 2021-04-03T03:15:18.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c067630 con 0x7fa73c14b700 2021-04-03T03:15:18.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c067aa0 con 0x7fa73c14b700 2021-04-03T03:15:18.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c068010 con 0x7fa73c14b700 2021-04-03T03:15:18.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa738ff9700 1 -- 172.21.2.111:0/2207986063 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa73401a410 con 0x7fa73c14b700 2021-04-03T03:15:18.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa738ff9700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa720057000 0x7fa7200594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa738ff9700 1 -- 172.21.2.111:0/2207986063 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6545+0+0 (secure 0 0 0) 0x7fa734070670 con 0x7fa73c14b700 2021-04-03T03:15:18.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73b7fe700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa720057000 0x7fa7200594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.063+0000 7fa73b7fe700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa720057000 0x7fa7200594c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fa72401a000 tx=0x7fa724015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:18.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.067+0000 7fa738ff9700 1 -- 172.21.2.111:0/2207986063 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa7340458f0 con 0x7fa73c14b700 2021-04-03T03:15:18.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa73c0650a0 con 0x7fa73c14b700 2021-04-03T03:15:18.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa738ff9700 1 -- 172.21.2.111:0/2207986063 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+1776 (secure 0 0 0) 0x7fa73400adb0 con 0x7fa73c14b700 2021-04-03T03:15:18.199 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:18.199 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/851714650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":851714650},{"type":"v1","addr":"172.21.2.115:6833","nonce":851714650}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:14:24.079081+0000","modified":"2021-04-03T03:15:17.984783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-04-03T03:15:18.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa720057000 msgr2=0x7fa7200594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa720057000 0x7fa7200594c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fa72401a000 tx=0x7fa724015000).stop 2021-04-03T03:15:18.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c14b700 msgr2=0x7fa73c0ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c14b700 0x7fa73c0ab3c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fa734007000 tx=0x7fa734008040).stop 2021-04-03T03:15:18.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 shutdown_connections 2021-04-03T03:15:18.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa73c14b700 0x7fa73c0ab3c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa73c0603f0 0x7fa73c062db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa720057000 0x7fa7200594c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 --2- 172.21.2.111:0/2207986063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa73c0b12b0 0x7fa73c0aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 >> 172.21.2.111:0/2207986063 conn(0x7fa73c0b61c0 msgr2=0x7fa73c0b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:18.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 shutdown_connections 2021-04-03T03:15:18.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.191+0000 7fa74087b700 1 -- 172.21.2.111:0/2207986063 wait complete. 2021-04-03T03:15:18.206 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 30 2021-04-03T03:15:18.208 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:15:18.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 Processor -- start 2021-04-03T03:15:18.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 -- start start 2021-04-03T03:15:18.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb41c0603f0 0x7fb41c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb41c0add50 0x7fb41c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c0607d0 con 0x7fb41c0b17b0 2021-04-03T03:15:18.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c1451c0 con 0x7fb41c0add50 2021-04-03T03:15:18.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c14b090 con 0x7fb41c0603f0 2021-04-03T03:15:18.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb41c0add50 0x7fb41c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb41c0add50 0x7fb41c144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54282/0 (socket says 172.21.2.111:54282) 2021-04-03T03:15:18.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 -- 172.21.2.111:0/2823403832 learned_addr learned my addr 172.21.2.111:0/2823403832 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:18.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 -- 172.21.2.111:0/2823403832 >> v2:172.21.2.115:3300/0 conn(0x7fb41c0603f0 msgr2=0x7fb41c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:18.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 --2- 172.21.2.111:0/2823403832 >> v2:172.21.2.115:3300/0 conn(0x7fb41c0603f0 0x7fb41c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:18.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 -- 172.21.2.111:0/2823403832 >> v1:172.21.2.115:6789/0 conn(0x7fb41c0b17b0 legacy=0x7fb41c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:18.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 -- 172.21.2.111:0/2823403832 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c14b330 con 0x7fb41c0add50 2021-04-03T03:15:18.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41a7fc700 1 --2- 172.21.2.111:0/2823403832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0add50 0x7fb41c144cd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fb41c0ae520 tx=0x7fb40c006e90).ready entity=mon.0 client_cookie=ce072548b6284447 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:18.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb419ffb700 1 -- 172.21.2.111:0/2823403832 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40c009930 con 0x7fb41c0add50 2021-04-03T03:15:18.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb419ffb700 1 -- 172.21.2.111:0/2823403832 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c009a90 con 0x7fb41c0add50 2021-04-03T03:15:18.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb419ffb700 1 -- 172.21.2.111:0/2823403832 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40c009e30 con 0x7fb41c0add50 2021-04-03T03:15:18.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 -- 172.21.2.111:0/2823403832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0add50 msgr2=0x7fb41c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.299+0000 7fb41bfff700 1 --2- 172.21.2.111:0/2823403832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0add50 0x7fb41c144cd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fb41c0ae520 tx=0x7fb40c006e90).stop 2021-04-03T03:15:18.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- 172.21.2.111:0/2823403832 shutdown_connections 2021-04-03T03:15:18.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 --2- 172.21.2.111:0/2823403832 >> v2:172.21.2.115:3300/0 conn(0x7fb41c0603f0 0x7fb41c1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:18.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 --2- 172.21.2.111:0/2823403832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0add50 0x7fb41c144cd0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:18.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- 172.21.2.111:0/2823403832 >> 172.21.2.111:0/2823403832 conn(0x7fb41c0b61c0 msgr2=0x7fb41c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:18.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- 172.21.2.111:0/2823403832 shutdown_connections 2021-04-03T03:15:18.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- 172.21.2.111:0/2823403832 wait complete. 2021-04-03T03:15:18.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 Processor -- start 2021-04-03T03:15:18.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- start start 2021-04-03T03:15:18.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0603f0 0x7fb41c0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb41c0add50 0x7fb41c065410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb41c0b17b0 0x7fb41c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0aaaf0 con 0x7fb41c0603f0 2021-04-03T03:15:18.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0ab700 con 0x7fb41c0b17b0 2021-04-03T03:15:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c063670 con 0x7fb41c0add50 2021-04-03T03:15:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0603f0 0x7fb41c0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0603f0 0x7fb41c0abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54284/0 (socket says 172.21.2.111:54284) 2021-04-03T03:15:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 -- 172.21.2.111:0/3931981646 learned_addr learned my addr 172.21.2.111:0/3931981646 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 -- 172.21.2.111:0/3931981646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb41c0add50 msgr2=0x7fb41c065410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb41c0add50 0x7fb41c065410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 -- 172.21.2.111:0/3931981646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb41c0b17b0 msgr2=0x7fb41c062c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb41c0b17b0 0x7fb41c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 -- 172.21.2.111:0/3931981646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0a9c50 con 0x7fb41c0603f0 2021-04-03T03:15:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41affd700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0603f0 0x7fb41c0abb50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fb404002f30 tx=0x7fb404006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb413fff700 1 -- 172.21.2.111:0/3931981646 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40400d840 con 0x7fb41c0603f0 2021-04-03T03:15:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb413fff700 1 -- 172.21.2.111:0/3931981646 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404000e90 con 0x7fb41c0603f0 2021-04-03T03:15:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c0ab040 con 0x7fb41c0603f0 2021-04-03T03:15:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb413fff700 1 -- 172.21.2.111:0/3931981646 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40400db90 con 0x7fb41c0603f0 2021-04-03T03:15:18.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c0aa8c0 con 0x7fb41c0603f0 2021-04-03T03:15:18.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c0aac60 con 0x7fb41c0603f0 2021-04-03T03:15:18.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb413fff700 1 -- 172.21.2.111:0/3931981646 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb404017030 con 0x7fb41c0603f0 2021-04-03T03:15:18.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb413fff700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb400056f20 0x7fb4000593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:18.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb413fff700 1 -- 172.21.2.111:0/3931981646 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6545+0+0 (secure 0 0 0) 0x7fb40406f6d0 con 0x7fb41c0603f0 2021-04-03T03:15:18.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41a7fc700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb400056f20 0x7fb4000593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:18.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.303+0000 7fb41a7fc700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb400056f20 0x7fb4000593e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fb41c0ae520 tx=0x7fb40c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:18.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.307+0000 7fb413fff700 1 -- 172.21.2.111:0/3931981646 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb41c0aac60 con 0x7fb41c0603f0 2021-04-03T03:15:18.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:18.431+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb41c0aae20 con 0x7fb41c0603f0 2021-04-03T03:15:19.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb413fff700 1 -- 172.21.2.111:0/3931981646 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb41c0aae20 con 0x7fb41c0603f0 2021-04-03T03:15:19.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb400056f20 msgr2=0x7fb4000593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:19.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb400056f20 0x7fb4000593e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fb41c0ae520 tx=0x7fb40c015000).stop 2021-04-03T03:15:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0603f0 msgr2=0x7fb41c0abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0603f0 0x7fb41c0abb50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fb404002f30 tx=0x7fb404006ea0).stop 2021-04-03T03:15:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 shutdown_connections 2021-04-03T03:15:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb400056f20 0x7fb4000593e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb41c0b17b0 0x7fb41c062c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb41c0603f0 0x7fb41c0abb50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 --2- 172.21.2.111:0/3931981646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb41c0add50 0x7fb41c065410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 >> 172.21.2.111:0/3931981646 conn(0x7fb41c0b61c0 msgr2=0x7fb41c147fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 shutdown_connections 2021-04-03T03:15:19.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.063+0000 7fb41bfff700 1 -- 172.21.2.111:0/3931981646 wait complete. 2021-04-03T03:15:19.079 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:15:19.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 Processor -- start 2021-04-03T03:15:19.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- start start 2021-04-03T03:15:19.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3b6c047130 0x7f3b6c042490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c03f550 0x7f3b6c0d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b6c042980 con 0x7f3b6c041660 2021-04-03T03:15:19.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6c047510 con 0x7f3b6c047130 2021-04-03T03:15:19.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c03da30 con 0x7f3b6c03f550 2021-04-03T03:15:19.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3b6c041660 0x7f3b6c03ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55086/0 (socket says 172.21.2.111:55086) 2021-04-03T03:15:19.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 -- 172.21.2.111:0/918711117 learned_addr learned my addr 172.21.2.111:0/918711117 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:19.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718053146 0 0) 0x7f3b6c042980 con 0x7f3b6c041660 2021-04-03T03:15:19.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b54003240 con 0x7f3b6c041660 2021-04-03T03:15:19.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7259c700 1 --2- 172.21.2.111:0/918711117 >> v2:172.21.2.115:3300/0 conn(0x7f3b6c047130 0x7f3b6c042490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:19.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b72d9d700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c03f550 0x7f3b6c0d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:19.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3717418149 0 0) 0x7f3b54003240 con 0x7f3b6c041660 2021-04-03T03:15:19.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c03f550 msgr2=0x7f3b6c0d87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:19.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c03f550 0x7f3b6c0d87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:19.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 >> v2:172.21.2.115:3300/0 conn(0x7f3b6c047130 msgr2=0x7f3b6c042490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:19.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 --2- 172.21.2.111:0/918711117 >> v2:172.21.2.115:3300/0 conn(0x7f3b6c047130 0x7f3b6c042490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:19.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c03dcd0 con 0x7f3b6c041660 2021-04-03T03:15:19.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f3b5c0016d0 con 0x7f3b6c041660 2021-04-03T03:15:19.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 >> v1:172.21.2.111:6789/0 conn(0x7f3b6c041660 legacy=0x7f3b6c03ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:19.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b540037b0 0x7f3b54003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b540044e0 0x7f3b54004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b54004e50 0x7f3b5400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c047510 con 0x7f3b540044e0 2021-04-03T03:15:19.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c03da30 con 0x7f3b540037b0 2021-04-03T03:15:19.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540052d0 con 0x7f3b54004e50 2021-04-03T03:15:19.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b540044e0 0x7f3b54004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:19.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 -- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b54004e50 msgr2=0x7f3b5400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:19.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b54004e50 0x7f3b5400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:19.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 -- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b540037b0 msgr2=0x7f3b54003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:19.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b540037b0 0x7f3b54003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:19.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 -- 172.21.2.111:0/918711117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c03dcd0 con 0x7f3b540044e0 2021-04-03T03:15:19.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b71d9b700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b540044e0 0x7f3b54004960 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f3b5c0013e0 tx=0x7f3b5c0077d0).ready entity=mon.0 client_cookie=363df74f6377e66b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:19.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b5c00ce10 con 0x7f3b540044e0 2021-04-03T03:15:19.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c0072e0 con 0x7f3b540044e0 2021-04-03T03:15:19.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7159a700 1 -- 172.21.2.111:0/918711117 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b5c0057e0 con 0x7f3b540044e0 2021-04-03T03:15:19.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- 172.21.2.111:0/918711117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b540044e0 msgr2=0x7f3b54004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:19.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 --2- 172.21.2.111:0/918711117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b540044e0 0x7f3b54004960 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f3b5c0013e0 tx=0x7f3b5c0077d0).stop 2021-04-03T03:15:19.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b72d9d700 1 -- 172.21.2.111:0/918711117 reap_dead start 2021-04-03T03:15:19.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- 172.21.2.111:0/918711117 shutdown_connections 2021-04-03T03:15:19.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- 172.21.2.111:0/918711117 >> 172.21.2.111:0/918711117 conn(0x7f3b6c00d510 msgr2=0x7f3b6c046700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:19.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- 172.21.2.111:0/918711117 shutdown_connections 2021-04-03T03:15:19.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.131+0000 7f3b7359e700 1 -- 172.21.2.111:0/918711117 wait complete. 2021-04-03T03:15:19.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 Processor -- start 2021-04-03T03:15:19.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 -- start start 2021-04-03T03:15:19.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c043210 0x7f3b6c0a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b6c03c0f0 0x7f3b6c0e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b6c0e6e10 0x7f3b6c0e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0a1e40 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0a3810 con 0x7f3b6c03c0f0 2021-04-03T03:15:19.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0435f0 con 0x7f3b6c043210 2021-04-03T03:15:19.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b6c0e6e10 0x7f3b6c0e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:19.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b6c0e6e10 0x7f3b6c0e9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54308/0 (socket says 172.21.2.111:54308) 2021-04-03T03:15:19.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 -- 172.21.2.111:0/67241873 learned_addr learned my addr 172.21.2.111:0/67241873 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:19.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b71d9b700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b6c03c0f0 0x7f3b6c0e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:19.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 -- 172.21.2.111:0/67241873 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c043210 msgr2=0x7f3b6c0a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:19.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c043210 0x7f3b6c0a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:19.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 -- 172.21.2.111:0/67241873 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b6c03c0f0 msgr2=0x7f3b6c0e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:19.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b6c03c0f0 0x7f3b6c0e6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:19.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 -- 172.21.2.111:0/67241873 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c0a20a0 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b72d9d700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b6c0e6e10 0x7f3b6c0e9ef0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f3b7405aec0 tx=0x7f3b7405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:19.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b6b7fe700 1 -- 172.21.2.111:0/67241873 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7405bda0 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b6b7fe700 1 -- 172.21.2.111:0/67241873 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7405df60 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6c03cea0 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6c0a2a30 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b6b7fe700 1 -- 172.21.2.111:0/67241873 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7405f4b0 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6c0a2230 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b6b7fe700 1 -- 172.21.2.111:0/67241873 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3b7405f610 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b6b7fe700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b58056f10 0x7f3b580593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:19.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b6b7fe700 1 -- 172.21.2.111:0/67241873 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6545+0+0 (secure 0 0 0) 0x7f3b740df030 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7259c700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b58056f10 0x7f3b580593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:19.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.135+0000 7f3b7259c700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b58056f10 0x7f3b580593d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3b6400a000 tx=0x7f3b64008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:19.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.139+0000 7f3b6b7fe700 1 -- 172.21.2.111:0/67241873 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3b740b4280 con 0x7f3b6c0e6e10 2021-04-03T03:15:19.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:19.231+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b6c005200 con 0x7f3b6c0e6e10 2021-04-03T03:15:20.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.067+0000 7f3b6b7fe700 1 -- 172.21.2.111:0/67241873 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 185+0+0 (secure 0 0 0) 0x7f3b740b78a0 con 0x7f3b6c0e6e10 2021-04-03T03:15:20.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.067+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b58056f10 msgr2=0x7f3b580593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:20.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.067+0000 7f3b7359e700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b58056f10 0x7f3b580593d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3b6400a000 tx=0x7f3b64008040).stop 2021-04-03T03:15:20.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.067+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b6c0e6e10 msgr2=0x7f3b6c0e9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:20.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.067+0000 7f3b7359e700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b6c0e6e10 0x7f3b6c0e9ef0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f3b7405aec0 tx=0x7f3b7405ad70).stop 2021-04-03T03:15:20.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.067+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 shutdown_connections 2021-04-03T03:15:20.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.071+0000 7f3b7359e700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b6c0e6e10 0x7f3b6c0e9ef0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:20.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.071+0000 7f3b7359e700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b6c043210 0x7f3b6c0a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:20.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.071+0000 7f3b7359e700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b58056f10 0x7f3b580593d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:20.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.071+0000 7f3b7359e700 1 --2- 172.21.2.111:0/67241873 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b6c03c0f0 0x7f3b6c0e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:20.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.071+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 >> 172.21.2.111:0/67241873 conn(0x7f3b6c00d510 msgr2=0x7f3b6c03a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:20.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.071+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 shutdown_connections 2021-04-03T03:15:20.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.071+0000 7f3b7359e700 1 -- 172.21.2.111:0/67241873 wait complete. 2021-04-03T03:15:20.079 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:15:20.083 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:15:20.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 Processor -- start 2021-04-03T03:15:20.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- start start 2021-04-03T03:15:20.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79d80add50 0x7f79d8147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:20.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80607d0 con 0x7f79d80603f0 2021-04-03T03:15:20.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d814b090 con 0x7f79d8144ef0 2021-04-03T03:15:20.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d814b330 con 0x7f79d80add50 2021-04-03T03:15:20.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d67fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79d80add50 0x7f79d8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:20.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d67fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79d80add50 0x7f79d8147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54316/0 (socket says 172.21.2.111:54316) 2021-04-03T03:15:20.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d67fc700 1 -- 172.21.2.111:0/3357489239 learned_addr learned my addr 172.21.2.111:0/3357489239 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:20.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d67fc700 1 -- 172.21.2.111:0/3357489239 >> v1:172.21.2.115:6789/0 conn(0x7f79d8144ef0 legacy=0x7f79d8147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:20.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d67fc700 1 -- 172.21.2.111:0/3357489239 >> v1:172.21.2.111:6789/0 conn(0x7f79d80603f0 legacy=0x7f79d81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:20.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d67fc700 1 -- 172.21.2.111:0/3357489239 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d814b5d0 con 0x7f79d80add50 2021-04-03T03:15:20.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d67fc700 1 --2- 172.21.2.111:0/3357489239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80add50 0x7f79d8147740 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f79d80ae520 tx=0x7f79c0006e90).ready entity=mon.0 client_cookie=3ecf998ee3975410 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:20.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d5ffb700 1 -- 172.21.2.111:0/3357489239 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79c00098f0 con 0x7f79d80add50 2021-04-03T03:15:20.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d5ffb700 1 -- 172.21.2.111:0/3357489239 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c0009a50 con 0x7f79d80add50 2021-04-03T03:15:20.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d5ffb700 1 -- 172.21.2.111:0/3357489239 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79c0009db0 con 0x7f79d80add50 2021-04-03T03:15:20.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- 172.21.2.111:0/3357489239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80add50 msgr2=0x7f79d8147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:20.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3357489239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80add50 0x7f79d8147740 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f79d80ae520 tx=0x7f79c0006e90).stop 2021-04-03T03:15:20.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- 172.21.2.111:0/3357489239 shutdown_connections 2021-04-03T03:15:20.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3357489239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80add50 0x7f79d8147740 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:20.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- 172.21.2.111:0/3357489239 >> 172.21.2.111:0/3357489239 conn(0x7f79d80b61c0 msgr2=0x7f79d805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:20.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- 172.21.2.111:0/3357489239 shutdown_connections 2021-04-03T03:15:20.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 -- 172.21.2.111:0/3357489239 wait complete. 2021-04-03T03:15:20.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.139+0000 7f79d7fff700 1 Processor -- start 2021-04-03T03:15:20.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 -- start start 2021-04-03T03:15:20.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80603f0 0x7f79d80aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:20.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d80add50 0x7f79d80a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:20.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d8144ef0 0x7f79d815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:20.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80642f0 con 0x7f79d80603f0 2021-04-03T03:15:20.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8064590 con 0x7f79d8144ef0 2021-04-03T03:15:20.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8064830 con 0x7f79d80add50 2021-04-03T03:15:20.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80603f0 0x7f79d80aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:20.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80603f0 0x7f79d80aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54318/0 (socket says 172.21.2.111:54318) 2021-04-03T03:15:20.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 -- 172.21.2.111:0/3580542416 learned_addr learned my addr 172.21.2.111:0/3580542416 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:20.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 -- 172.21.2.111:0/3580542416 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d80add50 msgr2=0x7f79d80a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:20.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d80add50 0x7f79d80a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:20.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 -- 172.21.2.111:0/3580542416 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d8144ef0 msgr2=0x7f79d815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:20.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d8144ef0 0x7f79d815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:20.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 -- 172.21.2.111:0/3580542416 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d8063fe0 con 0x7f79d80603f0 2021-04-03T03:15:20.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d6ffd700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80603f0 0x7f79d80aad70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f79c8001f60 tx=0x7f79c8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:20.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79cffff700 1 -- 172.21.2.111:0/3580542416 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79c800e4a0 con 0x7f79d80603f0 2021-04-03T03:15:20.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79cffff700 1 -- 172.21.2.111:0/3580542416 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c800e600 con 0x7f79d80603f0 2021-04-03T03:15:20.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79cffff700 1 -- 172.21.2.111:0/3580542416 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79c800e960 con 0x7f79d80603f0 2021-04-03T03:15:20.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8062a70 con 0x7f79d80603f0 2021-04-03T03:15:20.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d815a790 con 0x7f79d80603f0 2021-04-03T03:15:20.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d80649f0 con 0x7f79d80603f0 2021-04-03T03:15:20.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79cffff700 1 -- 172.21.2.111:0/3580542416 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f79c8019030 con 0x7f79d80603f0 2021-04-03T03:15:20.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79cffff700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79bc056ef0 0x7f79bc0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:20.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79cffff700 1 -- 172.21.2.111:0/3580542416 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6107+0+0 (secure 0 0 0) 0x7f79c806f580 con 0x7f79d80603f0 2021-04-03T03:15:20.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d67fc700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79bc056ef0 0x7f79bc0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:20.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.143+0000 7f79d67fc700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79bc056ef0 0x7f79bc0593b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f79d80ae520 tx=0x7f79c0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:20.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.147+0000 7f79cffff700 1 -- 172.21.2.111:0/3580542416 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f79c8042c50 con 0x7f79d80603f0 2021-04-03T03:15:20.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:20.239+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79d8057660 con 0x7f79d80603f0 2021-04-03T03:15:21.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.075+0000 7f79cffff700 1 -- 172.21.2.111:0/3580542416 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 173+0+0 (secure 0 0 0) 0x7f79c8048470 con 0x7f79d80603f0 2021-04-03T03:15:21.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79bc056ef0 msgr2=0x7f79bc0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:21.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79bc056ef0 0x7f79bc0593b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f79d80ae520 tx=0x7f79c0015000).stop 2021-04-03T03:15:21.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80603f0 msgr2=0x7f79d80aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:21.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80603f0 0x7f79d80aad70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f79c8001f60 tx=0x7f79c8007c10).stop 2021-04-03T03:15:21.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 shutdown_connections 2021-04-03T03:15:21.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f79bc056ef0 0x7f79bc0593b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:21.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79d8144ef0 0x7f79d815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:21.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79d80603f0 0x7f79d80aad70 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:21.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 --2- 172.21.2.111:0/3580542416 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79d80add50 0x7f79d80a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:21.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 >> 172.21.2.111:0/3580542416 conn(0x7f79d80b61c0 msgr2=0x7f79d81494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:21.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 shutdown_connections 2021-04-03T03:15:21.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.079+0000 7f79d7fff700 1 -- 172.21.2.111:0/3580542416 wait complete. 2021-04-03T03:15:21.091 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:15:21.101 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:15:21.102 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:15:21.102 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:15:21.103 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:15:21.103 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:15:21.103 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:15:21.104 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:15:21.104 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:15:21.108 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:15:21.108 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:15:21.124 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:15:21.124 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:15:21.153 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:15:21.154 DEBUG:teuthology.parallel:result is None 2021-04-03T03:15:21.168 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:15:21.168 DEBUG:teuthology.parallel:result is None 2021-04-03T03:15:21.169 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:15:21.193 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:15:21.208 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:15:21.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab89f5700 1 Processor -- start 2021-04-03T03:15:21.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab89f5700 1 -- start start 2021-04-03T03:15:21.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab89f5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6ab40add50 0x7f6ab4147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab89f5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab40607d0 con 0x7f6ab40603f0 2021-04-03T03:15:21.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab89f5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab414b090 con 0x7f6ab4144ef0 2021-04-03T03:15:21.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab89f5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab414b330 con 0x7f6ab40add50 2021-04-03T03:15:21.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab37fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6ab40603f0 0x7f6ab41447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55132/0 (socket says 172.21.2.111:55132) 2021-04-03T03:15:21.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab37fe700 1 -- 172.21.2.111:0/1056384044 learned_addr learned my addr 172.21.2.111:0/1056384044 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:21.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098581682 0 0) 0x7f6ab40607d0 con 0x7f6ab40603f0 2021-04-03T03:15:21.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a94003240 con 0x7f6ab40603f0 2021-04-03T03:15:21.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3527584085 0 0) 0x7f6a94003240 con 0x7f6ab40603f0 2021-04-03T03:15:21.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 >> v2:172.21.2.115:3300/0 conn(0x7f6ab40add50 msgr2=0x7f6ab4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:21.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 --2- 172.21.2.111:0/1056384044 >> v2:172.21.2.115:3300/0 conn(0x7f6ab40add50 0x7f6ab4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:21.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 >> v1:172.21.2.115:6789/0 conn(0x7f6ab4144ef0 legacy=0x7f6ab4147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:21.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab414b5d0 con 0x7f6ab40603f0 2021-04-03T03:15:21.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f6aa40016d0 con 0x7f6ab40603f0 2021-04-03T03:15:21.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.219+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 >> v1:172.21.2.111:6789/0 conn(0x7f6ab40603f0 legacy=0x7f6ab41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:21.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a94003730 0x7f6a94003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6a94004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a94004e50 0x7f6a9400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab414b330 con 0x7f6a940044e0 2021-04-03T03:15:21.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94004240 con 0x7f6a94003730 2021-04-03T03:15:21.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9400bc20 con 0x7f6a94004e50 2021-04-03T03:15:21.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6a94004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:21.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a94003730 0x7f6a94003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:21.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab3fff700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a94004e50 0x7f6a9400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:21.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 -- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a94004e50 msgr2=0x7f6a9400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:21.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a94004e50 0x7f6a9400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:21.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 -- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a94003730 msgr2=0x7f6a94003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:21.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a94003730 0x7f6a94003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:21.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 -- 172.21.2.111:0/1056384044 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94003240 con 0x7f6a940044e0 2021-04-03T03:15:21.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab3fff700 1 -- 172.21.2.111:0/1056384044 reap_dead start 2021-04-03T03:15:21.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6a94004960 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f6a9c002f30 tx=0x7f6a9c006e10).ready entity=mon.0 client_cookie=9f30f24190203bea server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:21.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9c0097f0 con 0x7f6a940044e0 2021-04-03T03:15:21.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c000e90 con 0x7f6a940044e0 2021-04-03T03:15:21.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab27fc700 1 -- 172.21.2.111:0/1056384044 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9c009b40 con 0x7f6a940044e0 2021-04-03T03:15:21.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/1056384044 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 msgr2=0x7f6a94004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:21.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6a94004960 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f6a9c002f30 tx=0x7f6a9c006e10).stop 2021-04-03T03:15:21.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/1056384044 shutdown_connections 2021-04-03T03:15:21.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/1056384044 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6a94004960 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:21.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/1056384044 >> 172.21.2.111:0/1056384044 conn(0x7f6ab40b61c0 msgr2=0x7f6ab405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:21.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/1056384044 shutdown_connections 2021-04-03T03:15:21.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/1056384044 wait complete. 2021-04-03T03:15:21.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 Processor -- start 2021-04-03T03:15:21.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- start start 2021-04-03T03:15:21.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6ab40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ab41476c0 0x7f6ab415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ab40aa9a0 0x7f6ab40aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4159c70 con 0x7f6a940044e0 2021-04-03T03:15:21.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4159f10 con 0x7f6ab41476c0 2021-04-03T03:15:21.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab415a8a0 con 0x7f6ab40aa9a0 2021-04-03T03:15:21.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6ab40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:21.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6ab40b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54350/0 (socket says 172.21.2.111:54350) 2021-04-03T03:15:21.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 -- 172.21.2.111:0/370885117 learned_addr learned my addr 172.21.2.111:0/370885117 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:21.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab3fff700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ab40aa9a0 0x7f6ab40aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:21.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ab41476c0 0x7f6ab415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:21.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 -- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ab40aa9a0 msgr2=0x7f6ab40aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:21.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ab40aa9a0 0x7f6ab40aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:21.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 -- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ab41476c0 msgr2=0x7f6ab415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:21.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ab41476c0 0x7f6ab415a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:21.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 -- 172.21.2.111:0/370885117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab41472c0 con 0x7f6a940044e0 2021-04-03T03:15:21.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab37fe700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6ab40b1f80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f6aa400b000 tx=0x7f6ab414b220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:21.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab0ff9700 1 -- 172.21.2.111:0/370885117 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa4005430 con 0x7f6a940044e0 2021-04-03T03:15:21.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab0ff9700 1 -- 172.21.2.111:0/370885117 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa400c610 con 0x7f6a940044e0 2021-04-03T03:15:21.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab0ff9700 1 -- 172.21.2.111:0/370885117 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa400c970 con 0x7f6a940044e0 2021-04-03T03:15:21.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab40a9960 con 0x7f6a940044e0 2021-04-03T03:15:21.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab40a9f50 con 0x7f6a940044e0 2021-04-03T03:15:21.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab415a0d0 con 0x7f6a940044e0 2021-04-03T03:15:21.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab0ff9700 1 -- 172.21.2.111:0/370885117 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6aa400cad0 con 0x7f6a940044e0 2021-04-03T03:15:21.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab0ff9700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a98056fb0 0x7f6a98059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:21.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab0ff9700 1 -- 172.21.2.111:0/370885117 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5725+0+0 (secure 0 0 0) 0x7f6aa40702a0 con 0x7f6a940044e0 2021-04-03T03:15:21.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a98056fb0 0x7f6a98059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:21.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.223+0000 7f6ab2ffd700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a98056fb0 0x7f6a98059470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f6a9c019000 tx=0x7f6a9c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:21.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.227+0000 7f6ab0ff9700 1 -- 172.21.2.111:0/370885117 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6aa4045940 con 0x7f6a940044e0 2021-04-03T03:15:21.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:21.331+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6ab4057660 con 0x7f6a940044e0 2021-04-03T03:15:22.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.083+0000 7f6ab0ff9700 1 -- 172.21.2.111:0/370885117 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6aa4048f60 con 0x7f6a940044e0 2021-04-03T03:15:22.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a98056fb0 msgr2=0x7f6a98059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a98056fb0 0x7f6a98059470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f6a9c019000 tx=0x7f6a9c015000).stop 2021-04-03T03:15:22.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 msgr2=0x7f6ab40b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6ab40b1f80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f6aa400b000 tx=0x7f6ab414b220).stop 2021-04-03T03:15:22.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 shutdown_connections 2021-04-03T03:15:22.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ab40aa9a0 0x7f6ab40aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a940044e0 0x7f6ab40b1f80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a98056fb0 0x7f6a98059470 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 --2- 172.21.2.111:0/370885117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ab41476c0 0x7f6ab415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 >> 172.21.2.111:0/370885117 conn(0x7f6ab40b61c0 msgr2=0x7f6ab40ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 shutdown_connections 2021-04-03T03:15:22.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.087+0000 7f6ab89f5700 1 -- 172.21.2.111:0/370885117 wait complete. 2021-04-03T03:15:22.096 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:15:22.100 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:15:22.100 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:15:22.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.219+0000 7fd4230af700 1 Processor -- start 2021-04-03T03:15:22.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.219+0000 7fd4230af700 1 -- start start 2021-04-03T03:15:22.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd41c0603f0 0x7fd41c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd41c0b0c90 0x7fd41c147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c0af400 con 0x7fd41c14b700 2021-04-03T03:15:22.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c060860 con 0x7fd41c0b0c90 2021-04-03T03:15:22.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c0af6a0 con 0x7fd41c0603f0 2021-04-03T03:15:22.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd41c0b0c90 0x7fd41c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd41c0b0c90 0x7fd41c147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54360/0 (socket says 172.21.2.111:54360) 2021-04-03T03:15:22.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 -- 172.21.2.111:0/465176069 learned_addr learned my addr 172.21.2.111:0/465176069 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4220ad700 1 --2- 172.21.2.111:0/465176069 >> v2:172.21.2.115:3300/0 conn(0x7fd41c0603f0 0x7fd41c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 -- 172.21.2.111:0/465176069 >> v2:172.21.2.115:3300/0 conn(0x7fd41c0603f0 msgr2=0x7fd41c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 --2- 172.21.2.111:0/465176069 >> v2:172.21.2.115:3300/0 conn(0x7fd41c0603f0 0x7fd41c0b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 -- 172.21.2.111:0/465176069 >> v1:172.21.2.115:6789/0 conn(0x7fd41c14b700 legacy=0x7fd41c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 -- 172.21.2.111:0/465176069 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0afb40 con 0x7fd41c0b0c90 2021-04-03T03:15:22.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4228ae700 1 --2- 172.21.2.111:0/465176069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c147680 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7fd41c06a3b0 tx=0x7fd414009270).ready entity=mon.0 client_cookie=6c98b9a58b2711fa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4210ab700 1 -- 172.21.2.111:0/465176069 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41400bbc0 con 0x7fd41c0b0c90 2021-04-03T03:15:22.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4210ab700 1 -- 172.21.2.111:0/465176069 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41400bd20 con 0x7fd41c0b0c90 2021-04-03T03:15:22.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4210ab700 1 -- 172.21.2.111:0/465176069 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41400be90 con 0x7fd41c0b0c90 2021-04-03T03:15:22.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- 172.21.2.111:0/465176069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 msgr2=0x7fd41c147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- 172.21.2.111:0/465176069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c147680 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7fd41c06a3b0 tx=0x7fd414009270).stop 2021-04-03T03:15:22.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- 172.21.2.111:0/465176069 shutdown_connections 2021-04-03T03:15:22.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- 172.21.2.111:0/465176069 >> v2:172.21.2.115:3300/0 conn(0x7fd41c0603f0 0x7fd41c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- 172.21.2.111:0/465176069 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c147680 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- 172.21.2.111:0/465176069 >> 172.21.2.111:0/465176069 conn(0x7fd41c0b61c0 msgr2=0x7fd41c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- 172.21.2.111:0/465176069 shutdown_connections 2021-04-03T03:15:22.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- 172.21.2.111:0/465176069 wait complete. 2021-04-03T03:15:22.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 Processor -- start 2021-04-03T03:15:22.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- start start 2021-04-03T03:15:22.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd41c0603f0 0x7fd41c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd41c14b700 0x7fd41c0a7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0a7500 con 0x7fd41c0b0c90 2021-04-03T03:15:22.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0663c0 con 0x7fd41c14b700 2021-04-03T03:15:22.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0677e0 con 0x7fd41c0603f0 2021-04-03T03:15:22.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54368/0 (socket says 172.21.2.111:54368) 2021-04-03T03:15:22.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 -- 172.21.2.111:0/3685377000 learned_addr learned my addr 172.21.2.111:0/3685377000 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 -- 172.21.2.111:0/3685377000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd41c0603f0 msgr2=0x7fd41c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd41c0603f0 0x7fd41c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 -- 172.21.2.111:0/3685377000 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd41c14b700 msgr2=0x7fd41c0a7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd41c14b700 0x7fd41c0a7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 -- 172.21.2.111:0/3685377000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0665e0 con 0x7fd41c0b0c90 2021-04-03T03:15:22.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4218ac700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c159a00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fd418002eb0 tx=0x7fd418006d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd40f7fe700 1 -- 172.21.2.111:0/3685377000 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41800d4a0 con 0x7fd41c0b0c90 2021-04-03T03:15:22.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd40f7fe700 1 -- 172.21.2.111:0/3685377000 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd418000e90 con 0x7fd41c0b0c90 2021-04-03T03:15:22.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c066880 con 0x7fd41c0b0c90 2021-04-03T03:15:22.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd40f7fe700 1 -- 172.21.2.111:0/3685377000 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41800d7f0 con 0x7fd41c0b0c90 2021-04-03T03:15:22.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.223+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c0add00 con 0x7fd41c0b0c90 2021-04-03T03:15:22.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.227+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c0a7670 con 0x7fd41c0b0c90 2021-04-03T03:15:22.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.227+0000 7fd40f7fe700 1 -- 172.21.2.111:0/3685377000 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd41800f030 con 0x7fd41c0b0c90 2021-04-03T03:15:22.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.227+0000 7fd40f7fe700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3fc056f20 0x7fd3fc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.246 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.227+0000 7fd40f7fe700 1 -- 172.21.2.111:0/3685377000 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fd41806fb60 con 0x7fd41c0b0c90 2021-04-03T03:15:22.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.227+0000 7fd4220ad700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3fc056f20 0x7fd3fc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.227+0000 7fd4220ad700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3fc056f20 0x7fd3fc0593e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fd410007000 tx=0x7fd410008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.247 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.227+0000 7fd40f7fe700 1 -- 172.21.2.111:0/3685377000 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd41c0a7670 con 0x7fd41c0b0c90 2021-04-03T03:15:22.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd41c0a7640 con 0x7fd41c0b0c90 2021-04-03T03:15:22.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd40f7fe700 1 -- 172.21.2.111:0/3685377000 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fd418045180 con 0x7fd41c0b0c90 2021-04-03T03:15:22.363 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:22.363 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:15:22.363 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:15:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:15:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:15:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.364 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.365 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.366 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.366 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:15:22.366 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:15:22.366 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.367 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:22.367 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.367 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.367 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.367 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:15:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:15:22.368 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.369 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.369 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.369 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.369 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.369 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.370 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.370 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:15:22.370 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:15:22.370 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.371 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.371 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.371 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.371 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:15:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:15:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.372 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.373 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.373 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.373 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.373 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:15:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:15:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.374 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.375 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.375 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.375 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.375 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.375 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:15:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:15:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.376 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.377 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.377 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.377 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.377 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.377 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:15:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:15:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.378 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.379 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.379 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.379 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.379 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:15:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:15:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.380 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.381 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.381 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.381 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.381 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:15:22.381 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:15:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.382 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.383 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.383 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.383 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:15:22.383 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:15:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:22.384 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:15:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:15:22.385 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.386 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:15:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:15:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.387 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.388 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:15:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:15:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.389 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.390 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:15:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:15:22.391 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:15:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.392 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:15:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:15:22.393 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:22.394 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:22.394 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:15:22.395 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:22.395 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.395 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:22.396 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:22.396 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:15:22.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3fc056f20 msgr2=0x7fd3fc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3fc056f20 0x7fd3fc0593e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fd410007000 tx=0x7fd410008040).stop 2021-04-03T03:15:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 msgr2=0x7fd41c159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c159a00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fd418002eb0 tx=0x7fd418006d10).stop 2021-04-03T03:15:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 shutdown_connections 2021-04-03T03:15:22.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3fc056f20 0x7fd3fc0593e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd41c14b700 0x7fd41c0a7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd41c0603f0 0x7fd41c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 --2- 172.21.2.111:0/3685377000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd41c0b0c90 0x7fd41c159a00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 >> 172.21.2.111:0/3685377000 conn(0x7fd41c0b61c0 msgr2=0x7fd41c146a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 shutdown_connections 2021-04-03T03:15:22.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.355+0000 7fd4230af700 1 -- 172.21.2.111:0/3685377000 wait complete. 2021-04-03T03:15:22.404 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:22.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 Processor -- start 2021-04-03T03:15:22.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- start start 2021-04-03T03:15:22.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fda440603f0 0x7fda440b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 0x7fda440b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440607d0 con 0x7fda440b12b0 2021-04-03T03:15:22.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440af400 con 0x7fda440603f0 2021-04-03T03:15:22.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440af6a0 con 0x7fda4414b700 2021-04-03T03:15:22.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fda440603f0 0x7fda440b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fda440603f0 0x7fda440b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54372/0 (socket says 172.21.2.111:54372) 2021-04-03T03:15:22.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 -- 172.21.2.111:0/1134378019 learned_addr learned my addr 172.21.2.111:0/1134378019 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda425e4700 1 --2- 172.21.2.111:0/1134378019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 0x7fda440b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 -- 172.21.2.111:0/1134378019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 msgr2=0x7fda440b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 --2- 172.21.2.111:0/1134378019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 0x7fda440b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 -- 172.21.2.111:0/1134378019 >> v1:172.21.2.115:6789/0 conn(0x7fda440b12b0 legacy=0x7fda441465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 -- 172.21.2.111:0/1134378019 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440afb40 con 0x7fda440603f0 2021-04-03T03:15:22.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda42de5700 1 --2- 172.21.2.111:0/1134378019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440603f0 0x7fda440b0090 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fda440af200 tx=0x7fda34006e90).ready entity=mon.0 client_cookie=7770a1b5a950a35f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda41de3700 1 -- 172.21.2.111:0/1134378019 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda340098e0 con 0x7fda440603f0 2021-04-03T03:15:22.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda41de3700 1 -- 172.21.2.111:0/1134378019 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda34009a40 con 0x7fda440603f0 2021-04-03T03:15:22.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda41de3700 1 -- 172.21.2.111:0/1134378019 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda34009de0 con 0x7fda440603f0 2021-04-03T03:15:22.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- 172.21.2.111:0/1134378019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440603f0 msgr2=0x7fda440b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 --2- 172.21.2.111:0/1134378019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440603f0 0x7fda440b0090 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fda440af200 tx=0x7fda34006e90).stop 2021-04-03T03:15:22.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- 172.21.2.111:0/1134378019 shutdown_connections 2021-04-03T03:15:22.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 --2- 172.21.2.111:0/1134378019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440603f0 0x7fda440b0090 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 --2- 172.21.2.111:0/1134378019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 0x7fda440b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- 172.21.2.111:0/1134378019 >> 172.21.2.111:0/1134378019 conn(0x7fda440b61c0 msgr2=0x7fda4405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- 172.21.2.111:0/1134378019 shutdown_connections 2021-04-03T03:15:22.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- 172.21.2.111:0/1134378019 wait complete. 2021-04-03T03:15:22.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 Processor -- start 2021-04-03T03:15:22.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.455+0000 7fda43de7700 1 -- start start 2021-04-03T03:15:22.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda440603f0 0x7fda44063da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440b12b0 0x7fda440aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 0x7fda44067670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440ab050 con 0x7fda440b12b0 2021-04-03T03:15:22.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440ab240 con 0x7fda440603f0 2021-04-03T03:15:22.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440ab4e0 con 0x7fda4414b700 2021-04-03T03:15:22.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440b12b0 0x7fda440aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440b12b0 0x7fda440aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54380/0 (socket says 172.21.2.111:54380) 2021-04-03T03:15:22.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 -- 172.21.2.111:0/56366901 learned_addr learned my addr 172.21.2.111:0/56366901 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 -- 172.21.2.111:0/56366901 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 msgr2=0x7fda44067670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 0x7fda44067670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 -- 172.21.2.111:0/56366901 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda440603f0 msgr2=0x7fda44063da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda440603f0 0x7fda44063da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 -- 172.21.2.111:0/56366901 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440a9350 con 0x7fda440b12b0 2021-04-03T03:15:22.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda425e4700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440b12b0 0x7fda440aab60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7fda2c003070 tx=0x7fda2c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda3bfff700 1 -- 172.21.2.111:0/56366901 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda2c009790 con 0x7fda440b12b0 2021-04-03T03:15:22.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda3bfff700 1 -- 172.21.2.111:0/56366901 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c0098f0 con 0x7fda440b12b0 2021-04-03T03:15:22.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda3bfff700 1 -- 172.21.2.111:0/56366901 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda2c009c50 con 0x7fda440b12b0 2021-04-03T03:15:22.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda440a9810 con 0x7fda440b12b0 2021-04-03T03:15:22.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda440aa500 con 0x7fda440b12b0 2021-04-03T03:15:22.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44057660 con 0x7fda440b12b0 2021-04-03T03:15:22.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda3bfff700 1 -- 172.21.2.111:0/56366901 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fda2c013480 con 0x7fda440b12b0 2021-04-03T03:15:22.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda3bfff700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda28056ea0 0x7fda28059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda3bfff700 1 -- 172.21.2.111:0/56366901 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fda2c070580 con 0x7fda440b12b0 2021-04-03T03:15:22.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda42de5700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda28056ea0 0x7fda28059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.459+0000 7fda42de5700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda28056ea0 0x7fda28059360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fda440af200 tx=0x7fda34015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.463+0000 7fda3bfff700 1 -- 172.21.2.111:0/56366901 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fda2c045c40 con 0x7fda440b12b0 2021-04-03T03:15:22.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.551+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda440a9540 con 0x7fda440b12b0 2021-04-03T03:15:22.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda3bfff700 1 -- 172.21.2.111:0/56366901 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13415 (secure 0 0 0) 0x7fda2c009db0 con 0x7fda440b12b0 2021-04-03T03:15:22.561 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:22.561 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":79,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:22.078267+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:22.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda28056ea0 msgr2=0x7fda28059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda28056ea0 0x7fda28059360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fda440af200 tx=0x7fda34015000).stop 2021-04-03T03:15:22.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440b12b0 msgr2=0x7fda440aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440b12b0 0x7fda440aab60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7fda2c003070 tx=0x7fda2c008040).stop 2021-04-03T03:15:22.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 shutdown_connections 2021-04-03T03:15:22.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda4414b700 0x7fda44067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda28056ea0 0x7fda28059360 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda440603f0 0x7fda44063da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 --2- 172.21.2.111:0/56366901 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda440b12b0 0x7fda440aab60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 >> 172.21.2.111:0/56366901 conn(0x7fda440b61c0 msgr2=0x7fda440ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 shutdown_connections 2021-04-03T03:15:22.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.555+0000 7fda43de7700 1 -- 172.21.2.111:0/56366901 wait complete. 2021-04-03T03:15:22.568 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:15:22.569 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:22.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 Processor -- start 2021-04-03T03:15:22.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 -- start start 2021-04-03T03:15:22.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83140603f0 0x7f83141447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f83140add50 0x7f8314144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83140607d0 con 0x7f83140b17b0 2021-04-03T03:15:22.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83141451c0 con 0x7f83140603f0 2021-04-03T03:15:22.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831414b090 con 0x7f83140add50 2021-04-03T03:15:22.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83140603f0 0x7f83141447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83140603f0 0x7f83141447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54386/0 (socket says 172.21.2.111:54386) 2021-04-03T03:15:22.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 -- 172.21.2.111:0/3473693060 learned_addr learned my addr 172.21.2.111:0/3473693060 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 -- 172.21.2.111:0/3473693060 >> v2:172.21.2.115:3300/0 conn(0x7f83140add50 msgr2=0x7f8314144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 --2- 172.21.2.111:0/3473693060 >> v2:172.21.2.115:3300/0 conn(0x7f83140add50 0x7f8314144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 -- 172.21.2.111:0/3473693060 >> v1:172.21.2.115:6789/0 conn(0x7f83140b17b0 legacy=0x7f8314147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 -- 172.21.2.111:0/3473693060 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831414b330 con 0x7f83140603f0 2021-04-03T03:15:22.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8318ac1700 1 --2- 172.21.2.111:0/3473693060 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141447e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f83140ae520 tx=0x7f8304006e90).ready entity=mon.0 client_cookie=837da444cd62dbb8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f83137fe700 1 -- 172.21.2.111:0/3473693060 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8304009930 con 0x7f83140603f0 2021-04-03T03:15:22.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f83137fe700 1 -- 172.21.2.111:0/3473693060 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8304009a90 con 0x7f83140603f0 2021-04-03T03:15:22.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f83137fe700 1 -- 172.21.2.111:0/3473693060 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8304009e30 con 0x7f83140603f0 2021-04-03T03:15:22.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.620+0000 7f8319ac3700 1 -- 172.21.2.111:0/3473693060 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 msgr2=0x7f83141447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 --2- 172.21.2.111:0/3473693060 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141447e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f83140ae520 tx=0x7f8304006e90).stop 2021-04-03T03:15:22.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- 172.21.2.111:0/3473693060 shutdown_connections 2021-04-03T03:15:22.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 --2- 172.21.2.111:0/3473693060 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141447e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 --2- 172.21.2.111:0/3473693060 >> v2:172.21.2.115:3300/0 conn(0x7f83140add50 0x7f8314144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- 172.21.2.111:0/3473693060 >> 172.21.2.111:0/3473693060 conn(0x7f83140b61c0 msgr2=0x7f831405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- 172.21.2.111:0/3473693060 shutdown_connections 2021-04-03T03:15:22.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- 172.21.2.111:0/3473693060 wait complete. 2021-04-03T03:15:22.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 Processor -- start 2021-04-03T03:15:22.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- start start 2021-04-03T03:15:22.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83140add50 0x7f8314141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83140b17b0 0x7f8314141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8314157300 con 0x7f83140603f0 2021-04-03T03:15:22.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8314142490 con 0x7f83140b17b0 2021-04-03T03:15:22.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8314157570 con 0x7f83140add50 2021-04-03T03:15:22.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54392/0 (socket says 172.21.2.111:54392) 2021-04-03T03:15:22.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 -- 172.21.2.111:0/1590613268 learned_addr learned my addr 172.21.2.111:0/1590613268 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 -- 172.21.2.111:0/1590613268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83140add50 msgr2=0x7f8314141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83140add50 0x7f8314141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 -- 172.21.2.111:0/1590613268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83140b17b0 msgr2=0x7f8314141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83140b17b0 0x7f8314141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 -- 172.21.2.111:0/1590613268 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8314157810 con 0x7f83140603f0 2021-04-03T03:15:22.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8318ac1700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141499b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f83140ae520 tx=0x7f8304015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8311ffb700 1 -- 172.21.2.111:0/1590613268 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8304011df0 con 0x7f83140603f0 2021-04-03T03:15:22.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8311ffb700 1 -- 172.21.2.111:0/1590613268 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830401e460 con 0x7f83140603f0 2021-04-03T03:15:22.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8314144410 con 0x7f83140603f0 2021-04-03T03:15:22.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8314147640 con 0x7f83140603f0 2021-04-03T03:15:22.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8311ffb700 1 -- 172.21.2.111:0/1590613268 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f830401e7c0 con 0x7f83140603f0 2021-04-03T03:15:22.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83141462a0 con 0x7f83140603f0 2021-04-03T03:15:22.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8311ffb700 1 -- 172.21.2.111:0/1590613268 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8304025030 con 0x7f83140603f0 2021-04-03T03:15:22.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8311ffb700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82f8056ef0 0x7f82f80593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8311ffb700 1 -- 172.21.2.111:0/1590613268 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f8304071580 con 0x7f83140603f0 2021-04-03T03:15:22.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.624+0000 7f8313fff700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82f8056ef0 0x7f82f80593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.628+0000 7f8313fff700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82f8056ef0 0x7f82f80593b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f82fc002f30 tx=0x7f82fc006a90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.628+0000 7f8311ffb700 1 -- 172.21.2.111:0/1590613268 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f83141462a0 con 0x7f83140603f0 2021-04-03T03:15:22.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.720+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f83141462a0 con 0x7f83140603f0 2021-04-03T03:15:22.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.720+0000 7f8311ffb700 1 -- 172.21.2.111:0/1590613268 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v79) v1 ==== 110+0+10 (secure 0 0 0) 0x7f83141462a0 con 0x7f83140603f0 2021-04-03T03:15:22.727 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:22.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.720+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82f8056ef0 msgr2=0x7f82f80593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.720+0000 7f8319ac3700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82f8056ef0 0x7f82f80593b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f82fc002f30 tx=0x7f82fc006a90).stop 2021-04-03T03:15:22.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.720+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 msgr2=0x7f83141499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.720+0000 7f8319ac3700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141499b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f83140ae520 tx=0x7f8304015000).stop 2021-04-03T03:15:22.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 shutdown_connections 2021-04-03T03:15:22.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f82f8056ef0 0x7f82f80593b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83140b17b0 0x7f8314141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83140603f0 0x7f83141499b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 --2- 172.21.2.111:0/1590613268 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83140add50 0x7f8314141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 >> 172.21.2.111:0/1590613268 conn(0x7f83140b61c0 msgr2=0x7f831414a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 shutdown_connections 2021-04-03T03:15:22.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.724+0000 7f8319ac3700 1 -- 172.21.2.111:0/1590613268 wait complete. 2021-04-03T03:15:22.735 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:22.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 Processor -- start 2021-04-03T03:15:22.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- start start 2021-04-03T03:15:22.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fae440b44a0 0x7fae440b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fae440b5620 0x7fae44142310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440607d0 con 0x7fae440603f0 2021-04-03T03:15:22.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae441433c0 con 0x7fae440b5620 2021-04-03T03:15:22.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae440b3f80 con 0x7fae440b44a0 2021-04-03T03:15:22.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae49d9e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fae440603f0 0x7fae440b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55192/0 (socket says 172.21.2.111:55192) 2021-04-03T03:15:22.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae49d9e700 1 -- 172.21.2.111:0/3435125680 learned_addr learned my addr 172.21.2.111:0/3435125680 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:22.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4a59f700 1 --2- 172.21.2.111:0/3435125680 >> v2:172.21.2.111:3300/0 conn(0x7fae440b5620 0x7fae44142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4959d700 1 --2- 172.21.2.111:0/3435125680 >> v2:172.21.2.115:3300/0 conn(0x7fae440b44a0 0x7fae440b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4a59f700 1 -- 172.21.2.111:0/3435125680 >> v2:172.21.2.115:3300/0 conn(0x7fae440b44a0 msgr2=0x7fae440b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4a59f700 1 --2- 172.21.2.111:0/3435125680 >> v2:172.21.2.115:3300/0 conn(0x7fae440b44a0 0x7fae440b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4a59f700 1 -- 172.21.2.111:0/3435125680 >> v1:172.21.2.111:6789/0 conn(0x7fae440603f0 legacy=0x7fae440b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4a59f700 1 -- 172.21.2.111:0/3435125680 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440b4c60 con 0x7fae440b5620 2021-04-03T03:15:22.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4959d700 1 --2- 172.21.2.111:0/3435125680 >> v2:172.21.2.115:3300/0 conn(0x7fae440b44a0 0x7fae440b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:22.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4a59f700 1 --2- 172.21.2.111:0/3435125680 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440b5620 0x7fae44142310 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fae440b2320 tx=0x7fae3c0091d0).ready entity=mon.0 client_cookie=8bbc471abcd17809 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae48d9c700 1 -- 172.21.2.111:0/3435125680 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae3c00ba10 con 0x7fae440b5620 2021-04-03T03:15:22.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae48d9c700 1 -- 172.21.2.111:0/3435125680 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c00bb70 con 0x7fae440b5620 2021-04-03T03:15:22.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae48d9c700 1 -- 172.21.2.111:0/3435125680 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae3c00bce0 con 0x7fae440b5620 2021-04-03T03:15:22.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- 172.21.2.111:0/3435125680 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440b5620 msgr2=0x7fae44142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 --2- 172.21.2.111:0/3435125680 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440b5620 0x7fae44142310 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fae440b2320 tx=0x7fae3c0091d0).stop 2021-04-03T03:15:22.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- 172.21.2.111:0/3435125680 shutdown_connections 2021-04-03T03:15:22.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 --2- 172.21.2.111:0/3435125680 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440b5620 0x7fae44142310 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 --2- 172.21.2.111:0/3435125680 >> v2:172.21.2.115:3300/0 conn(0x7fae440b44a0 0x7fae440b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- 172.21.2.111:0/3435125680 >> 172.21.2.111:0/3435125680 conn(0x7fae4405c7d0 msgr2=0x7fae4405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- 172.21.2.111:0/3435125680 shutdown_connections 2021-04-03T03:15:22.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.788+0000 7fae4ada0700 1 -- 172.21.2.111:0/3435125680 wait complete. 2021-04-03T03:15:22.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 Processor -- start 2021-04-03T03:15:22.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 -- start start 2021-04-03T03:15:22.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440603f0 0x7fae44148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae440b44a0 0x7fae4413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae440b5620 0x7fae44147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44146400 con 0x7fae440603f0 2021-04-03T03:15:22.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440afbb0 con 0x7fae440b5620 2021-04-03T03:15:22.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae44147c80 con 0x7fae440b44a0 2021-04-03T03:15:22.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440603f0 0x7fae44148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440603f0 0x7fae44148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54406/0 (socket says 172.21.2.111:54406) 2021-04-03T03:15:22.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 -- 172.21.2.111:0/1895697157 learned_addr learned my addr 172.21.2.111:0/1895697157 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 -- 172.21.2.111:0/1895697157 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae440b44a0 msgr2=0x7fae4413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae440b44a0 0x7fae4413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 -- 172.21.2.111:0/1895697157 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae440b5620 msgr2=0x7fae44147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae440b5620 0x7fae44147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 -- 172.21.2.111:0/1895697157 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440af730 con 0x7fae440603f0 2021-04-03T03:15:22.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae49d9e700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440603f0 0x7fae44148d40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7fae44062d70 tx=0x7fae40007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae36ffd700 1 -- 172.21.2.111:0/1895697157 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae40007320 con 0x7fae440603f0 2021-04-03T03:15:22.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae36ffd700 1 -- 172.21.2.111:0/1895697157 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40007480 con 0x7fae440603f0 2021-04-03T03:15:22.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae36ffd700 1 -- 172.21.2.111:0/1895697157 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae400057e0 con 0x7fae440603f0 2021-04-03T03:15:22.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae44149520 con 0x7fae440603f0 2021-04-03T03:15:22.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440b0730 con 0x7fae440603f0 2021-04-03T03:15:22.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4413e7b0 con 0x7fae440603f0 2021-04-03T03:15:22.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae36ffd700 1 -- 172.21.2.111:0/1895697157 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fae40010030 con 0x7fae440603f0 2021-04-03T03:15:22.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae36ffd700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae24057000 0x7fae240594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae36ffd700 1 -- 172.21.2.111:0/1895697157 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fae4006faf0 con 0x7fae440603f0 2021-04-03T03:15:22.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4959d700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae24057000 0x7fae240594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.792+0000 7fae4959d700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae24057000 0x7fae240594c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fae38007000 tx=0x7fae38008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.796+0000 7fae36ffd700 1 -- 172.21.2.111:0/1895697157 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fae4413e7b0 con 0x7fae440603f0 2021-04-03T03:15:22.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae4413e7b0 con 0x7fae440603f0 2021-04-03T03:15:22.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae36ffd700 1 -- 172.21.2.111:0/1895697157 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7fae4413e7b0 con 0x7fae440603f0 2021-04-03T03:15:22.894 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:22.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae24057000 msgr2=0x7fae240594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae24057000 0x7fae240594c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fae38007000 tx=0x7fae38008040).stop 2021-04-03T03:15:22.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440603f0 msgr2=0x7fae44148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440603f0 0x7fae44148d40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7fae44062d70 tx=0x7fae40007c20).stop 2021-04-03T03:15:22.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 shutdown_connections 2021-04-03T03:15:22.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fae24057000 0x7fae240594c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fae440b5620 0x7fae44147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fae440603f0 0x7fae44148d40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 --2- 172.21.2.111:0/1895697157 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fae440b44a0 0x7fae4413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 >> 172.21.2.111:0/1895697157 conn(0x7fae4405c7d0 msgr2=0x7fae440b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 shutdown_connections 2021-04-03T03:15:22.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.888+0000 7fae4ada0700 1 -- 172.21.2.111:0/1895697157 wait complete. 2021-04-03T03:15:22.901 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:15:22.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b2fe0700 1 Processor -- start 2021-04-03T03:15:22.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b2fe0700 1 -- start start 2021-04-03T03:15:22.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b2fe0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f26a40dbb90 0x7f26a4042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b2fe0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a40dc6e0 con 0x7f26a403d360 2021-04-03T03:15:22.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b2fe0700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a4042010 con 0x7f26a4006bc0 2021-04-03T03:15:22.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b2fe0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a40d9560 con 0x7f26a40dbb90 2021-04-03T03:15:22.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b17dd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f26a403d360 0x7f26a40064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55208/0 (socket says 172.21.2.111:55208) 2021-04-03T03:15:22.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b17dd700 1 -- 172.21.2.111:0/3000461450 learned_addr learned my addr 172.21.2.111:0/3000461450 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:22.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471623928 0 0) 0x7f26a40dc6e0 con 0x7f26a403d360 2021-04-03T03:15:22.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2694003240 con 0x7f26a403d360 2021-04-03T03:15:22.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b1fde700 1 --2- 172.21.2.111:0/3000461450 >> v2:172.21.2.115:3300/0 conn(0x7f26a40dbb90 0x7f26a4042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2120659627 0 0) 0x7f2694003240 con 0x7f26a403d360 2021-04-03T03:15:22.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 >> v2:172.21.2.115:3300/0 conn(0x7f26a40dbb90 msgr2=0x7f26a4042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 --2- 172.21.2.111:0/3000461450 >> v2:172.21.2.115:3300/0 conn(0x7f26a40dbb90 0x7f26a4042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 >> v1:172.21.2.115:6789/0 conn(0x7f26a4006bc0 legacy=0x7f26a40056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40422b0 con 0x7f26a403d360 2021-04-03T03:15:22.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f26ac0450f0 con 0x7f26a403d360 2021-04-03T03:15:22.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b1fde700 1 --2- 172.21.2.111:0/3000461450 >> v2:172.21.2.115:3300/0 conn(0x7f26a40dbb90 0x7f26a4042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:22.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 >> v1:172.21.2.111:6789/0 conn(0x7f26a403d360 legacy=0x7f26a40064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940037b0 0x7f2694003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26940044e0 0x7f2694004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2694004e50 0x7f269400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40d9560 con 0x7f26940044e0 2021-04-03T03:15:22.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26940041d0 con 0x7f26940037b0 2021-04-03T03:15:22.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.952+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269400bc20 con 0x7f2694004e50 2021-04-03T03:15:22.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26940044e0 0x7f2694004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b1fde700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940037b0 0x7f2694003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 -- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2694004e50 msgr2=0x7f269400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2694004e50 0x7f269400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 -- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940037b0 msgr2=0x7f2694003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:22.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940037b0 0x7f2694003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 -- 172.21.2.111:0/3000461450 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694003240 con 0x7f26940044e0 2021-04-03T03:15:22.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b1fde700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940037b0 0x7f2694003bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:22.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b27df700 1 -- 172.21.2.111:0/3000461450 reap_dead start 2021-04-03T03:15:22.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26940044e0 0x7f2694004960 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f26a40087e0 tx=0x7f26a403ec80).ready entity=mon.0 client_cookie=216586599b1947ed server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26ac0699f0 con 0x7f26940044e0 2021-04-03T03:15:22.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac05b040 con 0x7f26940044e0 2021-04-03T03:15:22.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b0fdc700 1 -- 172.21.2.111:0/3000461450 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26ac061300 con 0x7f26940044e0 2021-04-03T03:15:22.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/3000461450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26940044e0 msgr2=0x7f2694004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26940044e0 0x7f2694004960 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f26a40087e0 tx=0x7f26a403ec80).stop 2021-04-03T03:15:22.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/3000461450 shutdown_connections 2021-04-03T03:15:22.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/3000461450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26940044e0 0x7f2694004960 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:22.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/3000461450 >> 172.21.2.111:0/3000461450 conn(0x7f26a40462c0 msgr2=0x7f26a40466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:22.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/3000461450 shutdown_connections 2021-04-03T03:15:22.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/3000461450 wait complete. 2021-04-03T03:15:22.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 Processor -- start 2021-04-03T03:15:22.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- start start 2021-04-03T03:15:22.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940044e0 0x7f26a409fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26a4039cb0 0x7f26a40a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26a40a31b0 0x7f26a40e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a403b750 con 0x7f26a4039cb0 2021-04-03T03:15:22.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40a0690 con 0x7f26940044e0 2021-04-03T03:15:22.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40a3640 con 0x7f26a40a31b0 2021-04-03T03:15:22.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26a4039cb0 0x7f26a40a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26a4039cb0 0x7f26a40a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54430/0 (socket says 172.21.2.111:54430) 2021-04-03T03:15:22.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 -- 172.21.2.111:0/171604980 learned_addr learned my addr 172.21.2.111:0/171604980 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:22.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 -- 172.21.2.111:0/171604980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26a40a31b0 msgr2=0x7f26a40e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26a40a31b0 0x7f26a40e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 -- 172.21.2.111:0/171604980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940044e0 msgr2=0x7f26a409fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:22.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940044e0 0x7f26a409fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:22.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 -- 172.21.2.111:0/171604980 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40a03f0 con 0x7f26a4039cb0 2021-04-03T03:15:22.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b17dd700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26a4039cb0 0x7f26a40a2cc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f26a40087e0 tx=0x7f26ac085000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f269affd700 1 -- 172.21.2.111:0/171604980 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26ac08c040 con 0x7f26a4039cb0 2021-04-03T03:15:22.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f269affd700 1 -- 172.21.2.111:0/171604980 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac061260 con 0x7f26a4039cb0 2021-04-03T03:15:22.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f269affd700 1 -- 172.21.2.111:0/171604980 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26ac06a4c0 con 0x7f26a4039cb0 2021-04-03T03:15:22.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a40a00e0 con 0x7f26a4039cb0 2021-04-03T03:15:22.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a403bbb0 con 0x7f26a4039cb0 2021-04-03T03:15:22.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.956+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a4004da0 con 0x7f26a4039cb0 2021-04-03T03:15:22.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.960+0000 7f269affd700 1 -- 172.21.2.111:0/171604980 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f26ac06a620 con 0x7f26a4039cb0 2021-04-03T03:15:22.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.960+0000 7f269affd700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f268c056fb0 0x7f268c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:22.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.960+0000 7f269affd700 1 -- 172.21.2.111:0/171604980 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f26ac0dfe90 con 0x7f26a4039cb0 2021-04-03T03:15:22.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.960+0000 7f26b1fde700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f268c056fb0 0x7f268c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:22.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.960+0000 7f26b1fde700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f268c056fb0 0x7f268c059470 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f269c008000 tx=0x7f269c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:22.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:22.960+0000 7f269affd700 1 -- 172.21.2.111:0/171604980 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f26ac0b55b0 con 0x7f26a4039cb0 2021-04-03T03:15:23.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f26a4046ff0 con 0x7f26a4039cb0 2021-04-03T03:15:23.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f269affd700 1 -- 172.21.2.111:0/171604980 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+143 (secure 0 0 0) 0x7f26ac0b8bd0 con 0x7f26a4039cb0 2021-04-03T03:15:23.063 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:23.064 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:15:23.064 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:23.064 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:15:23.064 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:15:23.064 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:23.065 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:23.065 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:15:23.065 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:15:23.065 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:23.066 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:15:23.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f268c056fb0 msgr2=0x7f268c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:23.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f268c056fb0 0x7f268c059470 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f269c008000 tx=0x7f269c008040).stop 2021-04-03T03:15:23.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26a4039cb0 msgr2=0x7f26a40a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:23.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26a4039cb0 0x7f26a40a2cc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f26a40087e0 tx=0x7f26ac085000).stop 2021-04-03T03:15:23.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 shutdown_connections 2021-04-03T03:15:23.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26a40a31b0 0x7f26a40e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:23.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f268c056fb0 0x7f268c059470 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:23.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26940044e0 0x7f26a409fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:23.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 --2- 172.21.2.111:0/171604980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26a4039cb0 0x7f26a40a2cc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:23.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 >> 172.21.2.111:0/171604980 conn(0x7f26a40462c0 msgr2=0x7f26a4045260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:23.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.056+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 shutdown_connections 2021-04-03T03:15:23.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.060+0000 7f26b2fe0700 1 -- 172.21.2.111:0/171604980 wait complete. 2021-04-03T03:15:23.071 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:15:23.071 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:15:23.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 Processor -- start 2021-04-03T03:15:23.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- start start 2021-04-03T03:15:23.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1080603f0 0x7fe108144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:23.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe1080b17f0 0x7fe108144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:23.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1080b0be0 con 0x7fe1080b0290 2021-04-03T03:15:23.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080607d0 con 0x7fe1080603f0 2021-04-03T03:15:23.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080b0e10 con 0x7fe1080b17f0 2021-04-03T03:15:23.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1080603f0 0x7fe108144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:23.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1080603f0 0x7fe108144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54434/0 (socket says 172.21.2.111:54434) 2021-04-03T03:15:23.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 -- 172.21.2.111:0/835390808 learned_addr learned my addr 172.21.2.111:0/835390808 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:23.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe105d9b700 1 --2- 172.21.2.111:0/835390808 >> v2:172.21.2.115:3300/0 conn(0x7fe1080b17f0 0x7fe108144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:23.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 -- 172.21.2.111:0/835390808 >> v2:172.21.2.115:3300/0 conn(0x7fe1080b17f0 msgr2=0x7fe108144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:23.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 --2- 172.21.2.111:0/835390808 >> v2:172.21.2.115:3300/0 conn(0x7fe1080b17f0 0x7fe108144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:23.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 -- 172.21.2.111:0/835390808 >> v1:172.21.2.115:6789/0 conn(0x7fe1080b0290 legacy=0x7fe1080aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:23.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 -- 172.21.2.111:0/835390808 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080b10b0 con 0x7fe1080603f0 2021-04-03T03:15:23.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe105d9b700 1 --2- 172.21.2.111:0/835390808 >> v2:172.21.2.115:3300/0 conn(0x7fe1080b17f0 0x7fe108144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:23.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10659c700 1 --2- 172.21.2.111:0/835390808 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080603f0 0x7fe108144700 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fe0f0002f30 tx=0x7fe0f0006e90).ready entity=mon.0 client_cookie=19e7ad970acccedf server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:23.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10559a700 1 -- 172.21.2.111:0/835390808 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0f00099c0 con 0x7fe1080603f0 2021-04-03T03:15:23.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10559a700 1 -- 172.21.2.111:0/835390808 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0009b20 con 0x7fe1080603f0 2021-04-03T03:15:23.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10559a700 1 -- 172.21.2.111:0/835390808 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0f0009e80 con 0x7fe1080603f0 2021-04-03T03:15:23.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- 172.21.2.111:0/835390808 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080603f0 msgr2=0x7fe108144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:23.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 --2- 172.21.2.111:0/835390808 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080603f0 0x7fe108144700 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fe0f0002f30 tx=0x7fe0f0006e90).stop 2021-04-03T03:15:23.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- 172.21.2.111:0/835390808 shutdown_connections 2021-04-03T03:15:23.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 --2- 172.21.2.111:0/835390808 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080603f0 0x7fe108144700 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:23.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 --2- 172.21.2.111:0/835390808 >> v2:172.21.2.115:3300/0 conn(0x7fe1080b17f0 0x7fe108144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:23.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- 172.21.2.111:0/835390808 >> 172.21.2.111:0/835390808 conn(0x7fe1080b61c0 msgr2=0x7fe10805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:23.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- 172.21.2.111:0/835390808 shutdown_connections 2021-04-03T03:15:23.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.124+0000 7fe10759e700 1 -- 172.21.2.111:0/835390808 wait complete. 2021-04-03T03:15:23.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 Processor -- start 2021-04-03T03:15:23.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 -- start start 2021-04-03T03:15:23.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1080603f0 0x7fe108143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:23.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1080b0290 0x7fe108156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:23.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080b17f0 0x7fe108065200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:23.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080ad660 con 0x7fe1080b17f0 2021-04-03T03:15:23.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108066cd0 con 0x7fe1080b0290 2021-04-03T03:15:23.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe108155a50 con 0x7fe1080603f0 2021-04-03T03:15:23.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080b17f0 0x7fe108065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:23.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080b17f0 0x7fe108065200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54444/0 (socket says 172.21.2.111:54444) 2021-04-03T03:15:23.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe105d9b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1080b0290 0x7fe108156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:23.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 -- 172.21.2.111:0/3945273733 learned_addr learned my addr 172.21.2.111:0/3945273733 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:23.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10659c700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1080603f0 0x7fe108143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:23.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 -- 172.21.2.111:0/3945273733 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1080603f0 msgr2=0x7fe108143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:23.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1080603f0 0x7fe108143d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:23.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 -- 172.21.2.111:0/3945273733 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1080b0290 msgr2=0x7fe108156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:23.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1080b0290 0x7fe108156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:23.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 -- 172.21.2.111:0/3945273733 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe108155cf0 con 0x7fe1080b17f0 2021-04-03T03:15:23.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe106d9d700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080b17f0 0x7fe108065200 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fe1080ad7a0 tx=0x7fe100007ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:23.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe0ff7fe700 1 -- 172.21.2.111:0/3945273733 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000ce50 con 0x7fe1080b17f0 2021-04-03T03:15:23.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe0ff7fe700 1 -- 172.21.2.111:0/3945273733 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10000a770 con 0x7fe1080b17f0 2021-04-03T03:15:23.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe108066950 con 0x7fe1080b17f0 2021-04-03T03:15:23.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe0ff7fe700 1 -- 172.21.2.111:0/3945273733 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000aad0 con 0x7fe1080b17f0 2021-04-03T03:15:23.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10813f7a0 con 0x7fe1080b17f0 2021-04-03T03:15:23.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108155e80 con 0x7fe1080b17f0 2021-04-03T03:15:23.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe0ff7fe700 1 -- 172.21.2.111:0/3945273733 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe100019030 con 0x7fe1080b17f0 2021-04-03T03:15:23.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe0ff7fe700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe0ec056fb0 0x7fe0ec059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:23.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe0ff7fe700 1 -- 172.21.2.111:0/3945273733 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fe10006f6e0 con 0x7fe1080b17f0 2021-04-03T03:15:23.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10659c700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe0ec056fb0 0x7fe0ec059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:23.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.128+0000 7fe10659c700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe0ec056fb0 0x7fe0ec059470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fe0f001a000 tx=0x7fe0f0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:23.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.132+0000 7fe0ff7fe700 1 -- 172.21.2.111:0/3945273733 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe100044e00 con 0x7fe1080b17f0 2021-04-03T03:15:23.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:23.224+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fe108066ae0 con 0x7fe1080b17f0 2021-04-03T03:15:24.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.104+0000 7fe0ff7fe700 1 -- 172.21.2.111:0/3945273733 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v80) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe100048420 con 0x7fe1080b17f0 2021-04-03T03:15:24.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe0ec056fb0 msgr2=0x7fe0ec059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:24.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe0ec056fb0 0x7fe0ec059470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fe0f001a000 tx=0x7fe0f0015000).stop 2021-04-03T03:15:24.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080b17f0 msgr2=0x7fe108065200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:24.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080b17f0 0x7fe108065200 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fe1080ad7a0 tx=0x7fe100007ec0).stop 2021-04-03T03:15:24.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 shutdown_connections 2021-04-03T03:15:24.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1080b17f0 0x7fe108065200 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:24.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1080603f0 0x7fe108143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:24.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe0ec056fb0 0x7fe0ec059470 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:24.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 --2- 172.21.2.111:0/3945273733 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1080b0290 0x7fe108156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:24.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 >> 172.21.2.111:0/3945273733 conn(0x7fe1080b61c0 msgr2=0x7fe10813b500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:24.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 shutdown_connections 2021-04-03T03:15:24.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.108+0000 7fe10759e700 1 -- 172.21.2.111:0/3945273733 wait complete. 2021-04-03T03:15:24.118 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:15:24.121 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:15:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2f59e700 1 Processor -- start 2021-04-03T03:15:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2f59e700 1 -- start start 2021-04-03T03:15:24.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda28041f80 0x7fda28042360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280411e0 con 0x7fda2803d860 2021-04-03T03:15:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280d4b80 con 0x7fda2803ccc0 2021-04-03T03:15:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2803d130 con 0x7fda28041f80 2021-04-03T03:15:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2dd9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fda2803d860 0x7fda2803c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55244/0 (socket says 172.21.2.111:55244) 2021-04-03T03:15:24.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2dd9b700 1 -- 172.21.2.111:0/255582641 learned_addr learned my addr 172.21.2.111:0/255582641 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596384181 0 0) 0x7fda280411e0 con 0x7fda2803d860 2021-04-03T03:15:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda10003240 con 0x7fda2803d860 2021-04-03T03:15:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2e59c700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda28041f80 0x7fda28042360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:24.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1500728428 0 0) 0x7fda10003240 con 0x7fda2803d860 2021-04-03T03:15:24.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda28041f80 msgr2=0x7fda28042360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:24.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2d59a700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda28041f80 0x7fda28042360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:24.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 >> v1:172.21.2.115:6789/0 conn(0x7fda2803ccc0 legacy=0x7fda2803eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:24.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.172+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28041480 con 0x7fda2803d860 2021-04-03T03:15:24.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fda200016d0 con 0x7fda2803d860 2021-04-03T03:15:24.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 >> v1:172.21.2.111:6789/0 conn(0x7fda2803d860 legacy=0x7fda2803c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:24.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda28041f80 0x7fda10003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda10003c70 0x7fda100044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda100049d0 0x7fda10004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2803d130 con 0x7fda10003c70 2021-04-03T03:15:24.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda10004200 con 0x7fda100049d0 2021-04-03T03:15:24.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1000b730 con 0x7fda28041f80 2021-04-03T03:15:24.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda10003c70 0x7fda100044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:24.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 -- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda28041f80 msgr2=0x7fda10003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:24.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda28041f80 0x7fda10003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:24.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 -- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda100049d0 msgr2=0x7fda10004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:24.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda100049d0 0x7fda10004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:24.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 -- 172.21.2.111:0/255582641 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda10003240 con 0x7fda10003c70 2021-04-03T03:15:24.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda10003c70 0x7fda100044e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fda200013e0 tx=0x7fda20007780).ready entity=mon.0 client_cookie=f8f2bd792d23c31e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:24.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda200070d0 con 0x7fda10003c70 2021-04-03T03:15:24.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20007230 con 0x7fda10003c70 2021-04-03T03:15:24.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2d59a700 1 -- 172.21.2.111:0/255582641 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda200055f0 con 0x7fda10003c70 2021-04-03T03:15:24.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/255582641 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda10003c70 msgr2=0x7fda100044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:24.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 --2- 172.21.2.111:0/255582641 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda10003c70 0x7fda100044e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fda200013e0 tx=0x7fda20007780).stop 2021-04-03T03:15:24.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2ed9d700 1 -- 172.21.2.111:0/255582641 reap_dead start 2021-04-03T03:15:24.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/255582641 shutdown_connections 2021-04-03T03:15:24.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/255582641 >> 172.21.2.111:0/255582641 conn(0x7fda280462b0 msgr2=0x7fda280466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:24.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/255582641 shutdown_connections 2021-04-03T03:15:24.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/255582641 wait complete. 2021-04-03T03:15:24.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 Processor -- start 2021-04-03T03:15:24.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- start start 2021-04-03T03:15:24.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda280da750 0x7fda280d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda280d9230 0x7fda280e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda280d6440 0x7fda280d68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280daf50 con 0x7fda280da750 2021-04-03T03:15:24.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280dab30 con 0x7fda280d9230 2021-04-03T03:15:24.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda280d6db0 con 0x7fda280d6440 2021-04-03T03:15:24.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda280da750 0x7fda280d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:24.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda280da750 0x7fda280d9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54464/0 (socket says 172.21.2.111:54464) 2021-04-03T03:15:24.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 -- 172.21.2.111:0/476286105 learned_addr learned my addr 172.21.2.111:0/476286105 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:24.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda280d9230 0x7fda280e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:24.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2ed9d700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda280d6440 0x7fda280d68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:24.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 -- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda280d6440 msgr2=0x7fda280d68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:24.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda280d6440 0x7fda280d68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:24.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 -- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda280d9230 msgr2=0x7fda280e3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:24.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda280d9230 0x7fda280e3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:24.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 -- 172.21.2.111:0/476286105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280d8a30 con 0x7fda280da750 2021-04-03T03:15:24.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2e59c700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda280da750 0x7fda280d9ea0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fda1800a000 tx=0x7fda18008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:24.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda277fe700 1 -- 172.21.2.111:0/476286105 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda18017070 con 0x7fda280da750 2021-04-03T03:15:24.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda277fe700 1 -- 172.21.2.111:0/476286105 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda180127e0 con 0x7fda280da750 2021-04-03T03:15:24.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda277fe700 1 -- 172.21.2.111:0/476286105 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda18012b40 con 0x7fda280da750 2021-04-03T03:15:24.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda280d8cd0 con 0x7fda280da750 2021-04-03T03:15:24.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda280e60b0 con 0x7fda280da750 2021-04-03T03:15:24.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda28004db0 con 0x7fda280da750 2021-04-03T03:15:24.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda277fe700 1 -- 172.21.2.111:0/476286105 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fda18012ca0 con 0x7fda280da750 2021-04-03T03:15:24.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda277fe700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda14056ec0 0x7fda14059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:24.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda277fe700 1 -- 172.21.2.111:0/476286105 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5950+0+0 (secure 0 0 0) 0x7fda18070480 con 0x7fda280da750 2021-04-03T03:15:24.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.176+0000 7fda2dd9b700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda14056ec0 0x7fda14059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:24.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.180+0000 7fda2dd9b700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda14056ec0 0x7fda14059380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fda20017000 tx=0x7fda20015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:24.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.180+0000 7fda277fe700 1 -- 172.21.2.111:0/476286105 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fda18045a40 con 0x7fda280da750 2021-04-03T03:15:24.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:24.272+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fda280e35c0 con 0x7fda280da750 2021-04-03T03:15:25.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.116+0000 7fda277fe700 1 -- 172.21.2.111:0/476286105 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v81) v1 ==== 125+0+0 (secure 0 0 0) 0x7fda18049060 con 0x7fda280da750 2021-04-03T03:15:25.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda14056ec0 msgr2=0x7fda14059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda14056ec0 0x7fda14059380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fda20017000 tx=0x7fda20015000).stop 2021-04-03T03:15:25.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda280da750 msgr2=0x7fda280d9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda280da750 0x7fda280d9ea0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fda1800a000 tx=0x7fda18008040).stop 2021-04-03T03:15:25.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 shutdown_connections 2021-04-03T03:15:25.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fda280d6440 0x7fda280d68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fda280da750 0x7fda280d9ea0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fda14056ec0 0x7fda14059380 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 --2- 172.21.2.111:0/476286105 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fda280d9230 0x7fda280e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 >> 172.21.2.111:0/476286105 conn(0x7fda280462b0 msgr2=0x7fda28044fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 shutdown_connections 2021-04-03T03:15:25.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.120+0000 7fda2f59e700 1 -- 172.21.2.111:0/476286105 wait complete. 2021-04-03T03:15:25.128 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:15:25.133 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:15:25.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 Processor -- start 2021-04-03T03:15:25.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- start start 2021-04-03T03:15:25.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f2df003f880 0x7f2df00d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2df0039dd0 0x7f2df003fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df00d1c70 con 0x7f2df00ceed0 2021-04-03T03:15:25.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df0040150 con 0x7f2df0039dd0 2021-04-03T03:15:25.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df00d2f00 con 0x7f2df003f880 2021-04-03T03:15:25.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2df0039dd0 0x7f2df003fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2df0039dd0 0x7f2df003fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54470/0 (socket says 172.21.2.111:54470) 2021-04-03T03:15:25.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 -- 172.21.2.111:0/4055655287 learned_addr learned my addr 172.21.2.111:0/4055655287 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:25.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 -- 172.21.2.111:0/4055655287 >> v2:172.21.2.115:3300/0 conn(0x7f2df003f880 msgr2=0x7f2df00d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 --2- 172.21.2.111:0/4055655287 >> v2:172.21.2.115:3300/0 conn(0x7f2df003f880 0x7f2df00d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 -- 172.21.2.111:0/4055655287 >> v1:172.21.2.115:6789/0 conn(0x7f2df00ceed0 legacy=0x7f2df0005680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 -- 172.21.2.111:0/4055655287 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00d1ea0 con 0x7f2df0039dd0 2021-04-03T03:15:25.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df6ffd700 1 --2- 172.21.2.111:0/4055655287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df0039dd0 0x7f2df003fc60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f2df000c350 tx=0x7f2de8006e90).ready entity=mon.0 client_cookie=da02d4cef7bae0f3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df67fc700 1 -- 172.21.2.111:0/4055655287 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2de8009930 con 0x7f2df0039dd0 2021-04-03T03:15:25.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df67fc700 1 -- 172.21.2.111:0/4055655287 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8009a90 con 0x7f2df0039dd0 2021-04-03T03:15:25.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df67fc700 1 -- 172.21.2.111:0/4055655287 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2de8009e30 con 0x7f2df0039dd0 2021-04-03T03:15:25.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/4055655287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df0039dd0 msgr2=0x7f2df003fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/4055655287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df0039dd0 0x7f2df003fc60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f2df000c350 tx=0x7f2de8006e90).stop 2021-04-03T03:15:25.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/4055655287 shutdown_connections 2021-04-03T03:15:25.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/4055655287 >> v2:172.21.2.115:3300/0 conn(0x7f2df003f880 0x7f2df00d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/4055655287 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df0039dd0 0x7f2df003fc60 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/4055655287 >> 172.21.2.111:0/4055655287 conn(0x7f2df00462b0 msgr2=0x7f2df00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/4055655287 shutdown_connections 2021-04-03T03:15:25.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/4055655287 wait complete. 2021-04-03T03:15:25.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 Processor -- start 2021-04-03T03:15:25.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- start start 2021-04-03T03:15:25.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2df0039dd0 0x7f2df00ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2df003f880 0x7f2df00bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df00ceed0 0x7f2df00e2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df003d370 con 0x7f2df00ceed0 2021-04-03T03:15:25.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00bd120 con 0x7f2df003f880 2021-04-03T03:15:25.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00e2880 con 0x7f2df0039dd0 2021-04-03T03:15:25.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df00ceed0 0x7f2df00e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df00ceed0 0x7f2df00e2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54476/0 (socket says 172.21.2.111:54476) 2021-04-03T03:15:25.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 -- 172.21.2.111:0/1671517712 learned_addr learned my addr 172.21.2.111:0/1671517712 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:25.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 -- 172.21.2.111:0/1671517712 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2df0039dd0 msgr2=0x7f2df00ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2df0039dd0 0x7f2df00ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 -- 172.21.2.111:0/1671517712 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2df003f880 msgr2=0x7f2df00bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2df003f880 0x7f2df00bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 -- 172.21.2.111:0/1671517712 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df003cff0 con 0x7f2df00ceed0 2021-04-03T03:15:25.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df7fff700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df00ceed0 0x7f2df00e2390 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f2df806b5d0 tx=0x7f2df805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df4ff9700 1 -- 172.21.2.111:0/1671517712 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2df805df60 con 0x7f2df00ceed0 2021-04-03T03:15:25.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df4ff9700 1 -- 172.21.2.111:0/1671517712 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df80699f0 con 0x7f2df00ceed0 2021-04-03T03:15:25.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df003cbf0 con 0x7f2df00ceed0 2021-04-03T03:15:25.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2df4ff9700 1 -- 172.21.2.111:0/1671517712 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2df805f4b0 con 0x7f2df00ceed0 2021-04-03T03:15:25.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.188+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df00bd750 con 0x7f2df00ceed0 2021-04-03T03:15:25.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.192+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df00bd2e0 con 0x7f2df00ceed0 2021-04-03T03:15:25.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.192+0000 7f2df4ff9700 1 -- 172.21.2.111:0/1671517712 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2df808b020 con 0x7f2df00ceed0 2021-04-03T03:15:25.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.192+0000 7f2df4ff9700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2dd8056fb0 0x7f2dd8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.192+0000 7f2df4ff9700 1 -- 172.21.2.111:0/1671517712 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f2df80defb0 con 0x7f2df00ceed0 2021-04-03T03:15:25.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.192+0000 7f2df77fe700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2dd8056fb0 0x7f2dd8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.192+0000 7f2df77fe700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2dd8056fb0 0x7f2dd8059470 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f2dec002f30 tx=0x7f2dec006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.192+0000 7f2df4ff9700 1 -- 172.21.2.111:0/1671517712 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2df80b4410 con 0x7f2df00ceed0 2021-04-03T03:15:25.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.324+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f2df00bcf50 con 0x7f2df00ceed0 2021-04-03T03:15:25.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2df4ff9700 1 -- 172.21.2.111:0/1671517712 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v32) v1 ==== 189+0+0 (secure 0 0 0) 0x7f2df80b7a30 con 0x7f2df00ceed0 2021-04-03T03:15:25.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2dd8056fb0 msgr2=0x7f2dd8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2dd8056fb0 0x7f2dd8059470 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f2dec002f30 tx=0x7f2dec006a60).stop 2021-04-03T03:15:25.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df00ceed0 msgr2=0x7f2df00e2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df00ceed0 0x7f2df00e2390 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f2df806b5d0 tx=0x7f2df805ad70).stop 2021-04-03T03:15:25.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 shutdown_connections 2021-04-03T03:15:25.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2df00ceed0 0x7f2df00e2390 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2df0039dd0 0x7f2df00ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2dd8056fb0 0x7f2dd8059470 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 --2- 172.21.2.111:0/1671517712 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2df003f880 0x7f2df00bbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 >> 172.21.2.111:0/1671517712 conn(0x7f2df00462b0 msgr2=0x7f2df00cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 shutdown_connections 2021-04-03T03:15:25.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.328+0000 7f2dfcaeb700 1 -- 172.21.2.111:0/1671517712 wait complete. 2021-04-03T03:15:25.338 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 15 and data pool 16 2021-04-03T03:15:25.342 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:25.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.392+0000 7f614928b700 1 Processor -- start 2021-04-03T03:15:25.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.392+0000 7f614928b700 1 -- start start 2021-04-03T03:15:25.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.392+0000 7f614928b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f614414b700 0x7f61440b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.392+0000 7f614928b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440b0c90 0x7f6144147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.392+0000 7f614928b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440607d0 con 0x7f61440603f0 2021-04-03T03:15:25.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.392+0000 7f614928b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61440af400 con 0x7f614414b700 2021-04-03T03:15:25.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.392+0000 7f614928b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61440af6a0 con 0x7f61440b0c90 2021-04-03T03:15:25.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f61440603f0 0x7f61440b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55288/0 (socket says 172.21.2.111:55288) 2021-04-03T03:15:25.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 -- 172.21.2.111:0/3509426636 learned_addr learned my addr 172.21.2.111:0/3509426636 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:25.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f61437fe700 1 --2- 172.21.2.111:0/3509426636 >> v2:172.21.2.115:3300/0 conn(0x7f614414b700 0x7f61440b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440b0c90 0x7f6144147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817139225 0 0) 0x7f61440607d0 con 0x7f61440603f0 2021-04-03T03:15:25.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6124003240 con 0x7f61440603f0 2021-04-03T03:15:25.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1624470961 0 0) 0x7f6124003240 con 0x7f61440603f0 2021-04-03T03:15:25.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440b0c90 msgr2=0x7f6144147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440b0c90 0x7f6144147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 >> v2:172.21.2.115:3300/0 conn(0x7f614414b700 msgr2=0x7f61440b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 --2- 172.21.2.111:0/3509426636 >> v2:172.21.2.115:3300/0 conn(0x7f614414b700 0x7f61440b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440afb40 con 0x7f61440603f0 2021-04-03T03:15:25.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f612c0016d0 con 0x7f61440603f0 2021-04-03T03:15:25.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 >> v1:172.21.2.111:6789/0 conn(0x7f61440603f0 legacy=0x7f61440b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61240037b0 0x7f6124003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61240044e0 0x7f6124004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440b0c90 0x7f6124004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440af400 con 0x7f61240037b0 2021-04-03T03:15:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440af6a0 con 0x7f61240044e0 2021-04-03T03:15:25.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6124005340 con 0x7f61440b0c90 2021-04-03T03:15:25.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61240037b0 0x7f6124003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 -- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440b0c90 msgr2=0x7f6124004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440b0c90 0x7f6124004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 -- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61240044e0 msgr2=0x7f6124004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61240044e0 0x7f6124004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 -- 172.21.2.111:0/3509426636 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6124003240 con 0x7f61240037b0 2021-04-03T03:15:25.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6143fff700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61240037b0 0x7f6124003bd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f612c0013e0 tx=0x7f612c0076d0).ready entity=mon.0 client_cookie=10477e344a3fab07 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f612c0070d0 con 0x7f61240037b0 2021-04-03T03:15:25.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c007230 con 0x7f61240037b0 2021-04-03T03:15:25.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6142ffd700 1 -- 172.21.2.111:0/3509426636 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f612c0055f0 con 0x7f61240037b0 2021-04-03T03:15:25.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/3509426636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61240037b0 msgr2=0x7f6124003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 --2- 172.21.2.111:0/3509426636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61240037b0 0x7f6124003bd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f612c0013e0 tx=0x7f612c0076d0).stop 2021-04-03T03:15:25.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 -- 172.21.2.111:0/3509426636 reap_dead start 2021-04-03T03:15:25.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/3509426636 shutdown_connections 2021-04-03T03:15:25.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/3509426636 >> 172.21.2.111:0/3509426636 conn(0x7f61440b61c0 msgr2=0x7f614405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/3509426636 shutdown_connections 2021-04-03T03:15:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/3509426636 wait complete. 2021-04-03T03:15:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 Processor -- start 2021-04-03T03:15:25.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- start start 2021-04-03T03:15:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440a81d0 0x7f6144064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61440aa870 0x7f6144064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61440a8830 0x7f6144067490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440a85b0 con 0x7f61440a8830 2021-04-03T03:15:25.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440628d0 con 0x7f61440aa870 2021-04-03T03:15:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6144062b70 con 0x7f61440a81d0 2021-04-03T03:15:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61440a8830 0x7f6144067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61440a8830 0x7f6144067490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54510/0 (socket says 172.21.2.111:54510) 2021-04-03T03:15:25.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 -- 172.21.2.111:0/1496512421 learned_addr learned my addr 172.21.2.111:0/1496512421 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 -- 172.21.2.111:0/1496512421 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440a81d0 msgr2=0x7f6144064540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440a81d0 0x7f6144064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 -- 172.21.2.111:0/1496512421 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61440aa870 msgr2=0x7f6144064a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61440aa870 0x7f6144064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 -- 172.21.2.111:0/1496512421 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144062e10 con 0x7f61440a8830 2021-04-03T03:15:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f6148a8a700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61440a8830 0x7f6144067490 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f613c0011d0 tx=0x7f613c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f61417fa700 1 -- 172.21.2.111:0/1496512421 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f613c00c860 con 0x7f61440a8830 2021-04-03T03:15:25.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f61417fa700 1 -- 172.21.2.111:0/1496512421 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c00c9c0 con 0x7f61440a8830 2021-04-03T03:15:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6144064f20 con 0x7f61440a8830 2021-04-03T03:15:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f61417fa700 1 -- 172.21.2.111:0/1496512421 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f613c00cd20 con 0x7f61440a8830 2021-04-03T03:15:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61440abe80 con 0x7f61440a8830 2021-04-03T03:15:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.396+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6144064230 con 0x7f61440a8830 2021-04-03T03:15:25.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.400+0000 7f61417fa700 1 -- 172.21.2.111:0/1496512421 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f613c01c460 con 0x7f61440a8830 2021-04-03T03:15:25.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.400+0000 7f61417fa700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6128056ef0 0x7f61280593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.400+0000 7f61417fa700 1 -- 172.21.2.111:0/1496512421 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f613c070830 con 0x7f61440a8830 2021-04-03T03:15:25.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.400+0000 7f6143fff700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6128056ef0 0x7f61280593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.400+0000 7f6143fff700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6128056ef0 0x7f61280593b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f612c0013e0 tx=0x7f612c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.400+0000 7f61417fa700 1 -- 172.21.2.111:0/1496512421 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f613c045bc0 con 0x7f61440a8830 2021-04-03T03:15:25.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.492+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6144057660 con 0x7f61440a8830 2021-04-03T03:15:25.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f61417fa700 1 -- 172.21.2.111:0/1496512421 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f613c0491e0 con 0x7f61440a8830 2021-04-03T03:15:25.500 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:25.500 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":83,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:25.328587+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:23.240089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:24.288628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:25.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6128056ef0 msgr2=0x7f61280593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6128056ef0 0x7f61280593b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f612c0013e0 tx=0x7f612c015000).stop 2021-04-03T03:15:25.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61440a8830 msgr2=0x7f6144067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61440a8830 0x7f6144067490 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f613c0011d0 tx=0x7f613c0091d0).stop 2021-04-03T03:15:25.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 shutdown_connections 2021-04-03T03:15:25.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61440a8830 0x7f6144067490 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f61440a81d0 0x7f6144064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6128056ef0 0x7f61280593b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 --2- 172.21.2.111:0/1496512421 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61440aa870 0x7f6144064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 >> 172.21.2.111:0/1496512421 conn(0x7f61440b61c0 msgr2=0x7f6144147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 shutdown_connections 2021-04-03T03:15:25.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.496+0000 7f614928b700 1 -- 172.21.2.111:0/1496512421 wait complete. 2021-04-03T03:15:25.508 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:25.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 Processor -- start 2021-04-03T03:15:25.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- start start 2021-04-03T03:15:25.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f666405fb50 0x7f666414b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66640b1b70 con 0x7f66640b1e40 2021-04-03T03:15:25.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66640609d0 con 0x7f66640604c0 2021-04-03T03:15:25.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66640ae420 con 0x7f666405fb50 2021-04-03T03:15:25.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662b85700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f66640b1e40 0x7f6664067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55310/0 (socket says 172.21.2.111:55310) 2021-04-03T03:15:25.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662b85700 1 -- 172.21.2.111:0/2660729356 learned_addr learned my addr 172.21.2.111:0/2660729356 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:25.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002090180 0 0) 0x7f66640b1b70 con 0x7f66640b1e40 2021-04-03T03:15:25.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6644003240 con 0x7f66640b1e40 2021-04-03T03:15:25.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 --2- 172.21.2.111:0/2660729356 >> v2:172.21.2.115:3300/0 conn(0x7f666405fb50 0x7f666414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 487804114 0 0) 0x7f6644003240 con 0x7f66640b1e40 2021-04-03T03:15:25.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 >> v2:172.21.2.115:3300/0 conn(0x7f666405fb50 msgr2=0x7f666414b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 --2- 172.21.2.111:0/2660729356 >> v2:172.21.2.115:3300/0 conn(0x7f666405fb50 0x7f666414b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 >> v1:172.21.2.115:6789/0 conn(0x7f66640604c0 legacy=0x7f66641478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666414af50 con 0x7f66640b1e40 2021-04-03T03:15:25.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f665c001a50 con 0x7f66640b1e40 2021-04-03T03:15:25.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 >> v1:172.21.2.111:6789/0 conn(0x7f66640b1e40 legacy=0x7f6664067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6644003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66440044e0 0x7f6644004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6644004e50 0x7f664400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644005300 con 0x7f6644003800 2021-04-03T03:15:25.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664400bc20 con 0x7f6644004e50 2021-04-03T03:15:25.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664400bd60 con 0x7f66440044e0 2021-04-03T03:15:25.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6644003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661b83700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66440044e0 0x7f6644004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662b85700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6644004e50 0x7f664400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 -- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66440044e0 msgr2=0x7f6644004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66440044e0 0x7f6644004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 -- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6644004e50 msgr2=0x7f664400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:25.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6644004e50 0x7f664400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 -- 172.21.2.111:0/2660729356 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644003240 con 0x7f6644003800 2021-04-03T03:15:25.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662b85700 1 -- 172.21.2.111:0/2660729356 reap_dead start 2021-04-03T03:15:25.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6662384700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6644003c00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f6654007000 tx=0x7f6654008040).ready entity=mon.0 client_cookie=982581afac36f47c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665400a950 con 0x7f6644003800 2021-04-03T03:15:25.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665400c030 con 0x7f6644003800 2021-04-03T03:15:25.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6661382700 1 -- 172.21.2.111:0/2660729356 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665400aca0 con 0x7f6644003800 2021-04-03T03:15:25.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- 172.21.2.111:0/2660729356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 msgr2=0x7f6644003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6644003c00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f6654007000 tx=0x7f6654008040).stop 2021-04-03T03:15:25.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- 172.21.2.111:0/2660729356 shutdown_connections 2021-04-03T03:15:25.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 --2- 172.21.2.111:0/2660729356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6644003c00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- 172.21.2.111:0/2660729356 >> 172.21.2.111:0/2660729356 conn(0x7f66640b5fb0 msgr2=0x7f66640b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- 172.21.2.111:0/2660729356 shutdown_connections 2021-04-03T03:15:25.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.560+0000 7f6663386700 1 -- 172.21.2.111:0/2660729356 wait complete. 2021-04-03T03:15:25.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 Processor -- start 2021-04-03T03:15:25.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 -- start start 2021-04-03T03:15:25.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6664149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6664063330 0x7f6664065ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6664065fd0 0x7f6664063ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640acfb0 con 0x7f6644003800 2021-04-03T03:15:25.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66641564f0 con 0x7f6664063330 2021-04-03T03:15:25.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66640653a0 con 0x7f6664065fd0 2021-04-03T03:15:25.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6664149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6664149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54526/0 (socket says 172.21.2.111:54526) 2021-04-03T03:15:25.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 -- 172.21.2.111:0/2249161210 learned_addr learned my addr 172.21.2.111:0/2249161210 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:25.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 -- 172.21.2.111:0/2249161210 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6664065fd0 msgr2=0x7f6664063ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6664065fd0 0x7f6664063ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 -- 172.21.2.111:0/2249161210 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6664063330 msgr2=0x7f6664065ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6664063330 0x7f6664065ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 -- 172.21.2.111:0/2249161210 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664065640 con 0x7f6644003800 2021-04-03T03:15:25.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6662384700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6664149d60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f6654007000 tx=0x7f6654008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f665b7fe700 1 -- 172.21.2.111:0/2249161210 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6654021070 con 0x7f6644003800 2021-04-03T03:15:25.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f665b7fe700 1 -- 172.21.2.111:0/2249161210 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654012d50 con 0x7f6644003800 2021-04-03T03:15:25.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6664157230 con 0x7f6644003800 2021-04-03T03:15:25.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f665b7fe700 1 -- 172.21.2.111:0/2249161210 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665401d610 con 0x7f6644003800 2021-04-03T03:15:25.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666414c540 con 0x7f6644003800 2021-04-03T03:15:25.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66640657d0 con 0x7f6644003800 2021-04-03T03:15:25.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f665b7fe700 1 -- 172.21.2.111:0/2249161210 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f665401d770 con 0x7f6644003800 2021-04-03T03:15:25.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f665b7fe700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648056fb0 0x7f6648059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f665b7fe700 1 -- 172.21.2.111:0/2249161210 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f66540719c0 con 0x7f6644003800 2021-04-03T03:15:25.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6661b83700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648056fb0 0x7f6648059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.564+0000 7f6661b83700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648056fb0 0x7f6648059470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f664c00b000 tx=0x7f664c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.568+0000 7f665b7fe700 1 -- 172.21.2.111:0/2249161210 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6654046db0 con 0x7f6644003800 2021-04-03T03:15:25.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6664069270 con 0x7f6644003800 2021-04-03T03:15:25.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f665b7fe700 1 -- 172.21.2.111:0/2249161210 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f665404a3d0 con 0x7f6644003800 2021-04-03T03:15:25.665 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:25.666 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":83,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:25.328587+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:23.240089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:24.288628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:25.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648056fb0 msgr2=0x7f6648059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648056fb0 0x7f6648059470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f664c00b000 tx=0x7f664c009040).stop 2021-04-03T03:15:25.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 msgr2=0x7f6664149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6664149d60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f6654007000 tx=0x7f6654008040).stop 2021-04-03T03:15:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 shutdown_connections 2021-04-03T03:15:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6664065fd0 0x7f6664063ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6644003800 0x7f6664149d60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6648056fb0 0x7f6648059470 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 --2- 172.21.2.111:0/2249161210 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6664063330 0x7f6664065ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 >> 172.21.2.111:0/2249161210 conn(0x7f66640b5fb0 msgr2=0x7f66640b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 shutdown_connections 2021-04-03T03:15:25.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.660+0000 7f6663386700 1 -- 172.21.2.111:0/2249161210 wait complete. 2021-04-03T03:15:25.673 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:15:25.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.724+0000 7ffae9190700 1 Processor -- start 2021-04-03T03:15:25.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- start start 2021-04-03T03:15:25.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ffadc00bc90 0x7ffadc029ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ffadc0288c0 0x7ffadc0d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc00c070 con 0x7ffadc0d5930 2021-04-03T03:15:25.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc02af80 con 0x7ffadc00bc90 2021-04-03T03:15:25.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc029760 con 0x7ffadc0288c0 2021-04-03T03:15:25.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ffadc00bc90 0x7ffadc029ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ffadc00bc90 0x7ffadc029ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54534/0 (socket says 172.21.2.111:54534) 2021-04-03T03:15:25.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 -- 172.21.2.111:0/2674337617 learned_addr learned my addr 172.21.2.111:0/2674337617 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:25.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 -- 172.21.2.111:0/2674337617 >> v2:172.21.2.115:3300/0 conn(0x7ffadc0288c0 msgr2=0x7ffadc0d5440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:25.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 --2- 172.21.2.111:0/2674337617 >> v2:172.21.2.115:3300/0 conn(0x7ffadc0288c0 0x7ffadc0d5440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 -- 172.21.2.111:0/2674337617 >> v1:172.21.2.115:6789/0 conn(0x7ffadc0d5930 legacy=0x7ffadc005680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:25.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 -- 172.21.2.111:0/2674337617 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc029a00 con 0x7ffadc00bc90 2021-04-03T03:15:25.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae3fff700 1 --2- 172.21.2.111:0/2674337617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc00bc90 0x7ffadc029ea0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7ffadc029560 tx=0x7ffad4006e90).ready entity=mon.0 client_cookie=1539b7822b972aa6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae2ffd700 1 -- 172.21.2.111:0/2674337617 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffad4009930 con 0x7ffadc00bc90 2021-04-03T03:15:25.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae2ffd700 1 -- 172.21.2.111:0/2674337617 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad4009a90 con 0x7ffadc00bc90 2021-04-03T03:15:25.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae2ffd700 1 -- 172.21.2.111:0/2674337617 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffad4009e30 con 0x7ffadc00bc90 2021-04-03T03:15:25.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- 172.21.2.111:0/2674337617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc00bc90 msgr2=0x7ffadc029ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- 172.21.2.111:0/2674337617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc00bc90 0x7ffadc029ea0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7ffadc029560 tx=0x7ffad4006e90).stop 2021-04-03T03:15:25.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- 172.21.2.111:0/2674337617 shutdown_connections 2021-04-03T03:15:25.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- 172.21.2.111:0/2674337617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc00bc90 0x7ffadc029ea0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:25.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- 172.21.2.111:0/2674337617 >> v2:172.21.2.115:3300/0 conn(0x7ffadc0288c0 0x7ffadc0d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- 172.21.2.111:0/2674337617 >> 172.21.2.111:0/2674337617 conn(0x7ffadc0484c0 msgr2=0x7ffadc0488d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:25.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- 172.21.2.111:0/2674337617 shutdown_connections 2021-04-03T03:15:25.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- 172.21.2.111:0/2674337617 wait complete. 2021-04-03T03:15:25.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 Processor -- start 2021-04-03T03:15:25.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- start start 2021-04-03T03:15:25.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffadc00bc90 0x7ffadc043a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffadc0288c0 0x7ffadc02d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc0d5930 0x7ffadc0e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc02e150 con 0x7ffadc0d5930 2021-04-03T03:15:25.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc02c870 con 0x7ffadc00bc90 2021-04-03T03:15:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc02de00 con 0x7ffadc0288c0 2021-04-03T03:15:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc0d5930 0x7ffadc0e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc0d5930 0x7ffadc0e1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54544/0 (socket says 172.21.2.111:54544) 2021-04-03T03:15:25.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 -- 172.21.2.111:0/3044063525 learned_addr learned my addr 172.21.2.111:0/3044063525 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 -- 172.21.2.111:0/3044063525 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffadc0288c0 msgr2=0x7ffadc02d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffadc0288c0 0x7ffadc02d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 -- 172.21.2.111:0/3044063525 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffadc00bc90 msgr2=0x7ffadc043a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffadc00bc90 0x7ffadc043a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:25.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 -- 172.21.2.111:0/3044063525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc02d1d0 con 0x7ffadc0d5930 2021-04-03T03:15:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae898f700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc0d5930 0x7ffadc0e1ff0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7ffadc02e480 tx=0x7ffae405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae17fa700 1 -- 172.21.2.111:0/3044063525 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae405df60 con 0x7ffadc0d5930 2021-04-03T03:15:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae17fa700 1 -- 172.21.2.111:0/3044063525 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae40699f0 con 0x7ffadc0d5930 2021-04-03T03:15:25.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc02d470 con 0x7ffadc0d5930 2021-04-03T03:15:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae17fa700 1 -- 172.21.2.111:0/3044063525 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae405f4b0 con 0x7ffadc0d5930 2021-04-03T03:15:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.728+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc02c4a0 con 0x7ffadc0d5930 2021-04-03T03:15:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.732+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc0422d0 con 0x7ffadc0d5930 2021-04-03T03:15:25.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.732+0000 7ffae17fa700 1 -- 172.21.2.111:0/3044063525 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ffae405f610 con 0x7ffadc0d5930 2021-04-03T03:15:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.732+0000 7ffae17fa700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffac4056f20 0x7ffac40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.732+0000 7ffae17fa700 1 -- 172.21.2.111:0/3044063525 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ffae40df300 con 0x7ffadc0d5930 2021-04-03T03:15:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.732+0000 7ffae3fff700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffac4056f20 0x7ffac40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.732+0000 7ffae3fff700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffac4056f20 0x7ffac40593e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7ffadc029560 tx=0x7ffad4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:25.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.732+0000 7ffae17fa700 1 -- 172.21.2.111:0/3044063525 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ffadc0422d0 con 0x7ffadc0d5930 2021-04-03T03:15:25.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:25.864+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffadc0422d0 con 0x7ffadc0d5930 2021-04-03T03:15:26.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae17fa700 1 -- 172.21.2.111:0/3044063525 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffadc0422d0 con 0x7ffadc0d5930 2021-04-03T03:15:26.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffac4056f20 msgr2=0x7ffac40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffac4056f20 0x7ffac40593e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7ffadc029560 tx=0x7ffad4015000).stop 2021-04-03T03:15:26.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc0d5930 msgr2=0x7ffadc0e1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc0d5930 0x7ffadc0e1ff0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7ffadc02e480 tx=0x7ffae405ad70).stop 2021-04-03T03:15:26.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 shutdown_connections 2021-04-03T03:15:26.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffadc0d5930 0x7ffadc0e1ff0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffac4056f20 0x7ffac40593e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffadc00bc90 0x7ffadc043a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 --2- 172.21.2.111:0/3044063525 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffadc0288c0 0x7ffadc02d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 >> 172.21.2.111:0/3044063525 conn(0x7ffadc0484c0 msgr2=0x7ffadc004fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 shutdown_connections 2021-04-03T03:15:26.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.340+0000 7ffae9190700 1 -- 172.21.2.111:0/3044063525 wait complete. 2021-04-03T03:15:26.354 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:26.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 Processor -- start 2021-04-03T03:15:26.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- start start 2021-04-03T03:15:26.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5d940603f0 0x7f5d940b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940b0c90 0x7f5d94147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940607d0 con 0x7f5d9414b700 2021-04-03T03:15:26.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d940af400 con 0x7f5d940603f0 2021-04-03T03:15:26.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940af6a0 con 0x7f5d940b0c90 2021-04-03T03:15:26.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5d940603f0 0x7f5d940b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5d940603f0 0x7f5d940b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54548/0 (socket says 172.21.2.111:54548) 2021-04-03T03:15:26.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 -- 172.21.2.111:0/3287996027 learned_addr learned my addr 172.21.2.111:0/3287996027 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:26.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d92d9d700 1 --2- 172.21.2.111:0/3287996027 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940b0c90 0x7f5d94147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 -- 172.21.2.111:0/3287996027 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940b0c90 msgr2=0x7f5d94147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:26.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 --2- 172.21.2.111:0/3287996027 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940b0c90 0x7f5d94147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 -- 172.21.2.111:0/3287996027 >> v1:172.21.2.115:6789/0 conn(0x7f5d9414b700 legacy=0x7f5d940b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:26.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 -- 172.21.2.111:0/3287996027 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940afb40 con 0x7f5d940603f0 2021-04-03T03:15:26.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9259c700 1 --2- 172.21.2.111:0/3287996027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940603f0 0x7f5d940b0090 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f5d9406a3b0 tx=0x7f5d84006e90).ready entity=mon.0 client_cookie=a8df89a73410f9af server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9159a700 1 -- 172.21.2.111:0/3287996027 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d840098e0 con 0x7f5d940603f0 2021-04-03T03:15:26.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9159a700 1 -- 172.21.2.111:0/3287996027 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84009a40 con 0x7f5d940603f0 2021-04-03T03:15:26.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9159a700 1 -- 172.21.2.111:0/3287996027 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d84009de0 con 0x7f5d940603f0 2021-04-03T03:15:26.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- 172.21.2.111:0/3287996027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940603f0 msgr2=0x7f5d940b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 --2- 172.21.2.111:0/3287996027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940603f0 0x7f5d940b0090 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f5d9406a3b0 tx=0x7f5d84006e90).stop 2021-04-03T03:15:26.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- 172.21.2.111:0/3287996027 shutdown_connections 2021-04-03T03:15:26.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 --2- 172.21.2.111:0/3287996027 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940b0c90 0x7f5d94147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 --2- 172.21.2.111:0/3287996027 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940603f0 0x7f5d940b0090 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- 172.21.2.111:0/3287996027 >> 172.21.2.111:0/3287996027 conn(0x7f5d940b61c0 msgr2=0x7f5d9405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- 172.21.2.111:0/3287996027 shutdown_connections 2021-04-03T03:15:26.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.408+0000 7f5d9359e700 1 -- 172.21.2.111:0/3287996027 wait complete. 2021-04-03T03:15:26.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 Processor -- start 2021-04-03T03:15:26.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 -- start start 2021-04-03T03:15:26.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940603f0 0x7f5d940a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940b0c90 0x7f5d940ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d9414b700 0x7f5d940ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940aa2f0 con 0x7f5d940b0c90 2021-04-03T03:15:26.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940aab10 con 0x7f5d9414b700 2021-04-03T03:15:26.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940aa4f0 con 0x7f5d940603f0 2021-04-03T03:15:26.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940b0c90 0x7f5d940ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940b0c90 0x7f5d940ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54556/0 (socket says 172.21.2.111:54556) 2021-04-03T03:15:26.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 -- 172.21.2.111:0/2818532771 learned_addr learned my addr 172.21.2.111:0/2818532771 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:26.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9259c700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940603f0 0x7f5d940a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d92d9d700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d9414b700 0x7f5d940ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 -- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940603f0 msgr2=0x7f5d940a81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940603f0 0x7f5d940a81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 -- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d9414b700 msgr2=0x7f5d940ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d9414b700 0x7f5d940ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 -- 172.21.2.111:0/2818532771 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940aa790 con 0x7f5d940b0c90 2021-04-03T03:15:26.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9259c700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940603f0 0x7f5d940a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:26.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d91d9b700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940b0c90 0x7f5d940ab190 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f5d940abf50 tx=0x7f5d7c006d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d8b7fe700 1 -- 172.21.2.111:0/2818532771 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7c00d4a0 con 0x7f5d940b0c90 2021-04-03T03:15:26.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d8b7fe700 1 -- 172.21.2.111:0/2818532771 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c000e90 con 0x7f5d940b0c90 2021-04-03T03:15:26.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d940abb70 con 0x7f5d940b0c90 2021-04-03T03:15:26.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d8b7fe700 1 -- 172.21.2.111:0/2818532771 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7c00d7f0 con 0x7f5d940b0c90 2021-04-03T03:15:26.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d94159c90 con 0x7f5d940b0c90 2021-04-03T03:15:26.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d94057660 con 0x7f5d940b0c90 2021-04-03T03:15:26.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d8b7fe700 1 -- 172.21.2.111:0/2818532771 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5d7c00f030 con 0x7f5d940b0c90 2021-04-03T03:15:26.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d8b7fe700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d78056f70 0x7f5d78059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d8b7fe700 1 -- 172.21.2.111:0/2818532771 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5d7c06fd60 con 0x7f5d940b0c90 2021-04-03T03:15:26.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9259c700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d78056f70 0x7f5d78059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.412+0000 7f5d9259c700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d78056f70 0x7f5d78059430 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f5d84017000 tx=0x7f5d84015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.416+0000 7f5d8b7fe700 1 -- 172.21.2.111:0/2818532771 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5d7c045150 con 0x7f5d940b0c90 2021-04-03T03:15:26.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d94064ec0 con 0x7f5d940b0c90 2021-04-03T03:15:26.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d8b7fe700 1 -- 172.21.2.111:0/2818532771 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f5d7c048770 con 0x7f5d940b0c90 2021-04-03T03:15:26.546 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:26.546 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2065583567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2065583567},{"type":"v1","addr":"172.21.2.115:6833","nonce":2065583567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:25.328682+0000","modified":"2021-04-03T03:15:26.331659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/485806052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":485806052},{"type":"v1","addr":"172.21.2.111:6835","nonce":485806052}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-04-03T03:15:26.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d78056f70 msgr2=0x7f5d78059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d78056f70 0x7f5d78059430 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f5d84017000 tx=0x7f5d84015000).stop 2021-04-03T03:15:26.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940b0c90 msgr2=0x7f5d940ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940b0c90 0x7f5d940ab190 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f5d940abf50 tx=0x7f5d7c006d10).stop 2021-04-03T03:15:26.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 shutdown_connections 2021-04-03T03:15:26.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5d78056f70 0x7f5d78059430 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5d9414b700 0x7f5d940ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5d940603f0 0x7f5d940a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 --2- 172.21.2.111:0/2818532771 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5d940b0c90 0x7f5d940ab190 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 >> 172.21.2.111:0/2818532771 conn(0x7f5d940b61c0 msgr2=0x7f5d940b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 shutdown_connections 2021-04-03T03:15:26.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.540+0000 7f5d9359e700 1 -- 172.21.2.111:0/2818532771 wait complete. 2021-04-03T03:15:26.550 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 33 2021-04-03T03:15:26.553 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:26.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 Processor -- start 2021-04-03T03:15:26.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- start start 2021-04-03T03:15:26.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 0x7f95600b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f95600b0c90 0x7f9560147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95600607d0 con 0x7f956014b700 2021-04-03T03:15:26.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95600af400 con 0x7f95600b0c90 2021-04-03T03:15:26.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600af6a0 con 0x7f95600603f0 2021-04-03T03:15:26.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955f5e6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f95600b0c90 0x7f9560147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955e5e4700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f956014b700 0x7f95600b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55356/0 (socket says 172.21.2.111:55356) 2021-04-03T03:15:26.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955e5e4700 1 -- 172.21.2.111:0/2855120577 learned_addr learned my addr 172.21.2.111:0/2855120577 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:26.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955ede5700 1 --2- 172.21.2.111:0/2855120577 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 0x7f95600b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955f5e6700 1 -- 172.21.2.111:0/2855120577 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 msgr2=0x7f95600b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:26.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955f5e6700 1 --2- 172.21.2.111:0/2855120577 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 0x7f95600b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955f5e6700 1 -- 172.21.2.111:0/2855120577 >> v1:172.21.2.111:6789/0 conn(0x7f956014b700 legacy=0x7f95600b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955dde3700 1 -- 172.21.2.111:0/2855120577 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443628571 0 0) 0x7f95600607d0 con 0x7f956014b700 2021-04-03T03:15:26.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955f5e6700 1 -- 172.21.2.111:0/2855120577 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600afb40 con 0x7f95600b0c90 2021-04-03T03:15:26.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955f5e6700 1 --2- 172.21.2.111:0/2855120577 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f9560147680 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f95580032b0 tx=0x7f95580091d0).ready entity=mon.0 client_cookie=20485a685b2dfe79 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955dde3700 1 -- 172.21.2.111:0/2855120577 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f955800ba50 con 0x7f95600b0c90 2021-04-03T03:15:26.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955dde3700 1 -- 172.21.2.111:0/2855120577 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955800bbb0 con 0x7f95600b0c90 2021-04-03T03:15:26.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955dde3700 1 -- 172.21.2.111:0/2855120577 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f955800bd20 con 0x7f95600b0c90 2021-04-03T03:15:26.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- 172.21.2.111:0/2855120577 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 msgr2=0x7f9560147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 --2- 172.21.2.111:0/2855120577 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f9560147680 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f95580032b0 tx=0x7f95580091d0).stop 2021-04-03T03:15:26.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- 172.21.2.111:0/2855120577 shutdown_connections 2021-04-03T03:15:26.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 --2- 172.21.2.111:0/2855120577 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f9560147680 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 --2- 172.21.2.111:0/2855120577 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 0x7f95600b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- 172.21.2.111:0/2855120577 >> 172.21.2.111:0/2855120577 conn(0x7f95600b61c0 msgr2=0x7f956005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- 172.21.2.111:0/2855120577 shutdown_connections 2021-04-03T03:15:26.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.604+0000 7f955fde7700 1 -- 172.21.2.111:0/2855120577 wait complete. 2021-04-03T03:15:26.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 Processor -- start 2021-04-03T03:15:26.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 -- start start 2021-04-03T03:15:26.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 0x7f95601597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f95600a7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f956014b700 0x7f9560064350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600ab3c0 con 0x7f95600b0c90 2021-04-03T03:15:26.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600ab660 con 0x7f956014b700 2021-04-03T03:15:26.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600aba00 con 0x7f95600603f0 2021-04-03T03:15:26.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f95600a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f95600a7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54572/0 (socket says 172.21.2.111:54572) 2021-04-03T03:15:26.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 -- 172.21.2.111:0/4104441829 learned_addr learned my addr 172.21.2.111:0/4104441829 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:26.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 -- 172.21.2.111:0/4104441829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 msgr2=0x7f95601597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 0x7f95601597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 -- 172.21.2.111:0/4104441829 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f956014b700 msgr2=0x7f9560064350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f956014b700 0x7f9560064350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 -- 172.21.2.111:0/4104441829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600abbc0 con 0x7f95600b0c90 2021-04-03T03:15:26.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955e5e4700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f95600a7880 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f9550000b80 tx=0x7f9550007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f9557fff700 1 -- 172.21.2.111:0/4104441829 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95500072e0 con 0x7f95600b0c90 2021-04-03T03:15:26.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f9557fff700 1 -- 172.21.2.111:0/4104441829 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9550007440 con 0x7f95600b0c90 2021-04-03T03:15:26.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95600aa4a0 con 0x7f95600b0c90 2021-04-03T03:15:26.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f9557fff700 1 -- 172.21.2.111:0/4104441829 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95500057e0 con 0x7f95600b0c90 2021-04-03T03:15:26.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95600649d0 con 0x7f95600b0c90 2021-04-03T03:15:26.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95600ab7d0 con 0x7f95600b0c90 2021-04-03T03:15:26.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f9557fff700 1 -- 172.21.2.111:0/4104441829 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9550010070 con 0x7f95600b0c90 2021-04-03T03:15:26.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f9557fff700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9544056fb0 0x7f9544059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f9557fff700 1 -- 172.21.2.111:0/4104441829 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f955006f8b0 con 0x7f95600b0c90 2021-04-03T03:15:26.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955ede5700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9544056fb0 0x7f9544059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.608+0000 7f955ede5700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9544056fb0 0x7f9544059470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f9548007000 tx=0x7f9548008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.612+0000 7f9557fff700 1 -- 172.21.2.111:0/4104441829 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9550044ca0 con 0x7f95600b0c90 2021-04-03T03:15:26.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.700+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9560057660 con 0x7f95600b0c90 2021-04-03T03:15:26.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f9557fff700 1 -- 172.21.2.111:0/4104441829 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f95500482c0 con 0x7f95600b0c90 2021-04-03T03:15:26.708 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:26.709 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":84,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:26.331795+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:23.240089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:24.288628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:26.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9544056fb0 msgr2=0x7f9544059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9544056fb0 0x7f9544059470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f9548007000 tx=0x7f9548008040).stop 2021-04-03T03:15:26.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 msgr2=0x7f95600a7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f95600a7880 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f9550000b80 tx=0x7f9550007c40).stop 2021-04-03T03:15:26.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 shutdown_connections 2021-04-03T03:15:26.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9544056fb0 0x7f9544059470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f956014b700 0x7f9560064350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f95600603f0 0x7f95601597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 --2- 172.21.2.111:0/4104441829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f95600b0c90 0x7f95600a7880 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 >> 172.21.2.111:0/4104441829 conn(0x7f95600b61c0 msgr2=0x7f9560147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 shutdown_connections 2021-04-03T03:15:26.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.704+0000 7f955fde7700 1 -- 172.21.2.111:0/4104441829 wait complete. 2021-04-03T03:15:26.716 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:15:26.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 Processor -- start 2021-04-03T03:15:26.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- start start 2021-04-03T03:15:26.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe5240b38c0 0x7fe524025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe524153440 0x7fe524080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5240b3ca0 con 0x7fe524080a40 2021-04-03T03:15:26.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5240b0300 con 0x7fe524153440 2021-04-03T03:15:26.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe524025970 con 0x7fe5240b38c0 2021-04-03T03:15:26.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe5237fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe524153440 0x7fe524080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe529db6700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe524080a40 0x7fe52414feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55372/0 (socket says 172.21.2.111:55372) 2021-04-03T03:15:26.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe529db6700 1 -- 172.21.2.111:0/478209615 learned_addr learned my addr 172.21.2.111:0/478209615 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:26.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe523fff700 1 --2- 172.21.2.111:0/478209615 >> v2:172.21.2.115:3300/0 conn(0x7fe5240b38c0 0x7fe524025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe522ffd700 1 -- 172.21.2.111:0/478209615 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385099894 0 0) 0x7fe5240b3ca0 con 0x7fe524080a40 2021-04-03T03:15:26.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe522ffd700 1 -- 172.21.2.111:0/478209615 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe510003240 con 0x7fe524080a40 2021-04-03T03:15:26.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe5237fe700 1 -- 172.21.2.111:0/478209615 >> v2:172.21.2.115:3300/0 conn(0x7fe5240b38c0 msgr2=0x7fe524025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:26.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe5237fe700 1 --2- 172.21.2.111:0/478209615 >> v2:172.21.2.115:3300/0 conn(0x7fe5240b38c0 0x7fe524025480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe5237fe700 1 -- 172.21.2.111:0/478209615 >> v1:172.21.2.111:6789/0 conn(0x7fe524080a40 legacy=0x7fe52414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe5237fe700 1 -- 172.21.2.111:0/478209615 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524025b90 con 0x7fe524153440 2021-04-03T03:15:26.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe5237fe700 1 --2- 172.21.2.111:0/478209615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe524153440 0x7fe524080550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7fe50c002f30 tx=0x7fe50c006dd0).ready entity=mon.0 client_cookie=481c4546cfa06a71 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe522ffd700 1 -- 172.21.2.111:0/478209615 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50c0097b0 con 0x7fe524153440 2021-04-03T03:15:26.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe522ffd700 1 -- 172.21.2.111:0/478209615 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c009910 con 0x7fe524153440 2021-04-03T03:15:26.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe522ffd700 1 -- 172.21.2.111:0/478209615 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50c009c70 con 0x7fe524153440 2021-04-03T03:15:26.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- 172.21.2.111:0/478209615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe524153440 msgr2=0x7fe524080550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 --2- 172.21.2.111:0/478209615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe524153440 0x7fe524080550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7fe50c002f30 tx=0x7fe50c006dd0).stop 2021-04-03T03:15:26.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- 172.21.2.111:0/478209615 shutdown_connections 2021-04-03T03:15:26.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 --2- 172.21.2.111:0/478209615 >> v2:172.21.2.115:3300/0 conn(0x7fe5240b38c0 0x7fe524025480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 --2- 172.21.2.111:0/478209615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe524153440 0x7fe524080550 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- 172.21.2.111:0/478209615 >> 172.21.2.111:0/478209615 conn(0x7fe524061610 msgr2=0x7fe524061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- 172.21.2.111:0/478209615 shutdown_connections 2021-04-03T03:15:26.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.772+0000 7fe52893c700 1 -- 172.21.2.111:0/478209615 wait complete. 2021-04-03T03:15:26.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 Processor -- start 2021-04-03T03:15:26.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 -- start start 2021-04-03T03:15:26.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe524080a40 0x7fe5240b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5240b38c0 0x7fe524150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe524153440 0x7fe5240aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe524064880 con 0x7fe5240b38c0 2021-04-03T03:15:26.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52413e8c0 con 0x7fe524153440 2021-04-03T03:15:26.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe524064250 con 0x7fe524080a40 2021-04-03T03:15:26.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5240b38c0 0x7fe524150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5240b38c0 0x7fe524150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54586/0 (socket says 172.21.2.111:54586) 2021-04-03T03:15:26.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 -- 172.21.2.111:0/1628042526 learned_addr learned my addr 172.21.2.111:0/1628042526 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:26.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 -- 172.21.2.111:0/1628042526 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe524080a40 msgr2=0x7fe5240b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe524080a40 0x7fe5240b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 -- 172.21.2.111:0/1628042526 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe524153440 msgr2=0x7fe5240aaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe524153440 0x7fe5240aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 -- 172.21.2.111:0/1628042526 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5241429c0 con 0x7fe5240b38c0 2021-04-03T03:15:26.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe5237fe700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5240b38c0 0x7fe524150690 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7fe50c002f30 tx=0x7fe50c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe520ff9700 1 -- 172.21.2.111:0/1628042526 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50c01d450 con 0x7fe5240b38c0 2021-04-03T03:15:26.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe520ff9700 1 -- 172.21.2.111:0/1628042526 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c01d5b0 con 0x7fe5240b38c0 2021-04-03T03:15:26.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe520ff9700 1 -- 172.21.2.111:0/1628042526 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50c01d910 con 0x7fe5240b38c0 2021-04-03T03:15:26.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe524142c60 con 0x7fe5240b38c0 2021-04-03T03:15:26.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5240abf80 con 0x7fe5240b38c0 2021-04-03T03:15:26.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe524057660 con 0x7fe5240b38c0 2021-04-03T03:15:26.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe520ff9700 1 -- 172.21.2.111:0/1628042526 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe50c00f070 con 0x7fe5240b38c0 2021-04-03T03:15:26.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe520ff9700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe508057030 0x7fe5080594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe520ff9700 1 -- 172.21.2.111:0/1628042526 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe50c071ff0 con 0x7fe5240b38c0 2021-04-03T03:15:26.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe523fff700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe508057030 0x7fe5080594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.776+0000 7fe523fff700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe508057030 0x7fe5080594f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fe51400a000 tx=0x7fe514008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.780+0000 7fe520ff9700 1 -- 172.21.2.111:0/1628042526 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe50c0472f0 con 0x7fe5240b38c0 2021-04-03T03:15:26.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.904+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe524069270 con 0x7fe5240b38c0 2021-04-03T03:15:26.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.908+0000 7fe520ff9700 1 -- 172.21.2.111:0/1628042526 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe50c04a910 con 0x7fe5240b38c0 2021-04-03T03:15:26.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe508057030 msgr2=0x7fe5080594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe508057030 0x7fe5080594f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fe51400a000 tx=0x7fe514008040).stop 2021-04-03T03:15:26.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5240b38c0 msgr2=0x7fe524150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5240b38c0 0x7fe524150690 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7fe50c002f30 tx=0x7fe50c015000).stop 2021-04-03T03:15:26.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 shutdown_connections 2021-04-03T03:15:26.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe508057030 0x7fe5080594f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe524153440 0x7fe5240aaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe524080a40 0x7fe5240b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 --2- 172.21.2.111:0/1628042526 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5240b38c0 0x7fe524150690 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 >> 172.21.2.111:0/1628042526 conn(0x7fe524061610 msgr2=0x7fe5240b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 shutdown_connections 2021-04-03T03:15:26.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.912+0000 7fe52893c700 1 -- 172.21.2.111:0/1628042526 wait complete. 2021-04-03T03:15:26.919 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:15:26.923 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:15:26.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfede6700 1 Processor -- start 2021-04-03T03:15:26.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfede6700 1 -- start start 2021-04-03T03:15:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfede6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7facf8080c60 0x7facf8025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfede6700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf8081090 con 0x7facf8153440 2021-04-03T03:15:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfede6700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf80b3ca0 con 0x7facf80b38c0 2021-04-03T03:15:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfede6700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf8025970 con 0x7facf8080c60 2021-04-03T03:15:26.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7fad00260700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7facf8080c60 0x7facf8025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfdde4700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7facf8153440 0x7facf80af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55382/0 (socket says 172.21.2.111:55382) 2021-04-03T03:15:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.976+0000 7facfdde4700 1 -- 172.21.2.111:0/4108432513 learned_addr learned my addr 172.21.2.111:0/4108432513 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 -- 172.21.2.111:0/4108432513 >> v1:172.21.2.115:6789/0 conn(0x7facf80b38c0 legacy=0x7facf8080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:26.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 -- 172.21.2.111:0/4108432513 >> v1:172.21.2.111:6789/0 conn(0x7facf8153440 legacy=0x7facf80af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 -- 172.21.2.111:0/4108432513 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8025b90 con 0x7facf8080c60 2021-04-03T03:15:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 --2- 172.21.2.111:0/4108432513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8080c60 0x7facf8025480 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7facec001260 tx=0x7facf00091d0).ready entity=mon.0 client_cookie=2befdaf570619bd5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfd5e3700 1 -- 172.21.2.111:0/4108432513 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf000b910 con 0x7facf8080c60 2021-04-03T03:15:26.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfd5e3700 1 -- 172.21.2.111:0/4108432513 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf000ba70 con 0x7facf8080c60 2021-04-03T03:15:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfd5e3700 1 -- 172.21.2.111:0/4108432513 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf000bbe0 con 0x7facf8080c60 2021-04-03T03:15:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/4108432513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8080c60 msgr2=0x7facf8025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 --2- 172.21.2.111:0/4108432513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8080c60 0x7facf8025480 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7facec001260 tx=0x7facf00091d0).stop 2021-04-03T03:15:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/4108432513 shutdown_connections 2021-04-03T03:15:26.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 --2- 172.21.2.111:0/4108432513 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8080c60 0x7facf8025480 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/4108432513 >> 172.21.2.111:0/4108432513 conn(0x7facf8061610 msgr2=0x7facf8061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/4108432513 shutdown_connections 2021-04-03T03:15:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/4108432513 wait complete. 2021-04-03T03:15:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 Processor -- start 2021-04-03T03:15:26.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- start start 2021-04-03T03:15:26.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7facf8080c60 0x7facf8064610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7facf80b38c0 0x7facf80acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8153440 0x7facf8066b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf8063f40 con 0x7facf8153440 2021-04-03T03:15:26.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf8065460 con 0x7facf8080c60 2021-04-03T03:15:26.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf8080530 con 0x7facf80b38c0 2021-04-03T03:15:26.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8153440 0x7facf8066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8153440 0x7facf8066b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54594/0 (socket says 172.21.2.111:54594) 2021-04-03T03:15:26.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 -- 172.21.2.111:0/3033702447 learned_addr learned my addr 172.21.2.111:0/3033702447 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:26.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 -- 172.21.2.111:0/3033702447 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7facf80b38c0 msgr2=0x7facf80acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7facf80b38c0 0x7facf80acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 -- 172.21.2.111:0/3033702447 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7facf8080c60 msgr2=0x7facf8064610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:26.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7facf8080c60 0x7facf8064610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:26.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 -- 172.21.2.111:0/3033702447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8080790 con 0x7facf8153440 2021-04-03T03:15:26.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7fad00260700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8153440 0x7facf8066b30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7facec001260 tx=0x7facf0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7faceaffd700 1 -- 172.21.2.111:0/3033702447 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf00205f0 con 0x7facf8153440 2021-04-03T03:15:26.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7faceaffd700 1 -- 172.21.2.111:0/3033702447 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0020750 con 0x7facf8153440 2021-04-03T03:15:26.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7faceaffd700 1 -- 172.21.2.111:0/3033702447 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf0020ab0 con 0x7facf8153440 2021-04-03T03:15:26.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf8064170 con 0x7facf8153440 2021-04-03T03:15:26.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf8064b00 con 0x7facf8153440 2021-04-03T03:15:26.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.980+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf8057660 con 0x7facf8153440 2021-04-03T03:15:26.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.984+0000 7faceaffd700 1 -- 172.21.2.111:0/3033702447 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7facf0027030 con 0x7facf8153440 2021-04-03T03:15:26.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.984+0000 7faceaffd700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7face0056ee0 0x7face00593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:26.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.984+0000 7faceaffd700 1 -- 172.21.2.111:0/3033702447 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7facf00717e0 con 0x7facf8153440 2021-04-03T03:15:26.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.984+0000 7facfe5e5700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7face0056ee0 0x7face00593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:26.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.984+0000 7facfe5e5700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7face0056ee0 0x7face00593a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7facf4002f50 tx=0x7facf4006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:26.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:26.984+0000 7faceaffd700 1 -- 172.21.2.111:0/3033702447 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7facf0046ae0 con 0x7facf8153440 2021-04-03T03:15:27.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.112+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7facf8064300 con 0x7facf8153440 2021-04-03T03:15:27.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.116+0000 7faceaffd700 1 -- 172.21.2.111:0/3033702447 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7facf00150e0 con 0x7facf8153440 2021-04-03T03:15:27.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.116+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7face0056ee0 msgr2=0x7face00593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.116+0000 7facfede6700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7face0056ee0 0x7face00593a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7facf4002f50 tx=0x7facf4006990).stop 2021-04-03T03:15:27.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.116+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8153440 msgr2=0x7facf8066b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.116+0000 7facfede6700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8153440 0x7facf8066b30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7facec001260 tx=0x7facf0016000).stop 2021-04-03T03:15:27.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.116+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 shutdown_connections 2021-04-03T03:15:27.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.116+0000 7facfede6700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8153440 0x7facf8066b30 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.120+0000 7facfede6700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7face0056ee0 0x7face00593a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.120+0000 7facfede6700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7facf8080c60 0x7facf8064610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.120+0000 7facfede6700 1 --2- 172.21.2.111:0/3033702447 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7facf80b38c0 0x7facf80acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.120+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 >> 172.21.2.111:0/3033702447 conn(0x7facf8061610 msgr2=0x7facf80af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:27.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.120+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 shutdown_connections 2021-04-03T03:15:27.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.120+0000 7facfede6700 1 -- 172.21.2.111:0/3033702447 wait complete. 2021-04-03T03:15:27.127 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:15:27.130 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:15:27.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b5e7f700 1 Processor -- start 2021-04-03T03:15:27.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b5e7f700 1 -- start start 2021-04-03T03:15:27.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b5e7f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd1a8046ce0 0x7fd1a803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b5e7f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd1a8042380 0x7fd1a80f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b5e7f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a8041240 con 0x7fd1a803d350 2021-04-03T03:15:27.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b5e7f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a80f1850 con 0x7fd1a8042380 2021-04-03T03:15:27.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b5e7f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a80470c0 con 0x7fd1a8046ce0 2021-04-03T03:15:27.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b4e7d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd1a803d350 0x7fd1a8041c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55396/0 (socket says 172.21.2.111:55396) 2021-04-03T03:15:27.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b4e7d700 1 -- 172.21.2.111:0/3017687226 learned_addr learned my addr 172.21.2.111:0/3017687226 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:27.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b72f9700 1 --2- 172.21.2.111:0/3017687226 >> v2:172.21.2.111:3300/0 conn(0x7fd1a8042380 0x7fd1a80f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b72f9700 1 -- 172.21.2.111:0/3017687226 >> v2:172.21.2.115:3300/0 conn(0x7fd1a8046ce0 msgr2=0x7fd1a803f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:27.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b72f9700 1 --2- 172.21.2.111:0/3017687226 >> v2:172.21.2.115:3300/0 conn(0x7fd1a8046ce0 0x7fd1a803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b72f9700 1 -- 172.21.2.111:0/3017687226 >> v1:172.21.2.111:6789/0 conn(0x7fd1a803d350 legacy=0x7fd1a8041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b72f9700 1 -- 172.21.2.111:0/3017687226 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80400d0 con 0x7fd1a8042380 2021-04-03T03:15:27.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1a3fff700 1 -- 172.21.2.111:0/3017687226 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675869477 0 0) 0x7fd1a8041240 con 0x7fd1a803d350 2021-04-03T03:15:27.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.184+0000 7fd1b72f9700 1 --2- 172.21.2.111:0/3017687226 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a8042380 0x7fd1a80f51b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fd1b006b5d0 tx=0x7fd1b005ad70).ready entity=mon.0 client_cookie=cd189c4ee11f0bf server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a3fff700 1 -- 172.21.2.111:0/3017687226 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1b005bda0 con 0x7fd1a8042380 2021-04-03T03:15:27.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a3fff700 1 -- 172.21.2.111:0/3017687226 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b005df60 con 0x7fd1a8042380 2021-04-03T03:15:27.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a3fff700 1 -- 172.21.2.111:0/3017687226 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1b00699f0 con 0x7fd1a8042380 2021-04-03T03:15:27.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/3017687226 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a8042380 msgr2=0x7fd1a80f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/3017687226 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a8042380 0x7fd1a80f51b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fd1b006b5d0 tx=0x7fd1b005ad70).stop 2021-04-03T03:15:27.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/3017687226 shutdown_connections 2021-04-03T03:15:27.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/3017687226 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a8042380 0x7fd1a80f51b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/3017687226 >> v2:172.21.2.115:3300/0 conn(0x7fd1a8046ce0 0x7fd1a803f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/3017687226 >> 172.21.2.111:0/3017687226 conn(0x7fd1a800d0c0 msgr2=0x7fd1a80462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:27.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/3017687226 shutdown_connections 2021-04-03T03:15:27.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/3017687226 wait complete. 2021-04-03T03:15:27.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 Processor -- start 2021-04-03T03:15:27.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- start start 2021-04-03T03:15:27.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a803d350 0x7fd1a80f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd1a8042380 0x7fd1a80f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd1a8046ce0 0x7fd1a80f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80f8c10 con 0x7fd1a803d350 2021-04-03T03:15:27.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80f56f0 con 0x7fd1a8042380 2021-04-03T03:15:27.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80f9040 con 0x7fd1a8046ce0 2021-04-03T03:15:27.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a803d350 0x7fd1a80f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a803d350 0x7fd1a80f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54610/0 (socket says 172.21.2.111:54610) 2021-04-03T03:15:27.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 -- 172.21.2.111:0/1375943186 learned_addr learned my addr 172.21.2.111:0/1375943186 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:27.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b4e7d700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd1a8042380 0x7fd1a80f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b72f9700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd1a8046ce0 0x7fd1a80f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 -- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd1a8046ce0 msgr2=0x7fd1a80f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd1a8046ce0 0x7fd1a80f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 -- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd1a8042380 msgr2=0x7fd1a80f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd1a8042380 0x7fd1a80f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 -- 172.21.2.111:0/1375943186 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a803c0a0 con 0x7fd1a803d350 2021-04-03T03:15:27.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b567e700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a803d350 0x7fd1a80f8720 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fd1a4002f30 tx=0x7fd1a4006e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a1ffb700 1 -- 172.21.2.111:0/1375943186 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1a400d4a0 con 0x7fd1a803d350 2021-04-03T03:15:27.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a1ffb700 1 -- 172.21.2.111:0/1375943186 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a4000e90 con 0x7fd1a803d350 2021-04-03T03:15:27.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a803bca0 con 0x7fd1a803d350 2021-04-03T03:15:27.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a1ffb700 1 -- 172.21.2.111:0/1375943186 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1a400d7f0 con 0x7fd1a803d350 2021-04-03T03:15:27.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a80f5e00 con 0x7fd1a803d350 2021-04-03T03:15:27.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a80f5860 con 0x7fd1a803d350 2021-04-03T03:15:27.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a1ffb700 1 -- 172.21.2.111:0/1375943186 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd1a400f070 con 0x7fd1a803d350 2021-04-03T03:15:27.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a1ffb700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd190056e90 0x7fd190059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1a1ffb700 1 -- 172.21.2.111:0/1375943186 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd1a406f7b0 con 0x7fd1a803d350 2021-04-03T03:15:27.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.188+0000 7fd1b4e7d700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd190056e90 0x7fd190059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.192+0000 7fd1b4e7d700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd190056e90 0x7fd190059350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fd1ac00d000 tx=0x7fd1ac0079e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.192+0000 7fd1a1ffb700 1 -- 172.21.2.111:0/1375943186 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd1a80f5860 con 0x7fd1a803d350 2021-04-03T03:15:27.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.320+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd1a803be60 con 0x7fd1a803d350 2021-04-03T03:15:27.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.324+0000 7fd1a1ffb700 1 -- 172.21.2.111:0/1375943186 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd1a4018030 con 0x7fd1a803d350 2021-04-03T03:15:27.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd190056e90 msgr2=0x7fd190059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd190056e90 0x7fd190059350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fd1ac00d000 tx=0x7fd1ac0079e0).stop 2021-04-03T03:15:27.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a803d350 msgr2=0x7fd1a80f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a803d350 0x7fd1a80f8720 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fd1a4002f30 tx=0x7fd1a4006e00).stop 2021-04-03T03:15:27.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 shutdown_connections 2021-04-03T03:15:27.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd1a8046ce0 0x7fd1a80f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd1a803d350 0x7fd1a80f8720 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd190056e90 0x7fd190059350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 --2- 172.21.2.111:0/1375943186 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd1a8042380 0x7fd1a80f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 >> 172.21.2.111:0/1375943186 conn(0x7fd1a800d0c0 msgr2=0x7fd1a80f1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:27.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 shutdown_connections 2021-04-03T03:15:27.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.328+0000 7fd1b5e7f700 1 -- 172.21.2.111:0/1375943186 wait complete. 2021-04-03T03:15:27.335 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:15:27.339 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:27.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 Processor -- start 2021-04-03T03:15:27.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- start start 2021-04-03T03:15:27.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa17c00bca0 0x7fa17c02b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa17c0d5370 0x7fa17c0d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c00c080 con 0x7fa17c042e70 2021-04-03T03:15:27.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c02ad40 con 0x7fa17c0d5370 2021-04-03T03:15:27.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c029740 con 0x7fa17c00bca0 2021-04-03T03:15:27.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa17c0d5370 0x7fa17c0d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa17c0d5370 0x7fa17c0d57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54616/0 (socket says 172.21.2.111:54616) 2021-04-03T03:15:27.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 -- 172.21.2.111:0/4130588166 learned_addr learned my addr 172.21.2.111:0/4130588166 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:27.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 -- 172.21.2.111:0/4130588166 >> v2:172.21.2.115:3300/0 conn(0x7fa17c00bca0 msgr2=0x7fa17c02b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:27.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189292700 1 --2- 172.21.2.111:0/4130588166 >> v2:172.21.2.115:3300/0 conn(0x7fa17c00bca0 0x7fa17c02b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 --2- 172.21.2.111:0/4130588166 >> v2:172.21.2.115:3300/0 conn(0x7fa17c00bca0 0x7fa17c02b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 -- 172.21.2.111:0/4130588166 >> v1:172.21.2.111:6789/0 conn(0x7fa17c042e70 legacy=0x7fa17c029e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 -- 172.21.2.111:0/4130588166 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0299e0 con 0x7fa17c0d5370 2021-04-03T03:15:27.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa173fff700 1 -- 172.21.2.111:0/4130588166 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864793481 0 0) 0x7fa17c00c080 con 0x7fa17c042e70 2021-04-03T03:15:27.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa189a93700 1 --2- 172.21.2.111:0/4130588166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0d57e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fa18406b5d0 tx=0x7fa18405ad70).ready entity=mon.0 client_cookie=53f568ab598160cb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa173fff700 1 -- 172.21.2.111:0/4130588166 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18405bda0 con 0x7fa17c0d5370 2021-04-03T03:15:27.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa173fff700 1 -- 172.21.2.111:0/4130588166 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18405df60 con 0x7fa17c0d5370 2021-04-03T03:15:27.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa173fff700 1 -- 172.21.2.111:0/4130588166 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1840699f0 con 0x7fa17c0d5370 2021-04-03T03:15:27.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- 172.21.2.111:0/4130588166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 msgr2=0x7fa17c0d57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 --2- 172.21.2.111:0/4130588166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0d57e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fa18406b5d0 tx=0x7fa18405ad70).stop 2021-04-03T03:15:27.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- 172.21.2.111:0/4130588166 shutdown_connections 2021-04-03T03:15:27.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 --2- 172.21.2.111:0/4130588166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0d57e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 --2- 172.21.2.111:0/4130588166 >> v2:172.21.2.115:3300/0 conn(0x7fa17c00bca0 0x7fa17c02b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- 172.21.2.111:0/4130588166 >> 172.21.2.111:0/4130588166 conn(0x7fa17c0484d0 msgr2=0x7fa17c0488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:27.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- 172.21.2.111:0/4130588166 shutdown_connections 2021-04-03T03:15:27.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- 172.21.2.111:0/4130588166 wait complete. 2021-04-03T03:15:27.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 Processor -- start 2021-04-03T03:15:27.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.392+0000 7fa18a294700 1 -- start start 2021-04-03T03:15:27.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17c00bca0 0x7fa17c040a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa17c042e70 0x7fa17c0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c03f7e0 con 0x7fa17c0d5370 2021-04-03T03:15:27.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c01f760 con 0x7fa17c042e70 2021-04-03T03:15:27.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c01fa00 con 0x7fa17c00bca0 2021-04-03T03:15:27.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54624/0 (socket says 172.21.2.111:54624) 2021-04-03T03:15:27.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 -- 172.21.2.111:0/2153926331 learned_addr learned my addr 172.21.2.111:0/2153926331 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:27.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 -- 172.21.2.111:0/2153926331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17c00bca0 msgr2=0x7fa17c040a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17c00bca0 0x7fa17c040a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 -- 172.21.2.111:0/2153926331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa17c042e70 msgr2=0x7fa17c0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa188a91700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa17c042e70 0x7fa17c0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa17c042e70 0x7fa17c0cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 -- 172.21.2.111:0/2153926331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c03fc10 con 0x7fa17c0d5370 2021-04-03T03:15:27.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189a93700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0cfbd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7fa18406b5d0 tx=0x7fa184065000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa1727fc700 1 -- 172.21.2.111:0/2153926331 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1840441b0 con 0x7fa17c0d5370 2021-04-03T03:15:27.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa1727fc700 1 -- 172.21.2.111:0/2153926331 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18405f2c0 con 0x7fa17c0d5370 2021-04-03T03:15:27.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa1727fc700 1 -- 172.21.2.111:0/2153926331 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18405f620 con 0x7fa17c0d5370 2021-04-03T03:15:27.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c03f3e0 con 0x7fa17c0d5370 2021-04-03T03:15:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c0d0250 con 0x7fa17c0d5370 2021-04-03T03:15:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c03f950 con 0x7fa17c0d5370 2021-04-03T03:15:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa1727fc700 1 -- 172.21.2.111:0/2153926331 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa18408c020 con 0x7fa17c0d5370 2021-04-03T03:15:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa1727fc700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa164057070 0x7fa164059530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa1727fc700 1 -- 172.21.2.111:0/2153926331 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa1840e07a0 con 0x7fa17c0d5370 2021-04-03T03:15:27.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189292700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa164057070 0x7fa164059530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.396+0000 7fa189292700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa164057070 0x7fa164059530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fa180007000 tx=0x7fa180008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.400+0000 7fa1727fc700 1 -- 172.21.2.111:0/2153926331 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa17c03f950 con 0x7fa17c0d5370 2021-04-03T03:15:27.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa17c03f920 con 0x7fa17c0d5370 2021-04-03T03:15:27.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa1727fc700 1 -- 172.21.2.111:0/2153926331 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fa1840b5b90 con 0x7fa17c0d5370 2021-04-03T03:15:27.530 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:27.530 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2065583567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2065583567},{"type":"v1","addr":"172.21.2.115:6833","nonce":2065583567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:25.328682+0000","modified":"2021-04-03T03:15:26.331659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/485806052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":485806052},{"type":"v1","addr":"172.21.2.111:6835","nonce":485806052}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-04-03T03:15:27.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa164057070 msgr2=0x7fa164059530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa164057070 0x7fa164059530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fa180007000 tx=0x7fa180008040).stop 2021-04-03T03:15:27.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 msgr2=0x7fa17c0cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0cfbd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7fa18406b5d0 tx=0x7fa184065000).stop 2021-04-03T03:15:27.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 shutdown_connections 2021-04-03T03:15:27.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17c0d5370 0x7fa17c0cfbd0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17c00bca0 0x7fa17c040a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa164057070 0x7fa164059530 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 --2- 172.21.2.111:0/2153926331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa17c042e70 0x7fa17c0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 >> 172.21.2.111:0/2153926331 conn(0x7fa17c0484d0 msgr2=0x7fa17c044f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:27.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 shutdown_connections 2021-04-03T03:15:27.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.524+0000 7fa18a294700 1 -- 172.21.2.111:0/2153926331 wait complete. 2021-04-03T03:15:27.534 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 33 2021-04-03T03:15:27.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:15:25.328682+0000', 'modified': '2021-04-03T03:15:26.331659+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6573': {'gid': 6573, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/485806052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 485806052}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 485806052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:15:27.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:15:27.740 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:15:27.812 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6573,"want_state":"up:active","state":"up:active","rank_uptime":2.4756359379999999,"mdsmap_epoch":33,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":6.6142264480000001} 2021-04-03T03:15:27.813 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6573, "mdsmap_epoch": 33, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 2.475635938, "state": "up:active", "uptime": 6.614226448, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:15:27.813 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:27.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64f59e700 1 Processor -- start 2021-04-03T03:15:27.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64f59e700 1 -- start start 2021-04-03T03:15:27.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 0x7fb6480d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64f59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb648042300 0x7fb648043920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6480d4ef0 con 0x7fb648045690 2021-04-03T03:15:27.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64f59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb648046f50 con 0x7fb648042300 2021-04-03T03:15:27.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480d4020 con 0x7fb648046b70 2021-04-03T03:15:27.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64ed9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fb648045690 0x7fb6480d33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55424/0 (socket says 172.21.2.111:55424) 2021-04-03T03:15:27.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.864+0000 7fb64ed9d700 1 -- 172.21.2.111:0/3570217200 learned_addr learned my addr 172.21.2.111:0/3570217200 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:27.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296920906 0 0) 0x7fb6480d4ef0 con 0x7fb648045690 2021-04-03T03:15:27.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb630003240 con 0x7fb648045690 2021-04-03T03:15:27.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 --2- 172.21.2.111:0/3570217200 >> v2:172.21.2.115:3300/0 conn(0x7fb648042300 0x7fb648043920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 0x7fb6480d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3355321991 0 0) 0x7fb630003240 con 0x7fb648045690 2021-04-03T03:15:27.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 msgr2=0x7fb6480d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:27.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 0x7fb6480d4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 >> v2:172.21.2.115:3300/0 conn(0x7fb648042300 msgr2=0x7fb648043920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:27.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 --2- 172.21.2.111:0/3570217200 >> v2:172.21.2.115:3300/0 conn(0x7fb648042300 0x7fb648043920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480d42c0 con 0x7fb648045690 2021-04-03T03:15:27.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fb6500450f0 con 0x7fb648045690 2021-04-03T03:15:27.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 >> v1:172.21.2.111:6789/0 conn(0x7fb648045690 legacy=0x7fb6480d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb630003730 0x7fb630003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 0x7fb6300044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6300049d0 0x7fb630004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb648046f50 con 0x7fb630003730 2021-04-03T03:15:27.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480d4020 con 0x7fb6300049d0 2021-04-03T03:15:27.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb630004130 con 0x7fb648046b70 2021-04-03T03:15:27.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb630003730 0x7fb630003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 0x7fb6300044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64ed9d700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6300049d0 0x7fb630004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 -- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 msgr2=0x7fb6300044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:27.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648046b70 0x7fb6300044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 -- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6300049d0 msgr2=0x7fb630004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:27.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6300049d0 0x7fb630004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 -- 172.21.2.111:0/3570217200 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb630003240 con 0x7fb630003730 2021-04-03T03:15:27.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64e59c700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb630003730 0x7fb630003b50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7fb640007000 tx=0x7fb640008040).ready entity=mon.0 client_cookie=63fe0e7fe8e0320 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb64000a880 con 0x7fb630003730 2021-04-03T03:15:27.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64000c030 con 0x7fb630003730 2021-04-03T03:15:27.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64d59a700 1 -- 172.21.2.111:0/3570217200 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb64000abd0 con 0x7fb630003730 2021-04-03T03:15:27.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- 172.21.2.111:0/3570217200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb630003730 msgr2=0x7fb630003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 --2- 172.21.2.111:0/3570217200 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb630003730 0x7fb630003b50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7fb640007000 tx=0x7fb640008040).stop 2021-04-03T03:15:27.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64ed9d700 1 -- 172.21.2.111:0/3570217200 reap_dead start 2021-04-03T03:15:27.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- 172.21.2.111:0/3570217200 shutdown_connections 2021-04-03T03:15:27.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- 172.21.2.111:0/3570217200 >> 172.21.2.111:0/3570217200 conn(0x7fb64804a6e0 msgr2=0x7fb64804aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:27.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- 172.21.2.111:0/3570217200 shutdown_connections 2021-04-03T03:15:27.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- 172.21.2.111:0/3570217200 wait complete. 2021-04-03T03:15:27.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 Processor -- start 2021-04-03T03:15:27.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- start start 2021-04-03T03:15:27.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648005cd0 0x7fb64803e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6480cc0c0 0x7fb6480d7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6480d68a0 0x7fb6480d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480d8030 con 0x7fb6480cc0c0 2021-04-03T03:15:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64803d9b0 con 0x7fb6480d68a0 2021-04-03T03:15:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480c6c80 con 0x7fb648005cd0 2021-04-03T03:15:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6480cc0c0 0x7fb6480d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6480cc0c0 0x7fb6480d7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54642/0 (socket says 172.21.2.111:54642) 2021-04-03T03:15:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 -- 172.21.2.111:0/1481394564 learned_addr learned my addr 172.21.2.111:0/1481394564 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 -- 172.21.2.111:0/1481394564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648005cd0 msgr2=0x7fb64803e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648005cd0 0x7fb64803e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 -- 172.21.2.111:0/1481394564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6480d68a0 msgr2=0x7fb6480d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6480d68a0 0x7fb6480d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 -- 172.21.2.111:0/1481394564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64803e570 con 0x7fb6480cc0c0 2021-04-03T03:15:27.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.868+0000 7fb64dd9b700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6480cc0c0 0x7fb6480d7650 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fb63800b000 tx=0x7fb638008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb6477fe700 1 -- 172.21.2.111:0/1481394564 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb63800a9f0 con 0x7fb6480cc0c0 2021-04-03T03:15:27.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb6477fe700 1 -- 172.21.2.111:0/1481394564 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63800ab50 con 0x7fb6480cc0c0 2021-04-03T03:15:27.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb6477fe700 1 -- 172.21.2.111:0/1481394564 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb638013450 con 0x7fb6480cc0c0 2021-04-03T03:15:27.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6480060b0 con 0x7fb6480cc0c0 2021-04-03T03:15:27.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6480d53e0 con 0x7fb6480cc0c0 2021-04-03T03:15:27.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb648004da0 con 0x7fb6480cc0c0 2021-04-03T03:15:27.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb6477fe700 1 -- 172.21.2.111:0/1481394564 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb6380135b0 con 0x7fb6480cc0c0 2021-04-03T03:15:27.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb6477fe700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb634056f60 0x7fb634059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:27.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb6477fe700 1 -- 172.21.2.111:0/1481394564 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb6380708d0 con 0x7fb6480cc0c0 2021-04-03T03:15:27.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb64e59c700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb634056f60 0x7fb634059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:27.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb64e59c700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb634056f60 0x7fb634059420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fb640007000 tx=0x7fb640008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:27.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.872+0000 7fb6477fe700 1 -- 172.21.2.111:0/1481394564 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb638045c60 con 0x7fb6480cc0c0 2021-04-03T03:15:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb64804b560 con 0x7fb6480cc0c0 2021-04-03T03:15:27.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb6477fe700 1 -- 172.21.2.111:0/1481394564 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fb638017020 con 0x7fb6480cc0c0 2021-04-03T03:15:27.970 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:27.971 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":84,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:26.331795+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:23.240089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:24.288628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb634056f60 msgr2=0x7fb634059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb634056f60 0x7fb634059420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fb640007000 tx=0x7fb640008040).stop 2021-04-03T03:15:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6480cc0c0 msgr2=0x7fb6480d7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:27.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6480cc0c0 0x7fb6480d7650 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fb63800b000 tx=0x7fb638008040).stop 2021-04-03T03:15:27.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 shutdown_connections 2021-04-03T03:15:27.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb634056f60 0x7fb634059420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb6480d68a0 0x7fb6480d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb648005cd0 0x7fb64803e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 --2- 172.21.2.111:0/1481394564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb6480cc0c0 0x7fb6480d7650 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.964+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 >> 172.21.2.111:0/1481394564 conn(0x7fb64804a6e0 msgr2=0x7fb648043bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.968+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 shutdown_connections 2021-04-03T03:15:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:27.968+0000 7fb64f59e700 1 -- 172.21.2.111:0/1481394564 wait complete. 2021-04-03T03:15:27.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:15:23.240089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:15:24.288628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:15:27.979 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:28.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 Processor -- start 2021-04-03T03:15:28.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 -- start start 2021-04-03T03:15:28.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd884144ef0 0x7fd884147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8840607d0 con 0x7fd8840add50 2021-04-03T03:15:28.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88414b090 con 0x7fd8840603f0 2021-04-03T03:15:28.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88414b330 con 0x7fd884144ef0 2021-04-03T03:15:28.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd8837fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd884144ef0 0x7fd884147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd8837fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd884144ef0 0x7fd884147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54648/0 (socket says 172.21.2.111:54648) 2021-04-03T03:15:28.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd8837fe700 1 -- 172.21.2.111:0/2910058616 learned_addr learned my addr 172.21.2.111:0/2910058616 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd8837fe700 1 -- 172.21.2.111:0/2910058616 >> v1:172.21.2.115:6789/0 conn(0x7fd8840603f0 legacy=0x7fd8841447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:28.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd8837fe700 1 -- 172.21.2.111:0/2910058616 >> v1:172.21.2.111:6789/0 conn(0x7fd8840add50 legacy=0x7fd884147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd8837fe700 1 -- 172.21.2.111:0/2910058616 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88414b5d0 con 0x7fd884144ef0 2021-04-03T03:15:28.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd8837fe700 1 --2- 172.21.2.111:0/2910058616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd884144ef0 0x7fd884147e50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fd8840b5600 tx=0x7fd87c009270).ready entity=mon.0 client_cookie=a7d706a40c9090c2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd881ffb700 1 -- 172.21.2.111:0/2910058616 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87c00bbc0 con 0x7fd884144ef0 2021-04-03T03:15:28.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd881ffb700 1 -- 172.21.2.111:0/2910058616 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00bd20 con 0x7fd884144ef0 2021-04-03T03:15:28.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd881ffb700 1 -- 172.21.2.111:0/2910058616 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87c00be90 con 0x7fd884144ef0 2021-04-03T03:15:28.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 -- 172.21.2.111:0/2910058616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd884144ef0 msgr2=0x7fd884147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.032+0000 7fd883fff700 1 --2- 172.21.2.111:0/2910058616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd884144ef0 0x7fd884147e50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fd8840b5600 tx=0x7fd87c009270).stop 2021-04-03T03:15:28.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- 172.21.2.111:0/2910058616 shutdown_connections 2021-04-03T03:15:28.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 --2- 172.21.2.111:0/2910058616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd884144ef0 0x7fd884147e50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- 172.21.2.111:0/2910058616 >> 172.21.2.111:0/2910058616 conn(0x7fd8840b61c0 msgr2=0x7fd88405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- 172.21.2.111:0/2910058616 shutdown_connections 2021-04-03T03:15:28.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- 172.21.2.111:0/2910058616 wait complete. 2021-04-03T03:15:28.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 Processor -- start 2021-04-03T03:15:28.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- start start 2021-04-03T03:15:28.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8840603f0 0x7fd884063f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8840add50 0x7fd884155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd884144ef0 0x7fd884156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884157890 con 0x7fd8840603f0 2021-04-03T03:15:28.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884064780 con 0x7fd884144ef0 2021-04-03T03:15:28.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd884064470 con 0x7fd8840add50 2021-04-03T03:15:28.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8840603f0 0x7fd884063f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8840603f0 0x7fd884063f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54656/0 (socket says 172.21.2.111:54656) 2021-04-03T03:15:28.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 -- 172.21.2.111:0/3897752925 learned_addr learned my addr 172.21.2.111:0/3897752925 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 -- 172.21.2.111:0/3897752925 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8840add50 msgr2=0x7fd884155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8840add50 0x7fd884155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:28.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 -- 172.21.2.111:0/3897752925 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd884144ef0 msgr2=0x7fd884156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd884144ef0 0x7fd884156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:28.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 -- 172.21.2.111:0/3897752925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884156850 con 0x7fd8840603f0 2021-04-03T03:15:28.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd882ffd700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8840603f0 0x7fd884063f80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fd86c002f70 tx=0x7fd86c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd87bfff700 1 -- 172.21.2.111:0/3897752925 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd86c00d570 con 0x7fd8840603f0 2021-04-03T03:15:28.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd87bfff700 1 -- 172.21.2.111:0/3897752925 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c00d6d0 con 0x7fd8840603f0 2021-04-03T03:15:28.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8841574d0 con 0x7fd8840603f0 2021-04-03T03:15:28.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd87bfff700 1 -- 172.21.2.111:0/3897752925 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd86c00da30 con 0x7fd8840603f0 2021-04-03T03:15:28.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8840a9b30 con 0x7fd8840603f0 2021-04-03T03:15:28.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd884057660 con 0x7fd8840603f0 2021-04-03T03:15:28.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd87bfff700 1 -- 172.21.2.111:0/3897752925 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd86c00f070 con 0x7fd8840603f0 2021-04-03T03:15:28.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd87bfff700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd868056f70 0x7fd868059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd87bfff700 1 -- 172.21.2.111:0/3897752925 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd86c06fd40 con 0x7fd8840603f0 2021-04-03T03:15:28.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd8827fc700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd868056f70 0x7fd868059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.036+0000 7fd8827fc700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd868056f70 0x7fd868059430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fd8840ae520 tx=0x7fd8740077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.040+0000 7fd87bfff700 1 -- 172.21.2.111:0/3897752925 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd86c045130 con 0x7fd8840603f0 2021-04-03T03:15:28.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd8840648f0 con 0x7fd8840603f0 2021-04-03T03:15:28.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd87bfff700 1 -- 172.21.2.111:0/3897752925 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd86c048750 con 0x7fd8840603f0 2021-04-03T03:15:28.137 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:28.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd868056f70 msgr2=0x7fd868059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd868056f70 0x7fd868059430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fd8840ae520 tx=0x7fd8740077f0).stop 2021-04-03T03:15:28.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8840603f0 msgr2=0x7fd884063f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8840603f0 0x7fd884063f80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fd86c002f70 tx=0x7fd86c006dd0).stop 2021-04-03T03:15:28.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 shutdown_connections 2021-04-03T03:15:28.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd868056f70 0x7fd868059430 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd884144ef0 0x7fd884156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:28.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd8840603f0 0x7fd884063f80 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 --2- 172.21.2.111:0/3897752925 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd8840add50 0x7fd884155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:28.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 >> 172.21.2.111:0/3897752925 conn(0x7fd8840b61c0 msgr2=0x7fd884144060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 shutdown_connections 2021-04-03T03:15:28.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.132+0000 7fd883fff700 1 -- 172.21.2.111:0/3897752925 wait complete. 2021-04-03T03:15:28.145 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:28.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.196+0000 7f684bfff700 1 Processor -- start 2021-04-03T03:15:28.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.196+0000 7f684bfff700 1 -- start start 2021-04-03T03:15:28.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.196+0000 7f684bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0603f0 0x7f684c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.196+0000 7f684bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f684c0b17b0 0x7f684c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.196+0000 7f684bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0607d0 con 0x7f684c0add50 2021-04-03T03:15:28.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.196+0000 7f684bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c1451c0 con 0x7f684c0b17b0 2021-04-03T03:15:28.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.196+0000 7f684bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c14b090 con 0x7f684c0603f0 2021-04-03T03:15:28.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684b7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f684c0b17b0 0x7f684c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684affd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0603f0 0x7f684c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684b7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f684c0b17b0 0x7f684c144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:44374/0 (socket says 172.21.2.111:44374) 2021-04-03T03:15:28.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684affd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0603f0 0x7f684c1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:38010/0 (socket says 172.21.2.111:38010) 2021-04-03T03:15:28.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684b7fe700 1 -- 172.21.2.111:0/4216246512 learned_addr learned my addr 172.21.2.111:0/4216246512 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684affd700 1 -- 172.21.2.111:0/4216246512 >> v2:172.21.2.115:3300/0 conn(0x7f684c0b17b0 msgr2=0x7f684c144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:28.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684affd700 1 --2- 172.21.2.111:0/4216246512 >> v2:172.21.2.115:3300/0 conn(0x7f684c0b17b0 0x7f684c144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684affd700 1 -- 172.21.2.111:0/4216246512 >> v1:172.21.2.115:6789/0 conn(0x7f684c0add50 legacy=0x7f684c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684affd700 1 -- 172.21.2.111:0/4216246512 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c14b330 con 0x7f684c0603f0 2021-04-03T03:15:28.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684affd700 1 --2- 172.21.2.111:0/4216246512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0603f0 0x7f684c1447e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f683c002f30 tx=0x7f683c006dd0).ready entity=mon.2 client_cookie=d1bea264ae37c62f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f6849ffb700 1 -- 172.21.2.111:0/4216246512 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f683c009760 con 0x7f684c0603f0 2021-04-03T03:15:28.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f6849ffb700 1 -- 172.21.2.111:0/4216246512 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c0098c0 con 0x7f684c0603f0 2021-04-03T03:15:28.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f6849ffb700 1 -- 172.21.2.111:0/4216246512 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f683c009c20 con 0x7f684c0603f0 2021-04-03T03:15:28.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- 172.21.2.111:0/4216246512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0603f0 msgr2=0x7f684c1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 --2- 172.21.2.111:0/4216246512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0603f0 0x7f684c1447e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f683c002f30 tx=0x7f683c006dd0).stop 2021-04-03T03:15:28.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- 172.21.2.111:0/4216246512 shutdown_connections 2021-04-03T03:15:28.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 --2- 172.21.2.111:0/4216246512 >> v2:172.21.2.115:3300/0 conn(0x7f684c0b17b0 0x7f684c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 --2- 172.21.2.111:0/4216246512 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0603f0 0x7f684c1447e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- 172.21.2.111:0/4216246512 >> 172.21.2.111:0/4216246512 conn(0x7f684c0b61c0 msgr2=0x7f684c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- 172.21.2.111:0/4216246512 shutdown_connections 2021-04-03T03:15:28.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- 172.21.2.111:0/4216246512 wait complete. 2021-04-03T03:15:28.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 Processor -- start 2021-04-03T03:15:28.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- start start 2021-04-03T03:15:28.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f684c0603f0 0x7f684c0abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0add50 0x7f684c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f684c0b17b0 0x7f684c0a8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0a8ec0 con 0x7f684c0b17b0 2021-04-03T03:15:28.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.200+0000 7f684bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0a90f0 con 0x7f684c0603f0 2021-04-03T03:15:28.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0a81d0 con 0x7f684c0add50 2021-04-03T03:15:28.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f684c0b17b0 0x7f684c0a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f684c0b17b0 0x7f684c0a8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54668/0 (socket says 172.21.2.111:54668) 2021-04-03T03:15:28.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 -- 172.21.2.111:0/1807722717 learned_addr learned my addr 172.21.2.111:0/1807722717 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684affd700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f684c0603f0 0x7f684c0abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684a7fc700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0add50 0x7f684c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 -- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0add50 msgr2=0x7f684c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0add50 0x7f684c0b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 -- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f684c0603f0 msgr2=0x7f684c0abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f684c0603f0 0x7f684c0abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 -- 172.21.2.111:0/1807722717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0a8b60 con 0x7f684c0b17b0 2021-04-03T03:15:28.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684a7fc700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0add50 0x7f684c0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:28.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684b7fe700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f684c0b17b0 0x7f684c0a8670 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f684400c000 tx=0x7f6844009270).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f6843fff700 1 -- 172.21.2.111:0/1807722717 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684400ca00 con 0x7f684c0b17b0 2021-04-03T03:15:28.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f6843fff700 1 -- 172.21.2.111:0/1807722717 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400cb60 con 0x7f684c0b17b0 2021-04-03T03:15:28.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f6843fff700 1 -- 172.21.2.111:0/1807722717 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68440123f0 con 0x7f684c0b17b0 2021-04-03T03:15:28.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c0a95c0 con 0x7f684c0b17b0 2021-04-03T03:15:28.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c0a9ab0 con 0x7f684c0b17b0 2021-04-03T03:15:28.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c0a92b0 con 0x7f684c0b17b0 2021-04-03T03:15:28.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f6843fff700 1 -- 172.21.2.111:0/1807722717 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6844022070 con 0x7f684c0b17b0 2021-04-03T03:15:28.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f6843fff700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6830056fb0 0x7f6830059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f6843fff700 1 -- 172.21.2.111:0/1807722717 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f68440708a0 con 0x7f684c0b17b0 2021-04-03T03:15:28.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684affd700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6830056fb0 0x7f6830059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.204+0000 7f684affd700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6830056fb0 0x7f6830059470 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f683c019000 tx=0x7f683c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.208+0000 7f6843fff700 1 -- 172.21.2.111:0/1807722717 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6844045c90 con 0x7f684c0b17b0 2021-04-03T03:15:28.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.300+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f684c0a8390 con 0x7f684c0b17b0 2021-04-03T03:15:28.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.300+0000 7f6843fff700 1 -- 172.21.2.111:0/1807722717 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6844016020 con 0x7f684c0b17b0 2021-04-03T03:15:28.307 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:28.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6830056fb0 msgr2=0x7f6830059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6830056fb0 0x7f6830059470 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f683c019000 tx=0x7f683c015000).stop 2021-04-03T03:15:28.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f684c0b17b0 msgr2=0x7f684c0a8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f684c0b17b0 0x7f684c0a8670 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f684400c000 tx=0x7f6844009270).stop 2021-04-03T03:15:28.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 shutdown_connections 2021-04-03T03:15:28.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f684c0b17b0 0x7f684c0a8670 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6830056fb0 0x7f6830059470 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f684c0603f0 0x7f684c0abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 --2- 172.21.2.111:0/1807722717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f684c0add50 0x7f684c0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 >> 172.21.2.111:0/1807722717 conn(0x7f684c0b61c0 msgr2=0x7f684c0b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 shutdown_connections 2021-04-03T03:15:28.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.304+0000 7f684bfff700 1 -- 172.21.2.111:0/1807722717 wait complete. 2021-04-03T03:15:28.314 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:15:28.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cd058700 1 Processor -- start 2021-04-03T03:15:28.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cd058700 1 -- start start 2021-04-03T03:15:28.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cd058700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c00dbba0 0x7f30c00dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cd058700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f30c003d560 0x7f30c0041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cd058700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c00dc560 con 0x7f30c0041060 2021-04-03T03:15:28.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cd058700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c003de60 con 0x7f30c003d560 2021-04-03T03:15:28.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cd058700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00dc800 con 0x7f30c00dbba0 2021-04-03T03:15:28.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c77fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f30c0041060 0x7f30c003fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55472/0 (socket says 172.21.2.111:55472) 2021-04-03T03:15:28.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c77fe700 1 -- 172.21.2.111:0/1809828039 learned_addr learned my addr 172.21.2.111:0/1809828039 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:28.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30cc857700 1 --2- 172.21.2.111:0/1809828039 >> v2:172.21.2.115:3300/0 conn(0x7f30c003d560 0x7f30c0041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c7fff700 1 --2- 172.21.2.111:0/1809828039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c00dbba0 0x7f30c00dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c6ffd700 1 -- 172.21.2.111:0/1809828039 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097853576 0 0) 0x7f30c00dc560 con 0x7f30c0041060 2021-04-03T03:15:28.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c6ffd700 1 -- 172.21.2.111:0/1809828039 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30b4003240 con 0x7f30c0041060 2021-04-03T03:15:28.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c7fff700 1 -- 172.21.2.111:0/1809828039 >> v2:172.21.2.115:3300/0 conn(0x7f30c003d560 msgr2=0x7f30c0041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:28.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c7fff700 1 --2- 172.21.2.111:0/1809828039 >> v2:172.21.2.115:3300/0 conn(0x7f30c003d560 0x7f30c0041650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c7fff700 1 -- 172.21.2.111:0/1809828039 >> v1:172.21.2.111:6789/0 conn(0x7f30c0041060 legacy=0x7f30c003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c7fff700 1 -- 172.21.2.111:0/1809828039 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c003da30 con 0x7f30c00dbba0 2021-04-03T03:15:28.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.368+0000 7f30c7fff700 1 --2- 172.21.2.111:0/1809828039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c00dbba0 0x7f30c00dbf80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f30b8002f30 tx=0x7f30b8006dd0).ready entity=mon.2 client_cookie=ca38575fc9028005 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c6ffd700 1 -- 172.21.2.111:0/1809828039 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30b80096a0 con 0x7f30c00dbba0 2021-04-03T03:15:28.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c6ffd700 1 -- 172.21.2.111:0/1809828039 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8009800 con 0x7f30c00dbba0 2021-04-03T03:15:28.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c6ffd700 1 -- 172.21.2.111:0/1809828039 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30b8009b60 con 0x7f30c00dbba0 2021-04-03T03:15:28.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/1809828039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c00dbba0 msgr2=0x7f30c00dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 --2- 172.21.2.111:0/1809828039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c00dbba0 0x7f30c00dbf80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f30b8002f30 tx=0x7f30b8006dd0).stop 2021-04-03T03:15:28.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/1809828039 shutdown_connections 2021-04-03T03:15:28.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 --2- 172.21.2.111:0/1809828039 >> v2:172.21.2.115:3300/0 conn(0x7f30c003d560 0x7f30c0041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 --2- 172.21.2.111:0/1809828039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c00dbba0 0x7f30c00dbf80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/1809828039 >> 172.21.2.111:0/1809828039 conn(0x7f30c00462b0 msgr2=0x7f30c00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/1809828039 shutdown_connections 2021-04-03T03:15:28.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/1809828039 wait complete. 2021-04-03T03:15:28.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 Processor -- start 2021-04-03T03:15:28.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- start start 2021-04-03T03:15:28.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30c003d560 0x7f30c003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c0041060 0x7f30c00ea460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f30c00dbba0 0x7f30c00ea950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00444c0 con 0x7f30c003d560 2021-04-03T03:15:28.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c0044760 con 0x7f30c00dbba0 2021-04-03T03:15:28.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c0045230 con 0x7f30c0041060 2021-04-03T03:15:28.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30c003d560 0x7f30c003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30c003d560 0x7f30c003b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54682/0 (socket says 172.21.2.111:54682) 2021-04-03T03:15:28.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 -- 172.21.2.111:0/2131524045 learned_addr learned my addr 172.21.2.111:0/2131524045 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c77fe700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c0041060 0x7f30c00ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cc857700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f30c00dbba0 0x7f30c00ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 -- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c0041060 msgr2=0x7f30c00ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c0041060 0x7f30c00ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 -- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f30c00dbba0 msgr2=0x7f30c00ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f30c00dbba0 0x7f30c00ea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 -- 172.21.2.111:0/2131524045 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c00a2f00 con 0x7f30c003d560 2021-04-03T03:15:28.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cc857700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f30c00dbba0 0x7f30c00ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:28.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c7fff700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30c003d560 0x7f30c003b490 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7f30b8002f30 tx=0x7f30b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c57fa700 1 -- 172.21.2.111:0/2131524045 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30b8011b40 con 0x7f30c003d560 2021-04-03T03:15:28.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c57fa700 1 -- 172.21.2.111:0/2131524045 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8011ca0 con 0x7f30c003d560 2021-04-03T03:15:28.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c0045e10 con 0x7f30c003d560 2021-04-03T03:15:28.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c57fa700 1 -- 172.21.2.111:0/2131524045 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30b801e460 con 0x7f30c003d560 2021-04-03T03:15:28.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c00eafd0 con 0x7f30c003d560 2021-04-03T03:15:28.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c0004db0 con 0x7f30c003d560 2021-04-03T03:15:28.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c57fa700 1 -- 172.21.2.111:0/2131524045 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f30b801a040 con 0x7f30c003d560 2021-04-03T03:15:28.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c57fa700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30a8057050 0x7f30a8059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c57fa700 1 -- 172.21.2.111:0/2131524045 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f30b8071230 con 0x7f30c003d560 2021-04-03T03:15:28.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.372+0000 7f30c77fe700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30a8057050 0x7f30a8059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.376+0000 7f30c77fe700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30a8057050 0x7f30a8059510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f30bc00b000 tx=0x7f30bc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.376+0000 7f30c57fa700 1 -- 172.21.2.111:0/2131524045 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f30b8046620 con 0x7f30c003d560 2021-04-03T03:15:28.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30c00083c0 con 0x7f30c003d560 2021-04-03T03:15:28.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30c57fa700 1 -- 172.21.2.111:0/2131524045 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+10 (secure 0 0 0) 0x7f30b8049c40 con 0x7f30c003d560 2021-04-03T03:15:28.474 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:28.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30a8057050 msgr2=0x7f30a8059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30a8057050 0x7f30a8059510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f30bc00b000 tx=0x7f30bc009040).stop 2021-04-03T03:15:28.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30c003d560 msgr2=0x7f30c003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30c003d560 0x7f30c003b490 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7f30b8002f30 tx=0x7f30b8015000).stop 2021-04-03T03:15:28.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 shutdown_connections 2021-04-03T03:15:28.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f30a8057050 0x7f30a8059510 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f30c00dbba0 0x7f30c00ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f30c003d560 0x7f30c003b490 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 --2- 172.21.2.111:0/2131524045 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f30c0041060 0x7f30c00ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 >> 172.21.2.111:0/2131524045 conn(0x7f30c00462b0 msgr2=0x7f30c00055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 shutdown_connections 2021-04-03T03:15:28.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.468+0000 7f30cd058700 1 -- 172.21.2.111:0/2131524045 wait complete. 2021-04-03T03:15:28.481 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:15:28.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.532+0000 7ffb77de7700 1 Processor -- start 2021-04-03T03:15:28.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.532+0000 7ffb77de7700 1 -- start start 2021-04-03T03:15:28.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.532+0000 7ffb77de7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ffb780603f0 0x7ffb781447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.532+0000 7ffb77de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb78144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.532+0000 7ffb77de7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb780607d0 con 0x7ffb780b17b0 2021-04-03T03:15:28.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.532+0000 7ffb77de7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb781451c0 con 0x7ffb780603f0 2021-04-03T03:15:28.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.532+0000 7ffb77de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7814b090 con 0x7ffb780add50 2021-04-03T03:15:28.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb76de5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ffb780603f0 0x7ffb781447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb78144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb78144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:38042/0 (socket says 172.21.2.111:38042) 2021-04-03T03:15:28.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 -- 172.21.2.111:0/2003290693 learned_addr learned my addr 172.21.2.111:0/2003290693 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 -- 172.21.2.111:0/2003290693 >> v2:172.21.2.115:3300/0 conn(0x7ffb780603f0 msgr2=0x7ffb781447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:28.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 --2- 172.21.2.111:0/2003290693 >> v2:172.21.2.115:3300/0 conn(0x7ffb780603f0 0x7ffb781447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 -- 172.21.2.111:0/2003290693 >> v1:172.21.2.115:6789/0 conn(0x7ffb780b17b0 legacy=0x7ffb78147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 -- 172.21.2.111:0/2003290693 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7814b330 con 0x7ffb780add50 2021-04-03T03:15:28.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb76de5700 1 --2- 172.21.2.111:0/2003290693 >> v2:172.21.2.115:3300/0 conn(0x7ffb780603f0 0x7ffb781447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:15:28.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb765e4700 1 --2- 172.21.2.111:0/2003290693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb78144cd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ffb70001450 tx=0x7ffb60006e90).ready entity=mon.2 client_cookie=8964039864c26c0e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb75de3700 1 -- 172.21.2.111:0/2003290693 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb60009710 con 0x7ffb780add50 2021-04-03T03:15:28.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb75de3700 1 -- 172.21.2.111:0/2003290693 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60009870 con 0x7ffb780add50 2021-04-03T03:15:28.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb75de3700 1 -- 172.21.2.111:0/2003290693 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb60009bd0 con 0x7ffb780add50 2021-04-03T03:15:28.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 -- 172.21.2.111:0/2003290693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 msgr2=0x7ffb78144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2003290693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb78144cd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ffb70001450 tx=0x7ffb60006e90).stop 2021-04-03T03:15:28.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 -- 172.21.2.111:0/2003290693 shutdown_connections 2021-04-03T03:15:28.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2003290693 >> v2:172.21.2.115:3300/0 conn(0x7ffb780603f0 0x7ffb781447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2003290693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb78144cd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 -- 172.21.2.111:0/2003290693 >> 172.21.2.111:0/2003290693 conn(0x7ffb780b61c0 msgr2=0x7ffb7805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 -- 172.21.2.111:0/2003290693 shutdown_connections 2021-04-03T03:15:28.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 -- 172.21.2.111:0/2003290693 wait complete. 2021-04-03T03:15:28.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.536+0000 7ffb77de7700 1 Processor -- start 2021-04-03T03:15:28.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 -- start start 2021-04-03T03:15:28.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb780603f0 0x7ffb78062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb7815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb780b17b0 0x7ffb7815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb78064b70 con 0x7ffb780b17b0 2021-04-03T03:15:28.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780aa9d0 con 0x7ffb780603f0 2021-04-03T03:15:28.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780aa730 con 0x7ffb780add50 2021-04-03T03:15:28.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb780b17b0 0x7ffb7815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb780b17b0 0x7ffb7815a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54700/0 (socket says 172.21.2.111:54700) 2021-04-03T03:15:28.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 -- 172.21.2.111:0/2101362527 learned_addr learned my addr 172.21.2.111:0/2101362527 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb76de5700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb780603f0 0x7ffb78062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb765e4700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb7815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 -- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 msgr2=0x7ffb7815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb7815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 -- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb780603f0 msgr2=0x7ffb78062d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb780603f0 0x7ffb78062d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 -- 172.21.2.111:0/2101362527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780648d0 con 0x7ffb780b17b0 2021-04-03T03:15:28.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb765e4700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb7815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:28.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb775e6700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb780b17b0 0x7ffb7815a6e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ffb70001320 tx=0x7ffb70007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb6ffff700 1 -- 172.21.2.111:0/2101362527 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7000a880 con 0x7ffb780b17b0 2021-04-03T03:15:28.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb6ffff700 1 -- 172.21.2.111:0/2101362527 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7000a9e0 con 0x7ffb780b17b0 2021-04-03T03:15:28.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb6ffff700 1 -- 172.21.2.111:0/2101362527 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7000ad40 con 0x7ffb780b17b0 2021-04-03T03:15:28.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb780aacb0 con 0x7ffb780b17b0 2021-04-03T03:15:28.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb780aa4c0 con 0x7ffb780b17b0 2021-04-03T03:15:28.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb78057660 con 0x7ffb780b17b0 2021-04-03T03:15:28.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb6ffff700 1 -- 172.21.2.111:0/2101362527 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ffb70019030 con 0x7ffb780b17b0 2021-04-03T03:15:28.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb6ffff700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb5c056ef0 0x7ffb5c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb6ffff700 1 -- 172.21.2.111:0/2101362527 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ffb7006fc80 con 0x7ffb780b17b0 2021-04-03T03:15:28.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb76de5700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb5c056ef0 0x7ffb5c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.540+0000 7ffb76de5700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb5c056ef0 0x7ffb5c0593b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7ffb6800b000 tx=0x7ffb68008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.544+0000 7ffb6ffff700 1 -- 172.21.2.111:0/2101362527 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ffb70042c50 con 0x7ffb780b17b0 2021-04-03T03:15:28.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.636+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb78148850 con 0x7ffb780b17b0 2021-04-03T03:15:28.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.636+0000 7ffb6ffff700 1 -- 172.21.2.111:0/2101362527 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+10 (secure 0 0 0) 0x7ffb70075020 con 0x7ffb780b17b0 2021-04-03T03:15:28.642 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:28.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.636+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb5c056ef0 msgr2=0x7ffb5c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.636+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb5c056ef0 0x7ffb5c0593b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7ffb6800b000 tx=0x7ffb68008040).stop 2021-04-03T03:15:28.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.636+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb780b17b0 msgr2=0x7ffb7815a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.636+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb780b17b0 0x7ffb7815a6e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ffb70001320 tx=0x7ffb70007ef0).stop 2021-04-03T03:15:28.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 shutdown_connections 2021-04-03T03:15:28.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb780b17b0 0x7ffb7815a6e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb5c056ef0 0x7ffb5c0593b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb780603f0 0x7ffb78062d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 --2- 172.21.2.111:0/2101362527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb780add50 0x7ffb7815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 >> 172.21.2.111:0/2101362527 conn(0x7ffb780b61c0 msgr2=0x7ffb78144980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 shutdown_connections 2021-04-03T03:15:28.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.640+0000 7ffb77de7700 1 -- 172.21.2.111:0/2101362527 wait complete. 2021-04-03T03:15:28.650 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:15:28.651 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:28.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254afdf700 1 Processor -- start 2021-04-03T03:15:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254afdf700 1 -- start start 2021-04-03T03:15:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254afdf700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f254414b700 0x7f25440b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254afdf700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440b0c90 0x7f2544147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254afdf700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440607d0 con 0x7f25440603f0 2021-04-03T03:15:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254afdf700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25440af400 con 0x7f254414b700 2021-04-03T03:15:28.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254afdf700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440af6a0 con 0x7f25440b0c90 2021-04-03T03:15:28.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f2549fdd700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f25440603f0 0x7f25440b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:39800/0 (socket says 172.21.2.111:39800) 2021-04-03T03:15:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f25497dc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f254414b700 0x7f25440b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f254a7de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440b0c90 0x7f2544147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.704+0000 7f2549fdd700 1 -- 172.21.2.111:0/2102229805 learned_addr learned my addr 172.21.2.111:0/2102229805 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:28.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254a7de700 1 -- 172.21.2.111:0/2102229805 >> v2:172.21.2.115:3300/0 conn(0x7f254414b700 msgr2=0x7f25440b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:28.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254a7de700 1 --2- 172.21.2.111:0/2102229805 >> v2:172.21.2.115:3300/0 conn(0x7f254414b700 0x7f25440b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254a7de700 1 -- 172.21.2.111:0/2102229805 >> v1:172.21.2.115:6789/0 conn(0x7f25440603f0 legacy=0x7f25440b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254a7de700 1 -- 172.21.2.111:0/2102229805 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440afb40 con 0x7f25440b0c90 2021-04-03T03:15:28.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254a7de700 1 --2- 172.21.2.111:0/2102229805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440b0c90 0x7f2544147680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f253c0032b0 tx=0x7f253c0091d0).ready entity=mon.2 client_cookie=1d8956f3d1325e7b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f2548fdb700 1 -- 172.21.2.111:0/2102229805 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f253c00b9a0 con 0x7f25440b0c90 2021-04-03T03:15:28.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f2548fdb700 1 -- 172.21.2.111:0/2102229805 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c00bb00 con 0x7f25440b0c90 2021-04-03T03:15:28.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f2548fdb700 1 -- 172.21.2.111:0/2102229805 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f253c00bc70 con 0x7f25440b0c90 2021-04-03T03:15:28.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- 172.21.2.111:0/2102229805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440b0c90 msgr2=0x7f2544147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 --2- 172.21.2.111:0/2102229805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440b0c90 0x7f2544147680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f253c0032b0 tx=0x7f253c0091d0).stop 2021-04-03T03:15:28.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- 172.21.2.111:0/2102229805 shutdown_connections 2021-04-03T03:15:28.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 --2- 172.21.2.111:0/2102229805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440b0c90 0x7f2544147680 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 --2- 172.21.2.111:0/2102229805 >> v2:172.21.2.115:3300/0 conn(0x7f254414b700 0x7f25440b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- 172.21.2.111:0/2102229805 >> 172.21.2.111:0/2102229805 conn(0x7f25440b61c0 msgr2=0x7f254405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- 172.21.2.111:0/2102229805 shutdown_connections 2021-04-03T03:15:28.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- 172.21.2.111:0/2102229805 wait complete. 2021-04-03T03:15:28.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 Processor -- start 2021-04-03T03:15:28.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- start start 2021-04-03T03:15:28.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440603f0 0x7f25440abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25440b0c90 0x7f2544062a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f254414b700 0x7f2544067650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440a9960 con 0x7f25440b0c90 2021-04-03T03:15:28.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2544065180 con 0x7f254414b700 2021-04-03T03:15:28.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f254afdf700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440ab5b0 con 0x7f25440603f0 2021-04-03T03:15:28.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f25497dc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25440b0c90 0x7f2544062a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f25497dc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25440b0c90 0x7f2544062a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54714/0 (socket says 172.21.2.111:54714) 2021-04-03T03:15:28.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f25497dc700 1 -- 172.21.2.111:0/1243336463 learned_addr learned my addr 172.21.2.111:0/1243336463 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:28.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.708+0000 7f2549fdd700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440603f0 0x7f25440abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f254a7de700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f254414b700 0x7f2544067650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f25497dc700 1 -- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440603f0 msgr2=0x7f25440abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f25497dc700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440603f0 0x7f25440abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f25497dc700 1 -- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f254414b700 msgr2=0x7f2544067650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f25497dc700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f254414b700 0x7f2544067650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f25497dc700 1 -- 172.21.2.111:0/1243336463 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544062f50 con 0x7f25440b0c90 2021-04-03T03:15:28.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f25497dc700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25440b0c90 0x7f2544062a60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f253800c000 tx=0x7f2538006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2536ffd700 1 -- 172.21.2.111:0/1243336463 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f253800ca10 con 0x7f25440b0c90 2021-04-03T03:15:28.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2536ffd700 1 -- 172.21.2.111:0/1243336463 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253800cb70 con 0x7f25440b0c90 2021-04-03T03:15:28.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2536ffd700 1 -- 172.21.2.111:0/1243336463 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25380123f0 con 0x7f25440b0c90 2021-04-03T03:15:28.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25440aa360 con 0x7f25440b0c90 2021-04-03T03:15:28.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2544064f30 con 0x7f25440b0c90 2021-04-03T03:15:28.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25440aa580 con 0x7f25440b0c90 2021-04-03T03:15:28.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2536ffd700 1 -- 172.21.2.111:0/1243336463 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2538012550 con 0x7f25440b0c90 2021-04-03T03:15:28.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2536ffd700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2524056ef0 0x7f25240593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:28.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2536ffd700 1 -- 172.21.2.111:0/1243336463 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f2538070730 con 0x7f25440b0c90 2021-04-03T03:15:28.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2549fdd700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2524056ef0 0x7f25240593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:28.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.712+0000 7f2549fdd700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2524056ef0 0x7f25240593b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f2540008000 tx=0x7f2540009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:28.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.716+0000 7f2536ffd700 1 -- 172.21.2.111:0/1243336463 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f25380459b0 con 0x7f25440b0c90 2021-04-03T03:15:28.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.836+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2544057660 con 0x7f25440b0c90 2021-04-03T03:15:28.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.836+0000 7f2536ffd700 1 -- 172.21.2.111:0/1243336463 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f2538048fd0 con 0x7f25440b0c90 2021-04-03T03:15:28.844 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:28.844 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2065583567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2065583567},{"type":"v1","addr":"172.21.2.115:6833","nonce":2065583567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:25.328682+0000","modified":"2021-04-03T03:15:26.331659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/485806052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":485806052},{"type":"v1","addr":"172.21.2.111:6835","nonce":485806052}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-04-03T03:15:28.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2524056ef0 msgr2=0x7f25240593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2524056ef0 0x7f25240593b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f2540008000 tx=0x7f2540009040).stop 2021-04-03T03:15:28.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25440b0c90 msgr2=0x7f2544062a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:28.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25440b0c90 0x7f2544062a60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f253800c000 tx=0x7f2538006dd0).stop 2021-04-03T03:15:28.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 shutdown_connections 2021-04-03T03:15:28.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2524056ef0 0x7f25240593b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f254414b700 0x7f2544067650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25440603f0 0x7f25440abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 --2- 172.21.2.111:0/1243336463 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25440b0c90 0x7f2544062a60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:28.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 >> 172.21.2.111:0/1243336463 conn(0x7f25440b61c0 msgr2=0x7f2544148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:28.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 shutdown_connections 2021-04-03T03:15:28.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:28.840+0000 7f254afdf700 1 -- 172.21.2.111:0/1243336463 wait complete. 2021-04-03T03:15:28.848 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 33 2021-04-03T03:15:29.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:15:25.328682+0000', 'modified': '2021-04-03T03:15:26.331659+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6573': {'gid': 6573, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/485806052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 485806052}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 485806052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:15:29.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:15:29.054 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:15:29.112 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:29.114 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:29.117 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:29.120 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:29.123 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:29.130 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6573,"want_state":"up:active","state":"up:active","rank_uptime":3.79352483,"mdsmap_epoch":33,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":7.9321138480000002} 2021-04-03T03:15:29.131 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6573, "mdsmap_epoch": 33, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 3.79352483, "state": "up:active", "uptime": 7.932113848, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:15:29.131 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:15:29.132 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:15:29.132 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:15:29.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:29.133 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:29.193 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:15:29.247 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:29.247 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:29.295 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:29.296 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:29.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:29.297 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:15:29.349 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:15:29.349 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:29.394 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:15:29.402 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:29.402 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:29.433 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:15:29.431+0000 7f04cf9350c0 -1 init, newargv = 0x55a96ec2dd50 newargc=9ceph-fuse[15624]: starting ceph client 2021-04-03T03:15:29.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr: 2021-04-03T03:15:29.438 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:29.439 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:29.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:29.440 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:15:29.445 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15624]: starting fuse 2021-04-03T03:15:29.452 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:15:29.452 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:15:29.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:29.453 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:15:29.454 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:15:29.454 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:15:29.454 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:15:29.455 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:15:29.455 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:29.456 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:15:29.456 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:15:29.456 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:15:29.457 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:29.457 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:15:29.458 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:15:29.458 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:15:29.458 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:15:29.459 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:29.459 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:15:29.460 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:15:29.460 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:15:29.460 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:15:29.461 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:15:29.461 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:29.462 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:15:29.462 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:15:29.529 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15624.asok 2021-04-03T03:15:29.532 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15624.asok 2021-04-03T03:15:29.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:29.533 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15624.asok status 2021-04-03T03:15:29.645 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:15:29.646 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:15:29.646 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:15:29.646 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:15:29.646 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:15:29.647 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:15:29.647 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:15:29.647 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15624", 2021-04-03T03:15:29.647 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:15:29.648 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:29.648 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:15:29.648 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:15:29.648 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 4781, 2021-04-03T03:15:29.648 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:15:29.649 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:15:29.649 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:15:29.649 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 4781 2021-04-03T03:15:29.649 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:29.649 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:15:29.650 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:15:29.650 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:15:29.650 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2594442578 2021-04-03T03:15:29.650 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:15:29.651 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:29.651 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:15:29.651 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:15:29.651 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:15:29.651 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2594442578 2021-04-03T03:15:29.652 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:29.652 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.4781 172.21.2.126:0/2594442578", 2021-04-03T03:15:29.652 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/2594442578", 2021-04-03T03:15:29.652 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:15:29.652 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 33, 2021-04-03T03:15:29.653 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 84, 2021-04-03T03:15:29.653 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:15:29.653 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:15:29.653 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:15:29.653 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:15:29.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:29.655 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:29.656 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:29.661 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:15:29.662 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:29.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:29.662 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:29.714 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:29.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 Processor -- start 2021-04-03T03:15:29.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- start start 2021-04-03T03:15:29.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f93c40b12b0 0x7f93c41522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40607d0 con 0x7f93c414b700 2021-04-03T03:15:29.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40b0eb0 con 0x7f93c40603f0 2021-04-03T03:15:29.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40af400 con 0x7f93c40b12b0 2021-04-03T03:15:29.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c1d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f93c414b700 0x7f93c40b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55514/0 (socket says 172.21.2.111:55514) 2021-04-03T03:15:29.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c1d9b700 1 -- 172.21.2.111:0/1355672196 learned_addr learned my addr 172.21.2.111:0/1355672196 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:29.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497195534 0 0) 0x7f93c40607d0 con 0x7f93c414b700 2021-04-03T03:15:29.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a4003240 con 0x7f93c414b700 2021-04-03T03:15:29.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 --2- 172.21.2.111:0/1355672196 >> v2:172.21.2.115:3300/0 conn(0x7f93c40b12b0 0x7f93c41522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3989334862 0 0) 0x7f93a4003240 con 0x7f93c414b700 2021-04-03T03:15:29.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 >> v2:172.21.2.115:3300/0 conn(0x7f93c40b12b0 msgr2=0x7f93c41522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:29.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 --2- 172.21.2.111:0/1355672196 >> v2:172.21.2.115:3300/0 conn(0x7f93c40b12b0 0x7f93c41522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:29.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 >> v1:172.21.2.115:6789/0 conn(0x7f93c40603f0 legacy=0x7f93c40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:29.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40af6a0 con 0x7f93c414b700 2021-04-03T03:15:29.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f93ac0016d0 con 0x7f93c414b700 2021-04-03T03:15:29.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 >> v1:172.21.2.111:6789/0 conn(0x7f93c414b700 legacy=0x7f93c40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93a4003830 0x7f93a4003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93a40044e0 0x7f93a4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93a400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40af400 con 0x7f93a4004e50 2021-04-03T03:15:29.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4005330 con 0x7f93a4003830 2021-04-03T03:15:29.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a400bc20 con 0x7f93a40044e0 2021-04-03T03:15:29.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93a400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c1d9b700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93a40044e0 0x7f93a4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c259c700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93a4003830 0x7f93a4003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 -- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93a40044e0 msgr2=0x7f93a4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:29.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93a40044e0 0x7f93a4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:29.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 -- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93a4003830 msgr2=0x7f93a4003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:29.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93a4003830 0x7f93a4003c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:29.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 -- 172.21.2.111:0/1355672196 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a4003240 con 0x7f93a4004e50 2021-04-03T03:15:29.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 -- 172.21.2.111:0/1355672196 reap_dead start 2021-04-03T03:15:29.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c2d9d700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93a400b730 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f93bc00a000 tx=0x7f93bc008040).ready entity=mon.0 client_cookie=b06ce523b8be28eb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:29.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93bc00ddf0 con 0x7f93a4004e50 2021-04-03T03:15:29.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc012420 con 0x7f93a4004e50 2021-04-03T03:15:29.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c159a700 1 -- 172.21.2.111:0/1355672196 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93bc012780 con 0x7f93a4004e50 2021-04-03T03:15:29.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- 172.21.2.111:0/1355672196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 msgr2=0x7f93a400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93a400b730 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f93bc00a000 tx=0x7f93bc008040).stop 2021-04-03T03:15:29.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- 172.21.2.111:0/1355672196 shutdown_connections 2021-04-03T03:15:29.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 --2- 172.21.2.111:0/1355672196 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93a400b730 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:29.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- 172.21.2.111:0/1355672196 >> 172.21.2.111:0/1355672196 conn(0x7f93c40b61c0 msgr2=0x7f93c405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:29.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- 172.21.2.111:0/1355672196 shutdown_connections 2021-04-03T03:15:29.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.773+0000 7f93c359e700 1 -- 172.21.2.111:0/1355672196 wait complete. 2021-04-03T03:15:29.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 Processor -- start 2021-04-03T03:15:29.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 -- start start 2021-04-03T03:15:29.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93c4159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93c4063280 0x7f93c40a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93c40ab5f0 0x7f93c4147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40aa140 con 0x7f93a4004e50 2021-04-03T03:15:29.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40ab100 con 0x7f93c40ab5f0 2021-04-03T03:15:29.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40ab270 con 0x7f93c4063280 2021-04-03T03:15:29.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93c4159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93c4159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54730/0 (socket says 172.21.2.111:54730) 2021-04-03T03:15:29.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 -- 172.21.2.111:0/1918508239 learned_addr learned my addr 172.21.2.111:0/1918508239 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:29.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 -- 172.21.2.111:0/1918508239 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93c4063280 msgr2=0x7f93c40a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93c4063280 0x7f93c40a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 -- 172.21.2.111:0/1918508239 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93c40ab5f0 msgr2=0x7f93c4147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93c40ab5f0 0x7f93c4147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 -- 172.21.2.111:0/1918508239 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c41480f0 con 0x7f93a4004e50 2021-04-03T03:15:29.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c259c700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93c4159580 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f93b400b000 tx=0x7f93b4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:29.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93bb7fe700 1 -- 172.21.2.111:0/1918508239 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b4010680 con 0x7f93a4004e50 2021-04-03T03:15:29.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93bb7fe700 1 -- 172.21.2.111:0/1918508239 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4010040 con 0x7f93a4004e50 2021-04-03T03:15:29.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93bb7fe700 1 -- 172.21.2.111:0/1918508239 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b40109d0 con 0x7f93a4004e50 2021-04-03T03:15:29.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c41483a0 con 0x7f93a4004e50 2021-04-03T03:15:29.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c4064e00 con 0x7f93a4004e50 2021-04-03T03:15:29.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c40ab3e0 con 0x7f93a4004e50 2021-04-03T03:15:29.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93bb7fe700 1 -- 172.21.2.111:0/1918508239 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f93b4010b30 con 0x7f93a4004e50 2021-04-03T03:15:29.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93bb7fe700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93a8056ef0 0x7f93a80593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93bb7fe700 1 -- 172.21.2.111:0/1918508239 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f93b406fd60 con 0x7f93a4004e50 2021-04-03T03:15:29.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c1d9b700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93a8056ef0 0x7f93a80593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.777+0000 7f93c1d9b700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93a8056ef0 0x7f93a80593b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f93ac00b000 tx=0x7f93c40af830).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:29.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.781+0000 7f93bb7fe700 1 -- 172.21.2.111:0/1918508239 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f93b4045040 con 0x7f93a4004e50 2021-04-03T03:15:29.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93c4057660 con 0x7f93a4004e50 2021-04-03T03:15:29.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93bb7fe700 1 -- 172.21.2.111:0/1918508239 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f93b4048660 con 0x7f93a4004e50 2021-04-03T03:15:29.877 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:29.877 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":84,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:26.331795+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:23.240089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:24.288628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:29.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93a8056ef0 msgr2=0x7f93a80593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93a8056ef0 0x7f93a80593b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f93ac00b000 tx=0x7f93c40af830).stop 2021-04-03T03:15:29.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 msgr2=0x7f93c4159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93c4159580 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f93b400b000 tx=0x7f93b4009040).stop 2021-04-03T03:15:29.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 shutdown_connections 2021-04-03T03:15:29.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93a8056ef0 0x7f93a80593b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:29.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f93c40ab5f0 0x7f93c4147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f93a4004e50 0x7f93c4159580 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:29.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 --2- 172.21.2.111:0/1918508239 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f93c4063280 0x7f93c40a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 >> 172.21.2.111:0/1918508239 conn(0x7f93c40b61c0 msgr2=0x7f93c405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:29.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 shutdown_connections 2021-04-03T03:15:29.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.873+0000 7f93c359e700 1 -- 172.21.2.111:0/1918508239 wait complete. 2021-04-03T03:15:29.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:15:23.240089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:15:24.288628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:15:29.888 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:29.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb766cc700 1 Processor -- start 2021-04-03T03:15:29.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb766cc700 1 -- start start 2021-04-03T03:15:29.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb766cc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6803d560 0x7ffb680dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb766cc700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680dc560 con 0x7ffb68041060 2021-04-03T03:15:29.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb766cc700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680dc800 con 0x7ffb680dbba0 2021-04-03T03:15:29.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb766cc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6803de60 con 0x7ffb6803d560 2021-04-03T03:15:29.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb74ec9700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ffb68041060 0x7ffb68041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55534/0 (socket says 172.21.2.111:55534) 2021-04-03T03:15:29.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.941+0000 7ffb74ec9700 1 -- 172.21.2.111:0/3534019398 learned_addr learned my addr 172.21.2.111:0/3534019398 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:29.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222364966 0 0) 0x7ffb680dc560 con 0x7ffb68041060 2021-04-03T03:15:29.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb58003240 con 0x7ffb68041060 2021-04-03T03:15:29.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb75ecb700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6803d560 0x7ffb680dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3212655346 0 0) 0x7ffb58003240 con 0x7ffb68041060 2021-04-03T03:15:29.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6803d560 msgr2=0x7ffb680dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:29.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6803d560 0x7ffb680dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:29.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 >> v1:172.21.2.115:6789/0 conn(0x7ffb680dbba0 legacy=0x7ffb6803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6803da30 con 0x7ffb68041060 2021-04-03T03:15:29.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ffb6c0016d0 con 0x7ffb68041060 2021-04-03T03:15:29.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 >> v1:172.21.2.111:6789/0 conn(0x7ffb68041060 legacy=0x7ffb68041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb580037e0 0x7ffb58003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb580044e0 0x7ffb58004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6803d560 0x7ffb58004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6803de60 con 0x7ffb580044e0 2021-04-03T03:15:29.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58005340 con 0x7ffb580037e0 2021-04-03T03:15:29.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5800b730 con 0x7ffb6803d560 2021-04-03T03:15:29.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb74ec9700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb580044e0 0x7ffb58004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb74ec9700 1 -- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6803d560 msgr2=0x7ffb58004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:29.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb74ec9700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6803d560 0x7ffb58004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb74ec9700 1 -- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb580037e0 msgr2=0x7ffb58003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:29.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb74ec9700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb580037e0 0x7ffb58003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb74ec9700 1 -- 172.21.2.111:0/3534019398 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58003240 con 0x7ffb580044e0 2021-04-03T03:15:29.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb74ec9700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb580044e0 0x7ffb58004960 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7ffb6c0013e0 tx=0x7ffb6c0076a0).ready entity=mon.0 client_cookie=18cc513fb8a39797 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:29.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb6c0070d0 con 0x7ffb580044e0 2021-04-03T03:15:29.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c007230 con 0x7ffb580044e0 2021-04-03T03:15:29.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb63fff700 1 -- 172.21.2.111:0/3534019398 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb6c0055f0 con 0x7ffb580044e0 2021-04-03T03:15:29.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3534019398 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb580044e0 msgr2=0x7ffb58004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 --2- 172.21.2.111:0/3534019398 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb580044e0 0x7ffb58004960 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7ffb6c0013e0 tx=0x7ffb6c0076a0).stop 2021-04-03T03:15:29.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb75ecb700 1 -- 172.21.2.111:0/3534019398 reap_dead start 2021-04-03T03:15:29.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3534019398 shutdown_connections 2021-04-03T03:15:29.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3534019398 >> 172.21.2.111:0/3534019398 conn(0x7ffb680462b0 msgr2=0x7ffb680466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:29.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3534019398 shutdown_connections 2021-04-03T03:15:29.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3534019398 wait complete. 2021-04-03T03:15:29.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 Processor -- start 2021-04-03T03:15:29.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- start start 2021-04-03T03:15:29.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb68005c40 0x7ffb6803b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb6803c370 0x7ffb680a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6809f4a0 0x7ffb6809f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680a0e60 con 0x7ffb68005c40 2021-04-03T03:15:29.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6803b820 con 0x7ffb6803c370 2021-04-03T03:15:29.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68006020 con 0x7ffb6809f4a0 2021-04-03T03:15:29.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb68005c40 0x7ffb6803b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb68005c40 0x7ffb6803b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54754/0 (socket says 172.21.2.111:54754) 2021-04-03T03:15:29.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 -- 172.21.2.111:0/3393131784 learned_addr learned my addr 172.21.2.111:0/3393131784 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:29.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 -- 172.21.2.111:0/3393131784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6809f4a0 msgr2=0x7ffb6809f920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:29.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6809f4a0 0x7ffb6809f920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 -- 172.21.2.111:0/3393131784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb6803c370 msgr2=0x7ffb680a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:29.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb6803c370 0x7ffb680a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:29.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 -- 172.21.2.111:0/3393131784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb680a1a90 con 0x7ffb68005c40 2021-04-03T03:15:29.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb756ca700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb68005c40 0x7ffb6803b330 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7ffb68040a80 tx=0x7ffb64007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:29.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb627fc700 1 -- 172.21.2.111:0/3393131784 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb64013410 con 0x7ffb68005c40 2021-04-03T03:15:29.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb627fc700 1 -- 172.21.2.111:0/3393131784 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb64013570 con 0x7ffb68005c40 2021-04-03T03:15:29.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb680a1d30 con 0x7ffb68005c40 2021-04-03T03:15:29.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb680d4c60 con 0x7ffb68005c40 2021-04-03T03:15:29.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb627fc700 1 -- 172.21.2.111:0/3393131784 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb640138d0 con 0x7ffb68005c40 2021-04-03T03:15:29.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.945+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c0033f0 con 0x7ffb68005c40 2021-04-03T03:15:29.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.949+0000 7ffb627fc700 1 -- 172.21.2.111:0/3393131784 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ffb64010030 con 0x7ffb68005c40 2021-04-03T03:15:29.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.949+0000 7ffb627fc700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb50056f20 0x7ffb500593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:29.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.949+0000 7ffb627fc700 1 -- 172.21.2.111:0/3393131784 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ffb6406ff20 con 0x7ffb68005c40 2021-04-03T03:15:29.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.949+0000 7ffb74ec9700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb50056f20 0x7ffb500593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:29.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.949+0000 7ffb74ec9700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb50056f20 0x7ffb500593e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ffb6c0013e0 tx=0x7ffb6c013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:29.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:29.949+0000 7ffb627fc700 1 -- 172.21.2.111:0/3393131784 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ffb64045200 con 0x7ffb68005c40 2021-04-03T03:15:30.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ffb4c003c20 con 0x7ffb68005c40 2021-04-03T03:15:30.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb627fc700 1 -- 172.21.2.111:0/3393131784 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7ffb64048820 con 0x7ffb68005c40 2021-04-03T03:15:30.049 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:30.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb50056f20 msgr2=0x7ffb500593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb50056f20 0x7ffb500593e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ffb6c0013e0 tx=0x7ffb6c013000).stop 2021-04-03T03:15:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb68005c40 msgr2=0x7ffb6803b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb68005c40 0x7ffb6803b330 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7ffb68040a80 tx=0x7ffb64007bb0).stop 2021-04-03T03:15:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 shutdown_connections 2021-04-03T03:15:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb6809f4a0 0x7ffb6809f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb68005c40 0x7ffb6803b330 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb50056f20 0x7ffb500593e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 --2- 172.21.2.111:0/3393131784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb6803c370 0x7ffb680a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 >> 172.21.2.111:0/3393131784 conn(0x7ffb680462b0 msgr2=0x7ffb680db280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 shutdown_connections 2021-04-03T03:15:30.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.045+0000 7ffb766cc700 1 -- 172.21.2.111:0/3393131784 wait complete. 2021-04-03T03:15:30.057 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:30.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf3fff700 1 Processor -- start 2021-04-03T03:15:30.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf3fff700 1 -- start start 2021-04-03T03:15:30.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3bf40603f0 0x7f3bf41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf3fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3bf40add50 0x7f3bf4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf40607d0 con 0x7f3bf40b17b0 2021-04-03T03:15:30.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf41451c0 con 0x7f3bf40603f0 2021-04-03T03:15:30.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf3fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf414b090 con 0x7f3bf40add50 2021-04-03T03:15:30.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf37fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3bf40b17b0 0x7f3bf4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55554/0 (socket says 172.21.2.111:55554) 2021-04-03T03:15:30.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3bf40603f0 0x7f3bf41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.113+0000 7f3bf37fe700 1 -- 172.21.2.111:0/2927899881 learned_addr learned my addr 172.21.2.111:0/2927899881 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:30.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf2ffd700 1 -- 172.21.2.111:0/2927899881 >> v2:172.21.2.115:3300/0 conn(0x7f3bf40add50 msgr2=0x7f3bf4144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:30.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf2ffd700 1 --2- 172.21.2.111:0/2927899881 >> v2:172.21.2.115:3300/0 conn(0x7f3bf40add50 0x7f3bf4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf2ffd700 1 -- 172.21.2.111:0/2927899881 >> v1:172.21.2.111:6789/0 conn(0x7f3bf40b17b0 legacy=0x7f3bf4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf2ffd700 1 -- 172.21.2.111:0/2927899881 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf414b330 con 0x7f3bf40603f0 2021-04-03T03:15:30.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf2ffd700 1 --2- 172.21.2.111:0/2927899881 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40603f0 0x7f3bf41447e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f3bec001450 tx=0x7f3bdc006dd0).ready entity=mon.0 client_cookie=68ccd1161a34661a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf1ffb700 1 -- 172.21.2.111:0/2927899881 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bdc0096b0 con 0x7f3bf40603f0 2021-04-03T03:15:30.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf1ffb700 1 -- 172.21.2.111:0/2927899881 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc009810 con 0x7f3bf40603f0 2021-04-03T03:15:30.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf1ffb700 1 -- 172.21.2.111:0/2927899881 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bdc009b70 con 0x7f3bf40603f0 2021-04-03T03:15:30.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- 172.21.2.111:0/2927899881 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40603f0 msgr2=0x7f3bf41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/2927899881 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40603f0 0x7f3bf41447e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f3bec001450 tx=0x7f3bdc006dd0).stop 2021-04-03T03:15:30.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- 172.21.2.111:0/2927899881 shutdown_connections 2021-04-03T03:15:30.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/2927899881 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40603f0 0x7f3bf41447e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/2927899881 >> v2:172.21.2.115:3300/0 conn(0x7f3bf40add50 0x7f3bf4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- 172.21.2.111:0/2927899881 >> 172.21.2.111:0/2927899881 conn(0x7f3bf40b61c0 msgr2=0x7f3bf405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- 172.21.2.111:0/2927899881 shutdown_connections 2021-04-03T03:15:30.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- 172.21.2.111:0/2927899881 wait complete. 2021-04-03T03:15:30.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 Processor -- start 2021-04-03T03:15:30.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- start start 2021-04-03T03:15:30.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3bf40603f0 0x7f3bf40a8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40add50 0x7f3bf4156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3bf40b17b0 0x7f3bf414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40636b0 con 0x7f3bf40add50 2021-04-03T03:15:30.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf415a0a0 con 0x7f3bf40b17b0 2021-04-03T03:15:30.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4156e40 con 0x7f3bf40603f0 2021-04-03T03:15:30.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40add50 0x7f3bf4156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40add50 0x7f3bf4156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54764/0 (socket says 172.21.2.111:54764) 2021-04-03T03:15:30.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 -- 172.21.2.111:0/1392787440 learned_addr learned my addr 172.21.2.111:0/1392787440 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:30.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 -- 172.21.2.111:0/1392787440 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3bf40603f0 msgr2=0x7f3bf40a8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3bf40603f0 0x7f3bf40a8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 -- 172.21.2.111:0/1392787440 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3bf40b17b0 msgr2=0x7f3bf414a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3bf40b17b0 0x7f3bf414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 -- 172.21.2.111:0/1392787440 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf414aa70 con 0x7f3bf40add50 2021-04-03T03:15:30.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf27fc700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40add50 0x7f3bf4156950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f3be4002f30 tx=0x7f3be4006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bebfff700 1 -- 172.21.2.111:0/1392787440 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be400d7e0 con 0x7f3bf40add50 2021-04-03T03:15:30.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bebfff700 1 -- 172.21.2.111:0/1392787440 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400d940 con 0x7f3bf40add50 2021-04-03T03:15:30.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf414acc0 con 0x7f3bf40add50 2021-04-03T03:15:30.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bebfff700 1 -- 172.21.2.111:0/1392787440 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be400dca0 con 0x7f3bf40add50 2021-04-03T03:15:30.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.117+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf40a8f70 con 0x7f3bf40add50 2021-04-03T03:15:30.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.121+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf4057660 con 0x7f3bf40add50 2021-04-03T03:15:30.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.121+0000 7f3bebfff700 1 -- 172.21.2.111:0/1392787440 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3be4011480 con 0x7f3bf40add50 2021-04-03T03:15:30.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.121+0000 7f3bebfff700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3bd8056ec0 0x7f3bd8059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.121+0000 7f3bebfff700 1 -- 172.21.2.111:0/1392787440 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3be406fe40 con 0x7f3bf40add50 2021-04-03T03:15:30.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.121+0000 7f3bf2ffd700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3bd8056ec0 0x7f3bd8059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.121+0000 7f3bf2ffd700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3bd8056ec0 0x7f3bd8059380 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f3bec001450 tx=0x7f3bdc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.121+0000 7f3bebfff700 1 -- 172.21.2.111:0/1392787440 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3be40451d0 con 0x7f3bf40add50 2021-04-03T03:15:30.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3bf4148850 con 0x7f3bf40add50 2021-04-03T03:15:30.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bebfff700 1 -- 172.21.2.111:0/1392787440 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3be40487f0 con 0x7f3bf40add50 2021-04-03T03:15:30.221 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:30.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3bd8056ec0 msgr2=0x7f3bd8059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3bd8056ec0 0x7f3bd8059380 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f3bec001450 tx=0x7f3bdc015000).stop 2021-04-03T03:15:30.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40add50 msgr2=0x7f3bf4156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40add50 0x7f3bf4156950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f3be4002f30 tx=0x7f3be4006e50).stop 2021-04-03T03:15:30.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 shutdown_connections 2021-04-03T03:15:30.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3bd8056ec0 0x7f3bd8059380 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3bf40b17b0 0x7f3bf414a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3bf40603f0 0x7f3bf40a8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 --2- 172.21.2.111:0/1392787440 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3bf40add50 0x7f3bf4156950 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 >> 172.21.2.111:0/1392787440 conn(0x7f3bf40b61c0 msgr2=0x7f3bf4144600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 shutdown_connections 2021-04-03T03:15:30.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.217+0000 7f3bf3fff700 1 -- 172.21.2.111:0/1392787440 wait complete. 2021-04-03T03:15:30.229 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:15:30.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.281+0000 7f0423fff700 1 Processor -- start 2021-04-03T03:15:30.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.281+0000 7f0423fff700 1 -- start start 2021-04-03T03:15:30.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f041c0dbba0 0x7f041c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c041060 0x7f041c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f041c03d560 0x7f041c041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c03de60 con 0x7f041c03d560 2021-04-03T03:15:30.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0dc560 con 0x7f041c0dbba0 2021-04-03T03:15:30.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0dc800 con 0x7f041c041060 2021-04-03T03:15:30.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f041c03d560 0x7f041c041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f041c03d560 0x7f041c041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54774/0 (socket says 172.21.2.111:54774) 2021-04-03T03:15:30.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 -- 172.21.2.111:0/2727885466 learned_addr learned my addr 172.21.2.111:0/2727885466 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:30.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 -- 172.21.2.111:0/2727885466 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c041060 msgr2=0x7f041c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:30.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 --2- 172.21.2.111:0/2727885466 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c041060 0x7f041c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 -- 172.21.2.111:0/2727885466 >> v2:172.21.2.115:3300/0 conn(0x7f041c0dbba0 msgr2=0x7f041c0dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:30.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 --2- 172.21.2.111:0/2727885466 >> v2:172.21.2.115:3300/0 conn(0x7f041c0dbba0 0x7f041c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 -- 172.21.2.111:0/2727885466 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c03da30 con 0x7f041c03d560 2021-04-03T03:15:30.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04237fe700 1 --2- 172.21.2.111:0/2727885466 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c03d560 0x7f041c041650 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f042406b5d0 tx=0x7f042405ad70).ready entity=mon.0 client_cookie=c93cee41598b80b2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0421ffb700 1 -- 172.21.2.111:0/2727885466 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042405bda0 con 0x7f041c03d560 2021-04-03T03:15:30.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0421ffb700 1 -- 172.21.2.111:0/2727885466 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042405df60 con 0x7f041c03d560 2021-04-03T03:15:30.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0421ffb700 1 -- 172.21.2.111:0/2727885466 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04240699f0 con 0x7f041c03d560 2021-04-03T03:15:30.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- 172.21.2.111:0/2727885466 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c03d560 msgr2=0x7f041c041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- 172.21.2.111:0/2727885466 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c03d560 0x7f041c041650 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f042406b5d0 tx=0x7f042405ad70).stop 2021-04-03T03:15:30.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- 172.21.2.111:0/2727885466 shutdown_connections 2021-04-03T03:15:30.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- 172.21.2.111:0/2727885466 >> v2:172.21.2.115:3300/0 conn(0x7f041c0dbba0 0x7f041c0dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- 172.21.2.111:0/2727885466 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c03d560 0x7f041c041650 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- 172.21.2.111:0/2727885466 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c041060 0x7f041c03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- 172.21.2.111:0/2727885466 >> 172.21.2.111:0/2727885466 conn(0x7f041c0462b0 msgr2=0x7f041c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- 172.21.2.111:0/2727885466 shutdown_connections 2021-04-03T03:15:30.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- 172.21.2.111:0/2727885466 wait complete. 2021-04-03T03:15:30.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 Processor -- start 2021-04-03T03:15:30.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- start start 2021-04-03T03:15:30.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c03d560 0x7f041c0dee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c041060 0x7f041c0d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f041c0dbba0 0x7f041c0d7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c03c570 con 0x7f041c041060 2021-04-03T03:15:30.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c03c810 con 0x7f041c0dbba0 2021-04-03T03:15:30.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0423fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c03cab0 con 0x7f041c03d560 2021-04-03T03:15:30.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c041060 0x7f041c0d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c041060 0x7f041c0d77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54782/0 (socket says 172.21.2.111:54782) 2021-04-03T03:15:30.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 -- 172.21.2.111:0/2536741678 learned_addr learned my addr 172.21.2.111:0/2536741678 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:30.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 -- 172.21.2.111:0/2536741678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c03d560 msgr2=0x7f041c0dee60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:30.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c03d560 0x7f041c0dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 -- 172.21.2.111:0/2536741678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f041c0dbba0 msgr2=0x7f041c0d7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f041c0dbba0 0x7f041c0d7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 -- 172.21.2.111:0/2536741678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0d81b0 con 0x7f041c041060 2021-04-03T03:15:30.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f04227fc700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c041060 0x7f041c0d77d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f0418002f30 tx=0x7f0418006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0413fff700 1 -- 172.21.2.111:0/2536741678 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f041800d710 con 0x7f041c041060 2021-04-03T03:15:30.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.285+0000 7f0413fff700 1 -- 172.21.2.111:0/2536741678 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041800d870 con 0x7f041c041060 2021-04-03T03:15:30.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0413fff700 1 -- 172.21.2.111:0/2536741678 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f041800dbd0 con 0x7f041c041060 2021-04-03T03:15:30.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c03a730 con 0x7f041c041060 2021-04-03T03:15:30.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c039580 con 0x7f041c041060 2021-04-03T03:15:30.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c03b6e0 con 0x7f041c041060 2021-04-03T03:15:30.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0413fff700 1 -- 172.21.2.111:0/2536741678 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f041800dd30 con 0x7f041c041060 2021-04-03T03:15:30.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0413fff700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0400056ea0 0x7f0400059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0413fff700 1 -- 172.21.2.111:0/2536741678 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f041806fe70 con 0x7f041c041060 2021-04-03T03:15:30.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0422ffd700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0400056ea0 0x7f0400059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0422ffd700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0400056ea0 0x7f0400059360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0414002f30 tx=0x7f0414006970).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.289+0000 7f0413fff700 1 -- 172.21.2.111:0/2536741678 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0418045260 con 0x7f041c041060 2021-04-03T03:15:30.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f041c004db0 con 0x7f041c041060 2021-04-03T03:15:30.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0413fff700 1 -- 172.21.2.111:0/2536741678 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0418048880 con 0x7f041c041060 2021-04-03T03:15:30.388 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:30.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0400056ea0 msgr2=0x7f0400059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0400056ea0 0x7f0400059360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0414002f30 tx=0x7f0414006970).stop 2021-04-03T03:15:30.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c041060 msgr2=0x7f041c0d77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c041060 0x7f041c0d77d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f0418002f30 tx=0x7f0418006e80).stop 2021-04-03T03:15:30.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 shutdown_connections 2021-04-03T03:15:30.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0400056ea0 0x7f0400059360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f041c0dbba0 0x7f041c0d7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f041c03d560 0x7f041c0dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 --2- 172.21.2.111:0/2536741678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f041c041060 0x7f041c0d77d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 >> 172.21.2.111:0/2536741678 conn(0x7f041c0462b0 msgr2=0x7f041c03f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 shutdown_connections 2021-04-03T03:15:30.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.385+0000 7f0423fff700 1 -- 172.21.2.111:0/2536741678 wait complete. 2021-04-03T03:15:30.396 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:15:30.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.449+0000 7f6eac921700 1 Processor -- start 2021-04-03T03:15:30.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.449+0000 7f6eac921700 1 -- start start 2021-04-03T03:15:30.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80603f0 0x7f6ea81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6ea80add50 0x7f6ea8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea80607d0 con 0x7f6ea80b17b0 2021-04-03T03:15:30.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea81451c0 con 0x7f6ea80add50 2021-04-03T03:15:30.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea814b090 con 0x7f6ea80603f0 2021-04-03T03:15:30.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6ea80add50 0x7f6ea8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6ea80add50 0x7f6ea8144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54786/0 (socket says 172.21.2.111:54786) 2021-04-03T03:15:30.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 -- 172.21.2.111:0/2368941698 learned_addr learned my addr 172.21.2.111:0/2368941698 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:30.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 -- 172.21.2.111:0/2368941698 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80603f0 msgr2=0x7f6ea81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:30.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 --2- 172.21.2.111:0/2368941698 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80603f0 0x7f6ea81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 -- 172.21.2.111:0/2368941698 >> v1:172.21.2.115:6789/0 conn(0x7f6ea80b17b0 legacy=0x7f6ea8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:30.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 -- 172.21.2.111:0/2368941698 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea814b330 con 0x7f6ea80add50 2021-04-03T03:15:30.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea6ffd700 1 --2- 172.21.2.111:0/2368941698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80add50 0x7f6ea8144cd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f6ea80b5600 tx=0x7f6e98006e90).ready entity=mon.0 client_cookie=d250e51930a16c72 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea67fc700 1 -- 172.21.2.111:0/2368941698 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e980098e0 con 0x7f6ea80add50 2021-04-03T03:15:30.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea67fc700 1 -- 172.21.2.111:0/2368941698 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98009a40 con 0x7f6ea80add50 2021-04-03T03:15:30.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea67fc700 1 -- 172.21.2.111:0/2368941698 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e98009de0 con 0x7f6ea80add50 2021-04-03T03:15:30.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- 172.21.2.111:0/2368941698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80add50 msgr2=0x7f6ea8144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- 172.21.2.111:0/2368941698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80add50 0x7f6ea8144cd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f6ea80b5600 tx=0x7f6e98006e90).stop 2021-04-03T03:15:30.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- 172.21.2.111:0/2368941698 shutdown_connections 2021-04-03T03:15:30.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- 172.21.2.111:0/2368941698 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80603f0 0x7f6ea81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- 172.21.2.111:0/2368941698 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80add50 0x7f6ea8144cd0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- 172.21.2.111:0/2368941698 >> 172.21.2.111:0/2368941698 conn(0x7f6ea80b61c0 msgr2=0x7f6ea805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- 172.21.2.111:0/2368941698 shutdown_connections 2021-04-03T03:15:30.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- 172.21.2.111:0/2368941698 wait complete. 2021-04-03T03:15:30.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 Processor -- start 2021-04-03T03:15:30.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- start start 2021-04-03T03:15:30.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80603f0 0x7f6ea80a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80add50 0x7f6ea80abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ea80b17b0 0x7f6ea80ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80a95b0 con 0x7f6ea80603f0 2021-04-03T03:15:30.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80a9850 con 0x7f6ea80b17b0 2021-04-03T03:15:30.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8063a60 con 0x7f6ea80add50 2021-04-03T03:15:30.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80603f0 0x7f6ea80a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80603f0 0x7f6ea80a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54794/0 (socket says 172.21.2.111:54794) 2021-04-03T03:15:30.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 -- 172.21.2.111:0/1432115090 learned_addr learned my addr 172.21.2.111:0/1432115090 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:30.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 -- 172.21.2.111:0/1432115090 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80add50 msgr2=0x7f6ea80abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80add50 0x7f6ea80abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 -- 172.21.2.111:0/1432115090 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ea80b17b0 msgr2=0x7f6ea80ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ea80b17b0 0x7f6ea80ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 -- 172.21.2.111:0/1432115090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea80ab570 con 0x7f6ea80603f0 2021-04-03T03:15:30.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea77fe700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80603f0 0x7f6ea80a9b40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f6e9c002f30 tx=0x7f6e9c006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea4ff9700 1 -- 172.21.2.111:0/1432115090 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e9c00d840 con 0x7f6ea80603f0 2021-04-03T03:15:30.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea4ff9700 1 -- 172.21.2.111:0/1432115090 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c000e90 con 0x7f6ea80603f0 2021-04-03T03:15:30.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6ea4ff9700 1 -- 172.21.2.111:0/1432115090 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e9c00db90 con 0x7f6ea80603f0 2021-04-03T03:15:30.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea8063f00 con 0x7f6ea80603f0 2021-04-03T03:15:30.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.453+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea80643c0 con 0x7f6ea80603f0 2021-04-03T03:15:30.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.457+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea80ac090 con 0x7f6ea80603f0 2021-04-03T03:15:30.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.457+0000 7f6ea4ff9700 1 -- 172.21.2.111:0/1432115090 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6e9c017030 con 0x7f6ea80603f0 2021-04-03T03:15:30.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.457+0000 7f6ea4ff9700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e88056f20 0x7f6e880593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.457+0000 7f6ea4ff9700 1 -- 172.21.2.111:0/1432115090 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f6e9c06f4b0 con 0x7f6ea80603f0 2021-04-03T03:15:30.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.457+0000 7f6ea6ffd700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e88056f20 0x7f6e880593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.457+0000 7f6ea6ffd700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e88056f20 0x7f6e880593e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f6ea80b5600 tx=0x7f6e98015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.457+0000 7f6ea4ff9700 1 -- 172.21.2.111:0/1432115090 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6e9c0448f0 con 0x7f6ea80603f0 2021-04-03T03:15:30.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ea8063c20 con 0x7f6ea80603f0 2021-04-03T03:15:30.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6ea4ff9700 1 -- 172.21.2.111:0/1432115090 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6e9c047f10 con 0x7f6ea80603f0 2021-04-03T03:15:30.556 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:30.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e88056f20 msgr2=0x7f6e880593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e88056f20 0x7f6e880593e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f6ea80b5600 tx=0x7f6e98015000).stop 2021-04-03T03:15:30.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80603f0 msgr2=0x7f6ea80a9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80603f0 0x7f6ea80a9b40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f6e9c002f30 tx=0x7f6e9c006ea0).stop 2021-04-03T03:15:30.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 shutdown_connections 2021-04-03T03:15:30.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6e88056f20 0x7f6e880593e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6ea80b17b0 0x7f6ea80ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6ea80603f0 0x7f6ea80a9b40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 --2- 172.21.2.111:0/1432115090 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6ea80add50 0x7f6ea80abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 >> 172.21.2.111:0/1432115090 conn(0x7f6ea80b61c0 msgr2=0x7f6ea80b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 shutdown_connections 2021-04-03T03:15:30.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.553+0000 7f6eac921700 1 -- 172.21.2.111:0/1432115090 wait complete. 2021-04-03T03:15:30.564 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:15:30.564 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:30.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 Processor -- start 2021-04-03T03:15:30.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- start start 2021-04-03T03:15:30.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f554014b700 0x7f55400b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f55400b0c90 0x7f5540147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55400af400 con 0x7f55400603f0 2021-04-03T03:15:30.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5540060860 con 0x7f55400b0c90 2021-04-03T03:15:30.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400af6a0 con 0x7f554014b700 2021-04-03T03:15:30.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f55400b0c90 0x7f5540147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f55400b0c90 0x7f5540147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54802/0 (socket says 172.21.2.111:54802) 2021-04-03T03:15:30.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 -- 172.21.2.111:0/3894389149 learned_addr learned my addr 172.21.2.111:0/3894389149 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:30.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 -- 172.21.2.111:0/3894389149 >> v2:172.21.2.115:3300/0 conn(0x7f554014b700 msgr2=0x7f55400b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:30.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 --2- 172.21.2.111:0/3894389149 >> v2:172.21.2.115:3300/0 conn(0x7f554014b700 0x7f55400b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 -- 172.21.2.111:0/3894389149 >> v1:172.21.2.115:6789/0 conn(0x7f55400603f0 legacy=0x7f55400b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:30.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 -- 172.21.2.111:0/3894389149 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400afb40 con 0x7f55400b0c90 2021-04-03T03:15:30.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553ed9d700 1 --2- 172.21.2.111:0/3894389149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400b0c90 0x7f5540147680 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f554006a3b0 tx=0x7f5538009270).ready entity=mon.0 client_cookie=b1e5820ddde785b7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553d59a700 1 -- 172.21.2.111:0/3894389149 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f553800bbc0 con 0x7f55400b0c90 2021-04-03T03:15:30.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553d59a700 1 -- 172.21.2.111:0/3894389149 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553800bd20 con 0x7f55400b0c90 2021-04-03T03:15:30.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553d59a700 1 -- 172.21.2.111:0/3894389149 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f553800be90 con 0x7f55400b0c90 2021-04-03T03:15:30.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- 172.21.2.111:0/3894389149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400b0c90 msgr2=0x7f5540147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- 172.21.2.111:0/3894389149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400b0c90 0x7f5540147680 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f554006a3b0 tx=0x7f5538009270).stop 2021-04-03T03:15:30.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- 172.21.2.111:0/3894389149 shutdown_connections 2021-04-03T03:15:30.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- 172.21.2.111:0/3894389149 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400b0c90 0x7f5540147680 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- 172.21.2.111:0/3894389149 >> v2:172.21.2.115:3300/0 conn(0x7f554014b700 0x7f55400b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- 172.21.2.111:0/3894389149 >> 172.21.2.111:0/3894389149 conn(0x7f55400b61c0 msgr2=0x7f554005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- 172.21.2.111:0/3894389149 shutdown_connections 2021-04-03T03:15:30.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- 172.21.2.111:0/3894389149 wait complete. 2021-04-03T03:15:30.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 Processor -- start 2021-04-03T03:15:30.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- start start 2021-04-03T03:15:30.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400603f0 0x7f5540063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f55400b0c90 0x7f55400a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f554014b700 0x7f55400a8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5540063cf0 con 0x7f55400603f0 2021-04-03T03:15:30.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5540063f50 con 0x7f55400b0c90 2021-04-03T03:15:30.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55401571b0 con 0x7f554014b700 2021-04-03T03:15:30.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400603f0 0x7f5540063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400603f0 0x7f5540063800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54804/0 (socket says 172.21.2.111:54804) 2021-04-03T03:15:30.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 -- 172.21.2.111:0/1274104434 learned_addr learned my addr 172.21.2.111:0/1274104434 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:30.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 -- 172.21.2.111:0/1274104434 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f554014b700 msgr2=0x7f55400a8420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:30.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f554014b700 0x7f55400a8420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 -- 172.21.2.111:0/1274104434 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f55400b0c90 msgr2=0x7f55400a7f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:30.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f55400b0c90 0x7f55400a7f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 -- 172.21.2.111:0/1274104434 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55401576e0 con 0x7f55400603f0 2021-04-03T03:15:30.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.617+0000 7f553e59c700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400603f0 0x7f5540063800 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f5530002f50 tx=0x7f5530006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f552f7fe700 1 -- 172.21.2.111:0/1274104434 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f553000d4a0 con 0x7f55400603f0 2021-04-03T03:15:30.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f552f7fe700 1 -- 172.21.2.111:0/1274104434 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553000d600 con 0x7f55400603f0 2021-04-03T03:15:30.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55400a9370 con 0x7f55400603f0 2021-04-03T03:15:30.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55400a8aa0 con 0x7f55400603f0 2021-04-03T03:15:30.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f552f7fe700 1 -- 172.21.2.111:0/1274104434 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f553000d960 con 0x7f55400603f0 2021-04-03T03:15:30.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5540157370 con 0x7f55400603f0 2021-04-03T03:15:30.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f552f7fe700 1 -- 172.21.2.111:0/1274104434 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f553000f070 con 0x7f55400603f0 2021-04-03T03:15:30.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f552f7fe700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f551c056ea0 0x7f551c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:30.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f552f7fe700 1 -- 172.21.2.111:0/1274104434 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f553006fd40 con 0x7f55400603f0 2021-04-03T03:15:30.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f553dd9b700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f551c056ea0 0x7f551c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:30.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f553dd9b700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f551c056ea0 0x7f551c059360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f5534002f30 tx=0x7f5534006970).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:30.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.621+0000 7f552f7fe700 1 -- 172.21.2.111:0/1274104434 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5530045020 con 0x7f55400603f0 2021-04-03T03:15:30.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5540057660 con 0x7f55400603f0 2021-04-03T03:15:30.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f552f7fe700 1 -- 172.21.2.111:0/1274104434 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f553006f020 con 0x7f55400603f0 2021-04-03T03:15:30.750 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:30.750 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2065583567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2065583567},{"type":"v1","addr":"172.21.2.115:6833","nonce":2065583567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:25.328682+0000","modified":"2021-04-03T03:15:26.331659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/485806052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":485806052},{"type":"v1","addr":"172.21.2.111:6835","nonce":485806052}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-04-03T03:15:30.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f551c056ea0 msgr2=0x7f551c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f551c056ea0 0x7f551c059360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f5534002f30 tx=0x7f5534006970).stop 2021-04-03T03:15:30.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400603f0 msgr2=0x7f5540063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:30.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400603f0 0x7f5540063800 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f5530002f50 tx=0x7f5530006dd0).stop 2021-04-03T03:15:30.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 shutdown_connections 2021-04-03T03:15:30.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f554014b700 0x7f55400a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f55400603f0 0x7f5540063800 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f551c056ea0 0x7f551c059360 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:30.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 --2- 172.21.2.111:0/1274104434 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f55400b0c90 0x7f55400a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:30.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 >> 172.21.2.111:0/1274104434 conn(0x7f55400b61c0 msgr2=0x7f5540146a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:30.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 shutdown_connections 2021-04-03T03:15:30.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:30.745+0000 7f553f59e700 1 -- 172.21.2.111:0/1274104434 wait complete. 2021-04-03T03:15:30.757 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 33 2021-04-03T03:15:30.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:15:25.328682+0000', 'modified': '2021-04-03T03:15:26.331659+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6573': {'gid': 6573, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/485806052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 485806052}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 485806052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:15:30.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:15:30.961 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:15:31.037 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6573,"want_state":"up:active","state":"up:active","rank_uptime":5.7013748509999997,"mdsmap_epoch":33,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":9.8399648430000006} 2021-04-03T03:15:31.038 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6573, "mdsmap_epoch": 33, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 5.701374851, "state": "up:active", "uptime": 9.839964843, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:15:31.038 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:15:31.039 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:15:31.039 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:15:31.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.040 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:31.050 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:15:31.103 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:31.104 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:31.151 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:31.151 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:31.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.153 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:15:31.201 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:15:31.202 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:31.248 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:15:31.257 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:31.258 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:31.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:15:31.283+0000 7f0fdbb230c0 -1 init, newargv = 0x55bf764e1620 newargc=9 2021-04-03T03:15:31.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15152]: starting ceph client 2021-04-03T03:15:31.292 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:31.292 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:31.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.294 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:15:31.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15152]: starting fuse 2021-04-03T03:15:31.301 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:15:31.302 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:15:31.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.303 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:15:31.303 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:15:31.303 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:15:31.304 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:15:31.304 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:15:31.305 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:31.305 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:15:31.305 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:15:31.306 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:15:31.306 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:31.307 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:15:31.307 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:15:31.307 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:15:31.308 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:15:31.308 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:31.309 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:15:31.309 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:15:31.309 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:15:31.310 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:15:31.310 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:15:31.310 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:31.311 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:15:31.311 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:15:31.395 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15152.asok 2021-04-03T03:15:31.399 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15152.asok 2021-04-03T03:15:31.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.400 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15152.asok status 2021-04-03T03:15:31.466 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:15:31.466 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:15:31.466 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:15:31.467 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:15:31.467 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:15:31.467 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:15:31.467 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:15:31.467 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15152", 2021-04-03T03:15:31.468 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:15:31.468 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:31.468 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:15:31.468 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:15:31.469 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 5122, 2021-04-03T03:15:31.469 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:15:31.469 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:15:31.469 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:15:31.469 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 5122 2021-04-03T03:15:31.470 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:31.470 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:15:31.470 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:15:31.470 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:15:31.470 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 1651116299 2021-04-03T03:15:31.471 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:15:31.471 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:31.471 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:15:31.471 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:15:31.472 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:15:31.472 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 1651116299 2021-04-03T03:15:31.472 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:31.472 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.5122 172.21.2.128:0/1651116299", 2021-04-03T03:15:31.473 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/1651116299", 2021-04-03T03:15:31.473 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:15:31.473 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 33, 2021-04-03T03:15:31.473 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 84, 2021-04-03T03:15:31.473 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:15:31.474 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:15:31.474 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:15:31.474 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:15:31.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.476 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:31.476 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:31.481 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:15:31.482 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:31.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.482 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:31.534 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:31.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 Processor -- start 2021-04-03T03:15:31.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- start start 2021-04-03T03:15:31.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f19e01437f0 0x7f19e00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:31.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e00aee70 0x7f19e0139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:31.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f19e00ad3f0 0x7f19e013a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:31.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e005cb80 con 0x7f19e00ad3f0 2021-04-03T03:15:31.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00ae490 con 0x7f19e01437f0 2021-04-03T03:15:31.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e005c8e0 con 0x7f19e00aee70 2021-04-03T03:15:31.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f19e00ad3f0 0x7f19e013a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:31.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f19e00ad3f0 0x7f19e013a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54812/0 (socket says 172.21.2.111:54812) 2021-04-03T03:15:31.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 -- 172.21.2.111:0/770601502 learned_addr learned my addr 172.21.2.111:0/770601502 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:31.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 -- 172.21.2.111:0/770601502 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e00aee70 msgr2=0x7f19e0139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:31.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 --2- 172.21.2.111:0/770601502 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e00aee70 0x7f19e0139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 -- 172.21.2.111:0/770601502 >> v2:172.21.2.115:3300/0 conn(0x7f19e01437f0 msgr2=0x7f19e00b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:31.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 --2- 172.21.2.111:0/770601502 >> v2:172.21.2.115:3300/0 conn(0x7f19e01437f0 0x7f19e00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 -- 172.21.2.111:0/770601502 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00ace40 con 0x7f19e00ad3f0 2021-04-03T03:15:31.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e6b5c700 1 --2- 172.21.2.111:0/770601502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00ad3f0 0x7f19e013a2b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f19d00032b0 tx=0x7f19d0009270).ready entity=mon.0 client_cookie=cb5c0a7f61fda871 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:31.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5359700 1 -- 172.21.2.111:0/770601502 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d000bc00 con 0x7f19e00ad3f0 2021-04-03T03:15:31.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5359700 1 -- 172.21.2.111:0/770601502 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d000bd60 con 0x7f19e00ad3f0 2021-04-03T03:15:31.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5359700 1 -- 172.21.2.111:0/770601502 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d0011400 con 0x7f19e00ad3f0 2021-04-03T03:15:31.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- 172.21.2.111:0/770601502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00ad3f0 msgr2=0x7f19e013a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:31.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- 172.21.2.111:0/770601502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00ad3f0 0x7f19e013a2b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f19d00032b0 tx=0x7f19d0009270).stop 2021-04-03T03:15:31.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- 172.21.2.111:0/770601502 shutdown_connections 2021-04-03T03:15:31.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- 172.21.2.111:0/770601502 >> v2:172.21.2.115:3300/0 conn(0x7f19e01437f0 0x7f19e00b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- 172.21.2.111:0/770601502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00ad3f0 0x7f19e013a2b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:31.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- 172.21.2.111:0/770601502 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e00aee70 0x7f19e0139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- 172.21.2.111:0/770601502 >> 172.21.2.111:0/770601502 conn(0x7f19e00b5fb0 msgr2=0x7f19e00b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:31.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- 172.21.2.111:0/770601502 shutdown_connections 2021-04-03T03:15:31.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- 172.21.2.111:0/770601502 wait complete. 2021-04-03T03:15:31.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 Processor -- start 2021-04-03T03:15:31.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- start start 2021-04-03T03:15:31.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19e00ad3f0 0x7f19e013b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:31.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00aee70 0x7f19e0140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:31.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e01437f0 0x7f19e0065e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:31.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0064b00 con 0x7f19e00aee70 2021-04-03T03:15:31.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0140d90 con 0x7f19e00ad3f0 2021-04-03T03:15:31.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e735d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00656f0 con 0x7f19e01437f0 2021-04-03T03:15:31.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5b5a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00aee70 0x7f19e0140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:31.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5b5a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00aee70 0x7f19e0140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54822/0 (socket says 172.21.2.111:54822) 2021-04-03T03:15:31.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5b5a700 1 -- 172.21.2.111:0/863800920 learned_addr learned my addr 172.21.2.111:0/863800920 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:31.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5b5a700 1 -- 172.21.2.111:0/863800920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e01437f0 msgr2=0x7f19e0065e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:31.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5b5a700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e01437f0 0x7f19e0065e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.593+0000 7f19e5b5a700 1 -- 172.21.2.111:0/863800920 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19e00ad3f0 msgr2=0x7f19e013b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:31.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e5b5a700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19e00ad3f0 0x7f19e013b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e5b5a700 1 -- 172.21.2.111:0/863800920 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0065990 con 0x7f19e00aee70 2021-04-03T03:15:31.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e5b5a700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00aee70 0x7f19e0140560 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f19d4002f30 tx=0x7f19d4006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:31.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19df7fe700 1 -- 172.21.2.111:0/863800920 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d400d840 con 0x7f19e00aee70 2021-04-03T03:15:31.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19df7fe700 1 -- 172.21.2.111:0/863800920 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4000e90 con 0x7f19e00aee70 2021-04-03T03:15:31.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19df7fe700 1 -- 172.21.2.111:0/863800920 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d400db90 con 0x7f19e00aee70 2021-04-03T03:15:31.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e0140a50 con 0x7f19e00aee70 2021-04-03T03:15:31.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e013fc20 con 0x7f19e00aee70 2021-04-03T03:15:31.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e0065b20 con 0x7f19e00aee70 2021-04-03T03:15:31.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19df7fe700 1 -- 172.21.2.111:0/863800920 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f19d4017030 con 0x7f19e00aee70 2021-04-03T03:15:31.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19df7fe700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0057000 0x7f19c00594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19df7fe700 1 -- 172.21.2.111:0/863800920 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f19d406f760 con 0x7f19e00aee70 2021-04-03T03:15:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e635b700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0057000 0x7f19c00594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.597+0000 7f19e635b700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0057000 0x7f19c00594c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f19d8002f30 tx=0x7f19d8006950).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:31.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.601+0000 7f19df7fe700 1 -- 172.21.2.111:0/863800920 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f19d4044af0 con 0x7f19e00aee70 2021-04-03T03:15:31.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19e0140be0 con 0x7f19e00aee70 2021-04-03T03:15:31.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19df7fe700 1 -- 172.21.2.111:0/863800920 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f19d4048110 con 0x7f19e00aee70 2021-04-03T03:15:31.726 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:31.726 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2065583567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2065583567},{"type":"v1","addr":"172.21.2.115:6833","nonce":2065583567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:25.328682+0000","modified":"2021-04-03T03:15:26.331659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/485806052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":485806052},{"type":"v1","addr":"172.21.2.111:6835","nonce":485806052}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-04-03T03:15:31.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0057000 msgr2=0x7f19c00594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:31.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0057000 0x7f19c00594c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f19d8002f30 tx=0x7f19d8006950).stop 2021-04-03T03:15:31.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00aee70 msgr2=0x7f19e0140560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:31.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00aee70 0x7f19e0140560 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f19d4002f30 tx=0x7f19d4006ea0).stop 2021-04-03T03:15:31.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 shutdown_connections 2021-04-03T03:15:31.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f19e01437f0 0x7f19e0065e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f19c0057000 0x7f19c00594c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:31.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19e00ad3f0 0x7f19e013b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:31.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 --2- 172.21.2.111:0/863800920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f19e00aee70 0x7f19e0140560 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:31.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 >> 172.21.2.111:0/863800920 conn(0x7f19e00b5fb0 msgr2=0x7f19e00b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:31.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 shutdown_connections 2021-04-03T03:15:31.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:31.721+0000 7f19e735d700 1 -- 172.21.2.111:0/863800920 wait complete. 2021-04-03T03:15:31.730 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 33 2021-04-03T03:15:31.734 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1 2021-04-03T03:15:31.798 INFO:teuthology.orchestra.run.gibba011.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2021-04-03T03:15:29.709399+0000","btime":"2021-04-03T03:15:25.331147+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2021-04-03T03:15:25.331147+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2021-04-03T03:15:29.709399+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2021-04-03T03:15:25.331147+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":4781,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":5122,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2021-04-03T03:15:31.799 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2021-04-03T03:15:25.331147+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2021-04-03T03:15:25.331147+0000", "change_attr": 1, "client_caps": [ { "client_id": 4781, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 5122, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2021-04-03T03:15:29.709399+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2021-04-03T03:15:25.331147+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2021-04-03T03:15:29.709399+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2021-04-03T03:15:31.800 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:15:31.800 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:31.806 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:31.814 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:31.826 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:31.837 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:31.850 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:15:31.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.850 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:31.851 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:31.873 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:15:31.874 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:31.874 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:15:31.875 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:15:31.875 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:31.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15624]: fuse finished with error 0 and tester_r 0 2021-04-03T03:15:31.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:31.964 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:31.965 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:31.970 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:15:31.971 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:15:32.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:32.039 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:32.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:32.043 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:32.044 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:32.054 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:15:32.056 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:32.056 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:15:32.057 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:15:32.057 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:32.138 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15152]: fuse finished with error 0 and tester_r 0 2021-04-03T03:15:32.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:32.139 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:32.140 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:32.189 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:15:32.190 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:15:32.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:32.287 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:32.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:32.291 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:32.291 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:32.301 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:32.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:32.302 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:15:32.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:32.303 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:32.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:32.408 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:32.412 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:15:32.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6e57700 1 Processor -- start 2021-04-03T03:15:32.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6e57700 1 -- start start 2021-04-03T03:15:32.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6e57700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd2f00603f0 0x7fd2f01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6e57700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00b17b0 0x7fd2f0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6e57700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f00607d0 con 0x7fd2f00add50 2021-04-03T03:15:32.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6e57700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f01451c0 con 0x7fd2f00603f0 2021-04-03T03:15:32.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6e57700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f014b090 con 0x7fd2f00b17b0 2021-04-03T03:15:32.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f5e55700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd2f00603f0 0x7fd2f01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f6656700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00b17b0 0x7fd2f0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f5e55700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd2f00603f0 0x7fd2f01447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:44534/0 (socket says 172.21.2.111:44534) 2021-04-03T03:15:32.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.489+0000 7fd2f5e55700 1 -- 172.21.2.111:0/2551867172 learned_addr learned my addr 172.21.2.111:0/2551867172 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:32.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 -- 172.21.2.111:0/2551867172 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00b17b0 msgr2=0x7fd2f0144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 --2- 172.21.2.111:0/2551867172 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00b17b0 0x7fd2f0144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 -- 172.21.2.111:0/2551867172 >> v1:172.21.2.115:6789/0 conn(0x7fd2f00add50 legacy=0x7fd2f0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 -- 172.21.2.111:0/2551867172 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f014b330 con 0x7fd2f00603f0 2021-04-03T03:15:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 --2- 172.21.2.111:0/2551867172 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00603f0 0x7fd2f01447e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd2e4002f30 tx=0x7fd2e4006e90).ready entity=mon.1 client_cookie=ff24f791ab631e0e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f4e53700 1 -- 172.21.2.111:0/2551867172 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e4009850 con 0x7fd2f00603f0 2021-04-03T03:15:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f4e53700 1 -- 172.21.2.111:0/2551867172 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e40099b0 con 0x7fd2f00603f0 2021-04-03T03:15:32.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f4e53700 1 -- 172.21.2.111:0/2551867172 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e4009d10 con 0x7fd2f00603f0 2021-04-03T03:15:32.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- 172.21.2.111:0/2551867172 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00603f0 msgr2=0x7fd2f01447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/2551867172 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00603f0 0x7fd2f01447e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd2e4002f30 tx=0x7fd2e4006e90).stop 2021-04-03T03:15:32.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- 172.21.2.111:0/2551867172 shutdown_connections 2021-04-03T03:15:32.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/2551867172 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00603f0 0x7fd2f01447e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/2551867172 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00b17b0 0x7fd2f0144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- 172.21.2.111:0/2551867172 >> 172.21.2.111:0/2551867172 conn(0x7fd2f00b61c0 msgr2=0x7fd2f005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:32.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- 172.21.2.111:0/2551867172 shutdown_connections 2021-04-03T03:15:32.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- 172.21.2.111:0/2551867172 wait complete. 2021-04-03T03:15:32.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 Processor -- start 2021-04-03T03:15:32.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- start start 2021-04-03T03:15:32.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00603f0 0x7fd2f0149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00add50 0x7fd2f0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00b17b0 0x7fd2f0157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f015a530 con 0x7fd2f00603f0 2021-04-03T03:15:32.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f015a7d0 con 0x7fd2f00b17b0 2021-04-03T03:15:32.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6e57700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f015aa70 con 0x7fd2f00add50 2021-04-03T03:15:32.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00603f0 0x7fd2f0149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00603f0 0x7fd2f0149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54834/0 (socket says 172.21.2.111:54834) 2021-04-03T03:15:32.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 -- 172.21.2.111:0/3598252707 learned_addr learned my addr 172.21.2.111:0/3598252707 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:32.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f6656700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00b17b0 0x7fd2f0157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5654700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00add50 0x7fd2f0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 -- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00add50 msgr2=0x7fd2f0156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00add50 0x7fd2f0156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 -- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00b17b0 msgr2=0x7fd2f0157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00b17b0 0x7fd2f0157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 -- 172.21.2.111:0/3598252707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f015a1b0 con 0x7fd2f00603f0 2021-04-03T03:15:32.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.493+0000 7fd2f5e55700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00603f0 0x7fd2f0149f20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7fd2e4002f30 tx=0x7fd2e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:32.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2e2ffd700 1 -- 172.21.2.111:0/3598252707 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e4011bf0 con 0x7fd2f00603f0 2021-04-03T03:15:32.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2e2ffd700 1 -- 172.21.2.111:0/3598252707 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e4011d50 con 0x7fd2f00603f0 2021-04-03T03:15:32.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f00a8b30 con 0x7fd2f00603f0 2021-04-03T03:15:32.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2e2ffd700 1 -- 172.21.2.111:0/3598252707 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e401e650 con 0x7fd2f00603f0 2021-04-03T03:15:32.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f015b120 con 0x7fd2f00603f0 2021-04-03T03:15:32.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f015a340 con 0x7fd2f00603f0 2021-04-03T03:15:32.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2e2ffd700 1 -- 172.21.2.111:0/3598252707 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd2e4025030 con 0x7fd2f00603f0 2021-04-03T03:15:32.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2e2ffd700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2d0056f20 0x7fd2d00593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2e2ffd700 1 -- 172.21.2.111:0/3598252707 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd2e40712b0 con 0x7fd2f00603f0 2021-04-03T03:15:32.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2f5654700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2d0056f20 0x7fd2d00593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2f5654700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2d0056f20 0x7fd2d00593e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fd2ec00b000 tx=0x7fd2ec009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:32.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.497+0000 7fd2e2ffd700 1 -- 172.21.2.111:0/3598252707 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd2e40466a0 con 0x7fd2f00603f0 2021-04-03T03:15:32.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.589+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2f015b120 con 0x7fd2f00603f0 2021-04-03T03:15:32.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.593+0000 7fd2e2ffd700 1 -- 172.21.2.111:0/3598252707 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd2f015b120 con 0x7fd2f00603f0 2021-04-03T03:15:32.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2d0056f20 msgr2=0x7fd2d00593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2d0056f20 0x7fd2d00593e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fd2ec00b000 tx=0x7fd2ec009040).stop 2021-04-03T03:15:32.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00603f0 msgr2=0x7fd2f0149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00603f0 0x7fd2f0149f20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7fd2e4002f30 tx=0x7fd2e4015000).stop 2021-04-03T03:15:32.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 shutdown_connections 2021-04-03T03:15:32.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2d0056f20 0x7fd2d00593e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00b17b0 0x7fd2f0157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00603f0 0x7fd2f0149f20 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 --2- 172.21.2.111:0/3598252707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2f00add50 0x7fd2f0156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 >> 172.21.2.111:0/3598252707 conn(0x7fd2f00b61c0 msgr2=0x7fd2f00b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:32.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 shutdown_connections 2021-04-03T03:15:32.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.597+0000 7fd2f6e57700 1 -- 172.21.2.111:0/3598252707 wait complete. 2021-04-03T03:15:32.607 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2021-04-03T03:15:32.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 Processor -- start 2021-04-03T03:15:32.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- start start 2021-04-03T03:15:32.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 0x7f67180b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f67180b0c90 0x7f6718147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67180607d0 con 0x7f67180603f0 2021-04-03T03:15:32.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67180af400 con 0x7f67180b0c90 2021-04-03T03:15:32.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67180af6a0 con 0x7f671814b700 2021-04-03T03:15:32.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6717fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f67180603f0 0x7f67180b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55632/0 (socket says 172.21.2.111:55632) 2021-04-03T03:15:32.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6717fff700 1 -- 172.21.2.111:0/3199754707 learned_addr learned my addr 172.21.2.111:0/3199754707 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:32.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671cf04700 1 --2- 172.21.2.111:0/3199754707 >> v2:172.21.2.115:3300/0 conn(0x7f67180b0c90 0x7f6718147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 0x7f67180b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098701608 0 0) 0x7f67180607d0 con 0x7f67180603f0 2021-04-03T03:15:32.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f8003240 con 0x7f67180603f0 2021-04-03T03:15:32.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1449342871 0 0) 0x7f66f8003240 con 0x7f67180603f0 2021-04-03T03:15:32.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 msgr2=0x7f67180b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:32.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 0x7f67180b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 >> v2:172.21.2.115:3300/0 conn(0x7f67180b0c90 msgr2=0x7f6718147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:32.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 --2- 172.21.2.111:0/3199754707 >> v2:172.21.2.115:3300/0 conn(0x7f67180b0c90 0x7f6718147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180afb40 con 0x7f67180603f0 2021-04-03T03:15:32.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f67080016d0 con 0x7f67180603f0 2021-04-03T03:15:32.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 >> v1:172.21.2.111:6789/0 conn(0x7f67180603f0 legacy=0x7f67180b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 0x7f66f8003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66f8003c40 0x7f66f80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66f80049d0 0x7f66f8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180af400 con 0x7f66f8003c40 2021-04-03T03:15:32.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180af6a0 con 0x7f66f80049d0 2021-04-03T03:15:32.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f8004200 con 0x7f671814b700 2021-04-03T03:15:32.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66f8003c40 0x7f66f80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671cf04700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66f80049d0 0x7f66f8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6717fff700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 0x7f66f8003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 -- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 msgr2=0x7f66f8003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:32.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f671814b700 0x7f66f8003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 -- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66f80049d0 msgr2=0x7f66f8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:32.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66f80049d0 0x7f66f8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 -- 172.21.2.111:0/3199754707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8003240 con 0x7f66f8003c40 2021-04-03T03:15:32.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f67177fe700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66f8003c40 0x7f66f80044e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f670000c000 tx=0x7f6700008040).ready entity=mon.0 client_cookie=1e975bd2cbbde506 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:32.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670000e910 con 0x7f66f8003c40 2021-04-03T03:15:32.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670000ea70 con 0x7f66f8003c40 2021-04-03T03:15:32.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f6716ffd700 1 -- 172.21.2.111:0/3199754707 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670000edd0 con 0x7f66f8003c40 2021-04-03T03:15:32.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- 172.21.2.111:0/3199754707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66f8003c40 msgr2=0x7f66f80044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 --2- 172.21.2.111:0/3199754707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66f8003c40 0x7f66f80044e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f670000c000 tx=0x7f6700008040).stop 2021-04-03T03:15:32.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671cf04700 1 -- 172.21.2.111:0/3199754707 reap_dead start 2021-04-03T03:15:32.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- 172.21.2.111:0/3199754707 shutdown_connections 2021-04-03T03:15:32.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- 172.21.2.111:0/3199754707 >> 172.21.2.111:0/3199754707 conn(0x7f67180b61c0 msgr2=0x7f671805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:32.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- 172.21.2.111:0/3199754707 shutdown_connections 2021-04-03T03:15:32.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.661+0000 7f671d705700 1 -- 172.21.2.111:0/3199754707 wait complete. 2021-04-03T03:15:32.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 Processor -- start 2021-04-03T03:15:32.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 -- start start 2021-04-03T03:15:32.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f67180a8290 0x7f671815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f67180ab9e0 0x7f6718149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6718156220 0x7f67181566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671814a3a0 con 0x7f67180a8290 2021-04-03T03:15:32.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180a8670 con 0x7f6718156220 2021-04-03T03:15:32.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6718159810 con 0x7f67180ab9e0 2021-04-03T03:15:32.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f67180a8290 0x7f671815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f67180a8290 0x7f671815a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54852/0 (socket says 172.21.2.111:54852) 2021-04-03T03:15:32.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 -- 172.21.2.111:0/559516718 learned_addr learned my addr 172.21.2.111:0/559516718 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:32.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67177fe700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f67180ab9e0 0x7f6718149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671cf04700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6718156220 0x7f67181566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 -- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f67180ab9e0 msgr2=0x7f6718149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f67180ab9e0 0x7f6718149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 -- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6718156220 msgr2=0x7f67181566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:32.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6718156220 0x7f67181566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:32.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 -- 172.21.2.111:0/559516718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718159ab0 con 0x7f67180a8290 2021-04-03T03:15:32.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67177fe700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f67180ab9e0 0x7f6718149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:32.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671cf04700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6718156220 0x7f67181566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:32.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f6717fff700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f67180a8290 0x7f671815a1f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f670800b000 tx=0x7f6708001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:32.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67157fa700 1 -- 172.21.2.111:0/559516718 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670800ca10 con 0x7f67180a8290 2021-04-03T03:15:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67157fa700 1 -- 172.21.2.111:0/559516718 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670800cb70 con 0x7f67180a8290 2021-04-03T03:15:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6718159d50 con 0x7f67180a8290 2021-04-03T03:15:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67157fa700 1 -- 172.21.2.111:0/559516718 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6708014430 con 0x7f67180a8290 2021-04-03T03:15:32.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67181455c0 con 0x7f67180a8290 2021-04-03T03:15:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6718159f10 con 0x7f67180a8290 2021-04-03T03:15:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67157fa700 1 -- 172.21.2.111:0/559516718 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6708014590 con 0x7f67180a8290 2021-04-03T03:15:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67157fa700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f66fc056ea0 0x7f66fc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:32.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67157fa700 1 -- 172.21.2.111:0/559516718 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f67080700e0 con 0x7f67180a8290 2021-04-03T03:15:32.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67177fe700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f66fc056ea0 0x7f66fc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:32.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.665+0000 7f67177fe700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f66fc056ea0 0x7f66fc059360 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f6700007000 tx=0x7f670001a000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:32.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.669+0000 7f67157fa700 1 -- 172.21.2.111:0/559516718 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6708045470 con 0x7f67180a8290 2021-04-03T03:15:32.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:32.793+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f6718057660 con 0x7f67180a8290 2021-04-03T03:15:33.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f67157fa700 1 -- 172.21.2.111:0/559516718 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v288) v1 ==== 126+0+0 (secure 0 0 0) 0x7f6708048a90 con 0x7f67180a8290 2021-04-03T03:15:33.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f66fc056ea0 msgr2=0x7f66fc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f66fc056ea0 0x7f66fc059360 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f6700007000 tx=0x7f670001a000).stop 2021-04-03T03:15:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f67180a8290 msgr2=0x7f671815a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f67180a8290 0x7f671815a1f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f670800b000 tx=0x7f6708001560).stop 2021-04-03T03:15:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 shutdown_connections 2021-04-03T03:15:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f66fc056ea0 0x7f66fc059360 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6718156220 0x7f67181566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f67180a8290 0x7f671815a1f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 --2- 172.21.2.111:0/559516718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f67180ab9e0 0x7f6718149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 >> 172.21.2.111:0/559516718 conn(0x7f67180b61c0 msgr2=0x7f6718147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:33.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 shutdown_connections 2021-04-03T03:15:33.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.389+0000 7f671d705700 1 -- 172.21.2.111:0/559516718 wait complete. 2021-04-03T03:15:33.399 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2021-04-03T03:15:33.400 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:15:33.400 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2021-04-03T03:15:33.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 Processor -- start 2021-04-03T03:15:33.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 -- start start 2021-04-03T03:15:33.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5bdc14b700 0x7f5bdc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:33.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0af400 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0af6a0 con 0x7f5bdc0603f0 2021-04-03T03:15:33.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc060860 con 0x7f5bdc14b700 2021-04-03T03:15:33.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be11da700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5bdc14b700 0x7f5bdc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:33.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be21dc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5bdc0b0c90 0x7f5bdc147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55656/0 (socket says 172.21.2.111:55656) 2021-04-03T03:15:33.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be21dc700 1 -- 172.21.2.111:0/3644537022 learned_addr learned my addr 172.21.2.111:0/3644537022 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:33.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be11da700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5bdc14b700 0x7f5bdc0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54862/0 (socket says 172.21.2.111:54862) 2021-04-03T03:15:33.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be11da700 1 -- 172.21.2.111:0/3644537022 >> v1:172.21.2.115:6789/0 conn(0x7f5bdc0603f0 legacy=0x7f5bdc0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:33.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be11da700 1 -- 172.21.2.111:0/3644537022 >> v1:172.21.2.111:6789/0 conn(0x7f5bdc0b0c90 legacy=0x7f5bdc147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:33.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be11da700 1 -- 172.21.2.111:0/3644537022 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0afb40 con 0x7f5bdc14b700 2021-04-03T03:15:33.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be11da700 1 --2- 172.21.2.111:0/3644537022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc14b700 0x7f5bdc0b07a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f5bdc0620f0 tx=0x7f5bd8006dd0).ready entity=mon.0 client_cookie=b9ebf3b87beea8bb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:33.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be09d9700 1 -- 172.21.2.111:0/3644537022 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bd80097b0 con 0x7f5bdc14b700 2021-04-03T03:15:33.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be09d9700 1 -- 172.21.2.111:0/3644537022 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8009910 con 0x7f5bdc14b700 2021-04-03T03:15:33.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be09d9700 1 -- 172.21.2.111:0/3644537022 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bd8009c70 con 0x7f5bdc14b700 2021-04-03T03:15:33.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 -- 172.21.2.111:0/3644537022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc14b700 msgr2=0x7f5bdc0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:33.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.453+0000 7f5be29dd700 1 --2- 172.21.2.111:0/3644537022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc14b700 0x7f5bdc0b07a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f5bdc0620f0 tx=0x7f5bd8006dd0).stop 2021-04-03T03:15:33.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- 172.21.2.111:0/3644537022 shutdown_connections 2021-04-03T03:15:33.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 --2- 172.21.2.111:0/3644537022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc14b700 0x7f5bdc0b07a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:33.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- 172.21.2.111:0/3644537022 >> 172.21.2.111:0/3644537022 conn(0x7f5bdc0b61c0 msgr2=0x7f5bdc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:33.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- 172.21.2.111:0/3644537022 shutdown_connections 2021-04-03T03:15:33.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- 172.21.2.111:0/3644537022 wait complete. 2021-04-03T03:15:33.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 Processor -- start 2021-04-03T03:15:33.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- start start 2021-04-03T03:15:33.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bdc0603f0 0x7f5bdc0a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:33.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc0b0c90 0x7f5bdc15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:33.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bdc14b700 0x7f5bdc15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:33.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc159f00 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0ab6b0 con 0x7f5bdc14b700 2021-04-03T03:15:33.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0ab950 con 0x7f5bdc0603f0 2021-04-03T03:15:33.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc0b0c90 0x7f5bdc15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:33.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc0b0c90 0x7f5bdc15a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54868/0 (socket says 172.21.2.111:54868) 2021-04-03T03:15:33.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 -- 172.21.2.111:0/2585535265 learned_addr learned my addr 172.21.2.111:0/2585535265 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:33.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be21dc700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bdc14b700 0x7f5bdc15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:33.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be19db700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bdc0603f0 0x7f5bdc0a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:33.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 -- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bdc0603f0 msgr2=0x7f5bdc0a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:33.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bdc0603f0 0x7f5bdc0a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:33.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 -- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bdc14b700 msgr2=0x7f5bdc15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:33.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bdc14b700 0x7f5bdc15a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:33.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 -- 172.21.2.111:0/2585535265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc15ad10 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be11da700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc0b0c90 0x7f5bdc15a330 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f5bdc0620f0 tx=0x7f5bd8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:33.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5bceffd700 1 -- 172.21.2.111:0/2585535265 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bd8011bd0 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5bceffd700 1 -- 172.21.2.111:0/2585535265 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8011d30 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5bceffd700 1 -- 172.21.2.111:0/2585535265 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bd8011ea0 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc0ab330 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc063100 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc062bf0 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5bceffd700 1 -- 172.21.2.111:0/2585535265 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5bd801a040 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5bceffd700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bbc056ea0 0x7f5bbc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:33.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5bceffd700 1 -- 172.21.2.111:0/2585535265 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5bd8071280 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.457+0000 7f5be19db700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bbc056ea0 0x7f5bbc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:33.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.461+0000 7f5be19db700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bbc056ea0 0x7f5bbc059360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f5bd000d000 tx=0x7f5bd0007b50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:33.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.461+0000 7f5bceffd700 1 -- 172.21.2.111:0/2585535265 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5bd8046670 con 0x7f5bdc0b0c90 2021-04-03T03:15:33.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:33.589+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f5bdc057660 con 0x7f5bdc0b0c90 2021-04-03T03:15:34.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5bceffd700 1 -- 172.21.2.111:0/2585535265 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v289) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5bd80230d0 con 0x7f5bdc0b0c90 2021-04-03T03:15:34.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bbc056ea0 msgr2=0x7f5bbc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bbc056ea0 0x7f5bbc059360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f5bd000d000 tx=0x7f5bd0007b50).stop 2021-04-03T03:15:34.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc0b0c90 msgr2=0x7f5bdc15a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc0b0c90 0x7f5bdc15a330 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f5bdc0620f0 tx=0x7f5bd8015000).stop 2021-04-03T03:15:34.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 shutdown_connections 2021-04-03T03:15:34.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5bbc056ea0 0x7f5bbc059360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5bdc14b700 0x7f5bdc15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5bdc0603f0 0x7f5bdc0a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 --2- 172.21.2.111:0/2585535265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5bdc0b0c90 0x7f5bdc15a330 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 >> 172.21.2.111:0/2585535265 conn(0x7f5bdc0b61c0 msgr2=0x7f5bdc14ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:34.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 shutdown_connections 2021-04-03T03:15:34.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.393+0000 7f5be29dd700 1 -- 172.21.2.111:0/2585535265 wait complete. 2021-04-03T03:15:34.406 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:15:34.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 Processor -- start 2021-04-03T03:15:34.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- start start 2021-04-03T03:15:34.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f88b80423a0 0x7f88b803c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b8043400 con 0x7f88b8047580 2021-04-03T03:15:34.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b8047960 con 0x7f88b80d38b0 2021-04-03T03:15:34.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b8042ec0 con 0x7f88b80423a0 2021-04-03T03:15:34.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88beffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f88b8047580 0x7f88b803bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55666/0 (socket says 172.21.2.111:55666) 2021-04-03T03:15:34.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88beffd700 1 -- 172.21.2.111:0/504348887 learned_addr learned my addr 172.21.2.111:0/504348887 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:34.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88be7fc700 1 --2- 172.21.2.111:0/504348887 >> v2:172.21.2.111:3300/0 conn(0x7f88b80423a0 0x7f88b803c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:34.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88be7fc700 1 -- 172.21.2.111:0/504348887 >> v1:172.21.2.115:6789/0 conn(0x7f88b80d38b0 legacy=0x7f88b80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:34.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88be7fc700 1 -- 172.21.2.111:0/504348887 >> v1:172.21.2.111:6789/0 conn(0x7f88b8047580 legacy=0x7f88b803bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88be7fc700 1 -- 172.21.2.111:0/504348887 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8043160 con 0x7f88b80423a0 2021-04-03T03:15:34.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88be7fc700 1 --2- 172.21.2.111:0/504348887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b803c640 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f88b80413e0 tx=0x7f88b0006dd0).ready entity=mon.0 client_cookie=1aa645a549b42f54 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:34.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bdffb700 1 -- 172.21.2.111:0/504348887 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b0009ba0 con 0x7f88b80423a0 2021-04-03T03:15:34.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bdffb700 1 -- 172.21.2.111:0/504348887 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b0009d00 con 0x7f88b80423a0 2021-04-03T03:15:34.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bdffb700 1 -- 172.21.2.111:0/504348887 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b0010600 con 0x7f88b80423a0 2021-04-03T03:15:34.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- 172.21.2.111:0/504348887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 msgr2=0x7f88b803c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 --2- 172.21.2.111:0/504348887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b803c640 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f88b80413e0 tx=0x7f88b0006dd0).stop 2021-04-03T03:15:34.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- 172.21.2.111:0/504348887 shutdown_connections 2021-04-03T03:15:34.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 --2- 172.21.2.111:0/504348887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b803c640 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- 172.21.2.111:0/504348887 >> 172.21.2.111:0/504348887 conn(0x7f88b800d9a0 msgr2=0x7f88b8046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:34.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- 172.21.2.111:0/504348887 shutdown_connections 2021-04-03T03:15:34.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- 172.21.2.111:0/504348887 wait complete. 2021-04-03T03:15:34.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 Processor -- start 2021-04-03T03:15:34.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.461+0000 7f88bffff700 1 -- start start 2021-04-03T03:15:34.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b80406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88b8047580 0x7f88b80461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88b80d38b0 0x7f88b80e6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b803e550 con 0x7f88b80423a0 2021-04-03T03:15:34.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b803e7f0 con 0x7f88b80d38b0 2021-04-03T03:15:34.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b803dc30 con 0x7f88b8047580 2021-04-03T03:15:34.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b80406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:34.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b80406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54880/0 (socket says 172.21.2.111:54880) 2021-04-03T03:15:34.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 -- 172.21.2.111:0/1464298007 learned_addr learned my addr 172.21.2.111:0/1464298007 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:34.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 -- 172.21.2.111:0/1464298007 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88b8047580 msgr2=0x7f88b80461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88b8047580 0x7f88b80461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 -- 172.21.2.111:0/1464298007 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88b80d38b0 msgr2=0x7f88b80e6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88b80d38b0 0x7f88b80e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 -- 172.21.2.111:0/1464298007 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b803e0d0 con 0x7f88b80423a0 2021-04-03T03:15:34.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88beffd700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b80406e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f88b803bac0 tx=0x7f88a8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:34.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88b7fff700 1 -- 172.21.2.111:0/1464298007 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a8007320 con 0x7f88b80423a0 2021-04-03T03:15:34.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88b7fff700 1 -- 172.21.2.111:0/1464298007 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a8007480 con 0x7f88b80423a0 2021-04-03T03:15:34.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b803d600 con 0x7f88b80423a0 2021-04-03T03:15:34.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b80d2ee0 con 0x7f88b80423a0 2021-04-03T03:15:34.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88b7fff700 1 -- 172.21.2.111:0/1464298007 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a80057e0 con 0x7f88b80423a0 2021-04-03T03:15:34.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b803ddf0 con 0x7f88b80423a0 2021-04-03T03:15:34.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88b7fff700 1 -- 172.21.2.111:0/1464298007 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f88a8010030 con 0x7f88b80423a0 2021-04-03T03:15:34.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88b7fff700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88a4057050 0x7f88a4059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88b7fff700 1 -- 172.21.2.111:0/1464298007 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f88a806fe20 con 0x7f88b80423a0 2021-04-03T03:15:34.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88be7fc700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88a4057050 0x7f88a4059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:34.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.465+0000 7f88be7fc700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88a4057050 0x7f88a4059510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f88b80413e0 tx=0x7f88b0017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:34.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.469+0000 7f88b7fff700 1 -- 172.21.2.111:0/1464298007 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f88a8045210 con 0x7f88b80423a0 2021-04-03T03:15:34.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.561+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f88b80d3510 con 0x7f88b80423a0 2021-04-03T03:15:34.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88b7fff700 1 -- 172.21.2.111:0/1464298007 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88a8048830 con 0x7f88b80423a0 2021-04-03T03:15:34.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88a4057050 msgr2=0x7f88a4059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88a4057050 0x7f88a4059510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f88b80413e0 tx=0x7f88b0017000).stop 2021-04-03T03:15:34.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 msgr2=0x7f88b80406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b80406e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f88b803bac0 tx=0x7f88a8007c20).stop 2021-04-03T03:15:34.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 shutdown_connections 2021-04-03T03:15:34.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f88a4057050 0x7f88a4059510 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f88b80d38b0 0x7f88b80e6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f88b80423a0 0x7f88b80406e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 --2- 172.21.2.111:0/1464298007 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f88b8047580 0x7f88b80461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 >> 172.21.2.111:0/1464298007 conn(0x7f88b800d9a0 msgr2=0x7f88b8005ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:34.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 shutdown_connections 2021-04-03T03:15:34.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.565+0000 7f88bffff700 1 -- 172.21.2.111:0/1464298007 wait complete. 2021-04-03T03:15:34.577 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:15:34.577 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:34.584 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:34.591 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:34.603 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:34.615 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:34.628 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:15:34.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:34.629 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:34.629 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:34.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:34.649 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:15:34.649 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:34.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:34.650 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:34.651 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:34.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:34.661 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:15:34.662 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:34.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:34.663 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:34.663 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:34.675 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:34.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:34.676 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:15:34.677 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:15:34.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 Processor -- start 2021-04-03T03:15:34.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- start start 2021-04-03T03:15:34.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7faca40603f0 0x7faca40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca414b700 0x7faca40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7faca40b12b0 0x7faca40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca40b0f60 con 0x7faca40b12b0 2021-04-03T03:15:34.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca40607d0 con 0x7faca40603f0 2021-04-03T03:15:34.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40af400 con 0x7faca414b700 2021-04-03T03:15:34.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7faca40b12b0 0x7faca40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:34.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7faca40b12b0 0x7faca40b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54890/0 (socket says 172.21.2.111:54890) 2021-04-03T03:15:34.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 -- 172.21.2.111:0/697442009 learned_addr learned my addr 172.21.2.111:0/697442009 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:34.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 -- 172.21.2.111:0/697442009 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca414b700 msgr2=0x7faca40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:34.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 --2- 172.21.2.111:0/697442009 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca414b700 0x7faca40b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 -- 172.21.2.111:0/697442009 >> v2:172.21.2.115:3300/0 conn(0x7faca40603f0 msgr2=0x7faca40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:34.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 --2- 172.21.2.111:0/697442009 >> v2:172.21.2.115:3300/0 conn(0x7faca40603f0 0x7faca40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 -- 172.21.2.111:0/697442009 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40af6a0 con 0x7faca40b12b0 2021-04-03T03:15:34.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca94ac700 1 --2- 172.21.2.111:0/697442009 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40b0a70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fac9c0032b0 tx=0x7fac9c009270).ready entity=mon.0 client_cookie=58392ce9c2841abc server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:34.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca37fe700 1 -- 172.21.2.111:0/697442009 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9c00bc00 con 0x7faca40b12b0 2021-04-03T03:15:34.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca37fe700 1 -- 172.21.2.111:0/697442009 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c00bd60 con 0x7faca40b12b0 2021-04-03T03:15:34.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca37fe700 1 -- 172.21.2.111:0/697442009 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9c011400 con 0x7faca40b12b0 2021-04-03T03:15:34.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- 172.21.2.111:0/697442009 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 msgr2=0x7faca40b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 --2- 172.21.2.111:0/697442009 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40b0a70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fac9c0032b0 tx=0x7fac9c009270).stop 2021-04-03T03:15:34.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- 172.21.2.111:0/697442009 shutdown_connections 2021-04-03T03:15:34.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 --2- 172.21.2.111:0/697442009 >> v2:172.21.2.115:3300/0 conn(0x7faca40603f0 0x7faca40b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 --2- 172.21.2.111:0/697442009 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40b0a70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:34.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 --2- 172.21.2.111:0/697442009 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca414b700 0x7faca40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- 172.21.2.111:0/697442009 >> 172.21.2.111:0/697442009 conn(0x7faca40b61c0 msgr2=0x7faca405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:34.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- 172.21.2.111:0/697442009 shutdown_connections 2021-04-03T03:15:34.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- 172.21.2.111:0/697442009 wait complete. 2021-04-03T03:15:34.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 Processor -- start 2021-04-03T03:15:34.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.741+0000 7faca9cad700 1 -- start start 2021-04-03T03:15:34.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca40603f0 0x7faca40a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40a7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faca414b700 0x7faca4146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4062de0 con 0x7faca40b12b0 2021-04-03T03:15:34.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4063080 con 0x7faca414b700 2021-04-03T03:15:34.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40ab950 con 0x7faca40603f0 2021-04-03T03:15:34.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:34.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54896/0 (socket says 172.21.2.111:54896) 2021-04-03T03:15:34.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 -- 172.21.2.111:0/2281595922 learned_addr learned my addr 172.21.2.111:0/2281595922 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:34.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 -- 172.21.2.111:0/2281595922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca40603f0 msgr2=0x7faca40a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca40603f0 0x7faca40a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 -- 172.21.2.111:0/2281595922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faca414b700 msgr2=0x7faca4146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:34.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faca414b700 0x7faca4146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:34.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 -- 172.21.2.111:0/2281595922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40628d0 con 0x7faca40b12b0 2021-04-03T03:15:34.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca3fff700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40a7490 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7fac98002f30 tx=0x7fac98006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:34.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca1ffb700 1 -- 172.21.2.111:0/2281595922 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9800d7c0 con 0x7faca40b12b0 2021-04-03T03:15:34.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca1ffb700 1 -- 172.21.2.111:0/2281595922 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9800d920 con 0x7faca40b12b0 2021-04-03T03:15:34.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca4062b80 con 0x7faca40b12b0 2021-04-03T03:15:34.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca40aa3a0 con 0x7faca40b12b0 2021-04-03T03:15:34.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca1ffb700 1 -- 172.21.2.111:0/2281595922 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9800dc80 con 0x7faca40b12b0 2021-04-03T03:15:34.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4057660 con 0x7faca40b12b0 2021-04-03T03:15:34.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca1ffb700 1 -- 172.21.2.111:0/2281595922 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fac98011460 con 0x7faca40b12b0 2021-04-03T03:15:34.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca1ffb700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fac84056f70 0x7fac84059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:34.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca1ffb700 1 -- 172.21.2.111:0/2281595922 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fac9806fdb0 con 0x7faca40b12b0 2021-04-03T03:15:34.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca8cab700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fac84056f70 0x7fac84059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:34.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.745+0000 7faca8cab700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fac84056f70 0x7fac84059430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fac94002f30 tx=0x7fac94006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:34.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.749+0000 7faca1ffb700 1 -- 172.21.2.111:0/2281595922 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fac98045140 con 0x7faca40b12b0 2021-04-03T03:15:34.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faca40686d0 con 0x7faca40b12b0 2021-04-03T03:15:34.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca1ffb700 1 -- 172.21.2.111:0/2281595922 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v84) v1 ==== 81+0+34040 (secure 0 0 0) 0x7fac98048760 con 0x7faca40b12b0 2021-04-03T03:15:34.845 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:34.845 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:15:34.846 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 84, 2021-04-03T03:15:34.846 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:15:34.846 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:15:34.846 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:15:26.331795+0000", 2021-04-03T03:15:34.846 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:15:34.847 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:15:34.847 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:15:34.847 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:15:34.847 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:15:34.848 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:15:34.848 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:15:34.848 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:15:34.848 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:15:34.848 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:34.849 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:15:34.849 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:15:34.849 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:15:34.849 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:15:34.850 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:15:34.850 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 16, 2021-04-03T03:15:34.850 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:15:34.850 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:15:34.850 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:15:34.851 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:15:34.851 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:15:34.851 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.851 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:15:34.851 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:15:34.852 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:15:34.852 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:34.852 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:34.852 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:34.852 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:34.853 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:34.853 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:34.853 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:34.853 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:34.854 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:15:34.854 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:15:34.854 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:15:34.854 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:15:34.854 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:15:34.855 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:34.855 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:34.855 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:34.855 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:34.855 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:34.856 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:34.856 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:34.856 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.856 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:15:34.857 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:34.857 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:34.857 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:34.857 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:34.857 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:34.858 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:34.858 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:34.858 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:34.858 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:34.858 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:34.859 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:34.859 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:34.859 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:34.859 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:34.859 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:34.860 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:34.860 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:34.860 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:34.860 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:34.860 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:34.861 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:34.861 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:34.861 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:34.861 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:34.862 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:34.862 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:34.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.862 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:34.862 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:34.863 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:34.863 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:34.863 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:34.863 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:34.863 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:34.864 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:34.864 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:34.864 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:34.864 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:34.865 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:15:34.865 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:15:34.865 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.865 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:34.865 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:15:34.866 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.866 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.866 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.866 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:15:34.866 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:15:34.867 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:15:34.867 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:34.867 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:34.867 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:34.867 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:34.868 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:34.868 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:34.868 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:34.868 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:34.868 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:34.869 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:34.869 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:34.869 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:34.869 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:34.870 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:34.870 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:34.870 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:34.870 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:34.870 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:34.871 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:34.871 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:34.871 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.871 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:15:34.871 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:34.872 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:34.872 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:34.872 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:34.872 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:34.872 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:34.873 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:34.873 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:34.873 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:34.873 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:34.873 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:34.874 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:34.874 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:34.874 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:34.874 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:34.875 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:34.875 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:34.875 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:34.875 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:34.875 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:34.876 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:34.876 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:34.876 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:34.876 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:34.876 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:34.877 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:34.877 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.877 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:34.877 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:34.878 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:34.878 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:34.878 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:34.878 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:34.878 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:34.879 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:34.879 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:34.879 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:34.879 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:34.880 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:15:34.880 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:34.880 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:15:34.880 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.880 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.881 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.881 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 15, 2021-04-03T03:15:34.881 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:15:34.881 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:15:23.240089+0000", 2021-04-03T03:15:34.881 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:34.882 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:34.882 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:34.882 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:34.882 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:34.882 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:34.883 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:34.883 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:34.883 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:34.883 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:34.884 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:34.884 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:34.884 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:34.884 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:34.884 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:34.885 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:34.885 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:34.885 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:34.885 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:34.885 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:34.886 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.886 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "83", 2021-04-03T03:15:34.886 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:34.886 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:34.886 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:34.887 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:34.887 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:34.887 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:34.887 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:34.888 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:34.889 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:34.890 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:34.890 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:34.890 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:34.890 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:34.891 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:34.891 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:34.891 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:34.891 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:34.891 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:34.892 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.892 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:34.892 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:34.892 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:34.892 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:34.893 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:34.893 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:34.893 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:34.893 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:34.893 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:34.894 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:34.894 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:34.894 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:15:34.894 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:15:34.894 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:15:34.895 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:15:34.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.895 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:34.895 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:15:34.896 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:15:34.896 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.896 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.896 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.896 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.897 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 16, 2021-04-03T03:15:34.897 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:15:34.897 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:15:24.288628+0000", 2021-04-03T03:15:34.897 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:34.898 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:34.898 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:34.898 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:34.898 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:34.898 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:34.899 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:34.899 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:34.899 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:34.899 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:34.900 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:34.900 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:34.900 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:34.900 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:34.900 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:34.901 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:34.901 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:34.901 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:34.901 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:34.901 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:34.902 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.902 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "84", 2021-04-03T03:15:34.902 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:34.902 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:34.902 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:34.903 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:34.903 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:34.903 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:34.903 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:34.904 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:34.904 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:34.904 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:34.904 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:34.904 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:34.905 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:34.905 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:34.905 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:34.905 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:34.905 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:34.906 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:34.906 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:34.906 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:34.906 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:34.906 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:34.907 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:34.907 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:34.907 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:34.907 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:34.908 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.908 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:34.908 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:34.908 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:34.909 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:34.909 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:34.909 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:34.909 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:34.910 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:34.910 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:34.910 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:34.910 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:34.911 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:15:34.911 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:34.911 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:15:34.911 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:15:34.912 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.912 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.912 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.912 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:34.912 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:15:34.913 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.913 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:15:34.913 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:15:34.913 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:34.914 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:34.914 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:34.914 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:34.914 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:34.915 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:34.915 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:34.915 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 81, 2021-04-03T03:15:34.915 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:34.916 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:34.916 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:34.916 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.916 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.916 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.917 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:15:34.917 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.917 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.917 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.918 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.918 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:15:34.918 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.918 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.919 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.919 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:34.919 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.919 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.920 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.920 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:15:34.920 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.920 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.921 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.921 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.921 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:15:34.921 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.921 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.922 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.922 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.922 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:34.922 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.922 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.923 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.923 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:15:34.923 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.923 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.924 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.924 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.924 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:15:34.924 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.924 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.925 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.925 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.925 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:34.925 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.925 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.926 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.926 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:15:34.926 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.926 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.927 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.927 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.927 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:15:34.927 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:34.927 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.928 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.928 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.928 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:15:34.928 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:15:34.928 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:15:34.929 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:15:34.929 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:34.929 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:34.929 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:34.930 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.930 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.930 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.930 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:15:34.930 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:15:34.931 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:34.931 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:34.931 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:34.931 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:34.931 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:34.932 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:34.932 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:34.932 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 80, 2021-04-03T03:15:34.932 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:34.933 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:34.933 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:34.933 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.933 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.933 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.934 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:15:34.934 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.934 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.934 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.934 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.935 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:15:34.935 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.935 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.935 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.936 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.936 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:34.936 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.936 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.936 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.937 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:15:34.937 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.937 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.937 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.937 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.938 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:15:34.938 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.938 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.938 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.939 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.939 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:34.939 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.939 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.939 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.940 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:15:34.940 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.940 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.940 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.940 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.941 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:15:34.941 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.941 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.941 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.942 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.942 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:34.942 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.942 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.942 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.943 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:15:34.943 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.943 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.943 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.944 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.944 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:15:34.944 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:34.944 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.944 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.945 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.945 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:15:34.945 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:15:34.945 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:15:34.946 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:15:34.946 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:34.946 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:34.946 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:34.947 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.947 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.947 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.947 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:15:34.947 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:15:34.948 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:34.948 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:34.948 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:34.948 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:34.949 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:34.949 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:34.949 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:34.949 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 81, 2021-04-03T03:15:34.950 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:34.950 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:34.950 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:34.950 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.950 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.951 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.951 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:15:34.951 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.951 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.952 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.952 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.952 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:15:34.952 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.952 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.953 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.953 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.953 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:34.953 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.954 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.954 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.954 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:15:34.954 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.954 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.955 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.955 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.955 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:15:34.955 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.956 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.956 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.956 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.956 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:34.956 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.957 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.957 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.957 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:15:34.957 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.958 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.958 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.958 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.958 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:15:34.958 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.959 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.959 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.959 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.959 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:34.960 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.960 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.960 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.960 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:15:34.960 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.961 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.961 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.961 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.961 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:15:34.961 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:34.962 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.962 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.962 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.962 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:15:34.963 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:15:34.963 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:15:34.963 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:15:34.963 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:34.964 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:34.964 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:34.964 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.964 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.964 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.965 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:15:34.965 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:15:34.965 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:34.965 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:34.965 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:34.966 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:34.966 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:34.966 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:34.966 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:34.967 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 82, 2021-04-03T03:15:34.967 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:34.967 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:34.967 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:34.967 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.968 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.968 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.968 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:15:34.968 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.969 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.969 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.969 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.969 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:15:34.969 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.970 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.970 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.970 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.970 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:34.970 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:15:34.971 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:15:34.972 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:34.973 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.974 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.974 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.974 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:15:34.974 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.975 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.975 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.975 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.975 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:15:34.975 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.976 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.976 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.976 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.976 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:34.977 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.977 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.977 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.977 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:15:34.978 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.978 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.978 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.978 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.978 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:15:34.979 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:34.979 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.979 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.979 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.979 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:15:34.980 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:15:34.980 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:15:34.980 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:15:34.980 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:34.981 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:34.981 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:34.981 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.981 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.982 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.982 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:15:34.982 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:15:34.982 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:34.983 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:34.983 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:34.983 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:34.983 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:34.984 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:34.984 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:34.984 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 81, 2021-04-03T03:15:34.984 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:34.984 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:34.985 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:34.985 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.985 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.985 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.986 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:15:34.986 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.986 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.986 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.987 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.987 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:15:34.987 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.987 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.988 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.988 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.988 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:34.988 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.988 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.989 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.989 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:15:34.989 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.989 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.989 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.990 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.990 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:15:34.990 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.990 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.991 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.991 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.991 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:34.991 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.991 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.992 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.992 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:15:34.992 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.992 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.992 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.993 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.993 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:15:34.993 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.993 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.994 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.994 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.994 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:34.994 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:34.994 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.995 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:34.995 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:15:34.995 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.995 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.996 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.996 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:34.996 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:15:34.996 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:34.996 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:34.997 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.997 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.997 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:15:34.997 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:15:34.997 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:15:34.998 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:15:34.998 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:34.998 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:34.998 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:34.999 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:34.999 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:34.999 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:34.999 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:15:34.999 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:15:35.000 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:35.000 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:35.000 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:35.000 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:35.000 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:35.001 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:35.001 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:35.001 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 81, 2021-04-03T03:15:35.001 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:35.001 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:35.002 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:35.002 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.002 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.002 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.003 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:15:35.003 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.003 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.003 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.003 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.004 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:15:35.004 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.004 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.004 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.005 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.005 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:35.005 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.005 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.005 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.006 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:15:35.006 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.006 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.006 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.007 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.007 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:15:35.007 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.007 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.007 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.008 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.008 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:35.008 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.008 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.008 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.009 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:15:35.009 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.009 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.009 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.010 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.010 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:15:35.010 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.010 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.010 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.011 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.011 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:35.011 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.011 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.012 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.012 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:15:35.012 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.012 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.012 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.013 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.013 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:15:35.013 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:35.013 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.013 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.014 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.014 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:15:35.014 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:15:35.014 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:15:35.015 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:15:35.015 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:35.015 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:35.015 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:35.015 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.016 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.016 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.016 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:15:35.016 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:15:35.017 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:35.017 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:35.017 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:35.017 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:35.018 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:35.018 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:35.018 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:35.018 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 81, 2021-04-03T03:15:35.018 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:35.019 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:35.019 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:35.019 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.019 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.019 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.020 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:15:35.020 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.020 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.020 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.021 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.021 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:15:35.021 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.021 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.021 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.022 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.022 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:35.022 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.022 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.023 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.023 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:15:35.023 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.023 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.023 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.024 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.024 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:15:35.024 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.024 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.024 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.025 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.025 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:35.025 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.025 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.026 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.026 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:15:35.026 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.026 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.026 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.027 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.027 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:15:35.027 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.027 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.028 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.028 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.028 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:35.028 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.028 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.029 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.029 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:15:35.029 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.029 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.030 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.030 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.030 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:15:35.030 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:35.030 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.031 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.031 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.031 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:15:35.031 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:15:35.032 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:15:35.032 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:15:35.032 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:35.032 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:35.033 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:35.033 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.033 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.033 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.033 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:15:35.034 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:15:35.034 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:35.034 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:35.034 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:35.034 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:35.035 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:35.035 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:35.035 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:35.035 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 69, 2021-04-03T03:15:35.036 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:35.036 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:35.036 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:35.036 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.036 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.037 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.037 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:15:35.037 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.037 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.038 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.038 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.038 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:15:35.038 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.038 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.039 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.039 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.039 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:35.039 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.039 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.040 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.040 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:15:35.040 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.040 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.041 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.041 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.041 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:15:35.041 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.041 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.042 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.042 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.042 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:35.042 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.042 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.043 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.043 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:15:35.043 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.043 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.044 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.044 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.044 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:15:35.044 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.044 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.045 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.045 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.045 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:35.045 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:35.046 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.046 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:35.046 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:15:35.046 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.046 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.047 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.047 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:35.047 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:15:35.048 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:35.048 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.048 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.048 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.048 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:15:35.049 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:15:35.049 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:15:35.049 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:15:35.050 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:35.050 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:35.050 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:35.050 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:35.050 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.051 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:35.051 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:15:35.051 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.051 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:15:35.051 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.052 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.052 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.052 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.052 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.053 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:15:35.053 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.053 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.053 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.053 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:15:35.054 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.054 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.054 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.054 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.054 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.055 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:15:35.055 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.055 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.055 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.056 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:15:35.056 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.056 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.056 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.057 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.057 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.057 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:15:35.057 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.058 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.058 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.058 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:15:35.058 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.058 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.059 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.059 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.059 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.059 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:15:35.060 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.060 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.060 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.060 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:15:35.060 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.061 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.061 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.061 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.061 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:15:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.062 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:15:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.063 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.064 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:15:35.064 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.064 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.064 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.064 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:15:35.065 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.065 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.065 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.065 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.066 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.066 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:15:35.066 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.066 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.066 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:35.067 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:15:35.067 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:35.067 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:35.067 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:35.067 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:35.068 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:35.068 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:15:35.068 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:35.068 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.069 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:35.069 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:15:35.069 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:15:35.069 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:15:35.069 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:15:35.070 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:15:35.070 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:15:35.070 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:15:35.070 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:15:35.071 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:15:35.071 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:15:35.071 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:15:35.071 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:15:35.071 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:15:35.072 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:15:35.072 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:15:35.072 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:15:35.072 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:15:35.072 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.073 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:15:35.073 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:15:35.073 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:15:35.073 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:15:35.073 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:15:35.074 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:15:35.074 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:15:35.074 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:35.074 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:35.074 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:15:35.075 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:15:35.075 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:15:35.075 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:15:35.075 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:15:35.076 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:15:35.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fac84056f70 msgr2=0x7fac84059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fac84056f70 0x7fac84059430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fac94002f30 tx=0x7fac94006a60).stop 2021-04-03T03:15:35.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 msgr2=0x7faca40a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40a7490 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7fac98002f30 tx=0x7fac98006e10).stop 2021-04-03T03:15:35.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 shutdown_connections 2021-04-03T03:15:35.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fac84056f70 0x7fac84059430 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faca414b700 0x7faca4146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faca40603f0 0x7faca40a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 --2- 172.21.2.111:0/2281595922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faca40b12b0 0x7faca40a7490 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 >> 172.21.2.111:0/2281595922 conn(0x7faca40b61c0 msgr2=0x7faca40b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:35.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 shutdown_connections 2021-04-03T03:15:35.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:34.841+0000 7faca9cad700 1 -- 172.21.2.111:0/2281595922 wait complete. 2021-04-03T03:15:35.080 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:35.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.153+0000 7f3300a17700 1 Processor -- start 2021-04-03T03:15:35.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.153+0000 7f3300a17700 1 -- start start 2021-04-03T03:15:35.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f32f4046c60 0x7f32f40d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f4041f60 0x7f32f40438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f40469c0 con 0x7f32f40465e0 2021-04-03T03:15:35.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f4042bd0 con 0x7f32f4046c60 2021-04-03T03:15:35.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4042370 con 0x7f32f4041f60 2021-04-03T03:15:35.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32faffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f32f4046c60 0x7f32f40d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:35.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fb7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f32f40465e0 0x7f32f40d77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55694/0 (socket says 172.21.2.111:55694) 2021-04-03T03:15:35.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fb7fe700 1 -- 172.21.2.111:0/4025879483 learned_addr learned my addr 172.21.2.111:0/4025879483 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:35.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32faffd700 1 -- 172.21.2.111:0/4025879483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f4041f60 msgr2=0x7f32f40438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:35.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32faffd700 1 --2- 172.21.2.111:0/4025879483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f4041f60 0x7f32f40438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32faffd700 1 -- 172.21.2.111:0/4025879483 >> v1:172.21.2.111:6789/0 conn(0x7f32f40465e0 legacy=0x7f32f40d77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32faffd700 1 -- 172.21.2.111:0/4025879483 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f4045810 con 0x7f32f4046c60 2021-04-03T03:15:35.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32faffd700 1 --2- 172.21.2.111:0/4025879483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4046c60 0x7f32f40d7ee0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f32e8002f30 tx=0x7f32e8006dd0).ready entity=mon.0 client_cookie=26910e27602cb721 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:35.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fa7fc700 1 -- 172.21.2.111:0/4025879483 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32e80097f0 con 0x7f32f4046c60 2021-04-03T03:15:35.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fa7fc700 1 -- 172.21.2.111:0/4025879483 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8009950 con 0x7f32f4046c60 2021-04-03T03:15:35.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fa7fc700 1 -- 172.21.2.111:0/4025879483 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32e8009cb0 con 0x7f32f4046c60 2021-04-03T03:15:35.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- 172.21.2.111:0/4025879483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4046c60 msgr2=0x7f32f40d7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- 172.21.2.111:0/4025879483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4046c60 0x7f32f40d7ee0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f32e8002f30 tx=0x7f32e8006dd0).stop 2021-04-03T03:15:35.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- 172.21.2.111:0/4025879483 shutdown_connections 2021-04-03T03:15:35.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- 172.21.2.111:0/4025879483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f4041f60 0x7f32f40438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- 172.21.2.111:0/4025879483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4046c60 0x7f32f40d7ee0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- 172.21.2.111:0/4025879483 >> 172.21.2.111:0/4025879483 conn(0x7f32f404a6e0 msgr2=0x7f32f404aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:35.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- 172.21.2.111:0/4025879483 shutdown_connections 2021-04-03T03:15:35.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- 172.21.2.111:0/4025879483 wait complete. 2021-04-03T03:15:35.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 Processor -- start 2021-04-03T03:15:35.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- start start 2021-04-03T03:15:35.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4041f60 0x7f32f4049840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f40465e0 0x7f32f400c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f32f4046c60 0x7f32f400b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f400cd70 con 0x7f32f4041f60 2021-04-03T03:15:35.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f400cff0 con 0x7f32f4046c60 2021-04-03T03:15:35.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f3300a17700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f400d290 con 0x7f32f40465e0 2021-04-03T03:15:35.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4041f60 0x7f32f4049840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:35.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4041f60 0x7f32f4049840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54910/0 (socket says 172.21.2.111:54910) 2021-04-03T03:15:35.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.157+0000 7f32fb7fe700 1 -- 172.21.2.111:0/4121096784 learned_addr learned my addr 172.21.2.111:0/4121096784 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:35.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32fb7fe700 1 -- 172.21.2.111:0/4121096784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f40465e0 msgr2=0x7f32f400c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32fb7fe700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f40465e0 0x7f32f400c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32fb7fe700 1 -- 172.21.2.111:0/4121096784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f32f4046c60 msgr2=0x7f32f400b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32fb7fe700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f32f4046c60 0x7f32f400b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32fb7fe700 1 -- 172.21.2.111:0/4121096784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f400bd90 con 0x7f32f4041f60 2021-04-03T03:15:35.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32fb7fe700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4041f60 0x7f32f4049840 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f32f400e990 tx=0x7f32f0007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:35.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32f8ff9700 1 -- 172.21.2.111:0/4121096784 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32f0007320 con 0x7f32f4041f60 2021-04-03T03:15:35.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32f8ff9700 1 -- 172.21.2.111:0/4121096784 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0007480 con 0x7f32f4041f60 2021-04-03T03:15:35.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f40dc330 con 0x7f32f4041f60 2021-04-03T03:15:35.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32f8ff9700 1 -- 172.21.2.111:0/4121096784 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32f00057e0 con 0x7f32f4041f60 2021-04-03T03:15:35.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f4009c40 con 0x7f32f4041f60 2021-04-03T03:15:35.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f400d450 con 0x7f32f4041f60 2021-04-03T03:15:35.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32f8ff9700 1 -- 172.21.2.111:0/4121096784 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f32f0010030 con 0x7f32f4041f60 2021-04-03T03:15:35.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32f8ff9700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f32e4056f20 0x7f32e40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32f8ff9700 1 -- 172.21.2.111:0/4121096784 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f32f006fe10 con 0x7f32f4041f60 2021-04-03T03:15:35.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32faffd700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f32e4056f20 0x7f32e40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:35.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.161+0000 7f32faffd700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f32e4056f20 0x7f32e40593e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f32e8002f30 tx=0x7f32e8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:35.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.165+0000 7f32f8ff9700 1 -- 172.21.2.111:0/4121096784 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f32f0045200 con 0x7f32f4041f60 2021-04-03T03:15:35.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.285+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32f4004da0 con 0x7f32f4041f60 2021-04-03T03:15:35.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.285+0000 7f32f8ff9700 1 -- 172.21.2.111:0/4121096784 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f32f0048820 con 0x7f32f4041f60 2021-04-03T03:15:35.291 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:35.291 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":33,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2065583567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2065583567},{"type":"v1","addr":"172.21.2.115:6833","nonce":2065583567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:25.328682+0000","modified":"2021-04-03T03:15:26.331659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/485806052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":485806052},{"type":"v1","addr":"172.21.2.111:6835","nonce":485806052}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-04-03T03:15:35.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f32e4056f20 msgr2=0x7f32e40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f32e4056f20 0x7f32e40593e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f32e8002f30 tx=0x7f32e8015000).stop 2021-04-03T03:15:35.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4041f60 msgr2=0x7f32f4049840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4041f60 0x7f32f4049840 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f32f400e990 tx=0x7f32f0007be0).stop 2021-04-03T03:15:35.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 shutdown_connections 2021-04-03T03:15:35.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f32e4056f20 0x7f32e40593e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f32f4046c60 0x7f32f400b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f32f4041f60 0x7f32f4049840 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 --2- 172.21.2.111:0/4121096784 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f32f40465e0 0x7f32f400c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 >> 172.21.2.111:0/4121096784 conn(0x7f32f404a6e0 msgr2=0x7f32f4045080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:35.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 shutdown_connections 2021-04-03T03:15:35.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.289+0000 7f3300a17700 1 -- 172.21.2.111:0/4121096784 wait complete. 2021-04-03T03:15:35.295 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 33 2021-04-03T03:15:35.298 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:15:35.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.349+0000 7ff8d6e72700 1 Processor -- start 2021-04-03T03:15:35.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.349+0000 7ff8d6e72700 1 -- start start 2021-04-03T03:15:35.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6e72700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff8d00acb30 0x7ff8d013d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6e72700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d0143310 0x7ff8d01437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6e72700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d00607d0 con 0x7ff8d00603f0 2021-04-03T03:15:35.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6e72700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d0147860 con 0x7ff8d00acb30 2021-04-03T03:15:35.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6e72700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00af840 con 0x7ff8d0143310 2021-04-03T03:15:35.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d566f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff8d00acb30 0x7ff8d013d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:35.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6671700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d0143310 0x7ff8d01437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:35.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d5e70700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7ff8d00603f0 0x7ff8d013cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:40012/0 (socket says 172.21.2.111:40012) 2021-04-03T03:15:35.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d5e70700 1 -- 172.21.2.111:0/3963462166 learned_addr learned my addr 172.21.2.111:0/3963462166 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:35.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d566f700 1 -- 172.21.2.111:0/3963462166 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d0143310 msgr2=0x7ff8d01437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:35.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d566f700 1 --2- 172.21.2.111:0/3963462166 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d0143310 0x7ff8d01437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d566f700 1 -- 172.21.2.111:0/3963462166 >> v1:172.21.2.115:6789/0 conn(0x7ff8d00603f0 legacy=0x7ff8d013cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d566f700 1 -- 172.21.2.111:0/3963462166 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d01473c0 con 0x7ff8d00acb30 2021-04-03T03:15:35.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d566f700 1 --2- 172.21.2.111:0/3963462166 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d00acb30 0x7ff8d013d440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff8c4002f30 tx=0x7ff8c4006dd0).ready entity=mon.1 client_cookie=f6a3bb24941b952d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:35.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d4e6e700 1 -- 172.21.2.111:0/3963462166 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8c4009760 con 0x7ff8d00acb30 2021-04-03T03:15:35.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d4e6e700 1 -- 172.21.2.111:0/3963462166 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c40098c0 con 0x7ff8d00acb30 2021-04-03T03:15:35.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d4e6e700 1 -- 172.21.2.111:0/3963462166 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8c4009c20 con 0x7ff8d00acb30 2021-04-03T03:15:35.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6e72700 1 -- 172.21.2.111:0/3963462166 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d00acb30 msgr2=0x7ff8d013d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.353+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/3963462166 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d00acb30 0x7ff8d013d440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff8c4002f30 tx=0x7ff8c4006dd0).stop 2021-04-03T03:15:35.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- 172.21.2.111:0/3963462166 shutdown_connections 2021-04-03T03:15:35.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/3963462166 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d0143310 0x7ff8d01437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/3963462166 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d00acb30 0x7ff8d013d440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:35.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- 172.21.2.111:0/3963462166 >> 172.21.2.111:0/3963462166 conn(0x7ff8d00b61c0 msgr2=0x7ff8d005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:35.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- 172.21.2.111:0/3963462166 shutdown_connections 2021-04-03T03:15:35.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- 172.21.2.111:0/3963462166 wait complete. 2021-04-03T03:15:35.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 Processor -- start 2021-04-03T03:15:35.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- start start 2021-04-03T03:15:35.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff8d00603f0 0x7ff8d013ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d0143310 0x7ff8d0142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d01556b0 0x7ff8d0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0140ab0 con 0x7ff8d00603f0 2021-04-03T03:15:35.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0142c30 con 0x7ff8d0143310 2021-04-03T03:15:35.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01532f0 con 0x7ff8d01556b0 2021-04-03T03:15:35.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff8d00603f0 0x7ff8d013ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:35.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff8d00603f0 0x7ff8d013ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54922/0 (socket says 172.21.2.111:54922) 2021-04-03T03:15:35.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 -- 172.21.2.111:0/1318929571 learned_addr learned my addr 172.21.2.111:0/1318929571 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:35.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 -- 172.21.2.111:0/1318929571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d01556b0 msgr2=0x7ff8d0155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d01556b0 0x7ff8d0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 -- 172.21.2.111:0/1318929571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d0143310 msgr2=0x7ff8d0142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:35.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d0143310 0x7ff8d0142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:35.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 -- 172.21.2.111:0/1318929571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0156030 con 0x7ff8d00603f0 2021-04-03T03:15:35.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d5e70700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff8d00603f0 0x7ff8d013ef20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7ff8d00b5600 tx=0x7ff8cc007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:35.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8c2ffd700 1 -- 172.21.2.111:0/1318929571 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8cc007320 con 0x7ff8d00603f0 2021-04-03T03:15:35.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8c2ffd700 1 -- 172.21.2.111:0/1318929571 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc007480 con 0x7ff8d00603f0 2021-04-03T03:15:35.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d0142e20 con 0x7ff8d00603f0 2021-04-03T03:15:35.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8c2ffd700 1 -- 172.21.2.111:0/1318929571 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8cc0057e0 con 0x7ff8d00603f0 2021-04-03T03:15:35.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d013e990 con 0x7ff8d00603f0 2021-04-03T03:15:35.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d0057660 con 0x7ff8d00603f0 2021-04-03T03:15:35.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8c2ffd700 1 -- 172.21.2.111:0/1318929571 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff8cc010030 con 0x7ff8d00603f0 2021-04-03T03:15:35.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8c2ffd700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff8b8056f20 0x7ff8b80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:35.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8c2ffd700 1 -- 172.21.2.111:0/1318929571 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff8cc06ff30 con 0x7ff8d00603f0 2021-04-03T03:15:35.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d566f700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff8b8056f20 0x7ff8b80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:35.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.357+0000 7ff8d566f700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff8b8056f20 0x7ff8b80593e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7ff8c4002f30 tx=0x7ff8c4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:35.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.361+0000 7ff8c2ffd700 1 -- 172.21.2.111:0/1318929571 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff8cc045210 con 0x7ff8d00603f0 2021-04-03T03:15:35.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:35.485+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff8d00008d0 con 0x7ff8d00603f0 2021-04-03T03:15:36.395 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:15:36.393+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:15:36.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8c2ffd700 1 -- 172.21.2.111:0/1318929571 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff8cc048830 con 0x7ff8d00603f0 2021-04-03T03:15:36.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff8b8056f20 msgr2=0x7ff8b80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff8b8056f20 0x7ff8b80593e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7ff8c4002f30 tx=0x7ff8c4015000).stop 2021-04-03T03:15:36.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff8d00603f0 msgr2=0x7ff8d013ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff8d00603f0 0x7ff8d013ef20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7ff8d00b5600 tx=0x7ff8cc007c20).stop 2021-04-03T03:15:36.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 shutdown_connections 2021-04-03T03:15:36.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff8d01556b0 0x7ff8d0155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff8d00603f0 0x7ff8d013ef20 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:36.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff8b8056f20 0x7ff8b80593e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:36.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 --2- 172.21.2.111:0/1318929571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff8d0143310 0x7ff8d0142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 >> 172.21.2.111:0/1318929571 conn(0x7ff8d00b61c0 msgr2=0x7ff8d005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:36.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 shutdown_connections 2021-04-03T03:15:36.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.405+0000 7ff8d6e72700 1 -- 172.21.2.111:0/1318929571 wait complete. 2021-04-03T03:15:36.411 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:15:36.415 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:36.419 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:15:36.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 Processor -- start 2021-04-03T03:15:36.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- start start 2021-04-03T03:15:36.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 0x7f40500b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f40500b0c90 0x7f4050147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500607d0 con 0x7f40500603f0 2021-04-03T03:15:36.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40500af400 con 0x7f40500b0c90 2021-04-03T03:15:36.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500af6a0 con 0x7f405014b700 2021-04-03T03:15:36.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4055b1d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f40500603f0 0x7f40500b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55730/0 (socket says 172.21.2.111:55730) 2021-04-03T03:15:36.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4055b1d700 1 -- 172.21.2.111:0/242650120 learned_addr learned my addr 172.21.2.111:0/242650120 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:36.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405631e700 1 --2- 172.21.2.111:0/242650120 >> v2:172.21.2.111:3300/0 conn(0x7f40500b0c90 0x7f4050147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:36.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405631e700 1 -- 172.21.2.111:0/242650120 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 msgr2=0x7f40500b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:36.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405631e700 1 --2- 172.21.2.111:0/242650120 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 0x7f40500b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405631e700 1 -- 172.21.2.111:0/242650120 >> v1:172.21.2.111:6789/0 conn(0x7f40500603f0 legacy=0x7f40500b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405631e700 1 -- 172.21.2.111:0/242650120 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500afb40 con 0x7f40500b0c90 2021-04-03T03:15:36.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405631e700 1 --2- 172.21.2.111:0/242650120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050147680 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f40480032b0 tx=0x7f40480091d0).ready entity=mon.0 client_cookie=5c47ec73facb1ac9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:36.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4054b1b700 1 -- 172.21.2.111:0/242650120 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404800ba50 con 0x7f40500b0c90 2021-04-03T03:15:36.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4054b1b700 1 -- 172.21.2.111:0/242650120 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404800bbb0 con 0x7f40500b0c90 2021-04-03T03:15:36.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4054b1b700 1 -- 172.21.2.111:0/242650120 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404800bd20 con 0x7f40500b0c90 2021-04-03T03:15:36.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- 172.21.2.111:0/242650120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 msgr2=0x7f4050147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- 172.21.2.111:0/242650120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050147680 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f40480032b0 tx=0x7f40480091d0).stop 2021-04-03T03:15:36.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- 172.21.2.111:0/242650120 shutdown_connections 2021-04-03T03:15:36.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- 172.21.2.111:0/242650120 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050147680 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:36.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- 172.21.2.111:0/242650120 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 0x7f40500b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- 172.21.2.111:0/242650120 >> 172.21.2.111:0/242650120 conn(0x7f40500b61c0 msgr2=0x7f405005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:36.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- 172.21.2.111:0/242650120 shutdown_connections 2021-04-03T03:15:36.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- 172.21.2.111:0/242650120 wait complete. 2021-04-03T03:15:36.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 Processor -- start 2021-04-03T03:15:36.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- start start 2021-04-03T03:15:36.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40500603f0 0x7f4050064ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 0x7f4050067650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500abd60 con 0x7f40500b0c90 2021-04-03T03:15:36.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050064500 con 0x7f40500603f0 2021-04-03T03:15:36.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f4056b1f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500647a0 con 0x7f405014b700 2021-04-03T03:15:36.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:36.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050062aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54944/0 (socket says 172.21.2.111:54944) 2021-04-03T03:15:36.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 -- 172.21.2.111:0/3943740999 learned_addr learned my addr 172.21.2.111:0/3943740999 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:36.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 -- 172.21.2.111:0/3943740999 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 msgr2=0x7f4050067650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:36.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 0x7f4050067650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 -- 172.21.2.111:0/3943740999 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40500603f0 msgr2=0x7f4050064ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:36.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40500603f0 0x7f4050064ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.469+0000 7f405531c700 1 -- 172.21.2.111:0/3943740999 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050157870 con 0x7f40500b0c90 2021-04-03T03:15:36.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f405531c700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050062aa0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f404c002f30 tx=0x7f404c006e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:36.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f403effd700 1 -- 172.21.2.111:0/3943740999 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404c00d570 con 0x7f40500b0c90 2021-04-03T03:15:36.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f403effd700 1 -- 172.21.2.111:0/3943740999 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c000e90 con 0x7f40500b0c90 2021-04-03T03:15:36.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500aa140 con 0x7f40500b0c90 2021-04-03T03:15:36.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40500aa6d0 con 0x7f40500b0c90 2021-04-03T03:15:36.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f403effd700 1 -- 172.21.2.111:0/3943740999 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404c00d8c0 con 0x7f40500b0c90 2021-04-03T03:15:36.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40500aa3b0 con 0x7f40500b0c90 2021-04-03T03:15:36.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f403effd700 1 -- 172.21.2.111:0/3943740999 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f404c00f070 con 0x7f40500b0c90 2021-04-03T03:15:36.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f403effd700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4030056ea0 0x7f4030059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f403effd700 1 -- 172.21.2.111:0/3943740999 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f404c06fdc0 con 0x7f40500b0c90 2021-04-03T03:15:36.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f4055b1d700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4030056ea0 0x7f4030059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:36.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f4055b1d700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4030056ea0 0x7f4030059360 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f40500af200 tx=0x7f4040007750).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:36.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.473+0000 7f403effd700 1 -- 172.21.2.111:0/3943740999 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f404c045040 con 0x7f40500b0c90 2021-04-03T03:15:36.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4050057660 con 0x7f40500b0c90 2021-04-03T03:15:36.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f403effd700 1 -- 172.21.2.111:0/3943740999 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f404c048660 con 0x7f40500b0c90 2021-04-03T03:15:36.602 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:36.602 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":34,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/2065583567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":2065583567},{"type":"v1","addr":"172.21.2.115:6833","nonce":2065583567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:25.328682+0000","modified":"2021-04-03T03:15:36.395046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-04-03T03:15:36.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4030056ea0 msgr2=0x7f4030059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4030056ea0 0x7f4030059360 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f40500af200 tx=0x7f4040007750).stop 2021-04-03T03:15:36.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 msgr2=0x7f4050062aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050062aa0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f404c002f30 tx=0x7f404c006e00).stop 2021-04-03T03:15:36.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 shutdown_connections 2021-04-03T03:15:36.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f405014b700 0x7f4050067650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4030056ea0 0x7f4030059360 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:36.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f40500603f0 0x7f4050064ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 --2- 172.21.2.111:0/3943740999 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b0c90 0x7f4050062aa0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:36.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 >> 172.21.2.111:0/3943740999 conn(0x7f40500b61c0 msgr2=0x7f4050147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:36.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 shutdown_connections 2021-04-03T03:15:36.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.597+0000 7f4056b1f700 1 -- 172.21.2.111:0/3943740999 wait complete. 2021-04-03T03:15:36.606 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 34 2021-04-03T03:15:36.609 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:15:36.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a37007700 1 Processor -- start 2021-04-03T03:15:36.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a37007700 1 -- start start 2021-04-03T03:15:36.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a37007700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9a280d90b0 0x7f9a280e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a37007700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2803d840 con 0x7f9a2803f120 2021-04-03T03:15:36.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a37007700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2803f800 con 0x7f9a28005640 2021-04-03T03:15:36.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a37007700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2803faa0 con 0x7f9a280d90b0 2021-04-03T03:15:36.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a36806700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9a280d90b0 0x7f9a280e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:36.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a36806700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9a280d90b0 0x7f9a280e3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54954/0 (socket says 172.21.2.111:54954) 2021-04-03T03:15:36.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a36806700 1 -- 172.21.2.111:0/3939965067 learned_addr learned my addr 172.21.2.111:0/3939965067 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:36.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a36806700 1 -- 172.21.2.111:0/3939965067 >> v1:172.21.2.115:6789/0 conn(0x7f9a28005640 legacy=0x7f9a280d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:36.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a36806700 1 -- 172.21.2.111:0/3939965067 >> v1:172.21.2.111:6789/0 conn(0x7f9a2803f120 legacy=0x7f9a28041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a36806700 1 -- 172.21.2.111:0/3939965067 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28005ad0 con 0x7f9a280d90b0 2021-04-03T03:15:36.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a35003700 1 -- 172.21.2.111:0/3939965067 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244492517 0 0) 0x7f9a2803d840 con 0x7f9a2803f120 2021-04-03T03:15:36.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.721+0000 7f9a36806700 1 --2- 172.21.2.111:0/3939965067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280d90b0 0x7f9a280e3bd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f9a28043d60 tx=0x7f9a3005ad70).ready entity=mon.0 client_cookie=b8000d6ab6e46bdc server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:36.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a35003700 1 -- 172.21.2.111:0/3939965067 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a3005bda0 con 0x7f9a280d90b0 2021-04-03T03:15:36.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a35003700 1 -- 172.21.2.111:0/3939965067 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3005df60 con 0x7f9a280d90b0 2021-04-03T03:15:36.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a35003700 1 -- 172.21.2.111:0/3939965067 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a300699f0 con 0x7f9a280d90b0 2021-04-03T03:15:36.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/3939965067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280d90b0 msgr2=0x7f9a280e3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 --2- 172.21.2.111:0/3939965067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280d90b0 0x7f9a280e3bd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f9a28043d60 tx=0x7f9a3005ad70).stop 2021-04-03T03:15:36.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/3939965067 shutdown_connections 2021-04-03T03:15:36.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 --2- 172.21.2.111:0/3939965067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280d90b0 0x7f9a280e3bd0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:36.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/3939965067 >> 172.21.2.111:0/3939965067 conn(0x7f9a280462b0 msgr2=0x7f9a280466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:36.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/3939965067 shutdown_connections 2021-04-03T03:15:36.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/3939965067 wait complete. 2021-04-03T03:15:36.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 Processor -- start 2021-04-03T03:15:36.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- start start 2021-04-03T03:15:36.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a28005640 0x7f9a2803bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a2803f120 0x7f9a280e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280d90b0 0x7f9a280e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280a32b0 con 0x7f9a28005640 2021-04-03T03:15:36.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2809fa20 con 0x7f9a280d90b0 2021-04-03T03:15:36.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2809fcc0 con 0x7f9a2803f120 2021-04-03T03:15:36.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a28005640 0x7f9a2803bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:36.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a28005640 0x7f9a2803bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54958/0 (socket says 172.21.2.111:54958) 2021-04-03T03:15:36.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 -- 172.21.2.111:0/2112918419 learned_addr learned my addr 172.21.2.111:0/2112918419 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:36.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 -- 172.21.2.111:0/2112918419 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a2803f120 msgr2=0x7f9a280e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a2803f120 0x7f9a280e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 -- 172.21.2.111:0/2112918419 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280d90b0 msgr2=0x7f9a280e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:36.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280d90b0 0x7f9a280e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:36.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 -- 172.21.2.111:0/2112918419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2809ff60 con 0x7f9a28005640 2021-04-03T03:15:36.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a36005700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a28005640 0x7f9a2803bb60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f9a2c000b80 tx=0x7f9a2c007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:36.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a237fe700 1 -- 172.21.2.111:0/2112918419 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a2c00e4a0 con 0x7f9a28005640 2021-04-03T03:15:36.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a237fe700 1 -- 172.21.2.111:0/2112918419 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c00e600 con 0x7f9a28005640 2021-04-03T03:15:36.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a280e7620 con 0x7f9a28005640 2021-04-03T03:15:36.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a237fe700 1 -- 172.21.2.111:0/2112918419 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a2c00e960 con 0x7f9a28005640 2021-04-03T03:15:36.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2803c980 con 0x7f9a28005640 2021-04-03T03:15:36.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a280a00f0 con 0x7f9a28005640 2021-04-03T03:15:36.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a237fe700 1 -- 172.21.2.111:0/2112918419 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9a2c019030 con 0x7f9a28005640 2021-04-03T03:15:36.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a237fe700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a10056f20 0x7f9a100593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:36.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a237fe700 1 -- 172.21.2.111:0/2112918419 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f9a2c06f630 con 0x7f9a28005640 2021-04-03T03:15:36.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a35804700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a10056f20 0x7f9a100593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:36.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.725+0000 7f9a35804700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a10056f20 0x7f9a100593e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f9a24002f50 tx=0x7f9a24006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:36.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.729+0000 7f9a237fe700 1 -- 172.21.2.111:0/2112918419 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9a2c044a10 con 0x7f9a28005640 2021-04-03T03:15:36.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:36.853+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9a28004d30 con 0x7f9a28005640 2021-04-03T03:15:37.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.798+0000 7f9a237fe700 1 -- 172.21.2.111:0/2112918419 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9a2c017020 con 0x7f9a28005640 2021-04-03T03:15:37.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a10056f20 msgr2=0x7f9a100593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:37.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a10056f20 0x7f9a100593e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f9a24002f50 tx=0x7f9a24006990).stop 2021-04-03T03:15:37.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a28005640 msgr2=0x7f9a2803bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:37.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a28005640 0x7f9a2803bb60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f9a2c000b80 tx=0x7f9a2c007a50).stop 2021-04-03T03:15:37.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 shutdown_connections 2021-04-03T03:15:37.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a10056f20 0x7f9a100593e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280d90b0 0x7f9a280e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:37.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a28005640 0x7f9a2803bb60 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 --2- 172.21.2.111:0/2112918419 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a2803f120 0x7f9a280e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:37.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 >> 172.21.2.111:0/2112918419 conn(0x7f9a280462b0 msgr2=0x7f9a280433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:37.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 shutdown_connections 2021-04-03T03:15:37.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.802+0000 7f9a37007700 1 -- 172.21.2.111:0/2112918419 wait complete. 2021-04-03T03:15:37.810 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:15:37.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c766fa700 1 Processor -- start 2021-04-03T03:15:37.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c766fa700 1 -- start start 2021-04-03T03:15:37.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c766fa700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8c680dbba0 0x7f8c680dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c766fa700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c680dc560 con 0x7f8c68041060 2021-04-03T03:15:37.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c766fa700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c680dc800 con 0x7f8c6803d560 2021-04-03T03:15:37.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c766fa700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6803de60 con 0x7f8c680dbba0 2021-04-03T03:15:37.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c74ef7700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8c68041060 0x7f8c6803fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55762/0 (socket says 172.21.2.111:55762) 2021-04-03T03:15:37.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c74ef7700 1 -- 172.21.2.111:0/2085004899 learned_addr learned my addr 172.21.2.111:0/2085004899 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:37.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293696776 0 0) 0x7f8c680dc560 con 0x7f8c68041060 2021-04-03T03:15:37.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c58003240 con 0x7f8c68041060 2021-04-03T03:15:37.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.866+0000 7f8c756f8700 1 --2- 172.21.2.111:0/2085004899 >> v2:172.21.2.115:3300/0 conn(0x7f8c680dbba0 0x7f8c680dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1299263742 0 0) 0x7f8c58003240 con 0x7f8c68041060 2021-04-03T03:15:37.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 >> v2:172.21.2.115:3300/0 conn(0x7f8c680dbba0 msgr2=0x7f8c680dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:37.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 --2- 172.21.2.111:0/2085004899 >> v2:172.21.2.115:3300/0 conn(0x7f8c680dbba0 0x7f8c680dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 >> v1:172.21.2.115:6789/0 conn(0x7f8c6803d560 legacy=0x7f8c68041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:37.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6803da30 con 0x7f8c68041060 2021-04-03T03:15:37.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8c640016d0 con 0x7f8c68041060 2021-04-03T03:15:37.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 >> v1:172.21.2.111:6789/0 conn(0x7f8c68041060 legacy=0x7f8c6803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:37.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c58003780 0x7f8c58003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c580044e0 0x7f8c58004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c5800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6803de60 con 0x7f8c58004e50 2021-04-03T03:15:37.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c58005300 con 0x7f8c58003780 2021-04-03T03:15:37.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5800bc20 con 0x7f8c580044e0 2021-04-03T03:15:37.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c5800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c58003780 0x7f8c58003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c74ef7700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c580044e0 0x7f8c58004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 -- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c580044e0 msgr2=0x7f8c58004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:37.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c580044e0 0x7f8c58004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 -- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c58003780 msgr2=0x7f8c58003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:37.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c58003780 0x7f8c58003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 -- 172.21.2.111:0/2085004899 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c58003240 con 0x7f8c58004e50 2021-04-03T03:15:37.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 -- 172.21.2.111:0/2085004899 reap_dead start 2021-04-03T03:15:37.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c5800b730 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f8c68040a80 tx=0x7f8c7005ad70).ready entity=mon.0 client_cookie=2124b607ddc319f6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:37.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c7005df60 con 0x7f8c58004e50 2021-04-03T03:15:37.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c700699f0 con 0x7f8c58004e50 2021-04-03T03:15:37.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c63fff700 1 -- 172.21.2.111:0/2085004899 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c7005b040 con 0x7f8c58004e50 2021-04-03T03:15:37.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2085004899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 msgr2=0x7f8c5800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:37.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c5800b730 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f8c68040a80 tx=0x7f8c7005ad70).stop 2021-04-03T03:15:37.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2085004899 shutdown_connections 2021-04-03T03:15:37.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2085004899 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c5800b730 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2085004899 >> 172.21.2.111:0/2085004899 conn(0x7f8c680462b0 msgr2=0x7f8c680466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:37.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2085004899 shutdown_connections 2021-04-03T03:15:37.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2085004899 wait complete. 2021-04-03T03:15:37.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 Processor -- start 2021-04-03T03:15:37.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- start start 2021-04-03T03:15:37.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c6803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c680d6730 0x7f8c680a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c680a1a70 0x7f8c680e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680d6bb0 con 0x7f8c58004e50 2021-04-03T03:15:37.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6803bd40 con 0x7f8c680d6730 2021-04-03T03:15:37.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6803bfe0 con 0x7f8c680a1a70 2021-04-03T03:15:37.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c6803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c6803acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54980/0 (socket says 172.21.2.111:54980) 2021-04-03T03:15:37.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 -- 172.21.2.111:0/2795375709 learned_addr learned my addr 172.21.2.111:0/2795375709 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:37.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c74ef7700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c680d6730 0x7f8c680a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c680a1a70 0x7f8c680e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 -- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c680a1a70 msgr2=0x7f8c680e6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:37.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c680a1a70 0x7f8c680e6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 -- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c680d6730 msgr2=0x7f8c680a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:37.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c680d6730 0x7f8c680a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:37.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 -- 172.21.2.111:0/2795375709 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6803c280 con 0x7f8c58004e50 2021-04-03T03:15:37.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c75ef9700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c680a1a70 0x7f8c680e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:37.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c74ef7700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c680d6730 0x7f8c680a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:37.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c756f8700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c6803acd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f8c6c008000 tx=0x7f8c6c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:37.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c627fc700 1 -- 172.21.2.111:0/2795375709 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6c00e970 con 0x7f8c58004e50 2021-04-03T03:15:37.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c627fc700 1 -- 172.21.2.111:0/2795375709 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c00ead0 con 0x7f8c58004e50 2021-04-03T03:15:37.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6809ff20 con 0x7f8c58004e50 2021-04-03T03:15:37.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c627fc700 1 -- 172.21.2.111:0/2795375709 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6c00ee30 con 0x7f8c58004e50 2021-04-03T03:15:37.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c680ea170 con 0x7f8c58004e50 2021-04-03T03:15:37.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.870+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c68004db0 con 0x7f8c58004e50 2021-04-03T03:15:37.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.874+0000 7f8c627fc700 1 -- 172.21.2.111:0/2795375709 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8c6c024070 con 0x7f8c58004e50 2021-04-03T03:15:37.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.874+0000 7f8c627fc700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c50056ec0 0x7f8c50059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:37.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.874+0000 7f8c627fc700 1 -- 172.21.2.111:0/2795375709 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f8c6c070a70 con 0x7f8c58004e50 2021-04-03T03:15:37.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.874+0000 7f8c74ef7700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c50056ec0 0x7f8c50059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:37.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.874+0000 7f8c74ef7700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c50056ec0 0x7f8c50059380 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f8c6400d000 tx=0x7f8c64009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:37.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.874+0000 7f8c627fc700 1 -- 172.21.2.111:0/2795375709 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8c6c045e00 con 0x7f8c58004e50 2021-04-03T03:15:37.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:37.966+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c680083c0 con 0x7f8c58004e50 2021-04-03T03:15:38.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c627fc700 1 -- 172.21.2.111:0/2795375709 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8c6c070e60 con 0x7f8c58004e50 2021-04-03T03:15:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c50056ec0 msgr2=0x7f8c50059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c50056ec0 0x7f8c50059380 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f8c6400d000 tx=0x7f8c64009040).stop 2021-04-03T03:15:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 msgr2=0x7f8c6803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c6803acd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f8c6c008000 tx=0x7f8c6c008040).stop 2021-04-03T03:15:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 shutdown_connections 2021-04-03T03:15:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c680a1a70 0x7f8c680e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:38.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c58004e50 0x7f8c6803acd0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:38.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c50056ec0 0x7f8c50059380 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:38.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 --2- 172.21.2.111:0/2795375709 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c680d6730 0x7f8c680a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:38.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 >> 172.21.2.111:0/2795375709 conn(0x7f8c680462b0 msgr2=0x7f8c6803f020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:38.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 shutdown_connections 2021-04-03T03:15:38.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.814+0000 7f8c766fa700 1 -- 172.21.2.111:0/2795375709 wait complete. 2021-04-03T03:15:38.822 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:15:38.825 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:15:38.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 Processor -- start 2021-04-03T03:15:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- start start 2021-04-03T03:15:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96140dbba0 0x7f96140dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140dc560 con 0x7f9614041060 2021-04-03T03:15:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961403de60 con 0x7f961403d560 2021-04-03T03:15:38.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140dc800 con 0x7f96140dbba0 2021-04-03T03:15:38.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:38.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54990/0 (socket says 172.21.2.111:54990) 2021-04-03T03:15:38.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 -- 172.21.2.111:0/679916284 learned_addr learned my addr 172.21.2.111:0/679916284 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:38.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 -- 172.21.2.111:0/679916284 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96140dbba0 msgr2=0x7f96140dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:38.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 --2- 172.21.2.111:0/679916284 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96140dbba0 0x7f96140dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:38.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 -- 172.21.2.111:0/679916284 >> v1:172.21.2.115:6789/0 conn(0x7f9614041060 legacy=0x7f961403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:38.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 -- 172.21.2.111:0/679916284 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961403da30 con 0x7f961403d560 2021-04-03T03:15:38.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9621b05700 1 --2- 172.21.2.111:0/679916284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f961403d560 0x7f9614041650 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f961403f960 tx=0x7f961c05ad70).ready entity=mon.0 client_cookie=fbeafc87912a6bdb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:38.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f960bfff700 1 -- 172.21.2.111:0/679916284 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c05bda0 con 0x7f961403d560 2021-04-03T03:15:38.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f960bfff700 1 -- 172.21.2.111:0/679916284 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c05df60 con 0x7f961403d560 2021-04-03T03:15:38.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f960bfff700 1 -- 172.21.2.111:0/679916284 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c0699f0 con 0x7f961403d560 2021-04-03T03:15:38.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- 172.21.2.111:0/679916284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f961403d560 msgr2=0x7f9614041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:38.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- 172.21.2.111:0/679916284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f961403d560 0x7f9614041650 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f961403f960 tx=0x7f961c05ad70).stop 2021-04-03T03:15:38.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- 172.21.2.111:0/679916284 shutdown_connections 2021-04-03T03:15:38.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- 172.21.2.111:0/679916284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f961403d560 0x7f9614041650 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:38.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- 172.21.2.111:0/679916284 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96140dbba0 0x7f96140dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- 172.21.2.111:0/679916284 >> 172.21.2.111:0/679916284 conn(0x7f96140462b0 msgr2=0x7f96140466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- 172.21.2.111:0/679916284 shutdown_connections 2021-04-03T03:15:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- 172.21.2.111:0/679916284 wait complete. 2021-04-03T03:15:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 Processor -- start 2021-04-03T03:15:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- start start 2021-04-03T03:15:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 0x7f96140df040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9614041060 0x7f96140e7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96140dbba0 0x7f961409f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140a0a50 con 0x7f9614041060 2021-04-03T03:15:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961403b8e0 con 0x7f96140dbba0 2021-04-03T03:15:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9622306700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140eb270 con 0x7f961403d560 2021-04-03T03:15:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9620b03700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9614041060 0x7f96140e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9620b03700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9614041060 0x7f96140e7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:54994/0 (socket says 172.21.2.111:54994) 2021-04-03T03:15:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.882+0000 7f9620b03700 1 -- 172.21.2.111:0/2848119678 learned_addr learned my addr 172.21.2.111:0/2848119678 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9620b03700 1 -- 172.21.2.111:0/2848119678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 msgr2=0x7f96140df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9620b03700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 0x7f96140df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9620b03700 1 -- 172.21.2.111:0/2848119678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96140dbba0 msgr2=0x7f961409f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9620b03700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96140dbba0 0x7f961409f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9620b03700 1 -- 172.21.2.111:0/2848119678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140eb510 con 0x7f9614041060 2021-04-03T03:15:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9620b03700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9614041060 0x7f96140e7200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f9618001f60 tx=0x7f9618007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f960a7fc700 1 -- 172.21.2.111:0/2848119678 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961800e4a0 con 0x7f9614041060 2021-04-03T03:15:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f960a7fc700 1 -- 172.21.2.111:0/2848119678 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961800e600 con 0x7f9614041060 2021-04-03T03:15:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f960a7fc700 1 -- 172.21.2.111:0/2848119678 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961800e960 con 0x7f9614041060 2021-04-03T03:15:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961403b5d0 con 0x7f9614041060 2021-04-03T03:15:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961403c9a0 con 0x7f9614041060 2021-04-03T03:15:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614004db0 con 0x7f9614041060 2021-04-03T03:15:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f960a7fc700 1 -- 172.21.2.111:0/2848119678 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9618019030 con 0x7f9614041060 2021-04-03T03:15:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f960a7fc700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056f20 0x7f95fc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:38.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f960a7fc700 1 -- 172.21.2.111:0/2848119678 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6064+0+0 (secure 0 0 0) 0x7f961806f570 con 0x7f9614041060 2021-04-03T03:15:38.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9621304700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056f20 0x7f95fc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:38.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f9621304700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056f20 0x7f95fc0593e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f960c002f30 tx=0x7f960c0069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:38.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.886+0000 7f960a7fc700 1 -- 172.21.2.111:0/2848119678 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9618044ac0 con 0x7f9614041060 2021-04-03T03:15:38.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:38.982+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96140083c0 con 0x7f9614041060 2021-04-03T03:15:39.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.818+0000 7f960a7fc700 1 -- 172.21.2.111:0/2848119678 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 173+0+0 (secure 0 0 0) 0x7f96180480e0 con 0x7f9614041060 2021-04-03T03:15:39.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056f20 msgr2=0x7f95fc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:39.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056f20 0x7f95fc0593e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f960c002f30 tx=0x7f960c0069f0).stop 2021-04-03T03:15:39.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9614041060 msgr2=0x7f96140e7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:39.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9614041060 0x7f96140e7200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f9618001f60 tx=0x7f9618007c00).stop 2021-04-03T03:15:39.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 shutdown_connections 2021-04-03T03:15:39.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f95fc056f20 0x7f95fc0593e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:39.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96140dbba0 0x7f961409f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:39.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f961403d560 0x7f96140df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:39.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 --2- 172.21.2.111:0/2848119678 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9614041060 0x7f96140e7200 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:39.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 >> 172.21.2.111:0/2848119678 conn(0x7f96140462b0 msgr2=0x7f9614041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:39.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 shutdown_connections 2021-04-03T03:15:39.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.822+0000 7f9622306700 1 -- 172.21.2.111:0/2848119678 wait complete. 2021-04-03T03:15:39.825 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:15:39.829 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:15:39.830 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:15:39.830 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:15:39.830 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:15:39.831 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:15:39.831 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:15:39.831 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:15:39.831 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:15:39.836 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:15:39.836 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:15:39.837 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:15:39.837 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:15:39.880 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:15:39.881 DEBUG:teuthology.parallel:result is None 2021-04-03T03:15:39.882 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:15:39.883 DEBUG:teuthology.parallel:result is None 2021-04-03T03:15:39.884 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:15:39.917 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:15:39.921 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:15:39.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f690a46a700 1 Processor -- start 2021-04-03T03:15:39.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f690a46a700 1 -- start start 2021-04-03T03:15:39.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f690a46a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f69040af8c0 0x7f69040ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:39.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f690a46a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040b19c0 con 0x7f69040b6210 2021-04-03T03:15:39.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f690a46a700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040ad2c0 con 0x7f6904139690 2021-04-03T03:15:39.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f690a46a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6904139ab0 con 0x7f69040af8c0 2021-04-03T03:15:39.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f6908c67700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f69040af8c0 0x7f69040ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:39.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f6909468700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f69040b6210 0x7f69040adb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55806/0 (socket says 172.21.2.111:55806) 2021-04-03T03:15:39.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f6909468700 1 -- 172.21.2.111:0/3986186404 learned_addr learned my addr 172.21.2.111:0/3986186404 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:39.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f6908c67700 1 -- 172.21.2.111:0/3986186404 >> v1:172.21.2.115:6789/0 conn(0x7f6904139690 legacy=0x7f6904149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:39.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f6908c67700 1 -- 172.21.2.111:0/3986186404 >> v1:172.21.2.111:6789/0 conn(0x7f69040b6210 legacy=0x7f69040adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:39.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f6908c67700 1 -- 172.21.2.111:0/3986186404 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040ad560 con 0x7f69040af8c0 2021-04-03T03:15:39.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.942+0000 7f6908c67700 1 --2- 172.21.2.111:0/3986186404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040af8c0 0x7f69040ae240 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f68f4002f30 tx=0x7f68f4006dd0).ready entity=mon.0 client_cookie=15932232870811e1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:39.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f3fff700 1 -- 172.21.2.111:0/3986186404 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f40097f0 con 0x7f69040af8c0 2021-04-03T03:15:39.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f3fff700 1 -- 172.21.2.111:0/3986186404 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f4009950 con 0x7f69040af8c0 2021-04-03T03:15:39.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f3fff700 1 -- 172.21.2.111:0/3986186404 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f4009cb0 con 0x7f69040af8c0 2021-04-03T03:15:39.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/3986186404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040af8c0 msgr2=0x7f69040ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:39.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 --2- 172.21.2.111:0/3986186404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040af8c0 0x7f69040ae240 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f68f4002f30 tx=0x7f68f4006dd0).stop 2021-04-03T03:15:39.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/3986186404 shutdown_connections 2021-04-03T03:15:39.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 --2- 172.21.2.111:0/3986186404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040af8c0 0x7f69040ae240 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:39.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/3986186404 >> 172.21.2.111:0/3986186404 conn(0x7f69040b56f0 msgr2=0x7f69040b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:39.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/3986186404 shutdown_connections 2021-04-03T03:15:39.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/3986186404 wait complete. 2021-04-03T03:15:39.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 Processor -- start 2021-04-03T03:15:39.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- start start 2021-04-03T03:15:39.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69040af8c0 0x7f69041422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:39.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040b6210 0x7f6904063d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:39.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6904139690 0x7f6904145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:39.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040641f0 con 0x7f69040b6210 2021-04-03T03:15:39.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6904148290 con 0x7f6904139690 2021-04-03T03:15:39.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6904063b10 con 0x7f69040af8c0 2021-04-03T03:15:39.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909468700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69040af8c0 0x7f69041422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:39.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909c69700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6904139690 0x7f6904145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:39.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6908c67700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040b6210 0x7f6904063d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:39.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909c69700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6904139690 0x7f6904145c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:44728/0 (socket says 172.21.2.111:44728) 2021-04-03T03:15:39.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909c69700 1 -- 172.21.2.111:0/746044582 learned_addr learned my addr 172.21.2.111:0/746044582 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:39.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6908c67700 1 -- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69040af8c0 msgr2=0x7f69041422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:39.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6908c67700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69040af8c0 0x7f69041422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:39.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6908c67700 1 -- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6904139690 msgr2=0x7f6904145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:39.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6908c67700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6904139690 0x7f6904145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:39.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6908c67700 1 -- 172.21.2.111:0/746044582 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904063870 con 0x7f69040b6210 2021-04-03T03:15:39.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909468700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69040af8c0 0x7f69041422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:15:39.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909c69700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6904139690 0x7f6904145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:15:39.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6908c67700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040b6210 0x7f6904063d00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f69041483d0 tx=0x7f68f4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:39.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f27fc700 1 -- 172.21.2.111:0/746044582 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f401e460 con 0x7f69040b6210 2021-04-03T03:15:39.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f27fc700 1 -- 172.21.2.111:0/746044582 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f401e5c0 con 0x7f69040b6210 2021-04-03T03:15:39.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6904147df0 con 0x7f69040b6210 2021-04-03T03:15:39.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f27fc700 1 -- 172.21.2.111:0/746044582 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f401e920 con 0x7f69040b6210 2021-04-03T03:15:39.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69040677d0 con 0x7f69040b6210 2021-04-03T03:15:39.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6904147f80 con 0x7f69040b6210 2021-04-03T03:15:39.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f27fc700 1 -- 172.21.2.111:0/746044582 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f68f401a040 con 0x7f69040b6210 2021-04-03T03:15:39.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f27fc700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f68e4056ea0 0x7f68e4059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:39.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f68f27fc700 1 -- 172.21.2.111:0/746044582 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f68f4071500 con 0x7f69040b6210 2021-04-03T03:15:39.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909468700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f68e4056ea0 0x7f68e4059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:39.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.946+0000 7f6909468700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f68e4056ea0 0x7f68e4059360 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f690000d000 tx=0x7f6900007b60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:39.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:39.950+0000 7f68f27fc700 1 -- 172.21.2.111:0/746044582 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f68f4046c70 con 0x7f69040b6210 2021-04-03T03:15:40.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.054+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6904057660 con 0x7f69040b6210 2021-04-03T03:15:40.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f68f27fc700 1 -- 172.21.2.111:0/746044582 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7f68f404a290 con 0x7f69040b6210 2021-04-03T03:15:40.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f68e4056ea0 msgr2=0x7f68e4059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:40.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f68e4056ea0 0x7f68e4059360 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f690000d000 tx=0x7f6900007b60).stop 2021-04-03T03:15:40.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040b6210 msgr2=0x7f6904063d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:40.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040b6210 0x7f6904063d00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f69041483d0 tx=0x7f68f4015000).stop 2021-04-03T03:15:40.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 shutdown_connections 2021-04-03T03:15:40.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f68e4056ea0 0x7f68e4059360 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:40.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6904139690 0x7f6904145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:40.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69040af8c0 0x7f69041422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:40.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 --2- 172.21.2.111:0/746044582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69040b6210 0x7f6904063d00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:40.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.954+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 >> 172.21.2.111:0/746044582 conn(0x7f69040b56f0 msgr2=0x7f69040b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:40.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.958+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 shutdown_connections 2021-04-03T03:15:40.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:40.958+0000 7f690a46a700 1 -- 172.21.2.111:0/746044582 wait complete. 2021-04-03T03:15:40.961 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:15:40.965 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:15:40.965 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:15:41.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.062+0000 7fbe5f131700 1 Processor -- start 2021-04-03T03:15:41.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.062+0000 7fbe5f131700 1 -- start start 2021-04-03T03:15:41.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbe580b5d60 0x7fbe580b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580b4bc0 con 0x7fbe580b44a0 2021-04-03T03:15:41.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580607d0 con 0x7fbe580603f0 2021-04-03T03:15:41.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe580b4df0 con 0x7fbe580b5d60 2021-04-03T03:15:41.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fbe580b44a0 0x7fbe58142490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55824/0 (socket says 172.21.2.111:55824) 2021-04-03T03:15:41.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 -- 172.21.2.111:0/2536149673 learned_addr learned my addr 172.21.2.111:0/2536149673 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:41.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 --2- 172.21.2.111:0/2536149673 >> v2:172.21.2.115:3300/0 conn(0x7fbe580b5d60 0x7fbe580b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779937045 0 0) 0x7fbe580b4bc0 con 0x7fbe580b44a0 2021-04-03T03:15:41.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe40003240 con 0x7fbe580b44a0 2021-04-03T03:15:41.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2492598778 0 0) 0x7fbe40003240 con 0x7fbe580b44a0 2021-04-03T03:15:41.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 >> v2:172.21.2.115:3300/0 conn(0x7fbe580b5d60 msgr2=0x7fbe580b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 --2- 172.21.2.111:0/2536149673 >> v2:172.21.2.115:3300/0 conn(0x7fbe580b5d60 0x7fbe580b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 >> v1:172.21.2.115:6789/0 conn(0x7fbe580603f0 legacy=0x7fbe580b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580b5090 con 0x7fbe580b44a0 2021-04-03T03:15:41.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fbe50001aa0 con 0x7fbe580b44a0 2021-04-03T03:15:41.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 >> v1:172.21.2.111:6789/0 conn(0x7fbe580b44a0 legacy=0x7fbe58142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe40003730 0x7fbe40003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe400044e0 0x7fbe40004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe40004e50 0x7fbe4000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580b4df0 con 0x7fbe40004e50 2021-04-03T03:15:41.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40004260 con 0x7fbe400044e0 2021-04-03T03:15:41.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4000bc20 con 0x7fbe40003730 2021-04-03T03:15:41.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe40004e50 0x7fbe4000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 -- 172.21.2.111:0/2536149673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe40003730 msgr2=0x7fbe40003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe40003730 0x7fbe40003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 -- 172.21.2.111:0/2536149673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe400044e0 msgr2=0x7fbe40004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe400044e0 0x7fbe40004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 -- 172.21.2.111:0/2536149673 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe40003240 con 0x7fbe40004e50 2021-04-03T03:15:41.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 -- 172.21.2.111:0/2536149673 reap_dead start 2021-04-03T03:15:41.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5e930700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe40004e50 0x7fbe4000b730 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7fbe400054b0 tx=0x7fbe58062d40).ready entity=mon.0 client_cookie=113711a227f15223 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe50005ca0 con 0x7fbe40004e50 2021-04-03T03:15:41.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50005e00 con 0x7fbe40004e50 2021-04-03T03:15:41.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d12d700 1 -- 172.21.2.111:0/2536149673 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe500072c0 con 0x7fbe40004e50 2021-04-03T03:15:41.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- 172.21.2.111:0/2536149673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe40004e50 msgr2=0x7fbe4000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe40004e50 0x7fbe4000b730 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7fbe400054b0 tx=0x7fbe58062d40).stop 2021-04-03T03:15:41.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- 172.21.2.111:0/2536149673 shutdown_connections 2021-04-03T03:15:41.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2536149673 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe40004e50 0x7fbe4000b730 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- 172.21.2.111:0/2536149673 >> 172.21.2.111:0/2536149673 conn(0x7fbe5805c7d0 msgr2=0x7fbe5805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- 172.21.2.111:0/2536149673 shutdown_connections 2021-04-03T03:15:41.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- 172.21.2.111:0/2536149673 wait complete. 2021-04-03T03:15:41.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 Processor -- start 2021-04-03T03:15:41.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- start start 2021-04-03T03:15:41.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe40004e50 0x7fbe580afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe580b0330 0x7fbe5813e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe5813eb20 0x7fbe581520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58113f50 con 0x7fbe580b0330 2021-04-03T03:15:41.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe581141f0 con 0x7fbe40004e50 2021-04-03T03:15:41.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5f131700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58114490 con 0x7fbe5813eb20 2021-04-03T03:15:41.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe580b0330 0x7fbe5813e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe580b0330 0x7fbe5813e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55042/0 (socket says 172.21.2.111:55042) 2021-04-03T03:15:41.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 -- 172.21.2.111:0/2627486300 learned_addr learned my addr 172.21.2.111:0/2627486300 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 -- 172.21.2.111:0/2627486300 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe5813eb20 msgr2=0x7fbe581520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe5813eb20 0x7fbe581520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 -- 172.21.2.111:0/2627486300 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe40004e50 msgr2=0x7fbe580afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe40004e50 0x7fbe580afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 -- 172.21.2.111:0/2627486300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58114730 con 0x7fbe580b0330 2021-04-03T03:15:41.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.066+0000 7fbe5d92e700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe580b0330 0x7fbe5813e630 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7fbe4800c000 tx=0x7fbe48006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe477fe700 1 -- 172.21.2.111:0/2627486300 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe48016070 con 0x7fbe580b0330 2021-04-03T03:15:41.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe477fe700 1 -- 172.21.2.111:0/2627486300 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4800c860 con 0x7fbe580b0330 2021-04-03T03:15:41.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5813efa0 con 0x7fbe580b0330 2021-04-03T03:15:41.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe477fe700 1 -- 172.21.2.111:0/2627486300 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe4800cbc0 con 0x7fbe580b0330 2021-04-03T03:15:41.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe580aec40 con 0x7fbe580b0330 2021-04-03T03:15:41.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe581148c0 con 0x7fbe580b0330 2021-04-03T03:15:41.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe477fe700 1 -- 172.21.2.111:0/2627486300 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbe4800cd20 con 0x7fbe580b0330 2021-04-03T03:15:41.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe477fe700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbe38056fe0 0x7fbe380594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe477fe700 1 -- 172.21.2.111:0/2627486300 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fbe4806fef0 con 0x7fbe580b0330 2021-04-03T03:15:41.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe5e12f700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbe38056fe0 0x7fbe380594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe5e12f700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbe38056fe0 0x7fbe380594a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fbe580b76a0 tx=0x7fbe540077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.070+0000 7fbe477fe700 1 -- 172.21.2.111:0/2627486300 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbe48045610 con 0x7fbe580b0330 2021-04-03T03:15:41.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbe58057660 con 0x7fbe580b0330 2021-04-03T03:15:41.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe477fe700 1 -- 172.21.2.111:0/2627486300 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fbe48048c30 con 0x7fbe580b0330 2021-04-03T03:15:41.200 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:41.201 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:15:41.201 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:15:41.201 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.201 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:15:41.202 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:15:41.202 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.202 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:41.202 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.202 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.203 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.203 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.203 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.203 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.204 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:15:41.204 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:15:41.204 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.204 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:41.204 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.205 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.205 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.205 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.205 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.205 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.206 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:15:41.206 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:15:41.206 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.206 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.207 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.207 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.207 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.207 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.207 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.208 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:15:41.208 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:15:41.208 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.208 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.209 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.209 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.209 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.209 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.209 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.210 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:15:41.210 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:15:41.210 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.210 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.210 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.211 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.211 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.211 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.211 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.212 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:15:41.212 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:15:41.212 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.212 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.212 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.213 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.213 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.213 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.213 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.213 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:15:41.214 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:15:41.214 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.214 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.214 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.214 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.215 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.215 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.215 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.215 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:15:41.216 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:15:41.216 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.216 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.216 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.216 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.217 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.217 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.217 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.217 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:15:41.217 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:15:41.218 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.218 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.218 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.218 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.219 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.219 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.219 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.219 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:15:41.219 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:15:41.220 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.220 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.220 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.220 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.220 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.221 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.221 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.221 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:15:41.221 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:15:41.222 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.222 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:41.222 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:41.222 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:41.222 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.223 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.223 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.223 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:15:41.223 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:15:41.224 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.224 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:41.224 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:41.224 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:41.225 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.225 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.225 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.225 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:15:41.225 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:15:41.226 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.226 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:41.226 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:41.226 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:41.226 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.227 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.227 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.227 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:15:41.227 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:15:41.228 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.228 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:41.228 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:41.228 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:41.228 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.229 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.229 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.229 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.229 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:15:41.229 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:15:41.230 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.230 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:41.230 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:15:41.230 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.230 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.231 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.231 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.231 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:15:41.231 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:15:41.231 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:41.232 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:41.232 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:15:41.232 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:41.232 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.233 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.233 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:41.233 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:15:41.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbe38056fe0 msgr2=0x7fbe380594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbe38056fe0 0x7fbe380594a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fbe580b76a0 tx=0x7fbe540077f0).stop 2021-04-03T03:15:41.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe580b0330 msgr2=0x7fbe5813e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe580b0330 0x7fbe5813e630 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7fbe4800c000 tx=0x7fbe48006dd0).stop 2021-04-03T03:15:41.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 shutdown_connections 2021-04-03T03:15:41.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe5813eb20 0x7fbe581520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbe38056fe0 0x7fbe380594a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe40004e50 0x7fbe580afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 --2- 172.21.2.111:0/2627486300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe580b0330 0x7fbe5813e630 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 >> 172.21.2.111:0/2627486300 conn(0x7fbe5805c7d0 msgr2=0x7fbe5813fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 shutdown_connections 2021-04-03T03:15:41.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.198+0000 7fbe5f131700 1 -- 172.21.2.111:0/2627486300 wait complete. 2021-04-03T03:15:41.238 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:41.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 Processor -- start 2021-04-03T03:15:41.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- start start 2021-04-03T03:15:41.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9d340dbb90 0x7f9d34042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 0x7f9d340dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9d340064b0 0x7f9d34006940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d340d9560 con 0x7f9d340064b0 2021-04-03T03:15:41.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3403dcb0 con 0x7f9d340dbb90 2021-04-03T03:15:41.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d34006e30 con 0x7f9d3403d360 2021-04-03T03:15:41.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9d340064b0 0x7f9d34006940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9d340064b0 0x7f9d34006940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55052/0 (socket says 172.21.2.111:55052) 2021-04-03T03:15:41.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2082443034 learned_addr learned my addr 172.21.2.111:0/2082443034 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2082443034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 msgr2=0x7f9d340dbf70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 --2- 172.21.2.111:0/2082443034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 0x7f9d340dbf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2082443034 >> v2:172.21.2.115:3300/0 conn(0x7f9d340dbb90 msgr2=0x7f9d34042750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 --2- 172.21.2.111:0/2082443034 >> v2:172.21.2.115:3300/0 conn(0x7f9d340dbb90 0x7f9d34042750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2082443034 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d340dc6e0 con 0x7f9d340064b0 2021-04-03T03:15:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3b7fe700 1 --2- 172.21.2.111:0/2082443034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340064b0 0x7f9d34006940 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f9d34006f70 tx=0x7f9d3c05ad70).ready entity=mon.0 client_cookie=2e1285334b848df2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d39ffb700 1 -- 172.21.2.111:0/2082443034 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d3c05bda0 con 0x7f9d340064b0 2021-04-03T03:15:41.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d39ffb700 1 -- 172.21.2.111:0/2082443034 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c05df60 con 0x7f9d340064b0 2021-04-03T03:15:41.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d39ffb700 1 -- 172.21.2.111:0/2082443034 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d3c0699f0 con 0x7f9d340064b0 2021-04-03T03:15:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2082443034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340064b0 msgr2=0x7f9d34006940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2082443034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340064b0 0x7f9d34006940 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f9d34006f70 tx=0x7f9d3c05ad70).stop 2021-04-03T03:15:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2082443034 shutdown_connections 2021-04-03T03:15:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2082443034 >> v2:172.21.2.115:3300/0 conn(0x7f9d340dbb90 0x7f9d34042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2082443034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340064b0 0x7f9d34006940 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2082443034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 0x7f9d340dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2082443034 >> 172.21.2.111:0/2082443034 conn(0x7f9d340462c0 msgr2=0x7f9d340466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2082443034 shutdown_connections 2021-04-03T03:15:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.298+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2082443034 wait complete. 2021-04-03T03:15:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 Processor -- start 2021-04-03T03:15:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 -- start start 2021-04-03T03:15:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d340064b0 0x7f9d340dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 0x7f9d34039900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340dbb90 0x7f9d34039df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340df1d0 con 0x7f9d340dbb90 2021-04-03T03:15:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340df400 con 0x7f9d340064b0 2021-04-03T03:15:41.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340de920 con 0x7f9d3403d360 2021-04-03T03:15:41.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340dbb90 0x7f9d34039df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340dbb90 0x7f9d34039df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55058/0 (socket says 172.21.2.111:55058) 2021-04-03T03:15:41.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2940195634 learned_addr learned my addr 172.21.2.111:0/2940195634 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2940195634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 msgr2=0x7f9d34039900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 0x7f9d34039900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2940195634 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d340064b0 msgr2=0x7f9d340dd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d340064b0 0x7f9d340dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 -- 172.21.2.111:0/2940195634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3403a2e0 con 0x7f9d340dbb90 2021-04-03T03:15:41.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3b7fe700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340dbb90 0x7f9d34039df0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f9d34006f70 tx=0x7f9d3c064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d33fff700 1 -- 172.21.2.111:0/2940195634 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d3c0699f0 con 0x7f9d340dbb90 2021-04-03T03:15:41.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d33fff700 1 -- 172.21.2.111:0/2940195634 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0441b0 con 0x7f9d340dbb90 2021-04-03T03:15:41.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3409f4b0 con 0x7f9d340dbb90 2021-04-03T03:15:41.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d33fff700 1 -- 172.21.2.111:0/2940195634 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d3c05f4b0 con 0x7f9d340dbb90 2021-04-03T03:15:41.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d340de350 con 0x7f9d340dbb90 2021-04-03T03:15:41.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3409f790 con 0x7f9d340dbb90 2021-04-03T03:15:41.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d33fff700 1 -- 172.21.2.111:0/2940195634 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9d3c098060 con 0x7f9d340dbb90 2021-04-03T03:15:41.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d33fff700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d20057000 0x7f9d200594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d33fff700 1 -- 172.21.2.111:0/2940195634 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f9d3c0e0990 con 0x7f9d340dbb90 2021-04-03T03:15:41.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3affd700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d20057000 0x7f9d200594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.302+0000 7f9d3affd700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d20057000 0x7f9d200594c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9d2c000e90 tx=0x7f9d2c006900).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.306+0000 7f9d33fff700 1 -- 172.21.2.111:0/2940195634 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9d3c0b60b0 con 0x7f9d340dbb90 2021-04-03T03:15:41.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.398+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d3403a470 con 0x7f9d340dbb90 2021-04-03T03:15:41.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.398+0000 7f9d33fff700 1 -- 172.21.2.111:0/2940195634 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f9d3c0b96d0 con 0x7f9d340dbb90 2021-04-03T03:15:41.402 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:41.402 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":89,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:40.945758+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:41.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d20057000 msgr2=0x7f9d200594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d20057000 0x7f9d200594c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9d2c000e90 tx=0x7f9d2c006900).stop 2021-04-03T03:15:41.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340dbb90 msgr2=0x7f9d34039df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340dbb90 0x7f9d34039df0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f9d34006f70 tx=0x7f9d3c064030).stop 2021-04-03T03:15:41.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 shutdown_connections 2021-04-03T03:15:41.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d340dbb90 0x7f9d34039df0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d20057000 0x7f9d200594c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d340064b0 0x7f9d340dd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 --2- 172.21.2.111:0/2940195634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d3403d360 0x7f9d34039900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 >> 172.21.2.111:0/2940195634 conn(0x7f9d340462c0 msgr2=0x7f9d340454d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 shutdown_connections 2021-04-03T03:15:41.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.402+0000 7f9d3bfff700 1 -- 172.21.2.111:0/2940195634 wait complete. 2021-04-03T03:15:41.410 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:15:41.411 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:41.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02989de700 1 Processor -- start 2021-04-03T03:15:41.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02989de700 1 -- start start 2021-04-03T03:15:41.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02989de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02940603f0 0x7f02941447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02989de700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02940607d0 con 0x7f02940b17b0 2021-04-03T03:15:41.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02989de700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029414b090 con 0x7f02940add50 2021-04-03T03:15:41.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02989de700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029414b330 con 0x7f02940603f0 2021-04-03T03:15:41.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f0293fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f02940b17b0 0x7f0294147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55856/0 (socket says 172.21.2.111:55856) 2021-04-03T03:15:41.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f0293fff700 1 -- 172.21.2.111:0/671118705 learned_addr learned my addr 172.21.2.111:0/671118705 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:41.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565874538 0 0) 0x7f02940607d0 con 0x7f02940b17b0 2021-04-03T03:15:41.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0274003240 con 0x7f02940b17b0 2021-04-03T03:15:41.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2137926272 0 0) 0x7f0274003240 con 0x7f02940b17b0 2021-04-03T03:15:41.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02940603f0 msgr2=0x7f02941447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02940603f0 0x7f02941447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 >> v1:172.21.2.115:6789/0 conn(0x7f02940add50 legacy=0x7f0294147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029414b5d0 con 0x7f02940b17b0 2021-04-03T03:15:41.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f028c001a50 con 0x7f02940b17b0 2021-04-03T03:15:41.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 >> v1:172.21.2.111:6789/0 conn(0x7f02940b17b0 legacy=0x7f0294147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02740037e0 0x7f0274003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02940603f0 0x7f02740044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02740049d0 0x7f0274004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029414b330 con 0x7f02740037e0 2021-04-03T03:15:41.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740042b0 con 0x7f02740049d0 2021-04-03T03:15:41.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027400b730 con 0x7f02940603f0 2021-04-03T03:15:41.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02937fe700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02740037e0 0x7f0274003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f0293fff700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02740049d0 0x7f0274004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02937fe700 1 -- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02940603f0 msgr2=0x7f02740044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02937fe700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02940603f0 0x7f02740044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02937fe700 1 -- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02740049d0 msgr2=0x7f0274004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02937fe700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02740049d0 0x7f0274004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.466+0000 7f02937fe700 1 -- 172.21.2.111:0/671118705 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274003240 con 0x7f02740037e0 2021-04-03T03:15:41.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02937fe700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02740037e0 0x7f0274003be0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f027c002f30 tx=0x7f027c006e10).ready entity=mon.0 client_cookie=9bfd0d7f151090e8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f027c0097f0 con 0x7f02740037e0 2021-04-03T03:15:41.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c009950 con 0x7f02740037e0 2021-04-03T03:15:41.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02927fc700 1 -- 172.21.2.111:0/671118705 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f027c009cb0 con 0x7f02740037e0 2021-04-03T03:15:41.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/671118705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02740037e0 msgr2=0x7f0274003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 --2- 172.21.2.111:0/671118705 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02740037e0 0x7f0274003be0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f027c002f30 tx=0x7f027c006e10).stop 2021-04-03T03:15:41.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 -- 172.21.2.111:0/671118705 reap_dead start 2021-04-03T03:15:41.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/671118705 shutdown_connections 2021-04-03T03:15:41.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/671118705 >> 172.21.2.111:0/671118705 conn(0x7f02940b61c0 msgr2=0x7f029405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/671118705 shutdown_connections 2021-04-03T03:15:41.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/671118705 wait complete. 2021-04-03T03:15:41.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 Processor -- start 2021-04-03T03:15:41.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- start start 2021-04-03T03:15:41.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0294062d90 0x7f0294067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02940aa290 0x7f0294067a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0294064450 0x7f02940648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294063170 con 0x7f0294064450 2021-04-03T03:15:41.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02940aba50 con 0x7f02940aa290 2021-04-03T03:15:41.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940b4130 con 0x7f0294062d90 2021-04-03T03:15:41.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0294064450 0x7f02940648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0294064450 0x7f02940648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55078/0 (socket says 172.21.2.111:55078) 2021-04-03T03:15:41.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 -- 172.21.2.111:0/1646229262 learned_addr learned my addr 172.21.2.111:0/1646229262 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 -- 172.21.2.111:0/1646229262 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0294062d90 msgr2=0x7f0294067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0294062d90 0x7f0294067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 -- 172.21.2.111:0/1646229262 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02940aa290 msgr2=0x7f0294067a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02940aa290 0x7f0294067a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 -- 172.21.2.111:0/1646229262 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02940b43d0 con 0x7f0294064450 2021-04-03T03:15:41.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0293fff700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0294064450 0x7f02940648d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f028c001c30 tx=0x7f028c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0290ff9700 1 -- 172.21.2.111:0/1646229262 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028c004b50 con 0x7f0294064450 2021-04-03T03:15:41.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0290ff9700 1 -- 172.21.2.111:0/1646229262 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c004cb0 con 0x7f0294064450 2021-04-03T03:15:41.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0290ff9700 1 -- 172.21.2.111:0/1646229262 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028c00b900 con 0x7f0294064450 2021-04-03T03:15:41.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0294067f00 con 0x7f0294064450 2021-04-03T03:15:41.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02940aa020 con 0x7f0294064450 2021-04-03T03:15:41.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0294057660 con 0x7f0294064450 2021-04-03T03:15:41.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0290ff9700 1 -- 172.21.2.111:0/1646229262 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f028c00ba60 con 0x7f0294064450 2021-04-03T03:15:41.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.470+0000 7f0290ff9700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0278056fb0 0x7f0278059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.474+0000 7f0290ff9700 1 -- 172.21.2.111:0/1646229262 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f028c06fd00 con 0x7f0294064450 2021-04-03T03:15:41.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.474+0000 7f02937fe700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0278056fb0 0x7f0278059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.474+0000 7f02937fe700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0278056fb0 0x7f0278059470 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f027c002f30 tx=0x7f027c016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.474+0000 7f0290ff9700 1 -- 172.21.2.111:0/1646229262 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f028c045310 con 0x7f0294064450 2021-04-03T03:15:41.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f029405feb0 con 0x7f0294064450 2021-04-03T03:15:41.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f0290ff9700 1 -- 172.21.2.111:0/1646229262 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v89) v1 ==== 110+0+10 (secure 0 0 0) 0x7f028c048930 con 0x7f0294064450 2021-04-03T03:15:41.571 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:41.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0278056fb0 msgr2=0x7f0278059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0278056fb0 0x7f0278059470 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f027c002f30 tx=0x7f027c016000).stop 2021-04-03T03:15:41.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0294064450 msgr2=0x7f02940648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0294064450 0x7f02940648d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f028c001c30 tx=0x7f028c009040).stop 2021-04-03T03:15:41.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 shutdown_connections 2021-04-03T03:15:41.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0294064450 0x7f02940648d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0294062d90 0x7f0294067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0278056fb0 0x7f0278059470 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 --2- 172.21.2.111:0/1646229262 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02940aa290 0x7f0294067a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 >> 172.21.2.111:0/1646229262 conn(0x7f02940b61c0 msgr2=0x7f02940affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 shutdown_connections 2021-04-03T03:15:41.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.570+0000 7f02989de700 1 -- 172.21.2.111:0/1646229262 wait complete. 2021-04-03T03:15:41.578 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:41.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 Processor -- start 2021-04-03T03:15:41.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 -- start start 2021-04-03T03:15:41.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6c280603f0 0x7f6c281447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280add50 0x7f6c28144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6c280b17b0 0x7f6c28147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c281451c0 con 0x7f6c280b17b0 2021-04-03T03:15:41.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c280607d0 con 0x7f6c280603f0 2021-04-03T03:15:41.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c27fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2814b090 con 0x7f6c280add50 2021-04-03T03:15:41.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6c280b17b0 0x7f6c28147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6c280b17b0 0x7f6c28147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55086/0 (socket says 172.21.2.111:55086) 2021-04-03T03:15:41.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 -- 172.21.2.111:0/1815435261 learned_addr learned my addr 172.21.2.111:0/1815435261 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 -- 172.21.2.111:0/1815435261 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280add50 msgr2=0x7f6c28144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 --2- 172.21.2.111:0/1815435261 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280add50 0x7f6c28144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 -- 172.21.2.111:0/1815435261 >> v2:172.21.2.115:3300/0 conn(0x7f6c280603f0 msgr2=0x7f6c281447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 --2- 172.21.2.111:0/1815435261 >> v2:172.21.2.115:3300/0 conn(0x7f6c280603f0 0x7f6c281447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 -- 172.21.2.111:0/1815435261 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2814b330 con 0x7f6c280b17b0 2021-04-03T03:15:41.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.634+0000 7f6c277fe700 1 --2- 172.21.2.111:0/1815435261 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c28147740 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f6c200032b0 tx=0x7f6c20009270).ready entity=mon.0 client_cookie=e687cc03a0aa821d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c25ffb700 1 -- 172.21.2.111:0/1815435261 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c2000bc00 con 0x7f6c280b17b0 2021-04-03T03:15:41.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c25ffb700 1 -- 172.21.2.111:0/1815435261 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000bd60 con 0x7f6c280b17b0 2021-04-03T03:15:41.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c25ffb700 1 -- 172.21.2.111:0/1815435261 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c20011400 con 0x7f6c280b17b0 2021-04-03T03:15:41.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/1815435261 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 msgr2=0x7f6c28147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 --2- 172.21.2.111:0/1815435261 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c28147740 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f6c200032b0 tx=0x7f6c20009270).stop 2021-04-03T03:15:41.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/1815435261 shutdown_connections 2021-04-03T03:15:41.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 --2- 172.21.2.111:0/1815435261 >> v2:172.21.2.115:3300/0 conn(0x7f6c280603f0 0x7f6c281447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 --2- 172.21.2.111:0/1815435261 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c28147740 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 --2- 172.21.2.111:0/1815435261 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280add50 0x7f6c28144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/1815435261 >> 172.21.2.111:0/1815435261 conn(0x7f6c280b61c0 msgr2=0x7f6c2805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/1815435261 shutdown_connections 2021-04-03T03:15:41.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/1815435261 wait complete. 2021-04-03T03:15:41.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 Processor -- start 2021-04-03T03:15:41.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- start start 2021-04-03T03:15:41.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280603f0 0x7f6c28147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c280add50 0x7f6c28063050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c280648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280aad60 con 0x7f6c280b17b0 2021-04-03T03:15:41.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28063540 con 0x7f6c280add50 2021-04-03T03:15:41.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28062bf0 con 0x7f6c280603f0 2021-04-03T03:15:41.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c280648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c280648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55090/0 (socket says 172.21.2.111:55090) 2021-04-03T03:15:41.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 -- 172.21.2.111:0/49160175 learned_addr learned my addr 172.21.2.111:0/49160175 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 -- 172.21.2.111:0/49160175 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280603f0 msgr2=0x7f6c28147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280603f0 0x7f6c28147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 -- 172.21.2.111:0/49160175 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c280add50 msgr2=0x7f6c28063050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c280add50 0x7f6c28063050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 -- 172.21.2.111:0/49160175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280aaac0 con 0x7f6c280b17b0 2021-04-03T03:15:41.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c277fe700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c280648a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f6c200032b0 tx=0x7f6c20016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c1ffff700 1 -- 172.21.2.111:0/49160175 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c2001f420 con 0x7f6c280b17b0 2021-04-03T03:15:41.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c1ffff700 1 -- 172.21.2.111:0/49160175 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2001f580 con 0x7f6c280b17b0 2021-04-03T03:15:41.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c1ffff700 1 -- 172.21.2.111:0/49160175 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c2001f8e0 con 0x7f6c280b17b0 2021-04-03T03:15:41.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c280ab400 con 0x7f6c280b17b0 2021-04-03T03:15:41.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c280aa7e0 con 0x7f6c280b17b0 2021-04-03T03:15:41.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c28057660 con 0x7f6c280b17b0 2021-04-03T03:15:41.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c1ffff700 1 -- 172.21.2.111:0/49160175 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6c2000f070 con 0x7f6c280b17b0 2021-04-03T03:15:41.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c1ffff700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c0c056ea0 0x7f6c0c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c1ffff700 1 -- 172.21.2.111:0/49160175 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f6c20070690 con 0x7f6c280b17b0 2021-04-03T03:15:41.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c26ffd700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c0c056ea0 0x7f6c0c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.638+0000 7f6c26ffd700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c0c056ea0 0x7f6c0c059360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f6c18002f30 tx=0x7f6c18006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.642+0000 7f6c1ffff700 1 -- 172.21.2.111:0/49160175 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6c20045db0 con 0x7f6c280b17b0 2021-04-03T03:15:41.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.734+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6c28148850 con 0x7f6c280b17b0 2021-04-03T03:15:41.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c1ffff700 1 -- 172.21.2.111:0/49160175 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6c200150a0 con 0x7f6c280b17b0 2021-04-03T03:15:41.739 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:41.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c0c056ea0 msgr2=0x7f6c0c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c0c056ea0 0x7f6c0c059360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f6c18002f30 tx=0x7f6c18006a60).stop 2021-04-03T03:15:41.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 msgr2=0x7f6c280648a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c280648a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f6c200032b0 tx=0x7f6c20016000).stop 2021-04-03T03:15:41.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 shutdown_connections 2021-04-03T03:15:41.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c280b17b0 0x7f6c280648a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c280603f0 0x7f6c28147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c0c056ea0 0x7f6c0c059360 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 --2- 172.21.2.111:0/49160175 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c280add50 0x7f6c28063050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 >> 172.21.2.111:0/49160175 conn(0x7f6c280b61c0 msgr2=0x7f6c280b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 shutdown_connections 2021-04-03T03:15:41.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.738+0000 7f6c27fff700 1 -- 172.21.2.111:0/49160175 wait complete. 2021-04-03T03:15:41.746 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:15:41.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 Processor -- start 2021-04-03T03:15:41.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- start start 2021-04-03T03:15:41.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e4006bc0 0x7fa9e4042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa9e403d360 0x7fa9e40dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e40dc6e0 con 0x7fa9e40dbb90 2021-04-03T03:15:41.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e40d9560 con 0x7fa9e403d360 2021-04-03T03:15:41.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e403dcb0 con 0x7fa9e4006bc0 2021-04-03T03:15:41.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa9e403d360 0x7fa9e40dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa9e403d360 0x7fa9e40dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55100/0 (socket says 172.21.2.111:55100) 2021-04-03T03:15:41.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f1aaa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e4006bc0 0x7fa9e4042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f22ab700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fa9e40dbb90 0x7fa9e40064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:40192/0 (socket says 172.21.2.111:40192) 2021-04-03T03:15:41.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 -- 172.21.2.111:0/1650961520 learned_addr learned my addr 172.21.2.111:0/1650961520 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 -- 172.21.2.111:0/1650961520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e4006bc0 msgr2=0x7fa9e4042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:41.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 --2- 172.21.2.111:0/1650961520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e4006bc0 0x7fa9e4042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 -- 172.21.2.111:0/1650961520 >> v1:172.21.2.115:6789/0 conn(0x7fa9e40dbb90 legacy=0x7fa9e40064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 -- 172.21.2.111:0/1650961520 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40dc460 con 0x7fa9e403d360 2021-04-03T03:15:41.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 --2- 172.21.2.111:0/1650961520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403d360 0x7fa9e40dbf70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fa9ec06b5d0 tx=0x7fa9ec05ad70).ready entity=mon.0 client_cookie=daf55e325a24832f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f12a9700 1 -- 172.21.2.111:0/1650961520 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec05bda0 con 0x7fa9e403d360 2021-04-03T03:15:41.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f12a9700 1 -- 172.21.2.111:0/1650961520 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec05df60 con 0x7fa9e403d360 2021-04-03T03:15:41.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f12a9700 1 -- 172.21.2.111:0/1650961520 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec0699f0 con 0x7fa9e403d360 2021-04-03T03:15:41.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- 172.21.2.111:0/1650961520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403d360 msgr2=0x7fa9e40dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/1650961520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403d360 0x7fa9e40dbf70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fa9ec06b5d0 tx=0x7fa9ec05ad70).stop 2021-04-03T03:15:41.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- 172.21.2.111:0/1650961520 shutdown_connections 2021-04-03T03:15:41.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/1650961520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403d360 0x7fa9e40dbf70 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/1650961520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e4006bc0 0x7fa9e4042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- 172.21.2.111:0/1650961520 >> 172.21.2.111:0/1650961520 conn(0x7fa9e40462c0 msgr2=0x7fa9e40466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- 172.21.2.111:0/1650961520 shutdown_connections 2021-04-03T03:15:41.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- 172.21.2.111:0/1650961520 wait complete. 2021-04-03T03:15:41.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 Processor -- start 2021-04-03T03:15:41.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- start start 2021-04-03T03:15:41.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e403d360 0x7fa9e4005e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9e40dbb90 0x7fa9e40ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403c670 0x7fa9e40e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40e72a0 con 0x7fa9e403c670 2021-04-03T03:15:41.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40a25f0 con 0x7fa9e40dbb90 2021-04-03T03:15:41.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f32ad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40a2350 con 0x7fa9e403d360 2021-04-03T03:15:41.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403c670 0x7fa9e40e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403c670 0x7fa9e40e7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55104/0 (socket says 172.21.2.111:55104) 2021-04-03T03:15:41.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f2aac700 1 -- 172.21.2.111:0/621246890 learned_addr learned my addr 172.21.2.111:0/621246890 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:41.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.802+0000 7fa9f1aaa700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9e40dbb90 0x7fa9e40ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f22ab700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e403d360 0x7fa9e4005e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f2aac700 1 -- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e403d360 msgr2=0x7fa9e4005e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f2aac700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e403d360 0x7fa9e4005e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f2aac700 1 -- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9e40dbb90 msgr2=0x7fa9e40ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f2aac700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9e40dbb90 0x7fa9e40ea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f2aac700 1 -- 172.21.2.111:0/621246890 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e403b760 con 0x7fa9e403c670 2021-04-03T03:15:41.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f22ab700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e403d360 0x7fa9e4005e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:41.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f2aac700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403c670 0x7fa9e40e7700 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fa9ec06b5d0 tx=0x7fa9ec065000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9db7fe700 1 -- 172.21.2.111:0/621246890 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec05df60 con 0x7fa9e403c670 2021-04-03T03:15:41.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e403c2b0 con 0x7fa9e403c670 2021-04-03T03:15:41.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9db7fe700 1 -- 172.21.2.111:0/621246890 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec0699f0 con 0x7fa9e403c670 2021-04-03T03:15:41.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9db7fe700 1 -- 172.21.2.111:0/621246890 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec05f2c0 con 0x7fa9e403c670 2021-04-03T03:15:41.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e40e7d80 con 0x7fa9e403c670 2021-04-03T03:15:41.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e40e70e0 con 0x7fa9e403c670 2021-04-03T03:15:41.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9db7fe700 1 -- 172.21.2.111:0/621246890 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa9ec08c020 con 0x7fa9e403c670 2021-04-03T03:15:41.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9db7fe700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9cc057000 0x7fa9cc0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9db7fe700 1 -- 172.21.2.111:0/621246890 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fa9ec0e09b0 con 0x7fa9e403c670 2021-04-03T03:15:41.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f22ab700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9cc057000 0x7fa9cc0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9f22ab700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9cc057000 0x7fa9cc0594c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa9e8008000 tx=0x7fa9e8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.806+0000 7fa9db7fe700 1 -- 172.21.2.111:0/621246890 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa9ec0b5fc0 con 0x7fa9e403c670 2021-04-03T03:15:41.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.902+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa9e4004da0 con 0x7fa9e403c670 2021-04-03T03:15:41.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.902+0000 7fa9db7fe700 1 -- 172.21.2.111:0/621246890 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa9ec0b95e0 con 0x7fa9e403c670 2021-04-03T03:15:41.906 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:41.906 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:15:41.906 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.906 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:15:41.907 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:15:41.907 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:41.907 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:41.907 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:15:41.908 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:15:41.908 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:41.908 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:15:41.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9cc057000 msgr2=0x7fa9cc0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9cc057000 0x7fa9cc0594c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa9e8008000 tx=0x7fa9e8009040).stop 2021-04-03T03:15:41.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403c670 msgr2=0x7fa9e40e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403c670 0x7fa9e40e7700 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fa9ec06b5d0 tx=0x7fa9ec065000).stop 2021-04-03T03:15:41.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 shutdown_connections 2021-04-03T03:15:41.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9e403c670 0x7fa9e40e7700 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9e403d360 0x7fa9e4005e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa9cc057000 0x7fa9cc0594c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 --2- 172.21.2.111:0/621246890 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9e40dbb90 0x7fa9e40ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 >> 172.21.2.111:0/621246890 conn(0x7fa9e40462c0 msgr2=0x7fa9e4042950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:41.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 shutdown_connections 2021-04-03T03:15:41.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.906+0000 7fa9f32ad700 1 -- 172.21.2.111:0/621246890 wait complete. 2021-04-03T03:15:41.913 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:15:41.914 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:15:41.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 Processor -- start 2021-04-03T03:15:41.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- start start 2021-04-03T03:15:41.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb1c00add50 0x7fb1c0147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb1c0144ef0 0x7fb1c0147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:41.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00607d0 con 0x7fb1c00603f0 2021-04-03T03:15:41.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c014b090 con 0x7fb1c00add50 2021-04-03T03:15:41.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c014b330 con 0x7fb1c0144ef0 2021-04-03T03:15:41.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c612a700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fb1c00603f0 0x7fb1c01447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55902/0 (socket says 172.21.2.111:55902) 2021-04-03T03:15:41.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c612a700 1 -- 172.21.2.111:0/1917357090 learned_addr learned my addr 172.21.2.111:0/1917357090 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:41.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5929700 1 --2- 172.21.2.111:0/1917357090 >> v2:172.21.2.111:3300/0 conn(0x7fb1c00add50 0x7fb1c0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:41.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5929700 1 -- 172.21.2.111:0/1917357090 >> v2:172.21.2.115:3300/0 conn(0x7fb1c0144ef0 msgr2=0x7fb1c0147c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:41.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5929700 1 --2- 172.21.2.111:0/1917357090 >> v2:172.21.2.115:3300/0 conn(0x7fb1c0144ef0 0x7fb1c0147c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5929700 1 -- 172.21.2.111:0/1917357090 >> v1:172.21.2.111:6789/0 conn(0x7fb1c00603f0 legacy=0x7fb1c01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5929700 1 -- 172.21.2.111:0/1917357090 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c014b5d0 con 0x7fb1c00add50 2021-04-03T03:15:41.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5929700 1 --2- 172.21.2.111:0/1917357090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0147740 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7fb1b0002f30 tx=0x7fb1b0006dd0).ready entity=mon.0 client_cookie=ecb5f15187183b00 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:41.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5128700 1 -- 172.21.2.111:0/1917357090 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b00097f0 con 0x7fb1c00add50 2021-04-03T03:15:41.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5128700 1 -- 172.21.2.111:0/1917357090 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b0009950 con 0x7fb1c00add50 2021-04-03T03:15:41.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5128700 1 -- 172.21.2.111:0/1917357090 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b0009cb0 con 0x7fb1c00add50 2021-04-03T03:15:41.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- 172.21.2.111:0/1917357090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 msgr2=0x7fb1c0147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:41.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- 172.21.2.111:0/1917357090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0147740 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7fb1b0002f30 tx=0x7fb1b0006dd0).stop 2021-04-03T03:15:41.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- 172.21.2.111:0/1917357090 shutdown_connections 2021-04-03T03:15:41.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- 172.21.2.111:0/1917357090 >> v2:172.21.2.115:3300/0 conn(0x7fb1c0144ef0 0x7fb1c0147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:41.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- 172.21.2.111:0/1917357090 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0147740 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:41.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- 172.21.2.111:0/1917357090 >> 172.21.2.111:0/1917357090 conn(0x7fb1c00b61c0 msgr2=0x7fb1c005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:42.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- 172.21.2.111:0/1917357090 shutdown_connections 2021-04-03T03:15:42.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- 172.21.2.111:0/1917357090 wait complete. 2021-04-03T03:15:42.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 Processor -- start 2021-04-03T03:15:42.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- start start 2021-04-03T03:15:42.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1c00603f0 0x7fb1c0145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1c0144ef0 0x7fb1c0143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0067fc0 con 0x7fb1c00add50 2021-04-03T03:15:42.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00ab150 con 0x7fb1c0144ef0 2021-04-03T03:15:42.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c712c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00ab380 con 0x7fb1c00603f0 2021-04-03T03:15:42.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.982+0000 7fb1c5929700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0143970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55118/0 (socket says 172.21.2.111:55118) 2021-04-03T03:15:42.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 -- 172.21.2.111:0/2520945692 learned_addr learned my addr 172.21.2.111:0/2520945692 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:42.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 -- 172.21.2.111:0/2520945692 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1c00603f0 msgr2=0x7fb1c0145d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:42.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1c00603f0 0x7fb1c0145d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:42.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 -- 172.21.2.111:0/2520945692 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1c0144ef0 msgr2=0x7fb1c0143e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:42.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1c0144ef0 0x7fb1c0143e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:42.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 -- 172.21.2.111:0/2520945692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00ab620 con 0x7fb1c00add50 2021-04-03T03:15:42.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c5929700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0143970 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7fb1b0002f30 tx=0x7fb1b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:42.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1af7fe700 1 -- 172.21.2.111:0/2520945692 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b001e460 con 0x7fb1c00add50 2021-04-03T03:15:42.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1af7fe700 1 -- 172.21.2.111:0/2520945692 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b001e5c0 con 0x7fb1c00add50 2021-04-03T03:15:42.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c0157ae0 con 0x7fb1c00add50 2021-04-03T03:15:42.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c0144350 con 0x7fb1c00add50 2021-04-03T03:15:42.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1af7fe700 1 -- 172.21.2.111:0/2520945692 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b001e920 con 0x7fb1c00add50 2021-04-03T03:15:42.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c00ab7b0 con 0x7fb1c00add50 2021-04-03T03:15:42.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1af7fe700 1 -- 172.21.2.111:0/2520945692 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb1b0025030 con 0x7fb1c00add50 2021-04-03T03:15:42.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1af7fe700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1a0056fb0 0x7fb1a0059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1af7fe700 1 -- 172.21.2.111:0/2520945692 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fb1b0071340 con 0x7fb1c00add50 2021-04-03T03:15:42.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c612a700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1a0056fb0 0x7fb1a0059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1c612a700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1a0056fb0 0x7fb1a0059470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fb1c00ae520 tx=0x7fb1bc007750).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:42.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:41.986+0000 7fb1af7fe700 1 -- 172.21.2.111:0/2520945692 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb1b0046a60 con 0x7fb1c00add50 2021-04-03T03:15:42.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.082+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb1c0057660 con 0x7fb1c00add50 2021-04-03T03:15:42.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1af7fe700 1 -- 172.21.2.111:0/2520945692 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb1b00230a0 con 0x7fb1c00add50 2021-04-03T03:15:42.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1a0056fb0 msgr2=0x7fb1a0059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:42.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1a0056fb0 0x7fb1a0059470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fb1c00ae520 tx=0x7fb1bc007750).stop 2021-04-03T03:15:42.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 msgr2=0x7fb1c0143970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:42.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0143970 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7fb1b0002f30 tx=0x7fb1b0015000).stop 2021-04-03T03:15:42.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 shutdown_connections 2021-04-03T03:15:42.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb1a0056fb0 0x7fb1a0059470 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:42.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1c0144ef0 0x7fb1c0143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:42.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1c00603f0 0x7fb1c0145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:42.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 --2- 172.21.2.111:0/2520945692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1c00add50 0x7fb1c0143970 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:42.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 >> 172.21.2.111:0/2520945692 conn(0x7fb1c00b61c0 msgr2=0x7fb1c005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:42.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 shutdown_connections 2021-04-03T03:15:42.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.850+0000 7fb1c712c700 1 -- 172.21.2.111:0/2520945692 wait complete. 2021-04-03T03:15:42.855 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:15:42.859 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:15:42.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 Processor -- start 2021-04-03T03:15:42.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 -- start start 2021-04-03T03:15:42.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00603f0 0x7fc5f00b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc5f00b44a0 0x7fc5f00b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00607d0 con 0x7fc5f00b5620 2021-04-03T03:15:42.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f01433c0 con 0x7fc5f00b44a0 2021-04-03T03:15:42.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00b3f80 con 0x7fc5f00603f0 2021-04-03T03:15:42.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5effff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc5f00b44a0 0x7fc5f00b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f4ea1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00603f0 0x7fc5f00b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5effff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc5f00b44a0 0x7fc5f00b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:44832/0 (socket says 172.21.2.111:44832) 2021-04-03T03:15:42.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5effff700 1 -- 172.21.2.111:0/2092047515 learned_addr learned my addr 172.21.2.111:0/2092047515 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:42.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f56a2700 1 --1- 172.21.2.111:0/2092047515 >> v1:172.21.2.115:6789/0 conn(0x7fc5f00b5620 0x7fc5f01469f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:40220/0 (socket says 172.21.2.111:40220) 2021-04-03T03:15:42.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f4ea1700 1 -- 172.21.2.111:0/2092047515 >> v2:172.21.2.115:3300/0 conn(0x7fc5f00b44a0 msgr2=0x7fc5f00b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:42.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f4ea1700 1 --2- 172.21.2.111:0/2092047515 >> v2:172.21.2.115:3300/0 conn(0x7fc5f00b44a0 0x7fc5f00b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:42.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f4ea1700 1 -- 172.21.2.111:0/2092047515 >> v1:172.21.2.115:6789/0 conn(0x7fc5f00b5620 legacy=0x7fc5f01469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:42.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5ef7fe700 1 -- 172.21.2.111:0/2092047515 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180985123 0 0) 0x7fc5f00607d0 con 0x7fc5f00b5620 2021-04-03T03:15:42.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f4ea1700 1 -- 172.21.2.111:0/2092047515 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00b4c60 con 0x7fc5f00603f0 2021-04-03T03:15:42.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f4ea1700 1 --2- 172.21.2.111:0/2092047515 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00603f0 0x7fc5f00b2480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fc5e0002f30 tx=0x7fc5e0006dd0).ready entity=mon.2 client_cookie=fcb04d88c2a759ef server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:42.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5ef7fe700 1 -- 172.21.2.111:0/2092047515 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e0009760 con 0x7fc5f00603f0 2021-04-03T03:15:42.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5ef7fe700 1 -- 172.21.2.111:0/2092047515 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e00098c0 con 0x7fc5f00603f0 2021-04-03T03:15:42.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2092047515 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00603f0 msgr2=0x7fc5f00b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:42.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.918+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2092047515 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00603f0 0x7fc5f00b2480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fc5e0002f30 tx=0x7fc5e0006dd0).stop 2021-04-03T03:15:42.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2092047515 shutdown_connections 2021-04-03T03:15:42.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2092047515 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00603f0 0x7fc5f00b2480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:42.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2092047515 >> v2:172.21.2.115:3300/0 conn(0x7fc5f00b44a0 0x7fc5f00b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:42.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2092047515 >> 172.21.2.111:0/2092047515 conn(0x7fc5f005c7d0 msgr2=0x7fc5f005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:42.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2092047515 shutdown_connections 2021-04-03T03:15:42.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2092047515 wait complete. 2021-04-03T03:15:42.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 Processor -- start 2021-04-03T03:15:42.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- start start 2021-04-03T03:15:42.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5f00603f0 0x7fc5f0067470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00b44a0 0x7fc5f0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc5f00b5620 0x7fc5f0156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01448f0 con 0x7fc5f00603f0 2021-04-03T03:15:42.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0144b50 con 0x7fc5f00b5620 2021-04-03T03:15:42.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0065460 con 0x7fc5f00b44a0 2021-04-03T03:15:42.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5f00603f0 0x7fc5f0067470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5f00603f0 0x7fc5f0067470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55128/0 (socket says 172.21.2.111:55128) 2021-04-03T03:15:42.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 -- 172.21.2.111:0/2282432360 learned_addr learned my addr 172.21.2.111:0/2282432360 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:42.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f56a2700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc5f00b5620 0x7fc5f0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5effff700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00b44a0 0x7fc5f0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 -- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00b44a0 msgr2=0x7fc5f0156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:42.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00b44a0 0x7fc5f0156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:42.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 -- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc5f00b5620 msgr2=0x7fc5f0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:42.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc5f00b5620 0x7fc5f0156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:42.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 -- 172.21.2.111:0/2282432360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0067960 con 0x7fc5f00603f0 2021-04-03T03:15:42.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f4ea1700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5f00603f0 0x7fc5f0067470 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fc5e0002f30 tx=0x7fc5e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:42.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5edffb700 1 -- 172.21.2.111:0/2282432360 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e0011e60 con 0x7fc5f00603f0 2021-04-03T03:15:42.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5edffb700 1 -- 172.21.2.111:0/2282432360 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e001c5c0 con 0x7fc5f00603f0 2021-04-03T03:15:42.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f014b770 con 0x7fc5f00603f0 2021-04-03T03:15:42.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5edffb700 1 -- 172.21.2.111:0/2282432360 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e001c920 con 0x7fc5f00603f0 2021-04-03T03:15:42.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f0064940 con 0x7fc5f00603f0 2021-04-03T03:15:42.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0065620 con 0x7fc5f00603f0 2021-04-03T03:15:42.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5edffb700 1 -- 172.21.2.111:0/2282432360 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc5e000f070 con 0x7fc5f00603f0 2021-04-03T03:15:42.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5edffb700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc5d4056fb0 0x7fc5d4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:42.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5edffb700 1 -- 172.21.2.111:0/2282432360 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5950+0+0 (secure 0 0 0) 0x7fc5e0071ec0 con 0x7fc5f00603f0 2021-04-03T03:15:42.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5effff700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc5d4056fb0 0x7fc5d4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:42.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.922+0000 7fc5effff700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc5d4056fb0 0x7fc5d4059470 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fc5d800b000 tx=0x7fc5d8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:42.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:42.926+0000 7fc5edffb700 1 -- 172.21.2.111:0/2282432360 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc5e0047370 con 0x7fc5f00603f0 2021-04-03T03:15:43.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.022+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fc5f0057660 con 0x7fc5f00603f0 2021-04-03T03:15:43.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5edffb700 1 -- 172.21.2.111:0/2282432360 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v91) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc5e004a990 con 0x7fc5f00603f0 2021-04-03T03:15:43.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc5d4056fb0 msgr2=0x7fc5d4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:43.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc5d4056fb0 0x7fc5d4059470 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fc5d800b000 tx=0x7fc5d8008040).stop 2021-04-03T03:15:43.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5f00603f0 msgr2=0x7fc5f0067470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:43.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5f00603f0 0x7fc5f0067470 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fc5e0002f30 tx=0x7fc5e0015000).stop 2021-04-03T03:15:43.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 shutdown_connections 2021-04-03T03:15:43.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc5d4056fb0 0x7fc5d4059470 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc5f00b5620 0x7fc5f0156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5f00603f0 0x7fc5f0067470 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 --2- 172.21.2.111:0/2282432360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5f00b44a0 0x7fc5f0156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 >> 172.21.2.111:0/2282432360 conn(0x7fc5f005c7d0 msgr2=0x7fc5f00b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:43.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 shutdown_connections 2021-04-03T03:15:43.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.858+0000 7fc5f5ea3700 1 -- 172.21.2.111:0/2282432360 wait complete. 2021-04-03T03:15:43.866 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:15:43.871 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:15:43.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.922+0000 7f09214e2700 1 Processor -- start 2021-04-03T03:15:43.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.922+0000 7f09214e2700 1 -- start start 2021-04-03T03:15:43.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.922+0000 7f09214e2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09140dbba0 0x7f09140dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:43.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.922+0000 7f09214e2700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f091403d560 0x7f0914041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:43.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.922+0000 7f09214e2700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140dc560 con 0x7f0914041060 2021-04-03T03:15:43.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.922+0000 7f09214e2700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091403de60 con 0x7f09140dbba0 2021-04-03T03:15:43.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.922+0000 7f09214e2700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140dc800 con 0x7f091403d560 2021-04-03T03:15:43.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091b7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0914041060 0x7f091403fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55928/0 (socket says 172.21.2.111:55928) 2021-04-03T03:15:43.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09140dbba0 0x7f09140dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:43.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091b7fe700 1 -- 172.21.2.111:0/3627215304 learned_addr learned my addr 172.21.2.111:0/3627215304 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:43.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f0920ce1700 1 --2- 172.21.2.111:0/3627215304 >> v2:172.21.2.115:3300/0 conn(0x7f091403d560 0x7f0914041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:43.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 -- 172.21.2.111:0/3627215304 >> v2:172.21.2.115:3300/0 conn(0x7f091403d560 msgr2=0x7f0914041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:43.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- 172.21.2.111:0/3627215304 >> v2:172.21.2.115:3300/0 conn(0x7f091403d560 0x7f0914041650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 -- 172.21.2.111:0/3627215304 >> v1:172.21.2.111:6789/0 conn(0x7f0914041060 legacy=0x7f091403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:43.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091affd700 1 -- 172.21.2.111:0/3627215304 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36170340 0 0) 0x7f09140dc560 con 0x7f0914041060 2021-04-03T03:15:43.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 -- 172.21.2.111:0/3627215304 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091403da30 con 0x7f09140dbba0 2021-04-03T03:15:43.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f0920ce1700 1 --2- 172.21.2.111:0/3627215304 >> v2:172.21.2.115:3300/0 conn(0x7f091403d560 0x7f0914041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:43.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- 172.21.2.111:0/3627215304 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09140dbba0 0x7f09140dbf80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f090c002f30 tx=0x7f090c006dd0).ready entity=mon.0 client_cookie=86817bf323dd82c1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:43.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091affd700 1 -- 172.21.2.111:0/3627215304 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c0097f0 con 0x7f09140dbba0 2021-04-03T03:15:43.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091affd700 1 -- 172.21.2.111:0/3627215304 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c009950 con 0x7f09140dbba0 2021-04-03T03:15:43.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091affd700 1 -- 172.21.2.111:0/3627215304 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c009cb0 con 0x7f09140dbba0 2021-04-03T03:15:43.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- 172.21.2.111:0/3627215304 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09140dbba0 msgr2=0x7f09140dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:43.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 --2- 172.21.2.111:0/3627215304 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09140dbba0 0x7f09140dbf80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f090c002f30 tx=0x7f090c006dd0).stop 2021-04-03T03:15:43.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- 172.21.2.111:0/3627215304 shutdown_connections 2021-04-03T03:15:43.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 --2- 172.21.2.111:0/3627215304 >> v2:172.21.2.115:3300/0 conn(0x7f091403d560 0x7f0914041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 --2- 172.21.2.111:0/3627215304 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09140dbba0 0x7f09140dbf80 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- 172.21.2.111:0/3627215304 >> 172.21.2.111:0/3627215304 conn(0x7f09140462b0 msgr2=0x7f09140466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:43.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- 172.21.2.111:0/3627215304 shutdown_connections 2021-04-03T03:15:43.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- 172.21.2.111:0/3627215304 wait complete. 2021-04-03T03:15:43.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 Processor -- start 2021-04-03T03:15:43.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- start start 2021-04-03T03:15:43.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f091403d560 0x7f09140a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:43.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0914041060 0x7f0914044ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:43.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09140dbba0 0x7f09140a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:43.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09140df020 con 0x7f091403d560 2021-04-03T03:15:43.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09140df2c0 con 0x7f0914041060 2021-04-03T03:15:43.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09140df560 con 0x7f09140dbba0 2021-04-03T03:15:43.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f091403d560 0x7f09140a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:43.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f091403d560 0x7f09140a2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55144/0 (socket says 172.21.2.111:55144) 2021-04-03T03:15:43.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 -- 172.21.2.111:0/344538289 learned_addr learned my addr 172.21.2.111:0/344538289 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:43.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091b7fe700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0914041060 0x7f0914044ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:43.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 -- 172.21.2.111:0/344538289 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09140dbba0 msgr2=0x7f09140a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:43.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09140dbba0 0x7f09140a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:43.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 -- 172.21.2.111:0/344538289 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0914041060 msgr2=0x7f0914044ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:43.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0914041060 0x7f0914044ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:43.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 -- 172.21.2.111:0/344538289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091409f670 con 0x7f091403d560 2021-04-03T03:15:43.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f091bfff700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f091403d560 0x7f09140a2f10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f090c002f30 tx=0x7f090c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:43.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09197fa700 1 -- 172.21.2.111:0/344538289 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c011ce0 con 0x7f091403d560 2021-04-03T03:15:43.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09197fa700 1 -- 172.21.2.111:0/344538289 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c011e40 con 0x7f091403d560 2021-04-03T03:15:43.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091403c170 con 0x7f091403d560 2021-04-03T03:15:43.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09197fa700 1 -- 172.21.2.111:0/344538289 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c01e650 con 0x7f091403d560 2021-04-03T03:15:43.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.926+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09140a1150 con 0x7f091403d560 2021-04-03T03:15:43.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.930+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09140df900 con 0x7f091403d560 2021-04-03T03:15:43.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.930+0000 7f09197fa700 1 -- 172.21.2.111:0/344538289 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f090c025030 con 0x7f091403d560 2021-04-03T03:15:43.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.930+0000 7f09197fa700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f08fc057050 0x7f08fc059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:43.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.930+0000 7f09197fa700 1 -- 172.21.2.111:0/344538289 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f090c071640 con 0x7f091403d560 2021-04-03T03:15:43.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.930+0000 7f091b7fe700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f08fc057050 0x7f08fc059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:43.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.930+0000 7f091b7fe700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f08fc057050 0x7f08fc059510 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f091000b000 tx=0x7f0910009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:43.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:43.930+0000 7f09197fa700 1 -- 172.21.2.111:0/344538289 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f09140df900 con 0x7f091403d560 2021-04-03T03:15:44.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.058+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f09140df900 con 0x7f091403d560 2021-04-03T03:15:44.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.062+0000 7f09197fa700 1 -- 172.21.2.111:0/344538289 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v37) v1 ==== 189+0+0 (secure 0 0 0) 0x7f09140df900 con 0x7f091403d560 2021-04-03T03:15:44.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f08fc057050 msgr2=0x7f08fc059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f08fc057050 0x7f08fc059510 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f091000b000 tx=0x7f0910009040).stop 2021-04-03T03:15:44.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f091403d560 msgr2=0x7f09140a2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f091403d560 0x7f09140a2f10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f090c002f30 tx=0x7f090c015000).stop 2021-04-03T03:15:44.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 shutdown_connections 2021-04-03T03:15:44.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09140dbba0 0x7f09140a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f091403d560 0x7f09140a2f10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f08fc057050 0x7f08fc059510 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 --2- 172.21.2.111:0/344538289 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0914041060 0x7f0914044ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 >> 172.21.2.111:0/344538289 conn(0x7f09140462b0 msgr2=0x7f09140055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:44.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 shutdown_connections 2021-04-03T03:15:44.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.066+0000 7f09214e2700 1 -- 172.21.2.111:0/344538289 wait complete. 2021-04-03T03:15:44.074 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 17 and data pool 18 2021-04-03T03:15:44.079 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:44.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 Processor -- start 2021-04-03T03:15:44.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- start start 2021-04-03T03:15:44.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c0ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb62c060390 0x7fb62c0a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0afff0 con 0x7fb62c0aa150 2021-04-03T03:15:44.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c141260 con 0x7fb62c060390 2021-04-03T03:15:44.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b0290 con 0x7fb62c0acf10 2021-04-03T03:15:44.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb62c060390 0x7fb62c0a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c0ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb62c060390 0x7fb62c0a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:44870/0 (socket says 172.21.2.111:44870) 2021-04-03T03:15:44.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c0ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:38506/0 (socket says 172.21.2.111:38506) 2021-04-03T03:15:44.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 -- 172.21.2.111:0/1862668456 learned_addr learned my addr 172.21.2.111:0/1862668456 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:44.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 -- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 msgr2=0x7fb62c0ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:44.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 --2- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c0ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 -- 172.21.2.111:0/1862668456 >> v1:172.21.2.115:6789/0 conn(0x7fb62c0aa150 legacy=0x7fb62c0b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 -- 172.21.2.111:0/1862668456 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0b0530 con 0x7fb62c060390 2021-04-03T03:15:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62bfff700 1 --2- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c0ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:15:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62b7fe700 1 --2- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c060390 0x7fb62c0a9c60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fb61c002f30 tx=0x7fb61c006e90).ready entity=mon.1 client_cookie=2cf446ec7ee83746 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62affd700 1 -- 172.21.2.111:0/1862668456 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb61c009850 con 0x7fb62c060390 2021-04-03T03:15:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62affd700 1 -- 172.21.2.111:0/1862668456 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0099b0 con 0x7fb62c060390 2021-04-03T03:15:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb62affd700 1 -- 172.21.2.111:0/1862668456 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb61c009d10 con 0x7fb62c060390 2021-04-03T03:15:44.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c060390 msgr2=0x7fb62c0a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 --2- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c060390 0x7fb62c0a9c60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fb61c002f30 tx=0x7fb61c006e90).stop 2021-04-03T03:15:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- 172.21.2.111:0/1862668456 shutdown_connections 2021-04-03T03:15:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 --2- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c0ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 --2- 172.21.2.111:0/1862668456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c060390 0x7fb62c0a9c60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- 172.21.2.111:0/1862668456 >> 172.21.2.111:0/1862668456 conn(0x7fb62c063d70 msgr2=0x7fb62c064180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- 172.21.2.111:0/1862668456 shutdown_connections 2021-04-03T03:15:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.130+0000 7fb631221700 1 -- 172.21.2.111:0/1862668456 wait complete. 2021-04-03T03:15:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 Processor -- start 2021-04-03T03:15:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 -- start start 2021-04-03T03:15:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb62c060390 0x7fb62c0ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c0aa150 0x7fb62c064cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0ac370 con 0x7fb62c060390 2021-04-03T03:15:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0651c0 con 0x7fb62c0aa150 2021-04-03T03:15:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0aac50 con 0x7fb62c0acf10 2021-04-03T03:15:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb62c060390 0x7fb62c0ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb62c060390 0x7fb62c0ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55164/0 (socket says 172.21.2.111:55164) 2021-04-03T03:15:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 -- 172.21.2.111:0/702644688 learned_addr learned my addr 172.21.2.111:0/702644688 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62b7fe700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c0aa150 0x7fb62c064cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb630a20700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 -- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 msgr2=0x7fb62c156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 -- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c0aa150 msgr2=0x7fb62c064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c0aa150 0x7fb62c064cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 -- 172.21.2.111:0/702644688 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0afa70 con 0x7fb62c060390 2021-04-03T03:15:44.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62bfff700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb62c060390 0x7fb62c0ae550 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7fb62000c000 tx=0x7fb620008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb6297fa700 1 -- 172.21.2.111:0/702644688 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62000ca00 con 0x7fb62c060390 2021-04-03T03:15:44.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb6297fa700 1 -- 172.21.2.111:0/702644688 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62000cb60 con 0x7fb62c060390 2021-04-03T03:15:44.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb6297fa700 1 -- 172.21.2.111:0/702644688 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6200123f0 con 0x7fb62c060390 2021-04-03T03:15:44.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c0aaef0 con 0x7fb62c060390 2021-04-03T03:15:44.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c067900 con 0x7fb62c060390 2021-04-03T03:15:44.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c057660 con 0x7fb62c060390 2021-04-03T03:15:44.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb6297fa700 1 -- 172.21.2.111:0/702644688 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb620022070 con 0x7fb62c060390 2021-04-03T03:15:44.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb6297fa700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb60c056f70 0x7fb60c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb6297fa700 1 -- 172.21.2.111:0/702644688 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb620070880 con 0x7fb62c060390 2021-04-03T03:15:44.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62b7fe700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb60c056f70 0x7fb60c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.134+0000 7fb62b7fe700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb60c056f70 0x7fb60c059430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fb61c01a000 tx=0x7fb61c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.138+0000 7fb6297fa700 1 -- 172.21.2.111:0/702644688 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb620045c70 con 0x7fb62c060390 2021-04-03T03:15:44.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb62c067da0 con 0x7fb62c060390 2021-04-03T03:15:44.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb6297fa700 1 -- 172.21.2.111:0/702644688 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fb620049290 con 0x7fb62c060390 2021-04-03T03:15:44.239 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:44.239 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":92,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:44.064364+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:42.095114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:43.035163+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:44.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb60c056f70 msgr2=0x7fb60c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb60c056f70 0x7fb60c059430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fb61c01a000 tx=0x7fb61c015000).stop 2021-04-03T03:15:44.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb62c060390 msgr2=0x7fb62c0ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb62c060390 0x7fb62c0ae550 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7fb62000c000 tx=0x7fb620008040).stop 2021-04-03T03:15:44.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 shutdown_connections 2021-04-03T03:15:44.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb62c0acf10 0x7fb62c156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb62c060390 0x7fb62c0ae550 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb60c056f70 0x7fb60c059430 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 --2- 172.21.2.111:0/702644688 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb62c0aa150 0x7fb62c064cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 >> 172.21.2.111:0/702644688 conn(0x7fb62c063d70 msgr2=0x7fb62c0b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 shutdown_connections 2021-04-03T03:15:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.234+0000 7fb631221700 1 -- 172.21.2.111:0/702644688 wait complete. 2021-04-03T03:15:44.247 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:44.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 Processor -- start 2021-04-03T03:15:44.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- start start 2021-04-03T03:15:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f424c0603f0 0x7f424c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 0x7f424c147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424c0607d0 con 0x7f424c14b700 2021-04-03T03:15:44.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c0af400 con 0x7f424c0603f0 2021-04-03T03:15:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0af6a0 con 0x7f424c0b0c90 2021-04-03T03:15:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424affd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f424c0603f0 0x7f424c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424a7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f424c14b700 0x7f424c0b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55964/0 (socket says 172.21.2.111:55964) 2021-04-03T03:15:44.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424a7fc700 1 -- 172.21.2.111:0/701537248 learned_addr learned my addr 172.21.2.111:0/701537248 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424affd700 1 -- 172.21.2.111:0/701537248 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 msgr2=0x7f424c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424affd700 1 --2- 172.21.2.111:0/701537248 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 0x7f424c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424affd700 1 -- 172.21.2.111:0/701537248 >> v1:172.21.2.111:6789/0 conn(0x7f424c14b700 legacy=0x7f424c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424affd700 1 -- 172.21.2.111:0/701537248 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c0afb40 con 0x7f424c0603f0 2021-04-03T03:15:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424affd700 1 --2- 172.21.2.111:0/701537248 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c0603f0 0x7f424c0b0090 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f4234002f30 tx=0x7f4234006dd0).ready entity=mon.0 client_cookie=9904c3e82ba30336 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f4249ffb700 1 -- 172.21.2.111:0/701537248 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42340097f0 con 0x7f424c0603f0 2021-04-03T03:15:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f4249ffb700 1 -- 172.21.2.111:0/701537248 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234009950 con 0x7f424c0603f0 2021-04-03T03:15:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f4249ffb700 1 -- 172.21.2.111:0/701537248 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4234009cb0 con 0x7f424c0603f0 2021-04-03T03:15:44.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- 172.21.2.111:0/701537248 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c0603f0 msgr2=0x7f424c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 --2- 172.21.2.111:0/701537248 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c0603f0 0x7f424c0b0090 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f4234002f30 tx=0x7f4234006dd0).stop 2021-04-03T03:15:44.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- 172.21.2.111:0/701537248 shutdown_connections 2021-04-03T03:15:44.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 --2- 172.21.2.111:0/701537248 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 0x7f424c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 --2- 172.21.2.111:0/701537248 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c0603f0 0x7f424c0b0090 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- 172.21.2.111:0/701537248 >> 172.21.2.111:0/701537248 conn(0x7f424c0b61c0 msgr2=0x7f424c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:44.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- 172.21.2.111:0/701537248 shutdown_connections 2021-04-03T03:15:44.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.298+0000 7f424bfff700 1 -- 172.21.2.111:0/701537248 wait complete. 2021-04-03T03:15:44.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 Processor -- start 2021-04-03T03:15:44.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 -- start start 2021-04-03T03:15:44.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f424c0603f0 0x7f424c0a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 0x7f424c159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c14b700 0x7f424c064690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0a8630 con 0x7f424c14b700 2021-04-03T03:15:44.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0aa620 con 0x7f424c0603f0 2021-04-03T03:15:44.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0aa8c0 con 0x7f424c0b0c90 2021-04-03T03:15:44.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c14b700 0x7f424c064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c14b700 0x7f424c064690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55178/0 (socket says 172.21.2.111:55178) 2021-04-03T03:15:44.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 -- 172.21.2.111:0/3759822342 learned_addr learned my addr 172.21.2.111:0/3759822342 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:44.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 -- 172.21.2.111:0/3759822342 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 msgr2=0x7f424c159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 0x7f424c159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 -- 172.21.2.111:0/3759822342 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f424c0603f0 msgr2=0x7f424c0a78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f424c0603f0 0x7f424c0a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 -- 172.21.2.111:0/3759822342 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c0641f0 con 0x7f424c14b700 2021-04-03T03:15:44.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424b7fe700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c14b700 0x7f424c064690 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f42440032b0 tx=0x7f4244009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f4243fff700 1 -- 172.21.2.111:0/3759822342 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424400ca60 con 0x7f424c14b700 2021-04-03T03:15:44.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f4243fff700 1 -- 172.21.2.111:0/3759822342 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424400cbc0 con 0x7f424c14b700 2021-04-03T03:15:44.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c0ab270 con 0x7f424c14b700 2021-04-03T03:15:44.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c0aa0a0 con 0x7f424c14b700 2021-04-03T03:15:44.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f4243fff700 1 -- 172.21.2.111:0/3759822342 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4244015420 con 0x7f424c14b700 2021-04-03T03:15:44.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424c064380 con 0x7f424c14b700 2021-04-03T03:15:44.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f4243fff700 1 -- 172.21.2.111:0/3759822342 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f424401c070 con 0x7f424c14b700 2021-04-03T03:15:44.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f4243fff700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4230056f70 0x7f4230059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f4243fff700 1 -- 172.21.2.111:0/3759822342 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f424406feb0 con 0x7f424c14b700 2021-04-03T03:15:44.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424affd700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4230056f70 0x7f4230059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.302+0000 7f424affd700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4230056f70 0x7f4230059430 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f4234002f30 tx=0x7f4234015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.306+0000 7f4243fff700 1 -- 172.21.2.111:0/3759822342 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4244045130 con 0x7f424c14b700 2021-04-03T03:15:44.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.398+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f424c057660 con 0x7f424c14b700 2021-04-03T03:15:44.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f4243fff700 1 -- 172.21.2.111:0/3759822342 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f424401a030 con 0x7f424c14b700 2021-04-03T03:15:44.406 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:44.407 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":92,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:44.064364+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:42.095114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:43.035163+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:44.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4230056f70 msgr2=0x7f4230059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4230056f70 0x7f4230059430 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f4234002f30 tx=0x7f4234015000).stop 2021-04-03T03:15:44.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c14b700 msgr2=0x7f424c064690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c14b700 0x7f424c064690 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f42440032b0 tx=0x7f4244009220).stop 2021-04-03T03:15:44.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 shutdown_connections 2021-04-03T03:15:44.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f424c14b700 0x7f424c064690 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4230056f70 0x7f4230059430 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f424c0603f0 0x7f424c0a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 --2- 172.21.2.111:0/3759822342 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f424c0b0c90 0x7f424c159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 >> 172.21.2.111:0/3759822342 conn(0x7f424c0b61c0 msgr2=0x7f424c147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:44.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 shutdown_connections 2021-04-03T03:15:44.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.402+0000 7f424bfff700 1 -- 172.21.2.111:0/3759822342 wait complete. 2021-04-03T03:15:44.414 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:15:44.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f359e700 1 Processor -- start 2021-04-03T03:15:44.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f359e700 1 -- start start 2021-04-03T03:15:44.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd4ec0dbba0 0x7fd4ec0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 0x7fd4ec041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec0dc560 con 0x7fd4ec041060 2021-04-03T03:15:44.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec03de60 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0dc800 con 0x7fd4ec03d560 2021-04-03T03:15:44.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd4ec0dbba0 0x7fd4ec0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd4ec0dbba0 0x7fd4ec0dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55184/0 (socket says 172.21.2.111:55184) 2021-04-03T03:15:44.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f259c700 1 -- 172.21.2.111:0/4255876991 learned_addr learned my addr 172.21.2.111:0/4255876991 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:44.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f259c700 1 -- 172.21.2.111:0/4255876991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 msgr2=0x7fd4ec041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:44.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f259c700 1 --2- 172.21.2.111:0/4255876991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 0x7fd4ec041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f259c700 1 -- 172.21.2.111:0/4255876991 >> v1:172.21.2.115:6789/0 conn(0x7fd4ec041060 legacy=0x7fd4ec03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:44.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.466+0000 7fd4f259c700 1 -- 172.21.2.111:0/4255876991 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec03da30 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f259c700 1 --2- 172.21.2.111:0/4255876991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0dbf80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7fd4ec0087d0 tx=0x7fd4e8006e90).ready entity=mon.0 client_cookie=49817f5e039d3d2b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f159a700 1 -- 172.21.2.111:0/4255876991 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4e80098e0 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f159a700 1 -- 172.21.2.111:0/4255876991 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e8009a40 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f159a700 1 -- 172.21.2.111:0/4255876991 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4e8009de0 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/4255876991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 msgr2=0x7fd4ec0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 --2- 172.21.2.111:0/4255876991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0dbf80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7fd4ec0087d0 tx=0x7fd4e8006e90).stop 2021-04-03T03:15:44.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/4255876991 shutdown_connections 2021-04-03T03:15:44.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 --2- 172.21.2.111:0/4255876991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 0x7fd4ec041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 --2- 172.21.2.111:0/4255876991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0dbf80 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:44.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/4255876991 >> 172.21.2.111:0/4255876991 conn(0x7fd4ec0462b0 msgr2=0x7fd4ec0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:44.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/4255876991 shutdown_connections 2021-04-03T03:15:44.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/4255876991 wait complete. 2021-04-03T03:15:44.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 Processor -- start 2021-04-03T03:15:44.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- start start 2021-04-03T03:15:44.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 0x7fd4ec0de240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4ec041060 0x7fd4ec039580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0e70a0 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0e7290 con 0x7fd4ec041060 2021-04-03T03:15:44.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0e7530 con 0x7fd4ec03d560 2021-04-03T03:15:44.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0e79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55194/0 (socket says 172.21.2.111:55194) 2021-04-03T03:15:44.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 -- 172.21.2.111:0/2018032608 learned_addr learned my addr 172.21.2.111:0/2018032608 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:44.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 -- 172.21.2.111:0/2018032608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 msgr2=0x7fd4ec0de240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 0x7fd4ec0de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 -- 172.21.2.111:0/2018032608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4ec041060 msgr2=0x7fd4ec039580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:44.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4ec041060 0x7fd4ec039580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:44.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 -- 172.21.2.111:0/2018032608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec039a70 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f2d9d700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0e79d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fd4f406b5d0 tx=0x7fd4f405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4e37fe700 1 -- 172.21.2.111:0/2018032608 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f405f2c0 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4e37fe700 1 -- 172.21.2.111:0/2018032608 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f405f420 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec03bf10 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec09f4a0 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4e37fe700 1 -- 172.21.2.111:0/2018032608 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f405f780 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec004db0 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.470+0000 7fd4e37fe700 1 -- 172.21.2.111:0/2018032608 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd4f408b020 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.474+0000 7fd4e37fe700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4d0056fb0 0x7fd4d0059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:44.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.474+0000 7fd4e37fe700 1 -- 172.21.2.111:0/2018032608 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd4f40df0c0 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.474+0000 7fd4f259c700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4d0056fb0 0x7fd4d0059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:44.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.474+0000 7fd4f259c700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4d0056fb0 0x7fd4d0059470 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fd4ec0087d0 tx=0x7fd4e8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:44.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.474+0000 7fd4e37fe700 1 -- 172.21.2.111:0/2018032608 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd4f40b4560 con 0x7fd4ec0dbba0 2021-04-03T03:15:44.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:44.598+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd4ec0e76a0 con 0x7fd4ec0dbba0 2021-04-03T03:15:45.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.070+0000 7fd4e37fe700 1 -- 172.21.2.111:0/2018032608 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd4f40b7b80 con 0x7fd4ec0dbba0 2021-04-03T03:15:45.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.070+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4d0056fb0 msgr2=0x7fd4d0059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.070+0000 7fd4f359e700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4d0056fb0 0x7fd4d0059470 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fd4ec0087d0 tx=0x7fd4e8015000).stop 2021-04-03T03:15:45.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.070+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 msgr2=0x7fd4ec0e79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.070+0000 7fd4f359e700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0e79d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fd4f406b5d0 tx=0x7fd4f405ad70).stop 2021-04-03T03:15:45.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 shutdown_connections 2021-04-03T03:15:45.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd4ec0dbba0 0x7fd4ec0e79d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd4ec03d560 0x7fd4ec0de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4d0056fb0 0x7fd4d0059470 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 --2- 172.21.2.111:0/2018032608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd4ec041060 0x7fd4ec039580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 >> 172.21.2.111:0/2018032608 conn(0x7fd4ec0462b0 msgr2=0x7fd4ec0459a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 shutdown_connections 2021-04-03T03:15:45.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.074+0000 7fd4f359e700 1 -- 172.21.2.111:0/2018032608 wait complete. 2021-04-03T03:15:45.086 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:45.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc24e4a700 1 Processor -- start 2021-04-03T03:15:45.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc24e4a700 1 -- start start 2021-04-03T03:15:45.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc24e4a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdc18046b70 0x7fdc180d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc24e4a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdc18042300 0x7fdc18043920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc24e4a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc180d4ef0 con 0x7fdc18045690 2021-04-03T03:15:45.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc24e4a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc18046f50 con 0x7fdc18042300 2021-04-03T03:15:45.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc24e4a700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc180d4020 con 0x7fdc18046b70 2021-04-03T03:15:45.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdc18042300 0x7fdc18043920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fdc18045690 0x7fdc180d33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:55992/0 (socket says 172.21.2.111:55992) 2021-04-03T03:15:45.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1ffff700 1 -- 172.21.2.111:0/3781587648 learned_addr learned my addr 172.21.2.111:0/3781587648 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:45.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1f7fe700 1 --2- 172.21.2.111:0/3781587648 >> v2:172.21.2.115:3300/0 conn(0x7fdc18046b70 0x7fdc180d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1effd700 1 -- 172.21.2.111:0/3781587648 >> v2:172.21.2.115:3300/0 conn(0x7fdc18046b70 msgr2=0x7fdc180d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1effd700 1 --2- 172.21.2.111:0/3781587648 >> v2:172.21.2.115:3300/0 conn(0x7fdc18046b70 0x7fdc180d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1effd700 1 -- 172.21.2.111:0/3781587648 >> v1:172.21.2.111:6789/0 conn(0x7fdc18045690 legacy=0x7fdc180d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1e7fc700 1 -- 172.21.2.111:0/3781587648 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363997275 0 0) 0x7fdc180d4ef0 con 0x7fdc18045690 2021-04-03T03:15:45.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1effd700 1 -- 172.21.2.111:0/3781587648 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180d42c0 con 0x7fdc18042300 2021-04-03T03:15:45.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1effd700 1 --2- 172.21.2.111:0/3781587648 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18043920 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7fdc0c002f30 tx=0x7fdc0c006dd0).ready entity=mon.0 client_cookie=15cdfd003602bbaa server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.138+0000 7fdc1e7fc700 1 -- 172.21.2.111:0/3781587648 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc0c0097f0 con 0x7fdc18042300 2021-04-03T03:15:45.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1e7fc700 1 -- 172.21.2.111:0/3781587648 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c009950 con 0x7fdc18042300 2021-04-03T03:15:45.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1e7fc700 1 -- 172.21.2.111:0/3781587648 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc0c009cb0 con 0x7fdc18042300 2021-04-03T03:15:45.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/3781587648 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 msgr2=0x7fdc18043920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/3781587648 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18043920 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7fdc0c002f30 tx=0x7fdc0c006dd0).stop 2021-04-03T03:15:45.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/3781587648 shutdown_connections 2021-04-03T03:15:45.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/3781587648 >> v2:172.21.2.115:3300/0 conn(0x7fdc18046b70 0x7fdc180d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/3781587648 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18043920 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/3781587648 >> 172.21.2.111:0/3781587648 conn(0x7fdc1804a6e0 msgr2=0x7fdc1804aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/3781587648 shutdown_connections 2021-04-03T03:15:45.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/3781587648 wait complete. 2021-04-03T03:15:45.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 Processor -- start 2021-04-03T03:15:45.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- start start 2021-04-03T03:15:45.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc18045690 0x7fdc180d6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc18046b70 0x7fdc180a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180d8060 con 0x7fdc18042300 2021-04-03T03:15:45.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180da2c0 con 0x7fdc18045690 2021-04-03T03:15:45.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180a3e80 con 0x7fdc18046b70 2021-04-03T03:15:45.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18040e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55202/0 (socket says 172.21.2.111:55202) 2021-04-03T03:15:45.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 -- 172.21.2.111:0/1952023841 learned_addr learned my addr 172.21.2.111:0/1952023841 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:45.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1effd700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc18045690 0x7fdc180d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1ffff700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc18046b70 0x7fdc180a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 -- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc18046b70 msgr2=0x7fdc180a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc18046b70 0x7fdc180a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 -- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc18045690 msgr2=0x7fdc180d6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc18045690 0x7fdc180d6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 -- 172.21.2.111:0/1952023841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180d6860 con 0x7fdc18042300 2021-04-03T03:15:45.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1f7fe700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18040e60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fdc14003070 tx=0x7fdc14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1cff9700 1 -- 172.21.2.111:0/1952023841 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc14009790 con 0x7fdc18042300 2021-04-03T03:15:45.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1cff9700 1 -- 172.21.2.111:0/1952023841 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc140098f0 con 0x7fdc18042300 2021-04-03T03:15:45.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1cff9700 1 -- 172.21.2.111:0/1952023841 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc14009c50 con 0x7fdc18042300 2021-04-03T03:15:45.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc180d8260 con 0x7fdc18042300 2021-04-03T03:15:45.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc180d9200 con 0x7fdc18042300 2021-04-03T03:15:45.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc180da100 con 0x7fdc18042300 2021-04-03T03:15:45.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.142+0000 7fdc1cff9700 1 -- 172.21.2.111:0/1952023841 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdc14013480 con 0x7fdc18042300 2021-04-03T03:15:45.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.146+0000 7fdc1cff9700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc08056fb0 0x7fdc08059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.146+0000 7fdc1cff9700 1 -- 172.21.2.111:0/1952023841 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fdc14070930 con 0x7fdc18042300 2021-04-03T03:15:45.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.146+0000 7fdc1effd700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc08056fb0 0x7fdc08059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.146+0000 7fdc1effd700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc08056fb0 0x7fdc08059470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fdc0c01a000 tx=0x7fdc0c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.146+0000 7fdc1cff9700 1 -- 172.21.2.111:0/1952023841 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdc14045cc0 con 0x7fdc18042300 2021-04-03T03:15:45.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.270+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc180da480 con 0x7fdc18042300 2021-04-03T03:15:45.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc1cff9700 1 -- 172.21.2.111:0/1952023841 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fdc14009db0 con 0x7fdc18042300 2021-04-03T03:15:45.278 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:45.279 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":38,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3837436656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3837436656},{"type":"v1","addr":"172.21.2.115:6833","nonce":3837436656}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:44.064474+0000","modified":"2021-04-03T03:15:45.067347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/13282524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":13282524},{"type":"v1","addr":"172.21.2.111:6835","nonce":13282524}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-04-03T03:15:45.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc08056fb0 msgr2=0x7fdc08059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc08056fb0 0x7fdc08059470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fdc0c01a000 tx=0x7fdc0c015000).stop 2021-04-03T03:15:45.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 msgr2=0x7fdc18040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18040e60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fdc14003070 tx=0x7fdc14008040).stop 2021-04-03T03:15:45.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 shutdown_connections 2021-04-03T03:15:45.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdc18046b70 0x7fdc180a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdc18042300 0x7fdc18040e60 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdc08056fb0 0x7fdc08059470 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 --2- 172.21.2.111:0/1952023841 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdc18045690 0x7fdc180d6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 >> 172.21.2.111:0/1952023841 conn(0x7fdc1804a6e0 msgr2=0x7fdc18004410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 shutdown_connections 2021-04-03T03:15:45.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.274+0000 7fdc24e4a700 1 -- 172.21.2.111:0/1952023841 wait complete. 2021-04-03T03:15:45.282 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 38 2021-04-03T03:15:45.286 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:45.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 Processor -- start 2021-04-03T03:15:45.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- start start 2021-04-03T03:15:45.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ed00d7d90 0x7f7ed004b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed00d4550 con 0x7f7ed00456f0 2021-04-03T03:15:45.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed00d8300 con 0x7f7ed004ae10 2021-04-03T03:15:45.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00460a0 con 0x7f7ed00d7d90 2021-04-03T03:15:45.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed7fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7ed00456f0 0x7f7ed00467e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56004/0 (socket says 172.21.2.111:56004) 2021-04-03T03:15:45.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed7fff700 1 -- 172.21.2.111:0/3046240863 learned_addr learned my addr 172.21.2.111:0/3046240863 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:45.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed6ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ed00d7d90 0x7f7ed004b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed67fc700 1 -- 172.21.2.111:0/3046240863 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852205732 0 0) 0x7f7ed00d4550 con 0x7f7ed00456f0 2021-04-03T03:15:45.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed67fc700 1 -- 172.21.2.111:0/3046240863 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ebc003240 con 0x7f7ed00456f0 2021-04-03T03:15:45.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/3046240863 >> v1:172.21.2.115:6789/0 conn(0x7f7ed004ae10 legacy=0x7f7ed00d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/3046240863 >> v1:172.21.2.111:6789/0 conn(0x7f7ed00456f0 legacy=0x7f7ed00467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/3046240863 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0046340 con 0x7f7ed00d7d90 2021-04-03T03:15:45.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed6ffd700 1 --2- 172.21.2.111:0/3046240863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed00d7d90 0x7f7ed004b1f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f7ed00482e0 tx=0x7f7ec4006dd0).ready entity=mon.0 client_cookie=ffe9a3418817cfb8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed67fc700 1 -- 172.21.2.111:0/3046240863 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec4009770 con 0x7f7ed00d7d90 2021-04-03T03:15:45.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed67fc700 1 -- 172.21.2.111:0/3046240863 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec40098d0 con 0x7f7ed00d7d90 2021-04-03T03:15:45.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7ed67fc700 1 -- 172.21.2.111:0/3046240863 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec4009c70 con 0x7f7ed00d7d90 2021-04-03T03:15:45.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- 172.21.2.111:0/3046240863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed00d7d90 msgr2=0x7f7ed004b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3046240863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed00d7d90 0x7f7ed004b1f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f7ed00482e0 tx=0x7f7ec4006dd0).stop 2021-04-03T03:15:45.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- 172.21.2.111:0/3046240863 shutdown_connections 2021-04-03T03:15:45.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3046240863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed00d7d90 0x7f7ed004b1f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- 172.21.2.111:0/3046240863 >> 172.21.2.111:0/3046240863 conn(0x7f7ed0011530 msgr2=0x7f7ed004a700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- 172.21.2.111:0/3046240863 shutdown_connections 2021-04-03T03:15:45.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- 172.21.2.111:0/3046240863 wait complete. 2021-04-03T03:15:45.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 Processor -- start 2021-04-03T03:15:45.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- start start 2021-04-03T03:15:45.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00456f0 0x7f7ed00448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed004ae10 0x7f7ed00427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d7d90 0x7f7ed00d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00d06d0 con 0x7f7ed004ae10 2021-04-03T03:15:45.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00441a0 con 0x7f7ed00d7d90 2021-04-03T03:15:45.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.338+0000 7f7edcbed700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0044440 con 0x7f7ed00456f0 2021-04-03T03:15:45.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed004ae10 0x7f7ed00427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed004ae10 0x7f7ed00427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55218/0 (socket says 172.21.2.111:55218) 2021-04-03T03:15:45.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/3610147474 learned_addr learned my addr 172.21.2.111:0/3610147474 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:45.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/3610147474 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00456f0 msgr2=0x7f7ed00448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00456f0 0x7f7ed00448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/3610147474 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d7d90 msgr2=0x7f7ed00d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d7d90 0x7f7ed00d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/3610147474 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0041c70 con 0x7f7ed004ae10 2021-04-03T03:15:45.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed6ffd700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed004ae10 0x7f7ed00427c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f7ed00482e0 tx=0x7f7ec4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed4ff9700 1 -- 172.21.2.111:0/3610147474 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec401e460 con 0x7f7ed004ae10 2021-04-03T03:15:45.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed4ff9700 1 -- 172.21.2.111:0/3610147474 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec401e5c0 con 0x7f7ed004ae10 2021-04-03T03:15:45.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed0042cb0 con 0x7f7ed004ae10 2021-04-03T03:15:45.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed00d0420 con 0x7f7ed004ae10 2021-04-03T03:15:45.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed4ff9700 1 -- 172.21.2.111:0/3610147474 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec401e920 con 0x7f7ed004ae10 2021-04-03T03:15:45.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed0043e90 con 0x7f7ed004ae10 2021-04-03T03:15:45.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed4ff9700 1 -- 172.21.2.111:0/3610147474 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7ec4025030 con 0x7f7ed004ae10 2021-04-03T03:15:45.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed4ff9700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ec0057050 0x7f7ec0059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed4ff9700 1 -- 172.21.2.111:0/3610147474 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7ec40718a0 con 0x7f7ed004ae10 2021-04-03T03:15:45.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed77fe700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ec0057050 0x7f7ec0059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.342+0000 7f7ed77fe700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ec0057050 0x7f7ec0059510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f7ecc002f50 tx=0x7f7ecc006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.346+0000 7f7ed4ff9700 1 -- 172.21.2.111:0/3610147474 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7ec4046c90 con 0x7f7ed004ae10 2021-04-03T03:15:45.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed0044600 con 0x7f7ed004ae10 2021-04-03T03:15:45.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7ed4ff9700 1 -- 172.21.2.111:0/3610147474 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f7ec4023090 con 0x7f7ed004ae10 2021-04-03T03:15:45.444 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:45.444 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":93,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:45.068238+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:42.095114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:43.035163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:45.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ec0057050 msgr2=0x7f7ec0059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ec0057050 0x7f7ec0059510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f7ecc002f50 tx=0x7f7ecc006990).stop 2021-04-03T03:15:45.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed004ae10 msgr2=0x7f7ed00427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed004ae10 0x7f7ed00427c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f7ed00482e0 tx=0x7f7ec4015000).stop 2021-04-03T03:15:45.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 shutdown_connections 2021-04-03T03:15:45.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7ec0057050 0x7f7ec0059510 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d7d90 0x7f7ed00d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00456f0 0x7f7ed00448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 --2- 172.21.2.111:0/3610147474 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed004ae10 0x7f7ed00427c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 >> 172.21.2.111:0/3610147474 conn(0x7f7ed0011530 msgr2=0x7f7ed0049ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 shutdown_connections 2021-04-03T03:15:45.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.438+0000 7f7edcbed700 1 -- 172.21.2.111:0/3610147474 wait complete. 2021-04-03T03:15:45.451 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:15:45.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 Processor -- start 2021-04-03T03:15:45.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- start start 2021-04-03T03:15:45.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff3dc153440 0x7ff3dc080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc0b3ca0 con 0x7ff3dc0b38c0 2021-04-03T03:15:45.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc025b90 con 0x7ff3dc025480 2021-04-03T03:15:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc025e30 con 0x7ff3dc153440 2021-04-03T03:15:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e0a97700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff3dc0b38c0 0x7ff3dc080550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56016/0 (socket says 172.21.2.111:56016) 2021-04-03T03:15:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e0a97700 1 -- 172.21.2.111:0/2486381273 learned_addr learned my addr 172.21.2.111:0/2486381273 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3dbfff700 1 --2- 172.21.2.111:0/2486381273 >> v2:172.21.2.111:3300/0 conn(0x7ff3dc153440 0x7ff3dc080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3db7fe700 1 -- 172.21.2.111:0/2486381273 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493567953 0 0) 0x7ff3dc0b3ca0 con 0x7ff3dc0b38c0 2021-04-03T03:15:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3db7fe700 1 -- 172.21.2.111:0/2486381273 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c8003240 con 0x7ff3dc0b38c0 2021-04-03T03:15:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3dbfff700 1 -- 172.21.2.111:0/2486381273 >> v1:172.21.2.115:6789/0 conn(0x7ff3dc025480 legacy=0x7ff3dc0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3dbfff700 1 -- 172.21.2.111:0/2486381273 >> v1:172.21.2.111:6789/0 conn(0x7ff3dc0b38c0 legacy=0x7ff3dc080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3dbfff700 1 -- 172.21.2.111:0/2486381273 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc0260d0 con 0x7ff3dc153440 2021-04-03T03:15:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3dbfff700 1 --2- 172.21.2.111:0/2486381273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc153440 0x7ff3dc080c60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7ff3dc063a70 tx=0x7ff3cc006dd0).ready entity=mon.0 client_cookie=7fac3e400ba6cb5a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3db7fe700 1 -- 172.21.2.111:0/2486381273 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3cc009770 con 0x7ff3dc153440 2021-04-03T03:15:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3db7fe700 1 -- 172.21.2.111:0/2486381273 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc0098d0 con 0x7ff3dc153440 2021-04-03T03:15:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3db7fe700 1 -- 172.21.2.111:0/2486381273 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3cc009c70 con 0x7ff3dc153440 2021-04-03T03:15:45.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- 172.21.2.111:0/2486381273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc153440 msgr2=0x7ff3dc080c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 --2- 172.21.2.111:0/2486381273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc153440 0x7ff3dc080c60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7ff3dc063a70 tx=0x7ff3cc006dd0).stop 2021-04-03T03:15:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- 172.21.2.111:0/2486381273 shutdown_connections 2021-04-03T03:15:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 --2- 172.21.2.111:0/2486381273 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc153440 0x7ff3dc080c60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- 172.21.2.111:0/2486381273 >> 172.21.2.111:0/2486381273 conn(0x7ff3dc061610 msgr2=0x7ff3dc061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- 172.21.2.111:0/2486381273 shutdown_connections 2021-04-03T03:15:45.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.506+0000 7ff3e1298700 1 -- 172.21.2.111:0/2486381273 wait complete. 2021-04-03T03:15:45.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 Processor -- start 2021-04-03T03:15:45.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 -- start start 2021-04-03T03:15:45.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc025480 0x7ff3dc12dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3dc0b38c0 0x7ff3dc0acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3dc153440 0x7ff3dc0672a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc066100 con 0x7ff3dc025480 2021-04-03T03:15:45.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc12ebc0 con 0x7ff3dc153440 2021-04-03T03:15:45.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0ad720 con 0x7ff3dc0b38c0 2021-04-03T03:15:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc025480 0x7ff3dc12dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc025480 0x7ff3dc12dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55230/0 (socket says 172.21.2.111:55230) 2021-04-03T03:15:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 -- 172.21.2.111:0/1803927703 learned_addr learned my addr 172.21.2.111:0/1803927703 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 -- 172.21.2.111:0/1803927703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3dc0b38c0 msgr2=0x7ff3dc0acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3dc0b38c0 0x7ff3dc0acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 -- 172.21.2.111:0/1803927703 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3dc153440 msgr2=0x7ff3dc0672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3dc153440 0x7ff3dc0672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 -- 172.21.2.111:0/1803927703 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc064bc0 con 0x7ff3dc025480 2021-04-03T03:15:45.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e0a97700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc025480 0x7ff3dc12dbd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7ff3c40010d0 tx=0x7ff3c4007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3d97fa700 1 -- 172.21.2.111:0/1803927703 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3c4007320 con 0x7ff3dc025480 2021-04-03T03:15:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3d97fa700 1 -- 172.21.2.111:0/1803927703 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c4007480 con 0x7ff3dc025480 2021-04-03T03:15:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc0ad150 con 0x7ff3dc025480 2021-04-03T03:15:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3d97fa700 1 -- 172.21.2.111:0/1803927703 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3c40057e0 con 0x7ff3dc025480 2021-04-03T03:15:45.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc0640d0 con 0x7ff3dc025480 2021-04-03T03:15:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3dc057660 con 0x7ff3dc025480 2021-04-03T03:15:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3d97fa700 1 -- 172.21.2.111:0/1803927703 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff3c4010030 con 0x7ff3dc025480 2021-04-03T03:15:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3d97fa700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3c0057000 0x7ff3c00594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3d97fa700 1 -- 172.21.2.111:0/1803927703 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff3c406feb0 con 0x7ff3dc025480 2021-04-03T03:15:45.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3dbfff700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3c0057000 0x7ff3c00594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.510+0000 7ff3dbfff700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3c0057000 0x7ff3c00594c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7ff3dc063a70 tx=0x7ff3cc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.514+0000 7ff3d97fa700 1 -- 172.21.2.111:0/1803927703 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff3c4045190 con 0x7ff3dc025480 2021-04-03T03:15:45.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.638+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff3dc0aa6f0 con 0x7ff3dc025480 2021-04-03T03:15:45.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3d97fa700 1 -- 172.21.2.111:0/1803927703 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff3c40487b0 con 0x7ff3dc025480 2021-04-03T03:15:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3c0057000 msgr2=0x7ff3c00594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3c0057000 0x7ff3c00594c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7ff3dc063a70 tx=0x7ff3cc015000).stop 2021-04-03T03:15:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc025480 msgr2=0x7ff3dc12dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc025480 0x7ff3dc12dbd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7ff3c40010d0 tx=0x7ff3c4007ab0).stop 2021-04-03T03:15:45.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 shutdown_connections 2021-04-03T03:15:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3c0057000 0x7ff3c00594c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3dc153440 0x7ff3dc0672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3dc025480 0x7ff3dc12dbd0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 --2- 172.21.2.111:0/1803927703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3dc0b38c0 0x7ff3dc0acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 >> 172.21.2.111:0/1803927703 conn(0x7ff3dc061610 msgr2=0x7ff3dc0807c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 shutdown_connections 2021-04-03T03:15:45.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.646+0000 7ff3e1298700 1 -- 172.21.2.111:0/1803927703 wait complete. 2021-04-03T03:15:45.653 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:15:45.658 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:15:45.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02771ab700 1 Processor -- start 2021-04-03T03:15:45.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02771ab700 1 -- start start 2021-04-03T03:15:45.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02771ab700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0268046ce0 0x7f026803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02771ab700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f026803d350 0x7f0268041c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02771ab700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0268041240 con 0x7f0268042160 2021-04-03T03:15:45.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02771ab700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680f1850 con 0x7f026803d350 2021-04-03T03:15:45.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02771ab700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680470c0 con 0x7f0268046ce0 2021-04-03T03:15:45.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02761a9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f026803d350 0x7f0268041c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02761a9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f026803d350 0x7f0268041c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55242/0 (socket says 172.21.2.111:55242) 2021-04-03T03:15:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.710+0000 7f02761a9700 1 -- 172.21.2.111:0/2593039334 learned_addr learned my addr 172.21.2.111:0/2593039334 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 -- 172.21.2.111:0/2593039334 >> v2:172.21.2.115:3300/0 conn(0x7f0268046ce0 msgr2=0x7f026803f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 --2- 172.21.2.111:0/2593039334 >> v2:172.21.2.115:3300/0 conn(0x7f0268046ce0 0x7f026803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 -- 172.21.2.111:0/2593039334 >> v1:172.21.2.115:6789/0 conn(0x7f0268042160 legacy=0x7f02680f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 -- 172.21.2.111:0/2593039334 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680400d0 con 0x7f026803d350 2021-04-03T03:15:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 --2- 172.21.2.111:0/2593039334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f026803d350 0x7f0268041c70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f026800c2d0 tx=0x7f0264006e90).ready entity=mon.0 client_cookie=9b8463c933575fd0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02759a8700 1 -- 172.21.2.111:0/2593039334 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0264009930 con 0x7f026803d350 2021-04-03T03:15:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02759a8700 1 -- 172.21.2.111:0/2593039334 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264009a90 con 0x7f026803d350 2021-04-03T03:15:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02759a8700 1 -- 172.21.2.111:0/2593039334 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0264009e30 con 0x7f026803d350 2021-04-03T03:15:45.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/2593039334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f026803d350 msgr2=0x7f0268041c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 --2- 172.21.2.111:0/2593039334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f026803d350 0x7f0268041c70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f026800c2d0 tx=0x7f0264006e90).stop 2021-04-03T03:15:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/2593039334 shutdown_connections 2021-04-03T03:15:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 --2- 172.21.2.111:0/2593039334 >> v2:172.21.2.115:3300/0 conn(0x7f0268046ce0 0x7f026803f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 --2- 172.21.2.111:0/2593039334 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f026803d350 0x7f0268041c70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/2593039334 >> 172.21.2.111:0/2593039334 conn(0x7f026800d0c0 msgr2=0x7f02680462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/2593039334 shutdown_connections 2021-04-03T03:15:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/2593039334 wait complete. 2021-04-03T03:15:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 Processor -- start 2021-04-03T03:15:45.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- start start 2021-04-03T03:15:45.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f026803d350 0x7f02680a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0268042160 0x7f026803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0268046ce0 0x7f0268045b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680a0ff0 con 0x7f0268042160 2021-04-03T03:15:45.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680455a0 con 0x7f026803d350 2021-04-03T03:15:45.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680a1d80 con 0x7f0268046ce0 2021-04-03T03:15:45.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0268042160 0x7f026803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0268042160 0x7f026803b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55244/0 (socket says 172.21.2.111:55244) 2021-04-03T03:15:45.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 -- 172.21.2.111:0/1831781801 learned_addr learned my addr 172.21.2.111:0/1831781801 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:45.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 -- 172.21.2.111:0/1831781801 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0268046ce0 msgr2=0x7f0268045b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0268046ce0 0x7f0268045b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 -- 172.21.2.111:0/1831781801 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f026803d350 msgr2=0x7f02680a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f026803d350 0x7f02680a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 -- 172.21.2.111:0/1831781801 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680c89e0 con 0x7f0268042160 2021-04-03T03:15:45.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02761a9700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0268042160 0x7f026803b920 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f026800c2d0 tx=0x7f0264015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02637fe700 1 -- 172.21.2.111:0/1831781801 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0264011cc0 con 0x7f0268042160 2021-04-03T03:15:45.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02637fe700 1 -- 172.21.2.111:0/1831781801 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264011e20 con 0x7f0268042160 2021-04-03T03:15:45.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02680457d0 con 0x7f0268042160 2021-04-03T03:15:45.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02637fe700 1 -- 172.21.2.111:0/1831781801 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f026401e650 con 0x7f0268042160 2021-04-03T03:15:45.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026803be90 con 0x7f0268042160 2021-04-03T03:15:45.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.714+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0268004d90 con 0x7f0268042160 2021-04-03T03:15:45.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.718+0000 7f02637fe700 1 -- 172.21.2.111:0/1831781801 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0264025030 con 0x7f0268042160 2021-04-03T03:15:45.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.718+0000 7f02637fe700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0258056f20 0x7f02580593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.718+0000 7f02637fe700 1 -- 172.21.2.111:0/1831781801 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f02640716a0 con 0x7f0268042160 2021-04-03T03:15:45.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.718+0000 7f02769aa700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0258056f20 0x7f02580593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.718+0000 7f02769aa700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0258056f20 0x7f02580593e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f026c002f30 tx=0x7f026c0069c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.718+0000 7f02637fe700 1 -- 172.21.2.111:0/1831781801 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0264046980 con 0x7f0268042160 2021-04-03T03:15:45.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.846+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f02680083c0 con 0x7f0268042160 2021-04-03T03:15:45.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02637fe700 1 -- 172.21.2.111:0/1831781801 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f02640230e0 con 0x7f0268042160 2021-04-03T03:15:45.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0258056f20 msgr2=0x7f02580593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0258056f20 0x7f02580593e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f026c002f30 tx=0x7f026c0069c0).stop 2021-04-03T03:15:45.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0268042160 msgr2=0x7f026803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0268042160 0x7f026803b920 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f026800c2d0 tx=0x7f0264015000).stop 2021-04-03T03:15:45.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 shutdown_connections 2021-04-03T03:15:45.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0268046ce0 0x7f0268045b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0258056f20 0x7f02580593e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f026803d350 0x7f02680a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:45.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 --2- 172.21.2.111:0/1831781801 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0268042160 0x7f026803b920 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 >> 172.21.2.111:0/1831781801 conn(0x7f026800d0c0 msgr2=0x7f0268005d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 shutdown_connections 2021-04-03T03:15:45.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.850+0000 7f02771ab700 1 -- 172.21.2.111:0/1831781801 wait complete. 2021-04-03T03:15:45.860 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:15:45.864 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:15:45.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7e7386700 1 Processor -- start 2021-04-03T03:15:45.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7e7386700 1 -- start start 2021-04-03T03:15:45.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7e7386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe7e80b38c0 0x7fe7e8025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7e7386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8153440 0x7fe7e8080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7e7386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e80b3ca0 con 0x7fe7e8080a40 2021-04-03T03:15:45.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7e7386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e80b0300 con 0x7fe7e80b38c0 2021-04-03T03:15:45.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7e7386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e8025970 con 0x7fe7e8153440 2021-04-03T03:15:45.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7ec805700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe7e8080a40 0x7fe7e814feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56048/0 (socket says 172.21.2.111:56048) 2021-04-03T03:15:45.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.922+0000 7fe7ec805700 1 -- 172.21.2.111:0/510990264 learned_addr learned my addr 172.21.2.111:0/510990264 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:45.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103513419 0 0) 0x7fe7e80b3ca0 con 0x7fe7e8080a40 2021-04-03T03:15:45.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7d4003240 con 0x7fe7e8080a40 2021-04-03T03:15:45.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6b85700 1 --2- 172.21.2.111:0/510990264 >> v2:172.21.2.115:3300/0 conn(0x7fe7e80b38c0 0x7fe7e8025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8153440 0x7fe7e8080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2967344002 0 0) 0x7fe7d4003240 con 0x7fe7e8080a40 2021-04-03T03:15:45.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8153440 msgr2=0x7fe7e8080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8153440 0x7fe7e8080550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 >> v2:172.21.2.115:3300/0 conn(0x7fe7e80b38c0 msgr2=0x7fe7e8025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 --2- 172.21.2.111:0/510990264 >> v2:172.21.2.115:3300/0 conn(0x7fe7e80b38c0 0x7fe7e8025480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8025b90 con 0x7fe7e8080a40 2021-04-03T03:15:45.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe7e0001a50 con 0x7fe7e8080a40 2021-04-03T03:15:45.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 >> v1:172.21.2.111:6789/0 conn(0x7fe7e8080a40 legacy=0x7fe7e814feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7d40037b0 0x7fe7d4003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7d40044e0 0x7fe7d4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7d4004e50 0x7fe7d404f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80b0300 con 0x7fe7d40044e0 2021-04-03T03:15:45.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e8025970 con 0x7fe7d4004e50 2021-04-03T03:15:45.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4004240 con 0x7fe7d40037b0 2021-04-03T03:15:45.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8153440 0x7fe7e8080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:45.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7d40044e0 0x7fe7d4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6b85700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7d40037b0 0x7fe7d4003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7d4004e50 0x7fe7d404f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 -- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7d40037b0 msgr2=0x7fe7d4003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7d40037b0 0x7fe7d4003bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 -- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7d4004e50 msgr2=0x7fe7d404f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:45.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7d4004e50 0x7fe7d404f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 -- 172.21.2.111:0/510990264 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8025b90 con 0x7fe7d40044e0 2021-04-03T03:15:45.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7d4004e50 0x7fe7d404f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:45.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7d40044e0 0x7fe7d4004960 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fe7d8000b20 tx=0x7fe7d8008040).ready entity=mon.0 client_cookie=fe72b7b9c13120e7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7d8016070 con 0x7fe7d40044e0 2021-04-03T03:15:45.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d800ba00 con 0x7fe7d40044e0 2021-04-03T03:15:45.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e5b83700 1 -- 172.21.2.111:0/510990264 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7d800dea0 con 0x7fe7d40044e0 2021-04-03T03:15:45.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- 172.21.2.111:0/510990264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7d40044e0 msgr2=0x7fe7d4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 --2- 172.21.2.111:0/510990264 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7d40044e0 0x7fe7d4004960 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fe7d8000b20 tx=0x7fe7d8008040).stop 2021-04-03T03:15:45.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 -- 172.21.2.111:0/510990264 reap_dead start 2021-04-03T03:15:45.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- 172.21.2.111:0/510990264 shutdown_connections 2021-04-03T03:15:45.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- 172.21.2.111:0/510990264 >> 172.21.2.111:0/510990264 conn(0x7fe7e8061610 msgr2=0x7fe7e8061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:45.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- 172.21.2.111:0/510990264 shutdown_connections 2021-04-03T03:15:45.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- 172.21.2.111:0/510990264 wait complete. 2021-04-03T03:15:45.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 Processor -- start 2021-04-03T03:15:45.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- start start 2021-04-03T03:15:45.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8063f40 0x7fe7e8064320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7e80acee0 0x7fe7e80a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7e812dac0 0x7fe7e80aa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80abdf0 con 0x7fe7e812dac0 2021-04-03T03:15:45.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80ac020 con 0x7fe7e80acee0 2021-04-03T03:15:45.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80ac2c0 con 0x7fe7e8063f40 2021-04-03T03:15:45.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7e80acee0 0x7fe7e80a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7e812dac0 0x7fe7e80aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7e80acee0 0x7fe7e80a9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:44972/0 (socket says 172.21.2.111:44972) 2021-04-03T03:15:45.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6384700 1 -- 172.21.2.111:0/1638589710 learned_addr learned my addr 172.21.2.111:0/1638589710 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:45.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e6b85700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8063f40 0x7fe7e8064320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 -- 172.21.2.111:0/1638589710 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8063f40 msgr2=0x7fe7e8064320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8063f40 0x7fe7e8064320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 -- 172.21.2.111:0/1638589710 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7e80acee0 msgr2=0x7fe7e80a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:45.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7e80acee0 0x7fe7e80a9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:45.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 -- 172.21.2.111:0/1638589710 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80b1dd0 con 0x7fe7e812dac0 2021-04-03T03:15:45.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7ec805700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7e812dac0 0x7fe7e80aa980 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fe7e00011d0 tx=0x7fe7e0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7df7fe700 1 -- 172.21.2.111:0/1638589710 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7e0004b50 con 0x7fe7e812dac0 2021-04-03T03:15:45.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7df7fe700 1 -- 172.21.2.111:0/1638589710 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e0004cb0 con 0x7fe7e812dac0 2021-04-03T03:15:45.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7df7fe700 1 -- 172.21.2.111:0/1638589710 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7e0007360 con 0x7fe7e812dac0 2021-04-03T03:15:45.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e80aa4c0 con 0x7fe7e812dac0 2021-04-03T03:15:45.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.926+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e80ad890 con 0x7fe7e812dac0 2021-04-03T03:15:45.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.930+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e80ab190 con 0x7fe7e812dac0 2021-04-03T03:15:45.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.930+0000 7fe7df7fe700 1 -- 172.21.2.111:0/1638589710 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe7e001e070 con 0x7fe7e812dac0 2021-04-03T03:15:45.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.930+0000 7fe7df7fe700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7cc056fe0 0x7fe7cc0594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:45.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.930+0000 7fe7df7fe700 1 -- 172.21.2.111:0/1638589710 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe7e006f710 con 0x7fe7e812dac0 2021-04-03T03:15:45.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.930+0000 7fe7e6b85700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7cc056fe0 0x7fe7cc0594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:45.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.930+0000 7fe7e6b85700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7cc056fe0 0x7fe7cc0594a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fe7d0008000 tx=0x7fe7d0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:45.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:45.930+0000 7fe7df7fe700 1 -- 172.21.2.111:0/1638589710 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe7e00449f0 con 0x7fe7e812dac0 2021-04-03T03:15:46.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.058+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe7e80ac480 con 0x7fe7e812dac0 2021-04-03T03:15:46.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.062+0000 7fe7df7fe700 1 -- 172.21.2.111:0/1638589710 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe7e0048010 con 0x7fe7e812dac0 2021-04-03T03:15:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7cc056fe0 msgr2=0x7fe7cc0594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7cc056fe0 0x7fe7cc0594a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fe7d0008000 tx=0x7fe7d0008040).stop 2021-04-03T03:15:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7e812dac0 msgr2=0x7fe7e80aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7e812dac0 0x7fe7e80aa980 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fe7e00011d0 tx=0x7fe7e0009040).stop 2021-04-03T03:15:46.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 shutdown_connections 2021-04-03T03:15:46.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7e812dac0 0x7fe7e80aa980 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe7e8063f40 0x7fe7e8064320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe7cc056fe0 0x7fe7cc0594a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 --2- 172.21.2.111:0/1638589710 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7e80acee0 0x7fe7e80a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 >> 172.21.2.111:0/1638589710 conn(0x7fe7e8061610 msgr2=0x7fe7e812e260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 shutdown_connections 2021-04-03T03:15:46.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.066+0000 7fe7e7386700 1 -- 172.21.2.111:0/1638589710 wait complete. 2021-04-03T03:15:46.074 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:15:46.079 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:46.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 Processor -- start 2021-04-03T03:15:46.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- start start 2021-04-03T03:15:46.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 0x7f99e00dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f99e00456f0 0x7f99e00dc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f99e00059a0 0x7f99e0005dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00db880 con 0x7f99e00456f0 2021-04-03T03:15:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e004b510 con 0x7f99e00059a0 2021-04-03T03:15:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0009890 con 0x7f99e004b130 2021-04-03T03:15:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f99e00456f0 0x7f99e00dc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f99e00456f0 0x7f99e00dc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55272/0 (socket says 172.21.2.111:55272) 2021-04-03T03:15:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/3782817777 learned_addr learned my addr 172.21.2.111:0/3782817777 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/3782817777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 msgr2=0x7f99e00dbcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- 172.21.2.111:0/3782817777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 0x7f99e00dbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/3782817777 >> v2:172.21.2.115:3300/0 conn(0x7f99e00059a0 msgr2=0x7f99e0005dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- 172.21.2.111:0/3782817777 >> v2:172.21.2.115:3300/0 conn(0x7f99e00059a0 0x7f99e0005dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/3782817777 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009b30 con 0x7f99e00456f0 2021-04-03T03:15:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- 172.21.2.111:0/3782817777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e00dc1a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f99dc002f30 tx=0x7f99dc006e90).ready entity=mon.0 client_cookie=778320a9259ff055 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e6ffd700 1 -- 172.21.2.111:0/3782817777 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc009920 con 0x7f99e00456f0 2021-04-03T03:15:46.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e6ffd700 1 -- 172.21.2.111:0/3782817777 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc009a80 con 0x7f99e00456f0 2021-04-03T03:15:46.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e6ffd700 1 -- 172.21.2.111:0/3782817777 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc009de0 con 0x7f99e00456f0 2021-04-03T03:15:46.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- 172.21.2.111:0/3782817777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 msgr2=0x7f99e00dc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- 172.21.2.111:0/3782817777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e00dc1a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f99dc002f30 tx=0x7f99dc006e90).stop 2021-04-03T03:15:46.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- 172.21.2.111:0/3782817777 shutdown_connections 2021-04-03T03:15:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- 172.21.2.111:0/3782817777 >> v2:172.21.2.115:3300/0 conn(0x7f99e00059a0 0x7f99e0005dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- 172.21.2.111:0/3782817777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 0x7f99e00dbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- 172.21.2.111:0/3782817777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e00dc1a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- 172.21.2.111:0/3782817777 >> 172.21.2.111:0/3782817777 conn(0x7f99e0011530 msgr2=0x7f99e004a700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- 172.21.2.111:0/3782817777 shutdown_connections 2021-04-03T03:15:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- 172.21.2.111:0/3782817777 wait complete. 2021-04-03T03:15:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 Processor -- start 2021-04-03T03:15:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- start start 2021-04-03T03:15:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99e00059a0 0x7f99e00e7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e000b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 0x7f99e00d3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00df3d0 con 0x7f99e00456f0 2021-04-03T03:15:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00debf0 con 0x7f99e00059a0 2021-04-03T03:15:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99ed797700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00dee90 con 0x7f99e004b130 2021-04-03T03:15:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e000b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e000b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55278/0 (socket says 172.21.2.111:55278) 2021-04-03T03:15:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/2518489504 learned_addr learned my addr 172.21.2.111:0/2518489504 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/2518489504 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 msgr2=0x7f99e00d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 0x7f99e00d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/2518489504 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99e00059a0 msgr2=0x7f99e00e7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99e00059a0 0x7f99e00e7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 -- 172.21.2.111:0/2518489504 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00df130 con 0x7f99e00456f0 2021-04-03T03:15:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.130+0000 7f99e77fe700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e000b550 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f99dc002f30 tx=0x7f99dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e57fa700 1 -- 172.21.2.111:0/2518489504 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc011db0 con 0x7f99e00456f0 2021-04-03T03:15:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e57fa700 1 -- 172.21.2.111:0/2518489504 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc01e460 con 0x7f99e00456f0 2021-04-03T03:15:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e000ce90 con 0x7f99e00456f0 2021-04-03T03:15:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e000c040 con 0x7f99e00456f0 2021-04-03T03:15:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e57fa700 1 -- 172.21.2.111:0/2518489504 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc01e7c0 con 0x7f99e00456f0 2021-04-03T03:15:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e0004da0 con 0x7f99e00456f0 2021-04-03T03:15:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e57fa700 1 -- 172.21.2.111:0/2518489504 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f99dc025030 con 0x7f99e00456f0 2021-04-03T03:15:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e57fa700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f99c8056fb0 0x7f99c8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e57fa700 1 -- 172.21.2.111:0/2518489504 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f99dc071770 con 0x7f99e00456f0 2021-04-03T03:15:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e7fff700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f99c8056fb0 0x7f99c8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e7fff700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f99c8056fb0 0x7f99c8059470 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f99d8002eb0 tx=0x7f99d80068f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.134+0000 7f99e57fa700 1 -- 172.21.2.111:0/2518489504 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f99dc046a50 con 0x7f99e00456f0 2021-04-03T03:15:46.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99e0046160 con 0x7f99e00456f0 2021-04-03T03:15:46.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99e57fa700 1 -- 172.21.2.111:0/2518489504 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f99dc023020 con 0x7f99e00456f0 2021-04-03T03:15:46.267 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:46.268 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":38,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3837436656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3837436656},{"type":"v1","addr":"172.21.2.115:6833","nonce":3837436656}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:44.064474+0000","modified":"2021-04-03T03:15:45.067347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/13282524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":13282524},{"type":"v1","addr":"172.21.2.111:6835","nonce":13282524}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-04-03T03:15:46.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f99c8056fb0 msgr2=0x7f99c8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f99c8056fb0 0x7f99c8059470 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f99d8002eb0 tx=0x7f99d80068f0).stop 2021-04-03T03:15:46.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 msgr2=0x7f99e000b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e000b550 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f99dc002f30 tx=0x7f99dc015000).stop 2021-04-03T03:15:46.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 shutdown_connections 2021-04-03T03:15:46.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f99e004b130 0x7f99e00d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f99c8056fb0 0x7f99c8059470 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f99e00059a0 0x7f99e00e7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 --2- 172.21.2.111:0/2518489504 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f99e00456f0 0x7f99e000b550 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 >> 172.21.2.111:0/2518489504 conn(0x7f99e0011530 msgr2=0x7f99e00d7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 shutdown_connections 2021-04-03T03:15:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.262+0000 7f99ed797700 1 -- 172.21.2.111:0/2518489504 wait complete. 2021-04-03T03:15:46.272 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 38 2021-04-03T03:15:46.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:15:44.064474+0000', 'modified': '2021-04-03T03:15:45.067347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6879}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6879': {'gid': 6879, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/13282524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 13282524}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 13282524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:15:46.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:15:46.477 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:15:46.553 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6879,"want_state":"up:active","state":"up:active","rank_uptime":2.4815288930000001,"mdsmap_epoch":38,"osdmap_epoch":93,"osdmap_epoch_barrier":92,"uptime":6.6307473610000001} 2021-04-03T03:15:46.554 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6879, "mdsmap_epoch": 38, "osdmap_epoch": 93, "osdmap_epoch_barrier": 92, "rank_uptime": 2.481528893, "state": "up:active", "uptime": 6.630747361, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:15:46.554 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:46.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 Processor -- start 2021-04-03T03:15:46.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- start start 2021-04-03T03:15:46.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8a200603f0 0x7f8a200b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a2014b700 0x7f8a200b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a200607d0 con 0x7f8a200b12b0 2021-04-03T03:15:46.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a200af400 con 0x7f8a200603f0 2021-04-03T03:15:46.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200af6a0 con 0x7f8a2014b700 2021-04-03T03:15:46.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8a200603f0 0x7f8a200b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8a200603f0 0x7f8a200b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55286/0 (socket says 172.21.2.111:55286) 2021-04-03T03:15:46.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 -- 172.21.2.111:0/2328440446 learned_addr learned my addr 172.21.2.111:0/2328440446 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 -- 172.21.2.111:0/2328440446 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a2014b700 msgr2=0x7f8a200b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:46.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 --2- 172.21.2.111:0/2328440446 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a2014b700 0x7f8a200b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 -- 172.21.2.111:0/2328440446 >> v1:172.21.2.115:6789/0 conn(0x7f8a200b12b0 legacy=0x7f8a201465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:46.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 -- 172.21.2.111:0/2328440446 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a200afb40 con 0x7f8a200603f0 2021-04-03T03:15:46.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a25dad700 1 --2- 172.21.2.111:0/2328440446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200603f0 0x7f8a200b0090 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f8a200af200 tx=0x7f8a14006e90).ready entity=mon.0 client_cookie=d2032b78542ca23e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a24dab700 1 -- 172.21.2.111:0/2328440446 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a140098e0 con 0x7f8a200603f0 2021-04-03T03:15:46.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a24dab700 1 -- 172.21.2.111:0/2328440446 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14009a40 con 0x7f8a200603f0 2021-04-03T03:15:46.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a24dab700 1 -- 172.21.2.111:0/2328440446 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a14009de0 con 0x7f8a200603f0 2021-04-03T03:15:46.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- 172.21.2.111:0/2328440446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200603f0 msgr2=0x7f8a200b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2328440446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200603f0 0x7f8a200b0090 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f8a200af200 tx=0x7f8a14006e90).stop 2021-04-03T03:15:46.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- 172.21.2.111:0/2328440446 shutdown_connections 2021-04-03T03:15:46.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2328440446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200603f0 0x7f8a200b0090 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2328440446 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a2014b700 0x7f8a200b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- 172.21.2.111:0/2328440446 >> 172.21.2.111:0/2328440446 conn(0x7f8a200b61c0 msgr2=0x7f8a2005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- 172.21.2.111:0/2328440446 shutdown_connections 2021-04-03T03:15:46.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.606+0000 7f8a26daf700 1 -- 172.21.2.111:0/2328440446 wait complete. 2021-04-03T03:15:46.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 Processor -- start 2021-04-03T03:15:46.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 -- start start 2021-04-03T03:15:46.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a200603f0 0x7f8a2014e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200b12b0 0x7f8a20064200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a2014b700 0x7f8a200646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a20062910 con 0x7f8a200b12b0 2021-04-03T03:15:46.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a20062b40 con 0x7f8a2014b700 2021-04-03T03:15:46.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a20062de0 con 0x7f8a200603f0 2021-04-03T03:15:46.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200b12b0 0x7f8a20064200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200b12b0 0x7f8a20064200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55294/0 (socket says 172.21.2.111:55294) 2021-04-03T03:15:46.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 -- 172.21.2.111:0/2410635460 learned_addr learned my addr 172.21.2.111:0/2410635460 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 -- 172.21.2.111:0/2410635460 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a200603f0 msgr2=0x7f8a2014e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a200603f0 0x7f8a2014e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 -- 172.21.2.111:0/2410635460 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a2014b700 msgr2=0x7f8a200646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a2014b700 0x7f8a200646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 -- 172.21.2.111:0/2410635460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a20063080 con 0x7f8a200b12b0 2021-04-03T03:15:46.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a255ac700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200b12b0 0x7f8a20064200 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f8a1c002f30 tx=0x7f8a1c006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a12ffd700 1 -- 172.21.2.111:0/2410635460 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a1c00d7c0 con 0x7f8a200b12b0 2021-04-03T03:15:46.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a12ffd700 1 -- 172.21.2.111:0/2410635460 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c00d920 con 0x7f8a200b12b0 2021-04-03T03:15:46.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a20159ae0 con 0x7f8a200b12b0 2021-04-03T03:15:46.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a12ffd700 1 -- 172.21.2.111:0/2410635460 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a1c00dc80 con 0x7f8a200b12b0 2021-04-03T03:15:46.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a200ab810 con 0x7f8a200b12b0 2021-04-03T03:15:46.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a200ab230 con 0x7f8a200b12b0 2021-04-03T03:15:46.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a12ffd700 1 -- 172.21.2.111:0/2410635460 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8a1c0103c0 con 0x7f8a200b12b0 2021-04-03T03:15:46.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a12ffd700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a00056ef0 0x7f8a000593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a12ffd700 1 -- 172.21.2.111:0/2410635460 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8a1c06fbe0 con 0x7f8a200b12b0 2021-04-03T03:15:46.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a25dad700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a00056ef0 0x7f8a000593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.610+0000 7f8a25dad700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a00056ef0 0x7f8a000593b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8a200af200 tx=0x7f8a14015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.614+0000 7f8a12ffd700 1 -- 172.21.2.111:0/2410635460 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8a1c044e60 con 0x7f8a200b12b0 2021-04-03T03:15:46.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.706+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a20063210 con 0x7f8a200b12b0 2021-04-03T03:15:46.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.706+0000 7f8a12ffd700 1 -- 172.21.2.111:0/2410635460 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f8a1c048480 con 0x7f8a200b12b0 2021-04-03T03:15:46.713 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:46.714 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":93,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:45.068238+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:42.095114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:43.035163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:46.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a00056ef0 msgr2=0x7f8a000593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a00056ef0 0x7f8a000593b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8a200af200 tx=0x7f8a14015000).stop 2021-04-03T03:15:46.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200b12b0 msgr2=0x7f8a20064200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200b12b0 0x7f8a20064200 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f8a1c002f30 tx=0x7f8a1c006e30).stop 2021-04-03T03:15:46.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 shutdown_connections 2021-04-03T03:15:46.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a00056ef0 0x7f8a000593b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a2014b700 0x7f8a200646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a200603f0 0x7f8a2014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 --2- 172.21.2.111:0/2410635460 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a200b12b0 0x7f8a20064200 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 >> 172.21.2.111:0/2410635460 conn(0x7f8a200b61c0 msgr2=0x7f8a200b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 shutdown_connections 2021-04-03T03:15:46.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.710+0000 7f8a26daf700 1 -- 172.21.2.111:0/2410635460 wait complete. 2021-04-03T03:15:46.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:15:42.095114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:15:43.035163+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:15:46.722 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:46.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 Processor -- start 2021-04-03T03:15:46.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 -- start start 2021-04-03T03:15:46.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 0x7f1f680cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1f68043a00 0x7f1f680d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1f6803c670 0x7f1f6803cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f680481f0 con 0x7f1f68043a00 2021-04-03T03:15:46.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f68043760 con 0x7f1f6803c670 2021-04-03T03:15:46.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f7505f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6803b1b0 con 0x7f1f68047e10 2021-04-03T03:15:46.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1f68043a00 0x7f1f680d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1f68043a00 0x7f1f680d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55302/0 (socket says 172.21.2.111:55302) 2021-04-03T03:15:46.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 -- 172.21.2.111:0/3206973787 learned_addr learned my addr 172.21.2.111:0/3206973787 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 -- 172.21.2.111:0/3206973787 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 msgr2=0x7f1f680cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:46.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 --2- 172.21.2.111:0/3206973787 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 0x7f1f680cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 -- 172.21.2.111:0/3206973787 >> v2:172.21.2.115:3300/0 conn(0x7f1f6803c670 msgr2=0x7f1f6803cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:46.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 --2- 172.21.2.111:0/3206973787 >> v2:172.21.2.115:3300/0 conn(0x7f1f6803c670 0x7f1f6803cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 -- 172.21.2.111:0/3206973787 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680d41c0 con 0x7f1f68043a00 2021-04-03T03:15:46.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6f7fe700 1 --2- 172.21.2.111:0/3206973787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f68043a00 0x7f1f680d03d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f1f60002f30 tx=0x7f1f60006e90).ready entity=mon.0 client_cookie=25be4176c47f39c4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6effd700 1 -- 172.21.2.111:0/3206973787 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f60009920 con 0x7f1f68043a00 2021-04-03T03:15:46.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6effd700 1 -- 172.21.2.111:0/3206973787 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60009a80 con 0x7f1f68043a00 2021-04-03T03:15:46.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.774+0000 7f1f6effd700 1 -- 172.21.2.111:0/3206973787 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f60009de0 con 0x7f1f68043a00 2021-04-03T03:15:46.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/3206973787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f68043a00 msgr2=0x7f1f680d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 --2- 172.21.2.111:0/3206973787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f68043a00 0x7f1f680d03d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f1f60002f30 tx=0x7f1f60006e90).stop 2021-04-03T03:15:46.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/3206973787 shutdown_connections 2021-04-03T03:15:46.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 --2- 172.21.2.111:0/3206973787 >> v2:172.21.2.115:3300/0 conn(0x7f1f6803c670 0x7f1f6803cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 --2- 172.21.2.111:0/3206973787 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 0x7f1f680cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 --2- 172.21.2.111:0/3206973787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f68043a00 0x7f1f680d03d0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/3206973787 >> 172.21.2.111:0/3206973787 conn(0x7f1f6800e220 msgr2=0x7f1f680473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/3206973787 shutdown_connections 2021-04-03T03:15:46.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/3206973787 wait complete. 2021-04-03T03:15:46.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 Processor -- start 2021-04-03T03:15:46.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- start start 2021-04-03T03:15:46.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f6803c670 0x7f1f68007de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1f68043a00 0x7f1f680d3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 0x7f1f68040d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f68040670 con 0x7f1f6803c670 2021-04-03T03:15:46.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680408d0 con 0x7f1f68043a00 2021-04-03T03:15:46.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680c37f0 con 0x7f1f68047e10 2021-04-03T03:15:46.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f6803c670 0x7f1f68007de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f6803c670 0x7f1f68007de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55306/0 (socket says 172.21.2.111:55306) 2021-04-03T03:15:46.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 -- 172.21.2.111:0/199271021 learned_addr learned my addr 172.21.2.111:0/199271021 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 -- 172.21.2.111:0/199271021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 msgr2=0x7f1f68040d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 0x7f1f68040d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 -- 172.21.2.111:0/199271021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1f68043a00 msgr2=0x7f1f680d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1f68043a00 0x7f1f680d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 -- 172.21.2.111:0/199271021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680c34e0 con 0x7f1f6803c670 2021-04-03T03:15:46.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6ffff700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f6803c670 0x7f1f68007de0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f1f64002f30 tx=0x7f1f64006d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6d7fa700 1 -- 172.21.2.111:0/199271021 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f6400d4a0 con 0x7f1f6803c670 2021-04-03T03:15:46.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6d7fa700 1 -- 172.21.2.111:0/199271021 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f64000e90 con 0x7f1f6803c670 2021-04-03T03:15:46.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6803d280 con 0x7f1f6803c670 2021-04-03T03:15:46.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6d7fa700 1 -- 172.21.2.111:0/199271021 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f6400d7f0 con 0x7f1f6803c670 2021-04-03T03:15:46.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f680c2df0 con 0x7f1f6803c670 2021-04-03T03:15:46.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f68040bd0 con 0x7f1f6803c670 2021-04-03T03:15:46.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6d7fa700 1 -- 172.21.2.111:0/199271021 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1f6400f030 con 0x7f1f6803c670 2021-04-03T03:15:46.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6d7fa700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1f50056ef0 0x7f1f500593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6d7fa700 1 -- 172.21.2.111:0/199271021 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1f6406fef0 con 0x7f1f6803c670 2021-04-03T03:15:46.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6f7fe700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1f50056ef0 0x7f1f500593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.778+0000 7f1f6f7fe700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1f50056ef0 0x7f1f500593b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f1f60002f30 tx=0x7f1f60015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.782+0000 7f1f6d7fa700 1 -- 172.21.2.111:0/199271021 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1f640451d0 con 0x7f1f6803c670 2021-04-03T03:15:46.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1f68004db0 con 0x7f1f6803c670 2021-04-03T03:15:46.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f6d7fa700 1 -- 172.21.2.111:0/199271021 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v93) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1f640487f0 con 0x7f1f6803c670 2021-04-03T03:15:46.880 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:46.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1f50056ef0 msgr2=0x7f1f500593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1f50056ef0 0x7f1f500593b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f1f60002f30 tx=0x7f1f60015000).stop 2021-04-03T03:15:46.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f6803c670 msgr2=0x7f1f68007de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f6803c670 0x7f1f68007de0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f1f64002f30 tx=0x7f1f64006d80).stop 2021-04-03T03:15:46.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 shutdown_connections 2021-04-03T03:15:46.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1f68047e10 0x7f1f68040d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1f6803c670 0x7f1f68007de0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1f50056ef0 0x7f1f500593b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 --2- 172.21.2.111:0/199271021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1f68043a00 0x7f1f680d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 >> 172.21.2.111:0/199271021 conn(0x7f1f6800e220 msgr2=0x7f1f680cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 shutdown_connections 2021-04-03T03:15:46.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.874+0000 7f1f7505f700 1 -- 172.21.2.111:0/199271021 wait complete. 2021-04-03T03:15:46.888 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:46.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 Processor -- start 2021-04-03T03:15:46.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- start start 2021-04-03T03:15:46.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ffb040add50 0x7ffb04147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ffb04144ef0 0x7ffb04147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb040607d0 con 0x7ffb040603f0 2021-04-03T03:15:46.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0414b090 con 0x7ffb040add50 2021-04-03T03:15:46.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0414b330 con 0x7ffb04144ef0 2021-04-03T03:15:46.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ffb040add50 0x7ffb04147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ffb040add50 0x7ffb04147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55316/0 (socket says 172.21.2.111:55316) 2021-04-03T03:15:46.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 -- 172.21.2.111:0/1802036843 learned_addr learned my addr 172.21.2.111:0/1802036843 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 -- 172.21.2.111:0/1802036843 >> v2:172.21.2.115:3300/0 conn(0x7ffb04144ef0 msgr2=0x7ffb04147c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:46.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 --2- 172.21.2.111:0/1802036843 >> v2:172.21.2.115:3300/0 conn(0x7ffb04144ef0 0x7ffb04147c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 -- 172.21.2.111:0/1802036843 >> v1:172.21.2.115:6789/0 conn(0x7ffb040603f0 legacy=0x7ffb041447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:46.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 -- 172.21.2.111:0/1802036843 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0414b5d0 con 0x7ffb040add50 2021-04-03T03:15:46.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb08a5a700 1 --2- 172.21.2.111:0/1802036843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040add50 0x7ffb04147740 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7ffb040ae520 tx=0x7ffaf8006e90).ready entity=mon.0 client_cookie=4183278d0d27df8e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffaf7fff700 1 -- 172.21.2.111:0/1802036843 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffaf8009930 con 0x7ffb040add50 2021-04-03T03:15:46.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffaf7fff700 1 -- 172.21.2.111:0/1802036843 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8009a90 con 0x7ffb040add50 2021-04-03T03:15:46.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffaf7fff700 1 -- 172.21.2.111:0/1802036843 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffaf8009e30 con 0x7ffb040add50 2021-04-03T03:15:46.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- 172.21.2.111:0/1802036843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040add50 msgr2=0x7ffb04147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/1802036843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040add50 0x7ffb04147740 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7ffb040ae520 tx=0x7ffaf8006e90).stop 2021-04-03T03:15:46.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- 172.21.2.111:0/1802036843 shutdown_connections 2021-04-03T03:15:46.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/1802036843 >> v2:172.21.2.115:3300/0 conn(0x7ffb04144ef0 0x7ffb04147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/1802036843 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040add50 0x7ffb04147740 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:46.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- 172.21.2.111:0/1802036843 >> 172.21.2.111:0/1802036843 conn(0x7ffb040b61c0 msgr2=0x7ffb0405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:46.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- 172.21.2.111:0/1802036843 shutdown_connections 2021-04-03T03:15:46.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- 172.21.2.111:0/1802036843 wait complete. 2021-04-03T03:15:46.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 Processor -- start 2021-04-03T03:15:46.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- start start 2021-04-03T03:15:46.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040603f0 0x7ffb04146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb040add50 0x7ffb041574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb04144ef0 0x7ffb041579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.946+0000 7ffb0a25d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04157ea0 con 0x7ffb040603f0 2021-04-03T03:15:46.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0a25d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04156e30 con 0x7ffb040add50 2021-04-03T03:15:46.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0a25d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04063180 con 0x7ffb04144ef0 2021-04-03T03:15:46.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040603f0 0x7ffb04146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040603f0 0x7ffb04146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55322/0 (socket says 172.21.2.111:55322) 2021-04-03T03:15:46.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 -- 172.21.2.111:0/2352610300 learned_addr learned my addr 172.21.2.111:0/2352610300 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:46.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 -- 172.21.2.111:0/2352610300 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb04144ef0 msgr2=0x7ffb041579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb04144ef0 0x7ffb041579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 -- 172.21.2.111:0/2352610300 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb040add50 msgr2=0x7ffb041574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:46.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb040add50 0x7ffb041574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:46.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 -- 172.21.2.111:0/2352610300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0415b0b0 con 0x7ffb040603f0 2021-04-03T03:15:46.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0925b700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040603f0 0x7ffb04146580 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7ffb00002f70 tx=0x7ffb00006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffaf67fc700 1 -- 172.21.2.111:0/2352610300 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb0000d570 con 0x7ffb040603f0 2021-04-03T03:15:46.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffaf67fc700 1 -- 172.21.2.111:0/2352610300 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0000d6d0 con 0x7ffb040603f0 2021-04-03T03:15:46.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb04157020 con 0x7ffb040603f0 2021-04-03T03:15:46.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb041481b0 con 0x7ffb040603f0 2021-04-03T03:15:46.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffaf67fc700 1 -- 172.21.2.111:0/2352610300 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb0000da30 con 0x7ffb040603f0 2021-04-03T03:15:46.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb04057660 con 0x7ffb040603f0 2021-04-03T03:15:46.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffaf67fc700 1 -- 172.21.2.111:0/2352610300 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ffb0000f070 con 0x7ffb040603f0 2021-04-03T03:15:46.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffaf67fc700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffae4056ef0 0x7ffae40593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:46.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffaf67fc700 1 -- 172.21.2.111:0/2352610300 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ffb0006fce0 con 0x7ffb040603f0 2021-04-03T03:15:46.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb08a5a700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffae4056ef0 0x7ffae40593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:46.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.950+0000 7ffb08a5a700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffae4056ef0 0x7ffae40593b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7ffb040ae520 tx=0x7ffaf8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:46.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:46.954+0000 7ffaf67fc700 1 -- 172.21.2.111:0/2352610300 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ffb000450d0 con 0x7ffb040603f0 2021-04-03T03:15:47.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffb04064ec0 con 0x7ffb040603f0 2021-04-03T03:15:47.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffaf67fc700 1 -- 172.21.2.111:0/2352610300 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v93) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffb000486f0 con 0x7ffb040603f0 2021-04-03T03:15:47.053 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:47.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffae4056ef0 msgr2=0x7ffae40593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffae4056ef0 0x7ffae40593b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7ffb040ae520 tx=0x7ffaf8015000).stop 2021-04-03T03:15:47.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040603f0 msgr2=0x7ffb04146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040603f0 0x7ffb04146580 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7ffb00002f70 tx=0x7ffb00006dd0).stop 2021-04-03T03:15:47.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 shutdown_connections 2021-04-03T03:15:47.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb04144ef0 0x7ffb041579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb040603f0 0x7ffb04146580 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffae4056ef0 0x7ffae40593b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 --2- 172.21.2.111:0/2352610300 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb040add50 0x7ffb041574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 >> 172.21.2.111:0/2352610300 conn(0x7ffb040b61c0 msgr2=0x7ffb041493f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:47.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 shutdown_connections 2021-04-03T03:15:47.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.046+0000 7ffb0a25d700 1 -- 172.21.2.111:0/2352610300 wait complete. 2021-04-03T03:15:47.060 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:15:47.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 Processor -- start 2021-04-03T03:15:47.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- start start 2021-04-03T03:15:47.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1b5c047580 0x7f1b5c03bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c03c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1b5c0d38b0 0x7f1b5c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c043400 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c047960 con 0x7f1b5c047580 2021-04-03T03:15:47.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c042ec0 con 0x7f1b5c0423a0 2021-04-03T03:15:47.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1b5c0d38b0 0x7f1b5c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1b5c0d38b0 0x7f1b5c0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55328/0 (socket says 172.21.2.111:55328) 2021-04-03T03:15:47.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/298168543 learned_addr learned my addr 172.21.2.111:0/298168543 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:47.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6259c700 1 --2- 172.21.2.111:0/298168543 >> v2:172.21.2.115:3300/0 conn(0x7f1b5c047580 0x7f1b5c03bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b61d9b700 1 --2- 172.21.2.111:0/298168543 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c03c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/298168543 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 msgr2=0x7f1b5c03c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:47.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- 172.21.2.111:0/298168543 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c03c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/298168543 >> v2:172.21.2.115:3300/0 conn(0x7f1b5c047580 msgr2=0x7f1b5c03bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:47.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- 172.21.2.111:0/298168543 >> v2:172.21.2.115:3300/0 conn(0x7f1b5c047580 0x7f1b5c03bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/298168543 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c043160 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b61d9b700 1 --2- 172.21.2.111:0/298168543 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c03c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:47.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- 172.21.2.111:0/298168543 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0cd010 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f1b6406b5d0 tx=0x7f1b6405ad70).ready entity=mon.0 client_cookie=1606638ba2cb8e5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6159a700 1 -- 172.21.2.111:0/298168543 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b640699f0 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6159a700 1 -- 172.21.2.111:0/298168543 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6406b610 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6159a700 1 -- 172.21.2.111:0/298168543 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b6405d450 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- 172.21.2.111:0/298168543 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 msgr2=0x7f1b5c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- 172.21.2.111:0/298168543 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0cd010 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f1b6406b5d0 tx=0x7f1b6405ad70).stop 2021-04-03T03:15:47.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- 172.21.2.111:0/298168543 shutdown_connections 2021-04-03T03:15:47.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- 172.21.2.111:0/298168543 >> v2:172.21.2.115:3300/0 conn(0x7f1b5c047580 0x7f1b5c03bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- 172.21.2.111:0/298168543 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0cd010 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- 172.21.2.111:0/298168543 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c03c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- 172.21.2.111:0/298168543 >> 172.21.2.111:0/298168543 conn(0x7f1b5c00d9a0 msgr2=0x7f1b5c046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:47.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- 172.21.2.111:0/298168543 shutdown_connections 2021-04-03T03:15:47.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- 172.21.2.111:0/298168543 wait complete. 2021-04-03T03:15:47.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 Processor -- start 2021-04-03T03:15:47.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- start start 2021-04-03T03:15:47.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c045f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1b5c047580 0x7f1b5c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0d1a10 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0cf9e0 con 0x7f1b5c047580 2021-04-03T03:15:47.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b6359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0d2f80 con 0x7f1b5c0423a0 2021-04-03T03:15:47.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55336/0 (socket says 172.21.2.111:55336) 2021-04-03T03:15:47.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/957531605 learned_addr learned my addr 172.21.2.111:0/957531605 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:47.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/957531605 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 msgr2=0x7f1b5c045f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c045f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/957531605 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1b5c047580 msgr2=0x7f1b5c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1b5c047580 0x7f1b5c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.115+0000 7f1b62d9d700 1 -- 172.21.2.111:0/957531605 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0d3220 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b62d9d700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0ce980 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f1b6406b5d0 tx=0x7f1b64083000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b5b7fe700 1 -- 172.21.2.111:0/957531605 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b640687c0 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b5b7fe700 1 -- 172.21.2.111:0/957531605 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b64068920 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b5b7fe700 1 -- 172.21.2.111:0/957531605 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b6405f4b0 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c0cf6d0 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c0cee70 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c004db0 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b5b7fe700 1 -- 172.21.2.111:0/957531605 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1b6405f610 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b5b7fe700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1b48056ec0 0x7f1b48059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b5b7fe700 1 -- 172.21.2.111:0/957531605 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1b640dfaa0 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b6259c700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1b48056ec0 0x7f1b48059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b6259c700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1b48056ec0 0x7f1b48059380 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f1b5400a000 tx=0x7f1b54008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.119+0000 7f1b5b7fe700 1 -- 172.21.2.111:0/957531605 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1b640b4e90 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b5c008c80 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b5b7fe700 1 -- 172.21.2.111:0/957531605 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1b640b84b0 con 0x7f1b5c0d38b0 2021-04-03T03:15:47.219 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:47.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1b48056ec0 msgr2=0x7f1b48059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1b48056ec0 0x7f1b48059380 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f1b5400a000 tx=0x7f1b54008040).stop 2021-04-03T03:15:47.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 msgr2=0x7f1b5c0ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0ce980 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f1b6406b5d0 tx=0x7f1b64083000).stop 2021-04-03T03:15:47.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 shutdown_connections 2021-04-03T03:15:47.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1b5c0d38b0 0x7f1b5c0ce980 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1b5c0423a0 0x7f1b5c045f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1b48056ec0 0x7f1b48059380 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 --2- 172.21.2.111:0/957531605 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1b5c047580 0x7f1b5c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 >> 172.21.2.111:0/957531605 conn(0x7f1b5c00d9a0 msgr2=0x7f1b5c0067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:47.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 shutdown_connections 2021-04-03T03:15:47.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.215+0000 7f1b6359e700 1 -- 172.21.2.111:0/957531605 wait complete. 2021-04-03T03:15:47.227 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:15:47.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17618e700 1 Processor -- start 2021-04-03T03:15:47.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17618e700 1 -- start start 2021-04-03T03:15:47.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17618e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1700b1b70 0x7fa170143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17618e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700600c0 con 0x7fa1700b0130 2021-04-03T03:15:47.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17618e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700acb80 con 0x7fa17005fce0 2021-04-03T03:15:47.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17618e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700ace20 con 0x7fa1700b1b70 2021-04-03T03:15:47.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17498b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa1700b0130 0x7fa170067370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56136/0 (socket says 172.21.2.111:56136) 2021-04-03T03:15:47.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17498b700 1 -- 172.21.2.111:0/1481683997 learned_addr learned my addr 172.21.2.111:0/1481683997 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:47.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192546414 0 0) 0x7fa1700600c0 con 0x7fa1700b0130 2021-04-03T03:15:47.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa158003240 con 0x7fa1700b0130 2021-04-03T03:15:47.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17598d700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1700b1b70 0x7fa170143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1788947142 0 0) 0x7fa158003240 con 0x7fa1700b0130 2021-04-03T03:15:47.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1700b1b70 msgr2=0x7fa170143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:47.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1700b1b70 0x7fa170143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 >> v1:172.21.2.115:6789/0 conn(0x7fa17005fce0 legacy=0x7fa17013fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700afc60 con 0x7fa1700b0130 2021-04-03T03:15:47.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa16c0016d0 con 0x7fa1700b0130 2021-04-03T03:15:47.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 >> v1:172.21.2.111:6789/0 conn(0x7fa1700b0130 legacy=0x7fa170067370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa158003760 0x7fa158003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1580044e0 0x7fa158004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa158004e50 0x7fa15800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700ace20 con 0x7fa1580044e0 2021-04-03T03:15:47.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158004240 con 0x7fa158004e50 2021-04-03T03:15:47.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15800bc20 con 0x7fa158003760 2021-04-03T03:15:47.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17598d700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa1700b1b70 0x7fa170143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:47.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.279+0000 7fa17498b700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1580044e0 0x7fa158004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17498b700 1 -- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa158003760 msgr2=0x7fa158003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:47.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17498b700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa158003760 0x7fa158003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17498b700 1 -- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa158004e50 msgr2=0x7fa15800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:47.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17498b700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa158004e50 0x7fa15800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17498b700 1 -- 172.21.2.111:0/1481683997 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158003240 con 0x7fa1580044e0 2021-04-03T03:15:47.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17498b700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1580044e0 0x7fa158004960 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fa158004450 tx=0x7fa16c0076a0).ready entity=mon.0 client_cookie=aa1567f21e64a44 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa16c00cea0 con 0x7fa1580044e0 2021-04-03T03:15:47.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c0070d0 con 0x7fa1580044e0 2021-04-03T03:15:47.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa163fff700 1 -- 172.21.2.111:0/1481683997 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa16c007430 con 0x7fa1580044e0 2021-04-03T03:15:47.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- 172.21.2.111:0/1481683997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1580044e0 msgr2=0x7fa158004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 --2- 172.21.2.111:0/1481683997 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa1580044e0 0x7fa158004960 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fa158004450 tx=0x7fa16c0076a0).stop 2021-04-03T03:15:47.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 -- 172.21.2.111:0/1481683997 reap_dead start 2021-04-03T03:15:47.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- 172.21.2.111:0/1481683997 shutdown_connections 2021-04-03T03:15:47.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- 172.21.2.111:0/1481683997 >> 172.21.2.111:0/1481683997 conn(0x7fa1700b5fb0 msgr2=0x7fa1700b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:47.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- 172.21.2.111:0/1481683997 shutdown_connections 2021-04-03T03:15:47.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- 172.21.2.111:0/1481683997 wait complete. 2021-04-03T03:15:47.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 Processor -- start 2021-04-03T03:15:47.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- start start 2021-04-03T03:15:47.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1700ab680 0x7fa17013d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17013ab40 0x7fa17013b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17013b600 0x7fa1701374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700673a0 con 0x7fa17013b600 2021-04-03T03:15:47.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17013a440 con 0x7fa1700ab680 2021-04-03T03:15:47.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700aba60 con 0x7fa17013ab40 2021-04-03T03:15:47.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17013b600 0x7fa1701374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17013b600 0x7fa1701374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55358/0 (socket says 172.21.2.111:55358) 2021-04-03T03:15:47.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 -- 172.21.2.111:0/1294733797 learned_addr learned my addr 172.21.2.111:0/1294733797 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:47.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17518c700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1700ab680 0x7fa17013d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17498b700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17013ab40 0x7fa17013b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 -- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17013ab40 msgr2=0x7fa17013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17013ab40 0x7fa17013b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 -- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1700ab680 msgr2=0x7fa17013d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1700ab680 0x7fa17013d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 -- 172.21.2.111:0/1294733797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17013a6a0 con 0x7fa17013b600 2021-04-03T03:15:47.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17598d700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17013b600 0x7fa1701374f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fa1680011d0 tx=0x7fa168008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa1627fc700 1 -- 172.21.2.111:0/1294733797 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa168010580 con 0x7fa17013b600 2021-04-03T03:15:47.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa1627fc700 1 -- 172.21.2.111:0/1294733797 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1680106e0 con 0x7fa17013b600 2021-04-03T03:15:47.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa1627fc700 1 -- 172.21.2.111:0/1294733797 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa168010a80 con 0x7fa17013b600 2021-04-03T03:15:47.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa170140000 con 0x7fa17013b600 2021-04-03T03:15:47.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.283+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17013ba90 con 0x7fa17013b600 2021-04-03T03:15:47.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.287+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17013af20 con 0x7fa17013b600 2021-04-03T03:15:47.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.287+0000 7fa1627fc700 1 -- 172.21.2.111:0/1294733797 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa168010be0 con 0x7fa17013b600 2021-04-03T03:15:47.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.287+0000 7fa1627fc700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa150056f20 0x7fa1500593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.287+0000 7fa1627fc700 1 -- 172.21.2.111:0/1294733797 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa168070350 con 0x7fa17013b600 2021-04-03T03:15:47.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.287+0000 7fa17518c700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa150056f20 0x7fa1500593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.287+0000 7fa17518c700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa150056f20 0x7fa1500593e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fa16400b000 tx=0x7fa164009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.287+0000 7fa1627fc700 1 -- 172.21.2.111:0/1294733797 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa168045740 con 0x7fa17013b600 2021-04-03T03:15:47.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.379+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa170057660 con 0x7fa17013b600 2021-04-03T03:15:47.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.379+0000 7fa1627fc700 1 -- 172.21.2.111:0/1294733797 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa168048d60 con 0x7fa17013b600 2021-04-03T03:15:47.386 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:47.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa150056f20 msgr2=0x7fa1500593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa150056f20 0x7fa1500593e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fa16400b000 tx=0x7fa164009040).stop 2021-04-03T03:15:47.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17013b600 msgr2=0x7fa1701374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17013b600 0x7fa1701374f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fa1680011d0 tx=0x7fa168008040).stop 2021-04-03T03:15:47.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 shutdown_connections 2021-04-03T03:15:47.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa17013b600 0x7fa1701374f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa150056f20 0x7fa1500593e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa1700ab680 0x7fa17013d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 --2- 172.21.2.111:0/1294733797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa17013ab40 0x7fa17013b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 >> 172.21.2.111:0/1294733797 conn(0x7fa1700b5fb0 msgr2=0x7fa1700677b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:47.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 shutdown_connections 2021-04-03T03:15:47.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.383+0000 7fa17618e700 1 -- 172.21.2.111:0/1294733797 wait complete. 2021-04-03T03:15:47.393 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:15:47.394 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:47.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 Processor -- start 2021-04-03T03:15:47.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- start start 2021-04-03T03:15:47.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8ee00dbb90 0x7f8ee0042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee0006bc0 0x7f8ee00dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee00dc6e0 con 0x7f8ee003d360 2021-04-03T03:15:47.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee00d9560 con 0x7f8ee00dbb90 2021-04-03T03:15:47.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee003dcb0 con 0x7f8ee0006bc0 2021-04-03T03:15:47.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8ee00dbb90 0x7f8ee0042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8ee00dbb90 0x7f8ee0042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55362/0 (socket says 172.21.2.111:55362) 2021-04-03T03:15:47.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 -- 172.21.2.111:0/3162507493 learned_addr learned my addr 172.21.2.111:0/3162507493 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:47.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 -- 172.21.2.111:0/3162507493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee0006bc0 msgr2=0x7f8ee00dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:47.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 --2- 172.21.2.111:0/3162507493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee0006bc0 0x7f8ee00dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 -- 172.21.2.111:0/3162507493 >> v1:172.21.2.115:6789/0 conn(0x7f8ee003d360 legacy=0x7f8ee00064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:47.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 -- 172.21.2.111:0/3162507493 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee00dc460 con 0x7f8ee00dbb90 2021-04-03T03:15:47.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eede03700 1 --2- 172.21.2.111:0/3162507493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00dbb90 0x7f8ee0042750 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f8ee0046210 tx=0x7f8ee4006e90).ready entity=mon.0 client_cookie=509db779027b4d44 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eece01700 1 -- 172.21.2.111:0/3162507493 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ee40098a0 con 0x7f8ee00dbb90 2021-04-03T03:15:47.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eece01700 1 -- 172.21.2.111:0/3162507493 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee4009a00 con 0x7f8ee00dbb90 2021-04-03T03:15:47.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eece01700 1 -- 172.21.2.111:0/3162507493 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ee4009da0 con 0x7f8ee00dbb90 2021-04-03T03:15:47.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- 172.21.2.111:0/3162507493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00dbb90 msgr2=0x7f8ee0042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 --2- 172.21.2.111:0/3162507493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00dbb90 0x7f8ee0042750 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f8ee0046210 tx=0x7f8ee4006e90).stop 2021-04-03T03:15:47.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- 172.21.2.111:0/3162507493 shutdown_connections 2021-04-03T03:15:47.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 --2- 172.21.2.111:0/3162507493 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee0006bc0 0x7f8ee00dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 --2- 172.21.2.111:0/3162507493 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00dbb90 0x7f8ee0042750 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- 172.21.2.111:0/3162507493 >> 172.21.2.111:0/3162507493 conn(0x7f8ee00462c0 msgr2=0x7f8ee00466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:47.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- 172.21.2.111:0/3162507493 shutdown_connections 2021-04-03T03:15:47.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- 172.21.2.111:0/3162507493 wait complete. 2021-04-03T03:15:47.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 Processor -- start 2021-04-03T03:15:47.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.447+0000 7f8eeee05700 1 -- start start 2021-04-03T03:15:47.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ee003d360 0x7f8ee00a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee00dbb90 0x7f8ee00df350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00a30c0 0x7f8ee00ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee003bbc0 con 0x7f8ee00a30c0 2021-04-03T03:15:47.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee003c6d0 con 0x7f8ee003d360 2021-04-03T03:15:47.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee003c970 con 0x7f8ee00dbb90 2021-04-03T03:15:47.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00a30c0 0x7f8ee00ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00a30c0 0x7f8ee00ead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55372/0 (socket says 172.21.2.111:55372) 2021-04-03T03:15:47.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 -- 172.21.2.111:0/2416365614 learned_addr learned my addr 172.21.2.111:0/2416365614 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:47.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 -- 172.21.2.111:0/2416365614 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee00dbb90 msgr2=0x7f8ee00df350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee00dbb90 0x7f8ee00df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 -- 172.21.2.111:0/2416365614 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ee003d360 msgr2=0x7f8ee00a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ee003d360 0x7f8ee00a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 -- 172.21.2.111:0/2416365614 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee00ea8a0 con 0x7f8ee00a30c0 2021-04-03T03:15:47.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eee604700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00a30c0 0x7f8ee00ead40 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f8ee806b5d0 tx=0x7f8ee805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8edaffd700 1 -- 172.21.2.111:0/2416365614 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ee805f2c0 con 0x7f8ee00a30c0 2021-04-03T03:15:47.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8edaffd700 1 -- 172.21.2.111:0/2416365614 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee805f420 con 0x7f8ee00a30c0 2021-04-03T03:15:47.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8edaffd700 1 -- 172.21.2.111:0/2416365614 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ee805f780 con 0x7f8ee00a30c0 2021-04-03T03:15:47.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee009f4b0 con 0x7f8ee00a30c0 2021-04-03T03:15:47.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee00eb230 con 0x7f8ee00a30c0 2021-04-03T03:15:47.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee00eb770 con 0x7f8ee00a30c0 2021-04-03T03:15:47.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8edaffd700 1 -- 172.21.2.111:0/2416365614 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8ee806adb0 con 0x7f8ee00a30c0 2021-04-03T03:15:47.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8edaffd700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ec8056fb0 0x7f8ec8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:47.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8edaffd700 1 -- 172.21.2.111:0/2416365614 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8ee80df2f0 con 0x7f8ee00a30c0 2021-04-03T03:15:47.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eede03700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ec8056fb0 0x7f8ec8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:47.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.451+0000 7f8eede03700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ec8056fb0 0x7f8ec8059470 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f8ee0046210 tx=0x7f8ee4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:47.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.455+0000 7f8edaffd700 1 -- 172.21.2.111:0/2416365614 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8ee00eb770 con 0x7f8ee00a30c0 2021-04-03T03:15:47.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ee00eb770 con 0x7f8ee00a30c0 2021-04-03T03:15:47.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8edaffd700 1 -- 172.21.2.111:0/2416365614 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f8ee00eb770 con 0x7f8ee00a30c0 2021-04-03T03:15:47.583 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:47.584 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":38,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3837436656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3837436656},{"type":"v1","addr":"172.21.2.115:6833","nonce":3837436656}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:44.064474+0000","modified":"2021-04-03T03:15:45.067347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/13282524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":13282524},{"type":"v1","addr":"172.21.2.111:6835","nonce":13282524}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-04-03T03:15:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ec8056fb0 msgr2=0x7f8ec8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ec8056fb0 0x7f8ec8059470 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f8ee0046210 tx=0x7f8ee4015000).stop 2021-04-03T03:15:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00a30c0 msgr2=0x7f8ee00ead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:47.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00a30c0 0x7f8ee00ead40 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f8ee806b5d0 tx=0x7f8ee805ad70).stop 2021-04-03T03:15:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 shutdown_connections 2021-04-03T03:15:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ee00a30c0 0x7f8ee00ead40 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ec8056fb0 0x7f8ec8059470 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ee003d360 0x7f8ee00a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 --2- 172.21.2.111:0/2416365614 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ee00dbb90 0x7f8ee00df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 >> 172.21.2.111:0/2416365614 conn(0x7f8ee00462c0 msgr2=0x7f8ee0042670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 shutdown_connections 2021-04-03T03:15:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:47.579+0000 7f8eeee05700 1 -- 172.21.2.111:0/2416365614 wait complete. 2021-04-03T03:15:47.587 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 38 2021-04-03T03:15:47.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:15:44.064474+0000', 'modified': '2021-04-03T03:15:45.067347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6879}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6879': {'gid': 6879, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/13282524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 13282524}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 13282524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:15:47.793 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:15:47.794 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:15:47.882 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6879,"want_state":"up:active","state":"up:active","rank_uptime":3.8102075580000001,"mdsmap_epoch":38,"osdmap_epoch":93,"osdmap_epoch_barrier":92,"uptime":7.9594249960000001} 2021-04-03T03:15:47.882 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6879, "mdsmap_epoch": 38, "osdmap_epoch": 93, "osdmap_epoch_barrier": 92, "rank_uptime": 3.810207558, "state": "up:active", "uptime": 7.959424996, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:15:47.883 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:15:47.883 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:15:47.884 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:15:47.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:47.884 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:47.895 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:15:47.946 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:47.947 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:47.997 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:47.999 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:48.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:48.001 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:15:48.057 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:15:48.058 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:48.107 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:15:48.119 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:48.120 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:48.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15703]: starting ceph client 2021-04-03T03:15:48.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:15:48.143+0000 7fad421410c0 -1 init, newargv = 0x55d0ca3d5a00 newargc=9 2021-04-03T03:15:48.154 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:48.155 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:48.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:48.157 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:15:48.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15703]: starting fuse 2021-04-03T03:15:48.213 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:15:48.213 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:15:48.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:48.214 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:15:48.215 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:15:48.215 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:15:48.216 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:15:48.217 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:15:48.217 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:48.218 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:15:48.218 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:15:48.219 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:15:48.219 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:48.220 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:15:48.220 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:15:48.221 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:15:48.222 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:15:48.222 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:48.223 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:15:48.223 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:15:48.224 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:15:48.224 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:15:48.225 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:15:48.225 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:48.226 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:15:48.226 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:15:48.331 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15703.asok 2021-04-03T03:15:48.335 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15703.asok 2021-04-03T03:15:48.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:48.336 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15703.asok status 2021-04-03T03:15:48.391 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:15:48.392 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:15:48.392 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:15:48.392 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:15:48.392 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:15:48.393 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:15:48.393 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:15:48.393 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15703", 2021-04-03T03:15:48.393 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:15:48.393 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:48.394 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:15:48.394 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:15:48.394 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 5230, 2021-04-03T03:15:48.394 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:15:48.395 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:15:48.395 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:15:48.395 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 5230 2021-04-03T03:15:48.395 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:48.395 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:15:48.396 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:15:48.396 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:15:48.396 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2184137207 2021-04-03T03:15:48.396 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:15:48.396 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:48.397 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:15:48.397 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:15:48.397 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:15:48.397 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2184137207 2021-04-03T03:15:48.397 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:15:48.398 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.5230 172.21.2.126:0/2184137207", 2021-04-03T03:15:48.398 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/2184137207", 2021-04-03T03:15:48.398 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:15:48.398 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 38, 2021-04-03T03:15:48.399 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 93, 2021-04-03T03:15:48.399 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:15:48.399 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:15:48.399 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:15:48.400 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:15:48.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:48.401 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:48.402 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:48.408 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:15:48.408 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:48.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:48.409 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:48.457 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:48.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.511+0000 7f74662a4700 1 Processor -- start 2021-04-03T03:15:48.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.511+0000 7f74662a4700 1 -- start start 2021-04-03T03:15:48.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7458048f00 0x7f7458041a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580492e0 con 0x7f74580431a0 2021-04-03T03:15:48.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745803fac0 con 0x7f74580d5e10 2021-04-03T03:15:48.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745803fd60 con 0x7f7458048f00 2021-04-03T03:15:48.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74652a2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7458048f00 0x7f7458041a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74652a2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7458048f00 0x7f7458041a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55376/0 (socket says 172.21.2.111:55376) 2021-04-03T03:15:48.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74652a2700 1 -- 172.21.2.111:0/2385639385 learned_addr learned my addr 172.21.2.111:0/2385639385 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:48.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74652a2700 1 -- 172.21.2.111:0/2385639385 >> v1:172.21.2.115:6789/0 conn(0x7f74580d5e10 legacy=0x7f74580d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:48.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74652a2700 1 -- 172.21.2.111:0/2385639385 >> v1:172.21.2.111:6789/0 conn(0x7f74580431a0 legacy=0x7f74580d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74652a2700 1 -- 172.21.2.111:0/2385639385 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580d5260 con 0x7f7458048f00 2021-04-03T03:15:48.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74652a2700 1 --2- 172.21.2.111:0/2385639385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7458048f00 0x7f7458041a70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f745800dd60 tx=0x7f745c006e90).ready entity=mon.0 client_cookie=a32dd2869dfc62f3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7453fff700 1 -- 172.21.2.111:0/2385639385 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745c009980 con 0x7f7458048f00 2021-04-03T03:15:48.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7453fff700 1 -- 172.21.2.111:0/2385639385 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c009ae0 con 0x7f7458048f00 2021-04-03T03:15:48.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7453fff700 1 -- 172.21.2.111:0/2385639385 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745c009e80 con 0x7f7458048f00 2021-04-03T03:15:48.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- 172.21.2.111:0/2385639385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7458048f00 msgr2=0x7f7458041a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 --2- 172.21.2.111:0/2385639385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7458048f00 0x7f7458041a70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f745800dd60 tx=0x7f745c006e90).stop 2021-04-03T03:15:48.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- 172.21.2.111:0/2385639385 shutdown_connections 2021-04-03T03:15:48.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 --2- 172.21.2.111:0/2385639385 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7458048f00 0x7f7458041a70 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- 172.21.2.111:0/2385639385 >> 172.21.2.111:0/2385639385 conn(0x7f745800f2c0 msgr2=0x7f74580484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:48.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- 172.21.2.111:0/2385639385 shutdown_connections 2021-04-03T03:15:48.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- 172.21.2.111:0/2385639385 wait complete. 2021-04-03T03:15:48.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 Processor -- start 2021-04-03T03:15:48.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- start start 2021-04-03T03:15:48.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74580431a0 0x7f74580d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7458048f00 0x7f74580cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74580d5e10 0x7f74580cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745803e440 con 0x7f74580d5e10 2021-04-03T03:15:48.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580e2080 con 0x7f74580431a0 2021-04-03T03:15:48.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745803c120 con 0x7f7458048f00 2021-04-03T03:15:48.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74580d5e10 0x7f74580cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74580d5e10 0x7f74580cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55384/0 (socket says 172.21.2.111:55384) 2021-04-03T03:15:48.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 -- 172.21.2.111:0/21076833 learned_addr learned my addr 172.21.2.111:0/21076833 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:48.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 -- 172.21.2.111:0/21076833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7458048f00 msgr2=0x7f74580cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7458048f00 0x7f74580cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 -- 172.21.2.111:0/21076833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74580431a0 msgr2=0x7f74580d8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74580431a0 0x7f74580d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 -- 172.21.2.111:0/21076833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745803b920 con 0x7f74580d5e10 2021-04-03T03:15:48.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f7465aa3700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74580d5e10 0x7f74580cf1c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f745803e230 tx=0x7f746005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74527fc700 1 -- 172.21.2.111:0/21076833 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f746005df60 con 0x7f74580d5e10 2021-04-03T03:15:48.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74527fc700 1 -- 172.21.2.111:0/21076833 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746008a070 con 0x7f74580d5e10 2021-04-03T03:15:48.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580cf6b0 con 0x7f74580d5e10 2021-04-03T03:15:48.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74527fc700 1 -- 172.21.2.111:0/21076833 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f746005f4b0 con 0x7f74580d5e10 2021-04-03T03:15:48.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.515+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580e1010 con 0x7f74580d5e10 2021-04-03T03:15:48.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.519+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745803bd30 con 0x7f74580d5e10 2021-04-03T03:15:48.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.519+0000 7f74527fc700 1 -- 172.21.2.111:0/21076833 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f746005f610 con 0x7f74580d5e10 2021-04-03T03:15:48.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.519+0000 7f74527fc700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7440056f20 0x7f74400593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.519+0000 7f74527fc700 1 -- 172.21.2.111:0/21076833 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f74600e3220 con 0x7f74580d5e10 2021-04-03T03:15:48.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.519+0000 7f74652a2700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7440056f20 0x7f74400593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.519+0000 7f74652a2700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7440056f20 0x7f74400593e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f745800dd60 tx=0x7f745c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.519+0000 7f74527fc700 1 -- 172.21.2.111:0/21076833 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f745803bd30 con 0x7f74580d5e10 2021-04-03T03:15:48.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f745803bd30 con 0x7f74580d5e10 2021-04-03T03:15:48.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74527fc700 1 -- 172.21.2.111:0/21076833 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f745803bd30 con 0x7f74580d5e10 2021-04-03T03:15:48.619 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:48.619 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":93,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:45.068238+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:15:42.095114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-04-03T03:15:43.035163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:48.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7440056f20 msgr2=0x7f74400593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7440056f20 0x7f74400593e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f745800dd60 tx=0x7f745c015000).stop 2021-04-03T03:15:48.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74580d5e10 msgr2=0x7f74580cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74580d5e10 0x7f74580cf1c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f745803e230 tx=0x7f746005ad70).stop 2021-04-03T03:15:48.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 shutdown_connections 2021-04-03T03:15:48.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74580d5e10 0x7f74580cf1c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7440056f20 0x7f74400593e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74580431a0 0x7f74580d8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 --2- 172.21.2.111:0/21076833 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7458048f00 0x7f74580cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 >> 172.21.2.111:0/21076833 conn(0x7f745800f2c0 msgr2=0x7f7458043d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:48.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 shutdown_connections 2021-04-03T03:15:48.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.615+0000 7f74662a4700 1 -- 172.21.2.111:0/21076833 wait complete. 2021-04-03T03:15:48.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:15:42.095114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:15:43.035163+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:15:48.628 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:48.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 Processor -- start 2021-04-03T03:15:48.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 -- start start 2021-04-03T03:15:48.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6a5c0603f0 0x7f6a5c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6a5c0add50 0x7f6a5c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0b17b0 0x7f6a5c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c1451c0 con 0x7f6a5c0603f0 2021-04-03T03:15:48.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c0607d0 con 0x7f6a5c0add50 2021-04-03T03:15:48.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.679+0000 7f6a6194b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c14b090 con 0x7f6a5c0b17b0 2021-04-03T03:15:48.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6a5c0603f0 0x7f6a5c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6a5c0603f0 0x7f6a5c1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55390/0 (socket says 172.21.2.111:55390) 2021-04-03T03:15:48.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/3898398960 learned_addr learned my addr 172.21.2.111:0/3898398960 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:48.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6114a700 1 --2- 172.21.2.111:0/3898398960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0b17b0 0x7f6a5c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/3898398960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0b17b0 msgr2=0x7f6a5c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:48.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- 172.21.2.111:0/3898398960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0b17b0 0x7f6a5c147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a5bfff700 1 --2- 172.21.2.111:0/3898398960 >> v2:172.21.2.115:3300/0 conn(0x7f6a5c0add50 0x7f6a5c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/3898398960 >> v2:172.21.2.115:3300/0 conn(0x7f6a5c0add50 msgr2=0x7f6a5c144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:48.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- 172.21.2.111:0/3898398960 >> v2:172.21.2.115:3300/0 conn(0x7f6a5c0add50 0x7f6a5c144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/3898398960 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c14b330 con 0x7f6a5c0603f0 2021-04-03T03:15:48.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a5bfff700 1 --2- 172.21.2.111:0/3898398960 >> v2:172.21.2.115:3300/0 conn(0x7f6a5c0add50 0x7f6a5c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:48.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- 172.21.2.111:0/3898398960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c1447e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f6a4c002f30 tx=0x7f6a4c006e90).ready entity=mon.0 client_cookie=37870c4ca855b184 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a5b7fe700 1 -- 172.21.2.111:0/3898398960 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c009920 con 0x7f6a5c0603f0 2021-04-03T03:15:48.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a5b7fe700 1 -- 172.21.2.111:0/3898398960 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c009a80 con 0x7f6a5c0603f0 2021-04-03T03:15:48.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a5b7fe700 1 -- 172.21.2.111:0/3898398960 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c009de0 con 0x7f6a5c0603f0 2021-04-03T03:15:48.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- 172.21.2.111:0/3898398960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 msgr2=0x7f6a5c1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 --2- 172.21.2.111:0/3898398960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c1447e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f6a4c002f30 tx=0x7f6a4c006e90).stop 2021-04-03T03:15:48.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- 172.21.2.111:0/3898398960 shutdown_connections 2021-04-03T03:15:48.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 --2- 172.21.2.111:0/3898398960 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0b17b0 0x7f6a5c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 --2- 172.21.2.111:0/3898398960 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c1447e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 --2- 172.21.2.111:0/3898398960 >> v2:172.21.2.115:3300/0 conn(0x7f6a5c0add50 0x7f6a5c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- 172.21.2.111:0/3898398960 >> 172.21.2.111:0/3898398960 conn(0x7f6a5c0b61c0 msgr2=0x7f6a5c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:48.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- 172.21.2.111:0/3898398960 shutdown_connections 2021-04-03T03:15:48.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- 172.21.2.111:0/3898398960 wait complete. 2021-04-03T03:15:48.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 Processor -- start 2021-04-03T03:15:48.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- start start 2021-04-03T03:15:48.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c0a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0add50 0x7f6a5c0a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a5c0b17b0 0x7f6a5c0aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c068020 con 0x7f6a5c0603f0 2021-04-03T03:15:48.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c147ca0 con 0x7f6a5c0b17b0 2021-04-03T03:15:48.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c147760 con 0x7f6a5c0add50 2021-04-03T03:15:48.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c0a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c0a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55398/0 (socket says 172.21.2.111:55398) 2021-04-03T03:15:48.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/318511738 learned_addr learned my addr 172.21.2.111:0/318511738 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:48.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/318511738 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0add50 msgr2=0x7f6a5c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0add50 0x7f6a5c0a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/318511738 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a5c0b17b0 msgr2=0x7f6a5c0aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a5c0b17b0 0x7f6a5c0aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 -- 172.21.2.111:0/318511738 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c147a00 con 0x7f6a5c0603f0 2021-04-03T03:15:48.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a60949700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c0a7240 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f6a4c002f30 tx=0x7f6a4c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a59ffb700 1 -- 172.21.2.111:0/318511738 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c011db0 con 0x7f6a5c0603f0 2021-04-03T03:15:48.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a59ffb700 1 -- 172.21.2.111:0/318511738 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c01e460 con 0x7f6a5c0603f0 2021-04-03T03:15:48.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c147450 con 0x7f6a5c0603f0 2021-04-03T03:15:48.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a59ffb700 1 -- 172.21.2.111:0/318511738 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c01e7c0 con 0x7f6a5c0603f0 2021-04-03T03:15:48.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.683+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c0a8310 con 0x7f6a5c0603f0 2021-04-03T03:15:48.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.687+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c057660 con 0x7f6a5c0603f0 2021-04-03T03:15:48.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.687+0000 7f6a59ffb700 1 -- 172.21.2.111:0/318511738 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6a4c025030 con 0x7f6a5c0603f0 2021-04-03T03:15:48.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.687+0000 7f6a59ffb700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a40056f20 0x7f6a400593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.687+0000 7f6a59ffb700 1 -- 172.21.2.111:0/318511738 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f6a4c071760 con 0x7f6a5c0603f0 2021-04-03T03:15:48.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.687+0000 7f6a5bfff700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a40056f20 0x7f6a400593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.687+0000 7f6a5bfff700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a40056f20 0x7f6a400593e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f6a44007000 tx=0x7f6a44008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.687+0000 7f6a59ffb700 1 -- 172.21.2.111:0/318511738 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6a4c046b50 con 0x7f6a5c0603f0 2021-04-03T03:15:48.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6a5c148850 con 0x7f6a5c0603f0 2021-04-03T03:15:48.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a59ffb700 1 -- 172.21.2.111:0/318511738 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v93) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6a4c0230b0 con 0x7f6a5c0603f0 2021-04-03T03:15:48.787 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:48.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a40056f20 msgr2=0x7f6a400593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a40056f20 0x7f6a400593e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f6a44007000 tx=0x7f6a44008040).stop 2021-04-03T03:15:48.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 msgr2=0x7f6a5c0a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c0a7240 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f6a4c002f30 tx=0x7f6a4c015000).stop 2021-04-03T03:15:48.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 shutdown_connections 2021-04-03T03:15:48.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a40056f20 0x7f6a400593e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a5c0b17b0 0x7f6a5c0aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a5c0603f0 0x7f6a5c0a7240 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 --2- 172.21.2.111:0/318511738 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a5c0add50 0x7f6a5c0a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 >> 172.21.2.111:0/318511738 conn(0x7f6a5c0b61c0 msgr2=0x7f6a5c0b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:48.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 shutdown_connections 2021-04-03T03:15:48.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.783+0000 7f6a6194b700 1 -- 172.21.2.111:0/318511738 wait complete. 2021-04-03T03:15:48.794 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:48.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28d554b700 1 Processor -- start 2021-04-03T03:15:48.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28d554b700 1 -- start start 2021-04-03T03:15:48.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28d554b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f28c80dbba0 0x7f28c80dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28d554b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80dc560 con 0x7f28c8041060 2021-04-03T03:15:48.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28d554b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80dc800 con 0x7f28c803d560 2021-04-03T03:15:48.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28d554b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c803de60 con 0x7f28c80dbba0 2021-04-03T03:15:48.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28cf7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f28c8041060 0x7f28c803fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56198/0 (socket says 172.21.2.111:56198) 2021-04-03T03:15:48.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28cf7fe700 1 -- 172.21.2.111:0/2963377581 learned_addr learned my addr 172.21.2.111:0/2963377581 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:48.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106010400 0 0) 0x7f28c80dc560 con 0x7f28c8041060 2021-04-03T03:15:48.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.847+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28bc003240 con 0x7f28c8041060 2021-04-03T03:15:48.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 208559707 0 0) 0x7f28bc003240 con 0x7f28c8041060 2021-04-03T03:15:48.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 >> v2:172.21.2.115:3300/0 conn(0x7f28c80dbba0 msgr2=0x7f28c80dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:48.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 --2- 172.21.2.111:0/2963377581 >> v2:172.21.2.115:3300/0 conn(0x7f28c80dbba0 0x7f28c80dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 >> v1:172.21.2.115:6789/0 conn(0x7f28c803d560 legacy=0x7f28c8041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:48.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c803da30 con 0x7f28c8041060 2021-04-03T03:15:48.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f28c00016d0 con 0x7f28c8041060 2021-04-03T03:15:48.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 >> v1:172.21.2.111:6789/0 conn(0x7f28c8041060 legacy=0x7f28c803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28bc0037b0 0x7f28bc003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28bc0044e0 0x7f28bc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f28bc004e50 0x7f28bc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c803de60 con 0x7f28bc0037b0 2021-04-03T03:15:48.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc005300 con 0x7f28bc004e50 2021-04-03T03:15:48.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc00bc20 con 0x7f28bc0044e0 2021-04-03T03:15:48.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cffff700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28bc0037b0 0x7f28bc003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cffff700 1 -- 172.21.2.111:0/2963377581 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28bc0044e0 msgr2=0x7f28bc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:48.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cffff700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28bc0044e0 0x7f28bc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cffff700 1 -- 172.21.2.111:0/2963377581 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f28bc004e50 msgr2=0x7f28bc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:48.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cffff700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f28bc004e50 0x7f28bc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cffff700 1 -- 172.21.2.111:0/2963377581 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c803da30 con 0x7f28bc0037b0 2021-04-03T03:15:48.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 -- 172.21.2.111:0/2963377581 reap_dead start 2021-04-03T03:15:48.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cffff700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28bc0037b0 0x7f28bc003bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f28c4002f30 tx=0x7f28c4006e10).ready entity=mon.0 client_cookie=832a96833556dcf4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c40097f0 con 0x7f28bc0037b0 2021-04-03T03:15:48.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4009950 con 0x7f28bc0037b0 2021-04-03T03:15:48.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28ceffd700 1 -- 172.21.2.111:0/2963377581 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c4009cb0 con 0x7f28bc0037b0 2021-04-03T03:15:48.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/2963377581 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28bc0037b0 msgr2=0x7f28bc003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28bc0037b0 0x7f28bc003bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f28c4002f30 tx=0x7f28c4006e10).stop 2021-04-03T03:15:48.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/2963377581 shutdown_connections 2021-04-03T03:15:48.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 --2- 172.21.2.111:0/2963377581 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28bc0037b0 0x7f28bc003bb0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/2963377581 >> 172.21.2.111:0/2963377581 conn(0x7f28c80462b0 msgr2=0x7f28c80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:48.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/2963377581 shutdown_connections 2021-04-03T03:15:48.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/2963377581 wait complete. 2021-04-03T03:15:48.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 Processor -- start 2021-04-03T03:15:48.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- start start 2021-04-03T03:15:48.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f28bc0037b0 0x7f28c80e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28c80a16d0 0x7f28c80a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28c80d9910 0x7f28c80d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80a0db0 con 0x7f28c80d9910 2021-04-03T03:15:48.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c809ff70 con 0x7f28bc0037b0 2021-04-03T03:15:48.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80e7ba0 con 0x7f28c80a16d0 2021-04-03T03:15:48.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28c80d9910 0x7f28c80d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28c80d9910 0x7f28c80d9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55416/0 (socket says 172.21.2.111:55416) 2021-04-03T03:15:48.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 -- 172.21.2.111:0/3176700610 learned_addr learned my addr 172.21.2.111:0/3176700610 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:48.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 -- 172.21.2.111:0/3176700610 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28c80a16d0 msgr2=0x7f28c80a1b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:48.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28c80a16d0 0x7f28c80a1b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 -- 172.21.2.111:0/3176700610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f28bc0037b0 msgr2=0x7f28c80e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f28bc0037b0 0x7f28c80e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 -- 172.21.2.111:0/3176700610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80a0340 con 0x7f28c80d9910 2021-04-03T03:15:48.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d4d4a700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28c80d9910 0x7f28c80d9d90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f28c803ed80 tx=0x7f28d005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cd7fa700 1 -- 172.21.2.111:0/3176700610 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d005df60 con 0x7f28c80d9910 2021-04-03T03:15:48.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cd7fa700 1 -- 172.21.2.111:0/3176700610 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d00699f0 con 0x7f28c80d9910 2021-04-03T03:15:48.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c80a04d0 con 0x7f28c80d9910 2021-04-03T03:15:48.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c809fbc0 con 0x7f28c80d9910 2021-04-03T03:15:48.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cd7fa700 1 -- 172.21.2.111:0/3176700610 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d006b250 con 0x7f28c80d9910 2021-04-03T03:15:48.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8004db0 con 0x7f28c80d9910 2021-04-03T03:15:48.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.851+0000 7f28cd7fa700 1 -- 172.21.2.111:0/3176700610 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f28d0061070 con 0x7f28c80d9910 2021-04-03T03:15:48.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.855+0000 7f28cd7fa700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f28b0056f10 0x7f28b00593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:48.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.855+0000 7f28cd7fa700 1 -- 172.21.2.111:0/3176700610 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f28d00deb60 con 0x7f28c80d9910 2021-04-03T03:15:48.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.855+0000 7f28cffff700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f28b0056f10 0x7f28b00593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:48.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.855+0000 7f28cffff700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f28b0056f10 0x7f28b00593d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f28c4002f30 tx=0x7f28c4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:48.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.855+0000 7f28cd7fa700 1 -- 172.21.2.111:0/3176700610 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f28d00b3f50 con 0x7f28c80d9910 2021-04-03T03:15:48.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f28c80a04d0 con 0x7f28c80d9910 2021-04-03T03:15:48.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28cd7fa700 1 -- 172.21.2.111:0/3176700610 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v93) v1 ==== 92+0+10 (secure 0 0 0) 0x7f28c80a04d0 con 0x7f28c80d9910 2021-04-03T03:15:48.955 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:48.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f28b0056f10 msgr2=0x7f28b00593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f28b0056f10 0x7f28b00593d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f28c4002f30 tx=0x7f28c4016000).stop 2021-04-03T03:15:48.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28c80d9910 msgr2=0x7f28c80d9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:48.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28c80d9910 0x7f28c80d9d90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f28c803ed80 tx=0x7f28d005ad70).stop 2021-04-03T03:15:48.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 shutdown_connections 2021-04-03T03:15:48.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28c80d9910 0x7f28c80d9d90 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f28b0056f10 0x7f28b00593d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:48.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f28bc0037b0 0x7f28c80e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 --2- 172.21.2.111:0/3176700610 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28c80a16d0 0x7f28c80a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:48.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 >> 172.21.2.111:0/3176700610 conn(0x7f28c80462b0 msgr2=0x7f28c80056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:48.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 shutdown_connections 2021-04-03T03:15:48.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:48.951+0000 7f28d554b700 1 -- 172.21.2.111:0/3176700610 wait complete. 2021-04-03T03:15:48.963 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:15:49.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.015+0000 7fe85edf4700 1 Processor -- start 2021-04-03T03:15:49.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.015+0000 7fe85edf4700 1 -- start start 2021-04-03T03:15:49.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.015+0000 7fe85edf4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8581447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.015+0000 7fe85edf4700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe8580b17b0 0x7fe858144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.015+0000 7fe85edf4700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580607d0 con 0x7fe8580add50 2021-04-03T03:15:49.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.015+0000 7fe85edf4700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8581451c0 con 0x7fe8580b17b0 2021-04-03T03:15:49.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.015+0000 7fe85edf4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85814b090 con 0x7fe8580603f0 2021-04-03T03:15:49.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85e5f3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe8580b17b0 0x7fe858144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85ddf2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8581447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85e5f3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe8580b17b0 0x7fe858144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:45132/0 (socket says 172.21.2.111:45132) 2021-04-03T03:15:49.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85ddf2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8581447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:38770/0 (socket says 172.21.2.111:38770) 2021-04-03T03:15:49.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85e5f3700 1 -- 172.21.2.111:0/3462651823 learned_addr learned my addr 172.21.2.111:0/3462651823 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:49.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85d5f1700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fe8580add50 0x7fe858147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:40520/0 (socket says 172.21.2.111:40520) 2021-04-03T03:15:49.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85ddf2700 1 -- 172.21.2.111:0/3462651823 >> v2:172.21.2.115:3300/0 conn(0x7fe8580b17b0 msgr2=0x7fe858144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:49.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85ddf2700 1 --2- 172.21.2.111:0/3462651823 >> v2:172.21.2.115:3300/0 conn(0x7fe8580b17b0 0x7fe858144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85ddf2700 1 -- 172.21.2.111:0/3462651823 >> v1:172.21.2.115:6789/0 conn(0x7fe8580add50 legacy=0x7fe858147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85ddf2700 1 -- 172.21.2.111:0/3462651823 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85814b330 con 0x7fe8580603f0 2021-04-03T03:15:49.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85e5f3700 1 --2- 172.21.2.111:0/3462651823 >> v2:172.21.2.115:3300/0 conn(0x7fe8580b17b0 0x7fe858144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:15:49.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85ddf2700 1 --2- 172.21.2.111:0/3462651823 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8581447e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe854002f30 tx=0x7fe854006dd0).ready entity=mon.2 client_cookie=ec2748095d3b739e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85cdf0700 1 -- 172.21.2.111:0/3462651823 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe854009760 con 0x7fe8580603f0 2021-04-03T03:15:49.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85cdf0700 1 -- 172.21.2.111:0/3462651823 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8540098c0 con 0x7fe8580603f0 2021-04-03T03:15:49.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85cdf0700 1 -- 172.21.2.111:0/3462651823 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe854009c20 con 0x7fe8580603f0 2021-04-03T03:15:49.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 -- 172.21.2.111:0/3462651823 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 msgr2=0x7fe8581447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 --2- 172.21.2.111:0/3462651823 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8581447e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe854002f30 tx=0x7fe854006dd0).stop 2021-04-03T03:15:49.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 -- 172.21.2.111:0/3462651823 shutdown_connections 2021-04-03T03:15:49.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 --2- 172.21.2.111:0/3462651823 >> v2:172.21.2.115:3300/0 conn(0x7fe8580b17b0 0x7fe858144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 --2- 172.21.2.111:0/3462651823 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8581447e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 -- 172.21.2.111:0/3462651823 >> 172.21.2.111:0/3462651823 conn(0x7fe8580b61c0 msgr2=0x7fe85805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:49.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 -- 172.21.2.111:0/3462651823 shutdown_connections 2021-04-03T03:15:49.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.019+0000 7fe85edf4700 1 -- 172.21.2.111:0/3462651823 wait complete. 2021-04-03T03:15:49.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 Processor -- start 2021-04-03T03:15:49.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 -- start start 2021-04-03T03:15:49.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8580b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8580add50 0x7fe8580a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8580b17b0 0x7fe8580a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580a8240 con 0x7fe8580add50 2021-04-03T03:15:49.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580b4810 con 0x7fe8580b17b0 2021-04-03T03:15:49.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580a8430 con 0x7fe8580603f0 2021-04-03T03:15:49.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8580add50 0x7fe8580a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8580add50 0x7fe8580a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55430/0 (socket says 172.21.2.111:55430) 2021-04-03T03:15:49.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 -- 172.21.2.111:0/2887748928 learned_addr learned my addr 172.21.2.111:0/2887748928 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:49.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85ddf2700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8580b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85e5f3700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8580b17b0 0x7fe8580a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 -- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 msgr2=0x7fe8580b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8580b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 -- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8580b17b0 msgr2=0x7fe8580a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8580b17b0 0x7fe8580a99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 -- 172.21.2.111:0/2887748928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8581480b0 con 0x7fe8580add50 2021-04-03T03:15:49.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85e5f3700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8580b17b0 0x7fe8580a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:49.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85d5f1700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8580add50 0x7fe8580a7240 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fe8580b5600 tx=0x7fe84c007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe84affd700 1 -- 172.21.2.111:0/2887748928 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84c007320 con 0x7fe8580add50 2021-04-03T03:15:49.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe84affd700 1 -- 172.21.2.111:0/2887748928 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c005530 con 0x7fe8580add50 2021-04-03T03:15:49.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe84affd700 1 -- 172.21.2.111:0/2887748928 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84c005890 con 0x7fe8580add50 2021-04-03T03:15:49.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe858147380 con 0x7fe8580add50 2021-04-03T03:15:49.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580abfc0 con 0x7fe8580add50 2021-04-03T03:15:49.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8580b4950 con 0x7fe8580add50 2021-04-03T03:15:49.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe84affd700 1 -- 172.21.2.111:0/2887748928 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe84c010030 con 0x7fe8580add50 2021-04-03T03:15:49.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe84affd700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe838057000 0x7fe8380594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe84affd700 1 -- 172.21.2.111:0/2887748928 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe84c06fdf0 con 0x7fe8580add50 2021-04-03T03:15:49.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85ddf2700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe838057000 0x7fe8380594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.023+0000 7fe85ddf2700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe838057000 0x7fe8380594c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fe854016000 tx=0x7fe854015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.027+0000 7fe84affd700 1 -- 172.21.2.111:0/2887748928 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe84c045180 con 0x7fe8580add50 2021-04-03T03:15:49.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.119+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe858057660 con 0x7fe8580add50 2021-04-03T03:15:49.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe84affd700 1 -- 172.21.2.111:0/2887748928 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe84c0487a0 con 0x7fe8580add50 2021-04-03T03:15:49.126 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:49.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe838057000 msgr2=0x7fe8380594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe838057000 0x7fe8380594c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fe854016000 tx=0x7fe854015000).stop 2021-04-03T03:15:49.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8580add50 msgr2=0x7fe8580a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8580add50 0x7fe8580a7240 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fe8580b5600 tx=0x7fe84c007bf0).stop 2021-04-03T03:15:49.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 shutdown_connections 2021-04-03T03:15:49.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe838057000 0x7fe8380594c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe8580b17b0 0x7fe8580a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe8580603f0 0x7fe8580b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 --2- 172.21.2.111:0/2887748928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe8580add50 0x7fe8580a7240 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 >> 172.21.2.111:0/2887748928 conn(0x7fe8580b61c0 msgr2=0x7fe8580b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:49.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 shutdown_connections 2021-04-03T03:15:49.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.123+0000 7fe85edf4700 1 -- 172.21.2.111:0/2887748928 wait complete. 2021-04-03T03:15:49.134 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:15:49.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 Processor -- start 2021-04-03T03:15:49.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- start start 2021-04-03T03:15:49.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52580603f0 0x7f52581447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f52580add50 0x7f5258144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580b17b0 0x7f5258147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52581451c0 con 0x7f52580603f0 2021-04-03T03:15:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580607d0 con 0x7f52580add50 2021-04-03T03:15:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525814b090 con 0x7f52580b17b0 2021-04-03T03:15:49.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52580603f0 0x7f52581447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f52580603f0 0x7f52581447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55436/0 (socket says 172.21.2.111:55436) 2021-04-03T03:15:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 -- 172.21.2.111:0/2485052472 learned_addr learned my addr 172.21.2.111:0/2485052472 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 -- 172.21.2.111:0/2485052472 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580b17b0 msgr2=0x7f5258147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 --2- 172.21.2.111:0/2485052472 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580b17b0 0x7f5258147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 -- 172.21.2.111:0/2485052472 >> v2:172.21.2.115:3300/0 conn(0x7f52580add50 msgr2=0x7f5258144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:49.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 --2- 172.21.2.111:0/2485052472 >> v2:172.21.2.115:3300/0 conn(0x7f52580add50 0x7f5258144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 -- 172.21.2.111:0/2485052472 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525814b330 con 0x7f52580603f0 2021-04-03T03:15:49.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525d9f8700 1 --2- 172.21.2.111:0/2485052472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580603f0 0x7f52581447e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f5254002f30 tx=0x7f5254006e90).ready entity=mon.0 client_cookie=e3fce047cde12729 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525c9f6700 1 -- 172.21.2.111:0/2485052472 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5254009920 con 0x7f52580603f0 2021-04-03T03:15:49.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525c9f6700 1 -- 172.21.2.111:0/2485052472 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5254009a80 con 0x7f52580603f0 2021-04-03T03:15:49.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525c9f6700 1 -- 172.21.2.111:0/2485052472 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5254009de0 con 0x7f52580603f0 2021-04-03T03:15:49.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- 172.21.2.111:0/2485052472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580603f0 msgr2=0x7f52581447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2485052472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580603f0 0x7f52581447e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f5254002f30 tx=0x7f5254006e90).stop 2021-04-03T03:15:49.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- 172.21.2.111:0/2485052472 shutdown_connections 2021-04-03T03:15:49.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2485052472 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580b17b0 0x7f5258147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2485052472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580603f0 0x7f52581447e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2485052472 >> v2:172.21.2.115:3300/0 conn(0x7f52580add50 0x7f5258144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- 172.21.2.111:0/2485052472 >> 172.21.2.111:0/2485052472 conn(0x7f52580b61c0 msgr2=0x7f525805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:49.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- 172.21.2.111:0/2485052472 shutdown_connections 2021-04-03T03:15:49.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.187+0000 7f525e9fa700 1 -- 172.21.2.111:0/2485052472 wait complete. 2021-04-03T03:15:49.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 Processor -- start 2021-04-03T03:15:49.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 -- start start 2021-04-03T03:15:49.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580603f0 0x7f5258142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52580add50 0x7f52581568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580b17b0 0x7f525815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580a8920 con 0x7f52580b17b0 2021-04-03T03:15:49.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5258067fa0 con 0x7f52580add50 2021-04-03T03:15:49.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580a9c50 con 0x7f52580603f0 2021-04-03T03:15:49.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580b17b0 0x7f525815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580b17b0 0x7f525815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55442/0 (socket says 172.21.2.111:55442) 2021-04-03T03:15:49.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 -- 172.21.2.111:0/2622229082 learned_addr learned my addr 172.21.2.111:0/2622229082 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:49.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 -- 172.21.2.111:0/2622229082 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580603f0 msgr2=0x7f5258142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580603f0 0x7f5258142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 -- 172.21.2.111:0/2622229082 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52580add50 msgr2=0x7f52581568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52580add50 0x7f52581568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 -- 172.21.2.111:0/2622229082 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580a9eb0 con 0x7f52580b17b0 2021-04-03T03:15:49.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e1f9700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580b17b0 0x7f525815a630 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f52500032b0 tx=0x7f52500090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f524affd700 1 -- 172.21.2.111:0/2622229082 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f525000c900 con 0x7f52580b17b0 2021-04-03T03:15:49.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f524affd700 1 -- 172.21.2.111:0/2622229082 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525000ca60 con 0x7f52580b17b0 2021-04-03T03:15:49.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f524affd700 1 -- 172.21.2.111:0/2622229082 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f525000cdc0 con 0x7f52580b17b0 2021-04-03T03:15:49.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5258159ef0 con 0x7f52580b17b0 2021-04-03T03:15:49.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5258147900 con 0x7f52580b17b0 2021-04-03T03:15:49.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525815a320 con 0x7f52580b17b0 2021-04-03T03:15:49.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f524affd700 1 -- 172.21.2.111:0/2622229082 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5250015470 con 0x7f52580b17b0 2021-04-03T03:15:49.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f524affd700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52380570a0 0x7f5238059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f524affd700 1 -- 172.21.2.111:0/2622229082 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f52500700e0 con 0x7f52580b17b0 2021-04-03T03:15:49.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525d9f8700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52380570a0 0x7f5238059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.191+0000 7f525d9f8700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52380570a0 0x7f5238059560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f5254002f30 tx=0x7f5254015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.195+0000 7f524affd700 1 -- 172.21.2.111:0/2622229082 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5250045470 con 0x7f52580b17b0 2021-04-03T03:15:49.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.287+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5258057660 con 0x7f52580b17b0 2021-04-03T03:15:49.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.287+0000 7f524affd700 1 -- 172.21.2.111:0/2622229082 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5250048a90 con 0x7f52580b17b0 2021-04-03T03:15:49.293 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:15:49.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52380570a0 msgr2=0x7f5238059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52380570a0 0x7f5238059560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f5254002f30 tx=0x7f5254015000).stop 2021-04-03T03:15:49.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580b17b0 msgr2=0x7f525815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580b17b0 0x7f525815a630 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f52500032b0 tx=0x7f52500090f0).stop 2021-04-03T03:15:49.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 shutdown_connections 2021-04-03T03:15:49.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f52580b17b0 0x7f525815a630 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f52580603f0 0x7f5258142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f52380570a0 0x7f5238059560 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 --2- 172.21.2.111:0/2622229082 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f52580add50 0x7f52581568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 >> 172.21.2.111:0/2622229082 conn(0x7f52580b61c0 msgr2=0x7f52580b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:49.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 shutdown_connections 2021-04-03T03:15:49.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.291+0000 7f525e9fa700 1 -- 172.21.2.111:0/2622229082 wait complete. 2021-04-03T03:15:49.300 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:15:49.301 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:49.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.351+0000 7f43d2cd5700 1 Processor -- start 2021-04-03T03:15:49.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.351+0000 7f43d2cd5700 1 -- start start 2021-04-03T03:15:49.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 0x7f43cc0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f43cc0b0c90 0x7f43cc147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc0607d0 con 0x7f43cc14b700 2021-04-03T03:15:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc0af400 con 0x7f43cc0b0c90 2021-04-03T03:15:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0af6a0 con 0x7f43cc0603f0 2021-04-03T03:15:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f43cc14b700 0x7f43cc0b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56242/0 (socket says 172.21.2.111:56242) 2021-04-03T03:15:49.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 -- 172.21.2.111:0/2554340708 learned_addr learned my addr 172.21.2.111:0/2554340708 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866136417 0 0) 0x7f43cc0607d0 con 0x7f43cc14b700 2021-04-03T03:15:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b4003240 con 0x7f43cc14b700 2021-04-03T03:15:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d24d4700 1 --2- 172.21.2.111:0/2554340708 >> v2:172.21.2.115:3300/0 conn(0x7f43cc0b0c90 0x7f43cc147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d1cd3700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 0x7f43cc0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3417010584 0 0) 0x7f43b4003240 con 0x7f43cc14b700 2021-04-03T03:15:49.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 msgr2=0x7f43cc0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:49.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 0x7f43cc0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 >> v2:172.21.2.115:3300/0 conn(0x7f43cc0b0c90 msgr2=0x7f43cc147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:49.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 --2- 172.21.2.111:0/2554340708 >> v2:172.21.2.115:3300/0 conn(0x7f43cc0b0c90 0x7f43cc147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0afb40 con 0x7f43cc14b700 2021-04-03T03:15:49.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f43bc0016d0 con 0x7f43cc14b700 2021-04-03T03:15:49.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 >> v1:172.21.2.111:6789/0 conn(0x7f43cc14b700 legacy=0x7f43cc0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 0x7f43b40037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43b4003d10 0x7f43b40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43b40049d0 0x7f43b4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0af400 con 0x7f43b4003d10 2021-04-03T03:15:49.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0af6a0 con 0x7f43b40049d0 2021-04-03T03:15:49.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b40042d0 con 0x7f43cc0603f0 2021-04-03T03:15:49.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43b4003d10 0x7f43b40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d1cd3700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 0x7f43b40037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 -- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 msgr2=0x7f43b40037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:49.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 0x7f43b40037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 -- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43b40049d0 msgr2=0x7f43b4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:49.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43b40049d0 0x7f43b4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 -- 172.21.2.111:0/2554340708 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4003240 con 0x7f43b4003d10 2021-04-03T03:15:49.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d1cd3700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc0603f0 0x7f43b40037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:49.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d14d2700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43b4003d10 0x7f43b40044e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f43bc0013e0 tx=0x7f43cc0ae670).ready entity=mon.0 client_cookie=2283278e1727bfbf server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43bc00ce80 con 0x7f43b4003d10 2021-04-03T03:15:49.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc0070d0 con 0x7f43b4003d10 2021-04-03T03:15:49.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d0cd1700 1 -- 172.21.2.111:0/2554340708 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43bc007430 con 0x7f43b4003d10 2021-04-03T03:15:49.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2554340708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43b4003d10 msgr2=0x7f43b40044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 --2- 172.21.2.111:0/2554340708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43b4003d10 0x7f43b40044e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f43bc0013e0 tx=0x7f43cc0ae670).stop 2021-04-03T03:15:49.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d24d4700 1 -- 172.21.2.111:0/2554340708 reap_dead start 2021-04-03T03:15:49.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2554340708 shutdown_connections 2021-04-03T03:15:49.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2554340708 >> 172.21.2.111:0/2554340708 conn(0x7f43cc0b61c0 msgr2=0x7f43cc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:49.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2554340708 shutdown_connections 2021-04-03T03:15:49.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.355+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2554340708 wait complete. 2021-04-03T03:15:49.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 Processor -- start 2021-04-03T03:15:49.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 -- start start 2021-04-03T03:15:49.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43cc0a81d0 0x7f43cc0a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43cc0ab760 0x7f43cc0654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc063aa0 0x7f43cc0659b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0aac80 con 0x7f43cc0ab760 2021-04-03T03:15:49.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0a85b0 con 0x7f43cc0a81d0 2021-04-03T03:15:49.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc062b50 con 0x7f43cc063aa0 2021-04-03T03:15:49.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43cc0ab760 0x7f43cc0654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43cc0ab760 0x7f43cc0654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55462/0 (socket says 172.21.2.111:55462) 2021-04-03T03:15:49.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 -- 172.21.2.111:0/2904623564 learned_addr learned my addr 172.21.2.111:0/2904623564 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:49.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d1cd3700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43cc0a81d0 0x7f43cc0a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 -- 172.21.2.111:0/2904623564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc063aa0 msgr2=0x7f43cc0659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc063aa0 0x7f43cc0659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 -- 172.21.2.111:0/2904623564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43cc0a81d0 msgr2=0x7f43cc0a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43cc0a81d0 0x7f43cc0a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 -- 172.21.2.111:0/2904623564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0aae10 con 0x7f43cc0ab760 2021-04-03T03:15:49.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d14d2700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43cc0ab760 0x7f43cc0654c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f43bc0013e0 tx=0x7f43bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43baffd700 1 -- 172.21.2.111:0/2904623564 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43bc004e90 con 0x7f43cc0ab760 2021-04-03T03:15:49.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43baffd700 1 -- 172.21.2.111:0/2904623564 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc018070 con 0x7f43cc0ab760 2021-04-03T03:15:49.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43baffd700 1 -- 172.21.2.111:0/2904623564 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43bc013640 con 0x7f43cc0ab760 2021-04-03T03:15:49.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc065ea0 con 0x7f43cc0ab760 2021-04-03T03:15:49.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc1599b0 con 0x7f43cc0ab760 2021-04-03T03:15:49.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc057660 con 0x7f43cc0ab760 2021-04-03T03:15:49.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43baffd700 1 -- 172.21.2.111:0/2904623564 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f43bc024020 con 0x7f43cc0ab760 2021-04-03T03:15:49.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43baffd700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43ac056ea0 0x7f43ac059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:49.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43baffd700 1 -- 172.21.2.111:0/2904623564 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f43bc070ff0 con 0x7f43cc0ab760 2021-04-03T03:15:49.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d1cd3700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43ac056ea0 0x7f43ac059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:49.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.359+0000 7f43d1cd3700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43ac056ea0 0x7f43ac059360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f43c8009000 tx=0x7f43c8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:49.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.363+0000 7f43baffd700 1 -- 172.21.2.111:0/2904623564 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f43bc046310 con 0x7f43cc0ab760 2021-04-03T03:15:49.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.487+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43cc065140 con 0x7f43cc0ab760 2021-04-03T03:15:49.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.487+0000 7f43baffd700 1 -- 172.21.2.111:0/2904623564 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f43bc049930 con 0x7f43cc0ab760 2021-04-03T03:15:49.493 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:49.493 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":38,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3837436656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3837436656},{"type":"v1","addr":"172.21.2.115:6833","nonce":3837436656}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:44.064474+0000","modified":"2021-04-03T03:15:45.067347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/13282524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":13282524},{"type":"v1","addr":"172.21.2.111:6835","nonce":13282524}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-04-03T03:15:49.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43ac056ea0 msgr2=0x7f43ac059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43ac056ea0 0x7f43ac059360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f43c8009000 tx=0x7f43c8008040).stop 2021-04-03T03:15:49.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43cc0ab760 msgr2=0x7f43cc0654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:49.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43cc0ab760 0x7f43cc0654c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f43bc0013e0 tx=0x7f43bc015000).stop 2021-04-03T03:15:49.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 shutdown_connections 2021-04-03T03:15:49.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f43cc063aa0 0x7f43cc0659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:49.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f43ac056ea0 0x7f43ac059360 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f43cc0a81d0 0x7f43cc0a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 --2- 172.21.2.111:0/2904623564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f43cc0ab760 0x7f43cc0654c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:49.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 >> 172.21.2.111:0/2904623564 conn(0x7f43cc0b61c0 msgr2=0x7f43cc149ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:49.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 shutdown_connections 2021-04-03T03:15:49.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:49.491+0000 7f43d2cd5700 1 -- 172.21.2.111:0/2904623564 wait complete. 2021-04-03T03:15:49.497 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 38 2021-04-03T03:15:49.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:15:44.064474+0000', 'modified': '2021-04-03T03:15:45.067347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6879}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6879': {'gid': 6879, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/13282524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 13282524}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 13282524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:15:49.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:15:49.704 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:15:49.777 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":6879,"want_state":"up:active","state":"up:active","rank_uptime":5.7051246290000002,"mdsmap_epoch":38,"osdmap_epoch":93,"osdmap_epoch_barrier":92,"uptime":9.8543431160000008} 2021-04-03T03:15:49.777 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 6879, "mdsmap_epoch": 38, "osdmap_epoch": 93, "osdmap_epoch_barrier": 92, "rank_uptime": 5.705124629, "state": "up:active", "uptime": 9.854343116, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:15:49.778 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:15:49.778 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:15:49.778 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:15:49.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:49.779 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:49.789 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:15:49.843 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:49.844 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:49.891 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:49.892 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:49.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:49.893 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:15:49.945 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:15:49.945 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:49.991 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:15:50.002 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:15:50.002 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:15:50.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:15:50.023+0000 7f7b003d10c0 -1 init, newargv = 0x5652263ae1b0 newargc=9 2021-04-03T03:15:50.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15231]: starting ceph client 2021-04-03T03:15:50.037 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:15:50.037 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:15:50.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:50.038 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:15:50.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15231]: starting fuse 2021-04-03T03:15:50.042 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:15:50.042 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:15:50.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:50.043 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:15:50.044 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:15:50.044 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:15:50.045 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:15:50.045 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:15:50.045 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:50.046 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:15:50.047 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:15:50.048 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:15:50.048 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:50.048 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:15:50.049 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:15:50.049 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:15:50.050 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:15:50.050 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:50.050 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:15:50.051 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:15:50.051 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:15:50.052 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:15:50.052 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:15:50.052 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:15:50.053 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:15:50.053 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:15:50.134 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15231.asok 2021-04-03T03:15:50.138 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15231.asok 2021-04-03T03:15:50.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:50.140 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15231.asok status 2021-04-03T03:15:50.209 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:15:50.209 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:15:50.209 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:15:50.209 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:15:50.210 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:15:50.210 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:15:50.210 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:15:50.210 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15231", 2021-04-03T03:15:50.211 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:15:50.211 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:50.211 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:15:50.211 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:15:50.212 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 4859, 2021-04-03T03:15:50.212 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:15:50.212 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:15:50.212 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:15:50.212 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 4859 2021-04-03T03:15:50.213 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:50.213 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:15:50.213 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:15:50.213 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:15:50.213 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 1113893654 2021-04-03T03:15:50.214 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:15:50.214 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:50.214 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:15:50.214 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:15:50.215 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:15:50.215 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 1113893654 2021-04-03T03:15:50.215 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:15:50.215 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.4859 172.21.2.128:0/1113893654", 2021-04-03T03:15:50.215 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/1113893654", 2021-04-03T03:15:50.216 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:15:50.216 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 38, 2021-04-03T03:15:50.216 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 93, 2021-04-03T03:15:50.216 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:15:50.216 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:15:50.217 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:15:50.217 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:15:50.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:50.218 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:50.219 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:50.225 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:15:50.226 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:50.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:50.227 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:50.292 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-04-03T03:15:50.293 DEBUG:teuthology.orchestra.run.gibba026:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2021-04-03T03:15:50.325 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:15:50.326 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:15:50.326 DEBUG:teuthology.orchestra.run.gibba026:> import stat 2021-04-03T03:15:50.327 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:15:50.327 DEBUG:teuthology.orchestra.run.gibba026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2021-04-03T03:15:50.327 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:15:50.442 INFO:teuthology.orchestra.run.gibba026.stdout:1099511627776 2021-04-03T03:15:50.644 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:50.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f3768adc700 1 Processor -- start 2021-04-03T03:15:50.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f3768adc700 1 -- start start 2021-04-03T03:15:50.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f3768adc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c006bc0 0x7f375c042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f3768adc700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c0dc6e0 con 0x7f375c0dbb90 2021-04-03T03:15:50.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f3768adc700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c042010 con 0x7f375c03d360 2021-04-03T03:15:50.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f3768adc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0d9560 con 0x7f375c006bc0 2021-04-03T03:15:50.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f37637fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f375c0dbb90 0x7f375c0064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56262/0 (socket says 172.21.2.111:56262) 2021-04-03T03:15:50.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f37637fe700 1 -- 172.21.2.111:0/1317959640 learned_addr learned my addr 172.21.2.111:0/1317959640 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:50.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454370570 0 0) 0x7f375c0dc6e0 con 0x7f375c0dbb90 2021-04-03T03:15:50.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.703+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3748003240 con 0x7f375c0dbb90 2021-04-03T03:15:50.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c006bc0 0x7f375c042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:50.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3519086485 0 0) 0x7f3748003240 con 0x7f375c0dbb90 2021-04-03T03:15:50.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c006bc0 msgr2=0x7f375c042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:50.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c006bc0 0x7f375c042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:50.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 >> v1:172.21.2.115:6789/0 conn(0x7f375c03d360 legacy=0x7f375c0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:50.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c0422b0 con 0x7f375c0dbb90 2021-04-03T03:15:50.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f37580016d0 con 0x7f375c0dbb90 2021-04-03T03:15:50.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 >> v1:172.21.2.111:6789/0 conn(0x7f375c0dbb90 legacy=0x7f375c0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:50.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3748003760 0x7f3748003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f37480044e0 0x7f3748004920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3748004e10 0x7f374800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0d9560 con 0x7f3748004e10 2021-04-03T03:15:50.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748005380 con 0x7f3748003760 2021-04-03T03:15:50.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3748004240 con 0x7f37480044e0 2021-04-03T03:15:50.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3748004e10 0x7f374800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:50.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 -- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f37480044e0 msgr2=0x7f3748004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:50.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f37480044e0 0x7f3748004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:50.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 -- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3748003760 msgr2=0x7f3748003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:50.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3748003760 0x7f3748003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:50.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 -- 172.21.2.111:0/1317959640 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3748003240 con 0x7f3748004e10 2021-04-03T03:15:50.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3748004e10 0x7f374800b730 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f375c0087e0 tx=0x7f376405ad70).ready entity=mon.0 client_cookie=1ee9778567a63e73 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:50.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f376405df60 con 0x7f3748004e10 2021-04-03T03:15:50.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37640699f0 con 0x7f3748004e10 2021-04-03T03:15:50.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f37627fc700 1 -- 172.21.2.111:0/1317959640 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f376405b040 con 0x7f3748004e10 2021-04-03T03:15:50.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- 172.21.2.111:0/1317959640 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3748004e10 msgr2=0x7f374800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:50.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 --2- 172.21.2.111:0/1317959640 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3748004e10 0x7f374800b730 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f375c0087e0 tx=0x7f376405ad70).stop 2021-04-03T03:15:50.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3763fff700 1 -- 172.21.2.111:0/1317959640 reap_dead start 2021-04-03T03:15:50.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- 172.21.2.111:0/1317959640 shutdown_connections 2021-04-03T03:15:50.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- 172.21.2.111:0/1317959640 >> 172.21.2.111:0/1317959640 conn(0x7f375c0462c0 msgr2=0x7f375c0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:50.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- 172.21.2.111:0/1317959640 shutdown_connections 2021-04-03T03:15:50.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- 172.21.2.111:0/1317959640 wait complete. 2021-04-03T03:15:50.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 Processor -- start 2021-04-03T03:15:50.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- start start 2021-04-03T03:15:50.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c0a0560 0x7f375c0a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f375c0a0940 0x7f375c0e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f375c0e6fa0 0x7f375c0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0a3150 con 0x7f375c0a0940 2021-04-03T03:15:50.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0a0080 con 0x7f375c0e6fa0 2021-04-03T03:15:50.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3768adc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0a0e40 con 0x7f375c0a0560 2021-04-03T03:15:50.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f375c0a0940 0x7f375c0e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:50.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f375c0a0940 0x7f375c0e6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55484/0 (socket says 172.21.2.111:55484) 2021-04-03T03:15:50.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 -- 172.21.2.111:0/3530942855 learned_addr learned my addr 172.21.2.111:0/3530942855 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:50.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 -- 172.21.2.111:0/3530942855 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c0a0560 msgr2=0x7f375c0a2c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:50.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c0a0560 0x7f375c0a2c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:50.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 -- 172.21.2.111:0/3530942855 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f375c0e6fa0 msgr2=0x7f375c0d5750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:15:50.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f375c0e6fa0 0x7f375c0d5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:50.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.707+0000 7f3762ffd700 1 -- 172.21.2.111:0/3530942855 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c03c250 con 0x7f375c0a0940 2021-04-03T03:15:50.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3762ffd700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f375c0a0940 0x7f375c0e6ab0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f375c03b520 tx=0x7f3750006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:50.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3760ff9700 1 -- 172.21.2.111:0/3530942855 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3750016070 con 0x7f375c0a0940 2021-04-03T03:15:50.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3760ff9700 1 -- 172.21.2.111:0/3530942855 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375000c990 con 0x7f375c0a0940 2021-04-03T03:15:50.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c09fde0 con 0x7f375c0a0940 2021-04-03T03:15:50.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3760ff9700 1 -- 172.21.2.111:0/3530942855 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f375000eee0 con 0x7f375c0a0940 2021-04-03T03:15:50.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c03b0c0 con 0x7f375c0a0940 2021-04-03T03:15:50.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c004da0 con 0x7f375c0a0940 2021-04-03T03:15:50.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3760ff9700 1 -- 172.21.2.111:0/3530942855 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f375000ccf0 con 0x7f375c0a0940 2021-04-03T03:15:50.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3760ff9700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f374c056f20 0x7f374c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:50.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3760ff9700 1 -- 172.21.2.111:0/3530942855 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3750070380 con 0x7f375c0a0940 2021-04-03T03:15:50.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f37637fe700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f374c056f20 0x7f374c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:50.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f37637fe700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f374c056f20 0x7f374c0593e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f37580013e0 tx=0x7f375c03e7f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:50.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.711+0000 7f3760ff9700 1 -- 172.21.2.111:0/3530942855 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3750045660 con 0x7f375c0a0940 2021-04-03T03:15:50.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f375c046ff0 con 0x7f375c0a0940 2021-04-03T03:15:50.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3760ff9700 1 -- 172.21.2.111:0/3530942855 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f3750048c80 con 0x7f375c0a0940 2021-04-03T03:15:50.843 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:50.843 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":38,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3837436656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3837436656},{"type":"v1","addr":"172.21.2.115:6833","nonce":3837436656}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:44.064474+0000","modified":"2021-04-03T03:15:45.067347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/13282524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":13282524},{"type":"v1","addr":"172.21.2.111:6835","nonce":13282524}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-04-03T03:15:50.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f374c056f20 msgr2=0x7f374c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:50.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f374c056f20 0x7f374c0593e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f37580013e0 tx=0x7f375c03e7f0).stop 2021-04-03T03:15:50.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f375c0a0940 msgr2=0x7f375c0e6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:50.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f375c0a0940 0x7f375c0e6ab0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f375c03b520 tx=0x7f3750006dd0).stop 2021-04-03T03:15:50.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 shutdown_connections 2021-04-03T03:15:50.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f374c056f20 0x7f374c0593e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:50.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f375c0e6fa0 0x7f375c0d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:50.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f375c0a0560 0x7f375c0a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:50.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 --2- 172.21.2.111:0/3530942855 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f375c0a0940 0x7f375c0e6ab0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:50.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 >> 172.21.2.111:0/3530942855 conn(0x7f375c0462c0 msgr2=0x7f375c044f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:50.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 shutdown_connections 2021-04-03T03:15:50.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:50.839+0000 7f3768adc700 1 -- 172.21.2.111:0/3530942855 wait complete. 2021-04-03T03:15:50.847 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 38 2021-04-03T03:15:50.851 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 0x10000000000 2021-04-03T03:15:50.915 INFO:teuthology.orchestra.run.gibba011.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2021-04-03T03:15:50.313491+0000","btime":"2021-04-03T03:15:50.313491+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2021-04-03T03:15:50.313491+0000","atime":"2021-04-03T03:15:50.313491+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5230,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":5230,"want_loner":5230,"mds_caps_wanted":[]} 2021-04-03T03:15:50.916 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2021-04-03T03:15:50.313491+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2021-04-03T03:15:50.313491+0000", "change_attr": 0, "client_caps": [ { "client_id": 5230, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2021-04-03T03:15:50.313491+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5230, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2021-04-03T03:15:50.313491+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 5230, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2021-04-03T03:15:50.916 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:15:50.917 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:50.923 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:50.936 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:50.947 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:50.960 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:50.971 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:15:50.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:50.972 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:50.973 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:50.993 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:15:50.993 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:50.994 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:15:50.994 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:15:50.995 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:51.069 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15703]: fuse finished with error 0 and tester_r 0 2021-04-03T03:15:51.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:51.072 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:51.073 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:51.120 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:15:51.121 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:15:51.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:51.154 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:51.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:51.200 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:51.201 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:51.211 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:15:51.213 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:51.213 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:15:51.213 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:15:51.214 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:51.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15231]: fuse finished with error 0 and tester_r 0 2021-04-03T03:15:51.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:51.305 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:51.305 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:51.314 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:15:51.315 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:15:51.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:51.426 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:51.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:51.474 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:51.474 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:51.483 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:51.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:51.485 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:15:51.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:15:51.486 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:51.535 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:51.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:51.537 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:15:51.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 Processor -- start 2021-04-03T03:15:51.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 -- start start 2021-04-03T03:15:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f877c03f120 0x7f877c005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 0x7f877c0d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f877c042470 0x7f877c0d8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c03faa0 con 0x7f877c042470 2021-04-03T03:15:51.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c03dca0 con 0x7f877c03f120 2021-04-03T03:15:51.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c03d840 con 0x7f877c041fe0 2021-04-03T03:15:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f877c042470 0x7f877c0d8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f877c042470 0x7f877c0d8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55492/0 (socket says 172.21.2.111:55492) 2021-04-03T03:15:51.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 -- 172.21.2.111:0/2679654051 learned_addr learned my addr 172.21.2.111:0/2679654051 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 -- 172.21.2.111:0/2679654051 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 msgr2=0x7f877c0d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 --2- 172.21.2.111:0/2679654051 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 0x7f877c0d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 -- 172.21.2.111:0/2679654051 >> v2:172.21.2.115:3300/0 conn(0x7f877c03f120 msgr2=0x7f877c005640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:51.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 --2- 172.21.2.111:0/2679654051 >> v2:172.21.2.115:3300/0 conn(0x7f877c03f120 0x7f877c005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 -- 172.21.2.111:0/2679654051 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c005b30 con 0x7f877c042470 2021-04-03T03:15:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8789a1b700 1 --2- 172.21.2.111:0/2679654051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0d8e90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f877c03d2c0 tx=0x7f878405ad70).ready entity=mon.0 client_cookie=5128249f80598e73 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8777fff700 1 -- 172.21.2.111:0/2679654051 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878405bda0 con 0x7f877c042470 2021-04-03T03:15:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8777fff700 1 -- 172.21.2.111:0/2679654051 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878405df60 con 0x7f877c042470 2021-04-03T03:15:51.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f8777fff700 1 -- 172.21.2.111:0/2679654051 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87840699f0 con 0x7f877c042470 2021-04-03T03:15:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 -- 172.21.2.111:0/2679654051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 msgr2=0x7f877c0d8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.595+0000 7f878a21c700 1 --2- 172.21.2.111:0/2679654051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0d8e90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f877c03d2c0 tx=0x7f878405ad70).stop 2021-04-03T03:15:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- 172.21.2.111:0/2679654051 shutdown_connections 2021-04-03T03:15:51.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 --2- 172.21.2.111:0/2679654051 >> v2:172.21.2.115:3300/0 conn(0x7f877c03f120 0x7f877c005640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 --2- 172.21.2.111:0/2679654051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0d8e90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 --2- 172.21.2.111:0/2679654051 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 0x7f877c0d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- 172.21.2.111:0/2679654051 >> 172.21.2.111:0/2679654051 conn(0x7f877c0462b0 msgr2=0x7f877c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:51.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- 172.21.2.111:0/2679654051 shutdown_connections 2021-04-03T03:15:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- 172.21.2.111:0/2679654051 wait complete. 2021-04-03T03:15:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 Processor -- start 2021-04-03T03:15:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- start start 2021-04-03T03:15:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f877c03f120 0x7f877c0d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 0x7f877c0e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0a0cb0 con 0x7f877c042470 2021-04-03T03:15:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0a1870 con 0x7f877c03f120 2021-04-03T03:15:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c03ae60 con 0x7f877c041fe0 2021-04-03T03:15:51.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55500/0 (socket says 172.21.2.111:55500) 2021-04-03T03:15:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 -- 172.21.2.111:0/1046674608 learned_addr learned my addr 172.21.2.111:0/1046674608 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878921a700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f877c03f120 0x7f877c0d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 -- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 msgr2=0x7f877c0e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 0x7f877c0e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:51.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 -- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f877c03f120 msgr2=0x7f877c0d7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f877c03f120 0x7f877c0d7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 -- 172.21.2.111:0/1046674608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c03b2b0 con 0x7f877c042470 2021-04-03T03:15:51.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878921a700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f877c03f120 0x7f877c0d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:51.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f8789a1b700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0e7fe0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f877c0a19b0 tx=0x7f8784064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:51.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f87767fc700 1 -- 172.21.2.111:0/1046674608 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87840699f0 con 0x7f877c042470 2021-04-03T03:15:51.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f87767fc700 1 -- 172.21.2.111:0/1046674608 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87840441b0 con 0x7f877c042470 2021-04-03T03:15:51.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f87767fc700 1 -- 172.21.2.111:0/1046674608 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878405f4b0 con 0x7f877c042470 2021-04-03T03:15:51.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c03abc0 con 0x7f877c042470 2021-04-03T03:15:51.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c0e8550 con 0x7f877c042470 2021-04-03T03:15:51.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c004d30 con 0x7f877c042470 2021-04-03T03:15:51.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f87767fc700 1 -- 172.21.2.111:0/1046674608 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f878408c020 con 0x7f877c042470 2021-04-03T03:15:51.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f87767fc700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8764056ea0 0x7f8764059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f87767fc700 1 -- 172.21.2.111:0/1046674608 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f87840e0720 con 0x7f877c042470 2021-04-03T03:15:51.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878921a700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8764056ea0 0x7f8764059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.599+0000 7f878921a700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8764056ea0 0x7f8764059360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8778007000 tx=0x7f8778008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:51.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.603+0000 7f87767fc700 1 -- 172.21.2.111:0/1046674608 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f87840b5b10 con 0x7f877c042470 2021-04-03T03:15:51.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.695+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f877c006d20 con 0x7f877c042470 2021-04-03T03:15:51.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.699+0000 7f87767fc700 1 -- 172.21.2.111:0/1046674608 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f87840b9130 con 0x7f877c042470 2021-04-03T03:15:51.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8764056ea0 msgr2=0x7f8764059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8764056ea0 0x7f8764059360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8778007000 tx=0x7f8778008040).stop 2021-04-03T03:15:51.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 msgr2=0x7f877c0e7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0e7fe0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f877c0a19b0 tx=0x7f8784064030).stop 2021-04-03T03:15:51.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 shutdown_connections 2021-04-03T03:15:51.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f877c042470 0x7f877c0e7fe0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8764056ea0 0x7f8764059360 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f877c03f120 0x7f877c0d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 --2- 172.21.2.111:0/1046674608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f877c041fe0 0x7f877c0e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:51.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 >> 172.21.2.111:0/1046674608 conn(0x7f877c0462b0 msgr2=0x7f877c0dc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:51.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 shutdown_connections 2021-04-03T03:15:51.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.703+0000 7f878a21c700 1 -- 172.21.2.111:0/1046674608 wait complete. 2021-04-03T03:15:51.714 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2021-04-03T03:15:51.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 Processor -- start 2021-04-03T03:15:51.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 -- start start 2021-04-03T03:15:51.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe04c005dc0 0x7fe04c042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c0dc6e0 con 0x7fe04c0dbb90 2021-04-03T03:15:51.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c042010 con 0x7fe04c006bc0 2021-04-03T03:15:51.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c0d9560 con 0x7fe04c005dc0 2021-04-03T03:15:51.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe052ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe04c0dbb90 0x7fe04c0064b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56296/0 (socket says 172.21.2.111:56296) 2021-04-03T03:15:51.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe052ffd700 1 -- 172.21.2.111:0/2849588039 learned_addr learned my addr 172.21.2.111:0/2849588039 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:51.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442546148 0 0) 0x7fe04c0dc6e0 con 0x7fe04c0dbb90 2021-04-03T03:15:51.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe034003240 con 0x7fe04c0dbb90 2021-04-03T03:15:51.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 --2- 172.21.2.111:0/2849588039 >> v2:172.21.2.115:3300/0 conn(0x7fe04c005dc0 0x7fe04c042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2742588317 0 0) 0x7fe034003240 con 0x7fe04c0dbb90 2021-04-03T03:15:51.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 >> v2:172.21.2.115:3300/0 conn(0x7fe04c005dc0 msgr2=0x7fe04c042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:51.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 --2- 172.21.2.111:0/2849588039 >> v2:172.21.2.115:3300/0 conn(0x7fe04c005dc0 0x7fe04c042750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 >> v1:172.21.2.115:6789/0 conn(0x7fe04c006bc0 legacy=0x7fe04c0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0422b0 con 0x7fe04c0dbb90 2021-04-03T03:15:51.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe0440016d0 con 0x7fe04c0dbb90 2021-04-03T03:15:51.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 >> v1:172.21.2.111:6789/0 conn(0x7fe04c0dbb90 legacy=0x7fe04c0064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034003730 0x7fe034003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0340044e0 0x7fe034004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe034004e50 0x7fe03400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0d9560 con 0x7fe034004e50 2021-04-03T03:15:51.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034004260 con 0x7fe0340044e0 2021-04-03T03:15:51.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03400bc20 con 0x7fe034003730 2021-04-03T03:15:51.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe034004e50 0x7fe03400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0527fc700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0340044e0 0x7fe034004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 -- 172.21.2.111:0/2849588039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034003730 msgr2=0x7fe034003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:51.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034003730 0x7fe034003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:51.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 -- 172.21.2.111:0/2849588039 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0340044e0 msgr2=0x7fe034004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:51.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe0340044e0 0x7fe034004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 -- 172.21.2.111:0/2849588039 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0422b0 con 0x7fe034004e50 2021-04-03T03:15:51.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 -- 172.21.2.111:0/2849588039 reap_dead start 2021-04-03T03:15:51.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe0537fe700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe034004e50 0x7fe03400b730 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fe054082000 tx=0x7fe05405ad70).ready entity=mon.0 client_cookie=71215abec515b480 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:51.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe05405df60 con 0x7fe034004e50 2021-04-03T03:15:51.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe054068030 con 0x7fe034004e50 2021-04-03T03:15:51.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe051ffb700 1 -- 172.21.2.111:0/2849588039 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0540699f0 con 0x7fe034004e50 2021-04-03T03:15:51.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 -- 172.21.2.111:0/2849588039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe034004e50 msgr2=0x7fe03400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.771+0000 7fe053fff700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe034004e50 0x7fe03400b730 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fe054082000 tx=0x7fe05405ad70).stop 2021-04-03T03:15:51.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- 172.21.2.111:0/2849588039 shutdown_connections 2021-04-03T03:15:51.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 --2- 172.21.2.111:0/2849588039 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe034004e50 0x7fe03400b730 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- 172.21.2.111:0/2849588039 >> 172.21.2.111:0/2849588039 conn(0x7fe04c0462c0 msgr2=0x7fe04c0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:51.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- 172.21.2.111:0/2849588039 shutdown_connections 2021-04-03T03:15:51.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- 172.21.2.111:0/2849588039 wait complete. 2021-04-03T03:15:51.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 Processor -- start 2021-04-03T03:15:51.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- start start 2021-04-03T03:15:51.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034004e50 0x7fe04c0a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe04c03a780 0x7fe04c03d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe04c0a1740 0x7fe04c03d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0a1c10 con 0x7fe04c03a780 2021-04-03T03:15:51.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0a1e60 con 0x7fe04c0a1740 2021-04-03T03:15:51.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0a2100 con 0x7fe034004e50 2021-04-03T03:15:51.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe04c03a780 0x7fe04c03d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe04c03a780 0x7fe04c03d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55516/0 (socket says 172.21.2.111:55516) 2021-04-03T03:15:51.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 -- 172.21.2.111:0/3909488113 learned_addr learned my addr 172.21.2.111:0/3909488113 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:51.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0537fe700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe04c0a1740 0x7fe04c03d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe052ffd700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034004e50 0x7fe04c0a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 -- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034004e50 msgr2=0x7fe04c0a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034004e50 0x7fe04c0a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 -- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe04c0a1740 msgr2=0x7fe04c03d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:51.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe04c0a1740 0x7fe04c03d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:51.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 -- 172.21.2.111:0/3909488113 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0a25a0 con 0x7fe04c03a780 2021-04-03T03:15:51.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe052ffd700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034004e50 0x7fe04c0a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:51.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe0527fc700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe04c03a780 0x7fe04c03d0a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fe03c00b000 tx=0x7fe03c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:51.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe04bfff700 1 -- 172.21.2.111:0/3909488113 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe03c010490 con 0x7fe04c03a780 2021-04-03T03:15:51.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe04bfff700 1 -- 172.21.2.111:0/3909488113 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c010040 con 0x7fe04c03a780 2021-04-03T03:15:51.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c0a0340 con 0x7fe04c03a780 2021-04-03T03:15:51.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c0ddd90 con 0x7fe04c03a780 2021-04-03T03:15:51.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe04bfff700 1 -- 172.21.2.111:0/3909488113 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe03c0107e0 con 0x7fe04c03a780 2021-04-03T03:15:51.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c004da0 con 0x7fe04c03a780 2021-04-03T03:15:51.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe04bfff700 1 -- 172.21.2.111:0/3909488113 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe03c010940 con 0x7fe04c03a780 2021-04-03T03:15:51.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe04bfff700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe038056f70 0x7fe038059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:51.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe04bfff700 1 -- 172.21.2.111:0/3909488113 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe03c070270 con 0x7fe04c03a780 2021-04-03T03:15:51.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe052ffd700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe038056f70 0x7fe038059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:51.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.775+0000 7fe052ffd700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe038056f70 0x7fe038059430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fe044008000 tx=0x7fe044009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:51.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.779+0000 7fe04bfff700 1 -- 172.21.2.111:0/3909488113 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe03c045550 con 0x7fe04c03a780 2021-04-03T03:15:51.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:51.907+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fe04c046ff0 con 0x7fe04c03a780 2021-04-03T03:15:52.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.115+0000 7fe04bfff700 1 -- 172.21.2.111:0/3909488113 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v306) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe03c048b70 con 0x7fe04c03a780 2021-04-03T03:15:52.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe038056f70 msgr2=0x7fe038059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:52.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe038056f70 0x7fe038059430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fe044008000 tx=0x7fe044009040).stop 2021-04-03T03:15:52.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe04c03a780 msgr2=0x7fe04c03d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:52.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe04c03a780 0x7fe04c03d0a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fe03c00b000 tx=0x7fe03c009040).stop 2021-04-03T03:15:52.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 shutdown_connections 2021-04-03T03:15:52.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe038056f70 0x7fe038059430 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:52.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe04c0a1740 0x7fe04c03d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:52.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe034004e50 0x7fe04c0a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:52.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 --2- 172.21.2.111:0/3909488113 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe04c03a780 0x7fe04c03d0a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:52.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 >> 172.21.2.111:0/3909488113 conn(0x7fe04c0462c0 msgr2=0x7fe04c0d7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:52.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 shutdown_connections 2021-04-03T03:15:52.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.119+0000 7fe053fff700 1 -- 172.21.2.111:0/3909488113 wait complete. 2021-04-03T03:15:52.151 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2021-04-03T03:15:52.151 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:15:52.152 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2021-04-03T03:15:52.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.207+0000 7f359bfff700 1 Processor -- start 2021-04-03T03:15:52.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.207+0000 7f359bfff700 1 -- start start 2021-04-03T03:15:52.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.207+0000 7f359bfff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f359c0603f0 0x7f359c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.207+0000 7f359bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0af400 con 0x7f359c0b0c90 2021-04-03T03:15:52.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.207+0000 7f359bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0af6a0 con 0x7f359c14b700 2021-04-03T03:15:52.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.207+0000 7f359bfff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c060860 con 0x7f359c0603f0 2021-04-03T03:15:52.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f359c0b0c90 0x7f359c147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56320/0 (socket says 172.21.2.111:56320) 2021-04-03T03:15:52.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 -- 172.21.2.111:0/3597057782 learned_addr learned my addr 172.21.2.111:0/3597057782 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:52.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 --2- 172.21.2.111:0/3597057782 >> v2:172.21.2.115:3300/0 conn(0x7f359c0603f0 0x7f359c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:52.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951686816 0 0) 0x7f359c0af400 con 0x7f359c0b0c90 2021-04-03T03:15:52.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f357c003240 con 0x7f359c0b0c90 2021-04-03T03:15:52.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1906864187 0 0) 0x7f357c003240 con 0x7f359c0b0c90 2021-04-03T03:15:52.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 >> v2:172.21.2.115:3300/0 conn(0x7f359c0603f0 msgr2=0x7f359c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:52.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 --2- 172.21.2.111:0/3597057782 >> v2:172.21.2.115:3300/0 conn(0x7f359c0603f0 0x7f359c0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:52.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 >> v1:172.21.2.115:6789/0 conn(0x7f359c14b700 legacy=0x7f359c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:52.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0afb40 con 0x7f359c0b0c90 2021-04-03T03:15:52.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f3594001a50 con 0x7f359c0b0c90 2021-04-03T03:15:52.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 >> v1:172.21.2.111:6789/0 conn(0x7f359c0b0c90 legacy=0x7f359c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:52.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f357c003730 0x7f357c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f357c0044e0 0x7f357c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f357c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c060860 con 0x7f357c004e50 2021-04-03T03:15:52.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c004240 con 0x7f357c0044e0 2021-04-03T03:15:52.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357c00bc20 con 0x7f357c003730 2021-04-03T03:15:52.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f357c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:52.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 -- 172.21.2.111:0/3597057782 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f357c003730 msgr2=0x7f357c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:52.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f357c003730 0x7f357c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:52.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 -- 172.21.2.111:0/3597057782 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f357c0044e0 msgr2=0x7f357c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:52.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f357c0044e0 0x7f357c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:52.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 -- 172.21.2.111:0/3597057782 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c003240 con 0x7f357c004e50 2021-04-03T03:15:52.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 -- 172.21.2.111:0/3597057782 reap_dead start 2021-04-03T03:15:52.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359b7fe700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f357c00b730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f359c0620f0 tx=0x7f359c0af830).ready entity=mon.0 client_cookie=d60d08688b1553c2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:52.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3594005c50 con 0x7f357c004e50 2021-04-03T03:15:52.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3594005db0 con 0x7f357c004e50 2021-04-03T03:15:52.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3599ffb700 1 -- 172.21.2.111:0/3597057782 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35940072c0 con 0x7f357c004e50 2021-04-03T03:15:52.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- 172.21.2.111:0/3597057782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 msgr2=0x7f357c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:52.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f357c00b730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f359c0620f0 tx=0x7f359c0af830).stop 2021-04-03T03:15:52.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- 172.21.2.111:0/3597057782 shutdown_connections 2021-04-03T03:15:52.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 --2- 172.21.2.111:0/3597057782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f357c00b730 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:52.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- 172.21.2.111:0/3597057782 >> 172.21.2.111:0/3597057782 conn(0x7f359c0b61c0 msgr2=0x7f359c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:52.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- 172.21.2.111:0/3597057782 shutdown_connections 2021-04-03T03:15:52.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- 172.21.2.111:0/3597057782 wait complete. 2021-04-03T03:15:52.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 Processor -- start 2021-04-03T03:15:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- start start 2021-04-03T03:15:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f359c0ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f359c0a8bc0 0x7f359c0a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f359c146a80 0x7f359c146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0a91f0 con 0x7f357c004e50 2021-04-03T03:15:52.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0a9490 con 0x7f359c0a8bc0 2021-04-03T03:15:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0a9730 con 0x7f359c146a80 2021-04-03T03:15:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f359c0ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:52.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f359c0ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55536/0 (socket says 172.21.2.111:55536) 2021-04-03T03:15:52.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 -- 172.21.2.111:0/595752810 learned_addr learned my addr 172.21.2.111:0/595752810 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:52.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 -- 172.21.2.111:0/595752810 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f359c146a80 msgr2=0x7f359c146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:52.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f359c146a80 0x7f359c146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:52.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 -- 172.21.2.111:0/595752810 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f359c0a8bc0 msgr2=0x7f359c0a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:52.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f359c0a8bc0 0x7f359c0a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:52.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 -- 172.21.2.111:0/595752810 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0a9bd0 con 0x7f357c004e50 2021-04-03T03:15:52.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359affd700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f359c0ab060 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f358c00a000 tx=0x7f358c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:52.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3593fff700 1 -- 172.21.2.111:0/595752810 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f358c00f680 con 0x7f357c004e50 2021-04-03T03:15:52.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3593fff700 1 -- 172.21.2.111:0/595752810 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00f040 con 0x7f357c004e50 2021-04-03T03:15:52.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c1473f0 con 0x7f357c004e50 2021-04-03T03:15:52.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f3593fff700 1 -- 172.21.2.111:0/595752810 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f358c00f9d0 con 0x7f357c004e50 2021-04-03T03:15:52.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.211+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c1562b0 con 0x7f357c004e50 2021-04-03T03:15:52.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.215+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c0a9000 con 0x7f357c004e50 2021-04-03T03:15:52.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.215+0000 7f3593fff700 1 -- 172.21.2.111:0/595752810 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f358c00fb30 con 0x7f357c004e50 2021-04-03T03:15:52.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.215+0000 7f3593fff700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3580056f20 0x7f35800593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:52.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.215+0000 7f3593fff700 1 -- 172.21.2.111:0/595752810 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f358c06fa50 con 0x7f357c004e50 2021-04-03T03:15:52.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.215+0000 7f359a7fc700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3580056f20 0x7f35800593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:52.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.215+0000 7f359a7fc700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3580056f20 0x7f35800593e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f3584001f60 tx=0x7f35840077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:52.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.215+0000 7f3593fff700 1 -- 172.21.2.111:0/595752810 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f358c042c50 con 0x7f357c004e50 2021-04-03T03:15:52.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:52.347+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f359c057660 con 0x7f357c004e50 2021-04-03T03:15:53.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f3593fff700 1 -- 172.21.2.111:0/595752810 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v307) v1 ==== 130+0+0 (secure 0 0 0) 0x7f358c044e40 con 0x7f357c004e50 2021-04-03T03:15:53.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3580056f20 msgr2=0x7f35800593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3580056f20 0x7f35800593e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f3584001f60 tx=0x7f35840077d0).stop 2021-04-03T03:15:53.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 msgr2=0x7f359c0ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f359c0ab060 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f358c00a000 tx=0x7f358c008040).stop 2021-04-03T03:15:53.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 shutdown_connections 2021-04-03T03:15:53.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f359c146a80 0x7f359c146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f357c004e50 0x7f359c0ab060 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3580056f20 0x7f35800593e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 --2- 172.21.2.111:0/595752810 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f359c0a8bc0 0x7f359c0a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 >> 172.21.2.111:0/595752810 conn(0x7f359c0b61c0 msgr2=0x7f359c0b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:53.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 shutdown_connections 2021-04-03T03:15:53.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.119+0000 7f359bfff700 1 -- 172.21.2.111:0/595752810 wait complete. 2021-04-03T03:15:53.131 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:15:53.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 Processor -- start 2021-04-03T03:15:53.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 -- start start 2021-04-03T03:15:53.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 0x7f27ac147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac0607d0 con 0x7f27ac0add50 2021-04-03T03:15:53.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac14b090 con 0x7f27ac0603f0 2021-04-03T03:15:53.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac14b330 con 0x7f27ac144ef0 2021-04-03T03:15:53.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27ab7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f27ac0add50 0x7f27ac147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56340/0 (socket says 172.21.2.111:56340) 2021-04-03T03:15:53.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27ab7fe700 1 -- 172.21.2.111:0/2407453520 learned_addr learned my addr 172.21.2.111:0/2407453520 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:53.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351969303 0 0) 0x7f27ac0607d0 con 0x7f27ac0add50 2021-04-03T03:15:53.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f278c003240 con 0x7f27ac0add50 2021-04-03T03:15:53.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 0x7f27ac147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3687296622 0 0) 0x7f278c003240 con 0x7f27ac0add50 2021-04-03T03:15:53.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 msgr2=0x7f27ac147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:53.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 0x7f27ac147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 >> v1:172.21.2.115:6789/0 conn(0x7f27ac0603f0 legacy=0x7f27ac1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac14b5d0 con 0x7f27ac0add50 2021-04-03T03:15:53.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f279c0016d0 con 0x7f27ac0add50 2021-04-03T03:15:53.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 >> v1:172.21.2.111:6789/0 conn(0x7f27ac0add50 legacy=0x7f27ac147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 0x7f278c003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f278c003c70 0x7f278c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f278c0049d0 0x7f278c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac14b330 con 0x7f278c0049d0 2021-04-03T03:15:53.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c004200 con 0x7f278c003c70 2021-04-03T03:15:53.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c00b730 con 0x7f27ac144ef0 2021-04-03T03:15:53.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f278c0049d0 0x7f278c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27abfff700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 0x7f278c003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27ab7fe700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f278c003c70 0x7f278c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 -- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 msgr2=0x7f278c003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:53.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 0x7f278c003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 -- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f278c003c70 msgr2=0x7f278c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:53.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f278c003c70 0x7f278c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 -- 172.21.2.111:0/2407453520 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac14b5d0 con 0x7f278c0049d0 2021-04-03T03:15:53.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27abfff700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac144ef0 0x7f278c003760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:53.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27ab7fe700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f278c003c70 0x7f278c0044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:53.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f278c0049d0 0x7f278c004e50 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f27a400c000 tx=0x7f27a40091d0).ready entity=mon.0 client_cookie=dffa3b9acad9020a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a400eca0 con 0x7f278c0049d0 2021-04-03T03:15:53.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a400ee00 con 0x7f278c0049d0 2021-04-03T03:15:53.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27aaffd700 1 -- 172.21.2.111:0/2407453520 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a40125e0 con 0x7f278c0049d0 2021-04-03T03:15:53.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 -- 172.21.2.111:0/2407453520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f278c0049d0 msgr2=0x7f278c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b1327700 1 --2- 172.21.2.111:0/2407453520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f278c0049d0 0x7f278c004e50 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f27a400c000 tx=0x7f27a40091d0).stop 2021-04-03T03:15:53.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.191+0000 7f27b0b26700 1 -- 172.21.2.111:0/2407453520 reap_dead start 2021-04-03T03:15:53.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- 172.21.2.111:0/2407453520 shutdown_connections 2021-04-03T03:15:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- 172.21.2.111:0/2407453520 >> 172.21.2.111:0/2407453520 conn(0x7f27ac0b61c0 msgr2=0x7f27ac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- 172.21.2.111:0/2407453520 shutdown_connections 2021-04-03T03:15:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- 172.21.2.111:0/2407453520 wait complete. 2021-04-03T03:15:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 Processor -- start 2021-04-03T03:15:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- start start 2021-04-03T03:15:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac0b3670 0x7f27ac156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f27ac0a8570 0x7f27ac15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f27ac0a89f0 0x7f27ac15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0a8e80 con 0x7f27ac0a8570 2021-04-03T03:15:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac159db0 con 0x7f27ac0a89f0 2021-04-03T03:15:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac156b00 con 0x7f27ac0b3670 2021-04-03T03:15:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f27ac0a8570 0x7f27ac15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f27ac0a8570 0x7f27ac15a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55558/0 (socket says 172.21.2.111:55558) 2021-04-03T03:15:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 -- 172.21.2.111:0/2968575964 learned_addr learned my addr 172.21.2.111:0/2968575964 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b0b26700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f27ac0a89f0 0x7f27ac15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27abfff700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac0b3670 0x7f27ac156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 -- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac0b3670 msgr2=0x7f27ac156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac0b3670 0x7f27ac156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 -- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f27ac0a89f0 msgr2=0x7f27ac15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f27ac0a89f0 0x7f27ac15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 -- 172.21.2.111:0/2968575964 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0b3a50 con 0x7f27ac0a8570 2021-04-03T03:15:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27ab7fe700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f27ac0a8570 0x7f27ac15a250 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f279c00b000 tx=0x7f279c0016b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27a97fa700 1 -- 172.21.2.111:0/2968575964 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f279c00a9f0 con 0x7f27ac0a8570 2021-04-03T03:15:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27a97fa700 1 -- 172.21.2.111:0/2968575964 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c010040 con 0x7f27ac0a8570 2021-04-03T03:15:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac062a70 con 0x7f27ac0a8570 2021-04-03T03:15:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27a97fa700 1 -- 172.21.2.111:0/2968575964 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f279c00ad40 con 0x7f27ac0a8570 2021-04-03T03:15:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac062f10 con 0x7f27ac0a8570 2021-04-03T03:15:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac159f70 con 0x7f27ac0a8570 2021-04-03T03:15:53.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27a97fa700 1 -- 172.21.2.111:0/2968575964 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f279c01b020 con 0x7f27ac0a8570 2021-04-03T03:15:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27a97fa700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2790056ef0 0x7f27900593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27a97fa700 1 -- 172.21.2.111:0/2968575964 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f279c06fcf0 con 0x7f27ac0a8570 2021-04-03T03:15:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27abfff700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2790056ef0 0x7f27900593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.195+0000 7f27abfff700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2790056ef0 0x7f27900593b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f279400d000 tx=0x7f2794009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.199+0000 7f27a97fa700 1 -- 172.21.2.111:0/2968575964 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f279c044fd0 con 0x7f27ac0a8570 2021-04-03T03:15:53.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.291+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f27ac057660 con 0x7f27ac0a8570 2021-04-03T03:15:53.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.295+0000 7f27a97fa700 1 -- 172.21.2.111:0/2968575964 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f279c0485f0 con 0x7f27ac0a8570 2021-04-03T03:15:53.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2790056ef0 msgr2=0x7f27900593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2790056ef0 0x7f27900593b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f279400d000 tx=0x7f2794009040).stop 2021-04-03T03:15:53.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f27ac0a8570 msgr2=0x7f27ac15a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f27ac0a8570 0x7f27ac15a250 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f279c00b000 tx=0x7f279c0016b0).stop 2021-04-03T03:15:53.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 shutdown_connections 2021-04-03T03:15:53.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2790056ef0 0x7f27900593b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f27ac0a89f0 0x7f27ac15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f27ac0b3670 0x7f27ac156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 --2- 172.21.2.111:0/2968575964 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f27ac0a8570 0x7f27ac15a250 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 >> 172.21.2.111:0/2968575964 conn(0x7f27ac0b61c0 msgr2=0x7f27ac0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:53.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 shutdown_connections 2021-04-03T03:15:53.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.299+0000 7f27b1327700 1 -- 172.21.2.111:0/2968575964 wait complete. 2021-04-03T03:15:53.309 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:15:53.310 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:53.316 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:53.328 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:53.340 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:53.353 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:15:53.365 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:15:53.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:53.366 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:53.366 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:15:53.388 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:15:53.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:53.390 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:15:53.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:53.390 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:53.391 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:15:53.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:53.401 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:15:53.402 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:15:53.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:15:53.403 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:15:53.403 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:15:53.411 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:15:53.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:15:53.413 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:15:53.413 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:15:53.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 Processor -- start 2021-04-03T03:15:53.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 -- start start 2021-04-03T03:15:53.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9ba40603f0 0x7f9ba40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9ba40b0c90 0x7f9ba4147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba40af400 con 0x7f9ba414b700 2021-04-03T03:15:53.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba4060860 con 0x7f9ba40603f0 2021-04-03T03:15:53.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba40af6a0 con 0x7f9ba40b0c90 2021-04-03T03:15:53.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba3fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f9ba414b700 0x7f9ba40b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56358/0 (socket says 172.21.2.111:56358) 2021-04-03T03:15:53.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba3fff700 1 -- 172.21.2.111:0/725600180 learned_addr learned my addr 172.21.2.111:0/725600180 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:53.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba89fb700 1 --2- 172.21.2.111:0/725600180 >> v2:172.21.2.111:3300/0 conn(0x7f9ba40603f0 0x7f9ba40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba89fb700 1 -- 172.21.2.111:0/725600180 >> v2:172.21.2.115:3300/0 conn(0x7f9ba40b0c90 msgr2=0x7f9ba4147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:53.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba91fc700 1 --2- 172.21.2.111:0/725600180 >> v2:172.21.2.115:3300/0 conn(0x7f9ba40b0c90 0x7f9ba4147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba89fb700 1 --2- 172.21.2.111:0/725600180 >> v2:172.21.2.115:3300/0 conn(0x7f9ba40b0c90 0x7f9ba4147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba89fb700 1 -- 172.21.2.111:0/725600180 >> v1:172.21.2.111:6789/0 conn(0x7f9ba414b700 legacy=0x7f9ba40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba89fb700 1 -- 172.21.2.111:0/725600180 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba40afb40 con 0x7f9ba40603f0 2021-04-03T03:15:53.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba89fb700 1 --2- 172.21.2.111:0/725600180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba40603f0 0x7f9ba40b0090 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f9b940010d0 tx=0x7f9b8c006dd0).ready entity=mon.0 client_cookie=cebb84efb73e6046 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba37fe700 1 -- 172.21.2.111:0/725600180 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8c0096b0 con 0x7f9ba40603f0 2021-04-03T03:15:53.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba37fe700 1 -- 172.21.2.111:0/725600180 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c009810 con 0x7f9ba40603f0 2021-04-03T03:15:53.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba37fe700 1 -- 172.21.2.111:0/725600180 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8c009b70 con 0x7f9ba40603f0 2021-04-03T03:15:53.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 -- 172.21.2.111:0/725600180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba40603f0 msgr2=0x7f9ba40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.475+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/725600180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba40603f0 0x7f9ba40b0090 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f9b940010d0 tx=0x7f9b8c006dd0).stop 2021-04-03T03:15:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- 172.21.2.111:0/725600180 shutdown_connections 2021-04-03T03:15:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/725600180 >> v2:172.21.2.115:3300/0 conn(0x7f9ba40b0c90 0x7f9ba4147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/725600180 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba40603f0 0x7f9ba40b0090 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- 172.21.2.111:0/725600180 >> 172.21.2.111:0/725600180 conn(0x7f9ba40b61c0 msgr2=0x7f9ba405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:53.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- 172.21.2.111:0/725600180 shutdown_connections 2021-04-03T03:15:53.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- 172.21.2.111:0/725600180 wait complete. 2021-04-03T03:15:53.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 Processor -- start 2021-04-03T03:15:53.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- start start 2021-04-03T03:15:53.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9ba40603f0 0x7f9ba40a8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9ba40b0c90 0x7f9ba4062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba414b700 0x7f9ba4064620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40a79c0 con 0x7f9ba414b700 2021-04-03T03:15:53.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40a7c60 con 0x7f9ba40603f0 2021-04-03T03:15:53.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40a7640 con 0x7f9ba40b0c90 2021-04-03T03:15:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba414b700 0x7f9ba4064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba414b700 0x7f9ba4064620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55574/0 (socket says 172.21.2.111:55574) 2021-04-03T03:15:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 -- 172.21.2.111:0/4232271337 learned_addr learned my addr 172.21.2.111:0/4232271337 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:53.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 -- 172.21.2.111:0/4232271337 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9ba40b0c90 msgr2=0x7f9ba4062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9ba40b0c90 0x7f9ba4062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 -- 172.21.2.111:0/4232271337 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9ba40603f0 msgr2=0x7f9ba40a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9ba40603f0 0x7f9ba40a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 -- 172.21.2.111:0/4232271337 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba40aa260 con 0x7f9ba414b700 2021-04-03T03:15:53.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba91fc700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba414b700 0x7f9ba4064620 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f9b9c007000 tx=0x7f9b9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba1ffb700 1 -- 172.21.2.111:0/4232271337 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9c0078c0 con 0x7f9ba414b700 2021-04-03T03:15:53.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba1ffb700 1 -- 172.21.2.111:0/4232271337 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c007a20 con 0x7f9ba414b700 2021-04-03T03:15:53.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba40abe20 con 0x7f9ba414b700 2021-04-03T03:15:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba1ffb700 1 -- 172.21.2.111:0/4232271337 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9c007b90 con 0x7f9ba414b700 2021-04-03T03:15:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba4064ca0 con 0x7f9ba414b700 2021-04-03T03:15:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba40aa4b0 con 0x7f9ba414b700 2021-04-03T03:15:53.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba1ffb700 1 -- 172.21.2.111:0/4232271337 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9b9c007cf0 con 0x7f9ba414b700 2021-04-03T03:15:53.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.479+0000 7f9ba1ffb700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b88056fb0 0x7f9b88059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.483+0000 7f9ba1ffb700 1 -- 172.21.2.111:0/4232271337 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9b9c0702d0 con 0x7f9ba414b700 2021-04-03T03:15:53.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.483+0000 7f9ba89fb700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b88056fb0 0x7f9b88059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.483+0000 7f9ba89fb700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b88056fb0 0x7f9b88059470 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f9b940010d0 tx=0x7f9b8c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.483+0000 7f9ba1ffb700 1 -- 172.21.2.111:0/4232271337 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9b9c0456c0 con 0x7f9ba414b700 2021-04-03T03:15:53.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9ba40a7800 con 0x7f9ba414b700 2021-04-03T03:15:53.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba1ffb700 1 -- 172.21.2.111:0/4232271337 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v93) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f9b9c048ce0 con 0x7f9ba414b700 2021-04-03T03:15:53.580 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:53.580 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:15:53.581 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 93, 2021-04-03T03:15:53.581 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:15:53.581 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:15:53.581 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:15:45.068238+0000", 2021-04-03T03:15:53.581 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:15:53.582 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:15:53.582 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:15:53.582 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:15:53.582 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:15:53.583 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:15:53.583 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:15:53.583 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:15:53.583 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:15:53.584 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:53.584 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:15:53.584 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:15:53.584 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:15:53.585 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:15:53.585 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:15:53.585 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 18, 2021-04-03T03:15:53.585 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:15:53.586 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:15:53.586 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:15:53.586 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:15:53.586 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:15:53.586 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.587 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:15:53.587 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:15:53.587 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:15:53.587 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:53.588 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:53.588 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:53.588 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:53.588 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:53.589 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:53.589 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:53.589 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:53.589 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:15:53.589 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:15:53.590 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:15:53.590 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:15:53.590 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:15:53.590 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:53.591 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:53.591 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:53.591 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:53.591 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:53.591 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:53.592 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:53.592 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.592 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:15:53.592 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:53.592 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:53.593 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:53.593 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:53.593 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:53.593 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:53.594 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:53.594 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:53.594 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:53.594 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:53.594 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:53.595 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:53.595 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:53.595 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:53.595 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:53.595 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:53.596 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:53.596 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:53.596 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:53.596 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:53.596 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:53.597 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:53.597 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:53.597 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:53.598 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:53.598 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:53.598 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.598 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:53.599 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:53.599 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:53.599 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:53.599 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:53.599 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:53.600 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:53.600 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:53.600 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:53.600 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:53.600 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:53.601 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:15:53.601 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:15:53.601 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.601 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:53.602 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:15:53.602 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.602 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.602 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.603 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:15:53.603 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:15:53.603 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:15:53.603 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:53.603 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:53.604 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:53.604 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:53.604 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:53.604 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:53.604 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:53.605 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:53.605 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:53.605 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:53.605 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:53.605 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:53.606 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:53.606 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:53.606 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:53.606 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:53.607 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:53.607 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:53.607 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:53.607 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:53.607 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.608 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:15:53.608 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:53.608 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:53.608 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:53.608 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:53.609 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:53.609 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:53.609 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:53.609 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:53.610 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:53.610 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:53.610 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:53.610 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:53.610 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:53.611 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:53.611 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:53.611 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:53.611 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:53.611 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:53.612 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:53.612 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:53.612 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:53.612 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:53.613 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:53.613 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:53.613 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:53.613 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:53.613 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.614 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:53.614 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:53.614 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:53.614 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:53.614 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:53.615 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:53.615 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:53.615 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:53.615 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:53.616 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:53.616 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:53.616 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:15:53.616 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:53.616 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:15:53.617 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.617 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.617 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.617 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 17, 2021-04-03T03:15:53.618 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:15:53.618 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:15:42.095114+0000", 2021-04-03T03:15:53.618 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:53.618 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:53.618 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:53.619 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:53.619 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:53.619 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:53.619 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:53.619 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:53.620 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:53.620 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:53.620 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:53.620 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:53.621 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:53.621 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:53.621 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:53.621 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:53.621 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:53.622 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:53.622 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:53.622 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:53.622 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.622 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "92", 2021-04-03T03:15:53.623 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:53.623 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:53.623 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:53.623 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:53.623 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:53.624 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:53.624 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:53.624 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:53.624 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:53.625 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:53.625 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:53.625 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:53.625 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:53.625 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:53.626 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:53.626 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:53.626 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:53.626 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:53.626 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:53.627 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:53.627 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:53.627 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:53.627 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:53.628 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:53.628 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:53.628 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:53.629 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.629 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:53.629 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:53.630 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:53.630 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:53.630 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:53.630 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:53.631 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:53.631 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:53.631 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:53.631 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:53.632 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:53.632 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:15:53.632 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:15:53.632 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:15:53.632 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:15:53.633 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.633 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:53.633 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:15:53.634 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:15:53.634 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.634 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.634 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.635 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.635 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 18, 2021-04-03T03:15:53.635 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:15:53.636 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:15:43.035163+0000", 2021-04-03T03:15:53.636 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:15:53.636 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:15:53.636 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:15:53.637 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:15:53.637 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:15:53.637 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:15:53.637 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:15:53.637 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:15:53.638 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:15:53.638 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:15:53.638 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:15:53.638 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:15:53.639 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:15:53.639 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:15:53.639 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:15:53.639 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:15:53.639 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:15:53.640 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:15:53.640 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:15:53.640 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:15:53.640 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.640 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "93", 2021-04-03T03:15:53.641 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:15:53.641 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:15:53.641 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:15:53.641 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:15:53.642 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:15:53.642 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:15:53.642 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:15:53.642 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:15:53.642 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:15:53.643 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:15:53.643 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:15:53.643 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:15:53.643 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:15:53.644 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:15:53.644 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:15:53.644 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:15:53.644 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:15:53.645 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:15:53.645 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:15:53.645 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:15:53.645 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:15:53.645 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:15:53.646 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:15:53.646 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:15:53.646 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:15:53.646 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:15:53.646 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.647 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:15:53.647 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:15:53.647 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:15:53.647 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:15:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:15:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:15:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:15:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:15:53.648 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:15:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:15:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:15:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:15:53.649 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:15:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:15:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:15:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.650 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.651 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:53.651 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:15:53.651 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.651 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:15:53.652 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:15:53.652 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.652 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.652 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.653 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.653 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.653 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.653 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.653 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 91, 2021-04-03T03:15:53.654 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.654 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.654 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.654 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.655 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.655 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.655 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:15:53.655 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.656 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.656 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.656 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.656 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:15:53.656 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.657 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.657 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.657 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.657 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.657 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.658 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.658 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.658 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:15:53.658 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.659 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.659 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.659 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.659 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:15:53.659 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.660 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.660 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.660 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.660 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.661 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.661 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.661 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.661 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:15:53.661 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.662 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.662 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.662 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.662 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:15:53.663 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.663 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.663 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.663 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.663 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.664 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.664 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.664 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.664 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:15:53.664 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.665 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.665 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.665 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.665 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:15:53.666 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:15:53.666 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.666 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.666 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.666 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:15:53.667 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:15:53.667 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:15:53.667 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:15:53.667 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.667 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.668 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.668 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.668 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.668 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.668 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:15:53.669 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:15:53.669 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.669 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.669 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.670 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.670 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.670 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.670 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.670 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 91, 2021-04-03T03:15:53.671 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.671 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.671 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.671 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.672 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.672 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.672 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:15:53.672 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.672 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.673 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.673 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.673 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:15:53.673 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.673 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.674 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.674 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.674 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.674 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.675 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.675 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.675 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:15:53.676 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.676 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.676 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.677 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.677 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:15:53.677 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.677 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.678 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.678 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.678 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.678 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.679 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.679 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.679 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:15:53.679 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.679 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.680 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.680 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.680 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:15:53.680 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.681 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.681 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.681 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.681 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.681 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.682 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.682 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.682 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:15:53.682 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.682 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.683 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.683 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.683 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:15:53.683 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:15:53.684 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.684 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.684 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.684 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:15:53.684 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:15:53.685 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:15:53.685 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:15:53.685 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.685 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.686 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.686 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.686 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.686 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.686 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:15:53.687 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:15:53.687 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.687 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.687 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.687 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.688 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.688 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.688 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.688 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 91, 2021-04-03T03:15:53.689 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.689 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.689 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.689 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.689 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.690 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.690 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:15:53.690 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.690 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.690 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.691 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.691 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:15:53.691 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.691 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.692 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.692 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.692 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.692 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.692 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.693 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.693 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:15:53.693 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.693 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.693 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.694 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.694 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:15:53.694 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.694 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.695 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.695 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.695 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.696 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.696 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.696 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.696 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:15:53.697 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.697 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.697 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.697 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.697 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:15:53.698 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.698 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.698 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.698 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.698 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.699 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.699 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.699 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.699 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:15:53.700 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.700 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.700 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.700 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.700 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:15:53.701 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:15:53.701 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.701 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.701 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.702 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:15:53.702 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:15:53.702 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:15:53.702 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:15:53.702 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.703 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.703 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.703 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.703 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.704 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.704 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:15:53.704 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:15:53.704 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.704 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.705 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.705 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.705 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.705 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.706 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.706 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 91, 2021-04-03T03:15:53.706 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.706 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.706 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.707 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.707 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.707 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.707 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:15:53.708 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.708 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.708 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.708 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.708 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:15:53.709 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.709 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.709 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.709 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.710 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.710 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.710 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.710 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.710 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:15:53.711 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.711 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.711 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.711 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.712 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:15:53.712 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.712 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.712 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.712 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.713 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.713 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.713 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.713 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.714 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:15:53.714 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.714 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.714 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.714 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.715 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:15:53.715 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.715 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.715 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.716 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.716 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.716 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.716 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.717 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.717 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:15:53.717 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.717 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.718 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.718 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.718 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:15:53.718 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:15:53.719 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.719 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.719 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.719 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:15:53.719 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:15:53.720 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:15:53.720 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:15:53.720 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.720 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.721 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.721 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.721 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.721 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.721 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:15:53.722 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:15:53.722 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.722 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.722 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.722 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.723 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.723 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.723 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.723 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 90, 2021-04-03T03:15:53.724 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.724 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.724 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.724 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.724 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.725 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.725 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:15:53.725 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.725 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.725 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.726 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.726 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:15:53.726 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.726 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.726 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.727 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.727 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.727 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.727 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.728 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.728 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:15:53.728 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.728 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.728 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.729 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.729 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:15:53.729 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.729 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.730 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.730 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.730 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.730 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.731 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.731 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.732 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:15:53.732 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.732 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.733 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.733 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.733 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:15:53.734 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.734 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.735 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.735 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.736 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.736 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.737 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.737 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.738 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:15:53.738 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.738 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.739 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.739 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.740 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:15:53.740 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:15:53.740 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.741 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.741 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.742 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:15:53.743 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:15:53.743 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:15:53.743 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:15:53.744 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.744 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.745 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.745 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.746 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.746 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.746 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:15:53.747 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:15:53.747 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.748 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.748 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.748 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.749 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.749 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.750 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.750 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 81, 2021-04-03T03:15:53.751 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.751 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.751 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.752 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.752 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.752 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.752 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:15:53.753 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.753 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.753 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.754 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.754 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:15:53.754 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.755 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.755 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.755 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.756 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.756 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.757 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.757 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.757 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:15:53.757 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.758 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.758 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.758 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.759 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:15:53.759 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.759 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.760 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.760 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.761 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.761 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.761 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.762 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.762 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:15:53.762 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.763 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.764 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.764 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.765 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:15:53.765 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.765 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.766 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.766 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.767 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.767 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.767 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.768 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.768 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:15:53.769 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.769 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.770 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.770 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.771 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:15:53.771 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:15:53.772 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.772 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.772 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.773 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:15:53.773 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:15:53.774 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:15:53.774 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:15:53.774 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.775 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.775 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.776 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.776 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.777 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.777 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:15:53.778 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:15:53.778 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.779 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.779 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.779 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.780 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.780 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.781 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.781 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 90, 2021-04-03T03:15:53.781 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.782 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.782 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.782 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.783 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.783 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.783 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:15:53.783 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.784 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.784 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.784 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.784 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:15:53.784 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.785 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.785 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.785 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.785 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.785 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.786 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.786 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.786 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:15:53.786 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.786 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.787 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.787 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.787 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:15:53.787 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.788 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.788 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.788 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.788 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.788 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.789 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.789 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.789 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:15:53.789 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.789 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.790 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.790 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.790 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:15:53.790 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.790 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.791 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:15:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.792 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:15:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:15:53.793 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.794 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.794 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.794 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:15:53.794 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:15:53.794 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:15:53.795 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:15:53.795 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.795 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.795 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.795 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.796 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.796 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.796 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:15:53.796 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:15:53.797 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:15:53.797 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:15:53.797 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:15:53.797 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:15:53.797 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:15:53.798 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:15:53.798 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:15:53.798 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 91, 2021-04-03T03:15:53.798 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:15:53.798 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:15:53.799 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:15:53.799 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.799 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.799 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.799 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:15:53.800 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.800 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.800 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.800 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.801 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:15:53.801 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.801 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.801 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.801 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.802 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:15:53.802 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.802 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.802 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.802 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:15:53.803 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.803 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.803 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.803 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.804 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:15:53.804 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.804 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.804 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.804 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.805 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:15:53.805 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.805 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.805 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.805 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:15:53.806 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.806 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.806 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.806 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.806 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:15:53.807 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.807 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.807 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.807 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.808 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:15:53.808 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:15:53.808 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.808 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:15:53.808 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:15:53.809 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.809 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.809 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.809 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:15:53.809 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:15:53.810 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:15:53.810 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.810 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.810 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.810 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:15:53.811 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:15:53.811 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:15:53.811 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:15:53.811 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:15:53.812 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:15:53.812 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:15:53.812 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:53.812 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.812 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:53.813 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:15:53.813 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.813 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:15:53.813 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.813 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.814 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.814 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.814 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.814 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:15:53.814 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.815 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.815 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.815 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:15:53.815 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.816 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.816 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.816 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.816 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.816 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:15:53.817 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.817 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.817 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.817 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:15:53.817 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.818 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.818 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.818 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.818 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.818 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:15:53.819 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.819 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.819 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.819 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:15:53.820 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.820 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.820 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.820 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.820 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.821 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:15:53.821 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.821 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.821 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.821 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:15:53.822 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.822 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.822 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.822 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.823 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.823 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:15:53.823 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.823 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.823 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.824 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:15:53.824 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.824 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.824 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.824 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.825 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.825 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:15:53.825 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.825 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.826 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.826 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:15:53.826 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.826 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.826 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.827 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.827 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.827 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:15:53.827 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.827 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.828 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:53.828 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:15:53.828 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:15:53.828 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:15:53.829 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:15:53.829 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:15:53.829 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:15:53.829 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:15:53.829 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:15:53.830 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.830 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:15:53.830 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:15:53.830 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:15:53.830 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:15:53.831 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:15:53.831 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:15:53.831 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:15:53.831 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:15:53.831 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:15:53.832 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:15:53.832 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:15:53.832 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:15:53.832 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:15:53.832 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:15:53.833 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:15:53.833 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:15:53.833 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:15:53.833 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:15:53.833 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.834 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:15:53.834 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:15:53.834 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:15:53.834 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:15:53.835 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:15:53.835 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:15:53.835 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:15:53.835 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:53.835 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:53.836 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:15:53.836 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:15:53.836 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:15:53.836 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:15:53.836 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:15:53.837 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:15:53.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b88056fb0 msgr2=0x7f9b88059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b88056fb0 0x7f9b88059470 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f9b940010d0 tx=0x7f9b8c015000).stop 2021-04-03T03:15:53.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba414b700 msgr2=0x7f9ba4064620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba414b700 0x7f9ba4064620 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f9b9c007000 tx=0x7f9b9c008040).stop 2021-04-03T03:15:53.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 shutdown_connections 2021-04-03T03:15:53.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9ba414b700 0x7f9ba4064620 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b88056fb0 0x7f9b88059470 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9ba40603f0 0x7f9ba40a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 --2- 172.21.2.111:0/4232271337 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9ba40b0c90 0x7f9ba4062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 >> 172.21.2.111:0/4232271337 conn(0x7f9ba40b61c0 msgr2=0x7f9ba405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:53.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 shutdown_connections 2021-04-03T03:15:53.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.575+0000 7f9ba99fd700 1 -- 172.21.2.111:0/4232271337 wait complete. 2021-04-03T03:15:53.842 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:53.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 Processor -- start 2021-04-03T03:15:53.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 -- start start 2021-04-03T03:15:53.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1680b3a90 0x7fb1680ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680b24d0 con 0x7fb1680b8350 2021-04-03T03:15:53.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680b8730 con 0x7fb1680ad660 2021-04-03T03:15:53.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb168142090 con 0x7fb1680b3a90 2021-04-03T03:15:53.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fb1680b8350 0x7fb1680b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56370/0 (socket says 172.21.2.111:56370) 2021-04-03T03:15:53.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 -- 172.21.2.111:0/4192559412 learned_addr learned my addr 172.21.2.111:0/4192559412 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:53.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865644724 0 0) 0x7fb1680b24d0 con 0x7fb1680b8350 2021-04-03T03:15:53.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb14c003240 con 0x7fb1680b8350 2021-04-03T03:15:53.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165b83700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1680b3a90 0x7fb1680ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3128074121 0 0) 0x7fb14c003240 con 0x7fb1680b8350 2021-04-03T03:15:53.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1680b3a90 msgr2=0x7fb1680ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:53.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1680b3a90 0x7fb1680ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 >> v1:172.21.2.115:6789/0 conn(0x7fb1680ad660 legacy=0x7fb1680adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1681413c0 con 0x7fb1680b8350 2021-04-03T03:15:53.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fb1580016d0 con 0x7fb1680b8350 2021-04-03T03:15:53.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 >> v1:172.21.2.111:6789/0 conn(0x7fb1680b8350 legacy=0x7fb1680b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb14c003760 0x7fb14c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1680b3a90 0x7fb14c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb14c0049d0 0x7fb14c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168142090 con 0x7fb14c003760 2021-04-03T03:15:53.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c004250 con 0x7fb14c0049d0 2021-04-03T03:15:53.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c00b730 con 0x7fb1680b3a90 2021-04-03T03:15:53.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb14c003760 0x7fb14c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 -- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1680b3a90 msgr2=0x7fb14c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:53.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1680b3a90 0x7fb14c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 -- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb14c0049d0 msgr2=0x7fb14c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:53.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb14c0049d0 0x7fb14c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 -- 172.21.2.111:0/4192559412 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c003240 con 0x7fb14c003760 2021-04-03T03:15:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166384700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb14c003760 0x7fb14c003b80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7fb1580013e0 tx=0x7fb168141550).ready entity=mon.0 client_cookie=9026d2eb09299856 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb15800ce90 con 0x7fb14c003760 2021-04-03T03:15:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1580070d0 con 0x7fb14c003760 2021-04-03T03:15:53.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb165382700 1 -- 172.21.2.111:0/4192559412 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb158007430 con 0x7fb14c003760 2021-04-03T03:15:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 -- 172.21.2.111:0/4192559412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb14c003760 msgr2=0x7fb14c003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb167386700 1 --2- 172.21.2.111:0/4192559412 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb14c003760 0x7fb14c003b80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7fb1580013e0 tx=0x7fb168141550).stop 2021-04-03T03:15:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.903+0000 7fb166b85700 1 -- 172.21.2.111:0/4192559412 reap_dead start 2021-04-03T03:15:53.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- 172.21.2.111:0/4192559412 shutdown_connections 2021-04-03T03:15:53.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- 172.21.2.111:0/4192559412 >> 172.21.2.111:0/4192559412 conn(0x7fb168060490 msgr2=0x7fb1680b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:53.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- 172.21.2.111:0/4192559412 shutdown_connections 2021-04-03T03:15:53.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- 172.21.2.111:0/4192559412 wait complete. 2021-04-03T03:15:53.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 Processor -- start 2021-04-03T03:15:53.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- start start 2021-04-03T03:15:53.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1680673c0 0x7fb1680677a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1681445e0 0x7fb168142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb168143010 0x7fb1681434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16812aa90 con 0x7fb168143010 2021-04-03T03:15:53.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168143990 con 0x7fb1680673c0 2021-04-03T03:15:53.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680b1b20 con 0x7fb1681445e0 2021-04-03T03:15:53.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb168143010 0x7fb1681434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb168143010 0x7fb1681434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55592/0 (socket says 172.21.2.111:55592) 2021-04-03T03:15:53.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 -- 172.21.2.111:0/2274750178 learned_addr learned my addr 172.21.2.111:0/2274750178 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:53.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166384700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1680673c0 0x7fb1680677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 -- 172.21.2.111:0/2274750178 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1681445e0 msgr2=0x7fb168142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1681445e0 0x7fb168142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:53.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 -- 172.21.2.111:0/2274750178 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1680673c0 msgr2=0x7fb1680677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:53.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1680673c0 0x7fb1680677a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:53.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 -- 172.21.2.111:0/2274750178 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16813e700 con 0x7fb168143010 2021-04-03T03:15:53.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166b85700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb168143010 0x7fb1681434a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fb1580010d0 tx=0x7fb160007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb1577fe700 1 -- 172.21.2.111:0/2274750178 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb160005b40 con 0x7fb168143010 2021-04-03T03:15:53.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb1577fe700 1 -- 172.21.2.111:0/2274750178 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160005ca0 con 0x7fb168143010 2021-04-03T03:15:53.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb1577fe700 1 -- 172.21.2.111:0/2274750178 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1600070d0 con 0x7fb168143010 2021-04-03T03:15:53.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16812a7f0 con 0x7fb168143010 2021-04-03T03:15:53.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb168142a10 con 0x7fb168143010 2021-04-03T03:15:53.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb168057660 con 0x7fb168143010 2021-04-03T03:15:53.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb1577fe700 1 -- 172.21.2.111:0/2274750178 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb160010070 con 0x7fb168143010 2021-04-03T03:15:53.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb1577fe700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb144056ea0 0x7fb144059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:53.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb1577fe700 1 -- 172.21.2.111:0/2274750178 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb16006f2f0 con 0x7fb168143010 2021-04-03T03:15:53.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.907+0000 7fb166384700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb144056ea0 0x7fb144059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:53.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.911+0000 7fb166384700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb144056ea0 0x7fb144059360 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fb158016000 tx=0x7fb158015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:53.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:53.911+0000 7fb1577fe700 1 -- 172.21.2.111:0/2274750178 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb1600445d0 con 0x7fb168143010 2021-04-03T03:15:54.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1680686d0 con 0x7fb168143010 2021-04-03T03:15:54.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb1577fe700 1 -- 172.21.2.111:0/2274750178 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fb160047bf0 con 0x7fb168143010 2021-04-03T03:15:54.039 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:54.039 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":38,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3837436656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3837436656},{"type":"v1","addr":"172.21.2.115:6833","nonce":3837436656}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:44.064474+0000","modified":"2021-04-03T03:15:45.067347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/13282524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":13282524},{"type":"v1","addr":"172.21.2.111:6835","nonce":13282524}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-04-03T03:15:54.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb144056ea0 msgr2=0x7fb144059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:54.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb144056ea0 0x7fb144059360 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fb158016000 tx=0x7fb158015000).stop 2021-04-03T03:15:54.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb168143010 msgr2=0x7fb1681434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:54.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb168143010 0x7fb1681434a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fb1580010d0 tx=0x7fb160007d70).stop 2021-04-03T03:15:54.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 shutdown_connections 2021-04-03T03:15:54.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb168143010 0x7fb1681434a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:54.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb144056ea0 0x7fb144059360 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:54.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1680673c0 0x7fb1680677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:54.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 --2- 172.21.2.111:0/2274750178 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1681445e0 0x7fb168142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:54.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 >> 172.21.2.111:0/2274750178 conn(0x7fb168060490 msgr2=0x7fb168062760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:54.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 shutdown_connections 2021-04-03T03:15:54.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.035+0000 7fb167386700 1 -- 172.21.2.111:0/2274750178 wait complete. 2021-04-03T03:15:54.043 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 38 2021-04-03T03:15:54.046 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:15:54.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 Processor -- start 2021-04-03T03:15:54.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- start start 2021-04-03T03:15:54.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0d200603f0 0x7f0d201447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:54.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200b17b0 0x7f0d20144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:54.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200607d0 con 0x7f0d200add50 2021-04-03T03:15:54.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d201451c0 con 0x7f0d200603f0 2021-04-03T03:15:54.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2014b090 con 0x7f0d200b17b0 2021-04-03T03:15:54.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0d200603f0 0x7f0d201447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:54.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0d200603f0 0x7f0d201447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55596/0 (socket says 172.21.2.111:55596) 2021-04-03T03:15:54.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 -- 172.21.2.111:0/1184959842 learned_addr learned my addr 172.21.2.111:0/1184959842 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:54.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 -- 172.21.2.111:0/1184959842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200b17b0 msgr2=0x7f0d20144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:54.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 --2- 172.21.2.111:0/1184959842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200b17b0 0x7f0d20144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:54.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 -- 172.21.2.111:0/1184959842 >> v1:172.21.2.111:6789/0 conn(0x7f0d200add50 legacy=0x7f0d20147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:54.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 -- 172.21.2.111:0/1184959842 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2014b330 con 0x7f0d200603f0 2021-04-03T03:15:54.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1ffff700 1 --2- 172.21.2.111:0/1184959842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200603f0 0x7f0d201447e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f0d080010d0 tx=0x7f0d10006e90).ready entity=mon.0 client_cookie=d4d6196894e73c83 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:54.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1effd700 1 -- 172.21.2.111:0/1184959842 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d100097e0 con 0x7f0d200603f0 2021-04-03T03:15:54.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1effd700 1 -- 172.21.2.111:0/1184959842 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10009940 con 0x7f0d200603f0 2021-04-03T03:15:54.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d1effd700 1 -- 172.21.2.111:0/1184959842 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d10009ca0 con 0x7f0d200603f0 2021-04-03T03:15:54.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- 172.21.2.111:0/1184959842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200603f0 msgr2=0x7f0d201447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:54.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 --2- 172.21.2.111:0/1184959842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200603f0 0x7f0d201447e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f0d080010d0 tx=0x7f0d10006e90).stop 2021-04-03T03:15:54.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- 172.21.2.111:0/1184959842 shutdown_connections 2021-04-03T03:15:54.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 --2- 172.21.2.111:0/1184959842 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200b17b0 0x7f0d20144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:54.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 --2- 172.21.2.111:0/1184959842 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200603f0 0x7f0d201447e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:54.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- 172.21.2.111:0/1184959842 >> 172.21.2.111:0/1184959842 conn(0x7f0d200b61c0 msgr2=0x7f0d2005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:54.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- 172.21.2.111:0/1184959842 shutdown_connections 2021-04-03T03:15:54.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.099+0000 7f0d2541b700 1 -- 172.21.2.111:0/1184959842 wait complete. 2021-04-03T03:15:54.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 Processor -- start 2021-04-03T03:15:54.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 -- start start 2021-04-03T03:15:54.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d200603f0 0x7f0d200a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:54.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200add50 0x7f0d20147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:54.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200b17b0 0x7f0d201568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:54.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20156db0 con 0x7f0d200b17b0 2021-04-03T03:15:54.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20156fa0 con 0x7f0d200603f0 2021-04-03T03:15:54.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20157240 con 0x7f0d200add50 2021-04-03T03:15:54.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200b17b0 0x7f0d201568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:54.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200b17b0 0x7f0d201568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55606/0 (socket says 172.21.2.111:55606) 2021-04-03T03:15:54.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 -- 172.21.2.111:0/3632200204 learned_addr learned my addr 172.21.2.111:0/3632200204 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:54.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1ffff700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d200603f0 0x7f0d200a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:54.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1f7fe700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200add50 0x7f0d20147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:54.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 -- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200add50 msgr2=0x7f0d20147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:54.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200add50 0x7f0d20147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:54.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 -- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d200603f0 msgr2=0x7f0d200a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:54.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d200603f0 0x7f0d200a9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:54.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 -- 172.21.2.111:0/3632200204 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d201574e0 con 0x7f0d200b17b0 2021-04-03T03:15:54.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d24c1a700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200b17b0 0x7f0d201568c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f0d180032b0 tx=0x7f0d180091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:54.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1d7fa700 1 -- 172.21.2.111:0/3632200204 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1800c980 con 0x7f0d200b17b0 2021-04-03T03:15:54.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1d7fa700 1 -- 172.21.2.111:0/3632200204 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1800cae0 con 0x7f0d200b17b0 2021-04-03T03:15:54.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1d7fa700 1 -- 172.21.2.111:0/3632200204 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1800ce40 con 0x7f0d200b17b0 2021-04-03T03:15:54.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d200a7f30 con 0x7f0d200b17b0 2021-04-03T03:15:54.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d200a83d0 con 0x7f0d200b17b0 2021-04-03T03:15:54.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d200a8a00 con 0x7f0d200b17b0 2021-04-03T03:15:54.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1d7fa700 1 -- 172.21.2.111:0/3632200204 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0d18015470 con 0x7f0d200b17b0 2021-04-03T03:15:54.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1d7fa700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d04056f70 0x7f0d04059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:54.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1d7fa700 1 -- 172.21.2.111:0/3632200204 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0d18070310 con 0x7f0d200b17b0 2021-04-03T03:15:54.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1ffff700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d04056f70 0x7f0d04059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:54.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.103+0000 7f0d1ffff700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d04056f70 0x7f0d04059430 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f0d1001a000 tx=0x7f0d10015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:54.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.107+0000 7f0d1d7fa700 1 -- 172.21.2.111:0/3632200204 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0d18045590 con 0x7f0d200b17b0 2021-04-03T03:15:54.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:54.231+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0d20157670 con 0x7f0d200b17b0 2021-04-03T03:15:55.120 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:15:55.119+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:15:55.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.127+0000 7f0d1d7fa700 1 -- 172.21.2.111:0/3632200204 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0d18048bb0 con 0x7f0d200b17b0 2021-04-03T03:15:55.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d04056f70 msgr2=0x7f0d04059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d04056f70 0x7f0d04059430 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f0d1001a000 tx=0x7f0d10015000).stop 2021-04-03T03:15:55.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200b17b0 msgr2=0x7f0d201568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200b17b0 0x7f0d201568c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f0d180032b0 tx=0x7f0d180091a0).stop 2021-04-03T03:15:55.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 shutdown_connections 2021-04-03T03:15:55.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d200b17b0 0x7f0d201568c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d04056f70 0x7f0d04059430 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d200603f0 0x7f0d200a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 --2- 172.21.2.111:0/3632200204 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d200add50 0x7f0d20147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 >> 172.21.2.111:0/3632200204 conn(0x7f0d200b61c0 msgr2=0x7f0d2014a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:55.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 shutdown_connections 2021-04-03T03:15:55.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.131+0000 7f0d2541b700 1 -- 172.21.2.111:0/3632200204 wait complete. 2021-04-03T03:15:55.138 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:15:55.142 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:15:55.147 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:15:55.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.195+0000 7f0429f22700 1 Processor -- start 2021-04-03T03:15:55.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.195+0000 7f0429f22700 1 -- start start 2021-04-03T03:15:55.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.195+0000 7f0429f22700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f04240603f0 0x7f04240b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.195+0000 7f0429f22700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f042414b700 0x7f04240b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.195+0000 7f0429f22700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04240607d0 con 0x7f04240b12b0 2021-04-03T03:15:55.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.195+0000 7f0429f22700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04240af400 con 0x7f042414b700 2021-04-03T03:15:55.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.195+0000 7f0429f22700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04240af6a0 con 0x7f04240603f0 2021-04-03T03:15:55.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f042414b700 0x7f04240b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f042414b700 0x7f04240b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55628/0 (socket says 172.21.2.111:55628) 2021-04-03T03:15:55.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1388680503 learned_addr learned my addr 172.21.2.111:0/1388680503 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:55.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0428f20700 1 --2- 172.21.2.111:0/1388680503 >> v2:172.21.2.115:3300/0 conn(0x7f04240603f0 0x7f04240b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1388680503 >> v2:172.21.2.115:3300/0 conn(0x7f04240603f0 msgr2=0x7f04240b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:55.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- 172.21.2.111:0/1388680503 >> v2:172.21.2.115:3300/0 conn(0x7f04240603f0 0x7f04240b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1388680503 >> v1:172.21.2.115:6789/0 conn(0x7f04240b12b0 legacy=0x7f04241465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1388680503 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04240afb40 con 0x7f042414b700 2021-04-03T03:15:55.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- 172.21.2.111:0/1388680503 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f042414b700 0x7f04240b0580 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f040c002f30 tx=0x7f040c006e90).ready entity=mon.0 client_cookie=82923b25928b9ad1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:55.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f04237fe700 1 -- 172.21.2.111:0/1388680503 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f040c009850 con 0x7f042414b700 2021-04-03T03:15:55.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f04237fe700 1 -- 172.21.2.111:0/1388680503 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c0099b0 con 0x7f042414b700 2021-04-03T03:15:55.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f04237fe700 1 -- 172.21.2.111:0/1388680503 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f040c009d10 con 0x7f042414b700 2021-04-03T03:15:55.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1388680503 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f042414b700 msgr2=0x7f04240b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 --2- 172.21.2.111:0/1388680503 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f042414b700 0x7f04240b0580 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f040c002f30 tx=0x7f040c006e90).stop 2021-04-03T03:15:55.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1388680503 shutdown_connections 2021-04-03T03:15:55.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 --2- 172.21.2.111:0/1388680503 >> v2:172.21.2.115:3300/0 conn(0x7f04240603f0 0x7f04240b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 --2- 172.21.2.111:0/1388680503 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f042414b700 0x7f04240b0580 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1388680503 >> 172.21.2.111:0/1388680503 conn(0x7f04240b61c0 msgr2=0x7f042405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:55.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1388680503 shutdown_connections 2021-04-03T03:15:55.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1388680503 wait complete. 2021-04-03T03:15:55.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 Processor -- start 2021-04-03T03:15:55.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- start start 2021-04-03T03:15:55.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04240603f0 0x7f0424156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04240b12b0 0x7f0424062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f042414b700 0x7f0424146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042414eac0 con 0x7f04240b12b0 2021-04-03T03:15:55.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0424157460 con 0x7f042414b700 2021-04-03T03:15:55.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04240a9f70 con 0x7f04240603f0 2021-04-03T03:15:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04240b12b0 0x7f0424062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04240b12b0 0x7f0424062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55634/0 (socket says 172.21.2.111:55634) 2021-04-03T03:15:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1253308528 learned_addr learned my addr 172.21.2.111:0/1253308528 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429721700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f042414b700 0x7f0424146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0428f20700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04240603f0 0x7f0424156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04240603f0 msgr2=0x7f0424156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04240603f0 0x7f0424156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f042414b700 msgr2=0x7f0424146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f042414b700 0x7f0424146a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 -- 172.21.2.111:0/1253308528 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04240aa410 con 0x7f04240b12b0 2021-04-03T03:15:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0428f20700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04240603f0 0x7f0424156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0423fff700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04240b12b0 0x7f0424062bf0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f040c002f30 tx=0x7f040c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0421ffb700 1 -- 172.21.2.111:0/1253308528 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f040c011c30 con 0x7f04240b12b0 2021-04-03T03:15:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0421ffb700 1 -- 172.21.2.111:0/1253308528 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c011d90 con 0x7f04240b12b0 2021-04-03T03:15:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0421ffb700 1 -- 172.21.2.111:0/1253308528 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f040c01e650 con 0x7f04240b12b0 2021-04-03T03:15:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04240aa660 con 0x7f04240b12b0 2021-04-03T03:15:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04240ab1d0 con 0x7f04240b12b0 2021-04-03T03:15:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.199+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0424057660 con 0x7f04240b12b0 2021-04-03T03:15:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.203+0000 7f0421ffb700 1 -- 172.21.2.111:0/1253308528 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f040c01a040 con 0x7f04240b12b0 2021-04-03T03:15:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.203+0000 7f0421ffb700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0408056f60 0x7f0408059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.203+0000 7f0421ffb700 1 -- 172.21.2.111:0/1253308528 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f040c0710e0 con 0x7f04240b12b0 2021-04-03T03:15:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.203+0000 7f0428f20700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0408056f60 0x7f0408059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.203+0000 7f0428f20700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0408056f60 0x7f0408059420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f041400b000 tx=0x7f0414008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:55.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.203+0000 7f0421ffb700 1 -- 172.21.2.111:0/1253308528 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f040c046470 con 0x7f04240b12b0 2021-04-03T03:15:55.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04240aa130 con 0x7f04240b12b0 2021-04-03T03:15:55.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0421ffb700 1 -- 172.21.2.111:0/1253308528 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f040c023090 con 0x7f04240b12b0 2021-04-03T03:15:55.335 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:55.335 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":39,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3837436656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3837436656},{"type":"v1","addr":"172.21.2.115:6833","nonce":3837436656}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:15:44.064474+0000","modified":"2021-04-03T03:15:55.121548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-04-03T03:15:55.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0408056f60 msgr2=0x7f0408059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0408056f60 0x7f0408059420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f041400b000 tx=0x7f0414008040).stop 2021-04-03T03:15:55.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04240b12b0 msgr2=0x7f0424062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04240b12b0 0x7f0424062bf0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f040c002f30 tx=0x7f040c015000).stop 2021-04-03T03:15:55.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 shutdown_connections 2021-04-03T03:15:55.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0408056f60 0x7f0408059420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f042414b700 0x7f0424146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04240603f0 0x7f0424156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 --2- 172.21.2.111:0/1253308528 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04240b12b0 0x7f0424062bf0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 >> 172.21.2.111:0/1253308528 conn(0x7f04240b61c0 msgr2=0x7f0424147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:55.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 shutdown_connections 2021-04-03T03:15:55.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.331+0000 7f0429f22700 1 -- 172.21.2.111:0/1253308528 wait complete. 2021-04-03T03:15:55.338 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 39 2021-04-03T03:15:55.342 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:15:55.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 Processor -- start 2021-04-03T03:15:55.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 -- start start 2021-04-03T03:15:55.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 0x7f529c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f529c144ef0 0x7f529c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c0607d0 con 0x7f529c0603f0 2021-04-03T03:15:55.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c14b090 con 0x7f529c144ef0 2021-04-03T03:15:55.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c14b330 con 0x7f529c0add50 2021-04-03T03:15:55.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529a59c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f529c0603f0 0x7f529c1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56430/0 (socket says 172.21.2.111:56430) 2021-04-03T03:15:55.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529a59c700 1 -- 172.21.2.111:0/2271512197 learned_addr learned my addr 172.21.2.111:0/2271512197 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:55.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892637980 0 0) 0x7f529c0607d0 con 0x7f529c0603f0 2021-04-03T03:15:55.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f527c003240 con 0x7f529c0603f0 2021-04-03T03:15:55.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 --2- 172.21.2.111:0/2271512197 >> v2:172.21.2.115:3300/0 conn(0x7f529c144ef0 0x7f529c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f5299d9b700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 0x7f529c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2702962021 0 0) 0x7f527c003240 con 0x7f529c0603f0 2021-04-03T03:15:55.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 msgr2=0x7f529c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:55.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 0x7f529c147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 >> v2:172.21.2.115:3300/0 conn(0x7f529c144ef0 msgr2=0x7f529c147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:55.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 --2- 172.21.2.111:0/2271512197 >> v2:172.21.2.115:3300/0 conn(0x7f529c144ef0 0x7f529c147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c14b5d0 con 0x7f529c0603f0 2021-04-03T03:15:55.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f528c0016d0 con 0x7f529c0603f0 2021-04-03T03:15:55.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f5299d9b700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 0x7f529c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:55.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 >> v1:172.21.2.111:6789/0 conn(0x7f529c0603f0 legacy=0x7f529c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 0x7f527c0037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f527c003cf0 0x7f527c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f527c0049d0 0x7f527c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c14b090 con 0x7f527c0049d0 2021-04-03T03:15:55.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c14b330 con 0x7f527c003cf0 2021-04-03T03:15:55.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0042b0 con 0x7f529c0add50 2021-04-03T03:15:55.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 --2- 172.21.2.111:0/2271512197 >> v2:172.21.2.115:3300/0 conn(0x7f529c144ef0 0x7f529c147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:15:55.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f527c0049d0 0x7f527c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 -- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 msgr2=0x7f527c0037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:55.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0add50 0x7f527c0037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:55.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 -- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f527c003cf0 msgr2=0x7f527c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:55.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f527c003cf0 0x7f527c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:55.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 -- 172.21.2.111:0/2271512197 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c003240 con 0x7f527c0049d0 2021-04-03T03:15:55.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f527c0049d0 0x7f527c004e50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f529400a000 tx=0x7f5294008040).ready entity=mon.0 client_cookie=704b35b1f23956e2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:55.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529400c070 con 0x7f527c0049d0 2021-04-03T03:15:55.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400e9a0 con 0x7f527c0049d0 2021-04-03T03:15:55.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529959a700 1 -- 172.21.2.111:0/2271512197 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529400ed00 con 0x7f527c0049d0 2021-04-03T03:15:55.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 -- 172.21.2.111:0/2271512197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f527c0049d0 msgr2=0x7f527c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529b59e700 1 --2- 172.21.2.111:0/2271512197 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f527c0049d0 0x7f527c004e50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f529400a000 tx=0x7f5294008040).stop 2021-04-03T03:15:55.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.455+0000 7f529ad9d700 1 -- 172.21.2.111:0/2271512197 reap_dead start 2021-04-03T03:15:55.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- 172.21.2.111:0/2271512197 shutdown_connections 2021-04-03T03:15:55.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- 172.21.2.111:0/2271512197 >> 172.21.2.111:0/2271512197 conn(0x7f529c0b61c0 msgr2=0x7f529c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:55.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- 172.21.2.111:0/2271512197 shutdown_connections 2021-04-03T03:15:55.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- 172.21.2.111:0/2271512197 wait complete. 2021-04-03T03:15:55.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 Processor -- start 2021-04-03T03:15:55.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- start start 2021-04-03T03:15:55.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0a7240 0x7f529c13f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f529c0a79d0 0x7f529c13fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f529c0aa240 0x7f529c0aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0a8de0 con 0x7f529c0aa240 2021-04-03T03:15:55.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0a7620 con 0x7f529c0a79d0 2021-04-03T03:15:55.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0a8400 con 0x7f529c0a7240 2021-04-03T03:15:55.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f529c0aa240 0x7f529c0aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f529c0aa240 0x7f529c0aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55652/0 (socket says 172.21.2.111:55652) 2021-04-03T03:15:55.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 -- 172.21.2.111:0/466684331 learned_addr learned my addr 172.21.2.111:0/466684331 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:55.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529a59c700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0a7240 0x7f529c13f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 -- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0a7240 msgr2=0x7f529c13f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f5299d9b700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f529c0a79d0 0x7f529c13fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0a7240 0x7f529c13f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 -- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f529c0a79d0 msgr2=0x7f529c13fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:55.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f529c0a79d0 0x7f529c13fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:55.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 -- 172.21.2.111:0/466684331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0a86a0 con 0x7f529c0aa240 2021-04-03T03:15:55.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529ad9d700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f529c0aa240 0x7f529c0aa6c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f528c001830 tx=0x7f5294008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:55.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f52937fe700 1 -- 172.21.2.111:0/466684331 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5294013e00 con 0x7f529c0aa240 2021-04-03T03:15:55.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f52937fe700 1 -- 172.21.2.111:0/466684331 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529401d410 con 0x7f529c0aa240 2021-04-03T03:15:55.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c0a8940 con 0x7f529c0aa240 2021-04-03T03:15:55.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f52937fe700 1 -- 172.21.2.111:0/466684331 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529401d770 con 0x7f529c0aa240 2021-04-03T03:15:55.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c0a8060 con 0x7f529c0aa240 2021-04-03T03:15:55.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c0abd60 con 0x7f529c0aa240 2021-04-03T03:15:55.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f52937fe700 1 -- 172.21.2.111:0/466684331 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f529401d8d0 con 0x7f529c0aa240 2021-04-03T03:15:55.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f52937fe700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5280056fb0 0x7f5280059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:55.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f52937fe700 1 -- 172.21.2.111:0/466684331 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f5294071500 con 0x7f529c0aa240 2021-04-03T03:15:55.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529a59c700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5280056fb0 0x7f5280059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:55.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.459+0000 7f529a59c700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5280056fb0 0x7f5280059470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f528c008000 tx=0x7f528c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:55.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.463+0000 7f52937fe700 1 -- 172.21.2.111:0/466684331 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f529c0abd60 con 0x7f529c0aa240 2021-04-03T03:15:55.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:55.587+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f529c0abd60 con 0x7f529c0aa240 2021-04-03T03:15:56.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f52937fe700 1 -- 172.21.2.111:0/466684331 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f529c0abd60 con 0x7f529c0aa240 2021-04-03T03:15:56.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5280056fb0 msgr2=0x7f5280059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:56.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5280056fb0 0x7f5280059470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f528c008000 tx=0x7f528c009040).stop 2021-04-03T03:15:56.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f529c0aa240 msgr2=0x7f529c0aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:56.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f529c0aa240 0x7f529c0aa6c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f528c001830 tx=0x7f5294008040).stop 2021-04-03T03:15:56.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 shutdown_connections 2021-04-03T03:15:56.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f529c0aa240 0x7f529c0aa6c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f529c0a7240 0x7f529c13f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5280056fb0 0x7f5280059470 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 --2- 172.21.2.111:0/466684331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f529c0a79d0 0x7f529c13fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 >> 172.21.2.111:0/466684331 conn(0x7f529c0b61c0 msgr2=0x7f529c142d60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:56.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 shutdown_connections 2021-04-03T03:15:56.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.135+0000 7f529b59e700 1 -- 172.21.2.111:0/466684331 wait complete. 2021-04-03T03:15:56.146 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:15:56.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.199+0000 7f31a1fd0700 1 Processor -- start 2021-04-03T03:15:56.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- start start 2021-04-03T03:15:56.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f319c0b18b0 0x7f319c067b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c140b60 con 0x7f319c0603f0 2021-04-03T03:15:56.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c068000 con 0x7f319c13d010 2021-04-03T03:15:56.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319c0af3f0 con 0x7f319c0b18b0 2021-04-03T03:15:56.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a0fce700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f319c0603f0 0x7f319c0acb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56452/0 (socket says 172.21.2.111:56452) 2021-04-03T03:15:56.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a0fce700 1 -- 172.21.2.111:0/156806980 learned_addr learned my addr 172.21.2.111:0/156806980 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:56.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701957915 0 0) 0x7f319c140b60 con 0x7f319c0603f0 2021-04-03T03:15:56.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3188003240 con 0x7f319c0603f0 2021-04-03T03:15:56.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 178159249 0 0) 0x7f3188003240 con 0x7f319c0603f0 2021-04-03T03:15:56.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 >> v2:172.21.2.115:3300/0 conn(0x7f319c0b18b0 msgr2=0x7f319c067b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:56.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 --2- 172.21.2.111:0/156806980 >> v2:172.21.2.115:3300/0 conn(0x7f319c0b18b0 0x7f319c067b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:56.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 >> v1:172.21.2.115:6789/0 conn(0x7f319c13d010 legacy=0x7f319c13d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:56.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0607d0 con 0x7f319c0603f0 2021-04-03T03:15:56.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f3194001a50 con 0x7f319c0603f0 2021-04-03T03:15:56.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 >> v1:172.21.2.111:6789/0 conn(0x7f319c0603f0 legacy=0x7f319c0acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:56.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3188003800 0x7f3188003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31880044e0 0x7f3188004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3188004e50 0x7f318800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0af3f0 con 0x7f3188004e50 2021-04-03T03:15:56.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188005300 con 0x7f31880044e0 2021-04-03T03:15:56.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318800bc20 con 0x7f3188003800 2021-04-03T03:15:56.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3188004e50 0x7f318800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:56.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319bfff700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31880044e0 0x7f3188004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:56.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a0fce700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3188003800 0x7f3188003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:56.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 -- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3188003800 msgr2=0x7f3188003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:56.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3188003800 0x7f3188003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 -- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31880044e0 msgr2=0x7f3188004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:56.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31880044e0 0x7f3188004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 -- 172.21.2.111:0/156806980 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3188003240 con 0x7f3188004e50 2021-04-03T03:15:56.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319bfff700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31880044e0 0x7f3188004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:56.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a0fce700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3188003800 0x7f3188003c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:56.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 -- 172.21.2.111:0/156806980 reap_dead start 2021-04-03T03:15:56.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a17cf700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3188004e50 0x7f318800b730 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f318c000f30 tx=0x7f318c007b50).ready entity=mon.0 client_cookie=6d08b06af6ccc05 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:56.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318c00c660 con 0x7f3188004e50 2021-04-03T03:15:56.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c00c7c0 con 0x7f3188004e50 2021-04-03T03:15:56.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f319b7fe700 1 -- 172.21.2.111:0/156806980 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318c00cb20 con 0x7f3188004e50 2021-04-03T03:15:56.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- 172.21.2.111:0/156806980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3188004e50 msgr2=0x7f318800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:56.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3188004e50 0x7f318800b730 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f318c000f30 tx=0x7f318c007b50).stop 2021-04-03T03:15:56.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- 172.21.2.111:0/156806980 shutdown_connections 2021-04-03T03:15:56.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/156806980 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3188004e50 0x7f318800b730 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- 172.21.2.111:0/156806980 >> 172.21.2.111:0/156806980 conn(0x7f319c0b61c0 msgr2=0x7f319c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:56.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- 172.21.2.111:0/156806980 shutdown_connections 2021-04-03T03:15:56.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- 172.21.2.111:0/156806980 wait complete. 2021-04-03T03:15:56.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 Processor -- start 2021-04-03T03:15:56.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.203+0000 7f31a1fd0700 1 -- start start 2021-04-03T03:15:56.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3188004e50 0x7f319c0661d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f319c1435a0 0x7f319c066740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f319c152250 0x7f319c152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c143a20 con 0x7f319c152250 2021-04-03T03:15:56.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c066e00 con 0x7f3188004e50 2021-04-03T03:15:56.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0670a0 con 0x7f319c1435a0 2021-04-03T03:15:56.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f319c152250 0x7f319c152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:56.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f319c152250 0x7f319c152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55674/0 (socket says 172.21.2.111:55674) 2021-04-03T03:15:56.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 -- 172.21.2.111:0/2339204352 learned_addr learned my addr 172.21.2.111:0/2339204352 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:56.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f319bfff700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f319c1435a0 0x7f319c066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:56.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a0fce700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3188004e50 0x7f319c0661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:56.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 -- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f319c1435a0 msgr2=0x7f319c066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:56.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f319c1435a0 0x7f319c066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 -- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3188004e50 msgr2=0x7f319c0661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:56.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3188004e50 0x7f319c0661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:56.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 -- 172.21.2.111:0/2339204352 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c151d40 con 0x7f319c152250 2021-04-03T03:15:56.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a17cf700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f319c152250 0x7f319c152700 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f318c000f30 tx=0x7f318c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:56.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f3199ffb700 1 -- 172.21.2.111:0/2339204352 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318c013aa0 con 0x7f319c152250 2021-04-03T03:15:56.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f3199ffb700 1 -- 172.21.2.111:0/2339204352 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c013c00 con 0x7f319c152250 2021-04-03T03:15:56.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f3199ffb700 1 -- 172.21.2.111:0/2339204352 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318c020410 con 0x7f319c152250 2021-04-03T03:15:56.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c152bf0 con 0x7f319c152250 2021-04-03T03:15:56.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c155500 con 0x7f319c152250 2021-04-03T03:15:56.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c066c30 con 0x7f319c152250 2021-04-03T03:15:56.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f3199ffb700 1 -- 172.21.2.111:0/2339204352 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f318c010070 con 0x7f319c152250 2021-04-03T03:15:56.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f3199ffb700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f317c056f20 0x7f317c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:56.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f3199ffb700 1 -- 172.21.2.111:0/2339204352 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f318c019030 con 0x7f319c152250 2021-04-03T03:15:56.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a0fce700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f317c056f20 0x7f317c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:56.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.207+0000 7f31a0fce700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f317c056f20 0x7f317c0593e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f319400e000 tx=0x7f3194001b40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:56.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.211+0000 7f3199ffb700 1 -- 172.21.2.111:0/2339204352 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f318c045c90 con 0x7f319c152250 2021-04-03T03:15:56.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:56.303+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f319c151f30 con 0x7f319c152250 2021-04-03T03:15:57.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f3199ffb700 1 -- 172.21.2.111:0/2339204352 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 185+0+0 (secure 0 0 0) 0x7f318c0492b0 con 0x7f319c152250 2021-04-03T03:15:57.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f317c056f20 msgr2=0x7f317c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:57.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f317c056f20 0x7f317c0593e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f319400e000 tx=0x7f3194001b40).stop 2021-04-03T03:15:57.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f319c152250 msgr2=0x7f319c152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:57.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f319c152250 0x7f319c152700 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f318c000f30 tx=0x7f318c015000).stop 2021-04-03T03:15:57.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 shutdown_connections 2021-04-03T03:15:57.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f319c152250 0x7f319c152700 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:57.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f317c056f20 0x7f317c0593e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:57.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3188004e50 0x7f319c0661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:57.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 --2- 172.21.2.111:0/2339204352 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f319c1435a0 0x7f319c066740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:57.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.179+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 >> 172.21.2.111:0/2339204352 conn(0x7f319c0b61c0 msgr2=0x7f319c13c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:57.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.183+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 shutdown_connections 2021-04-03T03:15:57.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.183+0000 7f31a1fd0700 1 -- 172.21.2.111:0/2339204352 wait complete. 2021-04-03T03:15:57.188 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:15:57.195 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:15:57.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 Processor -- start 2021-04-03T03:15:57.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- start start 2021-04-03T03:15:57.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdcfc00ad10 0x7fdcfc0d7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc00b0f0 con 0x7fdcfc008820 2021-04-03T03:15:57.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc0265f0 con 0x7fdcfc0d44d0 2021-04-03T03:15:57.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc026a90 con 0x7fdcfc00ad10 2021-04-03T03:15:57.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fdcfc008820 0x7fdcfc0d7d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56474/0 (socket says 172.21.2.111:56474) 2021-04-03T03:15:57.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 -- 172.21.2.111:0/1771611130 learned_addr learned my addr 172.21.2.111:0/1771611130 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:57.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd08ff9700 1 --2- 172.21.2.111:0/1771611130 >> v2:172.21.2.115:3300/0 conn(0x7fdcfc00ad10 0x7fdcfc0d7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:57.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256398921 0 0) 0x7fdcfc00b0f0 con 0x7fdcfc008820 2021-04-03T03:15:57.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce8003240 con 0x7fdcfc008820 2021-04-03T03:15:57.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 737376495 0 0) 0x7fdce8003240 con 0x7fdcfc008820 2021-04-03T03:15:57.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 >> v2:172.21.2.115:3300/0 conn(0x7fdcfc00ad10 msgr2=0x7fdcfc0d7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:57.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 --2- 172.21.2.111:0/1771611130 >> v2:172.21.2.115:3300/0 conn(0x7fdcfc00ad10 0x7fdcfc0d7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:57.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 >> v1:172.21.2.115:6789/0 conn(0x7fdcfc0d44d0 legacy=0x7fdcfc0d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:57.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0d8470 con 0x7fdcfc008820 2021-04-03T03:15:57.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fdcf00016d0 con 0x7fdcfc008820 2021-04-03T03:15:57.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 >> v1:172.21.2.111:6789/0 conn(0x7fdcfc008820 legacy=0x7fdcfc0d7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:57.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdce8003780 0x7fdce8003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdce8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdce8004e50 0x7fdce800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8005300 con 0x7fdce80044e0 2021-04-03T03:15:57.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce800bc20 con 0x7fdce8003780 2021-04-03T03:15:57.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce800bd60 con 0x7fdce8004e50 2021-04-03T03:15:57.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdce8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:57.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 -- 172.21.2.111:0/1771611130 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdce8004e50 msgr2=0x7fdce800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:57.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdce8004e50 0x7fdce800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:57.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 -- 172.21.2.111:0/1771611130 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdce8003780 msgr2=0x7fdce8003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:57.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdce8003780 0x7fdce8003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:57.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 -- 172.21.2.111:0/1771611130 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8003240 con 0x7fdce80044e0 2021-04-03T03:15:57.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd097fa700 1 -- 172.21.2.111:0/1771611130 reap_dead start 2021-04-03T03:15:57.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd03fff700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdce8004960 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fdcf00013e0 tx=0x7fdcfc007180).ready entity=mon.0 client_cookie=d15199c54349eaca server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:57.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcf000ce00 con 0x7fdce80044e0 2021-04-03T03:15:57.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf00070d0 con 0x7fdce80044e0 2021-04-03T03:15:57.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd037fe700 1 -- 172.21.2.111:0/1771611130 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcf0007430 con 0x7fdce80044e0 2021-04-03T03:15:57.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- 172.21.2.111:0/1771611130 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 msgr2=0x7fdce8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:57.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdce8004960 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fdcf00013e0 tx=0x7fdcfc007180).stop 2021-04-03T03:15:57.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- 172.21.2.111:0/1771611130 shutdown_connections 2021-04-03T03:15:57.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/1771611130 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdce8004960 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:57.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- 172.21.2.111:0/1771611130 >> 172.21.2.111:0/1771611130 conn(0x7fdcfc0473d0 msgr2=0x7fdcfc0477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:57.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- 172.21.2.111:0/1771611130 shutdown_connections 2021-04-03T03:15:57.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.251+0000 7fdd09ffb700 1 -- 172.21.2.111:0/1771611130 wait complete. 2021-04-03T03:15:57.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 Processor -- start 2021-04-03T03:15:57.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 -- start start 2021-04-03T03:15:57.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdcfc0e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdcfc02c5c0 0x7fdcfc03cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdcfc03c330 0x7fdcfc02a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc03d090 con 0x7fdce80044e0 2021-04-03T03:15:57.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc02d200 con 0x7fdcfc02c5c0 2021-04-03T03:15:57.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc03d280 con 0x7fdcfc03c330 2021-04-03T03:15:57.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdcfc0e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:57.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdcfc0e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55692/0 (socket says 172.21.2.111:55692) 2021-04-03T03:15:57.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 -- 172.21.2.111:0/2955578214 learned_addr learned my addr 172.21.2.111:0/2955578214 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:57.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 -- 172.21.2.111:0/2955578214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdcfc03c330 msgr2=0x7fdcfc02a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:57.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdcfc03c330 0x7fdcfc02a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:57.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 -- 172.21.2.111:0/2955578214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdcfc02c5c0 msgr2=0x7fdcfc03cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:57.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdcfc02c5c0 0x7fdcfc03cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:57.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 -- 172.21.2.111:0/2955578214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc03d520 con 0x7fdce80044e0 2021-04-03T03:15:57.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd08ff9700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdcfc0e8070 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fdcfc02d340 tx=0x7fdd0405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:57.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd01ffb700 1 -- 172.21.2.111:0/2955578214 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd0405b040 con 0x7fdce80044e0 2021-04-03T03:15:57.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd01ffb700 1 -- 172.21.2.111:0/2955578214 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0405f2c0 con 0x7fdce80044e0 2021-04-03T03:15:57.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc02a850 con 0x7fdce80044e0 2021-04-03T03:15:57.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd01ffb700 1 -- 172.21.2.111:0/2955578214 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd0405b1a0 con 0x7fdce80044e0 2021-04-03T03:15:57.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc02acc0 con 0x7fdce80044e0 2021-04-03T03:15:57.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc005200 con 0x7fdce80044e0 2021-04-03T03:15:57.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd01ffb700 1 -- 172.21.2.111:0/2955578214 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdd0405f620 con 0x7fdce80044e0 2021-04-03T03:15:57.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd01ffb700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcec056fb0 0x7fdcec059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:57.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd01ffb700 1 -- 172.21.2.111:0/2955578214 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6064+0+0 (secure 0 0 0) 0x7fdd040dfb30 con 0x7fdce80044e0 2021-04-03T03:15:57.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd03fff700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcec056fb0 0x7fdcec059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:57.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.255+0000 7fdd03fff700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcec056fb0 0x7fdcec059470 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fdcf00013e0 tx=0x7fdcf0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:57.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.259+0000 7fdd01ffb700 1 -- 172.21.2.111:0/2955578214 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdd040b5080 con 0x7fdce80044e0 2021-04-03T03:15:57.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:57.351+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdcfc041a80 con 0x7fdce80044e0 2021-04-03T03:15:58.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.187+0000 7fdd01ffb700 1 -- 172.21.2.111:0/2955578214 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdd040b86a0 con 0x7fdce80044e0 2021-04-03T03:15:58.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcec056fb0 msgr2=0x7fdcec059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:58.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcec056fb0 0x7fdcec059470 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fdcf00013e0 tx=0x7fdcf0015000).stop 2021-04-03T03:15:58.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 msgr2=0x7fdcfc0e8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:58.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdcfc0e8070 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fdcfc02d340 tx=0x7fdd0405ad70).stop 2021-04-03T03:15:58.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 shutdown_connections 2021-04-03T03:15:58.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdcfc03c330 0x7fdcfc02a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:58.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdce80044e0 0x7fdcfc0e8070 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:58.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcec056fb0 0x7fdcec059470 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:58.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 --2- 172.21.2.111:0/2955578214 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdcfc02c5c0 0x7fdcfc03cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:58.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 >> 172.21.2.111:0/2955578214 conn(0x7fdcfc0473d0 msgr2=0x7fdcfc0062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:58.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 shutdown_connections 2021-04-03T03:15:58.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.191+0000 7fdd09ffb700 1 -- 172.21.2.111:0/2955578214 wait complete. 2021-04-03T03:15:58.199 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:15:58.201 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:15:58.204 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:15:58.205 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:15:58.205 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:15:58.205 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:15:58.206 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:15:58.207 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:15:58.207 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:15:58.211 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:15:58.211 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:15:58.214 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:15:58.214 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:15:58.261 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:15:58.262 DEBUG:teuthology.parallel:result is None 2021-04-03T03:15:58.263 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:15:58.264 DEBUG:teuthology.parallel:result is None 2021-04-03T03:15:58.265 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:15:58.306 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:15:58.313 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:15:58.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 Processor -- start 2021-04-03T03:15:58.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- start start 2021-04-03T03:15:58.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f594403dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:58.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440d52f0 con 0x7f5944048f00 2021-04-03T03:15:58.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440492e0 con 0x7f59440d5e30 2021-04-03T03:15:58.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594403be30 con 0x7f5944043480 2021-04-03T03:15:58.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5951afd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5944048f00 0x7f59440d5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56504/0 (socket says 172.21.2.111:56504) 2021-04-03T03:15:58.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5951afd700 1 -- 172.21.2.111:0/347869865 learned_addr learned my addr 172.21.2.111:0/347869865 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:58.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f59522fe700 1 --2- 172.21.2.111:0/347869865 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f594403dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:58.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5950afb700 1 -- 172.21.2.111:0/347869865 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736723773 0 0) 0x7f59440d52f0 con 0x7f5944048f00 2021-04-03T03:15:58.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5950afb700 1 -- 172.21.2.111:0/347869865 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5934003240 con 0x7f5944048f00 2021-04-03T03:15:58.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f59522fe700 1 -- 172.21.2.111:0/347869865 >> v1:172.21.2.115:6789/0 conn(0x7f59440d5e30 legacy=0x7f59440ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:58.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5950afb700 1 -- 172.21.2.111:0/347869865 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198797196 0 0) 0x7f59440492e0 con 0x7f59440d5e30 2021-04-03T03:15:58.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f59522fe700 1 -- 172.21.2.111:0/347869865 >> v1:172.21.2.111:6789/0 conn(0x7f5944048f00 legacy=0x7f59440d5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:58.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f59522fe700 1 -- 172.21.2.111:0/347869865 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594403c0d0 con 0x7f5944043480 2021-04-03T03:15:58.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f59522fe700 1 --2- 172.21.2.111:0/347869865 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f594403dfa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f594c06b5d0 tx=0x7f594c05ad70).ready entity=mon.2 client_cookie=528fb9f4e9b57b1b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:58.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5950afb700 1 -- 172.21.2.111:0/347869865 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594c05bda0 con 0x7f5944043480 2021-04-03T03:15:58.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5950afb700 1 -- 172.21.2.111:0/347869865 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c05df60 con 0x7f5944043480 2021-04-03T03:15:58.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5950afb700 1 -- 172.21.2.111:0/347869865 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594c0699f0 con 0x7f5944043480 2021-04-03T03:15:58.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- 172.21.2.111:0/347869865 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 msgr2=0x7f594403dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:58.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 --2- 172.21.2.111:0/347869865 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f594403dfa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f594c06b5d0 tx=0x7f594c05ad70).stop 2021-04-03T03:15:58.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- 172.21.2.111:0/347869865 shutdown_connections 2021-04-03T03:15:58.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 --2- 172.21.2.111:0/347869865 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f594403dfa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:58.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- 172.21.2.111:0/347869865 >> 172.21.2.111:0/347869865 conn(0x7f594400f320 msgr2=0x7f59440484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:58.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- 172.21.2.111:0/347869865 shutdown_connections 2021-04-03T03:15:58.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- 172.21.2.111:0/347869865 wait complete. 2021-04-03T03:15:58.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 Processor -- start 2021-04-03T03:15:58.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- start start 2021-04-03T03:15:58.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f5944042330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:58.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5944048f00 0x7f594403e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:58.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59440d5e30 0x7f594403f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:58.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440d2b90 con 0x7f59440d5e30 2021-04-03T03:15:58.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594403fee0 con 0x7f5944048f00 2021-04-03T03:15:58.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.335+0000 7f5952aff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440c4590 con 0x7f5944043480 2021-04-03T03:15:58.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59440d5e30 0x7f594403f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:58.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59440d5e30 0x7f594403f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55722/0 (socket says 172.21.2.111:55722) 2021-04-03T03:15:58.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 -- 172.21.2.111:0/1460138753 learned_addr learned my addr 172.21.2.111:0/1460138753 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:58.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59512fc700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5944048f00 0x7f594403e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:58.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f5951afd700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f5944042330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:58.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 -- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 msgr2=0x7f5944042330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:58.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f5944042330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:58.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 -- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5944048f00 msgr2=0x7f594403e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:58.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5944048f00 0x7f594403e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:58.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 -- 172.21.2.111:0/1460138753 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440c4830 con 0x7f59440d5e30 2021-04-03T03:15:58.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f59522fe700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59440d5e30 0x7f594403f510 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f594c06b5d0 tx=0x7f594c064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:58.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f593effd700 1 -- 172.21.2.111:0/1460138753 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594c0699f0 con 0x7f59440d5e30 2021-04-03T03:15:58.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f593effd700 1 -- 172.21.2.111:0/1460138753 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c05d770 con 0x7f59440d5e30 2021-04-03T03:15:58.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59440d2810 con 0x7f59440d5e30 2021-04-03T03:15:58.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f593effd700 1 -- 172.21.2.111:0/1460138753 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594c05f2c0 con 0x7f59440d5e30 2021-04-03T03:15:58.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59440df250 con 0x7f59440d5e30 2021-04-03T03:15:58.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944004d30 con 0x7f59440d5e30 2021-04-03T03:15:58.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f593effd700 1 -- 172.21.2.111:0/1460138753 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f594c08c020 con 0x7f59440d5e30 2021-04-03T03:15:58.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f593effd700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f592c056ec0 0x7f592c059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:58.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f593effd700 1 -- 172.21.2.111:0/1460138753 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f594c0e0330 con 0x7f59440d5e30 2021-04-03T03:15:58.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f5951afd700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f592c056ec0 0x7f592c059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:58.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f5951afd700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f592c056ec0 0x7f592c059380 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f594000d000 tx=0x7f5940009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:58.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.339+0000 7f593effd700 1 -- 172.21.2.111:0/1460138753 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f594c0b59f0 con 0x7f59440d5e30 2021-04-03T03:15:58.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:58.447+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f594400a600 con 0x7f59440d5e30 2021-04-03T03:15:59.156 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:59.159 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:59.162 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:59.165 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:59.168 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:15:59.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.327+0000 7f593effd700 1 -- 172.21.2.111:0/1460138753 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7f594c0b9010 con 0x7f59440d5e30 2021-04-03T03:15:59.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f592c056ec0 msgr2=0x7f592c059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f592c056ec0 0x7f592c059380 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f594000d000 tx=0x7f5940009040).stop 2021-04-03T03:15:59.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59440d5e30 msgr2=0x7f594403f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59440d5e30 0x7f594403f510 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f594c06b5d0 tx=0x7f594c064030).stop 2021-04-03T03:15:59.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 shutdown_connections 2021-04-03T03:15:59.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59440d5e30 0x7f594403f510 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5944043480 0x7f5944042330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f592c056ec0 0x7f592c059380 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 --2- 172.21.2.111:0/1460138753 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5944048f00 0x7f594403e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 >> 172.21.2.111:0/1460138753 conn(0x7f594400f320 msgr2=0x7f5944005db0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 shutdown_connections 2021-04-03T03:15:59.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.331+0000 7f5952aff700 1 -- 172.21.2.111:0/1460138753 wait complete. 2021-04-03T03:15:59.339 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:15:59.348 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:15:59.349 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:15:59.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 Processor -- start 2021-04-03T03:15:59.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- start start 2021-04-03T03:15:59.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d414b700 0x7ff3d40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d40607d0 con 0x7ff3d40b0c90 2021-04-03T03:15:59.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d40af400 con 0x7ff3d40603f0 2021-04-03T03:15:59.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40af6a0 con 0x7ff3d414b700 2021-04-03T03:15:59.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d3fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff3d40b0c90 0x7ff3d4147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56524/0 (socket says 172.21.2.111:56524) 2021-04-03T03:15:59.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d3fff700 1 -- 172.21.2.111:0/783642706 learned_addr learned my addr 172.21.2.111:0/783642706 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:59.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d2ffd700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d414b700 0x7ff3d40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301678400 0 0) 0x7ff3d40607d0 con 0x7ff3d40b0c90 2021-04-03T03:15:59.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b4003240 con 0x7ff3d40b0c90 2021-04-03T03:15:59.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 691902800 0 0) 0x7ff3b4003240 con 0x7ff3d40b0c90 2021-04-03T03:15:59.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d414b700 msgr2=0x7ff3d40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d414b700 0x7ff3d40b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 >> v1:172.21.2.115:6789/0 conn(0x7ff3d40603f0 legacy=0x7ff3d40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40afb40 con 0x7ff3d40b0c90 2021-04-03T03:15:59.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ff3cc001a50 con 0x7ff3d40b0c90 2021-04-03T03:15:59.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 >> v1:172.21.2.111:6789/0 conn(0x7ff3d40b0c90 legacy=0x7ff3d4147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b4003760 0x7ff3b4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d414b700 0x7ff3b40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3b40049d0 0x7ff3b4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40af6a0 con 0x7ff3b4003760 2021-04-03T03:15:59.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40041b0 con 0x7ff3b40049d0 2021-04-03T03:15:59.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b400b730 con 0x7ff3d414b700 2021-04-03T03:15:59.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d37fe700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b4003760 0x7ff3b4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d37fe700 1 -- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d414b700 msgr2=0x7ff3b40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d37fe700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d414b700 0x7ff3b40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d37fe700 1 -- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3b40049d0 msgr2=0x7ff3b4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d37fe700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3b40049d0 0x7ff3b4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d37fe700 1 -- 172.21.2.111:0/783642706 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40afb40 con 0x7ff3b4003760 2021-04-03T03:15:59.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d37fe700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b4003760 0x7ff3b4003b80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7ff3b40054b0 tx=0x7ff3bc007b70).ready entity=mon.0 client_cookie=dc0cc13ca29584d7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc00c620 con 0x7ff3b4003760 2021-04-03T03:15:59.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc00c780 con 0x7ff3b4003760 2021-04-03T03:15:59.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d27fc700 1 -- 172.21.2.111:0/783642706 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc00cae0 con 0x7ff3b4003760 2021-04-03T03:15:59.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- 172.21.2.111:0/783642706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b4003760 msgr2=0x7ff3b4003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 --2- 172.21.2.111:0/783642706 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b4003760 0x7ff3b4003b80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7ff3b40054b0 tx=0x7ff3bc007b70).stop 2021-04-03T03:15:59.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d3fff700 1 -- 172.21.2.111:0/783642706 reap_dead start 2021-04-03T03:15:59.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- 172.21.2.111:0/783642706 shutdown_connections 2021-04-03T03:15:59.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- 172.21.2.111:0/783642706 >> 172.21.2.111:0/783642706 conn(0x7ff3d40b61c0 msgr2=0x7ff3d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- 172.21.2.111:0/783642706 shutdown_connections 2021-04-03T03:15:59.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- 172.21.2.111:0/783642706 wait complete. 2021-04-03T03:15:59.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 Processor -- start 2021-04-03T03:15:59.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.403+0000 7ff3d896f700 1 -- start start 2021-04-03T03:15:59.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3d4149c00 0x7ff3d4062910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d4062e60 0x7ff3d4064270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3d4064760 0x7ff3d40a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40abb30 con 0x7ff3d4149c00 2021-04-03T03:15:59.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4149fe0 con 0x7ff3d4064760 2021-04-03T03:15:59.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40a9df0 con 0x7ff3d4062e60 2021-04-03T03:15:59.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3d4149c00 0x7ff3d4062910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3d4149c00 0x7ff3d4062910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55740/0 (socket says 172.21.2.111:55740) 2021-04-03T03:15:59.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 -- 172.21.2.111:0/1711253438 learned_addr learned my addr 172.21.2.111:0/1711253438 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:59.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 -- 172.21.2.111:0/1711253438 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d4062e60 msgr2=0x7ff3d4064270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d4062e60 0x7ff3d4064270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 -- 172.21.2.111:0/1711253438 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3d4064760 msgr2=0x7ff3d40a7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3d4064760 0x7ff3d40a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 -- 172.21.2.111:0/1711253438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40aa090 con 0x7ff3d4149c00 2021-04-03T03:15:59.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d37fe700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3d4149c00 0x7ff3d4062910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7ff3bc00d000 tx=0x7ff3bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d0ff9700 1 -- 172.21.2.111:0/1711253438 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc007210 con 0x7ff3d4149c00 2021-04-03T03:15:59.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d0ff9700 1 -- 172.21.2.111:0/1711253438 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc007370 con 0x7ff3d4149c00 2021-04-03T03:15:59.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d40aa330 con 0x7ff3d4149c00 2021-04-03T03:15:59.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d0ff9700 1 -- 172.21.2.111:0/1711253438 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc013a80 con 0x7ff3d4149c00 2021-04-03T03:15:59.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d4064c10 con 0x7ff3d4149c00 2021-04-03T03:15:59.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d4057660 con 0x7ff3d4149c00 2021-04-03T03:15:59.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d0ff9700 1 -- 172.21.2.111:0/1711253438 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff3bc010030 con 0x7ff3d4149c00 2021-04-03T03:15:59.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d0ff9700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3b8056f70 0x7ff3b8059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d0ff9700 1 -- 172.21.2.111:0/1711253438 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5596+0+0 (secure 0 0 0) 0x7ff3bc0708a0 con 0x7ff3d4149c00 2021-04-03T03:15:59.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d2ffd700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3b8056f70 0x7ff3b8059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.407+0000 7ff3d2ffd700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3b8056f70 0x7ff3b8059430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7ff3c400c000 tx=0x7ff3c4006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.411+0000 7ff3d0ff9700 1 -- 172.21.2.111:0/1711253438 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff3bc045fc0 con 0x7ff3d4149c00 2021-04-03T03:15:59.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff3d40ae7c0 con 0x7ff3d4149c00 2021-04-03T03:15:59.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d0ff9700 1 -- 172.21.2.111:0/1711253438 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+4242 (secure 0 0 0) 0x7ff3bc0495e0 con 0x7ff3d4149c00 2021-04-03T03:15:59.542 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:59.542 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:15:59.543 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:15:59.543 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.543 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:15:59.543 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:15:59.544 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.544 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:59.544 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.544 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.545 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.545 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.545 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.545 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.546 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:15:59.546 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:15:59.546 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.546 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:59.546 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.547 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.547 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.547 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.547 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.547 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.548 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:15:59.548 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:15:59.548 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.548 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.549 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.549 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.549 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.549 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.549 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.550 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:15:59.550 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:15:59.550 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.550 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.550 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.551 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.551 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.551 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.551 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.552 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:15:59.552 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:15:59.552 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.552 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.552 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.553 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.553 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.553 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.553 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.553 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:15:59.554 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:15:59.554 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.554 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.554 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.555 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.555 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.555 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.555 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.556 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:15:59.556 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:15:59.556 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.556 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.556 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.557 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.557 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.557 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.557 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.558 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:15:59.558 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:15:59.558 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.558 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.558 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.559 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.559 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.559 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.559 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.560 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:15:59.560 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:15:59.560 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.560 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.561 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.561 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.561 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.561 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.561 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.562 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:15:59.562 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:15:59.562 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.562 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.563 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.563 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.563 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.563 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.563 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.564 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:15:59.564 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:15:59.564 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.564 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:59.565 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:59.565 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:59.565 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.565 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.565 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.566 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:15:59.566 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:15:59.566 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.566 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:59.566 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:59.567 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:59.567 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.567 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.567 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.568 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:15:59.568 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:15:59.568 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.568 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:15:59.569 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:15:59.569 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:15:59.569 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.569 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.570 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.570 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:15:59.570 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:15:59.570 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.570 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:59.571 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:15:59.571 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:15:59.571 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.571 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.571 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.572 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.572 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:15:59.572 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:15:59.572 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.573 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:59.573 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:15:59.573 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.573 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.573 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:15:59.574 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:15:59.574 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:15:59.574 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:15:59.574 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:15:59.575 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:15:59.575 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:15:59.575 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:15:59.575 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.575 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:15:59.576 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:15:59.576 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:15:59.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3b8056f70 msgr2=0x7ff3b8059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3b8056f70 0x7ff3b8059430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7ff3c400c000 tx=0x7ff3c4006dd0).stop 2021-04-03T03:15:59.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3d4149c00 msgr2=0x7ff3d4062910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3d4149c00 0x7ff3d4062910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7ff3bc00d000 tx=0x7ff3bc016000).stop 2021-04-03T03:15:59.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 shutdown_connections 2021-04-03T03:15:59.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff3b8056f70 0x7ff3b8059430 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3d4064760 0x7ff3d40a7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3d4149c00 0x7ff3d4062910 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 --2- 172.21.2.111:0/1711253438 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3d4062e60 0x7ff3d4064270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 >> 172.21.2.111:0/1711253438 conn(0x7ff3d40b61c0 msgr2=0x7ff3d40b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 shutdown_connections 2021-04-03T03:15:59.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.540+0000 7ff3d896f700 1 -- 172.21.2.111:0/1711253438 wait complete. 2021-04-03T03:15:59.581 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:15:59.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 Processor -- start 2021-04-03T03:15:59.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 -- start start 2021-04-03T03:15:59.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f96300b12b0 0x7f96301522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96300607d0 con 0x7f963014b700 2021-04-03T03:15:59.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96300b0eb0 con 0x7f96300603f0 2021-04-03T03:15:59.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96300af400 con 0x7f96300b12b0 2021-04-03T03:15:59.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962e7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f963014b700 0x7f96300b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56544/0 (socket says 172.21.2.111:56544) 2021-04-03T03:15:59.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962e7fc700 1 -- 172.21.2.111:0/1019519707 learned_addr learned my addr 172.21.2.111:0/1019519707 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:59.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510843608 0 0) 0x7f96300607d0 con 0x7f963014b700 2021-04-03T03:15:59.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9610003240 con 0x7f963014b700 2021-04-03T03:15:59.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962f7fe700 1 --2- 172.21.2.111:0/1019519707 >> v2:172.21.2.115:3300/0 conn(0x7f96300b12b0 0x7f96301522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 182453206 0 0) 0x7f9610003240 con 0x7f963014b700 2021-04-03T03:15:59.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 >> v2:172.21.2.115:3300/0 conn(0x7f96300b12b0 msgr2=0x7f96301522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 --2- 172.21.2.111:0/1019519707 >> v2:172.21.2.115:3300/0 conn(0x7f96300b12b0 0x7f96301522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 >> v1:172.21.2.115:6789/0 conn(0x7f96300603f0 legacy=0x7f96300b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300af6a0 con 0x7f963014b700 2021-04-03T03:15:59.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f96200016d0 con 0x7f963014b700 2021-04-03T03:15:59.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 >> v1:172.21.2.111:6789/0 conn(0x7f963014b700 legacy=0x7f96300b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9610003830 0x7f9610003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96100044e0 0x7f9610004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9610004e50 0x7f961000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300af400 con 0x7f9610003830 2021-04-03T03:15:59.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9610005330 con 0x7f96100044e0 2021-04-03T03:15:59.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961000bc20 con 0x7f9610004e50 2021-04-03T03:15:59.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962effd700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9610003830 0x7f9610003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962e7fc700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96100044e0 0x7f9610004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962effd700 1 -- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9610004e50 msgr2=0x7f961000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962effd700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9610004e50 0x7f961000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962effd700 1 -- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96100044e0 msgr2=0x7f9610004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962effd700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96100044e0 0x7f9610004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962effd700 1 -- 172.21.2.111:0/1019519707 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610003240 con 0x7f9610003830 2021-04-03T03:15:59.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962f7fe700 1 -- 172.21.2.111:0/1019519707 reap_dead start 2021-04-03T03:15:59.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962e7fc700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96100044e0 0x7f9610004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:59.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962effd700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9610003830 0x7f9610003c30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f9618001f60 tx=0x7f9618007b70).ready entity=mon.0 client_cookie=2cddb35b6ade5c45 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961800c870 con 0x7f9610003830 2021-04-03T03:15:59.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961800c9d0 con 0x7f9610003830 2021-04-03T03:15:59.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962dffb700 1 -- 172.21.2.111:0/1019519707 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961800cd30 con 0x7f9610003830 2021-04-03T03:15:59.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 -- 172.21.2.111:0/1019519707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9610003830 msgr2=0x7f9610003c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.636+0000 7f962ffff700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9610003830 0x7f9610003c30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f9618001f60 tx=0x7f9618007b70).stop 2021-04-03T03:15:59.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- 172.21.2.111:0/1019519707 shutdown_connections 2021-04-03T03:15:59.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 --2- 172.21.2.111:0/1019519707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9610003830 0x7f9610003c30 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- 172.21.2.111:0/1019519707 >> 172.21.2.111:0/1019519707 conn(0x7f96300b61c0 msgr2=0x7f963005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- 172.21.2.111:0/1019519707 shutdown_connections 2021-04-03T03:15:59.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- 172.21.2.111:0/1019519707 wait complete. 2021-04-03T03:15:59.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 Processor -- start 2021-04-03T03:15:59.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- start start 2021-04-03T03:15:59.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9610003830 0x7f96300ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9630065860 0x7f96300aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9630063230 0x7f9630066f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9630064250 con 0x7f9630065860 2021-04-03T03:15:59.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300636b0 con 0x7f9610003830 2021-04-03T03:15:59.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9630064e80 con 0x7f9630063230 2021-04-03T03:15:59.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9630065860 0x7f96300aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9630065860 0x7f96300aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55760/0 (socket says 172.21.2.111:55760) 2021-04-03T03:15:59.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 -- 172.21.2.111:0/388072270 learned_addr learned my addr 172.21.2.111:0/388072270 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:59.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 -- 172.21.2.111:0/388072270 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9630063230 msgr2=0x7f9630066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9630063230 0x7f9630066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 -- 172.21.2.111:0/388072270 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9610003830 msgr2=0x7f96300ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9610003830 0x7f96300ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 -- 172.21.2.111:0/388072270 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630065120 con 0x7f9630065860 2021-04-03T03:15:59.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962e7fc700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9630065860 0x7f96300aa9b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f9630065740 tx=0x7f9620009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f9627fff700 1 -- 172.21.2.111:0/388072270 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9620007170 con 0x7f9630065860 2021-04-03T03:15:59.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f9627fff700 1 -- 172.21.2.111:0/388072270 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96200072d0 con 0x7f9630065860 2021-04-03T03:15:59.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f9627fff700 1 -- 172.21.2.111:0/388072270 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f962000b990 con 0x7f9630065860 2021-04-03T03:15:59.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96300653c0 con 0x7f9630065860 2021-04-03T03:15:59.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9630067650 con 0x7f9630065860 2021-04-03T03:15:59.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9630065550 con 0x7f9630065860 2021-04-03T03:15:59.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f9627fff700 1 -- 172.21.2.111:0/388072270 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f962000baf0 con 0x7f9630065860 2021-04-03T03:15:59.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f9627fff700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9614056f70 0x7f9614059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f9627fff700 1 -- 172.21.2.111:0/388072270 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f962006ffd0 con 0x7f9630065860 2021-04-03T03:15:59.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962effd700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9614056f70 0x7f9614059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.640+0000 7f962effd700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9614056f70 0x7f9614059430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f9618001f60 tx=0x7f9618016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.644+0000 7f9627fff700 1 -- 172.21.2.111:0/388072270 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9620045690 con 0x7f9630065860 2021-04-03T03:15:59.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.736+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9630057660 con 0x7f9630065860 2021-04-03T03:15:59.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.736+0000 7f9627fff700 1 -- 172.21.2.111:0/388072270 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f9620048cb0 con 0x7f9630065860 2021-04-03T03:15:59.741 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:15:59.741 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":98,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:15:59.320551+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:15:59.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9614056f70 msgr2=0x7f9614059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9614056f70 0x7f9614059430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f9618001f60 tx=0x7f9618016000).stop 2021-04-03T03:15:59.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9630065860 msgr2=0x7f96300aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9630065860 0x7f96300aa9b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f9630065740 tx=0x7f9620009040).stop 2021-04-03T03:15:59.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 shutdown_connections 2021-04-03T03:15:59.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9630063230 0x7f9630066f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9614056f70 0x7f9614059430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9610003830 0x7f96300ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 --2- 172.21.2.111:0/388072270 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9630065860 0x7f96300aa9b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 >> 172.21.2.111:0/388072270 conn(0x7f96300b61c0 msgr2=0x7f963014dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 shutdown_connections 2021-04-03T03:15:59.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.740+0000 7f962ffff700 1 -- 172.21.2.111:0/388072270 wait complete. 2021-04-03T03:15:59.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:15:59.749 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:15:59.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6953700 1 Processor -- start 2021-04-03T03:15:59.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6953700 1 -- start start 2021-04-03T03:15:59.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6953700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff0e00603f0 0x7ff0e01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6953700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00add50 0x7ff0e0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6953700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e00607d0 con 0x7ff0e00b17b0 2021-04-03T03:15:59.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6953700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e01451c0 con 0x7ff0e00603f0 2021-04-03T03:15:59.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6953700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e014b090 con 0x7ff0e00add50 2021-04-03T03:15:59.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6152700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff0e00b17b0 0x7ff0e0147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56562/0 (socket says 172.21.2.111:56562) 2021-04-03T03:15:59.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e6152700 1 -- 172.21.2.111:0/2832579004 learned_addr learned my addr 172.21.2.111:0/2832579004 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:15:59.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414772966 0 0) 0x7ff0e00607d0 con 0x7ff0e00b17b0 2021-04-03T03:15:59.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0c8003240 con 0x7ff0e00b17b0 2021-04-03T03:15:59.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e5951700 1 --2- 172.21.2.111:0/2832579004 >> v2:172.21.2.115:3300/0 conn(0x7ff0e00603f0 0x7ff0e01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3757983488 0 0) 0x7ff0c8003240 con 0x7ff0e00b17b0 2021-04-03T03:15:59.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00add50 msgr2=0x7ff0e0144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e5150700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00add50 0x7ff0e0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00add50 0x7ff0e0144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 >> v2:172.21.2.115:3300/0 conn(0x7ff0e00603f0 msgr2=0x7ff0e01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 --2- 172.21.2.111:0/2832579004 >> v2:172.21.2.115:3300/0 conn(0x7ff0e00603f0 0x7ff0e01447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e014b330 con 0x7ff0e00b17b0 2021-04-03T03:15:59.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ff0d8001a50 con 0x7ff0e00b17b0 2021-04-03T03:15:59.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 >> v1:172.21.2.111:6789/0 conn(0x7ff0e00b17b0 legacy=0x7ff0e0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0c8003730 0x7ff0c8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0c80044e0 0x7ff0c8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0c8004e50 0x7ff0c800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e014b090 con 0x7ff0c8003730 2021-04-03T03:15:59.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c8004210 con 0x7ff0c80044e0 2021-04-03T03:15:59.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c800bc20 con 0x7ff0c8004e50 2021-04-03T03:15:59.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e5951700 1 --2- 172.21.2.111:0/2832579004 >> v2:172.21.2.115:3300/0 conn(0x7ff0e00603f0 0x7ff0e01447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:59.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e5150700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00add50 0x7ff0e0144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:15:59.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.804+0000 7ff0e5951700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0c8003730 0x7ff0c8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 -- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0c8004e50 msgr2=0x7ff0c800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0c8004e50 0x7ff0c800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 -- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0c80044e0 msgr2=0x7ff0c8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:15:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0c80044e0 0x7ff0c8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 -- 172.21.2.111:0/2832579004 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8003240 con 0x7ff0c8003730 2021-04-03T03:15:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0c8003730 0x7ff0c8003b50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7ff0dc007000 tx=0x7ff0dc008040).ready entity=mon.0 client_cookie=8ad18d46dbe08576 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0dc015070 con 0x7ff0c8003730 2021-04-03T03:15:59.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc00c8a0 con 0x7ff0c8003730 2021-04-03T03:15:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e494f700 1 -- 172.21.2.111:0/2832579004 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0dc00cc00 con 0x7ff0c8003730 2021-04-03T03:15:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- 172.21.2.111:0/2832579004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0c8003730 msgr2=0x7ff0c8003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 --2- 172.21.2.111:0/2832579004 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0c8003730 0x7ff0c8003b50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7ff0dc007000 tx=0x7ff0dc008040).stop 2021-04-03T03:15:59.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6152700 1 -- 172.21.2.111:0/2832579004 reap_dead start 2021-04-03T03:15:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- 172.21.2.111:0/2832579004 shutdown_connections 2021-04-03T03:15:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- 172.21.2.111:0/2832579004 >> 172.21.2.111:0/2832579004 conn(0x7ff0e00b61c0 msgr2=0x7ff0e005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- 172.21.2.111:0/2832579004 shutdown_connections 2021-04-03T03:15:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- 172.21.2.111:0/2832579004 wait complete. 2021-04-03T03:15:59.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 Processor -- start 2021-04-03T03:15:59.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- start start 2021-04-03T03:15:59.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0e0066020 0x7ff0e0157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0e00a8f30 0x7ff0e0066400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00675d0 0x7ff0e0067a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e0066cb0 con 0x7ff0e0066020 2021-04-03T03:15:59.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00668f0 con 0x7ff0e00a8f30 2021-04-03T03:15:59.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00aafd0 con 0x7ff0e00675d0 2021-04-03T03:15:59.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0e0066020 0x7ff0e0157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0e0066020 0x7ff0e0157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55780/0 (socket says 172.21.2.111:55780) 2021-04-03T03:15:59.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 -- 172.21.2.111:0/245869949 learned_addr learned my addr 172.21.2.111:0/245869949 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:59.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 -- 172.21.2.111:0/245869949 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00675d0 msgr2=0x7ff0e0067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00675d0 0x7ff0e0067a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 -- 172.21.2.111:0/245869949 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0e00a8f30 msgr2=0x7ff0e0066400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0e00a8f30 0x7ff0e0066400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 -- 172.21.2.111:0/245869949 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0067f40 con 0x7ff0e0066020 2021-04-03T03:15:59.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e5951700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0e0066020 0x7ff0e0157130 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7ff0dc007000 tx=0x7ff0dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0d2ffd700 1 -- 172.21.2.111:0/245869949 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0dc021070 con 0x7ff0e0066020 2021-04-03T03:15:59.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0d2ffd700 1 -- 172.21.2.111:0/245869949 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc01b420 con 0x7ff0e0066020 2021-04-03T03:15:59.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e0066e90 con 0x7ff0e0066020 2021-04-03T03:15:59.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0d2ffd700 1 -- 172.21.2.111:0/245869949 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0dc01b780 con 0x7ff0e0066020 2021-04-03T03:15:59.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.808+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e013c890 con 0x7ff0e0066020 2021-04-03T03:15:59.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.812+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e0066cb0 con 0x7ff0e0066020 2021-04-03T03:15:59.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.812+0000 7ff0d2ffd700 1 -- 172.21.2.111:0/245869949 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff0dc01b8e0 con 0x7ff0e0066020 2021-04-03T03:15:59.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.812+0000 7ff0d2ffd700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff0c0056fb0 0x7ff0c0059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.812+0000 7ff0d2ffd700 1 -- 172.21.2.111:0/245869949 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5596+0+0 (secure 0 0 0) 0x7ff0dc0716d0 con 0x7ff0e0066020 2021-04-03T03:15:59.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.812+0000 7ff0e5150700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff0c0056fb0 0x7ff0c0059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.812+0000 7ff0e5150700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff0c0056fb0 0x7ff0c0059470 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7ff0d4007000 tx=0x7ff0d4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.812+0000 7ff0d2ffd700 1 -- 172.21.2.111:0/245869949 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff0dc046df0 con 0x7ff0e0066020 2021-04-03T03:15:59.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff0e00672c0 con 0x7ff0e0066020 2021-04-03T03:15:59.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0d2ffd700 1 -- 172.21.2.111:0/245869949 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v98) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff0dc04a410 con 0x7ff0e0066020 2021-04-03T03:15:59.911 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:15:59.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff0c0056fb0 msgr2=0x7ff0c0059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff0c0056fb0 0x7ff0c0059470 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7ff0d4007000 tx=0x7ff0d4008040).stop 2021-04-03T03:15:59.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0e0066020 msgr2=0x7ff0e0157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0e0066020 0x7ff0e0157130 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7ff0dc007000 tx=0x7ff0dc008040).stop 2021-04-03T03:15:59.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 shutdown_connections 2021-04-03T03:15:59.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff0e00675d0 0x7ff0e0067a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff0e0066020 0x7ff0e0157130 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff0c0056fb0 0x7ff0c0059470 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 --2- 172.21.2.111:0/245869949 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff0e00a8f30 0x7ff0e0066400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 >> 172.21.2.111:0/245869949 conn(0x7ff0e00b61c0 msgr2=0x7ff0e0146550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 shutdown_connections 2021-04-03T03:15:59.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.908+0000 7ff0e6953700 1 -- 172.21.2.111:0/245869949 wait complete. 2021-04-03T03:15:59.919 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:15:59.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 Processor -- start 2021-04-03T03:15:59.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- start start 2021-04-03T03:15:59.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c0dbba0 0x7fc21c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc21c03d560 0x7fc21c041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c0dc560 con 0x7fc21c041060 2021-04-03T03:15:59.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c03de60 con 0x7fc21c03d560 2021-04-03T03:15:59.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0dc800 con 0x7fc21c0dbba0 2021-04-03T03:15:59.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc21c03d560 0x7fc21c041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc21c03d560 0x7fc21c041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55792/0 (socket says 172.21.2.111:55792) 2021-04-03T03:15:59.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 -- 172.21.2.111:0/3896436951 learned_addr learned my addr 172.21.2.111:0/3896436951 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:59.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 -- 172.21.2.111:0/3896436951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c0dbba0 msgr2=0x7fc21c0dbf80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:59.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 --2- 172.21.2.111:0/3896436951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c0dbba0 0x7fc21c0dbf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 -- 172.21.2.111:0/3896436951 >> v1:172.21.2.115:6789/0 conn(0x7fc21c041060 legacy=0x7fc21c03fa80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:15:59.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 -- 172.21.2.111:0/3896436951 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c03da30 con 0x7fc21c03d560 2021-04-03T03:15:59.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229068700 1 --2- 172.21.2.111:0/3896436951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c03d560 0x7fc21c041650 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fc21c03f960 tx=0x7fc22405ad70).ready entity=mon.0 client_cookie=5dd1d5e17f1b9e84 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc2237fe700 1 -- 172.21.2.111:0/3896436951 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22405bda0 con 0x7fc21c03d560 2021-04-03T03:15:59.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc2237fe700 1 -- 172.21.2.111:0/3896436951 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22405df60 con 0x7fc21c03d560 2021-04-03T03:15:59.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc2237fe700 1 -- 172.21.2.111:0/3896436951 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2240699f0 con 0x7fc21c03d560 2021-04-03T03:15:59.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- 172.21.2.111:0/3896436951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c03d560 msgr2=0x7fc21c041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- 172.21.2.111:0/3896436951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c03d560 0x7fc21c041650 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fc21c03f960 tx=0x7fc22405ad70).stop 2021-04-03T03:15:59.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- 172.21.2.111:0/3896436951 shutdown_connections 2021-04-03T03:15:59.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- 172.21.2.111:0/3896436951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c03d560 0x7fc21c041650 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:15:59.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- 172.21.2.111:0/3896436951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c0dbba0 0x7fc21c0dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- 172.21.2.111:0/3896436951 >> 172.21.2.111:0/3896436951 conn(0x7fc21c0462b0 msgr2=0x7fc21c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:15:59.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- 172.21.2.111:0/3896436951 shutdown_connections 2021-04-03T03:15:59.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- 172.21.2.111:0/3896436951 wait complete. 2021-04-03T03:15:59.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 Processor -- start 2021-04-03T03:15:59.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- start start 2021-04-03T03:15:59.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c03d560 0x7fc21c0a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c041060 0x7fc21c0eb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc21c0dbba0 0x7fc21c0d76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0a0ba0 con 0x7fc21c041060 2021-04-03T03:15:59.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0ea8d0 con 0x7fc21c0dbba0 2021-04-03T03:15:59.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc229869700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0eab70 con 0x7fc21c03d560 2021-04-03T03:15:59.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c041060 0x7fc21c0eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c041060 0x7fc21c0eb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55794/0 (socket says 172.21.2.111:55794) 2021-04-03T03:15:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 -- 172.21.2.111:0/1611829514 learned_addr learned my addr 172.21.2.111:0/1611829514 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:15:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 -- 172.21.2.111:0/1611829514 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c03d560 msgr2=0x7fc21c0a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c03d560 0x7fc21c0a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 -- 172.21.2.111:0/1611829514 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc21c0dbba0 msgr2=0x7fc21c0d76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:15:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc21c0dbba0 0x7fc21c0d76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:15:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 -- 172.21.2.111:0/1611829514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c039e90 con 0x7fc21c041060 2021-04-03T03:15:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc223fff700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c041060 0x7fc21c0eb010 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fc218002f70 tx=0x7fc218006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.976+0000 7fc221ffb700 1 -- 172.21.2.111:0/1611829514 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc21800d570 con 0x7fc21c041060 2021-04-03T03:15:59.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc221ffb700 1 -- 172.21.2.111:0/1611829514 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21800d6d0 con 0x7fc21c041060 2021-04-03T03:15:59.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc221ffb700 1 -- 172.21.2.111:0/1611829514 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc21800da30 con 0x7fc21c041060 2021-04-03T03:15:59.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c0eb500 con 0x7fc21c041060 2021-04-03T03:15:59.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c0d7dd0 con 0x7fc21c041060 2021-04-03T03:15:59.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c0398a0 con 0x7fc21c041060 2021-04-03T03:15:59.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc221ffb700 1 -- 172.21.2.111:0/1611829514 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc21800f070 con 0x7fc21c041060 2021-04-03T03:15:59.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc221ffb700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc204056f20 0x7fc2040593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:15:59.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc221ffb700 1 -- 172.21.2.111:0/1611829514 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fc21806f570 con 0x7fc21c041060 2021-04-03T03:15:59.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc228867700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc204056f20 0x7fc2040593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:15:59.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc228867700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc204056f20 0x7fc2040593e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc214000e90 tx=0x7fc214006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:15:59.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:15:59.980+0000 7fc221ffb700 1 -- 172.21.2.111:0/1611829514 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc218044c90 con 0x7fc21c041060 2021-04-03T03:16:00.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.076+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc21c004db0 con 0x7fc21c041060 2021-04-03T03:16:00.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.076+0000 7fc221ffb700 1 -- 172.21.2.111:0/1611829514 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc2180482b0 con 0x7fc21c041060 2021-04-03T03:16:00.080 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:00.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.076+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc204056f20 msgr2=0x7fc2040593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.076+0000 7fc229869700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc204056f20 0x7fc2040593e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc214000e90 tx=0x7fc214006910).stop 2021-04-03T03:16:00.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.076+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c041060 msgr2=0x7fc21c0eb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.076+0000 7fc229869700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c041060 0x7fc21c0eb010 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fc218002f70 tx=0x7fc218006de0).stop 2021-04-03T03:16:00.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 shutdown_connections 2021-04-03T03:16:00.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc204056f20 0x7fc2040593e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc21c0dbba0 0x7fc21c0d76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc21c03d560 0x7fc21c0a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 --2- 172.21.2.111:0/1611829514 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc21c041060 0x7fc21c0eb010 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 >> 172.21.2.111:0/1611829514 conn(0x7fc21c0462b0 msgr2=0x7fc21c041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:00.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 shutdown_connections 2021-04-03T03:16:00.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.080+0000 7fc229869700 1 -- 172.21.2.111:0/1611829514 wait complete. 2021-04-03T03:16:00.088 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:16:00.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f90fa700 1 Processor -- start 2021-04-03T03:16:00.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f90fa700 1 -- start start 2021-04-03T03:16:00.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f90fa700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f77f414b700 0x7f77f40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f90fa700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f40af400 con 0x7f77f40603f0 2021-04-03T03:16:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f90fa700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f40af6a0 con 0x7f77f40b0c90 2021-04-03T03:16:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f90fa700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f4060860 con 0x7f77f414b700 2021-04-03T03:16:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f3fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f77f40603f0 0x7f77f40b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56594/0 (socket says 172.21.2.111:56594) 2021-04-03T03:16:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.140+0000 7f77f3fff700 1 -- 172.21.2.111:0/259966740 learned_addr learned my addr 172.21.2.111:0/259966740 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:00.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290034195 0 0) 0x7f77f40af400 con 0x7f77f40603f0 2021-04-03T03:16:00.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e0003240 con 0x7f77f40603f0 2021-04-03T03:16:00.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2669456134 0 0) 0x7f77e0003240 con 0x7f77f40603f0 2021-04-03T03:16:00.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 >> v2:172.21.2.115:3300/0 conn(0x7f77f414b700 msgr2=0x7f77f40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:00.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 --2- 172.21.2.111:0/259966740 >> v2:172.21.2.115:3300/0 conn(0x7f77f414b700 0x7f77f40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 --2- 172.21.2.111:0/259966740 >> v2:172.21.2.115:3300/0 conn(0x7f77f414b700 0x7f77f40b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 >> v1:172.21.2.115:6789/0 conn(0x7f77f40b0c90 legacy=0x7f77f4147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:00.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40afb40 con 0x7f77f40603f0 2021-04-03T03:16:00.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f77ec001a50 con 0x7f77f40603f0 2021-04-03T03:16:00.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 >> v1:172.21.2.111:6789/0 conn(0x7f77f40603f0 legacy=0x7f77f40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77e0003800 0x7f77e0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e00044e0 0x7f77e0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77e0004e50 0x7f77e000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4060860 con 0x7f77e0003800 2021-04-03T03:16:00.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e0005300 con 0x7f77e00044e0 2021-04-03T03:16:00.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000bc20 con 0x7f77e0004e50 2021-04-03T03:16:00.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 --2- 172.21.2.111:0/259966740 >> v2:172.21.2.115:3300/0 conn(0x7f77f414b700 0x7f77f40b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:00.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77e0003800 0x7f77e0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 -- 172.21.2.111:0/259966740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77e0004e50 msgr2=0x7f77e000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:00.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77e0004e50 0x7f77e000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 -- 172.21.2.111:0/259966740 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e00044e0 msgr2=0x7f77e0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:00.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e00044e0 0x7f77e0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 -- 172.21.2.111:0/259966740 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0003240 con 0x7f77e0003800 2021-04-03T03:16:00.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f88f9700 1 -- 172.21.2.111:0/259966740 reap_dead start 2021-04-03T03:16:00.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77e0003800 0x7f77e0003c00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f77f40620f0 tx=0x7f77f40af830).ready entity=mon.0 client_cookie=407af83f90958998 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:00.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77ec005b80 con 0x7f77e0003800 2021-04-03T03:16:00.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec005ce0 con 0x7f77e0003800 2021-04-03T03:16:00.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f2ffd700 1 -- 172.21.2.111:0/259966740 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77ec0072c0 con 0x7f77e0003800 2021-04-03T03:16:00.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- 172.21.2.111:0/259966740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77e0003800 msgr2=0x7f77e0003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77e0003800 0x7f77e0003c00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f77f40620f0 tx=0x7f77f40af830).stop 2021-04-03T03:16:00.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- 172.21.2.111:0/259966740 shutdown_connections 2021-04-03T03:16:00.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 --2- 172.21.2.111:0/259966740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77e0003800 0x7f77e0003c00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- 172.21.2.111:0/259966740 >> 172.21.2.111:0/259966740 conn(0x7f77f40b61c0 msgr2=0x7f77f405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:00.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- 172.21.2.111:0/259966740 shutdown_connections 2021-04-03T03:16:00.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- 172.21.2.111:0/259966740 wait complete. 2021-04-03T03:16:00.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 Processor -- start 2021-04-03T03:16:00.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- start start 2021-04-03T03:16:00.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e0003800 0x7f77f4156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77f4066560 0x7f77f4066dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77f40672c0 0x7f77f40628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40655e0 con 0x7f77f4066560 2021-04-03T03:16:00.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4065880 con 0x7f77e0003800 2021-04-03T03:16:00.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f90fa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4065b20 con 0x7f77f40672c0 2021-04-03T03:16:00.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77f4066560 0x7f77f4066dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77f4066560 0x7f77f4066dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55814/0 (socket says 172.21.2.111:55814) 2021-04-03T03:16:00.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 -- 172.21.2.111:0/3922339783 learned_addr learned my addr 172.21.2.111:0/3922339783 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:00.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f3fff700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e0003800 0x7f77f4156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 -- 172.21.2.111:0/3922339783 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77f40672c0 msgr2=0x7f77f40628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77f40672c0 0x7f77f40628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 -- 172.21.2.111:0/3922339783 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e0003800 msgr2=0x7f77f4156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e0003800 0x7f77f4156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 -- 172.21.2.111:0/3922339783 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4157250 con 0x7f77f4066560 2021-04-03T03:16:00.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.144+0000 7f77f37fe700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77f4066560 0x7f77f4066dd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f77e8007000 tx=0x7f77e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:00.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f17fa700 1 -- 172.21.2.111:0/3922339783 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e800e5e0 con 0x7f77f4066560 2021-04-03T03:16:00.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f17fa700 1 -- 172.21.2.111:0/3922339783 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e800c070 con 0x7f77f4066560 2021-04-03T03:16:00.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f17fa700 1 -- 172.21.2.111:0/3922339783 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e800e930 con 0x7f77f4066560 2021-04-03T03:16:00.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f4065330 con 0x7f77f4066560 2021-04-03T03:16:00.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f4062f50 con 0x7f77f4066560 2021-04-03T03:16:00.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f4067710 con 0x7f77f4066560 2021-04-03T03:16:00.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f17fa700 1 -- 172.21.2.111:0/3922339783 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f77e800ea90 con 0x7f77f4066560 2021-04-03T03:16:00.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f17fa700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77d4056f70 0x7f77d4059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f17fa700 1 -- 172.21.2.111:0/3922339783 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f77e806f980 con 0x7f77f4066560 2021-04-03T03:16:00.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f3fff700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77d4056f70 0x7f77d4059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f3fff700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77d4056f70 0x7f77d4059430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f77ec014000 tx=0x7f77ec015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:00.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.148+0000 7f77f17fa700 1 -- 172.21.2.111:0/3922339783 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f77f4067710 con 0x7f77f4066560 2021-04-03T03:16:00.249 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f77f4067710 con 0x7f77f4066560 2021-04-03T03:16:00.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f17fa700 1 -- 172.21.2.111:0/3922339783 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+143 (secure 0 0 0) 0x7f77f4067710 con 0x7f77f4066560 2021-04-03T03:16:00.250 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:00.250 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:16:00.251 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:00.251 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:16:00.251 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:16:00.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:00.252 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:00.252 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:16:00.252 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:16:00.252 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:00.252 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:16:00.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77d4056f70 msgr2=0x7f77d4059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77d4056f70 0x7f77d4059430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f77ec014000 tx=0x7f77ec015000).stop 2021-04-03T03:16:00.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77f4066560 msgr2=0x7f77f4066dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77f4066560 0x7f77f4066dd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f77e8007000 tx=0x7f77e8008040).stop 2021-04-03T03:16:00.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 shutdown_connections 2021-04-03T03:16:00.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f77f40672c0 0x7f77f40628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f77d4056f70 0x7f77d4059430 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f77e0003800 0x7f77f4156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 --2- 172.21.2.111:0/3922339783 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f77f4066560 0x7f77f4066dd0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 >> 172.21.2.111:0/3922339783 conn(0x7f77f40b61c0 msgr2=0x7f77f4067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:00.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 shutdown_connections 2021-04-03T03:16:00.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.248+0000 7f77f90fa700 1 -- 172.21.2.111:0/3922339783 wait complete. 2021-04-03T03:16:00.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:16:00.258 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:16:00.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 Processor -- start 2021-04-03T03:16:00.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 -- start start 2021-04-03T03:16:00.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440603f0 0x7f4e441447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4e440add50 0x7f4e44144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4e440b17b0 0x7f4e44147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e441451c0 con 0x7f4e440add50 2021-04-03T03:16:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e440607d0 con 0x7f4e440b17b0 2021-04-03T03:16:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4414b090 con 0x7f4e440603f0 2021-04-03T03:16:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4e440add50 0x7f4e44144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4e440add50 0x7f4e44144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55824/0 (socket says 172.21.2.111:55824) 2021-04-03T03:16:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 -- 172.21.2.111:0/69162887 learned_addr learned my addr 172.21.2.111:0/69162887 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 -- 172.21.2.111:0/69162887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440603f0 msgr2=0x7f4e441447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 --2- 172.21.2.111:0/69162887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440603f0 0x7f4e441447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 -- 172.21.2.111:0/69162887 >> v2:172.21.2.115:3300/0 conn(0x7f4e440b17b0 msgr2=0x7f4e44147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 --2- 172.21.2.111:0/69162887 >> v2:172.21.2.115:3300/0 conn(0x7f4e440b17b0 0x7f4e44147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 -- 172.21.2.111:0/69162887 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4414b330 con 0x7f4e440add50 2021-04-03T03:16:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e48c89700 1 --2- 172.21.2.111:0/69162887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440add50 0x7f4e44144cd0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f4e40002f30 tx=0x7f4e40006e90).ready entity=mon.0 client_cookie=a98cf232a2b09534 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e37fff700 1 -- 172.21.2.111:0/69162887 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e40009920 con 0x7f4e440add50 2021-04-03T03:16:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e37fff700 1 -- 172.21.2.111:0/69162887 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40009a80 con 0x7f4e440add50 2021-04-03T03:16:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e37fff700 1 -- 172.21.2.111:0/69162887 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e40009de0 con 0x7f4e440add50 2021-04-03T03:16:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 -- 172.21.2.111:0/69162887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440add50 msgr2=0x7f4e44144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.312+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/69162887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440add50 0x7f4e44144cd0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f4e40002f30 tx=0x7f4e40006e90).stop 2021-04-03T03:16:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- 172.21.2.111:0/69162887 shutdown_connections 2021-04-03T03:16:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/69162887 >> v2:172.21.2.115:3300/0 conn(0x7f4e440b17b0 0x7f4e44147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/69162887 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440603f0 0x7f4e441447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/69162887 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440add50 0x7f4e44144cd0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- 172.21.2.111:0/69162887 >> 172.21.2.111:0/69162887 conn(0x7f4e440b61c0 msgr2=0x7f4e4405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:00.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- 172.21.2.111:0/69162887 shutdown_connections 2021-04-03T03:16:00.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- 172.21.2.111:0/69162887 wait complete. 2021-04-03T03:16:00.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 Processor -- start 2021-04-03T03:16:00.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- start start 2021-04-03T03:16:00.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440603f0 0x7f4e44156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e440add50 0x7f4e44157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440b17b0 0x7f4e44064fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440aa9e0 con 0x7f4e440603f0 2021-04-03T03:16:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e44064b00 con 0x7f4e440add50 2021-04-03T03:16:00.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e44156fc0 con 0x7f4e440b17b0 2021-04-03T03:16:00.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440603f0 0x7f4e44156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440603f0 0x7f4e44156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55830/0 (socket says 172.21.2.111:55830) 2021-04-03T03:16:00.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 -- 172.21.2.111:0/9369547 learned_addr learned my addr 172.21.2.111:0/9369547 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:00.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 -- 172.21.2.111:0/9369547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440b17b0 msgr2=0x7f4e44064fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440b17b0 0x7f4e44064fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 -- 172.21.2.111:0/9369547 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e440add50 msgr2=0x7f4e44157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:00.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e440add50 0x7f4e44157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:00.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 -- 172.21.2.111:0/9369547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440aa460 con 0x7f4e440603f0 2021-04-03T03:16:00.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4948a700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440603f0 0x7f4e44156ad0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f4e38002f30 tx=0x7f4e38006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:00.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e367fc700 1 -- 172.21.2.111:0/9369547 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e3800d770 con 0x7f4e440603f0 2021-04-03T03:16:00.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e367fc700 1 -- 172.21.2.111:0/9369547 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e38000e90 con 0x7f4e440603f0 2021-04-03T03:16:00.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e440abb50 con 0x7f4e440603f0 2021-04-03T03:16:00.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e367fc700 1 -- 172.21.2.111:0/9369547 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e3800dac0 con 0x7f4e440603f0 2021-04-03T03:16:00.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e44157a10 con 0x7f4e440603f0 2021-04-03T03:16:00.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e44064950 con 0x7f4e440603f0 2021-04-03T03:16:00.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e367fc700 1 -- 172.21.2.111:0/9369547 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4e38017030 con 0x7f4e440603f0 2021-04-03T03:16:00.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e367fc700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e24056ef0 0x7f4e240593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:00.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e367fc700 1 -- 172.21.2.111:0/9369547 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f4e3806f2f0 con 0x7f4e440603f0 2021-04-03T03:16:00.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.316+0000 7f4e48c89700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e24056ef0 0x7f4e240593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:00.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.320+0000 7f4e48c89700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e24056ef0 0x7f4e240593b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f4e40002f30 tx=0x7f4e40015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:00.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.320+0000 7f4e367fc700 1 -- 172.21.2.111:0/9369547 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4e38044950 con 0x7f4e440603f0 2021-04-03T03:16:00.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:00.412+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4e44057660 con 0x7f4e440603f0 2021-04-03T03:16:01.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e367fc700 1 -- 172.21.2.111:0/9369547 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v99) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4e38047f70 con 0x7f4e440603f0 2021-04-03T03:16:01.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e24056ef0 msgr2=0x7f4e240593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:01.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e24056ef0 0x7f4e240593b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f4e40002f30 tx=0x7f4e40015000).stop 2021-04-03T03:16:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440603f0 msgr2=0x7f4e44156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440603f0 0x7f4e44156ad0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f4e38002f30 tx=0x7f4e38006ea0).stop 2021-04-03T03:16:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 shutdown_connections 2021-04-03T03:16:01.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4e440b17b0 0x7f4e44064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4e440603f0 0x7f4e44156ad0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4e24056ef0 0x7f4e240593b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 --2- 172.21.2.111:0/9369547 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4e440add50 0x7f4e44157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 >> 172.21.2.111:0/9369547 conn(0x7f4e440b61c0 msgr2=0x7f4e44144960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:01.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 shutdown_connections 2021-04-03T03:16:01.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.336+0000 7f4e4a48c700 1 -- 172.21.2.111:0/9369547 wait complete. 2021-04-03T03:16:01.342 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:16:01.346 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:16:01.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.400+0000 7f109c9d9700 1 Processor -- start 2021-04-03T03:16:01.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.400+0000 7f109c9d9700 1 -- start start 2021-04-03T03:16:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.400+0000 7f109c9d9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 0x7f109003ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.400+0000 7f109c9d9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1090042850 0x7f109003d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.400+0000 7f109c9d9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1090039580 con 0x7f10900cdcf0 2021-04-03T03:16:01.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.400+0000 7f109c9d9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1090042460 con 0x7f1090042850 2021-04-03T03:16:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10900d19c0 con 0x7f1090042080 2021-04-03T03:16:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1097fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f10900cdcf0 0x7f10900d4190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56630/0 (socket says 172.21.2.111:56630) 2021-04-03T03:16:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1097fff700 1 -- 172.21.2.111:0/2518990959 learned_addr learned my addr 172.21.2.111:0/2518990959 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:01.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- 172.21.2.111:0/2518990959 >> v2:172.21.2.111:3300/0 conn(0x7f1090042850 0x7f109003d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f10977fe700 1 --2- 172.21.2.111:0/2518990959 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 0x7f109003ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 -- 172.21.2.111:0/2518990959 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 msgr2=0x7f109003ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- 172.21.2.111:0/2518990959 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 0x7f109003ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 -- 172.21.2.111:0/2518990959 >> v1:172.21.2.111:6789/0 conn(0x7f10900cdcf0 legacy=0x7f10900d4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:01.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 -- 172.21.2.111:0/2518990959 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900397b0 con 0x7f1090042850 2021-04-03T03:16:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- 172.21.2.111:0/2518990959 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f109003d350 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f108c002f30 tx=0x7f108c006dd0).ready entity=mon.0 client_cookie=8dcbc8acc49aa45a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f10967fc700 1 -- 172.21.2.111:0/2518990959 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c0097f0 con 0x7f1090042850 2021-04-03T03:16:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f10967fc700 1 -- 172.21.2.111:0/2518990959 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c009950 con 0x7f1090042850 2021-04-03T03:16:01.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f10967fc700 1 -- 172.21.2.111:0/2518990959 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c009cb0 con 0x7f1090042850 2021-04-03T03:16:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/2518990959 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 msgr2=0x7f109003d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 --2- 172.21.2.111:0/2518990959 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f109003d350 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f108c002f30 tx=0x7f108c006dd0).stop 2021-04-03T03:16:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/2518990959 shutdown_connections 2021-04-03T03:16:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 --2- 172.21.2.111:0/2518990959 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 0x7f109003ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:01.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 --2- 172.21.2.111:0/2518990959 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f109003d350 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/2518990959 >> 172.21.2.111:0/2518990959 conn(0x7f10900462b0 msgr2=0x7f10900466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/2518990959 shutdown_connections 2021-04-03T03:16:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/2518990959 wait complete. 2021-04-03T03:16:01.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 Processor -- start 2021-04-03T03:16:01.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- start start 2021-04-03T03:16:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 0x7f10900b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:01.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f10900de9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f10900cdcf0 0x7f10900b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109003b5f0 con 0x7f1090042850 2021-04-03T03:16:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10900e22b0 con 0x7f10900cdcf0 2021-04-03T03:16:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10900e2010 con 0x7f1090042080 2021-04-03T03:16:01.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f10900de9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f10900de9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55846/0 (socket says 172.21.2.111:55846) 2021-04-03T03:16:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 -- 172.21.2.111:0/3587070992 learned_addr learned my addr 172.21.2.111:0/3587070992 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1097fff700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f10900cdcf0 0x7f10900b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:01.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 -- 172.21.2.111:0/3587070992 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 msgr2=0x7f10900b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 0x7f10900b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 -- 172.21.2.111:0/3587070992 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f10900cdcf0 msgr2=0x7f10900b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f10900cdcf0 0x7f10900b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 -- 172.21.2.111:0/3587070992 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090039fb0 con 0x7f1090042850 2021-04-03T03:16:01.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1096ffd700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f10900de9d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f108c002f30 tx=0x7f108c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1094ff9700 1 -- 172.21.2.111:0/3587070992 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c011d90 con 0x7f1090042850 2021-04-03T03:16:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1094ff9700 1 -- 172.21.2.111:0/3587070992 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c01e460 con 0x7f1090042850 2021-04-03T03:16:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109003aff0 con 0x7f1090042850 2021-04-03T03:16:01.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109003a290 con 0x7f1090042850 2021-04-03T03:16:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f1094ff9700 1 -- 172.21.2.111:0/3587070992 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108c01e7c0 con 0x7f1090042850 2021-04-03T03:16:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.404+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1090004db0 con 0x7f1090042850 2021-04-03T03:16:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.408+0000 7f1094ff9700 1 -- 172.21.2.111:0/3587070992 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f108c025030 con 0x7f1090042850 2021-04-03T03:16:01.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.408+0000 7f1094ff9700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1078057000 0x7f10780594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.408+0000 7f1094ff9700 1 -- 172.21.2.111:0/3587070992 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5950+0+0 (secure 0 0 0) 0x7f108c0716e0 con 0x7f1090042850 2021-04-03T03:16:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.408+0000 7f10977fe700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1078057000 0x7f10780594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.408+0000 7f10977fe700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1078057000 0x7f10780594c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f1088007000 tx=0x7f1088008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:01.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.408+0000 7f1094ff9700 1 -- 172.21.2.111:0/3587070992 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f108c046b90 con 0x7f1090042850 2021-04-03T03:16:01.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:01.504+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f109003b730 con 0x7f1090042850 2021-04-03T03:16:02.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.340+0000 7f1094ff9700 1 -- 172.21.2.111:0/3587070992 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v100) v1 ==== 125+0+0 (secure 0 0 0) 0x7f108c04a1b0 con 0x7f1090042850 2021-04-03T03:16:02.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1078057000 msgr2=0x7f10780594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1078057000 0x7f10780594c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f1088007000 tx=0x7f1088008040).stop 2021-04-03T03:16:02.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 msgr2=0x7f10900de9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f10900de9d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f108c002f30 tx=0x7f108c015000).stop 2021-04-03T03:16:02.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 shutdown_connections 2021-04-03T03:16:02.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1078057000 0x7f10780594c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f10900cdcf0 0x7f10900b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1090042080 0x7f10900b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 --2- 172.21.2.111:0/3587070992 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1090042850 0x7f10900de9d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 >> 172.21.2.111:0/3587070992 conn(0x7f10900462b0 msgr2=0x7f1090005920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 shutdown_connections 2021-04-03T03:16:02.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.344+0000 7f109c9d9700 1 -- 172.21.2.111:0/3587070992 wait complete. 2021-04-03T03:16:02.349 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:16:02.354 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:16:02.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11fffff700 1 Processor -- start 2021-04-03T03:16:02.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11fffff700 1 -- start start 2021-04-03T03:16:02.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11fffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12000603f0 0x7f12001447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11fffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f12000add50 0x7f1200144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11fffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12000607d0 con 0x7f12000b17b0 2021-04-03T03:16:02.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11fffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12001451c0 con 0x7f12000603f0 2021-04-03T03:16:02.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11fffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120014b090 con 0x7f12000add50 2021-04-03T03:16:02.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.408+0000 7f11feffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12000603f0 0x7f12001447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12000603f0 0x7f12001447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55852/0 (socket says 172.21.2.111:55852) 2021-04-03T03:16:02.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 -- 172.21.2.111:0/3434768930 learned_addr learned my addr 172.21.2.111:0/3434768930 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:02.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 --1- 172.21.2.111:0/3434768930 >> v1:172.21.2.111:6789/0 conn(0x7f12000b17b0 0x7f1200147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56642/0 (socket says 172.21.2.111:56642) 2021-04-03T03:16:02.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fe7fc700 1 --2- 172.21.2.111:0/3434768930 >> v2:172.21.2.115:3300/0 conn(0x7f12000add50 0x7f1200144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 -- 172.21.2.111:0/3434768930 >> v2:172.21.2.115:3300/0 conn(0x7f12000add50 msgr2=0x7f1200144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:02.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 --2- 172.21.2.111:0/3434768930 >> v2:172.21.2.115:3300/0 conn(0x7f12000add50 0x7f1200144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 -- 172.21.2.111:0/3434768930 >> v1:172.21.2.111:6789/0 conn(0x7f12000b17b0 legacy=0x7f1200147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 -- 172.21.2.111:0/3434768930 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120014b330 con 0x7f12000603f0 2021-04-03T03:16:02.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 --2- 172.21.2.111:0/3434768930 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000603f0 0x7f12001447e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f11e8002f30 tx=0x7f11e8006e90).ready entity=mon.0 client_cookie=9ee82c50bdccb0f5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fdffb700 1 -- 172.21.2.111:0/3434768930 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e80099c0 con 0x7f12000603f0 2021-04-03T03:16:02.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fdffb700 1 -- 172.21.2.111:0/3434768930 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8009b20 con 0x7f12000603f0 2021-04-03T03:16:02.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fdffb700 1 -- 172.21.2.111:0/3434768930 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e8009e80 con 0x7f12000603f0 2021-04-03T03:16:02.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- 172.21.2.111:0/3434768930 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000603f0 msgr2=0x7f12001447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 --2- 172.21.2.111:0/3434768930 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000603f0 0x7f12001447e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f11e8002f30 tx=0x7f11e8006e90).stop 2021-04-03T03:16:02.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- 172.21.2.111:0/3434768930 shutdown_connections 2021-04-03T03:16:02.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 --2- 172.21.2.111:0/3434768930 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000603f0 0x7f12001447e0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 --2- 172.21.2.111:0/3434768930 >> v2:172.21.2.115:3300/0 conn(0x7f12000add50 0x7f1200144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- 172.21.2.111:0/3434768930 >> 172.21.2.111:0/3434768930 conn(0x7f12000b61c0 msgr2=0x7f120005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- 172.21.2.111:0/3434768930 shutdown_connections 2021-04-03T03:16:02.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- 172.21.2.111:0/3434768930 wait complete. 2021-04-03T03:16:02.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 Processor -- start 2021-04-03T03:16:02.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- start start 2021-04-03T03:16:02.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12000603f0 0x7f12000aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12000add50 0x7f1200149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000b17b0 0x7f120015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120015a220 con 0x7f12000b17b0 2021-04-03T03:16:02.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000aabe0 con 0x7f12000603f0 2021-04-03T03:16:02.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1200147c00 con 0x7f12000add50 2021-04-03T03:16:02.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000b17b0 0x7f120015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000b17b0 0x7f120015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55860/0 (socket says 172.21.2.111:55860) 2021-04-03T03:16:02.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 -- 172.21.2.111:0/3091657092 learned_addr learned my addr 172.21.2.111:0/3091657092 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:02.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11feffd700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12000603f0 0x7f12000aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fe7fc700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12000add50 0x7f1200149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 -- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12000add50 msgr2=0x7f1200149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12000add50 0x7f1200149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 -- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12000603f0 msgr2=0x7f12000aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12000603f0 0x7f12000aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 -- 172.21.2.111:0/3091657092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200147ea0 con 0x7f12000b17b0 2021-04-03T03:16:02.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11ff7fe700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000b17b0 0x7f120015a650 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f12000ae520 tx=0x7f11f8007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11f7fff700 1 -- 172.21.2.111:0/3091657092 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f80058b0 con 0x7f12000b17b0 2021-04-03T03:16:02.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11f7fff700 1 -- 172.21.2.111:0/3091657092 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8005a10 con 0x7f12000b17b0 2021-04-03T03:16:02.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120014a070 con 0x7f12000b17b0 2021-04-03T03:16:02.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1200149b80 con 0x7f12000b17b0 2021-04-03T03:16:02.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.412+0000 7f11f7fff700 1 -- 172.21.2.111:0/3091657092 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f8005d70 con 0x7f12000b17b0 2021-04-03T03:16:02.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.416+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120015a390 con 0x7f12000b17b0 2021-04-03T03:16:02.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.416+0000 7f11f7fff700 1 -- 172.21.2.111:0/3091657092 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f11f8010030 con 0x7f12000b17b0 2021-04-03T03:16:02.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.416+0000 7f11f7fff700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11e40570a0 0x7f11e4059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.416+0000 7f11f7fff700 1 -- 172.21.2.111:0/3091657092 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f11f8070230 con 0x7f12000b17b0 2021-04-03T03:16:02.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.416+0000 7f11feffd700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11e40570a0 0x7f11e4059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.416+0000 7f11feffd700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11e40570a0 0x7f11e4059560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f11e801a000 tx=0x7f11e8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.416+0000 7f11f7fff700 1 -- 172.21.2.111:0/3091657092 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f11f8045690 con 0x7f12000b17b0 2021-04-03T03:16:02.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.544+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f1200057660 con 0x7f12000b17b0 2021-04-03T03:16:02.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11f7fff700 1 -- 172.21.2.111:0/3091657092 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v42) v1 ==== 189+0+0 (secure 0 0 0) 0x7f11f8048cb0 con 0x7f12000b17b0 2021-04-03T03:16:02.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11e40570a0 msgr2=0x7f11e4059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11e40570a0 0x7f11e4059560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f11e801a000 tx=0x7f11e8015000).stop 2021-04-03T03:16:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000b17b0 msgr2=0x7f120015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000b17b0 0x7f120015a650 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f12000ae520 tx=0x7f11f8007f50).stop 2021-04-03T03:16:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 shutdown_connections 2021-04-03T03:16:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12000b17b0 0x7f120015a650 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11e40570a0 0x7f11e4059560 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12000603f0 0x7f12000aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 --2- 172.21.2.111:0/3091657092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12000add50 0x7f1200149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 >> 172.21.2.111:0/3091657092 conn(0x7f12000b61c0 msgr2=0x7f12001440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 shutdown_connections 2021-04-03T03:16:02.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.552+0000 7f11fffff700 1 -- 172.21.2.111:0/3091657092 wait complete. 2021-04-03T03:16:02.558 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 19 and data pool 20 2021-04-03T03:16:02.562 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:02.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 Processor -- start 2021-04-03T03:16:02.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 -- start start 2021-04-03T03:16:02.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d140b12b0 0x7f0d141522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140607d0 con 0x7f0d140603f0 2021-04-03T03:16:02.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140b0eb0 con 0x7f0d1414b700 2021-04-03T03:16:02.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140af400 con 0x7f0d140b12b0 2021-04-03T03:16:02.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19a07700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0d140603f0 0x7f0d140b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56668/0 (socket says 172.21.2.111:56668) 2021-04-03T03:16:02.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19a07700 1 -- 172.21.2.111:0/1124304511 learned_addr learned my addr 172.21.2.111:0/1124304511 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:02.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1a208700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d140b12b0 0x7f0d141522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606242323 0 0) 0x7f0d140607d0 con 0x7f0d140603f0 2021-04-03T03:16:02.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cfc003240 con 0x7f0d140603f0 2021-04-03T03:16:02.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1917877356 0 0) 0x7f0cfc003240 con 0x7f0d140603f0 2021-04-03T03:16:02.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d140b12b0 msgr2=0x7f0d141522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:02.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d140b12b0 0x7f0d141522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 >> v1:172.21.2.115:6789/0 conn(0x7f0d1414b700 legacy=0x7f0d140b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140af6a0 con 0x7f0d140603f0 2021-04-03T03:16:02.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f0d040016d0 con 0x7f0d140603f0 2021-04-03T03:16:02.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 >> v1:172.21.2.111:6789/0 conn(0x7f0d140603f0 legacy=0x7f0d140b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d140b12b0 0x7f0cfc003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cfc003cc0 0x7f0cfc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cfc0049d0 0x7f0cfc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140af400 con 0x7f0cfc003cc0 2021-04-03T03:16:02.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc004250 con 0x7f0cfc0049d0 2021-04-03T03:16:02.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00b730 con 0x7f0d140b12b0 2021-04-03T03:16:02.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19206700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cfc003cc0 0x7f0cfc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19206700 1 -- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d140b12b0 msgr2=0x7f0cfc003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:02.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19206700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d140b12b0 0x7f0cfc003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19206700 1 -- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cfc0049d0 msgr2=0x7f0cfc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:02.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19206700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cfc0049d0 0x7f0cfc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19206700 1 -- 172.21.2.111:0/1124304511 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc003240 con 0x7f0cfc003cc0 2021-04-03T03:16:02.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d19206700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cfc003cc0 0x7f0cfc0044e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f0d140620f0 tx=0x7f0d10007b70).ready entity=mon.0 client_cookie=af76b317f5f7ce1c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1000c600 con 0x7f0cfc003cc0 2021-04-03T03:16:02.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1000c760 con 0x7f0cfc003cc0 2021-04-03T03:16:02.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d18a05700 1 -- 172.21.2.111:0/1124304511 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1000cac0 con 0x7f0cfc003cc0 2021-04-03T03:16:02.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 -- 172.21.2.111:0/1124304511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cfc003cc0 msgr2=0x7f0cfc0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1aa09700 1 --2- 172.21.2.111:0/1124304511 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cfc003cc0 0x7f0cfc0044e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f0d140620f0 tx=0x7f0d10007b70).stop 2021-04-03T03:16:02.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.616+0000 7f0d1a208700 1 -- 172.21.2.111:0/1124304511 reap_dead start 2021-04-03T03:16:02.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- 172.21.2.111:0/1124304511 shutdown_connections 2021-04-03T03:16:02.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- 172.21.2.111:0/1124304511 >> 172.21.2.111:0/1124304511 conn(0x7f0d140b61c0 msgr2=0x7f0d1405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- 172.21.2.111:0/1124304511 shutdown_connections 2021-04-03T03:16:02.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- 172.21.2.111:0/1124304511 wait complete. 2021-04-03T03:16:02.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 Processor -- start 2021-04-03T03:16:02.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- start start 2021-04-03T03:16:02.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d140a7950 0x7f0d140ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d14159b50 0x7f0d140628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d14062dc0 0x7f0d14067660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140a7d30 con 0x7f0d140a7950 2021-04-03T03:16:02.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14064f90 con 0x7f0d14062dc0 2021-04-03T03:16:02.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140aa210 con 0x7f0d14159b50 2021-04-03T03:16:02.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d140a7950 0x7f0d140ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d140a7950 0x7f0d140ab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55890/0 (socket says 172.21.2.111:55890) 2021-04-03T03:16:02.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 -- 172.21.2.111:0/3128057097 learned_addr learned my addr 172.21.2.111:0/3128057097 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:02.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1a208700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d14062dc0 0x7f0d14067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19206700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d14159b50 0x7f0d140628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 -- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d14159b50 msgr2=0x7f0d140628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d14159b50 0x7f0d140628d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 -- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d14062dc0 msgr2=0x7f0d14067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d14062dc0 0x7f0d14067660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 -- 172.21.2.111:0/3128057097 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14064c10 con 0x7f0d140a7950 2021-04-03T03:16:02.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19a07700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d140a7950 0x7f0d140ab340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f0d040013e0 tx=0x7f0d04001880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d02ffd700 1 -- 172.21.2.111:0/3128057097 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d04007170 con 0x7f0d140a7950 2021-04-03T03:16:02.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d02ffd700 1 -- 172.21.2.111:0/3128057097 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d040072d0 con 0x7f0d140a7950 2021-04-03T03:16:02.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d140a8570 con 0x7f0d140a7950 2021-04-03T03:16:02.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d02ffd700 1 -- 172.21.2.111:0/3128057097 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d040049c0 con 0x7f0d140a7950 2021-04-03T03:16:02.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1415a160 con 0x7f0d140a7950 2021-04-03T03:16:02.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d14064e00 con 0x7f0d140a7950 2021-04-03T03:16:02.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d02ffd700 1 -- 172.21.2.111:0/3128057097 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0d0401b060 con 0x7f0d140a7950 2021-04-03T03:16:02.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d02ffd700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cf4056ec0 0x7f0cf4059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d02ffd700 1 -- 172.21.2.111:0/3128057097 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0d0406faf0 con 0x7f0d140a7950 2021-04-03T03:16:02.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19206700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cf4056ec0 0x7f0cf4059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.620+0000 7f0d19206700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cf4056ec0 0x7f0cf4059380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f0d10002000 tx=0x7f0d10016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.624+0000 7f0d02ffd700 1 -- 172.21.2.111:0/3128057097 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0d04044dd0 con 0x7f0d140a7950 2021-04-03T03:16:02.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d1415a160 con 0x7f0d140a7950 2021-04-03T03:16:02.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d02ffd700 1 -- 172.21.2.111:0/3128057097 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16295 (secure 0 0 0) 0x7f0d1415a160 con 0x7f0d140a7950 2021-04-03T03:16:02.723 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:02.724 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":102,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:02.550086+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:00.429170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:01.518005+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:02.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cf4056ec0 msgr2=0x7f0cf4059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cf4056ec0 0x7f0cf4059380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f0d10002000 tx=0x7f0d10016000).stop 2021-04-03T03:16:02.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d140a7950 msgr2=0x7f0d140ab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d140a7950 0x7f0d140ab340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f0d040013e0 tx=0x7f0d04001880).stop 2021-04-03T03:16:02.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 shutdown_connections 2021-04-03T03:16:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0cf4056ec0 0x7f0cf4059380 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d14062dc0 0x7f0d14067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d140a7950 0x7f0d140ab340 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 --2- 172.21.2.111:0/3128057097 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d14159b50 0x7f0d140628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 >> 172.21.2.111:0/3128057097 conn(0x7f0d140b61c0 msgr2=0x7f0d140b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 shutdown_connections 2021-04-03T03:16:02.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.720+0000 7f0d1aa09700 1 -- 172.21.2.111:0/3128057097 wait complete. 2021-04-03T03:16:02.731 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:02.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 Processor -- start 2021-04-03T03:16:02.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 -- start start 2021-04-03T03:16:02.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac0dbb90 0x7f54ac042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f54ac03d360 0x7f54ac0dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0dc6e0 con 0x7f54ac0064b0 2021-04-03T03:16:02.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac0d9560 con 0x7f54ac03d360 2021-04-03T03:16:02.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac03dcb0 con 0x7f54ac0dbb90 2021-04-03T03:16:02.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b37fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f54ac0064b0 0x7f54ac0056b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56692/0 (socket says 172.21.2.111:56692) 2021-04-03T03:16:02.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b37fe700 1 -- 172.21.2.111:0/3286249890 learned_addr learned my addr 172.21.2.111:0/3286249890 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:02.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b27fc700 1 --2- 172.21.2.111:0/3286249890 >> v2:172.21.2.111:3300/0 conn(0x7f54ac03d360 0x7f54ac0dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b27fc700 1 -- 172.21.2.111:0/3286249890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac0dbb90 msgr2=0x7f54ac042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:02.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b27fc700 1 --2- 172.21.2.111:0/3286249890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac0dbb90 0x7f54ac042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b27fc700 1 -- 172.21.2.111:0/3286249890 >> v1:172.21.2.111:6789/0 conn(0x7f54ac0064b0 legacy=0x7f54ac0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b27fc700 1 -- 172.21.2.111:0/3286249890 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac0dc460 con 0x7f54ac03d360 2021-04-03T03:16:02.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b27fc700 1 --2- 172.21.2.111:0/3286249890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac03d360 0x7f54ac0dbf70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f54a4002f30 tx=0x7f54a4006dd0).ready entity=mon.0 client_cookie=16647fb16665166e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b1ffb700 1 -- 172.21.2.111:0/3286249890 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54a40097f0 con 0x7f54ac03d360 2021-04-03T03:16:02.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b1ffb700 1 -- 172.21.2.111:0/3286249890 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4009950 con 0x7f54ac03d360 2021-04-03T03:16:02.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b1ffb700 1 -- 172.21.2.111:0/3286249890 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54a4009cb0 con 0x7f54ac03d360 2021-04-03T03:16:02.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 -- 172.21.2.111:0/3286249890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac03d360 msgr2=0x7f54ac0dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 --2- 172.21.2.111:0/3286249890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac03d360 0x7f54ac0dbf70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f54a4002f30 tx=0x7f54a4006dd0).stop 2021-04-03T03:16:02.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 -- 172.21.2.111:0/3286249890 shutdown_connections 2021-04-03T03:16:02.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 --2- 172.21.2.111:0/3286249890 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac0dbb90 0x7f54ac042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 --2- 172.21.2.111:0/3286249890 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac03d360 0x7f54ac0dbf70 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.784+0000 7f54b3fff700 1 -- 172.21.2.111:0/3286249890 >> 172.21.2.111:0/3286249890 conn(0x7f54ac0462c0 msgr2=0x7f54ac0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- 172.21.2.111:0/3286249890 shutdown_connections 2021-04-03T03:16:02.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- 172.21.2.111:0/3286249890 wait complete. 2021-04-03T03:16:02.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 Processor -- start 2021-04-03T03:16:02.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- start start 2021-04-03T03:16:02.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f54ac0064b0 0x7f54ac0ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac03d360 0x7f54ac039d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac0dbb90 0x7f54ac0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac03c920 con 0x7f54ac0dbb90 2021-04-03T03:16:02.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0a1040 con 0x7f54ac0064b0 2021-04-03T03:16:02.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0a12e0 con 0x7f54ac03d360 2021-04-03T03:16:02.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac0dbb90 0x7f54ac0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac0dbb90 0x7f54ac0a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55904/0 (socket says 172.21.2.111:55904) 2021-04-03T03:16:02.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 -- 172.21.2.111:0/4084513737 learned_addr learned my addr 172.21.2.111:0/4084513737 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:02.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 -- 172.21.2.111:0/4084513737 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac03d360 msgr2=0x7f54ac039d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac03d360 0x7f54ac039d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 -- 172.21.2.111:0/4084513737 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f54ac0064b0 msgr2=0x7f54ac0ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f54ac0064b0 0x7f54ac0ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 -- 172.21.2.111:0/4084513737 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac03c5a0 con 0x7f54ac0dbb90 2021-04-03T03:16:02.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b37fe700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac0dbb90 0x7f54ac0a29e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f54ac047210 tx=0x7f54b405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54abfff700 1 -- 172.21.2.111:0/4084513737 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b405df60 con 0x7f54ac0dbb90 2021-04-03T03:16:02.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54abfff700 1 -- 172.21.2.111:0/4084513737 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b40699f0 con 0x7f54ac0dbb90 2021-04-03T03:16:02.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac09f520 con 0x7f54ac0dbb90 2021-04-03T03:16:02.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac0a01b0 con 0x7f54ac0dbb90 2021-04-03T03:16:02.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54abfff700 1 -- 172.21.2.111:0/4084513737 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b406b5d0 con 0x7f54ac0dbb90 2021-04-03T03:16:02.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac0a2740 con 0x7f54ac0dbb90 2021-04-03T03:16:02.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54abfff700 1 -- 172.21.2.111:0/4084513737 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f54b4061110 con 0x7f54ac0dbb90 2021-04-03T03:16:02.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54abfff700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5498056fb0 0x7f5498059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54abfff700 1 -- 172.21.2.111:0/4084513737 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f54b40df710 con 0x7f54ac0dbb90 2021-04-03T03:16:02.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b2ffd700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5498056fb0 0x7f5498059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.788+0000 7f54b2ffd700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5498056fb0 0x7f5498059470 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f549c002f30 tx=0x7f549c0069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.792+0000 7f54abfff700 1 -- 172.21.2.111:0/4084513737 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f54ac0a2740 con 0x7f54ac0dbb90 2021-04-03T03:16:02.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54ac0a2740 con 0x7f54ac0dbb90 2021-04-03T03:16:02.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54abfff700 1 -- 172.21.2.111:0/4084513737 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16295 (secure 0 0 0) 0x7f54ac0a2740 con 0x7f54ac0dbb90 2021-04-03T03:16:02.887 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:02.888 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":102,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:02.550086+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:00.429170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:01.518005+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5498056fb0 msgr2=0x7f5498059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5498056fb0 0x7f5498059470 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f549c002f30 tx=0x7f549c0069f0).stop 2021-04-03T03:16:02.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac0dbb90 msgr2=0x7f54ac0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac0dbb90 0x7f54ac0a29e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f54ac047210 tx=0x7f54b405ad70).stop 2021-04-03T03:16:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 shutdown_connections 2021-04-03T03:16:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f54ac0dbb90 0x7f54ac0a29e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5498056fb0 0x7f5498059470 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f54ac0064b0 0x7f54ac0ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 --2- 172.21.2.111:0/4084513737 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f54ac03d360 0x7f54ac039d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 >> 172.21.2.111:0/4084513737 conn(0x7f54ac0462c0 msgr2=0x7f54ac03f930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 shutdown_connections 2021-04-03T03:16:02.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.884+0000 7f54b3fff700 1 -- 172.21.2.111:0/4084513737 wait complete. 2021-04-03T03:16:02.895 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:16:02.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 Processor -- start 2021-04-03T03:16:02.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- start start 2021-04-03T03:16:02.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f250400bc90 0x7f2504029a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f250402aac0 0x7f2504029f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402a3f0 0x7f2504005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25040285c0 con 0x7f250400bc90 2021-04-03T03:16:02.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250400c070 con 0x7f250402aac0 2021-04-03T03:16:02.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25040287f0 con 0x7f250402a3f0 2021-04-03T03:16:02.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f250400bc90 0x7f2504029a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f250400bc90 0x7f2504029a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55910/0 (socket says 172.21.2.111:55910) 2021-04-03T03:16:02.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 -- 172.21.2.111:0/2124631607 learned_addr learned my addr 172.21.2.111:0/2124631607 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:02.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 -- 172.21.2.111:0/2124631607 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402a3f0 msgr2=0x7f2504005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:02.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 --2- 172.21.2.111:0/2124631607 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402a3f0 0x7f2504005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 -- 172.21.2.111:0/2124631607 >> v2:172.21.2.115:3300/0 conn(0x7f250402aac0 msgr2=0x7f2504029f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:02.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 --2- 172.21.2.111:0/2124631607 >> v2:172.21.2.115:3300/0 conn(0x7f250402aac0 0x7f2504029f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 -- 172.21.2.111:0/2124631607 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504044220 con 0x7f250400bc90 2021-04-03T03:16:02.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251125c700 1 --2- 172.21.2.111:0/2124631607 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250400bc90 0x7f2504029a10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f2508002f30 tx=0x7f2508006e90).ready entity=mon.0 client_cookie=ed85cf02a19fe3ff server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f24fffff700 1 -- 172.21.2.111:0/2124631607 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2508009b10 con 0x7f250400bc90 2021-04-03T03:16:02.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f24fffff700 1 -- 172.21.2.111:0/2124631607 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2508009c70 con 0x7f250400bc90 2021-04-03T03:16:02.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f24fffff700 1 -- 172.21.2.111:0/2124631607 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2508010560 con 0x7f250400bc90 2021-04-03T03:16:02.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- 172.21.2.111:0/2124631607 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250400bc90 msgr2=0x7f2504029a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- 172.21.2.111:0/2124631607 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250400bc90 0x7f2504029a10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f2508002f30 tx=0x7f2508006e90).stop 2021-04-03T03:16:02.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- 172.21.2.111:0/2124631607 shutdown_connections 2021-04-03T03:16:02.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- 172.21.2.111:0/2124631607 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402a3f0 0x7f2504005a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- 172.21.2.111:0/2124631607 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250400bc90 0x7f2504029a10 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:02.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- 172.21.2.111:0/2124631607 >> v2:172.21.2.115:3300/0 conn(0x7f250402aac0 0x7f2504029f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- 172.21.2.111:0/2124631607 >> 172.21.2.111:0/2124631607 conn(0x7f25040484c0 msgr2=0x7f25040488d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:02.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- 172.21.2.111:0/2124631607 shutdown_connections 2021-04-03T03:16:02.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- 172.21.2.111:0/2124631607 wait complete. 2021-04-03T03:16:02.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 Processor -- start 2021-04-03T03:16:02.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- start start 2021-04-03T03:16:02.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f250400bc90 0x7f25040df300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250402a3f0 0x7f2504040c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402aac0 0x7f250403ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250403e2d0 con 0x7f250402a3f0 2021-04-03T03:16:02.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250403e500 con 0x7f250400bc90 2021-04-03T03:16:02.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f251225e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250403e7a0 con 0x7f250402aac0 2021-04-03T03:16:02.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250402a3f0 0x7f2504040c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250402a3f0 0x7f2504040c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55916/0 (socket says 172.21.2.111:55916) 2021-04-03T03:16:02.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 -- 172.21.2.111:0/2951741419 learned_addr learned my addr 172.21.2.111:0/2951741419 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:02.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 -- 172.21.2.111:0/2951741419 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402aac0 msgr2=0x7f250403ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402aac0 0x7f250403ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 -- 172.21.2.111:0/2951741419 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f250400bc90 msgr2=0x7f25040df300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:02.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f250400bc90 0x7f25040df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:02.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 -- 172.21.2.111:0/2951741419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504042690 con 0x7f250402a3f0 2021-04-03T03:16:02.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f2510a5b700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250402a3f0 0x7f2504040c40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f2500002f30 tx=0x7f2500006d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f24fe7fc700 1 -- 172.21.2.111:0/2951741419 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f250000d4a0 con 0x7f250402a3f0 2021-04-03T03:16:02.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.952+0000 7f24fe7fc700 1 -- 172.21.2.111:0/2951741419 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2500000e90 con 0x7f250402a3f0 2021-04-03T03:16:02.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250402d510 con 0x7f250402a3f0 2021-04-03T03:16:02.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250403d420 con 0x7f250402a3f0 2021-04-03T03:16:02.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f24fe7fc700 1 -- 172.21.2.111:0/2951741419 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f250000d7f0 con 0x7f250402a3f0 2021-04-03T03:16:02.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250403d730 con 0x7f250402a3f0 2021-04-03T03:16:02.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f24fe7fc700 1 -- 172.21.2.111:0/2951741419 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f250000f030 con 0x7f250402a3f0 2021-04-03T03:16:02.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f24fe7fc700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24ec056fb0 0x7f24ec059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:02.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f24fe7fc700 1 -- 172.21.2.111:0/2951741419 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f250006ff20 con 0x7f250402a3f0 2021-04-03T03:16:02.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f251125c700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24ec056fb0 0x7f24ec059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:02.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f251125c700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24ec056fb0 0x7f24ec059470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f2508002f30 tx=0x7f2508017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:02.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:02.956+0000 7f24fe7fc700 1 -- 172.21.2.111:0/2951741419 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2500045200 con 0x7f250402a3f0 2021-04-03T03:16:03.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.084+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2504004db0 con 0x7f250402a3f0 2021-04-03T03:16:03.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f24fe7fc700 1 -- 172.21.2.111:0/2951741419 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2500048820 con 0x7f250402a3f0 2021-04-03T03:16:03.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24ec056fb0 msgr2=0x7f24ec059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24ec056fb0 0x7f24ec059470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f2508002f30 tx=0x7f2508017000).stop 2021-04-03T03:16:03.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250402a3f0 msgr2=0x7f2504040c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250402a3f0 0x7f2504040c40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f2500002f30 tx=0x7f2500006d80).stop 2021-04-03T03:16:03.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 shutdown_connections 2021-04-03T03:16:03.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f250402aac0 0x7f250403ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24ec056fb0 0x7f24ec059470 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f250400bc90 0x7f25040df300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 --2- 172.21.2.111:0/2951741419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f250402a3f0 0x7f2504040c40 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 >> 172.21.2.111:0/2951741419 conn(0x7f25040484c0 msgr2=0x7f25040079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:03.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 shutdown_connections 2021-04-03T03:16:03.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.560+0000 7f251225e700 1 -- 172.21.2.111:0/2951741419 wait complete. 2021-04-03T03:16:03.570 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:03.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672a12700 1 Processor -- start 2021-04-03T03:16:03.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672a12700 1 -- start start 2021-04-03T03:16:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672a12700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 0x7f166c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672a12700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f166c0b0c90 0x7f166c147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672a12700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c0607d0 con 0x7f166c0603f0 2021-04-03T03:16:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672a12700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c0af400 con 0x7f166c0b0c90 2021-04-03T03:16:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672a12700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0af6a0 con 0x7f166c14b700 2021-04-03T03:16:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672211700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f166c0b0c90 0x7f166c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:03.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672211700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f166c0b0c90 0x7f166c147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55928/0 (socket says 172.21.2.111:55928) 2021-04-03T03:16:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672211700 1 -- 172.21.2.111:0/1142683429 learned_addr learned my addr 172.21.2.111:0/1142683429 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672211700 1 -- 172.21.2.111:0/1142683429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 msgr2=0x7f166c0b07a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672211700 1 --2- 172.21.2.111:0/1142683429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 0x7f166c0b07a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672211700 1 -- 172.21.2.111:0/1142683429 >> v1:172.21.2.115:6789/0 conn(0x7f166c0603f0 legacy=0x7f166c0b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.628+0000 7f1672211700 1 -- 172.21.2.111:0/1142683429 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0afb40 con 0x7f166c0b0c90 2021-04-03T03:16:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672211700 1 --2- 172.21.2.111:0/1142683429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c147680 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f166c0af200 tx=0x7f1664009270).ready entity=mon.0 client_cookie=72f08ff0d333accf server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1670a0e700 1 -- 172.21.2.111:0/1142683429 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166400bbc0 con 0x7f166c0b0c90 2021-04-03T03:16:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1670a0e700 1 -- 172.21.2.111:0/1142683429 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166400bd20 con 0x7f166c0b0c90 2021-04-03T03:16:03.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1670a0e700 1 -- 172.21.2.111:0/1142683429 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1664011400 con 0x7f166c0b0c90 2021-04-03T03:16:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/1142683429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 msgr2=0x7f166c147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 --2- 172.21.2.111:0/1142683429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c147680 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f166c0af200 tx=0x7f1664009270).stop 2021-04-03T03:16:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/1142683429 shutdown_connections 2021-04-03T03:16:03.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 --2- 172.21.2.111:0/1142683429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c147680 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 --2- 172.21.2.111:0/1142683429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 0x7f166c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/1142683429 >> 172.21.2.111:0/1142683429 conn(0x7f166c0b61c0 msgr2=0x7f166c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:03.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/1142683429 shutdown_connections 2021-04-03T03:16:03.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/1142683429 wait complete. 2021-04-03T03:16:03.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 Processor -- start 2021-04-03T03:16:03.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- start start 2021-04-03T03:16:03.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f166c0603f0 0x7f166c062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 0x7f166c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0aaa40 con 0x7f166c0b0c90 2021-04-03T03:16:03.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c063400 con 0x7f166c0603f0 2021-04-03T03:16:03.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c064300 con 0x7f166c14b700 2021-04-03T03:16:03.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:03.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55932/0 (socket says 172.21.2.111:55932) 2021-04-03T03:16:03.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 -- 172.21.2.111:0/3824378592 learned_addr learned my addr 172.21.2.111:0/3824378592 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:03.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 -- 172.21.2.111:0/3824378592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 msgr2=0x7f166c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 0x7f166c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 -- 172.21.2.111:0/3824378592 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f166c0603f0 msgr2=0x7f166c062aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f166c0603f0 0x7f166c062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 -- 172.21.2.111:0/3824378592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0645a0 con 0x7f166c0b0c90 2021-04-03T03:16:03.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f167120f700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c159b00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f1668000e90 tx=0x7f1668006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:03.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f165effd700 1 -- 172.21.2.111:0/3824378592 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166800d4a0 con 0x7f166c0b0c90 2021-04-03T03:16:03.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f165effd700 1 -- 172.21.2.111:0/3824378592 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166800d600 con 0x7f166c0b0c90 2021-04-03T03:16:03.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c15a4e0 con 0x7f166c0b0c90 2021-04-03T03:16:03.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c0abf10 con 0x7f166c0b0c90 2021-04-03T03:16:03.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f165effd700 1 -- 172.21.2.111:0/3824378592 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166800d960 con 0x7f166c0b0c90 2021-04-03T03:16:03.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.632+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c065000 con 0x7f166c0b0c90 2021-04-03T03:16:03.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.636+0000 7f165effd700 1 -- 172.21.2.111:0/3824378592 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f166800f030 con 0x7f166c0b0c90 2021-04-03T03:16:03.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.636+0000 7f165effd700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f164c056ef0 0x7f164c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.636+0000 7f165effd700 1 -- 172.21.2.111:0/3824378592 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f166806ff60 con 0x7f166c0b0c90 2021-04-03T03:16:03.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.636+0000 7f1671a10700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f164c056ef0 0x7f164c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:03.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.636+0000 7f1671a10700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f164c056ef0 0x7f164c0593b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f1660002f70 tx=0x7f1660006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:03.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.636+0000 7f165effd700 1 -- 172.21.2.111:0/3824378592 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1668045350 con 0x7f166c0b0c90 2021-04-03T03:16:03.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f166c057660 con 0x7f166c0b0c90 2021-04-03T03:16:03.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f165effd700 1 -- 172.21.2.111:0/3824378592 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f1668048970 con 0x7f166c0b0c90 2021-04-03T03:16:03.766 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:03.767 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/3373357469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3373357469},{"type":"v1","addr":"172.21.2.115:6833","nonce":3373357469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:02.550193+0000","modified":"2021-04-03T03:16:03.553657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/3012485417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":3012485417},{"type":"v1","addr":"172.21.2.111:6835","nonce":3012485417}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-04-03T03:16:03.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f164c056ef0 msgr2=0x7f164c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f164c056ef0 0x7f164c0593b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f1660002f70 tx=0x7f1660006990).stop 2021-04-03T03:16:03.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 msgr2=0x7f166c159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c159b00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f1668000e90 tx=0x7f1668006d50).stop 2021-04-03T03:16:03.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 shutdown_connections 2021-04-03T03:16:03.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f166c14b700 0x7f166c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f164c056ef0 0x7f164c0593b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f166c0603f0 0x7f166c062aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 --2- 172.21.2.111:0/3824378592 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f166c0b0c90 0x7f166c159b00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 >> 172.21.2.111:0/3824378592 conn(0x7f166c0b61c0 msgr2=0x7f166c147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:03.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 shutdown_connections 2021-04-03T03:16:03.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.764+0000 7f1672a12700 1 -- 172.21.2.111:0/3824378592 wait complete. 2021-04-03T03:16:03.770 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 43 2021-04-03T03:16:03.774 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:03.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 Processor -- start 2021-04-03T03:16:03.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- start start 2021-04-03T03:16:03.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f370c0dbb90 0x7f370c042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f370c03d360 0x7f370c0dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0dc6e0 con 0x7f370c0064b0 2021-04-03T03:16:03.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c0d9560 con 0x7f370c0dbb90 2021-04-03T03:16:03.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c03dcb0 con 0x7f370c03d360 2021-04-03T03:16:03.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f370c0dbb90 0x7f370c042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:03.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f3712d9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f370c0064b0 0x7f370c0056b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56734/0 (socket says 172.21.2.111:56734) 2021-04-03T03:16:03.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f3712d9d700 1 -- 172.21.2.111:0/1619468326 learned_addr learned my addr 172.21.2.111:0/1619468326 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:03.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371259c700 1 -- 172.21.2.111:0/1619468326 >> v2:172.21.2.115:3300/0 conn(0x7f370c03d360 msgr2=0x7f370c0dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:03.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371259c700 1 --2- 172.21.2.111:0/1619468326 >> v2:172.21.2.115:3300/0 conn(0x7f370c03d360 0x7f370c0dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371259c700 1 -- 172.21.2.111:0/1619468326 >> v1:172.21.2.111:6789/0 conn(0x7f370c0064b0 legacy=0x7f370c0056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371259c700 1 -- 172.21.2.111:0/1619468326 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0dc460 con 0x7f370c0dbb90 2021-04-03T03:16:03.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371259c700 1 --2- 172.21.2.111:0/1619468326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c0dbb90 0x7f370c042750 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704006dd0).ready entity=mon.0 client_cookie=d888f030aa81efc4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:03.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371159a700 1 -- 172.21.2.111:0/1619468326 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37040097f0 con 0x7f370c0dbb90 2021-04-03T03:16:03.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371159a700 1 -- 172.21.2.111:0/1619468326 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704009950 con 0x7f370c0dbb90 2021-04-03T03:16:03.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371159a700 1 -- 172.21.2.111:0/1619468326 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3704009cb0 con 0x7f370c0dbb90 2021-04-03T03:16:03.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- 172.21.2.111:0/1619468326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c0dbb90 msgr2=0x7f370c042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 --2- 172.21.2.111:0/1619468326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c0dbb90 0x7f370c042750 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704006dd0).stop 2021-04-03T03:16:03.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- 172.21.2.111:0/1619468326 shutdown_connections 2021-04-03T03:16:03.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 --2- 172.21.2.111:0/1619468326 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c0dbb90 0x7f370c042750 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 --2- 172.21.2.111:0/1619468326 >> v2:172.21.2.115:3300/0 conn(0x7f370c03d360 0x7f370c0dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- 172.21.2.111:0/1619468326 >> 172.21.2.111:0/1619468326 conn(0x7f370c0462c0 msgr2=0x7f370c0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:03.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- 172.21.2.111:0/1619468326 shutdown_connections 2021-04-03T03:16:03.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- 172.21.2.111:0/1619468326 wait complete. 2021-04-03T03:16:03.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 Processor -- start 2021-04-03T03:16:03.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.828+0000 7f371359e700 1 -- start start 2021-04-03T03:16:03.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f370c0064b0 0x7f370c0d3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c03d360 0x7f370c0dae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f370c0dbb90 0x7f370c0cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0d8950 con 0x7f370c03d360 2021-04-03T03:16:03.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0eb6d0 con 0x7f370c0064b0 2021-04-03T03:16:03.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0d8b80 con 0x7f370c0dbb90 2021-04-03T03:16:03.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c03d360 0x7f370c0dae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:03.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c03d360 0x7f370c0dae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55948/0 (socket says 172.21.2.111:55948) 2021-04-03T03:16:03.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 -- 172.21.2.111:0/969445435 learned_addr learned my addr 172.21.2.111:0/969445435 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:03.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 -- 172.21.2.111:0/969445435 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f370c0dbb90 msgr2=0x7f370c0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f370c0dbb90 0x7f370c0cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 -- 172.21.2.111:0/969445435 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f370c0064b0 msgr2=0x7f370c0d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f370c0064b0 0x7f370c0d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 -- 172.21.2.111:0/969445435 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0d5a70 con 0x7f370c03d360 2021-04-03T03:16:03.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f3711d9b700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c03d360 0x7f370c0dae60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f3708002f30 tx=0x7f3708006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:03.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f37037fe700 1 -- 172.21.2.111:0/969445435 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370800d840 con 0x7f370c03d360 2021-04-03T03:16:03.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f37037fe700 1 -- 172.21.2.111:0/969445435 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3708000e90 con 0x7f370c03d360 2021-04-03T03:16:03.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f37037fe700 1 -- 172.21.2.111:0/969445435 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370800db90 con 0x7f370c03d360 2021-04-03T03:16:03.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c0eb1f0 con 0x7f370c03d360 2021-04-03T03:16:03.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c0cfa00 con 0x7f370c03d360 2021-04-03T03:16:03.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c0d8f40 con 0x7f370c03d360 2021-04-03T03:16:03.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f37037fe700 1 -- 172.21.2.111:0/969445435 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3708017030 con 0x7f370c03d360 2021-04-03T03:16:03.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f37037fe700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f36f0057000 0x7f36f00594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:03.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f37037fe700 1 -- 172.21.2.111:0/969445435 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f370806f720 con 0x7f370c03d360 2021-04-03T03:16:03.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371259c700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f36f0057000 0x7f36f00594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:03.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.832+0000 7f371259c700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f36f0057000 0x7f36f00594c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:03.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.836+0000 7f37037fe700 1 -- 172.21.2.111:0/969445435 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3708044ab0 con 0x7f370c03d360 2021-04-03T03:16:03.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.924+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f370c0eb500 con 0x7f370c03d360 2021-04-03T03:16:03.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.924+0000 7f37037fe700 1 -- 172.21.2.111:0/969445435 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16282 (secure 0 0 0) 0x7f37080480d0 con 0x7f370c03d360 2021-04-03T03:16:03.929 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:03.929 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":103,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:03.554720+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:00.429170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:01.518005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:03.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f36f0057000 msgr2=0x7f36f00594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f36f0057000 0x7f36f00594c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704015000).stop 2021-04-03T03:16:03.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c03d360 msgr2=0x7f370c0dae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:03.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c03d360 0x7f370c0dae60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f3708002f30 tx=0x7f3708006ea0).stop 2021-04-03T03:16:03.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 shutdown_connections 2021-04-03T03:16:03.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f370c0dbb90 0x7f370c0cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f36f0057000 0x7f36f00594c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f370c0064b0 0x7f370c0d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:03.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 --2- 172.21.2.111:0/969445435 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f370c03d360 0x7f370c0dae60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:03.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 >> 172.21.2.111:0/969445435 conn(0x7f370c0462c0 msgr2=0x7f370c0441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:03.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 shutdown_connections 2021-04-03T03:16:03.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.928+0000 7f371359e700 1 -- 172.21.2.111:0/969445435 wait complete. 2021-04-03T03:16:03.938 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:04.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 Processor -- start 2021-04-03T03:16:04.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- start start 2021-04-03T03:16:04.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1210153440 0x7f1210080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12100b3ca0 con 0x7f12100b38c0 2021-04-03T03:16:04.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1210025b90 con 0x7f1210025480 2021-04-03T03:16:04.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1210025e30 con 0x7f1210153440 2021-04-03T03:16:04.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120e59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1210153440 0x7f1210080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120e59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1210153440 0x7f1210080c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55952/0 (socket says 172.21.2.111:55952) 2021-04-03T03:16:04.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120e59c700 1 -- 172.21.2.111:0/3251680812 learned_addr learned my addr 172.21.2.111:0/3251680812 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:04.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120ed9d700 1 --1- 172.21.2.111:0/3251680812 >> v1:172.21.2.111:6789/0 conn(0x7f12100b38c0 0x7f1210080550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56746/0 (socket says 172.21.2.111:56746) 2021-04-03T03:16:04.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120e59c700 1 -- 172.21.2.111:0/3251680812 >> v1:172.21.2.115:6789/0 conn(0x7f1210025480 legacy=0x7f12100af2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:04.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120e59c700 1 -- 172.21.2.111:0/3251680812 >> v1:172.21.2.111:6789/0 conn(0x7f12100b38c0 legacy=0x7f1210080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120e59c700 1 -- 172.21.2.111:0/3251680812 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12100260d0 con 0x7f1210153440 2021-04-03T03:16:04.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120e59c700 1 --2- 172.21.2.111:0/3251680812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210153440 0x7f1210080c60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f1210063a70 tx=0x7f11f8006e90).ready entity=mon.0 client_cookie=3bdf1dcffac0ced6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120dd9b700 1 -- 172.21.2.111:0/3251680812 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f8009980 con 0x7f1210153440 2021-04-03T03:16:04.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120dd9b700 1 -- 172.21.2.111:0/3251680812 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8009ae0 con 0x7f1210153440 2021-04-03T03:16:04.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120dd9b700 1 -- 172.21.2.111:0/3251680812 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f8009e80 con 0x7f1210153440 2021-04-03T03:16:04.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- 172.21.2.111:0/3251680812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210153440 msgr2=0x7f1210080c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 --2- 172.21.2.111:0/3251680812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210153440 0x7f1210080c60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f1210063a70 tx=0x7f11f8006e90).stop 2021-04-03T03:16:04.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- 172.21.2.111:0/3251680812 shutdown_connections 2021-04-03T03:16:04.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 --2- 172.21.2.111:0/3251680812 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210153440 0x7f1210080c60 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- 172.21.2.111:0/3251680812 >> 172.21.2.111:0/3251680812 conn(0x7f1210061610 msgr2=0x7f1210061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- 172.21.2.111:0/3251680812 shutdown_connections 2021-04-03T03:16:04.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- 172.21.2.111:0/3251680812 wait complete. 2021-04-03T03:16:04.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 Processor -- start 2021-04-03T03:16:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- start start 2021-04-03T03:16:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210025480 0x7f1210132e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12100b38c0 0x7f1210066da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1210153440 0x7f1210067290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12100801d0 con 0x7f1210025480 2021-04-03T03:16:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12100ac690 con 0x7f12100b38c0 2021-04-03T03:16:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1210064050 con 0x7f1210153440 2021-04-03T03:16:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210025480 0x7f1210132e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210025480 0x7f1210132e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55960/0 (socket says 172.21.2.111:55960) 2021-04-03T03:16:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:03.996+0000 7f120ed9d700 1 -- 172.21.2.111:0/772774797 learned_addr learned my addr 172.21.2.111:0/772774797 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120ed9d700 1 -- 172.21.2.111:0/772774797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1210153440 msgr2=0x7f1210067290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:04.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120ed9d700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1210153440 0x7f1210067290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120ed9d700 1 -- 172.21.2.111:0/772774797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12100b38c0 msgr2=0x7f1210066da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:04.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120ed9d700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12100b38c0 0x7f1210066da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120ed9d700 1 -- 172.21.2.111:0/772774797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12100642f0 con 0x7f1210025480 2021-04-03T03:16:04.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120ed9d700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210025480 0x7f1210132e40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f1210133b70 tx=0x7f1200007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f12077fe700 1 -- 172.21.2.111:0/772774797 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1200007320 con 0x7f1210025480 2021-04-03T03:16:04.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f12077fe700 1 -- 172.21.2.111:0/772774797 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1200005530 con 0x7f1210025480 2021-04-03T03:16:04.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f12077fe700 1 -- 172.21.2.111:0/772774797 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1200005890 con 0x7f1210025480 2021-04-03T03:16:04.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12100ad810 con 0x7f1210025480 2021-04-03T03:16:04.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1210133710 con 0x7f1210025480 2021-04-03T03:16:04.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12100ac0a0 con 0x7f1210025480 2021-04-03T03:16:04.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f12077fe700 1 -- 172.21.2.111:0/772774797 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1200010030 con 0x7f1210025480 2021-04-03T03:16:04.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f12077fe700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f4056f20 0x7f11f40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f12077fe700 1 -- 172.21.2.111:0/772774797 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f120006ff70 con 0x7f1210025480 2021-04-03T03:16:04.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120e59c700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f4056f20 0x7f11f40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.000+0000 7f120e59c700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f4056f20 0x7f11f40593e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f1210063a70 tx=0x7f11f8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.004+0000 7f12077fe700 1 -- 172.21.2.111:0/772774797 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f12000451f0 con 0x7f1210025480 2021-04-03T03:16:04.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.132+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f12100ac4a0 con 0x7f1210025480 2021-04-03T03:16:04.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f12077fe700 1 -- 172.21.2.111:0/772774797 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1200048810 con 0x7f1210025480 2021-04-03T03:16:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f4056f20 msgr2=0x7f11f40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f4056f20 0x7f11f40593e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f1210063a70 tx=0x7f11f8015000).stop 2021-04-03T03:16:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210025480 msgr2=0x7f1210132e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210025480 0x7f1210132e40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f1210133b70 tx=0x7f1200007c20).stop 2021-04-03T03:16:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 shutdown_connections 2021-04-03T03:16:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1210153440 0x7f1210067290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1210025480 0x7f1210132e40 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f4056f20 0x7f11f40593e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 --2- 172.21.2.111:0/772774797 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12100b38c0 0x7f1210066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 >> 172.21.2.111:0/772774797 conn(0x7f1210061610 msgr2=0x7f12100b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 shutdown_connections 2021-04-03T03:16:04.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.136+0000 7f120f59e700 1 -- 172.21.2.111:0/772774797 wait complete. 2021-04-03T03:16:04.142 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:16:04.146 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:04.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 Processor -- start 2021-04-03T03:16:04.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- start start 2021-04-03T03:16:04.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1980153440 0x7f1980080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19800b3ca0 con 0x7f1980025480 2021-04-03T03:16:04.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1980025b90 con 0x7f19800b38c0 2021-04-03T03:16:04.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1980025e30 con 0x7f1980153440 2021-04-03T03:16:04.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1980153440 0x7f1980080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1980153440 0x7f1980080c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55968/0 (socket says 172.21.2.111:55968) 2021-04-03T03:16:04.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1986054700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1980025480 0x7f19800af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56762/0 (socket says 172.21.2.111:56762) 2021-04-03T03:16:04.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197f7fe700 1 -- 172.21.2.111:0/3263557642 learned_addr learned my addr 172.21.2.111:0/3263557642 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:04.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197f7fe700 1 -- 172.21.2.111:0/3263557642 >> v1:172.21.2.115:6789/0 conn(0x7f19800b38c0 legacy=0x7f1980080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197effd700 1 -- 172.21.2.111:0/3263557642 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162649198 0 0) 0x7f19800b3ca0 con 0x7f1980025480 2021-04-03T03:16:04.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197f7fe700 1 -- 172.21.2.111:0/3263557642 >> v1:172.21.2.111:6789/0 conn(0x7f1980025480 legacy=0x7f19800af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197f7fe700 1 -- 172.21.2.111:0/3263557642 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19800260d0 con 0x7f1980153440 2021-04-03T03:16:04.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197f7fe700 1 --2- 172.21.2.111:0/3263557642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980080c60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f1968002f30 tx=0x7f1968006e90).ready entity=mon.0 client_cookie=13c342b5a1947ee3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197effd700 1 -- 172.21.2.111:0/3263557642 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1968009a10 con 0x7f1980153440 2021-04-03T03:16:04.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197effd700 1 -- 172.21.2.111:0/3263557642 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1968009b70 con 0x7f1980153440 2021-04-03T03:16:04.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f197effd700 1 -- 172.21.2.111:0/3263557642 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1968011400 con 0x7f1980153440 2021-04-03T03:16:04.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- 172.21.2.111:0/3263557642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 msgr2=0x7f1980080c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 --2- 172.21.2.111:0/3263557642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980080c60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f1968002f30 tx=0x7f1968006e90).stop 2021-04-03T03:16:04.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- 172.21.2.111:0/3263557642 shutdown_connections 2021-04-03T03:16:04.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 --2- 172.21.2.111:0/3263557642 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980080c60 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- 172.21.2.111:0/3263557642 >> 172.21.2.111:0/3263557642 conn(0x7f1980061610 msgr2=0x7f1980061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- 172.21.2.111:0/3263557642 shutdown_connections 2021-04-03T03:16:04.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 -- 172.21.2.111:0/3263557642 wait complete. 2021-04-03T03:16:04.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.204+0000 7f1984bda700 1 Processor -- start 2021-04-03T03:16:04.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 -- start start 2021-04-03T03:16:04.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1980025480 0x7f1980064450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19800b38c0 0x7f198012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980130e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980067f30 con 0x7f1980153440 2021-04-03T03:16:04.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980064df0 con 0x7f19800b38c0 2021-04-03T03:16:04.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1980065090 con 0x7f1980025480 2021-04-03T03:16:04.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1986054700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980130e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1986054700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980130e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55976/0 (socket says 172.21.2.111:55976) 2021-04-03T03:16:04.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1986054700 1 -- 172.21.2.111:0/3146947144 learned_addr learned my addr 172.21.2.111:0/3146947144 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:04.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197ffff700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1980025480 0x7f1980064450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197f7fe700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19800b38c0 0x7f198012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197ffff700 1 -- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19800b38c0 msgr2=0x7f198012ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197ffff700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19800b38c0 0x7f198012ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197ffff700 1 -- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 msgr2=0x7f1980130e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197ffff700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980130e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197ffff700 1 -- 172.21.2.111:0/3146947144 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980065350 con 0x7f1980025480 2021-04-03T03:16:04.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1986054700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980130e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:16:04.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197ffff700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1980025480 0x7f1980064450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1980063a70 tx=0x7f1970007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197cff9700 1 -- 172.21.2.111:0/3146947144 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1970013410 con 0x7f1980025480 2021-04-03T03:16:04.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1980131360 con 0x7f1980025480 2021-04-03T03:16:04.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19800657f0 con 0x7f1980025480 2021-04-03T03:16:04.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197cff9700 1 -- 172.21.2.111:0/3146947144 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1970013570 con 0x7f1980025480 2021-04-03T03:16:04.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197cff9700 1 -- 172.21.2.111:0/3146947144 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19700138d0 con 0x7f1980025480 2021-04-03T03:16:04.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19800654e0 con 0x7f1980025480 2021-04-03T03:16:04.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197cff9700 1 -- 172.21.2.111:0/3146947144 <== mon.2 v2:172.21.2.115:3301/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1970010030 con 0x7f1980025480 2021-04-03T03:16:04.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197cff9700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1964056f20 0x7f19640593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197cff9700 1 -- 172.21.2.111:0/3146947144 <== mon.2 v2:172.21.2.115:3301/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1970001e40 con 0x7f1980025480 2021-04-03T03:16:04.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197f7fe700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1964056f20 0x7f19640593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.208+0000 7f197f7fe700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1964056f20 0x7f19640593e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f196801c000 tx=0x7f1968015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.216+0000 7f197cff9700 1 -- 172.21.2.111:0/3146947144 <== mon.2 v2:172.21.2.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1970045700 con 0x7f1980025480 2021-04-03T03:16:04.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.344+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1980057660 con 0x7f1980025480 2021-04-03T03:16:04.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.356+0000 7f197cff9700 1 -- 172.21.2.111:0/3146947144 <== mon.2 v2:172.21.2.115:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1970048d20 con 0x7f1980025480 2021-04-03T03:16:04.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.356+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1964056f20 msgr2=0x7f19640593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.356+0000 7f1984bda700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1964056f20 0x7f19640593e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f196801c000 tx=0x7f1968015000).stop 2021-04-03T03:16:04.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.356+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1980025480 msgr2=0x7f1980064450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.356+0000 7f1984bda700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1980025480 0x7f1980064450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1980063a70 tx=0x7f1970007b70).stop 2021-04-03T03:16:04.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 shutdown_connections 2021-04-03T03:16:04.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1980153440 0x7f1980130e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1980025480 0x7f1980064450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1964056f20 0x7f19640593e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 --2- 172.21.2.111:0/3146947144 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f19800b38c0 0x7f198012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 >> 172.21.2.111:0/3146947144 conn(0x7f1980061610 msgr2=0x7f19800b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 shutdown_connections 2021-04-03T03:16:04.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.360+0000 7f1984bda700 1 -- 172.21.2.111:0/3146947144 wait complete. 2021-04-03T03:16:04.364 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:16:04.368 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:04.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 Processor -- start 2021-04-03T03:16:04.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 -- start start 2021-04-03T03:16:04.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff65803d350 0x7ff65803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff658041240 con 0x7ff658046ce0 2021-04-03T03:16:04.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6580f1850 con 0x7ff658042380 2021-04-03T03:16:04.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff658042810 con 0x7ff65803d350 2021-04-03T03:16:04.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff658046ce0 0x7ff658041c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56772/0 (socket says 172.21.2.111:56772) 2021-04-03T03:16:04.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 -- 172.21.2.111:0/152185324 learned_addr learned my addr 172.21.2.111:0/152185324 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:04.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65ede5700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff65803d350 0x7ff65803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235423196 0 0) 0x7ff658041240 con 0x7ff658046ce0 2021-04-03T03:16:04.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff64c003240 con 0x7ff658046ce0 2021-04-03T03:16:04.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1194316442 0 0) 0x7ff64c003240 con 0x7ff658046ce0 2021-04-03T03:16:04.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff65803d350 msgr2=0x7ff65803f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:04.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff65803d350 0x7ff65803f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 >> v1:172.21.2.115:6789/0 conn(0x7ff658042380 legacy=0x7ff6580f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580470c0 con 0x7ff658046ce0 2021-04-03T03:16:04.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ff6480016d0 con 0x7ff658046ce0 2021-04-03T03:16:04.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 >> v1:172.21.2.111:6789/0 conn(0x7ff658046ce0 legacy=0x7ff658041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff64c003760 0x7ff64c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff64c0044e0 0x7ff64c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff64c004e50 0x7ff64c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff658042810 con 0x7ff64c003760 2021-04-03T03:16:04.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0041a0 con 0x7ff64c004e50 2021-04-03T03:16:04.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c04fc30 con 0x7ff64c0044e0 2021-04-03T03:16:04.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff64c003760 0x7ff64c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65ede5700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff64c0044e0 0x7ff64c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff665452700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff64c004e50 0x7ff64c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 -- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff64c0044e0 msgr2=0x7ff64c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:04.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff64c0044e0 0x7ff64c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 -- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff64c004e50 msgr2=0x7ff64c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:04.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff64c004e50 0x7ff64c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 -- 172.21.2.111:0/152185324 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580470c0 con 0x7ff64c003760 2021-04-03T03:16:04.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff64c003760 0x7ff64c003b80 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7ff6480013e0 tx=0x7ff6480077e0).ready entity=mon.0 client_cookie=d42466fbf33c22f0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff64800ce50 con 0x7ff64c003760 2021-04-03T03:16:04.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6480072e0 con 0x7ff64c003760 2021-04-03T03:16:04.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65e5e4700 1 -- 172.21.2.111:0/152185324 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6480055f0 con 0x7ff64c003760 2021-04-03T03:16:04.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 -- 172.21.2.111:0/152185324 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff64c003760 msgr2=0x7ff64c003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff65fde7700 1 --2- 172.21.2.111:0/152185324 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff64c003760 0x7ff64c003b80 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7ff6480013e0 tx=0x7ff6480077e0).stop 2021-04-03T03:16:04.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.424+0000 7ff665452700 1 -- 172.21.2.111:0/152185324 reap_dead start 2021-04-03T03:16:04.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- 172.21.2.111:0/152185324 shutdown_connections 2021-04-03T03:16:04.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- 172.21.2.111:0/152185324 >> 172.21.2.111:0/152185324 conn(0x7ff65800d0c0 msgr2=0x7ff6580462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- 172.21.2.111:0/152185324 shutdown_connections 2021-04-03T03:16:04.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- 172.21.2.111:0/152185324 wait complete. 2021-04-03T03:16:04.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 Processor -- start 2021-04-03T03:16:04.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- start start 2021-04-03T03:16:04.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff65803bb80 0x7ff65803bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6580a2500 0x7ff6580df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff65803c930 0x7ff6580dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580a0da0 con 0x7ff65803c930 2021-04-03T03:16:04.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580f16b0 con 0x7ff65803bb80 2021-04-03T03:16:04.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580b10e0 con 0x7ff6580a2500 2021-04-03T03:16:04.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff65803c930 0x7ff6580dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff65803c930 0x7ff6580dfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:55996/0 (socket says 172.21.2.111:55996) 2021-04-03T03:16:04.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 -- 172.21.2.111:0/3189310146 learned_addr learned my addr 172.21.2.111:0/3189310146 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:04.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65ede5700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6580a2500 0x7ff6580df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff65803bb80 0x7ff65803bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 -- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6580a2500 msgr2=0x7ff6580df5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6580a2500 0x7ff6580df5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 -- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff65803bb80 msgr2=0x7ff65803bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff65803bb80 0x7ff65803bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 -- 172.21.2.111:0/3189310146 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580b1380 con 0x7ff65803c930 2021-04-03T03:16:04.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65ede5700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6580a2500 0x7ff6580df5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:04.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff65803bb80 0x7ff65803bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:04.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff665452700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff65803c930 0x7ff6580dfaa0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7ff6580a0920 tx=0x7ff66005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff657fff700 1 -- 172.21.2.111:0/3189310146 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6600699f0 con 0x7ff65803c930 2021-04-03T03:16:04.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff657fff700 1 -- 172.21.2.111:0/3189310146 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6600685b0 con 0x7ff65803c930 2021-04-03T03:16:04.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6580ce6e0 con 0x7ff65803c930 2021-04-03T03:16:04.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff657fff700 1 -- 172.21.2.111:0/3189310146 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff660068910 con 0x7ff65803c930 2021-04-03T03:16:04.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6580e01a0 con 0x7ff65803c930 2021-04-03T03:16:04.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6580a2930 con 0x7ff65803c930 2021-04-03T03:16:04.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff657fff700 1 -- 172.21.2.111:0/3189310146 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff66005f2c0 con 0x7ff65803c930 2021-04-03T03:16:04.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff657fff700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff644056e90 0x7ff644059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff657fff700 1 -- 172.21.2.111:0/3189310146 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff6600d0160 con 0x7ff65803c930 2021-04-03T03:16:04.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.428+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff644056e90 0x7ff644059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.432+0000 7ff65f5e6700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff644056e90 0x7ff644059350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ff648018000 tx=0x7ff648015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.432+0000 7ff657fff700 1 -- 172.21.2.111:0/3189310146 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff6600a54f0 con 0x7ff65803c930 2021-04-03T03:16:04.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.560+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff6580b1720 con 0x7ff65803c930 2021-04-03T03:16:04.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.564+0000 7ff657fff700 1 -- 172.21.2.111:0/3189310146 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff6600a8b10 con 0x7ff65803c930 2021-04-03T03:16:04.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff644056e90 msgr2=0x7ff644059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff644056e90 0x7ff644059350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ff648018000 tx=0x7ff648015000).stop 2021-04-03T03:16:04.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff65803c930 msgr2=0x7ff6580dfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff65803c930 0x7ff6580dfaa0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7ff6580a0920 tx=0x7ff66005ad70).stop 2021-04-03T03:16:04.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 shutdown_connections 2021-04-03T03:16:04.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff65803c930 0x7ff6580dfaa0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff644056e90 0x7ff644059350 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff65803bb80 0x7ff65803bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 --2- 172.21.2.111:0/3189310146 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6580a2500 0x7ff6580df5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 >> 172.21.2.111:0/3189310146 conn(0x7ff65800d0c0 msgr2=0x7ff6580f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 shutdown_connections 2021-04-03T03:16:04.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.568+0000 7ff65fde7700 1 -- 172.21.2.111:0/3189310146 wait complete. 2021-04-03T03:16:04.572 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:16:04.576 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:04.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 Processor -- start 2021-04-03T03:16:04.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- start start 2021-04-03T03:16:04.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd5180603f0 0x7fd5180b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd51814b700 0x7fd5180b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180607d0 con 0x7fd5180b12b0 2021-04-03T03:16:04.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180af400 con 0x7fd51814b700 2021-04-03T03:16:04.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180af6a0 con 0x7fd5180603f0 2021-04-03T03:16:04.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d8c8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd51814b700 0x7fd5180b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51e8ca700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd5180b12b0 0x7fd5181465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56792/0 (socket says 172.21.2.111:56792) 2021-04-03T03:16:04.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51e8ca700 1 -- 172.21.2.111:0/3318119804 learned_addr learned my addr 172.21.2.111:0/3318119804 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:04.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d8c8700 1 -- 172.21.2.111:0/3318119804 >> v2:172.21.2.115:3300/0 conn(0x7fd5180603f0 msgr2=0x7fd5180b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:04.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d8c8700 1 --2- 172.21.2.111:0/3318119804 >> v2:172.21.2.115:3300/0 conn(0x7fd5180603f0 0x7fd5180b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d8c8700 1 -- 172.21.2.111:0/3318119804 >> v1:172.21.2.111:6789/0 conn(0x7fd5180b12b0 legacy=0x7fd5181465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d8c8700 1 -- 172.21.2.111:0/3318119804 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180afb40 con 0x7fd51814b700 2021-04-03T03:16:04.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d8c8700 1 --2- 172.21.2.111:0/3318119804 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180b0580 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fd514002f30 tx=0x7fd514006dd0).ready entity=mon.0 client_cookie=724dfe06738a7edf server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d0c7700 1 -- 172.21.2.111:0/3318119804 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5140097f0 con 0x7fd51814b700 2021-04-03T03:16:04.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d0c7700 1 -- 172.21.2.111:0/3318119804 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd514009950 con 0x7fd51814b700 2021-04-03T03:16:04.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51d0c7700 1 -- 172.21.2.111:0/3318119804 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd514009cb0 con 0x7fd51814b700 2021-04-03T03:16:04.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- 172.21.2.111:0/3318119804 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 msgr2=0x7fd5180b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/3318119804 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180b0580 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fd514002f30 tx=0x7fd514006dd0).stop 2021-04-03T03:16:04.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- 172.21.2.111:0/3318119804 shutdown_connections 2021-04-03T03:16:04.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/3318119804 >> v2:172.21.2.115:3300/0 conn(0x7fd5180603f0 0x7fd5180b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/3318119804 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180b0580 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- 172.21.2.111:0/3318119804 >> 172.21.2.111:0/3318119804 conn(0x7fd5180b61c0 msgr2=0x7fd51805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- 172.21.2.111:0/3318119804 shutdown_connections 2021-04-03T03:16:04.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.676+0000 7fd51f0cb700 1 -- 172.21.2.111:0/3318119804 wait complete. 2021-04-03T03:16:04.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 Processor -- start 2021-04-03T03:16:04.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 -- start start 2021-04-03T03:16:04.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd5180603f0 0x7fd5180a8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd5180b12b0 0x7fd5180abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518062e50 con 0x7fd51814b700 2021-04-03T03:16:04.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180630f0 con 0x7fd5180603f0 2021-04-03T03:16:04.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180628d0 con 0x7fd5180b12b0 2021-04-03T03:16:04.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56010/0 (socket says 172.21.2.111:56010) 2021-04-03T03:16:04.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 -- 172.21.2.111:0/2730486717 learned_addr learned my addr 172.21.2.111:0/2730486717 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:04.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e0c9700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd5180603f0 0x7fd5180a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 -- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd5180b12b0 msgr2=0x7fd5180abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd5180b12b0 0x7fd5180abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 -- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd5180603f0 msgr2=0x7fd5180a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd5180603f0 0x7fd5180a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 -- 172.21.2.111:0/2730486717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518062a90 con 0x7fd51814b700 2021-04-03T03:16:04.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e0c9700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd5180603f0 0x7fd5180a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:04.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e8ca700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180a78a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fd51806a3b0 tx=0x7fd510007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd5077fe700 1 -- 172.21.2.111:0/2730486717 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5100020b0 con 0x7fd51814b700 2021-04-03T03:16:04.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd5077fe700 1 -- 172.21.2.111:0/2730486717 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510005b80 con 0x7fd51814b700 2021-04-03T03:16:04.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5180675c0 con 0x7fd51814b700 2021-04-03T03:16:04.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd5077fe700 1 -- 172.21.2.111:0/2730486717 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd510007110 con 0x7fd51814b700 2021-04-03T03:16:04.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd518067a80 con 0x7fd51814b700 2021-04-03T03:16:04.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5180632b0 con 0x7fd51814b700 2021-04-03T03:16:04.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd5077fe700 1 -- 172.21.2.111:0/2730486717 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd510010030 con 0x7fd51814b700 2021-04-03T03:16:04.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd5077fe700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4f8056f20 0x7fd4f80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:04.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd5077fe700 1 -- 172.21.2.111:0/2730486717 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd510070100 con 0x7fd51814b700 2021-04-03T03:16:04.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e0c9700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4f8056f20 0x7fd4f80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:04.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.680+0000 7fd51e0c9700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4f8056f20 0x7fd4f80593e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fd508007000 tx=0x7fd508008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:04.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.684+0000 7fd5077fe700 1 -- 172.21.2.111:0/2730486717 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd5100454f0 con 0x7fd51814b700 2021-04-03T03:16:04.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.804+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5180aa3a0 con 0x7fd51814b700 2021-04-03T03:16:04.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.804+0000 7fd5077fe700 1 -- 172.21.2.111:0/2730486717 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fd510048b10 con 0x7fd51814b700 2021-04-03T03:16:04.809 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:04.809 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/3373357469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3373357469},{"type":"v1","addr":"172.21.2.115:6833","nonce":3373357469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:02.550193+0000","modified":"2021-04-03T03:16:03.553657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/3012485417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":3012485417},{"type":"v1","addr":"172.21.2.111:6835","nonce":3012485417}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-04-03T03:16:04.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4f8056f20 msgr2=0x7fd4f80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4f8056f20 0x7fd4f80593e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fd508007000 tx=0x7fd508008040).stop 2021-04-03T03:16:04.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 msgr2=0x7fd5180a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:04.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180a78a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fd51806a3b0 tx=0x7fd510007ee0).stop 2021-04-03T03:16:04.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 shutdown_connections 2021-04-03T03:16:04.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd51814b700 0x7fd5180a78a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd4f8056f20 0x7fd4f80593e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd5180603f0 0x7fd5180a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:04.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 --2- 172.21.2.111:0/2730486717 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd5180b12b0 0x7fd5180abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:04.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 >> 172.21.2.111:0/2730486717 conn(0x7fd5180b61c0 msgr2=0x7fd518148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:04.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 shutdown_connections 2021-04-03T03:16:04.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:04.808+0000 7fd51f0cb700 1 -- 172.21.2.111:0/2730486717 wait complete. 2021-04-03T03:16:04.813 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 43 2021-04-03T03:16:05.018 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:02.550193+0000', 'modified': '2021-04-03T03:16:03.553657+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7179': {'gid': 7179, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/3012485417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 3012485417}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 3012485417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:05.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:05.019 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:05.091 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7179,"want_state":"up:active","state":"up:active","rank_uptime":2.5340733809999998,"mdsmap_epoch":43,"osdmap_epoch":103,"osdmap_epoch_barrier":102,"uptime":6.7723356749999999} 2021-04-03T03:16:05.091 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7179, "mdsmap_epoch": 43, "osdmap_epoch": 103, "osdmap_epoch_barrier": 102, "rank_uptime": 2.534073381, "state": "up:active", "uptime": 6.772335675, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:05.092 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:05.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.144+0000 7f1807246700 1 Processor -- start 2021-04-03T03:16:05.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.144+0000 7f1807246700 1 -- start start 2021-04-03T03:16:05.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.144+0000 7f1807246700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000603f0 0x7f18000b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.144+0000 7f1807246700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f18000b44a0 0x7f18000b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.144+0000 7f1807246700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18000b4bc0 con 0x7f18000b5d60 2021-04-03T03:16:05.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.144+0000 7f1807246700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18000607d0 con 0x7f18000b44a0 2021-04-03T03:16:05.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.144+0000 7f1807246700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000b4df0 con 0x7f18000603f0 2021-04-03T03:16:05.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f18000b44a0 0x7f18000b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806244700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000603f0 0x7f18000b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f18000b44a0 0x7f18000b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:45726/0 (socket says 172.21.2.111:45726) 2021-04-03T03:16:05.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806244700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000603f0 0x7f18000b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:39362/0 (socket says 172.21.2.111:39362) 2021-04-03T03:16:05.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 -- 172.21.2.111:0/2245614928 learned_addr learned my addr 172.21.2.111:0/2245614928 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 -- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000603f0 msgr2=0x7f18000b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 --2- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000603f0 0x7f18000b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 -- 172.21.2.111:0/2245614928 >> v1:172.21.2.115:6789/0 conn(0x7f18000b5d60 legacy=0x7f18000b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1805242700 1 -- 172.21.2.111:0/2245614928 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051217498 0 0) 0x7f18000b4bc0 con 0x7f18000b5d60 2021-04-03T03:16:05.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 -- 172.21.2.111:0/2245614928 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000b5090 con 0x7f18000b44a0 2021-04-03T03:16:05.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806244700 1 --2- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000603f0 0x7f18000b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:16:05.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1806a45700 1 --2- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 0x7f18000b2b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f17f8003300 tx=0x7f17f8009270).ready entity=mon.1 client_cookie=95dc30d1fea3f7f7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1805242700 1 -- 172.21.2.111:0/2245614928 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17f8009e60 con 0x7f18000b44a0 2021-04-03T03:16:05.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1805242700 1 -- 172.21.2.111:0/2245614928 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f800bc00 con 0x7f18000b44a0 2021-04-03T03:16:05.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1805242700 1 -- 172.21.2.111:0/2245614928 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17f800bd70 con 0x7f18000b44a0 2021-04-03T03:16:05.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1807246700 1 -- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 msgr2=0x7f18000b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.148+0000 7f1807246700 1 --2- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 0x7f18000b2b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f17f8003300 tx=0x7f17f8009270).stop 2021-04-03T03:16:05.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 -- 172.21.2.111:0/2245614928 shutdown_connections 2021-04-03T03:16:05.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 --2- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 0x7f18000b2b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 --2- 172.21.2.111:0/2245614928 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000603f0 0x7f18000b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 -- 172.21.2.111:0/2245614928 >> 172.21.2.111:0/2245614928 conn(0x7f180005c7d0 msgr2=0x7f180005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 -- 172.21.2.111:0/2245614928 shutdown_connections 2021-04-03T03:16:05.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 -- 172.21.2.111:0/2245614928 wait complete. 2021-04-03T03:16:05.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 Processor -- start 2021-04-03T03:16:05.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.152+0000 7f1807246700 1 -- start start 2021-04-03T03:16:05.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1807246700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18000603f0 0x7f1800144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1807246700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 0x7f1800146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1807246700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000b5d60 0x7f1800146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1807246700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1800141680 con 0x7f18000603f0 2021-04-03T03:16:05.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1807246700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1800149b50 con 0x7f18000b44a0 2021-04-03T03:16:05.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1807246700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1800148be0 con 0x7f18000b5d60 2021-04-03T03:16:05.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18000603f0 0x7f1800144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18000603f0 0x7f1800144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56020/0 (socket says 172.21.2.111:56020) 2021-04-03T03:16:05.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 -- 172.21.2.111:0/2479087170 learned_addr learned my addr 172.21.2.111:0/2479087170 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806a45700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000b5d60 0x7f1800146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1805a43700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 0x7f1800146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 -- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000b5d60 msgr2=0x7f1800146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000b5d60 0x7f1800146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 -- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 msgr2=0x7f1800146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 0x7f1800146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 -- 172.21.2.111:0/2479087170 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1800147ff0 con 0x7f18000603f0 2021-04-03T03:16:05.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.156+0000 7f1806244700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18000603f0 0x7f1800144de0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f17fc00c000 tx=0x7f17fc006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.160+0000 7f17ef7fe700 1 -- 172.21.2.111:0/2479087170 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17fc00cb40 con 0x7f18000603f0 2021-04-03T03:16:05.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.160+0000 7f17ef7fe700 1 -- 172.21.2.111:0/2479087170 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc00cca0 con 0x7f18000603f0 2021-04-03T03:16:05.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.160+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1800147aa0 con 0x7f18000603f0 2021-04-03T03:16:05.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.160+0000 7f17ef7fe700 1 -- 172.21.2.111:0/2479087170 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17fc0123f0 con 0x7f18000603f0 2021-04-03T03:16:05.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.160+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1800146ca0 con 0x7f18000603f0 2021-04-03T03:16:05.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.160+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1800057660 con 0x7f18000603f0 2021-04-03T03:16:05.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.160+0000 7f17ef7fe700 1 -- 172.21.2.111:0/2479087170 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f17fc012550 con 0x7f18000603f0 2021-04-03T03:16:05.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.164+0000 7f17ef7fe700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f17e0056fe0 0x7f17e00594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.164+0000 7f17ef7fe700 1 -- 172.21.2.111:0/2479087170 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f17fc0708b0 con 0x7f18000603f0 2021-04-03T03:16:05.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.164+0000 7f1805a43700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f17e0056fe0 0x7f17e00594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.164+0000 7f1805a43700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f17e0056fe0 0x7f17e00594a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f17f000d000 tx=0x7f17f0003040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.168+0000 7f17ef7fe700 1 -- 172.21.2.111:0/2479087170 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f17fc045b30 con 0x7f18000603f0 2021-04-03T03:16:05.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.260+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1800069270 con 0x7f18000603f0 2021-04-03T03:16:05.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.260+0000 7f17ef7fe700 1 -- 172.21.2.111:0/2479087170 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16282 (secure 0 0 0) 0x7f17fc049150 con 0x7f18000603f0 2021-04-03T03:16:05.264 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:05.265 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":103,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:03.554720+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:00.429170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:01.518005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:05.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f17e0056fe0 msgr2=0x7f17e00594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f17e0056fe0 0x7f17e00594a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f17f000d000 tx=0x7f17f0003040).stop 2021-04-03T03:16:05.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18000603f0 msgr2=0x7f1800144de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18000603f0 0x7f1800144de0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f17fc00c000 tx=0x7f17fc006dd0).stop 2021-04-03T03:16:05.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 shutdown_connections 2021-04-03T03:16:05.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18000b5d60 0x7f1800146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18000603f0 0x7f1800144de0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f17e0056fe0 0x7f17e00594a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 --2- 172.21.2.111:0/2479087170 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18000b44a0 0x7f1800146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 >> 172.21.2.111:0/2479087170 conn(0x7f180005c7d0 msgr2=0x7f1800143bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 shutdown_connections 2021-04-03T03:16:05.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.264+0000 7f1807246700 1 -- 172.21.2.111:0/2479087170 wait complete. 2021-04-03T03:16:05.273 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:16:00.429170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:16:01.518005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:16:05.274 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:05.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 Processor -- start 2021-04-03T03:16:05.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- start start 2021-04-03T03:16:05.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180603f0 0x7fbe181447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbe180add50 0x7fbe18144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe180607d0 con 0x7fbe180b17b0 2021-04-03T03:16:05.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe181451c0 con 0x7fbe180add50 2021-04-03T03:16:05.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1814b090 con 0x7fbe180603f0 2021-04-03T03:16:05.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbe180add50 0x7fbe18144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbe180add50 0x7fbe18144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56032/0 (socket says 172.21.2.111:56032) 2021-04-03T03:16:05.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 -- 172.21.2.111:0/22042638 learned_addr learned my addr 172.21.2.111:0/22042638 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 -- 172.21.2.111:0/22042638 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180603f0 msgr2=0x7fbe181447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 --2- 172.21.2.111:0/22042638 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180603f0 0x7fbe181447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 -- 172.21.2.111:0/22042638 >> v1:172.21.2.115:6789/0 conn(0x7fbe180b17b0 legacy=0x7fbe18147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 -- 172.21.2.111:0/22042638 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1814b330 con 0x7fbe180add50 2021-04-03T03:16:05.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe15d9b700 1 --2- 172.21.2.111:0/22042638 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180add50 0x7fbe18144cd0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7fbe180b5600 tx=0x7fbe08006e90).ready entity=mon.0 client_cookie=b48a22305f6f79e6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1559a700 1 -- 172.21.2.111:0/22042638 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe080098e0 con 0x7fbe180add50 2021-04-03T03:16:05.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1559a700 1 -- 172.21.2.111:0/22042638 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe08009a40 con 0x7fbe180add50 2021-04-03T03:16:05.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1559a700 1 -- 172.21.2.111:0/22042638 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe08009de0 con 0x7fbe180add50 2021-04-03T03:16:05.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- 172.21.2.111:0/22042638 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180add50 msgr2=0x7fbe18144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 --2- 172.21.2.111:0/22042638 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180add50 0x7fbe18144cd0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7fbe180b5600 tx=0x7fbe08006e90).stop 2021-04-03T03:16:05.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- 172.21.2.111:0/22042638 shutdown_connections 2021-04-03T03:16:05.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 --2- 172.21.2.111:0/22042638 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180603f0 0x7fbe181447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 --2- 172.21.2.111:0/22042638 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180add50 0x7fbe18144cd0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- 172.21.2.111:0/22042638 >> 172.21.2.111:0/22042638 conn(0x7fbe180b61c0 msgr2=0x7fbe1805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.328+0000 7fbe1759e700 1 -- 172.21.2.111:0/22042638 shutdown_connections 2021-04-03T03:16:05.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- 172.21.2.111:0/22042638 wait complete. 2021-04-03T03:16:05.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 Processor -- start 2021-04-03T03:16:05.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- start start 2021-04-03T03:16:05.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe180603f0 0x7fbe180a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180add50 0x7fbe180abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180b17b0 0x7fbe180ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180a95b0 con 0x7fbe180b17b0 2021-04-03T03:16:05.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180a9850 con 0x7fbe180603f0 2021-04-03T03:16:05.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe18063a60 con 0x7fbe180add50 2021-04-03T03:16:05.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180b17b0 0x7fbe180ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180b17b0 0x7fbe180ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56038/0 (socket says 172.21.2.111:56038) 2021-04-03T03:16:05.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 -- 172.21.2.111:0/2216128533 learned_addr learned my addr 172.21.2.111:0/2216128533 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 -- 172.21.2.111:0/2216128533 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180add50 msgr2=0x7fbe180abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180add50 0x7fbe180abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 -- 172.21.2.111:0/2216128533 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe180603f0 msgr2=0x7fbe180a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe180603f0 0x7fbe180a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 -- 172.21.2.111:0/2216128533 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180ab570 con 0x7fbe180b17b0 2021-04-03T03:16:05.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe16d9d700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180b17b0 0x7fbe180ab080 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fbe10003250 tx=0x7fbe100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe0f7fe700 1 -- 172.21.2.111:0/2216128533 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe1000c900 con 0x7fbe180b17b0 2021-04-03T03:16:05.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe0f7fe700 1 -- 172.21.2.111:0/2216128533 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1000ca60 con 0x7fbe180b17b0 2021-04-03T03:16:05.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18063f00 con 0x7fbe180b17b0 2021-04-03T03:16:05.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe180643c0 con 0x7fbe180b17b0 2021-04-03T03:16:05.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe0f7fe700 1 -- 172.21.2.111:0/2216128533 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe1000cdc0 con 0x7fbe180b17b0 2021-04-03T03:16:05.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe180ac090 con 0x7fbe180b17b0 2021-04-03T03:16:05.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe0f7fe700 1 -- 172.21.2.111:0/2216128533 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbe10015470 con 0x7fbe180b17b0 2021-04-03T03:16:05.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe0f7fe700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdfc056f70 0x7fbdfc059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe0f7fe700 1 -- 172.21.2.111:0/2216128533 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fbe100706e0 con 0x7fbe180b17b0 2021-04-03T03:16:05.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.332+0000 7fbe1659c700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdfc056f70 0x7fbdfc059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.336+0000 7fbe1659c700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdfc056f70 0x7fbdfc059430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fbe00002f30 tx=0x7fbe00006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.336+0000 7fbe0f7fe700 1 -- 172.21.2.111:0/2216128533 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbe10045a70 con 0x7fbe180b17b0 2021-04-03T03:16:05.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbe18063c20 con 0x7fbe180b17b0 2021-04-03T03:16:05.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe0f7fe700 1 -- 172.21.2.111:0/2216128533 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v103) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbe10049090 con 0x7fbe180b17b0 2021-04-03T03:16:05.430 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:05.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdfc056f70 msgr2=0x7fbdfc059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdfc056f70 0x7fbdfc059430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fbe00002f30 tx=0x7fbe00006a60).stop 2021-04-03T03:16:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180b17b0 msgr2=0x7fbe180ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180b17b0 0x7fbe180ab080 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fbe10003250 tx=0x7fbe100090f0).stop 2021-04-03T03:16:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 shutdown_connections 2021-04-03T03:16:05.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbe180b17b0 0x7fbe180ab080 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdfc056f70 0x7fbdfc059430 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbe180603f0 0x7fbe180a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 --2- 172.21.2.111:0/2216128533 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbe180add50 0x7fbe180abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 >> 172.21.2.111:0/2216128533 conn(0x7fbe180b61c0 msgr2=0x7fbe180b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 shutdown_connections 2021-04-03T03:16:05.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.428+0000 7fbe1759e700 1 -- 172.21.2.111:0/2216128533 wait complete. 2021-04-03T03:16:05.437 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:05.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afffff700 1 Processor -- start 2021-04-03T03:16:05.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afffff700 1 -- start start 2021-04-03T03:16:05.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8b00144ef0 0x7f8b00147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b000607d0 con 0x7f8b000add50 2021-04-03T03:16:05.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0014b090 con 0x7f8b000603f0 2021-04-03T03:16:05.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0014b330 con 0x7f8b00144ef0 2021-04-03T03:16:05.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8b000add50 0x7f8b00147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56836/0 (socket says 172.21.2.111:56836) 2021-04-03T03:16:05.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 -- 172.21.2.111:0/2210301143 learned_addr learned my addr 172.21.2.111:0/2210301143 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:05.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581634805 0 0) 0x7f8b000607d0 con 0x7f8b000add50 2021-04-03T03:16:05.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae0003240 con 0x7f8b000add50 2021-04-03T03:16:05.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8aff7fe700 1 --2- 172.21.2.111:0/2210301143 >> v2:172.21.2.115:3300/0 conn(0x7f8b00144ef0 0x7f8b00147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 443646382 0 0) 0x7f8ae0003240 con 0x7f8b000add50 2021-04-03T03:16:05.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 >> v2:172.21.2.115:3300/0 conn(0x7f8b00144ef0 msgr2=0x7f8b00147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 --2- 172.21.2.111:0/2210301143 >> v2:172.21.2.115:3300/0 conn(0x7f8b00144ef0 0x7f8b00147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 >> v1:172.21.2.115:6789/0 conn(0x7f8b000603f0 legacy=0x7f8b001447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0014b5d0 con 0x7f8b000add50 2021-04-03T03:16:05.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8af00016d0 con 0x7f8b000add50 2021-04-03T03:16:05.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 >> v1:172.21.2.111:6789/0 conn(0x7f8b000add50 legacy=0x7f8b00147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ae0003800 0x7f8ae0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8ae0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ae0004e50 0x7f8ae000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0014b330 con 0x7f8ae00044e0 2021-04-03T03:16:05.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0005300 con 0x7f8ae0004e50 2021-04-03T03:16:05.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae000bc20 con 0x7f8ae0003800 2021-04-03T03:16:05.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8ae0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 -- 172.21.2.111:0/2210301143 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ae0003800 msgr2=0x7f8ae0003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ae0003800 0x7f8ae0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 -- 172.21.2.111:0/2210301143 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ae0004e50 msgr2=0x7f8ae000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ae0004e50 0x7f8ae000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 -- 172.21.2.111:0/2210301143 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0003240 con 0x7f8ae00044e0 2021-04-03T03:16:05.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8aff7fe700 1 -- 172.21.2.111:0/2210301143 reap_dead start 2021-04-03T03:16:05.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afe7fc700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8ae0004960 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f8b000b5600 tx=0x7f8b0014b220).ready entity=mon.0 client_cookie=d3669d961e50b8b1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af000ce80 con 0x7f8ae00044e0 2021-04-03T03:16:05.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af00070d0 con 0x7f8ae00044e0 2021-04-03T03:16:05.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.492+0000 7f8afdffb700 1 -- 172.21.2.111:0/2210301143 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af0007430 con 0x7f8ae00044e0 2021-04-03T03:16:05.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2210301143 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 msgr2=0x7f8ae0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8ae0004960 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f8b000b5600 tx=0x7f8b0014b220).stop 2021-04-03T03:16:05.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2210301143 shutdown_connections 2021-04-03T03:16:05.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 --2- 172.21.2.111:0/2210301143 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8ae0004960 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2210301143 >> 172.21.2.111:0/2210301143 conn(0x7f8b000b61c0 msgr2=0x7f8b0005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2210301143 shutdown_connections 2021-04-03T03:16:05.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2210301143 wait complete. 2021-04-03T03:16:05.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 Processor -- start 2021-04-03T03:16:05.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- start start 2021-04-03T03:16:05.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8b00159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b001477d0 0x7f8b000a7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b0014a2e0 0x7f8b000a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000aa3c0 con 0x7f8ae00044e0 2021-04-03T03:16:05.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000aa120 con 0x7f8b0014a2e0 2021-04-03T03:16:05.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000a78f0 con 0x7f8b001477d0 2021-04-03T03:16:05.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8b00159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8b00159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56054/0 (socket says 172.21.2.111:56054) 2021-04-03T03:16:05.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 -- 172.21.2.111:0/2132250832 learned_addr learned my addr 172.21.2.111:0/2132250832 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 -- 172.21.2.111:0/2132250832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b001477d0 msgr2=0x7f8b000a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b001477d0 0x7f8b000a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 -- 172.21.2.111:0/2132250832 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b0014a2e0 msgr2=0x7f8b000a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b0014a2e0 0x7f8b000a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 -- 172.21.2.111:0/2132250832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00147460 con 0x7f8ae00044e0 2021-04-03T03:16:05.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afeffd700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8b00159c20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f8ae8001f60 tx=0x7f8ae8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8af7fff700 1 -- 172.21.2.111:0/2132250832 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ae800e4a0 con 0x7f8ae00044e0 2021-04-03T03:16:05.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8af7fff700 1 -- 172.21.2.111:0/2132250832 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae800e600 con 0x7f8ae00044e0 2021-04-03T03:16:05.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b000a7c30 con 0x7f8ae00044e0 2021-04-03T03:16:05.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b001483b0 con 0x7f8ae00044e0 2021-04-03T03:16:05.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8af7fff700 1 -- 172.21.2.111:0/2132250832 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ae800e9a0 con 0x7f8ae00044e0 2021-04-03T03:16:05.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad80033f0 con 0x7f8ae00044e0 2021-04-03T03:16:05.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8af7fff700 1 -- 172.21.2.111:0/2132250832 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8ae8010070 con 0x7f8ae00044e0 2021-04-03T03:16:05.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8af7fff700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ae4057030 0x7f8ae40594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8af7fff700 1 -- 172.21.2.111:0/2132250832 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8ae806f930 con 0x7f8ae00044e0 2021-04-03T03:16:05.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afe7fc700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ae4057030 0x7f8ae40594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.496+0000 7f8afe7fc700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ae4057030 0x7f8ae40594f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8b000b5600 tx=0x7f8af0013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.500+0000 7f8af7fff700 1 -- 172.21.2.111:0/2132250832 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8ae8044c10 con 0x7f8ae00044e0 2021-04-03T03:16:05.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.592+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8ad8003c20 con 0x7f8ae00044e0 2021-04-03T03:16:05.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.592+0000 7f8af7fff700 1 -- 172.21.2.111:0/2132250832 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8ae8017210 con 0x7f8ae00044e0 2021-04-03T03:16:05.597 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:05.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ae4057030 msgr2=0x7f8ae40594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ae4057030 0x7f8ae40594f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8b000b5600 tx=0x7f8af0013000).stop 2021-04-03T03:16:05.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 msgr2=0x7f8b00159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8b00159c20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f8ae8001f60 tx=0x7f8ae8007c10).stop 2021-04-03T03:16:05.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 shutdown_connections 2021-04-03T03:16:05.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8ae4057030 0x7f8ae40594f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b0014a2e0 0x7f8b000a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ae00044e0 0x7f8b00159c20 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 --2- 172.21.2.111:0/2132250832 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b001477d0 0x7f8b000a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 >> 172.21.2.111:0/2132250832 conn(0x7f8b000b61c0 msgr2=0x7f8b000b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 shutdown_connections 2021-04-03T03:16:05.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.596+0000 7f8afffff700 1 -- 172.21.2.111:0/2132250832 wait complete. 2021-04-03T03:16:05.604 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:16:05.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 Processor -- start 2021-04-03T03:16:05.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- start start 2021-04-03T03:16:05.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f61680dbba0 0x7f61680dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6168041060 0x7f616803fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f616803d560 0x7f6168041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616803de60 con 0x7f61680dbba0 2021-04-03T03:16:05.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61680dc560 con 0x7f6168041060 2021-04-03T03:16:05.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680dc800 con 0x7f616803d560 2021-04-03T03:16:05.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f61680dbba0 0x7f61680dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f61680dbba0 0x7f61680dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56062/0 (socket says 172.21.2.111:56062) 2021-04-03T03:16:05.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 -- 172.21.2.111:0/3141854495 learned_addr learned my addr 172.21.2.111:0/3141854495 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 -- 172.21.2.111:0/3141854495 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f616803d560 msgr2=0x7f6168041650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:05.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 --2- 172.21.2.111:0/3141854495 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f616803d560 0x7f6168041650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 -- 172.21.2.111:0/3141854495 >> v2:172.21.2.115:3300/0 conn(0x7f6168041060 msgr2=0x7f616803fa80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:05.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 --2- 172.21.2.111:0/3141854495 >> v2:172.21.2.115:3300/0 conn(0x7f6168041060 0x7f616803fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 -- 172.21.2.111:0/3141854495 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616803da30 con 0x7f61680dbba0 2021-04-03T03:16:05.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616f7fe700 1 --2- 172.21.2.111:0/3141854495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61680dbba0 0x7f61680dbf80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f6160002f30 tx=0x7f6160006e90).ready entity=mon.0 client_cookie=c172161ecc4f573b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616e7fc700 1 -- 172.21.2.111:0/3141854495 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6160009920 con 0x7f61680dbba0 2021-04-03T03:16:05.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616e7fc700 1 -- 172.21.2.111:0/3141854495 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160009a80 con 0x7f61680dbba0 2021-04-03T03:16:05.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f616e7fc700 1 -- 172.21.2.111:0/3141854495 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6160009de0 con 0x7f61680dbba0 2021-04-03T03:16:05.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- 172.21.2.111:0/3141854495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61680dbba0 msgr2=0x7f61680dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 --2- 172.21.2.111:0/3141854495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61680dbba0 0x7f61680dbf80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f6160002f30 tx=0x7f6160006e90).stop 2021-04-03T03:16:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- 172.21.2.111:0/3141854495 shutdown_connections 2021-04-03T03:16:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 --2- 172.21.2.111:0/3141854495 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f616803d560 0x7f6168041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 --2- 172.21.2.111:0/3141854495 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f61680dbba0 0x7f61680dbf80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 --2- 172.21.2.111:0/3141854495 >> v2:172.21.2.115:3300/0 conn(0x7f6168041060 0x7f616803fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- 172.21.2.111:0/3141854495 >> 172.21.2.111:0/3141854495 conn(0x7f61680462b0 msgr2=0x7f61680466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- 172.21.2.111:0/3141854495 shutdown_connections 2021-04-03T03:16:05.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.660+0000 7f6174a98700 1 -- 172.21.2.111:0/3141854495 wait complete. 2021-04-03T03:16:05.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 Processor -- start 2021-04-03T03:16:05.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 -- start start 2021-04-03T03:16:05.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f616803d560 0x7f61680e7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6168041060 0x7f61680daa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61680dbba0 0x7f61680d7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680a0490 con 0x7f616803d560 2021-04-03T03:16:05.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616809fa00 con 0x7f61680dbba0 2021-04-03T03:16:05.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616809fca0 con 0x7f6168041060 2021-04-03T03:16:05.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f616803d560 0x7f61680e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f616803d560 0x7f61680e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56070/0 (socket says 172.21.2.111:56070) 2021-04-03T03:16:05.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 -- 172.21.2.111:0/1540975096 learned_addr learned my addr 172.21.2.111:0/1540975096 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 -- 172.21.2.111:0/1540975096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6168041060 msgr2=0x7f61680daa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6168041060 0x7f61680daa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 -- 172.21.2.111:0/1540975096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61680dbba0 msgr2=0x7f61680d7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61680dbba0 0x7f61680d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 -- 172.21.2.111:0/1540975096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168039580 con 0x7f616803d560 2021-04-03T03:16:05.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616f7fe700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f616803d560 0x7f61680e7220 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f6160002f30 tx=0x7f6160015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616cff9700 1 -- 172.21.2.111:0/1540975096 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6160011db0 con 0x7f616803d560 2021-04-03T03:16:05.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616cff9700 1 -- 172.21.2.111:0/1540975096 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616001e460 con 0x7f616803d560 2021-04-03T03:16:05.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616cff9700 1 -- 172.21.2.111:0/1540975096 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f616001e7c0 con 0x7f616803d560 2021-04-03T03:16:05.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616803ade0 con 0x7f616803d560 2021-04-03T03:16:05.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616809f5d0 con 0x7f616803d560 2021-04-03T03:16:05.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616809f850 con 0x7f616803d560 2021-04-03T03:16:05.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616cff9700 1 -- 172.21.2.111:0/1540975096 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f616001a040 con 0x7f616803d560 2021-04-03T03:16:05.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616cff9700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6150057050 0x7f6150059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616cff9700 1 -- 172.21.2.111:0/1540975096 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f6160071380 con 0x7f616803d560 2021-04-03T03:16:05.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616effd700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6150057050 0x7f6150059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.664+0000 7f616effd700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6150057050 0x7f6150059510 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f6164002f70 tx=0x7f6164006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.668+0000 7f616cff9700 1 -- 172.21.2.111:0/1540975096 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6160046770 con 0x7f616803d560 2021-04-03T03:16:05.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f616803b9c0 con 0x7f616803d560 2021-04-03T03:16:05.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f616cff9700 1 -- 172.21.2.111:0/1540975096 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6160049d90 con 0x7f616803d560 2021-04-03T03:16:05.764 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:05.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6150057050 msgr2=0x7f6150059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6150057050 0x7f6150059510 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f6164002f70 tx=0x7f6164006990).stop 2021-04-03T03:16:05.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f616803d560 msgr2=0x7f61680e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f616803d560 0x7f61680e7220 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f6160002f30 tx=0x7f6160015000).stop 2021-04-03T03:16:05.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 shutdown_connections 2021-04-03T03:16:05.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6150057050 0x7f6150059510 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f61680dbba0 0x7f61680d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f616803d560 0x7f61680e7220 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 --2- 172.21.2.111:0/1540975096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6168041060 0x7f61680daa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 >> 172.21.2.111:0/1540975096 conn(0x7f61680462b0 msgr2=0x7f6168045720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 shutdown_connections 2021-04-03T03:16:05.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.760+0000 7f6174a98700 1 -- 172.21.2.111:0/1540975096 wait complete. 2021-04-03T03:16:05.771 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:16:05.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.824+0000 7efceeecd700 1 Processor -- start 2021-04-03T03:16:05.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.824+0000 7efceeecd700 1 -- start start 2021-04-03T03:16:05.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7efce0042080 0x7efce003ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce0042850 0x7efce003d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce0039580 con 0x7efce00cdcf0 2021-04-03T03:16:05.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce0042460 con 0x7efce0042080 2021-04-03T03:16:05.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00d19c0 con 0x7efce0042850 2021-04-03T03:16:05.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcee6cc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7efce00cdcf0 0x7efce00d4190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56872/0 (socket says 172.21.2.111:56872) 2021-04-03T03:16:05.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcee6cc700 1 -- 172.21.2.111:0/3081301344 learned_addr learned my addr 172.21.2.111:0/3081301344 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:05.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89171244 0 0) 0x7efce0039580 con 0x7efce00cdcf0 2021-04-03T03:16:05.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd0003240 con 0x7efce00cdcf0 2021-04-03T03:16:05.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3922697016 0 0) 0x7efcd0003240 con 0x7efce00cdcf0 2021-04-03T03:16:05.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce0042850 msgr2=0x7efce003d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce0042850 0x7efce003d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 >> v2:172.21.2.115:3300/0 conn(0x7efce0042080 msgr2=0x7efce003ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 --2- 172.21.2.111:0/3081301344 >> v2:172.21.2.115:3300/0 conn(0x7efce0042080 0x7efce003ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00397b0 con 0x7efce00cdcf0 2021-04-03T03:16:05.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7efce40016d0 con 0x7efce00cdcf0 2021-04-03T03:16:05.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 >> v1:172.21.2.111:6789/0 conn(0x7efce00cdcf0 legacy=0x7efce00d4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efcd00037b0 0x7efcd0003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efcd00044e0 0x7efcd0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce0042850 0x7efcd0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0042460 con 0x7efcd00044e0 2021-04-03T03:16:05.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00d19c0 con 0x7efcd00037b0 2021-04-03T03:16:05.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd0005340 con 0x7efce0042850 2021-04-03T03:16:05.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efced6ca700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efcd00044e0 0x7efcd0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efced6ca700 1 -- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce0042850 msgr2=0x7efcd0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efced6ca700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce0042850 0x7efcd0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efced6ca700 1 -- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efcd00037b0 msgr2=0x7efcd0003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:05.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efced6ca700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efcd00037b0 0x7efcd0003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efced6ca700 1 -- 172.21.2.111:0/3081301344 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0003240 con 0x7efcd00044e0 2021-04-03T03:16:05.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efced6ca700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efcd00044e0 0x7efcd0004960 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7efce806b5d0 tx=0x7efce805ad70).ready entity=mon.0 client_cookie=58901dab58d8b784 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce805df60 con 0x7efcd00044e0 2021-04-03T03:16:05.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce80699f0 con 0x7efcd00044e0 2021-04-03T03:16:05.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcecec9700 1 -- 172.21.2.111:0/3081301344 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce805b040 con 0x7efcd00044e0 2021-04-03T03:16:05.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- 172.21.2.111:0/3081301344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efcd00044e0 msgr2=0x7efcd0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 --2- 172.21.2.111:0/3081301344 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efcd00044e0 0x7efcd0004960 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7efce806b5d0 tx=0x7efce805ad70).stop 2021-04-03T03:16:05.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcee6cc700 1 -- 172.21.2.111:0/3081301344 reap_dead start 2021-04-03T03:16:05.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- 172.21.2.111:0/3081301344 shutdown_connections 2021-04-03T03:16:05.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- 172.21.2.111:0/3081301344 >> 172.21.2.111:0/3081301344 conn(0x7efce00462b0 msgr2=0x7efce00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- 172.21.2.111:0/3081301344 shutdown_connections 2021-04-03T03:16:05.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- 172.21.2.111:0/3081301344 wait complete. 2021-04-03T03:16:05.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 Processor -- start 2021-04-03T03:16:05.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- start start 2021-04-03T03:16:05.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efce00d23b0 0x7efce00df340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce00b7980 0x7efce00ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efce00ceeb0 0x7efce00cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00cf850 con 0x7efce00ceeb0 2021-04-03T03:16:05.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00df830 con 0x7efce00d23b0 2021-04-03T03:16:05.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efceeecd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00d3080 con 0x7efce00b7980 2021-04-03T03:16:05.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcee6cc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efce00ceeb0 0x7efce00cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcee6cc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efce00ceeb0 0x7efce00cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56092/0 (socket says 172.21.2.111:56092) 2021-04-03T03:16:05.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.828+0000 7efcee6cc700 1 -- 172.21.2.111:0/600172259 learned_addr learned my addr 172.21.2.111:0/600172259 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:05.846 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcee6cc700 1 -- 172.21.2.111:0/600172259 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce00b7980 msgr2=0x7efce00ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcee6cc700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce00b7980 0x7efce00ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcee6cc700 1 -- 172.21.2.111:0/600172259 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efce00d23b0 msgr2=0x7efce00df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcee6cc700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efce00d23b0 0x7efce00df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcee6cc700 1 -- 172.21.2.111:0/600172259 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00d2790 con 0x7efce00ceeb0 2021-04-03T03:16:05.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcee6cc700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efce00ceeb0 0x7efce00cf360 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7efce00df970 tx=0x7efce4001710).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcd6ffd700 1 -- 172.21.2.111:0/600172259 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce40070d0 con 0x7efce00ceeb0 2021-04-03T03:16:05.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce00ce280 con 0x7efce00ceeb0 2021-04-03T03:16:05.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcd6ffd700 1 -- 172.21.2.111:0/600172259 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce4007230 con 0x7efce00ceeb0 2021-04-03T03:16:05.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcd6ffd700 1 -- 172.21.2.111:0/600172259 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce40047d0 con 0x7efce00ceeb0 2021-04-03T03:16:05.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce00e1da0 con 0x7efce00ceeb0 2021-04-03T03:16:05.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce00df9b0 con 0x7efce00ceeb0 2021-04-03T03:16:05.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcd6ffd700 1 -- 172.21.2.111:0/600172259 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7efce401c070 con 0x7efce00ceeb0 2021-04-03T03:16:05.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcd6ffd700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efcc8057000 0x7efcc80594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcd6ffd700 1 -- 172.21.2.111:0/600172259 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7efce406f2e0 con 0x7efce00ceeb0 2021-04-03T03:16:05.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcedecb700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efcc8057000 0x7efcc80594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.832+0000 7efcedecb700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efcc8057000 0x7efcc80594c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7efcd8002f30 tx=0x7efcd8006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:05.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.836+0000 7efcd6ffd700 1 -- 172.21.2.111:0/600172259 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7efce40445c0 con 0x7efce00ceeb0 2021-04-03T03:16:05.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efce00ce6b0 con 0x7efce00ceeb0 2021-04-03T03:16:05.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efcd6ffd700 1 -- 172.21.2.111:0/600172259 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+10 (secure 0 0 0) 0x7efce4047be0 con 0x7efce00ceeb0 2021-04-03T03:16:05.930 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:05.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efcc8057000 msgr2=0x7efcc80594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efcc8057000 0x7efcc80594c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7efcd8002f30 tx=0x7efcd8006a60).stop 2021-04-03T03:16:05.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efce00ceeb0 msgr2=0x7efce00cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:05.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efce00ceeb0 0x7efce00cf360 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7efce00df970 tx=0x7efce4001710).stop 2021-04-03T03:16:05.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 shutdown_connections 2021-04-03T03:16:05.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efce00ceeb0 0x7efce00cf360 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efcc8057000 0x7efcc80594c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:05.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efce00d23b0 0x7efce00df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 --2- 172.21.2.111:0/600172259 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efce00b7980 0x7efce00ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:05.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 >> 172.21.2.111:0/600172259 conn(0x7efce00462b0 msgr2=0x7efce00055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:05.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 shutdown_connections 2021-04-03T03:16:05.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.928+0000 7efceeecd700 1 -- 172.21.2.111:0/600172259 wait complete. 2021-04-03T03:16:05.938 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:16:05.939 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:05.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080ffff700 1 Processor -- start 2021-04-03T03:16:05.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080ffff700 1 -- start start 2021-04-03T03:16:05.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080ffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f08100603f0 0x7f08100b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f08100b0c90 0x7f0810147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:05.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080ffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100af400 con 0x7f081014b700 2021-04-03T03:16:05.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0810060860 con 0x7f08100b0c90 2021-04-03T03:16:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080ffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100af6a0 con 0x7f08100603f0 2021-04-03T03:16:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080e7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f081014b700 0x7f08100b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56890/0 (socket says 172.21.2.111:56890) 2021-04-03T03:16:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080e7fc700 1 -- 172.21.2.111:0/1287131736 learned_addr learned my addr 172.21.2.111:0/1287131736 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080f7fe700 1 --2- 172.21.2.111:0/1287131736 >> v2:172.21.2.111:3300/0 conn(0x7f08100b0c90 0x7f0810147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:05.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080f7fe700 1 -- 172.21.2.111:0/1287131736 >> v2:172.21.2.115:3300/0 conn(0x7f08100603f0 msgr2=0x7f08100b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080f7fe700 1 --2- 172.21.2.111:0/1287131736 >> v2:172.21.2.115:3300/0 conn(0x7f08100603f0 0x7f08100b0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080f7fe700 1 -- 172.21.2.111:0/1287131736 >> v1:172.21.2.111:6789/0 conn(0x7f081014b700 legacy=0x7f08100b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080f7fe700 1 -- 172.21.2.111:0/1287131736 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08100afb40 con 0x7f08100b0c90 2021-04-03T03:16:06.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080dffb700 1 -- 172.21.2.111:0/1287131736 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956587133 0 0) 0x7f08100af400 con 0x7f081014b700 2021-04-03T03:16:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080f7fe700 1 --2- 172.21.2.111:0/1287131736 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100b0c90 0x7f0810147680 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f08080032b0 tx=0x7f08080091d0).ready entity=mon.0 client_cookie=1484e1466db08894 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.992+0000 7f080dffb700 1 -- 172.21.2.111:0/1287131736 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080800ba50 con 0x7f08100b0c90 2021-04-03T03:16:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080dffb700 1 -- 172.21.2.111:0/1287131736 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080800bbb0 con 0x7f08100b0c90 2021-04-03T03:16:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080dffb700 1 -- 172.21.2.111:0/1287131736 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080800bd20 con 0x7f08100b0c90 2021-04-03T03:16:06.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/1287131736 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100b0c90 msgr2=0x7f0810147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 --2- 172.21.2.111:0/1287131736 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100b0c90 0x7f0810147680 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f08080032b0 tx=0x7f08080091d0).stop 2021-04-03T03:16:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/1287131736 shutdown_connections 2021-04-03T03:16:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 --2- 172.21.2.111:0/1287131736 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100b0c90 0x7f0810147680 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 --2- 172.21.2.111:0/1287131736 >> v2:172.21.2.115:3300/0 conn(0x7f08100603f0 0x7f08100b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:06.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/1287131736 >> 172.21.2.111:0/1287131736 conn(0x7f08100b61c0 msgr2=0x7f081005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:06.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/1287131736 shutdown_connections 2021-04-03T03:16:06.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/1287131736 wait complete. 2021-04-03T03:16:06.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 Processor -- start 2021-04-03T03:16:06.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- start start 2021-04-03T03:16:06.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100603f0 0x7f0810062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:06.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f08100b0c90 0x7f0810159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:06.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f081014b700 0x7f08100a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:06.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100ab1d0 con 0x7f08100603f0 2021-04-03T03:16:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100a8360 con 0x7f081014b700 2021-04-03T03:16:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08100a8600 con 0x7f08100b0c90 2021-04-03T03:16:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100603f0 0x7f0810062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100603f0 0x7f0810062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56100/0 (socket says 172.21.2.111:56100) 2021-04-03T03:16:06.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 -- 172.21.2.111:0/3910388378 learned_addr learned my addr 172.21.2.111:0/3910388378 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:06.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 -- 172.21.2.111:0/3910388378 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f08100b0c90 msgr2=0x7f0810159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:06.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f08100b0c90 0x7f0810159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:06.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 -- 172.21.2.111:0/3910388378 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f081014b700 msgr2=0x7f08100a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:06.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f081014b700 0x7f08100a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 -- 172.21.2.111:0/3910388378 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810149990 con 0x7f08100603f0 2021-04-03T03:16:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080effd700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100603f0 0x7f0810062bf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f07f8002f70 tx=0x7f07f8006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f0807fff700 1 -- 172.21.2.111:0/3910388378 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f800d570 con 0x7f08100603f0 2021-04-03T03:16:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f0807fff700 1 -- 172.21.2.111:0/3910388378 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f800d6d0 con 0x7f08100603f0 2021-04-03T03:16:06.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08100a88a0 con 0x7f08100603f0 2021-04-03T03:16:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08100a99f0 con 0x7f08100603f0 2021-04-03T03:16:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f0807fff700 1 -- 172.21.2.111:0/3910388378 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f800da30 con 0x7f08100603f0 2021-04-03T03:16:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0810057660 con 0x7f08100603f0 2021-04-03T03:16:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f0807fff700 1 -- 172.21.2.111:0/3910388378 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f07f800f070 con 0x7f08100603f0 2021-04-03T03:16:06.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f0807fff700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f07f4056f70 0x7f07f4059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f0807fff700 1 -- 172.21.2.111:0/3910388378 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f07f806fe10 con 0x7f08100603f0 2021-04-03T03:16:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:05.996+0000 7f080e7fc700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f07f4056f70 0x7f07f4059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.000+0000 7f080e7fc700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f07f4056f70 0x7f07f4059430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f0800000b80 tx=0x7f0800007860).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:06.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.000+0000 7f0807fff700 1 -- 172.21.2.111:0/3910388378 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f07f80450f0 con 0x7f08100603f0 2021-04-03T03:16:06.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08100a9c40 con 0x7f08100603f0 2021-04-03T03:16:06.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f0807fff700 1 -- 172.21.2.111:0/3910388378 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f07f8048710 con 0x7f08100603f0 2021-04-03T03:16:06.126 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:06.127 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/3373357469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3373357469},{"type":"v1","addr":"172.21.2.115:6833","nonce":3373357469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:02.550193+0000","modified":"2021-04-03T03:16:03.553657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/3012485417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":3012485417},{"type":"v1","addr":"172.21.2.111:6835","nonce":3012485417}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-04-03T03:16:06.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f07f4056f70 msgr2=0x7f07f4059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:06.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f07f4056f70 0x7f07f4059430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f0800000b80 tx=0x7f0800007860).stop 2021-04-03T03:16:06.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100603f0 msgr2=0x7f0810062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:06.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100603f0 0x7f0810062bf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f07f8002f70 tx=0x7f07f8006dd0).stop 2021-04-03T03:16:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 shutdown_connections 2021-04-03T03:16:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f07f4056f70 0x7f07f4059430 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f081014b700 0x7f08100a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f08100603f0 0x7f0810062bf0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 --2- 172.21.2.111:0/3910388378 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f08100b0c90 0x7f0810159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 >> 172.21.2.111:0/3910388378 conn(0x7f08100b61c0 msgr2=0x7f08100b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 shutdown_connections 2021-04-03T03:16:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:06.124+0000 7f080ffff700 1 -- 172.21.2.111:0/3910388378 wait complete. 2021-04-03T03:16:06.130 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 43 2021-04-03T03:16:06.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:02.550193+0000', 'modified': '2021-04-03T03:16:03.553657+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7179': {'gid': 7179, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/3012485417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 3012485417}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 3012485417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:06.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:06.336 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:06.422 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7179,"want_state":"up:active","state":"up:active","rank_uptime":3.8649474270000002,"mdsmap_epoch":43,"osdmap_epoch":103,"osdmap_epoch_barrier":102,"uptime":8.1032099029999998} 2021-04-03T03:16:06.422 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7179, "mdsmap_epoch": 43, "osdmap_epoch": 103, "osdmap_epoch_barrier": 102, "rank_uptime": 3.864947427, "state": "up:active", "uptime": 8.103209903, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:06.422 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:16:06.423 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:16:06.423 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:16:06.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:06.424 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:06.434 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:16:06.486 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:06.487 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:06.535 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:06.536 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:06.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:06.537 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:16:06.588 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:16:06.588 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:06.634 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:16:06.641 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:06.641 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:06.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:16:06.668+0000 7ff64434d0c0 -1 init, newargv = 0x5585134b16a0 newargc=9 2021-04-03T03:16:06.674 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15791]: starting ceph client 2021-04-03T03:16:06.678 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:06.679 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:06.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:06.680 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:16:06.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15791]: starting fuse 2021-04-03T03:16:06.692 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:16:06.692 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:16:06.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:06.693 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:16:06.694 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:16:06.694 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:16:06.695 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:16:06.695 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:16:06.695 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:06.696 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:16:06.696 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:16:06.697 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:16:06.697 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:06.697 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:16:06.698 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:16:06.698 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:16:06.699 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:16:06.699 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:06.699 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:16:06.700 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:16:06.700 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:16:06.701 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:16:06.701 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:16:06.701 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:06.702 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:16:06.702 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:16:06.779 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15791.asok 2021-04-03T03:16:06.782 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15791.asok 2021-04-03T03:16:06.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:06.783 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15791.asok status 2021-04-03T03:16:06.909 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:16:06.909 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:16:06.910 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:16:06.910 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:16:06.910 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:16:06.910 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:16:06.910 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:16:06.911 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15791", 2021-04-03T03:16:06.911 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:16:06.911 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:06.911 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:16:06.912 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:16:06.912 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 7353, 2021-04-03T03:16:06.912 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:16:06.912 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:16:06.912 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:16:06.913 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 7353 2021-04-03T03:16:06.913 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:06.913 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:16:06.913 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:16:06.913 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:16:06.914 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2888145897 2021-04-03T03:16:06.914 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:16:06.914 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:06.914 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:16:06.915 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:16:06.915 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:16:06.915 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2888145897 2021-04-03T03:16:06.915 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:06.915 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.7353 172.21.2.126:0/2888145897", 2021-04-03T03:16:06.916 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/2888145897", 2021-04-03T03:16:06.916 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:16:06.916 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 43, 2021-04-03T03:16:06.916 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 103, 2021-04-03T03:16:06.916 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:16:06.917 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:16:06.917 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:16:06.917 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:16:06.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:06.919 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:06.919 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:06.926 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:16:06.928 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:06.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:06.928 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:06.988 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:07.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.044+0000 7f928b0c2700 1 Processor -- start 2021-04-03T03:16:07.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.044+0000 7f928b0c2700 1 -- start start 2021-04-03T03:16:07.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f927c00bca0 0x7f927c0090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f927c007c30 0x7f927c043780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c042f10 con 0x7f927c005b40 2021-04-03T03:16:07.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c02b020 con 0x7f927c00bca0 2021-04-03T03:16:07.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c00c080 con 0x7f927c007c30 2021-04-03T03:16:07.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a0c0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f927c00bca0 0x7f927c0090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92898bf700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f927c005b40 0x7f927c007520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56900/0 (socket says 172.21.2.111:56900) 2021-04-03T03:16:07.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92898bf700 1 -- 172.21.2.111:0/1169149663 learned_addr learned my addr 172.21.2.111:0/1169149663 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:07.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 --2- 172.21.2.111:0/1169149663 >> v2:172.21.2.115:3300/0 conn(0x7f927c007c30 0x7f927c043780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a0c0700 1 -- 172.21.2.111:0/1169149663 >> v2:172.21.2.115:3300/0 conn(0x7f927c007c30 msgr2=0x7f927c043780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a0c0700 1 --2- 172.21.2.111:0/1169149663 >> v2:172.21.2.115:3300/0 conn(0x7f927c007c30 0x7f927c043780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a0c0700 1 -- 172.21.2.111:0/1169149663 >> v1:172.21.2.111:6789/0 conn(0x7f927c005b40 legacy=0x7f927c007520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a0c0700 1 -- 172.21.2.111:0/1169149663 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c043170 con 0x7f927c00bca0 2021-04-03T03:16:07.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92890be700 1 -- 172.21.2.111:0/1169149663 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525793924 0 0) 0x7f927c042f10 con 0x7f927c005b40 2021-04-03T03:16:07.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a0c0700 1 --2- 172.21.2.111:0/1169149663 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0090c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f9280002f30 tx=0x7f9280006dd0).ready entity=mon.0 client_cookie=dc88358e72cad964 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92890be700 1 -- 172.21.2.111:0/1169149663 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92800097f0 con 0x7f927c00bca0 2021-04-03T03:16:07.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92890be700 1 -- 172.21.2.111:0/1169149663 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9280009950 con 0x7f927c00bca0 2021-04-03T03:16:07.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92890be700 1 -- 172.21.2.111:0/1169149663 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9280009cb0 con 0x7f927c00bca0 2021-04-03T03:16:07.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- 172.21.2.111:0/1169149663 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 msgr2=0x7f927c0090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- 172.21.2.111:0/1169149663 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0090c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f9280002f30 tx=0x7f9280006dd0).stop 2021-04-03T03:16:07.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- 172.21.2.111:0/1169149663 shutdown_connections 2021-04-03T03:16:07.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- 172.21.2.111:0/1169149663 >> v2:172.21.2.115:3300/0 conn(0x7f927c007c30 0x7f927c043780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- 172.21.2.111:0/1169149663 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0090c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- 172.21.2.111:0/1169149663 >> 172.21.2.111:0/1169149663 conn(0x7f927c0484d0 msgr2=0x7f927c0488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- 172.21.2.111:0/1169149663 shutdown_connections 2021-04-03T03:16:07.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- 172.21.2.111:0/1169149663 wait complete. 2021-04-03T03:16:07.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 Processor -- start 2021-04-03T03:16:07.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- start start 2021-04-03T03:16:07.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f927c005b40 0x7f927c0e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f927c007c30 0x7f927c02b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0e7160 con 0x7f927c00bca0 2021-04-03T03:16:07.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0d4850 con 0x7f927c005b40 2021-04-03T03:16:07.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0d45b0 con 0x7f927c007c30 2021-04-03T03:16:07.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0e6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56118/0 (socket says 172.21.2.111:56118) 2021-04-03T03:16:07.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 -- 172.21.2.111:0/3159484646 learned_addr learned my addr 172.21.2.111:0/3159484646 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 -- 172.21.2.111:0/3159484646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f927c007c30 msgr2=0x7f927c02b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f927c007c30 0x7f927c02b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 -- 172.21.2.111:0/3159484646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f927c005b40 msgr2=0x7f927c0e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f927c005b40 0x7f927c0e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 -- 172.21.2.111:0/3159484646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c02c660 con 0x7f927c00bca0 2021-04-03T03:16:07.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928a8c1700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0e6c70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f9284066000 tx=0x7f928405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92777fe700 1 -- 172.21.2.111:0/3159484646 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92840699f0 con 0x7f927c00bca0 2021-04-03T03:16:07.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92777fe700 1 -- 172.21.2.111:0/3159484646 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928405f2c0 con 0x7f927c00bca0 2021-04-03T03:16:07.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f92777fe700 1 -- 172.21.2.111:0/3159484646 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f928405f620 con 0x7f927c00bca0 2021-04-03T03:16:07.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c0d4030 con 0x7f927c00bca0 2021-04-03T03:16:07.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.048+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c02cb00 con 0x7f927c00bca0 2021-04-03T03:16:07.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.052+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c02c7f0 con 0x7f927c00bca0 2021-04-03T03:16:07.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.052+0000 7f92777fe700 1 -- 172.21.2.111:0/3159484646 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f928406a2d0 con 0x7f927c00bca0 2021-04-03T03:16:07.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.052+0000 7f92777fe700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9264056fb0 0x7f9264059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.052+0000 7f92777fe700 1 -- 172.21.2.111:0/3159484646 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f92840df3c0 con 0x7f927c00bca0 2021-04-03T03:16:07.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.052+0000 7f928a0c0700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9264056fb0 0x7f9264059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.052+0000 7f928a0c0700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9264056fb0 0x7f9264059470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f9280002f30 tx=0x7f9280015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.052+0000 7f92777fe700 1 -- 172.21.2.111:0/3159484646 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f92840b4750 con 0x7f927c00bca0 2021-04-03T03:16:07.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.144+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f927c004da0 con 0x7f927c00bca0 2021-04-03T03:16:07.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f92777fe700 1 -- 172.21.2.111:0/3159484646 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16282 (secure 0 0 0) 0x7f928408b090 con 0x7f927c00bca0 2021-04-03T03:16:07.149 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:07.150 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":103,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:03.554720+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:00.429170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:01.518005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:07.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9264056fb0 msgr2=0x7f9264059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9264056fb0 0x7f9264059470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f9280002f30 tx=0x7f9280015000).stop 2021-04-03T03:16:07.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 msgr2=0x7f927c0e6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0e6c70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f9284066000 tx=0x7f928405ad70).stop 2021-04-03T03:16:07.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 shutdown_connections 2021-04-03T03:16:07.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f927c00bca0 0x7f927c0e6c70 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9264056fb0 0x7f9264059470 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f927c005b40 0x7f927c0e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 --2- 172.21.2.111:0/3159484646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f927c007c30 0x7f927c02b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 >> 172.21.2.111:0/3159484646 conn(0x7f927c0484d0 msgr2=0x7f927c0d77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 shutdown_connections 2021-04-03T03:16:07.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.148+0000 7f928b0c2700 1 -- 172.21.2.111:0/3159484646 wait complete. 2021-04-03T03:16:07.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:16:00.429170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:16:01.518005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:16:07.158 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:07.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 Processor -- start 2021-04-03T03:16:07.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 -- start start 2021-04-03T03:16:07.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000603f0 0x7f2d001447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f2d000add50 0x7f2d00144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2d000b17b0 0x7f2d00147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d001451c0 con 0x7f2d000b17b0 2021-04-03T03:16:07.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d000607d0 con 0x7f2d000add50 2021-04-03T03:16:07.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0014b090 con 0x7f2d000603f0 2021-04-03T03:16:07.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2d000b17b0 0x7f2d00147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2d000b17b0 0x7f2d00147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56126/0 (socket says 172.21.2.111:56126) 2021-04-03T03:16:07.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 -- 172.21.2.111:0/873995860 learned_addr learned my addr 172.21.2.111:0/873995860 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 -- 172.21.2.111:0/873995860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000603f0 msgr2=0x7f2d001447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 --2- 172.21.2.111:0/873995860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000603f0 0x7f2d001447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 -- 172.21.2.111:0/873995860 >> v2:172.21.2.115:3300/0 conn(0x7f2d000add50 msgr2=0x7f2d00144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 --2- 172.21.2.111:0/873995860 >> v2:172.21.2.115:3300/0 conn(0x7f2d000add50 0x7f2d00144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 -- 172.21.2.111:0/873995860 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0014b330 con 0x7f2d000b17b0 2021-04-03T03:16:07.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d05ddf700 1 --2- 172.21.2.111:0/873995860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000b17b0 0x7f2d00147740 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f2cf80032b0 tx=0x7f2cf8009270).ready entity=mon.0 client_cookie=13dfa565b04915e0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2cf3fff700 1 -- 172.21.2.111:0/873995860 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cf800bc00 con 0x7f2d000b17b0 2021-04-03T03:16:07.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2cf3fff700 1 -- 172.21.2.111:0/873995860 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf800bd60 con 0x7f2d000b17b0 2021-04-03T03:16:07.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2cf3fff700 1 -- 172.21.2.111:0/873995860 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cf8011400 con 0x7f2d000b17b0 2021-04-03T03:16:07.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 -- 172.21.2.111:0/873995860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000b17b0 msgr2=0x7f2d00147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.212+0000 7f2d065e0700 1 --2- 172.21.2.111:0/873995860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000b17b0 0x7f2d00147740 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f2cf80032b0 tx=0x7f2cf8009270).stop 2021-04-03T03:16:07.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- 172.21.2.111:0/873995860 shutdown_connections 2021-04-03T03:16:07.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 --2- 172.21.2.111:0/873995860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000b17b0 0x7f2d00147740 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 --2- 172.21.2.111:0/873995860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000603f0 0x7f2d001447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 --2- 172.21.2.111:0/873995860 >> v2:172.21.2.115:3300/0 conn(0x7f2d000add50 0x7f2d00144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- 172.21.2.111:0/873995860 >> 172.21.2.111:0/873995860 conn(0x7f2d000b61c0 msgr2=0x7f2d0005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- 172.21.2.111:0/873995860 shutdown_connections 2021-04-03T03:16:07.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- 172.21.2.111:0/873995860 wait complete. 2021-04-03T03:16:07.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 Processor -- start 2021-04-03T03:16:07.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- start start 2021-04-03T03:16:07.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000603f0 0x7f2d00147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d000add50 0x7f2d00063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000b17b0 0x7f2d00064870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000aaa00 con 0x7f2d000603f0 2021-04-03T03:16:07.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d00062bf0 con 0x7f2d000add50 2021-04-03T03:16:07.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d00067ea0 con 0x7f2d000b17b0 2021-04-03T03:16:07.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000603f0 0x7f2d00147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000603f0 0x7f2d00147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56128/0 (socket says 172.21.2.111:56128) 2021-04-03T03:16:07.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 -- 172.21.2.111:0/1223112123 learned_addr learned my addr 172.21.2.111:0/1223112123 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 -- 172.21.2.111:0/1223112123 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000b17b0 msgr2=0x7f2d00064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000b17b0 0x7f2d00064870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 -- 172.21.2.111:0/1223112123 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d000add50 msgr2=0x7f2d00063090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d000add50 0x7f2d00063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 -- 172.21.2.111:0/1223112123 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000aa560 con 0x7f2d000603f0 2021-04-03T03:16:07.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d055de700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000603f0 0x7f2d00147640 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f2cfc002f30 tx=0x7f2cfc006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2cf27fc700 1 -- 172.21.2.111:0/1223112123 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cfc00d840 con 0x7f2d000603f0 2021-04-03T03:16:07.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2cf27fc700 1 -- 172.21.2.111:0/1223112123 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc000e90 con 0x7f2d000603f0 2021-04-03T03:16:07.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d000abd00 con 0x7f2d000603f0 2021-04-03T03:16:07.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d00064d60 con 0x7f2d000603f0 2021-04-03T03:16:07.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2cf27fc700 1 -- 172.21.2.111:0/1223112123 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cfc00db90 con 0x7f2d000603f0 2021-04-03T03:16:07.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d000aac00 con 0x7f2d000603f0 2021-04-03T03:16:07.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2cf27fc700 1 -- 172.21.2.111:0/1223112123 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2cfc017030 con 0x7f2d000603f0 2021-04-03T03:16:07.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2cf27fc700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ce0056ef0 0x7f2ce00593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2cf27fc700 1 -- 172.21.2.111:0/1223112123 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f2cfc06f6e0 con 0x7f2d000603f0 2021-04-03T03:16:07.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.216+0000 7f2d04ddd700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ce0056ef0 0x7f2ce00593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.220+0000 7f2d04ddd700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ce0056ef0 0x7f2ce00593b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f2cf4002f30 tx=0x7f2cf40069e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.220+0000 7f2cf27fc700 1 -- 172.21.2.111:0/1223112123 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2cfc044a70 con 0x7f2d000603f0 2021-04-03T03:16:07.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2d00062db0 con 0x7f2d000603f0 2021-04-03T03:16:07.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2cf27fc700 1 -- 172.21.2.111:0/1223112123 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v103) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2cfc048090 con 0x7f2d000603f0 2021-04-03T03:16:07.316 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:07.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ce0056ef0 msgr2=0x7f2ce00593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ce0056ef0 0x7f2ce00593b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f2cf4002f30 tx=0x7f2cf40069e0).stop 2021-04-03T03:16:07.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000603f0 msgr2=0x7f2d00147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000603f0 0x7f2d00147640 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f2cfc002f30 tx=0x7f2cfc006ea0).stop 2021-04-03T03:16:07.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 shutdown_connections 2021-04-03T03:16:07.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d000b17b0 0x7f2d00064870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d000603f0 0x7f2d00147640 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ce0056ef0 0x7f2ce00593b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 --2- 172.21.2.111:0/1223112123 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d000add50 0x7f2d00063090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 >> 172.21.2.111:0/1223112123 conn(0x7f2d000b61c0 msgr2=0x7f2d000b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 shutdown_connections 2021-04-03T03:16:07.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.312+0000 7f2d065e0700 1 -- 172.21.2.111:0/1223112123 wait complete. 2021-04-03T03:16:07.323 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:07.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 Processor -- start 2021-04-03T03:16:07.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- start start 2021-04-03T03:16:07.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f82a4060260 0x7f82a40ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a4147bd0 con 0x7f82a4147750 2021-04-03T03:16:07.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a40b56f0 con 0x7f82a414b1c0 2021-04-03T03:16:07.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a4141c50 con 0x7f82a4060260 2021-04-03T03:16:07.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a27fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f82a4147750 0x7f82a40b37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56932/0 (socket says 172.21.2.111:56932) 2021-04-03T03:16:07.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a27fc700 1 -- 172.21.2.111:0/938594087 learned_addr learned my addr 172.21.2.111:0/938594087 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:07.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a2ffd700 1 --2- 172.21.2.111:0/938594087 >> v2:172.21.2.111:3300/0 conn(0x7f82a4060260 0x7f82a40ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a2ffd700 1 -- 172.21.2.111:0/938594087 >> v1:172.21.2.115:6789/0 conn(0x7f82a414b1c0 legacy=0x7f82a414b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a2ffd700 1 -- 172.21.2.111:0/938594087 >> v1:172.21.2.111:6789/0 conn(0x7f82a4147750 legacy=0x7f82a40b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a2ffd700 1 -- 172.21.2.111:0/938594087 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a40b5990 con 0x7f82a4060260 2021-04-03T03:16:07.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a2ffd700 1 --2- 172.21.2.111:0/938594087 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad170 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f82a405d0c0 tx=0x7f828c006dd0).ready entity=mon.0 client_cookie=719b9ea5f92c8ac3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a1ffb700 1 -- 172.21.2.111:0/938594087 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f828c0097b0 con 0x7f82a4060260 2021-04-03T03:16:07.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a1ffb700 1 -- 172.21.2.111:0/938594087 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c009910 con 0x7f82a4060260 2021-04-03T03:16:07.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a1ffb700 1 -- 172.21.2.111:0/938594087 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f828c009c70 con 0x7f82a4060260 2021-04-03T03:16:07.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- 172.21.2.111:0/938594087 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 msgr2=0x7f82a40ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 --2- 172.21.2.111:0/938594087 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad170 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f82a405d0c0 tx=0x7f828c006dd0).stop 2021-04-03T03:16:07.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- 172.21.2.111:0/938594087 shutdown_connections 2021-04-03T03:16:07.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 --2- 172.21.2.111:0/938594087 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad170 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- 172.21.2.111:0/938594087 >> 172.21.2.111:0/938594087 conn(0x7f82a405c7d0 msgr2=0x7f82a405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- 172.21.2.111:0/938594087 shutdown_connections 2021-04-03T03:16:07.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- 172.21.2.111:0/938594087 wait complete. 2021-04-03T03:16:07.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 Processor -- start 2021-04-03T03:16:07.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.380+0000 7f82a3fff700 1 -- start start 2021-04-03T03:16:07.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f82a4147750 0x7f82a40b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82a414b1c0 0x7f82a40af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a41456a0 con 0x7f82a4060260 2021-04-03T03:16:07.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40ad170 con 0x7f82a414b1c0 2021-04-03T03:16:07.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40ad410 con 0x7f82a4147750 2021-04-03T03:16:07.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56142/0 (socket says 172.21.2.111:56142) 2021-04-03T03:16:07.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 -- 172.21.2.111:0/2017589967 learned_addr learned my addr 172.21.2.111:0/2017589967 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 -- 172.21.2.111:0/2017589967 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f82a4147750 msgr2=0x7f82a40b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f82a4147750 0x7f82a40b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 -- 172.21.2.111:0/2017589967 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82a414b1c0 msgr2=0x7f82a40af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82a414b1c0 0x7f82a40af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 -- 172.21.2.111:0/2017589967 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4145320 con 0x7f82a4060260 2021-04-03T03:16:07.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a2ffd700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad8b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f82a405d0c0 tx=0x7f828c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f829bfff700 1 -- 172.21.2.111:0/2017589967 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f828c011db0 con 0x7f82a4060260 2021-04-03T03:16:07.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f829bfff700 1 -- 172.21.2.111:0/2017589967 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c01e460 con 0x7f82a4060260 2021-04-03T03:16:07.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f829bfff700 1 -- 172.21.2.111:0/2017589967 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f828c01e7c0 con 0x7f82a4060260 2021-04-03T03:16:07.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a414aae0 con 0x7f82a4060260 2021-04-03T03:16:07.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a4145f50 con 0x7f82a4060260 2021-04-03T03:16:07.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a40ad5d0 con 0x7f82a4060260 2021-04-03T03:16:07.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f829bfff700 1 -- 172.21.2.111:0/2017589967 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f828c01a040 con 0x7f82a4060260 2021-04-03T03:16:07.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f829bfff700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8288056f20 0x7f82880593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f829bfff700 1 -- 172.21.2.111:0/2017589967 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f828c0713e0 con 0x7f82a4060260 2021-04-03T03:16:07.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a27fc700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8288056f20 0x7f82880593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.384+0000 7f82a27fc700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8288056f20 0x7f82880593e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f82a405d020 tx=0x7f82940077e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.388+0000 7f829bfff700 1 -- 172.21.2.111:0/2017589967 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f828c0467d0 con 0x7f82a4060260 2021-04-03T03:16:07.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f82a4069270 con 0x7f82a4060260 2021-04-03T03:16:07.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f829bfff700 1 -- 172.21.2.111:0/2017589967 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f828c0230a0 con 0x7f82a4060260 2021-04-03T03:16:07.482 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:07.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8288056f20 msgr2=0x7f82880593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8288056f20 0x7f82880593e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f82a405d020 tx=0x7f82940077e0).stop 2021-04-03T03:16:07.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 msgr2=0x7f82a40ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad8b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f82a405d0c0 tx=0x7f828c015000).stop 2021-04-03T03:16:07.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 shutdown_connections 2021-04-03T03:16:07.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8288056f20 0x7f82880593e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f82a414b1c0 0x7f82a40af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f82a4060260 0x7f82a40ad8b0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 --2- 172.21.2.111:0/2017589967 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f82a4147750 0x7f82a40b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 >> 172.21.2.111:0/2017589967 conn(0x7f82a405c7d0 msgr2=0x7f82a40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 shutdown_connections 2021-04-03T03:16:07.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.480+0000 7f82a3fff700 1 -- 172.21.2.111:0/2017589967 wait complete. 2021-04-03T03:16:07.489 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:16:07.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4f59e700 1 Processor -- start 2021-04-03T03:16:07.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4f59e700 1 -- start start 2021-04-03T03:16:07.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b500685b0 0x7f9b500ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b500600c0 con 0x7f9b500b1990 2021-04-03T03:16:07.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b500afd90 con 0x7f9b5005fce0 2021-04-03T03:16:07.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500b2430 con 0x7f9b500685b0 2021-04-03T03:16:07.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4dd9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f9b500b1990 0x7f9b501429f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56944/0 (socket says 172.21.2.111:56944) 2021-04-03T03:16:07.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.544+0000 7f9b4dd9b700 1 -- 172.21.2.111:0/2744088083 learned_addr learned my addr 172.21.2.111:0/2744088083 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:07.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029744232 0 0) 0x7f9b500600c0 con 0x7f9b500b1990 2021-04-03T03:16:07.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b30003240 con 0x7f9b500b1990 2021-04-03T03:16:07.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b500685b0 0x7f9b500ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3206121611 0 0) 0x7f9b30003240 con 0x7f9b500b1990 2021-04-03T03:16:07.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b500685b0 msgr2=0x7f9b500ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b500685b0 0x7f9b500ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 >> v1:172.21.2.115:6789/0 conn(0x7f9b5005fce0 legacy=0x7f9b5013e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b500b0030 con 0x7f9b500b1990 2021-04-03T03:16:07.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f9b380016d0 con 0x7f9b500b1990 2021-04-03T03:16:07.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 >> v1:172.21.2.111:6789/0 conn(0x7f9b500b1990 legacy=0x7f9b501429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b500685b0 0x7f9b30003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b30003cc0 0x7f9b300044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b300049d0 0x7f9b30004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500b2430 con 0x7f9b300049d0 2021-04-03T03:16:07.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30004250 con 0x7f9b30003cc0 2021-04-03T03:16:07.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3000b730 con 0x7f9b500685b0 2021-04-03T03:16:07.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b300049d0 0x7f9b30004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4dd9b700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b30003cc0 0x7f9b300044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b500685b0 msgr2=0x7f9b30003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b500685b0 0x7f9b30003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b30003cc0 msgr2=0x7f9b300044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b30003cc0 0x7f9b300044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/2744088083 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b30003240 con 0x7f9b300049d0 2021-04-03T03:16:07.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b300049d0 0x7f9b30004e50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f9b4800a000 tx=0x7f9b48008040).ready entity=mon.0 client_cookie=7080018305c83482 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b4800dd90 con 0x7f9b300049d0 2021-04-03T03:16:07.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48013480 con 0x7f9b300049d0 2021-04-03T03:16:07.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4d59a700 1 -- 172.21.2.111:0/2744088083 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b480137e0 con 0x7f9b300049d0 2021-04-03T03:16:07.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- 172.21.2.111:0/2744088083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b300049d0 msgr2=0x7f9b30004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 --2- 172.21.2.111:0/2744088083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b300049d0 0x7f9b30004e50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f9b4800a000 tx=0x7f9b48008040).stop 2021-04-03T03:16:07.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/2744088083 reap_dead start 2021-04-03T03:16:07.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- 172.21.2.111:0/2744088083 shutdown_connections 2021-04-03T03:16:07.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- 172.21.2.111:0/2744088083 >> 172.21.2.111:0/2744088083 conn(0x7f9b500b5b50 msgr2=0x7f9b500b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- 172.21.2.111:0/2744088083 shutdown_connections 2021-04-03T03:16:07.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- 172.21.2.111:0/2744088083 wait complete. 2021-04-03T03:16:07.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 Processor -- start 2021-04-03T03:16:07.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- start start 2021-04-03T03:16:07.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b5013ecb0 0x7f9b5013f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b501417e0 0x7f9b500640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b50141c50 0x7f9b50064590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50142190 con 0x7f9b50141c50 2021-04-03T03:16:07.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50141340 con 0x7f9b501417e0 2021-04-03T03:16:07.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50143780 con 0x7f9b5013ecb0 2021-04-03T03:16:07.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b50141c50 0x7f9b50064590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b50141c50 0x7f9b50064590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56166/0 (socket says 172.21.2.111:56166) 2021-04-03T03:16:07.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/3729801131 learned_addr learned my addr 172.21.2.111:0/3729801131 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4dd9b700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b501417e0 0x7f9b500640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b5013ecb0 msgr2=0x7f9b5013f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b5013ecb0 0x7f9b5013f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b501417e0 msgr2=0x7f9b500640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b501417e0 0x7f9b500640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 -- 172.21.2.111:0/3729801131 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b50064a80 con 0x7f9b50141c50 2021-04-03T03:16:07.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4dd9b700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b501417e0 0x7f9b500640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:07.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4ed9d700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b50141c50 0x7f9b50064590 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f9b380018b0 tx=0x7f9b48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b477fe700 1 -- 172.21.2.111:0/3729801131 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b4801e610 con 0x7f9b50141c50 2021-04-03T03:16:07.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b477fe700 1 -- 172.21.2.111:0/3729801131 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4801e770 con 0x7f9b50141c50 2021-04-03T03:16:07.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b50064c70 con 0x7f9b50141c50 2021-04-03T03:16:07.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b477fe700 1 -- 172.21.2.111:0/3729801131 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b4801ead0 con 0x7f9b50141c50 2021-04-03T03:16:07.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.548+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b50067d90 con 0x7f9b50141c50 2021-04-03T03:16:07.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.552+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b50142300 con 0x7f9b50141c50 2021-04-03T03:16:07.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.552+0000 7f9b477fe700 1 -- 172.21.2.111:0/3729801131 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9b4801ec30 con 0x7f9b50141c50 2021-04-03T03:16:07.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.552+0000 7f9b477fe700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b34057050 0x7f9b34059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.552+0000 7f9b477fe700 1 -- 172.21.2.111:0/3729801131 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9b48002030 con 0x7f9b50141c50 2021-04-03T03:16:07.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.552+0000 7f9b4e59c700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b34057050 0x7f9b34059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.552+0000 7f9b4e59c700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b34057050 0x7f9b34059510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f9b500b6a50 tx=0x7f9b400077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.552+0000 7f9b477fe700 1 -- 172.21.2.111:0/3729801131 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9b48047110 con 0x7f9b50141c50 2021-04-03T03:16:07.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b50141500 con 0x7f9b50141c50 2021-04-03T03:16:07.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b477fe700 1 -- 172.21.2.111:0/3729801131 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9b4804a730 con 0x7f9b50141c50 2021-04-03T03:16:07.651 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:07.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b34057050 msgr2=0x7f9b34059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b34057050 0x7f9b34059510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f9b500b6a50 tx=0x7f9b400077f0).stop 2021-04-03T03:16:07.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b50141c50 msgr2=0x7f9b50064590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b50141c50 0x7f9b50064590 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f9b380018b0 tx=0x7f9b48008040).stop 2021-04-03T03:16:07.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 shutdown_connections 2021-04-03T03:16:07.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9b50141c50 0x7f9b50064590 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9b5013ecb0 0x7f9b5013f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9b34057050 0x7f9b34059510 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 --2- 172.21.2.111:0/3729801131 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9b501417e0 0x7f9b500640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 >> 172.21.2.111:0/3729801131 conn(0x7f9b500b5b50 msgr2=0x7f9b500b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 shutdown_connections 2021-04-03T03:16:07.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.648+0000 7f9b4f59e700 1 -- 172.21.2.111:0/3729801131 wait complete. 2021-04-03T03:16:07.659 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:16:07.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.712+0000 7f5c9d306700 1 Processor -- start 2021-04-03T03:16:07.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.712+0000 7f5c9d306700 1 -- start start 2021-04-03T03:16:07.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5c980603f0 0x7f5c981447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980b17b0 0x7f5c98144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c980607d0 con 0x7f5c980add50 2021-04-03T03:16:07.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c981451c0 con 0x7f5c980603f0 2021-04-03T03:16:07.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9814b090 con 0x7f5c980b17b0 2021-04-03T03:16:07.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c97fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5c980603f0 0x7f5c981447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c977fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5c980add50 0x7f5c98147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:56964/0 (socket says 172.21.2.111:56964) 2021-04-03T03:16:07.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c97fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5c980603f0 0x7f5c981447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56170/0 (socket says 172.21.2.111:56170) 2021-04-03T03:16:07.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c977fe700 1 -- 172.21.2.111:0/1212665321 learned_addr learned my addr 172.21.2.111:0/1212665321 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:07.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c97fff700 1 -- 172.21.2.111:0/1212665321 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980b17b0 msgr2=0x7f5c98144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c97fff700 1 --2- 172.21.2.111:0/1212665321 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980b17b0 0x7f5c98144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c97fff700 1 -- 172.21.2.111:0/1212665321 >> v1:172.21.2.111:6789/0 conn(0x7f5c980add50 legacy=0x7f5c98147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c97fff700 1 -- 172.21.2.111:0/1212665321 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9814b330 con 0x7f5c980603f0 2021-04-03T03:16:07.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c97fff700 1 --2- 172.21.2.111:0/1212665321 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980603f0 0x7f5c981447e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f5c88002f30 tx=0x7f5c88006dd0).ready entity=mon.0 client_cookie=581868717481a645 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c96ffd700 1 -- 172.21.2.111:0/1212665321 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c880097f0 con 0x7f5c980603f0 2021-04-03T03:16:07.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c96ffd700 1 -- 172.21.2.111:0/1212665321 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c88009950 con 0x7f5c980603f0 2021-04-03T03:16:07.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c96ffd700 1 -- 172.21.2.111:0/1212665321 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c88009cb0 con 0x7f5c980603f0 2021-04-03T03:16:07.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- 172.21.2.111:0/1212665321 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980603f0 msgr2=0x7f5c981447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- 172.21.2.111:0/1212665321 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980603f0 0x7f5c981447e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f5c88002f30 tx=0x7f5c88006dd0).stop 2021-04-03T03:16:07.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- 172.21.2.111:0/1212665321 shutdown_connections 2021-04-03T03:16:07.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- 172.21.2.111:0/1212665321 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980b17b0 0x7f5c98144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- 172.21.2.111:0/1212665321 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980603f0 0x7f5c981447e0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- 172.21.2.111:0/1212665321 >> 172.21.2.111:0/1212665321 conn(0x7f5c980b61c0 msgr2=0x7f5c9805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- 172.21.2.111:0/1212665321 shutdown_connections 2021-04-03T03:16:07.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- 172.21.2.111:0/1212665321 wait complete. 2021-04-03T03:16:07.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 Processor -- start 2021-04-03T03:16:07.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- start start 2021-04-03T03:16:07.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5c980603f0 0x7f5c9815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980add50 0x7f5c98145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980b17b0 0x7f5c980a8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c980a7930 con 0x7f5c980b17b0 2021-04-03T03:16:07.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c98157cd0 con 0x7f5c980603f0 2021-04-03T03:16:07.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9d306700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9815abb0 con 0x7f5c980add50 2021-04-03T03:16:07.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980b17b0 0x7f5c980a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980b17b0 0x7f5c980a8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56180/0 (socket says 172.21.2.111:56180) 2021-04-03T03:16:07.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 -- 172.21.2.111:0/3937074272 learned_addr learned my addr 172.21.2.111:0/3937074272 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 -- 172.21.2.111:0/3937074272 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980add50 msgr2=0x7f5c98145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980add50 0x7f5c98145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 -- 172.21.2.111:0/3937074272 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5c980603f0 msgr2=0x7f5c9815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5c980603f0 0x7f5c9815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.716+0000 7f5c9cb05700 1 -- 172.21.2.111:0/3937074272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c980a7b90 con 0x7f5c980b17b0 2021-04-03T03:16:07.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c9cb05700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980b17b0 0x7f5c980a8030 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f5c900032b0 tx=0x7f5c900091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c957fa700 1 -- 172.21.2.111:0/3937074272 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c9000c960 con 0x7f5c980b17b0 2021-04-03T03:16:07.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c957fa700 1 -- 172.21.2.111:0/3937074272 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9000cac0 con 0x7f5c980b17b0 2021-04-03T03:16:07.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9815a8a0 con 0x7f5c980b17b0 2021-04-03T03:16:07.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c957fa700 1 -- 172.21.2.111:0/3937074272 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c9000ce20 con 0x7f5c980b17b0 2021-04-03T03:16:07.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c98147680 con 0x7f5c980b17b0 2021-04-03T03:16:07.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c980a7d20 con 0x7f5c980b17b0 2021-04-03T03:16:07.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c957fa700 1 -- 172.21.2.111:0/3937074272 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5c90014420 con 0x7f5c980b17b0 2021-04-03T03:16:07.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c957fa700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5c7c056ea0 0x7f5c7c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c957fa700 1 -- 172.21.2.111:0/3937074272 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5c90070000 con 0x7f5c980b17b0 2021-04-03T03:16:07.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c97fff700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5c7c056ea0 0x7f5c7c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.720+0000 7f5c97fff700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5c7c056ea0 0x7f5c7c059360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f5c88002f30 tx=0x7f5c88015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.724+0000 7f5c957fa700 1 -- 172.21.2.111:0/3937074272 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5c900452f0 con 0x7f5c980b17b0 2021-04-03T03:16:07.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c98057660 con 0x7f5c980b17b0 2021-04-03T03:16:07.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c957fa700 1 -- 172.21.2.111:0/3937074272 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5c90019070 con 0x7f5c980b17b0 2021-04-03T03:16:07.820 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:07.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5c7c056ea0 msgr2=0x7f5c7c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5c7c056ea0 0x7f5c7c059360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f5c88002f30 tx=0x7f5c88015000).stop 2021-04-03T03:16:07.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980b17b0 msgr2=0x7f5c980a8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980b17b0 0x7f5c980a8030 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f5c900032b0 tx=0x7f5c900091a0).stop 2021-04-03T03:16:07.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 shutdown_connections 2021-04-03T03:16:07.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5c980b17b0 0x7f5c980a8030 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5c7c056ea0 0x7f5c7c059360 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5c980603f0 0x7f5c9815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 --2- 172.21.2.111:0/3937074272 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5c980add50 0x7f5c98145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 >> 172.21.2.111:0/3937074272 conn(0x7f5c980b61c0 msgr2=0x7f5c9805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 shutdown_connections 2021-04-03T03:16:07.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.816+0000 7f5c9d306700 1 -- 172.21.2.111:0/3937074272 wait complete. 2021-04-03T03:16:07.828 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:16:07.828 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:07.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.880+0000 7f3f6a7a7700 1 Processor -- start 2021-04-03T03:16:07.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.880+0000 7f3f6a7a7700 1 -- start start 2021-04-03T03:16:07.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3f640603f0 0x7f3f640b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3f6414b700 0x7f3f640b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f640607d0 con 0x7f3f640b12b0 2021-04-03T03:16:07.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640af400 con 0x7f3f6414b700 2021-04-03T03:16:07.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640af6a0 con 0x7f3f640603f0 2021-04-03T03:16:07.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3f6414b700 0x7f3f640b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3f6414b700 0x7f3f640b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56186/0 (socket says 172.21.2.111:56186) 2021-04-03T03:16:07.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 -- 172.21.2.111:0/2117464316 learned_addr learned my addr 172.21.2.111:0/2117464316 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 -- 172.21.2.111:0/2117464316 >> v2:172.21.2.115:3300/0 conn(0x7f3f640603f0 msgr2=0x7f3f640b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 --2- 172.21.2.111:0/2117464316 >> v2:172.21.2.115:3300/0 conn(0x7f3f640603f0 0x7f3f640b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 -- 172.21.2.111:0/2117464316 >> v1:172.21.2.115:6789/0 conn(0x7f3f640b12b0 legacy=0x7f3f641465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:07.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 -- 172.21.2.111:0/2117464316 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640afb40 con 0x7f3f6414b700 2021-04-03T03:16:07.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 --2- 172.21.2.111:0/2117464316 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f640b0580 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f3f6406a3b0 tx=0x7f3f58006e90).ready entity=mon.0 client_cookie=9fbdc6444952ccb3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f57fff700 1 -- 172.21.2.111:0/2117464316 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f58009930 con 0x7f3f6414b700 2021-04-03T03:16:07.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f57fff700 1 -- 172.21.2.111:0/2117464316 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f58009a90 con 0x7f3f6414b700 2021-04-03T03:16:07.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f57fff700 1 -- 172.21.2.111:0/2117464316 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f58009e30 con 0x7f3f6414b700 2021-04-03T03:16:07.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/2117464316 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 msgr2=0x7f3f640b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/2117464316 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f640b0580 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f3f6406a3b0 tx=0x7f3f58006e90).stop 2021-04-03T03:16:07.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/2117464316 shutdown_connections 2021-04-03T03:16:07.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/2117464316 >> v2:172.21.2.115:3300/0 conn(0x7f3f640603f0 0x7f3f640b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:07.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/2117464316 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f640b0580 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/2117464316 >> 172.21.2.111:0/2117464316 conn(0x7f3f640b61c0 msgr2=0x7f3f6405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:07.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/2117464316 shutdown_connections 2021-04-03T03:16:07.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/2117464316 wait complete. 2021-04-03T03:16:07.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 Processor -- start 2021-04-03T03:16:07.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- start start 2021-04-03T03:16:07.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3f640603f0 0x7f3f640aa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3f640b12b0 0x7f3f64064a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f64064f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640abd70 con 0x7f3f6414b700 2021-04-03T03:16:07.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64062d70 con 0x7f3f640603f0 2021-04-03T03:16:07.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64063010 con 0x7f3f640b12b0 2021-04-03T03:16:07.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f64064f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f64064f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56190/0 (socket says 172.21.2.111:56190) 2021-04-03T03:16:07.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 -- 172.21.2.111:0/893368468 learned_addr learned my addr 172.21.2.111:0/893368468 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:07.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f697a5700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3f640603f0 0x7f3f640aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3f640b12b0 0x7f3f64064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 -- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3f640b12b0 msgr2=0x7f3f64064a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3f640b12b0 0x7f3f64064a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 -- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3f640603f0 msgr2=0x7f3f640aa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:07.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3f640603f0 0x7f3f640aa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:07.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 -- 172.21.2.111:0/893368468 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640632b0 con 0x7f3f6414b700 2021-04-03T03:16:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f68fa4700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3f640b12b0 0x7f3f64064a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f69fa6700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f64064f00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f3f5c0032d0 tx=0x7f3f5c006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f567fc700 1 -- 172.21.2.111:0/893368468 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f5c00c960 con 0x7f3f6414b700 2021-04-03T03:16:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f567fc700 1 -- 172.21.2.111:0/893368468 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c00cac0 con 0x7f3f6414b700 2021-04-03T03:16:07.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f640a9990 con 0x7f3f6414b700 2021-04-03T03:16:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f567fc700 1 -- 172.21.2.111:0/893368468 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f5c00ce20 con 0x7f3f6414b700 2021-04-03T03:16:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.884+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f640628d0 con 0x7f3f6414b700 2021-04-03T03:16:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.888+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f64062be0 con 0x7f3f6414b700 2021-04-03T03:16:07.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.888+0000 7f3f567fc700 1 -- 172.21.2.111:0/893368468 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3f5c01c070 con 0x7f3f6414b700 2021-04-03T03:16:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.888+0000 7f3f567fc700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3f44056ea0 0x7f3f44059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.888+0000 7f3f567fc700 1 -- 172.21.2.111:0/893368468 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3f5c06ff00 con 0x7f3f6414b700 2021-04-03T03:16:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.888+0000 7f3f697a5700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3f44056ea0 0x7f3f44059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.888+0000 7f3f697a5700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3f44056ea0 0x7f3f44059360 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f3f6000a000 tx=0x7f3f60008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:07.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:07.888+0000 7f3f567fc700 1 -- 172.21.2.111:0/893368468 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3f5c0451e0 con 0x7f3f6414b700 2021-04-03T03:16:08.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f64067f10 con 0x7f3f6414b700 2021-04-03T03:16:08.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f567fc700 1 -- 172.21.2.111:0/893368468 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f3f5c048800 con 0x7f3f6414b700 2021-04-03T03:16:08.014 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:08.014 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/3373357469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3373357469},{"type":"v1","addr":"172.21.2.115:6833","nonce":3373357469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:02.550193+0000","modified":"2021-04-03T03:16:03.553657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/3012485417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":3012485417},{"type":"v1","addr":"172.21.2.111:6835","nonce":3012485417}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-04-03T03:16:08.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3f44056ea0 msgr2=0x7f3f44059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:08.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3f44056ea0 0x7f3f44059360 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f3f6000a000 tx=0x7f3f60008040).stop 2021-04-03T03:16:08.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 msgr2=0x7f3f64064f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:08.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f64064f00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f3f5c0032d0 tx=0x7f3f5c006f90).stop 2021-04-03T03:16:08.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 shutdown_connections 2021-04-03T03:16:08.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3f6414b700 0x7f3f64064f00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:08.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3f44056ea0 0x7f3f44059360 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:08.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3f640603f0 0x7f3f640aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:08.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 --2- 172.21.2.111:0/893368468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3f640b12b0 0x7f3f64064a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:08.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 >> 172.21.2.111:0/893368468 conn(0x7f3f640b61c0 msgr2=0x7f3f64147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:08.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 shutdown_connections 2021-04-03T03:16:08.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.012+0000 7f3f6a7a7700 1 -- 172.21.2.111:0/893368468 wait complete. 2021-04-03T03:16:08.018 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 43 2021-04-03T03:16:08.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:02.550193+0000', 'modified': '2021-04-03T03:16:03.553657+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7179': {'gid': 7179, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/3012485417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 3012485417}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 3012485417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:08.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:08.224 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:08.296 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7179,"want_state":"up:active","state":"up:active","rank_uptime":5.7395811730000004,"mdsmap_epoch":43,"osdmap_epoch":103,"osdmap_epoch_barrier":102,"uptime":9.9778436429999999} 2021-04-03T03:16:08.297 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7179, "mdsmap_epoch": 43, "osdmap_epoch": 103, "osdmap_epoch_barrier": 102, "rank_uptime": 5.739581173, "state": "up:active", "uptime": 9.977843643, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:08.297 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:16:08.298 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:16:08.298 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:16:08.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:08.299 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:08.309 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:16:08.363 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:08.363 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:08.411 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:08.412 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:08.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:08.413 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:16:08.484 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:16:08.485 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:08.545 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:16:08.559 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:08.560 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:08.587 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:16:08.580+0000 7f02ee39b0c0 -1 init, newargv = 0x56279b89db20 newargc=9 2021-04-03T03:16:08.587 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15312]: starting ceph client 2021-04-03T03:16:08.592 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:08.594 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:08.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:08.596 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:16:08.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15312]: starting fuse 2021-04-03T03:16:08.604 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:16:08.604 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:16:08.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:08.606 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:16:08.606 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:16:08.607 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:16:08.608 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:16:08.609 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:16:08.609 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:08.610 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:16:08.611 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:16:08.611 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:16:08.612 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:08.612 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:16:08.613 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:16:08.613 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:16:08.614 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:16:08.614 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:08.615 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:16:08.616 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:16:08.616 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:16:08.617 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:16:08.617 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:16:08.617 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:08.618 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:16:08.619 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:16:08.673 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15312.asok 2021-04-03T03:16:08.676 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15312.asok 2021-04-03T03:16:08.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:08.677 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15312.asok status 2021-04-03T03:16:08.804 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:16:08.804 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:16:08.804 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:16:08.805 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:16:08.805 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:16:08.805 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:16:08.806 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:16:08.806 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15312", 2021-04-03T03:16:08.806 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:16:08.807 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:08.807 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:16:08.807 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:16:08.808 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 5371, 2021-04-03T03:16:08.808 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:16:08.808 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:16:08.808 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:16:08.809 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 5371 2021-04-03T03:16:08.809 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:08.809 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:16:08.809 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:16:08.810 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:16:08.810 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 918782922 2021-04-03T03:16:08.810 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:16:08.810 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:08.811 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:16:08.811 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:16:08.811 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:16:08.812 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 918782922 2021-04-03T03:16:08.812 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:08.812 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.5371 172.21.2.128:0/918782922", 2021-04-03T03:16:08.812 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/918782922", 2021-04-03T03:16:08.813 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:16:08.813 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 43, 2021-04-03T03:16:08.813 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 103, 2021-04-03T03:16:08.813 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:16:08.814 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:16:08.814 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:16:08.814 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:16:08.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:08.816 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:08.817 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:08.823 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:08.824 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:08.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:08.825 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:08.874 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2021-04-03T03:16:08.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.940+0000 7f59f9e02700 1 Processor -- start 2021-04-03T03:16:08.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.940+0000 7f59f9e02700 1 -- start start 2021-04-03T03:16:08.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f59f40603f0 0x7f59f4139940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:08.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f59f40b11a0 0x7f59f4139e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:08.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f4067940 con 0x7f59f41414b0 2021-04-03T03:16:08.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f4068010 con 0x7f59f40b11a0 2021-04-03T03:16:08.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f40607d0 con 0x7f59f40603f0 2021-04-03T03:16:08.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f59f40b11a0 0x7f59f4139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:08.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f59f40b11a0 0x7f59f4139e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56198/0 (socket says 172.21.2.111:56198) 2021-04-03T03:16:08.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 -- 172.21.2.111:0/2449810284 learned_addr learned my addr 172.21.2.111:0/2449810284 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:08.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 -- 172.21.2.111:0/2449810284 >> v2:172.21.2.115:3300/0 conn(0x7f59f40603f0 msgr2=0x7f59f4139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:08.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 --2- 172.21.2.111:0/2449810284 >> v2:172.21.2.115:3300/0 conn(0x7f59f40603f0 0x7f59f4139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:08.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 -- 172.21.2.111:0/2449810284 >> v1:172.21.2.115:6789/0 conn(0x7f59f41414b0 legacy=0x7f59f4141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:08.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 -- 172.21.2.111:0/2449810284 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4140de0 con 0x7f59f40b11a0 2021-04-03T03:16:08.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f3fff700 1 --2- 172.21.2.111:0/2449810284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f40b11a0 0x7f59f4139e30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f59f4067ef0 tx=0x7f59e8006f20).ready entity=mon.0 client_cookie=d99236573615eb75 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:08.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f37fe700 1 -- 172.21.2.111:0/2449810284 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59e800de10 con 0x7f59f40b11a0 2021-04-03T03:16:08.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f37fe700 1 -- 172.21.2.111:0/2449810284 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e8010470 con 0x7f59f40b11a0 2021-04-03T03:16:08.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f37fe700 1 -- 172.21.2.111:0/2449810284 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59e80107d0 con 0x7f59f40b11a0 2021-04-03T03:16:08.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- 172.21.2.111:0/2449810284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f40b11a0 msgr2=0x7f59f4139e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:08.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2449810284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f40b11a0 0x7f59f4139e30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f59f4067ef0 tx=0x7f59e8006f20).stop 2021-04-03T03:16:08.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- 172.21.2.111:0/2449810284 shutdown_connections 2021-04-03T03:16:08.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2449810284 >> v2:172.21.2.115:3300/0 conn(0x7f59f40603f0 0x7f59f4139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:08.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2449810284 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f40b11a0 0x7f59f4139e30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:08.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- 172.21.2.111:0/2449810284 >> 172.21.2.111:0/2449810284 conn(0x7f59f40b61c0 msgr2=0x7f59f405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:08.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- 172.21.2.111:0/2449810284 shutdown_connections 2021-04-03T03:16:08.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- 172.21.2.111:0/2449810284 wait complete. 2021-04-03T03:16:08.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 Processor -- start 2021-04-03T03:16:08.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- start start 2021-04-03T03:16:08.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59f40603f0 0x7f59f4062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:08.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f59f40b11a0 0x7f59f4139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:08.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f41414b0 0x7f59f4063530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:08.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4064120 con 0x7f59f41414b0 2021-04-03T03:16:08.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4063a20 con 0x7f59f40b11a0 2021-04-03T03:16:08.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f40668f0 con 0x7f59f40603f0 2021-04-03T03:16:08.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f41414b0 0x7f59f4063530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:08.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f41414b0 0x7f59f4063530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56206/0 (socket says 172.21.2.111:56206) 2021-04-03T03:16:08.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 -- 172.21.2.111:0/2416063489 learned_addr learned my addr 172.21.2.111:0/2416063489 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:08.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 -- 172.21.2.111:0/2416063489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59f40603f0 msgr2=0x7f59f4062de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:08.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59f40603f0 0x7f59f4062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:08.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 -- 172.21.2.111:0/2416063489 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f59f40b11a0 msgr2=0x7f59f4139020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:08.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f59f40b11a0 0x7f59f4139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:08.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 -- 172.21.2.111:0/2416063489 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4065a40 con 0x7f59f41414b0 2021-04-03T03:16:08.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9601700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f41414b0 0x7f59f4063530 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f59f4152270 tx=0x7f59ec0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:08.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f1ffb700 1 -- 172.21.2.111:0/2416063489 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ec00c900 con 0x7f59f41414b0 2021-04-03T03:16:08.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f1ffb700 1 -- 172.21.2.111:0/2416063489 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec00ca60 con 0x7f59f41414b0 2021-04-03T03:16:08.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f1ffb700 1 -- 172.21.2.111:0/2416063489 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ec00cdc0 con 0x7f59f41414b0 2021-04-03T03:16:08.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f41370e0 con 0x7f59f41414b0 2021-04-03T03:16:08.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.944+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f4137d50 con 0x7f59f41414b0 2021-04-03T03:16:08.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.948+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f4066af0 con 0x7f59f41414b0 2021-04-03T03:16:08.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.948+0000 7f59f1ffb700 1 -- 172.21.2.111:0/2416063489 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f59ec015470 con 0x7f59f41414b0 2021-04-03T03:16:08.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.948+0000 7f59f1ffb700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f59d4056fe0 0x7f59d40594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:08.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.948+0000 7f59f1ffb700 1 -- 172.21.2.111:0/2416063489 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f59ec070330 con 0x7f59f41414b0 2021-04-03T03:16:08.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.948+0000 7f59f8e00700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f59d4056fe0 0x7f59d40594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:08.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.948+0000 7f59f8e00700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f59d4056fe0 0x7f59d40594a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f59e4002f30 tx=0x7f59e4006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:08.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:08.948+0000 7f59f1ffb700 1 -- 172.21.2.111:0/2416063489 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f59ec0456c0 con 0x7f59f41414b0 2021-04-03T03:16:09.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f59f4065bd0 con 0x7f59f41414b0 2021-04-03T03:16:09.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f1ffb700 1 -- 172.21.2.111:0/2416063489 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7f59ec048ce0 con 0x7f59f41414b0 2021-04-03T03:16:09.084 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:09.084 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:09.084 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:09.085 INFO:teuthology.orchestra.run.gibba011.stdout: "total_bytes": 773094113280, 2021-04-03T03:16:09.085 INFO:teuthology.orchestra.run.gibba011.stdout: "total_avail_bytes": 764493725696, 2021-04-03T03:16:09.085 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_bytes": 10452992, 2021-04-03T03:16:09.085 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_bytes": 8600387584, 2021-04-03T03:16:09.086 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_ratio": 0.011124632321298122, 2021-04-03T03:16:09.086 INFO:teuthology.orchestra.run.gibba011.stdout: "num_osds": 8, 2021-04-03T03:16:09.086 INFO:teuthology.orchestra.run.gibba011.stdout: "num_per_pool_osds": 8, 2021-04-03T03:16:09.086 INFO:teuthology.orchestra.run.gibba011.stdout: "num_per_pool_omap_osds": 8 2021-04-03T03:16:09.086 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:09.087 INFO:teuthology.orchestra.run.gibba011.stdout: "stats_by_class": { 2021-04-03T03:16:09.087 INFO:teuthology.orchestra.run.gibba011.stdout: "ssd": { 2021-04-03T03:16:09.087 INFO:teuthology.orchestra.run.gibba011.stdout: "total_bytes": 773094113280, 2021-04-03T03:16:09.087 INFO:teuthology.orchestra.run.gibba011.stdout: "total_avail_bytes": 764493725696, 2021-04-03T03:16:09.087 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_bytes": 10452992, 2021-04-03T03:16:09.088 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_bytes": 8600387584, 2021-04-03T03:16:09.088 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_ratio": 0.011124632321298122 2021-04-03T03:16:09.088 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:09.088 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:09.088 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:16:09.089 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:09.089 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "device_health_metrics", 2021-04-03T03:16:09.089 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 1, 2021-04-03T03:16:09.089 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:09.090 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:09.090 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 2, 2021-04-03T03:16:09.090 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:09.090 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:09.090 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:09.091 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362919460864 2021-04-03T03:16:09.091 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:09.091 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:09.091 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:09.091 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "rbd", 2021-04-03T03:16:09.092 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 2, 2021-04-03T03:16:09.092 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:09.092 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:09.092 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 0, 2021-04-03T03:16:09.093 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:09.093 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:09.093 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:09.093 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362919460864 2021-04-03T03:16:09.093 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:09.094 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:09.094 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:09.094 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "cephfs_metadata", 2021-04-03T03:16:09.094 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 19, 2021-04-03T03:16:09.094 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:09.095 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 2304, 2021-04-03T03:16:09.095 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 21, 2021-04-03T03:16:09.095 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 64, 2021-04-03T03:16:09.095 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 65536, 2021-04-03T03:16:09.095 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 9.0289994147951802e-08, 2021-04-03T03:16:09.096 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362919460864 2021-04-03T03:16:09.096 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:09.096 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:09.096 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:09.096 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "cephfs_data", 2021-04-03T03:16:09.097 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 20, 2021-04-03T03:16:09.097 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:09.097 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:09.097 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 0, 2021-04-03T03:16:09.098 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:09.098 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:09.098 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:09.098 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362919460864 2021-04-03T03:16:09.098 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:09.099 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:09.099 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:09.099 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:09.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f59d4056fe0 msgr2=0x7f59d40594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f59d4056fe0 0x7f59d40594a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f59e4002f30 tx=0x7f59e4006a60).stop 2021-04-03T03:16:09.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f41414b0 msgr2=0x7f59f4063530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f41414b0 0x7f59f4063530 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f59f4152270 tx=0x7f59ec0090f0).stop 2021-04-03T03:16:09.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 shutdown_connections 2021-04-03T03:16:09.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f59f41414b0 0x7f59f4063530 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:09.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f59f40603f0 0x7f59f4062de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f59d4056fe0 0x7f59d40594a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:09.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.080+0000 7f59f9e02700 1 --2- 172.21.2.111:0/2416063489 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f59f40b11a0 0x7f59f4139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.084+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 >> 172.21.2.111:0/2416063489 conn(0x7f59f40b61c0 msgr2=0x7f59f4142ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:09.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.084+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 shutdown_connections 2021-04-03T03:16:09.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.084+0000 7f59f9e02700 1 -- 172.21.2.111:0/2416063489 wait complete. 2021-04-03T03:16:09.103 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data size -f json-pretty 2021-04-03T03:16:09.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.156+0000 7ffb2b223700 1 Processor -- start 2021-04-03T03:16:09.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.156+0000 7ffb2b223700 1 -- start start 2021-04-03T03:16:09.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ffb240603f0 0x7ffb241447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:09.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ffb240b17b0 0x7ffb24144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:09.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb240607d0 con 0x7ffb240add50 2021-04-03T03:16:09.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb241451c0 con 0x7ffb240603f0 2021-04-03T03:16:09.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2414b090 con 0x7ffb240b17b0 2021-04-03T03:16:09.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ffb240add50 0x7ffb24147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57006/0 (socket says 172.21.2.111:57006) 2021-04-03T03:16:09.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 -- 172.21.2.111:0/1134380707 learned_addr learned my addr 172.21.2.111:0/1134380707 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:09.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2a221700 1 --2- 172.21.2.111:0/1134380707 >> v2:172.21.2.111:3300/0 conn(0x7ffb240603f0 0x7ffb241447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:09.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2a221700 1 -- 172.21.2.111:0/1134380707 >> v2:172.21.2.115:3300/0 conn(0x7ffb240b17b0 msgr2=0x7ffb24144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:09.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2a221700 1 --2- 172.21.2.111:0/1134380707 >> v2:172.21.2.115:3300/0 conn(0x7ffb240b17b0 0x7ffb24144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2a221700 1 -- 172.21.2.111:0/1134380707 >> v1:172.21.2.111:6789/0 conn(0x7ffb240add50 legacy=0x7ffb24147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2a221700 1 -- 172.21.2.111:0/1134380707 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2414b330 con 0x7ffb240603f0 2021-04-03T03:16:09.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2a221700 1 --2- 172.21.2.111:0/1134380707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240603f0 0x7ffb241447e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7ffb18002f30 tx=0x7ffb18006dd0).ready entity=mon.0 client_cookie=7e707d029c7e75cf server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:09.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2921f700 1 -- 172.21.2.111:0/1134380707 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb180097f0 con 0x7ffb240603f0 2021-04-03T03:16:09.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2921f700 1 -- 172.21.2.111:0/1134380707 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb18009950 con 0x7ffb240603f0 2021-04-03T03:16:09.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2921f700 1 -- 172.21.2.111:0/1134380707 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb18009cb0 con 0x7ffb240603f0 2021-04-03T03:16:09.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- 172.21.2.111:0/1134380707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240603f0 msgr2=0x7ffb241447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- 172.21.2.111:0/1134380707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240603f0 0x7ffb241447e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7ffb18002f30 tx=0x7ffb18006dd0).stop 2021-04-03T03:16:09.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- 172.21.2.111:0/1134380707 shutdown_connections 2021-04-03T03:16:09.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- 172.21.2.111:0/1134380707 >> v2:172.21.2.115:3300/0 conn(0x7ffb240b17b0 0x7ffb24144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- 172.21.2.111:0/1134380707 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240603f0 0x7ffb241447e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:09.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- 172.21.2.111:0/1134380707 >> 172.21.2.111:0/1134380707 conn(0x7ffb240b61c0 msgr2=0x7ffb2405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:09.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- 172.21.2.111:0/1134380707 shutdown_connections 2021-04-03T03:16:09.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- 172.21.2.111:0/1134380707 wait complete. 2021-04-03T03:16:09.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 Processor -- start 2021-04-03T03:16:09.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- start start 2021-04-03T03:16:09.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb240603f0 0x7ffb240a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:09.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240add50 0x7ffb240aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:09.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb240b17b0 0x7ffb240a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:09.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240a9170 con 0x7ffb240add50 2021-04-03T03:16:09.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb24157910 con 0x7ffb240b17b0 2021-04-03T03:16:09.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb2b223700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb241479e0 con 0x7ffb240603f0 2021-04-03T03:16:09.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240add50 0x7ffb240aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:09.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240add50 0x7ffb240aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56220/0 (socket says 172.21.2.111:56220) 2021-04-03T03:16:09.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 -- 172.21.2.111:0/3970609782 learned_addr learned my addr 172.21.2.111:0/3970609782 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:09.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 -- 172.21.2.111:0/3970609782 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb240603f0 msgr2=0x7ffb240a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb240603f0 0x7ffb240a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 -- 172.21.2.111:0/3970609782 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb240b17b0 msgr2=0x7ffb240a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb240b17b0 0x7ffb240a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 -- 172.21.2.111:0/3970609782 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb24147c80 con 0x7ffb240add50 2021-04-03T03:16:09.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.160+0000 7ffb29a20700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240add50 0x7ffb240aa410 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7ffb240ae520 tx=0x7ffb20007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:09.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb177fe700 1 -- 172.21.2.111:0/3970609782 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb200072e0 con 0x7ffb240add50 2021-04-03T03:16:09.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb177fe700 1 -- 172.21.2.111:0/3970609782 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20007440 con 0x7ffb240add50 2021-04-03T03:16:09.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb177fe700 1 -- 172.21.2.111:0/3970609782 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb200057e0 con 0x7ffb240add50 2021-04-03T03:16:09.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb24067e90 con 0x7ffb240add50 2021-04-03T03:16:09.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb24157680 con 0x7ffb240add50 2021-04-03T03:16:09.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb24148040 con 0x7ffb240add50 2021-04-03T03:16:09.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb177fe700 1 -- 172.21.2.111:0/3970609782 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ffb20010070 con 0x7ffb240add50 2021-04-03T03:16:09.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb177fe700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb04056ea0 0x7ffb04059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:09.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb177fe700 1 -- 172.21.2.111:0/3970609782 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ffb2006f7b0 con 0x7ffb240add50 2021-04-03T03:16:09.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb2a221700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb04056ea0 0x7ffb04059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:09.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb2a221700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb04056ea0 0x7ffb04059360 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7ffb18002f30 tx=0x7ffb18015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:09.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.164+0000 7ffb177fe700 1 -- 172.21.2.111:0/3970609782 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ffb20044b00 con 0x7ffb240add50 2021-04-03T03:16:09.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7ffb24147e10 con 0x7ffb240add50 2021-04-03T03:16:09.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb177fe700 1 -- 172.21.2.111:0/3970609782 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v103) v1 ==== 123+0+64 (secure 0 0 0) 0x7ffb20018070 con 0x7ffb240add50 2021-04-03T03:16:09.261 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:09.261 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:09.262 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": "cephfs_data", 2021-04-03T03:16:09.262 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_id": 20, 2021-04-03T03:16:09.263 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2 2021-04-03T03:16:09.263 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:09.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb04056ea0 msgr2=0x7ffb04059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb04056ea0 0x7ffb04059360 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7ffb18002f30 tx=0x7ffb18015000).stop 2021-04-03T03:16:09.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240add50 msgr2=0x7ffb240aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:09.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240add50 0x7ffb240aa410 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7ffb240ae520 tx=0x7ffb20007bc0).stop 2021-04-03T03:16:09.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 shutdown_connections 2021-04-03T03:16:09.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ffb04056ea0 0x7ffb04059360 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:09.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ffb240b17b0 0x7ffb240a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ffb240603f0 0x7ffb240a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:09.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 --2- 172.21.2.111:0/3970609782 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ffb240add50 0x7ffb240aa410 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:09.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.256+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 >> 172.21.2.111:0/3970609782 conn(0x7ffb240b61c0 msgr2=0x7ffb241440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:09.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.260+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 shutdown_connections 2021-04-03T03:16:09.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:09.260+0000 7ffb2b223700 1 -- 172.21.2.111:0/3970609782 wait complete. 2021-04-03T03:16:09.272 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-04-03T03:16:09.273 DEBUG:teuthology.orchestra.run.gibba026:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2021-04-03T03:16:09.300 INFO:teuthology.orchestra.run.gibba026.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2021-04-03T03:16:09.301 INFO:teuthology.orchestra.run.gibba026.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:09.301 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:16:09.302 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:09.311 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:09.324 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:09.354 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:09.368 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:09.382 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:16:09.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:09.383 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:09.384 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:09.406 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:16:09.408 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:09.409 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:16:09.409 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:16:09.410 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:09.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15791]: fuse finished with error 0 and tester_r 0 2021-04-03T03:16:09.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:09.487 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:09.488 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:09.536 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:16:09.538 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:16:09.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:09.649 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:09.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:09.696 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:09.697 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:09.707 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:09.708 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:09.708 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:16:09.709 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:16:09.709 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:09.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15312]: fuse finished with error 0 and tester_r 0 2021-04-03T03:16:09.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:09.801 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:09.801 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:09.810 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:16:09.812 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:16:09.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:09.922 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:09.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:09.969 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:09.969 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:16:09.978 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:16:09.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:09.980 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:16:09.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:09.981 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:16:10.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:10.024 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:16:10.024 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:16:10.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8095ec6700 1 Processor -- start 2021-04-03T03:16:10.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8095ec6700 1 -- start start 2021-04-03T03:16:10.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8095ec6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f809005fce0 0x7f80900b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8095ec6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8090143ac0 0x7f809013fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8095ec6700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80900600c0 con 0x7f80900b1b70 2021-04-03T03:16:10.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8095ec6700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809005cee0 con 0x7f809005fce0 2021-04-03T03:16:10.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8095ec6700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80900acb80 con 0x7f8090143ac0 2021-04-03T03:16:10.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f808ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f80900b1b70 0x7f809005c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57018/0 (socket says 172.21.2.111:57018) 2021-04-03T03:16:10.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f808ffff700 1 -- 172.21.2.111:0/1547399227 learned_addr learned my addr 172.21.2.111:0/1547399227 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:10.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.080+0000 7f8094ec4700 1 --2- 172.21.2.111:0/1547399227 >> v2:172.21.2.111:3300/0 conn(0x7f809005fce0 0x7f80900b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808f7fe700 1 -- 172.21.2.111:0/1547399227 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119360246 0 0) 0x7f80900600c0 con 0x7f80900b1b70 2021-04-03T03:16:10.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808f7fe700 1 -- 172.21.2.111:0/1547399227 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8070003240 con 0x7f80900b1b70 2021-04-03T03:16:10.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8094ec4700 1 -- 172.21.2.111:0/1547399227 >> v2:172.21.2.115:3300/0 conn(0x7f8090143ac0 msgr2=0x7f809013fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:10.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8094ec4700 1 --2- 172.21.2.111:0/1547399227 >> v2:172.21.2.115:3300/0 conn(0x7f8090143ac0 0x7f809013fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8094ec4700 1 -- 172.21.2.111:0/1547399227 >> v1:172.21.2.111:6789/0 conn(0x7f80900b1b70 legacy=0x7f809005c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8094ec4700 1 -- 172.21.2.111:0/1547399227 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80900ace20 con 0x7f809005fce0 2021-04-03T03:16:10.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8094ec4700 1 --2- 172.21.2.111:0/1547399227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f809005fce0 0x7f80900b0130 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f8078002f30 tx=0x7f8078006dd0).ready entity=mon.0 client_cookie=e7e752e4e84ce535 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808f7fe700 1 -- 172.21.2.111:0/1547399227 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80780097b0 con 0x7f809005fce0 2021-04-03T03:16:10.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808f7fe700 1 -- 172.21.2.111:0/1547399227 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8078009910 con 0x7f809005fce0 2021-04-03T03:16:10.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808f7fe700 1 -- 172.21.2.111:0/1547399227 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8078009c70 con 0x7f809005fce0 2021-04-03T03:16:10.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/1547399227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f809005fce0 msgr2=0x7f80900b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 --2- 172.21.2.111:0/1547399227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f809005fce0 0x7f80900b0130 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f8078002f30 tx=0x7f8078006dd0).stop 2021-04-03T03:16:10.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/1547399227 shutdown_connections 2021-04-03T03:16:10.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 --2- 172.21.2.111:0/1547399227 >> v2:172.21.2.115:3300/0 conn(0x7f8090143ac0 0x7f809013fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 --2- 172.21.2.111:0/1547399227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f809005fce0 0x7f80900b0130 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/1547399227 >> 172.21.2.111:0/1547399227 conn(0x7f80900b5fb0 msgr2=0x7f80900b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:10.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/1547399227 shutdown_connections 2021-04-03T03:16:10.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/1547399227 wait complete. 2021-04-03T03:16:10.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 Processor -- start 2021-04-03T03:16:10.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- start start 2021-04-03T03:16:10.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f809005fce0 0x7f8090138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f80900b1b70 0x7f8090155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8090143ac0 0x7f8090066a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8090135f40 con 0x7f80900b1b70 2021-04-03T03:16:10.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8090143080 con 0x7f809005fce0 2021-04-03T03:16:10.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80901416f0 con 0x7f8090143ac0 2021-04-03T03:16:10.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f80900b1b70 0x7f8090155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f80900b1b70 0x7f8090155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56232/0 (socket says 172.21.2.111:56232) 2021-04-03T03:16:10.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 -- 172.21.2.111:0/4135252790 learned_addr learned my addr 172.21.2.111:0/4135252790 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:10.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 -- 172.21.2.111:0/4135252790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8090143ac0 msgr2=0x7f8090066a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8090143ac0 0x7f8090066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 -- 172.21.2.111:0/4135252790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f809005fce0 msgr2=0x7f8090138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f809005fce0 0x7f8090138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 -- 172.21.2.111:0/4135252790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8090156290 con 0x7f80900b1b70 2021-04-03T03:16:10.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808ffff700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f80900b1b70 0x7f8090155da0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f80800010d0 tx=0x7f8080007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808dffb700 1 -- 172.21.2.111:0/4135252790 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8080007320 con 0x7f80900b1b70 2021-04-03T03:16:10.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808dffb700 1 -- 172.21.2.111:0/4135252790 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080007480 con 0x7f80900b1b70 2021-04-03T03:16:10.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8090141450 con 0x7f80900b1b70 2021-04-03T03:16:10.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8090066f70 con 0x7f80900b1b70 2021-04-03T03:16:10.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f808dffb700 1 -- 172.21.2.111:0/4135252790 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80800057e0 con 0x7f80900b1b70 2021-04-03T03:16:10.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.084+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80900647c0 con 0x7f80900b1b70 2021-04-03T03:16:10.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.088+0000 7f808dffb700 1 -- 172.21.2.111:0/4135252790 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8080010030 con 0x7f80900b1b70 2021-04-03T03:16:10.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.088+0000 7f808dffb700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8074056f70 0x7f8074059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.088+0000 7f808dffb700 1 -- 172.21.2.111:0/4135252790 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f808006fec0 con 0x7f80900b1b70 2021-04-03T03:16:10.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.088+0000 7f8094ec4700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8074056f70 0x7f8074059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.088+0000 7f8094ec4700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8074056f70 0x7f8074059430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f8078002f30 tx=0x7f8078015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.088+0000 7f808dffb700 1 -- 172.21.2.111:0/4135252790 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f80800452b0 con 0x7f80900b1b70 2021-04-03T03:16:10.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.180+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f80901401a0 con 0x7f80900b1b70 2021-04-03T03:16:10.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f808dffb700 1 -- 172.21.2.111:0/4135252790 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f80800488d0 con 0x7f80900b1b70 2021-04-03T03:16:10.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8074056f70 msgr2=0x7f8074059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8074056f70 0x7f8074059430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f8078002f30 tx=0x7f8078015000).stop 2021-04-03T03:16:10.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f80900b1b70 msgr2=0x7f8090155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f80900b1b70 0x7f8090155da0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f80800010d0 tx=0x7f8080007ab0).stop 2021-04-03T03:16:10.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 shutdown_connections 2021-04-03T03:16:10.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8090143ac0 0x7f8090066a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8074056f70 0x7f8074059430 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f809005fce0 0x7f8090138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 --2- 172.21.2.111:0/4135252790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f80900b1b70 0x7f8090155da0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 >> 172.21.2.111:0/4135252790 conn(0x7f80900b5fb0 msgr2=0x7f80900ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:10.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 shutdown_connections 2021-04-03T03:16:10.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.188+0000 7f8095ec6700 1 -- 172.21.2.111:0/4135252790 wait complete. 2021-04-03T03:16:10.197 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2021-04-03T03:16:10.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 Processor -- start 2021-04-03T03:16:10.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- start start 2021-04-03T03:16:10.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc5000603f0 0x7fc5000b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc50014b700 0x7fc5000b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc5000b12b0 0x7fc5000b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5000b0f60 con 0x7fc5000b12b0 2021-04-03T03:16:10.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5000607d0 con 0x7fc5000603f0 2021-04-03T03:16:10.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000af400 con 0x7fc50014b700 2021-04-03T03:16:10.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc5000b12b0 0x7fc5000b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc5000b12b0 0x7fc5000b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56244/0 (socket says 172.21.2.111:56244) 2021-04-03T03:16:10.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 -- 172.21.2.111:0/1063566164 learned_addr learned my addr 172.21.2.111:0/1063566164 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:10.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 -- 172.21.2.111:0/1063566164 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc50014b700 msgr2=0x7fc5000b0580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:10.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 --2- 172.21.2.111:0/1063566164 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc50014b700 0x7fc5000b0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 -- 172.21.2.111:0/1063566164 >> v2:172.21.2.115:3300/0 conn(0x7fc5000603f0 msgr2=0x7fc5000b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:10.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 --2- 172.21.2.111:0/1063566164 >> v2:172.21.2.115:3300/0 conn(0x7fc5000603f0 0x7fc5000b0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 -- 172.21.2.111:0/1063566164 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000af6a0 con 0x7fc5000b12b0 2021-04-03T03:16:10.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc505832700 1 --2- 172.21.2.111:0/1063566164 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000b12b0 0x7fc5000b0a70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fc4f80032b0 tx=0x7fc4f8009270).ready entity=mon.0 client_cookie=47c5c4a072ca481d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc4effff700 1 -- 172.21.2.111:0/1063566164 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4f800bc00 con 0x7fc5000b12b0 2021-04-03T03:16:10.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc4effff700 1 -- 172.21.2.111:0/1063566164 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f800bd60 con 0x7fc5000b12b0 2021-04-03T03:16:10.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc4effff700 1 -- 172.21.2.111:0/1063566164 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4f8011400 con 0x7fc5000b12b0 2021-04-03T03:16:10.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- 172.21.2.111:0/1063566164 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000b12b0 msgr2=0x7fc5000b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- 172.21.2.111:0/1063566164 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000b12b0 0x7fc5000b0a70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fc4f80032b0 tx=0x7fc4f8009270).stop 2021-04-03T03:16:10.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- 172.21.2.111:0/1063566164 shutdown_connections 2021-04-03T03:16:10.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- 172.21.2.111:0/1063566164 >> v2:172.21.2.115:3300/0 conn(0x7fc5000603f0 0x7fc5000b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- 172.21.2.111:0/1063566164 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000b12b0 0x7fc5000b0a70 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- 172.21.2.111:0/1063566164 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc50014b700 0x7fc5000b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- 172.21.2.111:0/1063566164 >> 172.21.2.111:0/1063566164 conn(0x7fc5000b61c0 msgr2=0x7fc50005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:10.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- 172.21.2.111:0/1063566164 shutdown_connections 2021-04-03T03:16:10.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- 172.21.2.111:0/1063566164 wait complete. 2021-04-03T03:16:10.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 Processor -- start 2021-04-03T03:16:10.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- start start 2021-04-03T03:16:10.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000603f0 0x7fc5000a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5000b12b0 0x7fc500156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc50014b700 0x7fc500156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc500063310 con 0x7fc5000603f0 2021-04-03T03:16:10.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000a9d60 con 0x7fc50014b700 2021-04-03T03:16:10.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.252+0000 7fc506033700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000aa000 con 0x7fc5000b12b0 2021-04-03T03:16:10.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000603f0 0x7fc5000a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000603f0 0x7fc5000a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56248/0 (socket says 172.21.2.111:56248) 2021-04-03T03:16:10.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 -- 172.21.2.111:0/1160922761 learned_addr learned my addr 172.21.2.111:0/1160922761 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:10.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc504830700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5000b12b0 0x7fc500156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 -- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5000b12b0 msgr2=0x7fc500156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5000b12b0 0x7fc500156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505832700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc50014b700 0x7fc500156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 -- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc50014b700 msgr2=0x7fc500156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc50014b700 0x7fc500156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 -- 172.21.2.111:0/1160922761 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5001565b0 con 0x7fc5000603f0 2021-04-03T03:16:10.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505832700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc50014b700 0x7fc500156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:10.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc505031700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000603f0 0x7fc5000a7240 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7fc4fc002f70 tx=0x7fc4fc006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc4ee7fc700 1 -- 172.21.2.111:0/1160922761 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4fc00d570 con 0x7fc5000603f0 2021-04-03T03:16:10.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc4ee7fc700 1 -- 172.21.2.111:0/1160922761 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc00d6d0 con 0x7fc5000603f0 2021-04-03T03:16:10.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5000a7dd0 con 0x7fc5000603f0 2021-04-03T03:16:10.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc4ee7fc700 1 -- 172.21.2.111:0/1160922761 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4fc00da30 con 0x7fc5000603f0 2021-04-03T03:16:10.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5000a99e0 con 0x7fc5000603f0 2021-04-03T03:16:10.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5000abc50 con 0x7fc5000603f0 2021-04-03T03:16:10.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc4ee7fc700 1 -- 172.21.2.111:0/1160922761 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc4fc00f070 con 0x7fc5000603f0 2021-04-03T03:16:10.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc4ee7fc700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4e0056f20 0x7fc4e00593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc4ee7fc700 1 -- 172.21.2.111:0/1160922761 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc4fc06fdd0 con 0x7fc5000603f0 2021-04-03T03:16:10.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc504830700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4e0056f20 0x7fc4e00593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.256+0000 7fc504830700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4e0056f20 0x7fc4e00593e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fc4f0007000 tx=0x7fc4f0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.260+0000 7fc4ee7fc700 1 -- 172.21.2.111:0/1160922761 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc4fc0450b0 con 0x7fc5000603f0 2021-04-03T03:16:10.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.384+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7fc500057660 con 0x7fc5000603f0 2021-04-03T03:16:10.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc4ee7fc700 1 -- 172.21.2.111:0/1160922761 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v324) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc4fc0486d0 con 0x7fc5000603f0 2021-04-03T03:16:10.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4e0056f20 msgr2=0x7fc4e00593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4e0056f20 0x7fc4e00593e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fc4f0007000 tx=0x7fc4f0008040).stop 2021-04-03T03:16:10.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000603f0 msgr2=0x7fc5000a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000603f0 0x7fc5000a7240 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7fc4fc002f70 tx=0x7fc4fc006dd0).stop 2021-04-03T03:16:10.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 shutdown_connections 2021-04-03T03:16:10.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc4e0056f20 0x7fc4e00593e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc50014b700 0x7fc500156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc5000603f0 0x7fc5000a7240 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 --2- 172.21.2.111:0/1160922761 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc5000b12b0 0x7fc500156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 >> 172.21.2.111:0/1160922761 conn(0x7fc5000b61c0 msgr2=0x7fc5000b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:10.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 shutdown_connections 2021-04-03T03:16:10.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.828+0000 7fc506033700 1 -- 172.21.2.111:0/1160922761 wait complete. 2021-04-03T03:16:10.838 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2021-04-03T03:16:10.839 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:16:10.839 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2021-04-03T03:16:10.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 Processor -- start 2021-04-03T03:16:10.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 -- start start 2021-04-03T03:16:10.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d9014b700 0x7f4d900b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4d900b0c90 0x7f4d90147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d900607d0 con 0x7f4d900603f0 2021-04-03T03:16:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d900af400 con 0x7f4d900b0c90 2021-04-03T03:16:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900af6a0 con 0x7f4d9014b700 2021-04-03T03:16:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d95c23700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4d900603f0 0x7f4d900b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57046/0 (socket says 172.21.2.111:57046) 2021-04-03T03:16:10.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d95c23700 1 -- 172.21.2.111:0/3283421441 learned_addr learned my addr 172.21.2.111:0/3283421441 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:10.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 --2- 172.21.2.111:0/3283421441 >> v2:172.21.2.115:3300/0 conn(0x7f4d900b0c90 0x7f4d90147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092574928 0 0) 0x7f4d900607d0 con 0x7f4d900603f0 2021-04-03T03:16:10.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d78003240 con 0x7f4d900603f0 2021-04-03T03:16:10.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d95422700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d9014b700 0x7f4d900b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3449113159 0 0) 0x7f4d78003240 con 0x7f4d900603f0 2021-04-03T03:16:10.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d9014b700 msgr2=0x7f4d900b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:10.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d9014b700 0x7f4d900b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 >> v2:172.21.2.115:3300/0 conn(0x7f4d900b0c90 msgr2=0x7f4d90147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:10.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 --2- 172.21.2.111:0/3283421441 >> v2:172.21.2.115:3300/0 conn(0x7f4d900b0c90 0x7f4d90147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:10.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d900afb40 con 0x7f4d900603f0 2021-04-03T03:16:10.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f4d840016d0 con 0x7f4d900603f0 2021-04-03T03:16:10.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 >> v1:172.21.2.111:6789/0 conn(0x7f4d900603f0 legacy=0x7f4d900b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4d780037b0 0x7f4d78003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d9014b700 0x7f4d780044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d780049d0 0x7f4d78004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900af400 con 0x7f4d780049d0 2021-04-03T03:16:10.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900af6a0 con 0x7f4d780037b0 2021-04-03T03:16:10.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780042a0 con 0x7f4d9014b700 2021-04-03T03:16:10.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d780049d0 0x7f4d78004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 -- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d9014b700 msgr2=0x7f4d780044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:10.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d9014b700 0x7f4d780044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 -- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4d780037b0 msgr2=0x7f4d78003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:10.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4d780037b0 0x7f4d78003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 -- 172.21.2.111:0/3283421441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d78003240 con 0x7f4d780049d0 2021-04-03T03:16:10.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96424700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d780049d0 0x7f4d78004e50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f4d780054b0 tx=0x7f4d880091d0).ready entity=mon.0 client_cookie=f19b5528537f27d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d8800ed00 con 0x7f4d780049d0 2021-04-03T03:16:10.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8800ee60 con 0x7f4d780049d0 2021-04-03T03:16:10.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d94c21700 1 -- 172.21.2.111:0/3283421441 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d880125e0 con 0x7f4d780049d0 2021-04-03T03:16:10.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 -- 172.21.2.111:0/3283421441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d780049d0 msgr2=0x7f4d78004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.900+0000 7f4d96c25700 1 --2- 172.21.2.111:0/3283421441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d780049d0 0x7f4d78004e50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f4d780054b0 tx=0x7f4d880091d0).stop 2021-04-03T03:16:10.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96424700 1 -- 172.21.2.111:0/3283421441 reap_dead start 2021-04-03T03:16:10.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- 172.21.2.111:0/3283421441 shutdown_connections 2021-04-03T03:16:10.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- 172.21.2.111:0/3283421441 >> 172.21.2.111:0/3283421441 conn(0x7f4d900b61c0 msgr2=0x7f4d9005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:10.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- 172.21.2.111:0/3283421441 shutdown_connections 2021-04-03T03:16:10.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- 172.21.2.111:0/3283421441 wait complete. 2021-04-03T03:16:10.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 Processor -- start 2021-04-03T03:16:10.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- start start 2021-04-03T03:16:10.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d900a7a00 0x7f4d900673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d90159b70 0x7f4d900678a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4d90062780 0x7f4d90064620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900a7de0 con 0x7f4d900a7a00 2021-04-03T03:16:10.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d90062f40 con 0x7f4d90062780 2021-04-03T03:16:10.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900ab9d0 con 0x7f4d90159b70 2021-04-03T03:16:10.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d900a7a00 0x7f4d900673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d900a7a00 0x7f4d900673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56266/0 (socket says 172.21.2.111:56266) 2021-04-03T03:16:10.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 -- 172.21.2.111:0/2829684740 learned_addr learned my addr 172.21.2.111:0/2829684740 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:10.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 -- 172.21.2.111:0/2829684740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d90159b70 msgr2=0x7f4d900678a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d90159b70 0x7f4d900678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 -- 172.21.2.111:0/2829684740 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4d90062780 msgr2=0x7f4d90064620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:10.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4d90062780 0x7f4d90064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:10.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 -- 172.21.2.111:0/2829684740 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d900abc70 con 0x7f4d900a7a00 2021-04-03T03:16:10.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95c23700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d900a7a00 0x7f4d900673b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f4d840013e0 tx=0x7f4d84001ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d82ffd700 1 -- 172.21.2.111:0/2829684740 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d84007170 con 0x7f4d900a7a00 2021-04-03T03:16:10.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d82ffd700 1 -- 172.21.2.111:0/2829684740 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d840072d0 con 0x7f4d900a7a00 2021-04-03T03:16:10.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d82ffd700 1 -- 172.21.2.111:0/2829684740 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d840049c0 con 0x7f4d900a7a00 2021-04-03T03:16:10.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d90067d90 con 0x7f4d900a7a00 2021-04-03T03:16:10.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d900ab5e0 con 0x7f4d900a7a00 2021-04-03T03:16:10.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d90057660 con 0x7f4d900a7a00 2021-04-03T03:16:10.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d82ffd700 1 -- 172.21.2.111:0/2829684740 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4d8401b060 con 0x7f4d900a7a00 2021-04-03T03:16:10.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d82ffd700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4d70056f10 0x7f4d700593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:10.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d82ffd700 1 -- 172.21.2.111:0/2829684740 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f4d8406fe70 con 0x7f4d900a7a00 2021-04-03T03:16:10.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.904+0000 7f4d95422700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4d70056f10 0x7f4d700593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:10.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.908+0000 7f4d95422700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4d70056f10 0x7f4d700593d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f4d8c00a000 tx=0x7f4d8c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:10.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:10.908+0000 7f4d82ffd700 1 -- 172.21.2.111:0/2829684740 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4d84045150 con 0x7f4d900a7a00 2021-04-03T03:16:11.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.036+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f4d900b17d0 con 0x7f4d900a7a00 2021-04-03T03:16:11.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d82ffd700 1 -- 172.21.2.111:0/2829684740 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v325) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4d900b17d0 con 0x7f4d900a7a00 2021-04-03T03:16:11.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4d70056f10 msgr2=0x7f4d700593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:11.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4d70056f10 0x7f4d700593d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f4d8c00a000 tx=0x7f4d8c008040).stop 2021-04-03T03:16:11.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d900a7a00 msgr2=0x7f4d900673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:11.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d900a7a00 0x7f4d900673b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f4d840013e0 tx=0x7f4d84001ad0).stop 2021-04-03T03:16:11.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 shutdown_connections 2021-04-03T03:16:11.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4d70056f10 0x7f4d700593d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:11.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4d90062780 0x7f4d90064620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:11.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4d900a7a00 0x7f4d900673b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:11.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 --2- 172.21.2.111:0/2829684740 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4d90159b70 0x7f4d900678a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:11.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 >> 172.21.2.111:0/2829684740 conn(0x7f4d900b61c0 msgr2=0x7f4d9005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:11.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 shutdown_connections 2021-04-03T03:16:11.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.828+0000 7f4d96c25700 1 -- 172.21.2.111:0/2829684740 wait complete. 2021-04-03T03:16:11.839 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:16:11.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 Processor -- start 2021-04-03T03:16:11.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- start start 2021-04-03T03:16:11.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f98940dbba0 0x7f98940dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:11.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98940dc560 con 0x7f989403d560 2021-04-03T03:16:11.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98940dc800 con 0x7f9894041060 2021-04-03T03:16:11.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989403de60 con 0x7f98940dbba0 2021-04-03T03:16:11.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2782700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f989403d560 0x7f9894041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57068/0 (socket says 172.21.2.111:57068) 2021-04-03T03:16:11.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2782700 1 -- 172.21.2.111:0/3379997938 learned_addr learned my addr 172.21.2.111:0/3379997938 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:11.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a1f81700 1 --2- 172.21.2.111:0/3379997938 >> v2:172.21.2.111:3300/0 conn(0x7f98940dbba0 0x7f98940dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:11.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a0f7f700 1 -- 172.21.2.111:0/3379997938 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436892832 0 0) 0x7f98940dc560 con 0x7f989403d560 2021-04-03T03:16:11.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a0f7f700 1 -- 172.21.2.111:0/3379997938 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9884003240 con 0x7f989403d560 2021-04-03T03:16:11.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a1f81700 1 -- 172.21.2.111:0/3379997938 >> v1:172.21.2.115:6789/0 conn(0x7f9894041060 legacy=0x7f989403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:11.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a1f81700 1 -- 172.21.2.111:0/3379997938 >> v1:172.21.2.111:6789/0 conn(0x7f989403d560 legacy=0x7f9894041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:11.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a1f81700 1 -- 172.21.2.111:0/3379997938 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989403da30 con 0x7f98940dbba0 2021-04-03T03:16:11.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a1f81700 1 --2- 172.21.2.111:0/3379997938 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f98940dbba0 0x7f98940dbf80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f9894040a80 tx=0x7f9890006dd0).ready entity=mon.0 client_cookie=cfc12064616a3025 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:11.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a0f7f700 1 -- 172.21.2.111:0/3379997938 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9890009770 con 0x7f98940dbba0 2021-04-03T03:16:11.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a0f7f700 1 -- 172.21.2.111:0/3379997938 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98900098d0 con 0x7f98940dbba0 2021-04-03T03:16:11.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a0f7f700 1 -- 172.21.2.111:0/3379997938 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9890009c70 con 0x7f98940dbba0 2021-04-03T03:16:11.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- 172.21.2.111:0/3379997938 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f98940dbba0 msgr2=0x7f98940dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:11.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3379997938 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f98940dbba0 0x7f98940dbf80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f9894040a80 tx=0x7f9890006dd0).stop 2021-04-03T03:16:11.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- 172.21.2.111:0/3379997938 shutdown_connections 2021-04-03T03:16:11.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3379997938 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f98940dbba0 0x7f98940dbf80 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:11.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- 172.21.2.111:0/3379997938 >> 172.21.2.111:0/3379997938 conn(0x7f98940462b0 msgr2=0x7f98940466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:11.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- 172.21.2.111:0/3379997938 shutdown_connections 2021-04-03T03:16:11.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- 172.21.2.111:0/3379997938 wait complete. 2021-04-03T03:16:11.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 Processor -- start 2021-04-03T03:16:11.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- start start 2021-04-03T03:16:11.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f989403d560 0x7f989403a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:11.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9894041060 0x7f98940d7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:11.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f98940dbba0 0x7f98940a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:11.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98940dda70 con 0x7f9894041060 2021-04-03T03:16:11.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98940dca00 con 0x7f989403d560 2021-04-03T03:16:11.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.896+0000 7f98a2f83700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989409f5b0 con 0x7f98940dbba0 2021-04-03T03:16:11.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9894041060 0x7f98940d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:11.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9894041060 0x7f98940d7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56282/0 (socket says 172.21.2.111:56282) 2021-04-03T03:16:11.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 -- 172.21.2.111:0/3911757329 learned_addr learned my addr 172.21.2.111:0/3911757329 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:11.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 -- 172.21.2.111:0/3911757329 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f98940dbba0 msgr2=0x7f98940a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:11.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f98940dbba0 0x7f98940a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:11.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 -- 172.21.2.111:0/3911757329 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f989403d560 msgr2=0x7f989403a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:11.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f989403d560 0x7f989403a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:11.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 -- 172.21.2.111:0/3911757329 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989409f850 con 0x7f9894041060 2021-04-03T03:16:11.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1780700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9894041060 0x7f98940d7370 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f9898001f60 tx=0x7f9898007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:11.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f988effd700 1 -- 172.21.2.111:0/3911757329 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f989800e4a0 con 0x7f9894041060 2021-04-03T03:16:11.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f988effd700 1 -- 172.21.2.111:0/3911757329 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989800e600 con 0x7f9894041060 2021-04-03T03:16:11.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989409faf0 con 0x7f9894041060 2021-04-03T03:16:11.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98940a31c0 con 0x7f9894041060 2021-04-03T03:16:11.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f988effd700 1 -- 172.21.2.111:0/3911757329 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f989800e960 con 0x7f9894041060 2021-04-03T03:16:11.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989409fc80 con 0x7f9894041060 2021-04-03T03:16:11.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f988effd700 1 -- 172.21.2.111:0/3911757329 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9898019030 con 0x7f9894041060 2021-04-03T03:16:11.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f988effd700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f987c0570a0 0x7f987c059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:11.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f988effd700 1 -- 172.21.2.111:0/3911757329 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f989806f700 con 0x7f9894041060 2021-04-03T03:16:11.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1f81700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f987c0570a0 0x7f987c059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:11.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.900+0000 7f98a1f81700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f987c0570a0 0x7f987c059560 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f9894040a80 tx=0x7f9890015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:11.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.904+0000 7f988effd700 1 -- 172.21.2.111:0/3911757329 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9898044a90 con 0x7f9894041060 2021-04-03T03:16:11.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:11.996+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9894004db0 con 0x7f9894041060 2021-04-03T03:16:12.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.000+0000 7f988effd700 1 -- 172.21.2.111:0/3911757329 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f98980480b0 con 0x7f9894041060 2021-04-03T03:16:12.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.000+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f987c0570a0 msgr2=0x7f987c059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.000+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f987c0570a0 0x7f987c059560 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f9894040a80 tx=0x7f9890015000).stop 2021-04-03T03:16:12.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.000+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9894041060 msgr2=0x7f98940d7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.000+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9894041060 0x7f98940d7370 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f9898001f60 tx=0x7f9898007c10).stop 2021-04-03T03:16:12.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 shutdown_connections 2021-04-03T03:16:12.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f98940dbba0 0x7f98940a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f987c0570a0 0x7f987c059560 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f989403d560 0x7f989403a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 --2- 172.21.2.111:0/3911757329 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9894041060 0x7f98940d7370 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 >> 172.21.2.111:0/3911757329 conn(0x7f98940462b0 msgr2=0x7f9894005c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:12.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 shutdown_connections 2021-04-03T03:16:12.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.004+0000 7f98a2f83700 1 -- 172.21.2.111:0/3911757329 wait complete. 2021-04-03T03:16:12.012 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:16:12.012 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:12.019 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:12.027 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:12.039 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:12.051 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:12.064 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:16:12.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:12.064 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:12.065 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:12.083 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:16:12.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:12.085 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:12.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:12.086 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:12.086 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:12.096 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:16:12.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:12.098 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:12.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:12.098 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:12.099 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:16:12.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:12.109 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:16:12.109 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:16:12.110 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:16:12.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.176+0000 7f31b4ede700 1 Processor -- start 2021-04-03T03:16:12.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- start start 2021-04-03T03:16:12.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f31b00b12b0 0x7f31b01522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b00607d0 con 0x7f31b00603f0 2021-04-03T03:16:12.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b00b0eb0 con 0x7f31b014b700 2021-04-03T03:16:12.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b00af400 con 0x7f31b00b12b0 2021-04-03T03:16:12.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31af7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f31b00603f0 0x7f31b00b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57080/0 (socket says 172.21.2.111:57080) 2021-04-03T03:16:12.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31af7fe700 1 -- 172.21.2.111:0/4270283926 learned_addr learned my addr 172.21.2.111:0/4270283926 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:12.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388548595 0 0) 0x7f31b00607d0 con 0x7f31b00603f0 2021-04-03T03:16:12.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3190003240 con 0x7f31b00603f0 2021-04-03T03:16:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 --2- 172.21.2.111:0/4270283926 >> v2:172.21.2.115:3300/0 conn(0x7f31b00b12b0 0x7f31b01522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2145800848 0 0) 0x7f3190003240 con 0x7f31b00603f0 2021-04-03T03:16:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 >> v2:172.21.2.115:3300/0 conn(0x7f31b00b12b0 msgr2=0x7f31b01522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 --2- 172.21.2.111:0/4270283926 >> v2:172.21.2.115:3300/0 conn(0x7f31b00b12b0 0x7f31b01522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 >> v1:172.21.2.115:6789/0 conn(0x7f31b014b700 legacy=0x7f31b00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00af6a0 con 0x7f31b00603f0 2021-04-03T03:16:12.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f31a00016d0 con 0x7f31b00603f0 2021-04-03T03:16:12.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 >> v1:172.21.2.111:6789/0 conn(0x7f31b00603f0 legacy=0x7f31b00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3190003760 0x7f3190003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31900044e0 0x7f3190004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3190004e50 0x7f319000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00af400 con 0x7f3190004e50 2021-04-03T03:16:12.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190004270 con 0x7f3190003760 2021-04-03T03:16:12.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319000bc20 con 0x7f31900044e0 2021-04-03T03:16:12.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3190004e50 0x7f319000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31af7fe700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3190003760 0x7f3190003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31aeffd700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31900044e0 0x7f3190004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 -- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31900044e0 msgr2=0x7f3190004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f31900044e0 0x7f3190004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 -- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3190003760 msgr2=0x7f3190003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3190003760 0x7f3190003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 -- 172.21.2.111:0/4270283926 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3190003240 con 0x7f3190004e50 2021-04-03T03:16:12.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 -- 172.21.2.111:0/4270283926 reap_dead start 2021-04-03T03:16:12.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31affff700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3190004e50 0x7f319000b730 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f31a800a000 tx=0x7f31a8008040).ready entity=mon.0 client_cookie=68d4b3f9a5966665 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a800dd90 con 0x7f3190004e50 2021-04-03T03:16:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8012420 con 0x7f3190004e50 2021-04-03T03:16:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31ae7fc700 1 -- 172.21.2.111:0/4270283926 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a8012780 con 0x7f3190004e50 2021-04-03T03:16:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- 172.21.2.111:0/4270283926 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3190004e50 msgr2=0x7f319000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3190004e50 0x7f319000b730 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f31a800a000 tx=0x7f31a8008040).stop 2021-04-03T03:16:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- 172.21.2.111:0/4270283926 shutdown_connections 2021-04-03T03:16:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 --2- 172.21.2.111:0/4270283926 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3190004e50 0x7f319000b730 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- 172.21.2.111:0/4270283926 >> 172.21.2.111:0/4270283926 conn(0x7f31b00b61c0 msgr2=0x7f31b005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- 172.21.2.111:0/4270283926 shutdown_connections 2021-04-03T03:16:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.180+0000 7f31b4ede700 1 -- 172.21.2.111:0/4270283926 wait complete. 2021-04-03T03:16:12.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 Processor -- start 2021-04-03T03:16:12.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 -- start start 2021-04-03T03:16:12.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3190004e50 0x7f31b0062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31b00aba90 0x7f31b00ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31b00a75f0 0x7f31b00a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00a7f60 con 0x7f31b00aba90 2021-04-03T03:16:12.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00a9730 con 0x7f31b00a75f0 2021-04-03T03:16:12.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00aa840 con 0x7f3190004e50 2021-04-03T03:16:12.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31b00aba90 0x7f31b00ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31b00aba90 0x7f31b00ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56302/0 (socket says 172.21.2.111:56302) 2021-04-03T03:16:12.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 -- 172.21.2.111:0/773464456 learned_addr learned my addr 172.21.2.111:0/773464456 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:12.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31affff700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31b00a75f0 0x7f31b00a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31af7fe700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3190004e50 0x7f31b0062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 -- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3190004e50 msgr2=0x7f31b0062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3190004e50 0x7f31b0062bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 -- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31b00a75f0 msgr2=0x7f31b00a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31b00a75f0 0x7f31b00a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 -- 172.21.2.111:0/773464456 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b0156ea0 con 0x7f31b00aba90 2021-04-03T03:16:12.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31aeffd700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31b00aba90 0x7f31b00ab3c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f319800b000 tx=0x7f3198009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31acff9700 1 -- 172.21.2.111:0/773464456 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3198011650 con 0x7f31b00aba90 2021-04-03T03:16:12.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31acff9700 1 -- 172.21.2.111:0/773464456 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3198010040 con 0x7f31b00aba90 2021-04-03T03:16:12.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b0157140 con 0x7f31b00aba90 2021-04-03T03:16:12.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b00aa130 con 0x7f31b00aba90 2021-04-03T03:16:12.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31acff9700 1 -- 172.21.2.111:0/773464456 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31980119a0 con 0x7f31b00aba90 2021-04-03T03:16:12.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b00ab8b0 con 0x7f31b00aba90 2021-04-03T03:16:12.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31acff9700 1 -- 172.21.2.111:0/773464456 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3198011b00 con 0x7f31b00aba90 2021-04-03T03:16:12.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31acff9700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3194056ef0 0x7f31940593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31acff9700 1 -- 172.21.2.111:0/773464456 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f319806fb50 con 0x7f31b00aba90 2021-04-03T03:16:12.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31af7fe700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3194056ef0 0x7f31940593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.184+0000 7f31af7fe700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3194056ef0 0x7f31940593b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f31a000d000 tx=0x7f31a0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.188+0000 7f31acff9700 1 -- 172.21.2.111:0/773464456 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3198043c50 con 0x7f31b00aba90 2021-04-03T03:16:12.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f31b01572d0 con 0x7f31b00aba90 2021-04-03T03:16:12.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31acff9700 1 -- 172.21.2.111:0/773464456 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+34048 (secure 0 0 0) 0x7f319802a510 con 0x7f31b00aba90 2021-04-03T03:16:12.284 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:12.284 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:12.284 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 103, 2021-04-03T03:16:12.284 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:16:12.285 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:16:12.285 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:16:03.554720+0000", 2021-04-03T03:16:12.285 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:16:12.285 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:16:12.286 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:16:12.286 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:16:12.286 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:16:12.286 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:16:12.286 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:16:12.287 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:16:12.287 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:16:12.287 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:12.287 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:16:12.288 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:16:12.288 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:16:12.288 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:16:12.288 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:16:12.288 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 20, 2021-04-03T03:16:12.289 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:16:12.289 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:16:12.289 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:16:12.289 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:16:12.290 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:16:12.290 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.290 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:16:12.290 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:16:12.290 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:16:12.291 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:12.291 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:12.291 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:12.291 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:12.291 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:12.292 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:12.292 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:12.292 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:12.292 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:16:12.293 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:16:12.293 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:16:12.293 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:16:12.293 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:16:12.293 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:12.294 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:12.294 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:12.294 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:12.294 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:12.295 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:12.295 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:12.295 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.295 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:16:12.295 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:12.296 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:12.296 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:12.296 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:12.296 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:12.297 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:12.297 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:12.297 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:12.297 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:12.297 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:12.298 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:12.298 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:12.298 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:12.298 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:12.298 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:12.299 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:12.299 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:12.299 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:12.299 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:12.300 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:12.300 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:12.300 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:12.300 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:12.300 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:12.301 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:12.301 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:12.301 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.301 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:12.302 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:12.302 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:12.302 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:12.302 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:12.302 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:12.303 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:12.303 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:12.303 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:12.303 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:12.304 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:12.304 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:16:12.304 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:16:12.304 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.304 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:12.305 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:16:12.305 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.305 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.305 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.306 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:16:12.306 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:16:12.306 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:16:12.306 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:12.306 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:12.307 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:12.307 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:12.307 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:12.307 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:12.308 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:12.308 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:12.308 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:16:12.308 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:16:12.308 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:16:12.309 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:16:12.309 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:16:12.309 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:12.309 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:12.309 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:12.310 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:12.310 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:12.310 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:12.310 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:12.311 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.311 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:16:12.311 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:12.311 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:12.311 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:12.312 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:12.312 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:12.312 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:12.312 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:12.313 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:12.313 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:12.313 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:12.313 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:12.313 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:12.314 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:12.314 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:12.314 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:12.314 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:12.315 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:12.315 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:12.315 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:12.315 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:12.315 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:12.316 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:12.316 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:12.316 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:12.316 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:12.316 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:12.317 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.317 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:12.317 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:12.317 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:12.318 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:12.318 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:12.318 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:12.318 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:12.318 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:12.319 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:12.319 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:12.319 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:12.319 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:16:12.319 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:12.320 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:16:12.320 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.320 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.320 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.320 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 19, 2021-04-03T03:16:12.321 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:16:12.321 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:16:00.429170+0000", 2021-04-03T03:16:12.321 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:12.321 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:12.322 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:12.322 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:12.322 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:12.322 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:12.322 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:12.323 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:12.323 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:16:12.323 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:16:12.323 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:16:12.323 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:16:12.324 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:16:12.324 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:12.324 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:12.324 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:12.324 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:12.325 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:12.325 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:12.325 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:12.325 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.326 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "102", 2021-04-03T03:16:12.326 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:12.326 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:12.326 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:12.326 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:12.327 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:12.327 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:12.327 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:12.327 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:12.327 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:12.328 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:12.328 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:12.328 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:12.328 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:12.328 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:12.329 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:12.329 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:12.329 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:12.329 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:12.329 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:12.330 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:12.330 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:12.330 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:12.330 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:12.330 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:12.331 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:12.331 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:12.331 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.331 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:12.332 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:12.332 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:12.332 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:12.332 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:12.332 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:12.333 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:12.333 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:12.333 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:12.333 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:12.333 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:12.334 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:16:12.334 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:16:12.334 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:16:12.334 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:16:12.335 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.335 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:12.335 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:16:12.335 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:16:12.335 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.336 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.336 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.336 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.336 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 20, 2021-04-03T03:16:12.336 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:16:12.337 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:16:01.518005+0000", 2021-04-03T03:16:12.337 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:12.337 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:12.337 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:12.337 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:12.338 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:12.338 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:12.338 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:12.338 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:12.338 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:16:12.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:16:12.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:16:12.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:16:12.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:16:12.340 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:12.340 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:12.340 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:12.340 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:12.340 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:12.341 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:12.341 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:12.341 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.341 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "103", 2021-04-03T03:16:12.341 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:12.342 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:12.342 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:12.342 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:12.342 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:12.342 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:12.343 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:12.343 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:12.343 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:12.343 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:12.344 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:12.344 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:12.344 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:12.344 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:12.344 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:12.345 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:12.345 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:12.345 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:12.345 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:12.345 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:12.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:12.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:12.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:12.346 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:12.347 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:12.347 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:12.347 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.347 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:12.347 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:12.348 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:12.348 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:12.348 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:12.348 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:12.348 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:12.349 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:12.349 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:12.349 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:12.349 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:12.349 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:16:12.350 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:12.350 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:16:12.350 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:16:12.350 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.351 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.351 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.351 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:12.351 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:16:12.351 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.352 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:16:12.352 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:16:12.352 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.352 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.352 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.353 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.353 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.353 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.353 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.354 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 100, 2021-04-03T03:16:12.354 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.354 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.354 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.354 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.355 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.355 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.355 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:16:12.355 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.356 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.356 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.356 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.356 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:16:12.356 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.357 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.357 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.357 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.357 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.357 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.358 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.358 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.358 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:16:12.358 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.358 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.359 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.359 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.359 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:16:12.359 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.360 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.360 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.360 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.360 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.360 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.361 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.361 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.361 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:16:12.361 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.361 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.362 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.362 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.362 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:16:12.362 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.362 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.363 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.363 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.363 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.363 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.364 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.364 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.364 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:16:12.364 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.364 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.365 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.365 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.365 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:16:12.365 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:12.365 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.366 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.366 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.366 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:16:12.366 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:16:12.366 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:16:12.367 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:16:12.367 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.367 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.367 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.368 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.368 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.368 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.368 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:16:12.368 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:16:12.369 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.369 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.369 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.369 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.369 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.370 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.370 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.370 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 100, 2021-04-03T03:16:12.370 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.370 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.371 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.371 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.371 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.371 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.372 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:16:12.372 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.372 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.372 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.372 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.373 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:16:12.373 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.373 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.373 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.374 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.374 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.374 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.374 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.374 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.375 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:16:12.375 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.375 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.375 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.375 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.376 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:16:12.376 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.376 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.376 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.377 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.377 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.377 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.377 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.378 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.378 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:16:12.378 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.378 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.378 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.379 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.379 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:16:12.379 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.379 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.380 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.380 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.380 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.380 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.380 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.381 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.381 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:16:12.381 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.381 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.381 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.382 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.382 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:16:12.382 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:12.382 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.382 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.383 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.383 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:16:12.383 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:16:12.383 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:16:12.384 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:16:12.384 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.384 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.384 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.384 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.385 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.385 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.385 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:16:12.385 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:16:12.386 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.386 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.386 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.386 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.386 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.387 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.387 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.387 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 99, 2021-04-03T03:16:12.387 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.387 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.388 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.388 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.388 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.388 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.388 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:16:12.389 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.389 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.389 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.389 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.390 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:16:12.390 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.390 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.390 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.390 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.391 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.391 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.391 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.391 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.391 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:16:12.392 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.392 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.392 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.392 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.392 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:16:12.393 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.393 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.393 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.393 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.394 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.394 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.394 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.394 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.394 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:16:12.395 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.395 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.395 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.395 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.395 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:16:12.396 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.396 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.396 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.396 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.396 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.397 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.397 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.397 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.397 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:16:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.398 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:16:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.399 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:16:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:16:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:16:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:16:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.400 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.401 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.401 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.401 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.401 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.402 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:16:12.402 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:16:12.402 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.402 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.402 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.403 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.403 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.403 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.403 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.403 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 100, 2021-04-03T03:16:12.404 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.404 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.404 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.404 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.404 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.405 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.405 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:16:12.405 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.405 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.406 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.406 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.406 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:16:12.406 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.406 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.407 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.407 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.407 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.407 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.407 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.408 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.408 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:16:12.408 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.408 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.409 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.409 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.409 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:16:12.409 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.409 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.410 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.410 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.410 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.410 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.410 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.411 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.411 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:16:12.411 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.411 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.411 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.412 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.412 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:16:12.412 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.412 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.413 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.413 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.413 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.413 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.413 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.414 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.414 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:16:12.414 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.414 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.414 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.415 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.415 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:16:12.415 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:12.415 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.416 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.416 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.416 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:16:12.416 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:16:12.416 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:16:12.417 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:16:12.417 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.417 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.417 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.417 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.418 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.418 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.418 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:16:12.418 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:16:12.418 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.419 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.419 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.419 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.419 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.420 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.420 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.420 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 99, 2021-04-03T03:16:12.420 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.420 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.421 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.421 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.421 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.421 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.422 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:16:12.422 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.422 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.422 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.422 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.423 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:16:12.423 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.423 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.423 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.424 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.424 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.424 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.424 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.424 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.425 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:16:12.425 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.425 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.425 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.425 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.426 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:16:12.426 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.426 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.426 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.426 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.427 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.427 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.427 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.427 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.427 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:16:12.428 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.428 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.428 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.428 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:16:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.429 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.430 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:16:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.431 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.432 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:16:12.432 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:12.432 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.432 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.432 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.433 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:16:12.433 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:16:12.433 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:16:12.433 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:16:12.433 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.434 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.434 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.434 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.434 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.435 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.435 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:16:12.435 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:16:12.435 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.435 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.436 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.436 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.436 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.436 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.436 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.437 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 99, 2021-04-03T03:16:12.437 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.437 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.437 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.437 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.438 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.438 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.438 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:16:12.438 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.438 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.439 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.439 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.439 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:16:12.439 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.440 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.440 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.440 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.440 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.440 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:16:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:16:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:16:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:16:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:16:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:16:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:16:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:16:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:16:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:16:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:16:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:16:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 100, 2021-04-03T03:16:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:16:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:16:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:16:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:16:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:16:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:16:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:16:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:16:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:16:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:16:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:16:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:16:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:16:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:16:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 100, 2021-04-03T03:16:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:16:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.472 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.472 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.472 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:16:12.472 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.473 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.473 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.473 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:12.473 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.474 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.474 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.474 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:16:12.474 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.474 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.475 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.475 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.475 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:16:12.475 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.475 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.476 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.476 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.476 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:12.476 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.477 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.477 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.477 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:16:12.477 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.477 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.478 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.478 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.478 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:16:12.478 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.479 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.479 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.479 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.479 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:12.480 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:12.480 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.480 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:12.480 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:16:12.480 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.481 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.481 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.481 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:12.481 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:16:12.481 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:12.482 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.482 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.482 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.482 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:16:12.482 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:16:12.483 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:16:12.483 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:16:12.483 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:12.483 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:12.484 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:12.484 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:12.484 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.484 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:12.484 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:16:12.485 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.485 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:16:12.485 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.485 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.485 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.486 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.486 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.486 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:16:12.486 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.486 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.487 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.487 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:16:12.487 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.487 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.488 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.488 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.488 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.488 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:16:12.488 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.489 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.489 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.489 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:16:12.489 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.490 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.490 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.490 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.490 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.490 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:16:12.491 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.492 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.493 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:16:12.494 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.494 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.494 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.494 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.494 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.495 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:16:12.495 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.495 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.495 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.495 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:16:12.496 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.496 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.496 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.496 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.496 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.497 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:16:12.497 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.497 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.497 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.498 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:16:12.498 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.498 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.498 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.498 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.499 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.499 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:16:12.499 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.499 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.500 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:12.500 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:16:12.500 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:12.500 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:12.500 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:12.501 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:12.501 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:12.501 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:16:12.501 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:12.501 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.502 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:12.502 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:16:12.502 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:16:12.502 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:16:12.503 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:16:12.503 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:16:12.503 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:16:12.503 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:16:12.503 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:16:12.504 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:16:12.504 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:16:12.504 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:16:12.504 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:16:12.504 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:16:12.505 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:16:12.505 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:16:12.505 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:16:12.505 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:16:12.505 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.506 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:16:12.506 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:16:12.506 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:16:12.506 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:16:12.506 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:16:12.507 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:16:12.507 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:16:12.507 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:12.507 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:12.508 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:16:12.508 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:16:12.508 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:16:12.508 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:16:12.508 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:16:12.509 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:12.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3194056ef0 msgr2=0x7f31940593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3194056ef0 0x7f31940593b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f31a000d000 tx=0x7f31a0009040).stop 2021-04-03T03:16:12.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31b00aba90 msgr2=0x7f31b00ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31b00aba90 0x7f31b00ab3c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f319800b000 tx=0x7f3198009040).stop 2021-04-03T03:16:12.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 shutdown_connections 2021-04-03T03:16:12.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3194056ef0 0x7f31940593b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f31b00a75f0 0x7f31b00a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3190004e50 0x7f31b0062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 --2- 172.21.2.111:0/773464456 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f31b00aba90 0x7f31b00ab3c0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 >> 172.21.2.111:0/773464456 conn(0x7f31b00b61c0 msgr2=0x7f31b00acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:12.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 shutdown_connections 2021-04-03T03:16:12.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.280+0000 7f31b4ede700 1 -- 172.21.2.111:0/773464456 wait complete. 2021-04-03T03:16:12.513 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:12.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 Processor -- start 2021-04-03T03:16:12.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 -- start start 2021-04-03T03:16:12.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f685c14b700 0x7f685c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 0x7f685c147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0607d0 con 0x7f685c0603f0 2021-04-03T03:16:12.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0af400 con 0x7f685c14b700 2021-04-03T03:16:12.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0af6a0 con 0x7f685c0b0c90 2021-04-03T03:16:12.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f685c14b700 0x7f685c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f685c14b700 0x7f685c0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56310/0 (socket says 172.21.2.111:56310) 2021-04-03T03:16:12.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 -- 172.21.2.111:0/593551841 learned_addr learned my addr 172.21.2.111:0/593551841 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:12.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 -- 172.21.2.111:0/593551841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 msgr2=0x7f685c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 --2- 172.21.2.111:0/593551841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 0x7f685c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 -- 172.21.2.111:0/593551841 >> v1:172.21.2.115:6789/0 conn(0x7f685c0603f0 legacy=0x7f685c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 -- 172.21.2.111:0/593551841 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0afb40 con 0x7f685c14b700 2021-04-03T03:16:12.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685bfff700 1 --2- 172.21.2.111:0/593551841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c14b700 0x7f685c0b07a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f685c06a3b0 tx=0x7f6844006e90).ready entity=mon.0 client_cookie=3b59499e587c237e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685b7fe700 1 -- 172.21.2.111:0/593551841 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68440098e0 con 0x7f685c14b700 2021-04-03T03:16:12.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685b7fe700 1 -- 172.21.2.111:0/593551841 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844009a40 con 0x7f685c14b700 2021-04-03T03:16:12.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f685b7fe700 1 -- 172.21.2.111:0/593551841 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6844009de0 con 0x7f685c14b700 2021-04-03T03:16:12.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 -- 172.21.2.111:0/593551841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c14b700 msgr2=0x7f685c0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 --2- 172.21.2.111:0/593551841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c14b700 0x7f685c0b07a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f685c06a3b0 tx=0x7f6844006e90).stop 2021-04-03T03:16:12.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 -- 172.21.2.111:0/593551841 shutdown_connections 2021-04-03T03:16:12.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 --2- 172.21.2.111:0/593551841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 0x7f685c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 --2- 172.21.2.111:0/593551841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c14b700 0x7f685c0b07a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.572+0000 7f6861da7700 1 -- 172.21.2.111:0/593551841 >> 172.21.2.111:0/593551841 conn(0x7f685c0b61c0 msgr2=0x7f685c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:12.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- 172.21.2.111:0/593551841 shutdown_connections 2021-04-03T03:16:12.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- 172.21.2.111:0/593551841 wait complete. 2021-04-03T03:16:12.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 Processor -- start 2021-04-03T03:16:12.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- start start 2021-04-03T03:16:12.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c0603f0 0x7f685c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 0x7f685c0aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f685c14b700 0x7f685c156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0a8b60 con 0x7f685c0603f0 2021-04-03T03:16:12.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c156790 con 0x7f685c14b700 2021-04-03T03:16:12.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c1569f0 con 0x7f685c0b0c90 2021-04-03T03:16:12.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c0603f0 0x7f685c0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c0603f0 0x7f685c0a9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56314/0 (socket says 172.21.2.111:56314) 2021-04-03T03:16:12.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 -- 172.21.2.111:0/553647569 learned_addr learned my addr 172.21.2.111:0/553647569 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:12.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 -- 172.21.2.111:0/553647569 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 msgr2=0x7f685c0aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 0x7f685c0aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 -- 172.21.2.111:0/553647569 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f685c14b700 msgr2=0x7f685c156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f685c14b700 0x7f685c156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 -- 172.21.2.111:0/553647569 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0635f0 con 0x7f685c0603f0 2021-04-03T03:16:12.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6860da5700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c0603f0 0x7f685c0a9c80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f684c002f50 tx=0x7f684c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6859ffb700 1 -- 172.21.2.111:0/553647569 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684c00d4a0 con 0x7f685c0603f0 2021-04-03T03:16:12.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6859ffb700 1 -- 172.21.2.111:0/553647569 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c00d600 con 0x7f685c0603f0 2021-04-03T03:16:12.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6859ffb700 1 -- 172.21.2.111:0/553647569 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684c00d960 con 0x7f685c0603f0 2021-04-03T03:16:12.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0638c0 con 0x7f685c0603f0 2021-04-03T03:16:12.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0a9560 con 0x7f685c0603f0 2021-04-03T03:16:12.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c057660 con 0x7f685c0603f0 2021-04-03T03:16:12.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6859ffb700 1 -- 172.21.2.111:0/553647569 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f684c00f070 con 0x7f685c0603f0 2021-04-03T03:16:12.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6859ffb700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6840057000 0x7f68400594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.576+0000 7f6859ffb700 1 -- 172.21.2.111:0/553647569 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f684c06fe40 con 0x7f685c0603f0 2021-04-03T03:16:12.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.580+0000 7f685bfff700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6840057000 0x7f68400594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.580+0000 7f685bfff700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6840057000 0x7f68400594c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f685c06a3b0 tx=0x7f6844015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.580+0000 7f6859ffb700 1 -- 172.21.2.111:0/553647569 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f684c045120 con 0x7f685c0603f0 2021-04-03T03:16:12.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f685c0686d0 con 0x7f685c0603f0 2021-04-03T03:16:12.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6859ffb700 1 -- 172.21.2.111:0/553647569 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f684c048740 con 0x7f685c0603f0 2021-04-03T03:16:12.706 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:12.706 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":43,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/3373357469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3373357469},{"type":"v1","addr":"172.21.2.115:6833","nonce":3373357469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:02.550193+0000","modified":"2021-04-03T03:16:03.553657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/3012485417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":3012485417},{"type":"v1","addr":"172.21.2.111:6835","nonce":3012485417}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-04-03T03:16:12.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6840057000 msgr2=0x7f68400594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6840057000 0x7f68400594c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f685c06a3b0 tx=0x7f6844015000).stop 2021-04-03T03:16:12.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c0603f0 msgr2=0x7f685c0a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c0603f0 0x7f685c0a9c80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f684c002f50 tx=0x7f684c006dd0).stop 2021-04-03T03:16:12.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 shutdown_connections 2021-04-03T03:16:12.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6840057000 0x7f68400594c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f685c14b700 0x7f685c156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f685c0603f0 0x7f685c0a9c80 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 --2- 172.21.2.111:0/553647569 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f685c0b0c90 0x7f685c0aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 >> 172.21.2.111:0/553647569 conn(0x7f685c0b61c0 msgr2=0x7f685c146a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:12.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 shutdown_connections 2021-04-03T03:16:12.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.704+0000 7f6861da7700 1 -- 172.21.2.111:0/553647569 wait complete. 2021-04-03T03:16:12.710 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 43 2021-04-03T03:16:12.714 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:16:12.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 Processor -- start 2021-04-03T03:16:12.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- start start 2021-04-03T03:16:12.771 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680603f0 0x7f96681447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f96680add50 0x7f9668144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96680607d0 con 0x7f96680b17b0 2021-04-03T03:16:12.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96681451c0 con 0x7f96680add50 2021-04-03T03:16:12.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966814b090 con 0x7f96680603f0 2021-04-03T03:16:12.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f96680add50 0x7f9668144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f96680add50 0x7f9668144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56324/0 (socket says 172.21.2.111:56324) 2021-04-03T03:16:12.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 -- 172.21.2.111:0/913927366 learned_addr learned my addr 172.21.2.111:0/913927366 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:12.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 -- 172.21.2.111:0/913927366 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680603f0 msgr2=0x7f96681447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 --2- 172.21.2.111:0/913927366 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680603f0 0x7f96681447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 -- 172.21.2.111:0/913927366 >> v1:172.21.2.115:6789/0 conn(0x7f96680b17b0 legacy=0x7f9668147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:12.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 -- 172.21.2.111:0/913927366 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966814b330 con 0x7f96680add50 2021-04-03T03:16:12.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f96665e4700 1 --2- 172.21.2.111:0/913927366 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680add50 0x7f9668144cd0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f96680b5600 tx=0x7f9650006e90).ready entity=mon.0 client_cookie=c86e2fdd525b4af6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9665de3700 1 -- 172.21.2.111:0/913927366 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96500098e0 con 0x7f96680add50 2021-04-03T03:16:12.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9665de3700 1 -- 172.21.2.111:0/913927366 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9650009a40 con 0x7f96680add50 2021-04-03T03:16:12.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9665de3700 1 -- 172.21.2.111:0/913927366 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9650009de0 con 0x7f96680add50 2021-04-03T03:16:12.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- 172.21.2.111:0/913927366 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680add50 msgr2=0x7f9668144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 --2- 172.21.2.111:0/913927366 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680add50 0x7f9668144cd0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f96680b5600 tx=0x7f9650006e90).stop 2021-04-03T03:16:12.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- 172.21.2.111:0/913927366 shutdown_connections 2021-04-03T03:16:12.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 --2- 172.21.2.111:0/913927366 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680603f0 0x7f96681447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:12.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 --2- 172.21.2.111:0/913927366 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680add50 0x7f9668144cd0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- 172.21.2.111:0/913927366 >> 172.21.2.111:0/913927366 conn(0x7f96680b61c0 msgr2=0x7f966805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:12.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- 172.21.2.111:0/913927366 shutdown_connections 2021-04-03T03:16:12.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.768+0000 7f9667de7700 1 -- 172.21.2.111:0/913927366 wait complete. 2021-04-03T03:16:12.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 Processor -- start 2021-04-03T03:16:12.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 -- start start 2021-04-03T03:16:12.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680603f0 0x7f96680a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680add50 0x7f96680abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96680b17b0 0x7f96680ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96680a95b0 con 0x7f96680603f0 2021-04-03T03:16:12.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96680a9850 con 0x7f96680b17b0 2021-04-03T03:16:12.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9668063a60 con 0x7f96680add50 2021-04-03T03:16:12.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680603f0 0x7f96680a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680603f0 0x7f96680a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56328/0 (socket says 172.21.2.111:56328) 2021-04-03T03:16:12.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 -- 172.21.2.111:0/3350183590 learned_addr learned my addr 172.21.2.111:0/3350183590 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:12.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f96675e6700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96680b17b0 0x7f96680ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f96665e4700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680add50 0x7f96680abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 -- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680add50 msgr2=0x7f96680abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680add50 0x7f96680abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 -- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96680b17b0 msgr2=0x7f96680ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:12.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96680b17b0 0x7f96680ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:12.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 -- 172.21.2.111:0/3350183590 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96680ab570 con 0x7f96680603f0 2021-04-03T03:16:12.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9666de5700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680603f0 0x7f96680a9b40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f9660002f30 tx=0x7f9660006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f965ffff700 1 -- 172.21.2.111:0/3350183590 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f966000d770 con 0x7f96680603f0 2021-04-03T03:16:12.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f965ffff700 1 -- 172.21.2.111:0/3350183590 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660000e90 con 0x7f96680603f0 2021-04-03T03:16:12.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9668063f00 con 0x7f96680603f0 2021-04-03T03:16:12.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f965ffff700 1 -- 172.21.2.111:0/3350183590 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f966000dac0 con 0x7f96680603f0 2021-04-03T03:16:12.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9668064410 con 0x7f96680603f0 2021-04-03T03:16:12.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96680ac090 con 0x7f96680603f0 2021-04-03T03:16:12.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f965ffff700 1 -- 172.21.2.111:0/3350183590 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9660017030 con 0x7f96680603f0 2021-04-03T03:16:12.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f965ffff700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f964c056f20 0x7f964c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:12.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f965ffff700 1 -- 172.21.2.111:0/3350183590 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f966006f4a0 con 0x7f96680603f0 2021-04-03T03:16:12.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f96665e4700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f964c056f20 0x7f964c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:12.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.772+0000 7f96665e4700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f964c056f20 0x7f964c0593e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f965001a000 tx=0x7f9650015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:12.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.776+0000 7f965ffff700 1 -- 172.21.2.111:0/3350183590 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f96600448e0 con 0x7f96680603f0 2021-04-03T03:16:12.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:12.900+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9668063c20 con 0x7f96680603f0 2021-04-03T03:16:13.839 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:16:13.836+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:16:13.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f965ffff700 1 -- 172.21.2.111:0/3350183590 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9660047f00 con 0x7f96680603f0 2021-04-03T03:16:13.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f964c056f20 msgr2=0x7f964c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:13.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f964c056f20 0x7f964c0593e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f965001a000 tx=0x7f9650015000).stop 2021-04-03T03:16:13.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680603f0 msgr2=0x7f96680a9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:13.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680603f0 0x7f96680a9b40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f9660002f30 tx=0x7f9660006ea0).stop 2021-04-03T03:16:13.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 shutdown_connections 2021-04-03T03:16:13.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f964c056f20 0x7f964c0593e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:13.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f96680b17b0 0x7f96680ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:13.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f96680603f0 0x7f96680a9b40 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:13.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 --2- 172.21.2.111:0/3350183590 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f96680add50 0x7f96680abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:13.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 >> 172.21.2.111:0/3350183590 conn(0x7f96680b61c0 msgr2=0x7f96680b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:13.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 shutdown_connections 2021-04-03T03:16:13.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.852+0000 7f9667de7700 1 -- 172.21.2.111:0/3350183590 wait complete. 2021-04-03T03:16:13.857 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:16:13.861 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:13.865 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:16:13.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 Processor -- start 2021-04-03T03:16:13.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- start start 2021-04-03T03:16:13.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadc0041e50 0x7fadc00056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:13.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fadc0005ba0 0x7fadc00d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:13.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc0041960 con 0x7fadc0041580 2021-04-03T03:16:13.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc00d2c80 con 0x7fadc0041e50 2021-04-03T03:16:13.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc00d1c10 con 0x7fadc0005ba0 2021-04-03T03:16:13.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadc0041e50 0x7fadc00056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:13.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadc0041e50 0x7fadc00056b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56350/0 (socket says 172.21.2.111:56350) 2021-04-03T03:16:13.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 -- 172.21.2.111:0/342264104 learned_addr learned my addr 172.21.2.111:0/342264104 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:13.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 -- 172.21.2.111:0/342264104 >> v2:172.21.2.115:3300/0 conn(0x7fadc0005ba0 msgr2=0x7fadc00d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:13.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 --2- 172.21.2.111:0/342264104 >> v2:172.21.2.115:3300/0 conn(0x7fadc0005ba0 0x7fadc00d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:13.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 -- 172.21.2.111:0/342264104 >> v1:172.21.2.115:6789/0 conn(0x7fadc0041580 legacy=0x7fadc00d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:13.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 -- 172.21.2.111:0/342264104 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00d1e30 con 0x7fadc0041e50 2021-04-03T03:16:13.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc67fc700 1 --2- 172.21.2.111:0/342264104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041e50 0x7fadc00056b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fadc0047320 tx=0x7fadb8006e90).ready entity=mon.0 client_cookie=be39e70e9dfed83 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:13.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc5ffb700 1 -- 172.21.2.111:0/342264104 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb80098f0 con 0x7fadc0041e50 2021-04-03T03:16:13.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc5ffb700 1 -- 172.21.2.111:0/342264104 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8009a50 con 0x7fadc0041e50 2021-04-03T03:16:13.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc5ffb700 1 -- 172.21.2.111:0/342264104 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb8009df0 con 0x7fadc0041e50 2021-04-03T03:16:13.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- 172.21.2.111:0/342264104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041e50 msgr2=0x7fadc00056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:13.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 --2- 172.21.2.111:0/342264104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041e50 0x7fadc00056b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fadc0047320 tx=0x7fadb8006e90).stop 2021-04-03T03:16:13.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- 172.21.2.111:0/342264104 shutdown_connections 2021-04-03T03:16:13.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 --2- 172.21.2.111:0/342264104 >> v2:172.21.2.115:3300/0 conn(0x7fadc0005ba0 0x7fadc00d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:13.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 --2- 172.21.2.111:0/342264104 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041e50 0x7fadc00056b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:13.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- 172.21.2.111:0/342264104 >> 172.21.2.111:0/342264104 conn(0x7fadc00462c0 msgr2=0x7fadc00466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:13.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- 172.21.2.111:0/342264104 shutdown_connections 2021-04-03T03:16:13.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.916+0000 7fadc7fff700 1 -- 172.21.2.111:0/342264104 wait complete. 2021-04-03T03:16:13.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 Processor -- start 2021-04-03T03:16:13.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 -- start start 2021-04-03T03:16:13.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadc0005ba0 0x7fadc003b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:13.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041580 0x7fadc00cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:13.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadc0041e50 0x7fadc00d0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:13.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00cd440 con 0x7fadc0041580 2021-04-03T03:16:13.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00cb2b0 con 0x7fadc0041e50 2021-04-03T03:16:13.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00ccf30 con 0x7fadc0005ba0 2021-04-03T03:16:13.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041580 0x7fadc00cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:13.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041580 0x7fadc00cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56358/0 (socket says 172.21.2.111:56358) 2021-04-03T03:16:13.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 -- 172.21.2.111:0/2196148078 learned_addr learned my addr 172.21.2.111:0/2196148078 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:13.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc77fe700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadc0041e50 0x7fadc00d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:13.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc6ffd700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadc0005ba0 0x7fadc003b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:13.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 -- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadc0005ba0 msgr2=0x7fadc003b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:13.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadc0005ba0 0x7fadc003b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:13.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 -- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadc0041e50 msgr2=0x7fadc00d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:13.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadc0041e50 0x7fadc00d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:13.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 -- 172.21.2.111:0/2196148078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00cb750 con 0x7fadc0041580 2021-04-03T03:16:13.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc67fc700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041580 0x7fadc00cfb60 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fadc0047320 tx=0x7fadb8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:13.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadbffff700 1 -- 172.21.2.111:0/2196148078 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb8021070 con 0x7fadc0041580 2021-04-03T03:16:13.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadbffff700 1 -- 172.21.2.111:0/2196148078 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8011b80 con 0x7fadc0041580 2021-04-03T03:16:13.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadbffff700 1 -- 172.21.2.111:0/2196148078 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb801d450 con 0x7fadc0041580 2021-04-03T03:16:13.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc00cba40 con 0x7fadc0041580 2021-04-03T03:16:13.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc00c9d20 con 0x7fadc0041580 2021-04-03T03:16:13.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc00cd2a0 con 0x7fadc0041580 2021-04-03T03:16:13.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadbffff700 1 -- 172.21.2.111:0/2196148078 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fadb800f070 con 0x7fadc0041580 2021-04-03T03:16:13.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadbffff700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fadac056f20 0x7fadac0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:13.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadbffff700 1 -- 172.21.2.111:0/2196148078 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fadb8071630 con 0x7fadc0041580 2021-04-03T03:16:13.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc6ffd700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fadac056f20 0x7fadac0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:13.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.920+0000 7fadc6ffd700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fadac056f20 0x7fadac0593e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fadb000c000 tx=0x7fadb0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:13.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:13.924+0000 7fadbffff700 1 -- 172.21.2.111:0/2196148078 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fadc00cd2a0 con 0x7fadc0041580 2021-04-03T03:16:14.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadc00cd2a0 con 0x7fadc0041580 2021-04-03T03:16:14.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadbffff700 1 -- 172.21.2.111:0/2196148078 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+1780 (secure 0 0 0) 0x7fadc00cd2a0 con 0x7fadc0041580 2021-04-03T03:16:14.051 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:14.051 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":44,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.115:6833/3373357469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3373357469},{"type":"v1","addr":"172.21.2.115:6833","nonce":3373357469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":44,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:02.550193+0000","modified":"2021-04-03T03:16:13.841208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-04-03T03:16:14.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fadac056f20 msgr2=0x7fadac0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fadac056f20 0x7fadac0593e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fadb000c000 tx=0x7fadb0008040).stop 2021-04-03T03:16:14.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041580 msgr2=0x7fadc00cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041580 0x7fadc00cfb60 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fadc0047320 tx=0x7fadb8015000).stop 2021-04-03T03:16:14.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 shutdown_connections 2021-04-03T03:16:14.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fadac056f20 0x7fadac0593e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadc0041e50 0x7fadc00d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadc0005ba0 0x7fadc003b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 --2- 172.21.2.111:0/2196148078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadc0041580 0x7fadc00cfb60 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 >> 172.21.2.111:0/2196148078 conn(0x7fadc00462c0 msgr2=0x7fadc00062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:14.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 shutdown_connections 2021-04-03T03:16:14.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.048+0000 7fadc7fff700 1 -- 172.21.2.111:0/2196148078 wait complete. 2021-04-03T03:16:14.055 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 44 2021-04-03T03:16:14.059 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:16:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 Processor -- start 2021-04-03T03:16:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 -- start start 2021-04-03T03:16:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4fe40603f0 0x7f4fe41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4fe40add50 0x7f4fe4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 0x7f4fe4147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe41451c0 con 0x7f4fe40603f0 2021-04-03T03:16:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40607d0 con 0x7f4fe40add50 2021-04-03T03:16:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe8f4b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe414b090 con 0x7f4fe40b17b0 2021-04-03T03:16:14.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4fe40603f0 0x7f4fe41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4fe40603f0 0x7f4fe41447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56362/0 (socket says 172.21.2.111:56362) 2021-04-03T03:16:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 -- 172.21.2.111:0/345776201 learned_addr learned my addr 172.21.2.111:0/345776201 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 -- 172.21.2.111:0/345776201 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 msgr2=0x7f4fe4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 --2- 172.21.2.111:0/345776201 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 0x7f4fe4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 -- 172.21.2.111:0/345776201 >> v2:172.21.2.115:3300/0 conn(0x7f4fe40add50 msgr2=0x7f4fe4144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 --2- 172.21.2.111:0/345776201 >> v2:172.21.2.115:3300/0 conn(0x7f4fe40add50 0x7f4fe4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 -- 172.21.2.111:0/345776201 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe414b330 con 0x7f4fe40603f0 2021-04-03T03:16:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.164+0000 7f4fe37fe700 1 --2- 172.21.2.111:0/345776201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe41447e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f4fcc002f30 tx=0x7f4fcc006e90).ready entity=mon.0 client_cookie=2c4e3059d158b8c3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe27fc700 1 -- 172.21.2.111:0/345776201 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fcc009920 con 0x7f4fe40603f0 2021-04-03T03:16:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe27fc700 1 -- 172.21.2.111:0/345776201 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc009a80 con 0x7f4fe40603f0 2021-04-03T03:16:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe27fc700 1 -- 172.21.2.111:0/345776201 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fcc009de0 con 0x7f4fe40603f0 2021-04-03T03:16:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/345776201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 msgr2=0x7f4fe41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/345776201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe41447e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f4fcc002f30 tx=0x7f4fcc006e90).stop 2021-04-03T03:16:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/345776201 shutdown_connections 2021-04-03T03:16:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/345776201 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 0x7f4fe4147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/345776201 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe41447e0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/345776201 >> v2:172.21.2.115:3300/0 conn(0x7f4fe40add50 0x7f4fe4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/345776201 >> 172.21.2.111:0/345776201 conn(0x7f4fe40b61c0 msgr2=0x7f4fe405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/345776201 shutdown_connections 2021-04-03T03:16:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/345776201 wait complete. 2021-04-03T03:16:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 Processor -- start 2021-04-03T03:16:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- start start 2021-04-03T03:16:14.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe40a7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4fe40add50 0x7f4fe4145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 0x7f4fe40ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40a7810 con 0x7f4fe40603f0 2021-04-03T03:16:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40a7ea0 con 0x7f4fe40add50 2021-04-03T03:16:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe415ac90 con 0x7f4fe40b17b0 2021-04-03T03:16:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe40a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe40a7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56368/0 (socket says 172.21.2.111:56368) 2021-04-03T03:16:14.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 -- 172.21.2.111:0/3070686314 learned_addr learned my addr 172.21.2.111:0/3070686314 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:14.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 -- 172.21.2.111:0/3070686314 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 msgr2=0x7f4fe40ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 0x7f4fe40ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 -- 172.21.2.111:0/3070686314 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4fe40add50 msgr2=0x7f4fe4145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4fe40add50 0x7f4fe4145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 -- 172.21.2.111:0/3070686314 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe415aec0 con 0x7f4fe40603f0 2021-04-03T03:16:14.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe37fe700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe40a7320 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f4fcc002f30 tx=0x7f4fcc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:14.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe0ff9700 1 -- 172.21.2.111:0/3070686314 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fcc011d90 con 0x7f4fe40603f0 2021-04-03T03:16:14.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe0ff9700 1 -- 172.21.2.111:0/3070686314 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc01e460 con 0x7f4fe40603f0 2021-04-03T03:16:14.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40a7a00 con 0x7f4fe40603f0 2021-04-03T03:16:14.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe0ff9700 1 -- 172.21.2.111:0/3070686314 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fcc01e7c0 con 0x7f4fe40603f0 2021-04-03T03:16:14.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe41404f0 con 0x7f4fe40603f0 2021-04-03T03:16:14.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40a7b90 con 0x7f4fe40603f0 2021-04-03T03:16:14.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.168+0000 7f4fe0ff9700 1 -- 172.21.2.111:0/3070686314 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4fcc025030 con 0x7f4fe40603f0 2021-04-03T03:16:14.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.172+0000 7f4fe0ff9700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fc8056ea0 0x7f4fc8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.172+0000 7f4fe0ff9700 1 -- 172.21.2.111:0/3070686314 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f4fcc0716c0 con 0x7f4fe40603f0 2021-04-03T03:16:14.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.172+0000 7f4fe2ffd700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fc8056ea0 0x7f4fc8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:14.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.172+0000 7f4fe2ffd700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fc8056ea0 0x7f4fc8059360 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f4fd4002f30 tx=0x7f4fd4006a70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:14.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.172+0000 7f4fe0ff9700 1 -- 172.21.2.111:0/3070686314 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4fcc046a50 con 0x7f4fe40603f0 2021-04-03T03:16:14.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.300+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4fe4057660 con 0x7f4fe40603f0 2021-04-03T03:16:14.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.856+0000 7f4fe0ff9700 1 -- 172.21.2.111:0/3070686314 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4fcc04a070 con 0x7f4fe40603f0 2021-04-03T03:16:14.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fc8056ea0 msgr2=0x7f4fc8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fc8056ea0 0x7f4fc8059360 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f4fd4002f30 tx=0x7f4fd4006a70).stop 2021-04-03T03:16:14.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 msgr2=0x7f4fe40a7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe40a7320 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f4fcc002f30 tx=0x7f4fcc015000).stop 2021-04-03T03:16:14.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 shutdown_connections 2021-04-03T03:16:14.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4fe40b17b0 0x7f4fe40ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4fe40603f0 0x7f4fe40a7320 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4fc8056ea0 0x7f4fc8059360 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 --2- 172.21.2.111:0/3070686314 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4fe40add50 0x7f4fe4145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 >> 172.21.2.111:0/3070686314 conn(0x7f4fe40b61c0 msgr2=0x7f4fe4149830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:14.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 shutdown_connections 2021-04-03T03:16:14.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.860+0000 7f4fe8f4b700 1 -- 172.21.2.111:0/3070686314 wait complete. 2021-04-03T03:16:14.868 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:16:14.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 Processor -- start 2021-04-03T03:16:14.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 -- start start 2021-04-03T03:16:14.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f66a80dbba0 0x7f66a80dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f66a803d560 0x7f66a8041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a80dc560 con 0x7f66a8041060 2021-04-03T03:16:14.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a803de60 con 0x7f66a803d560 2021-04-03T03:16:14.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80dc800 con 0x7f66a80dbba0 2021-04-03T03:16:14.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f66a803d560 0x7f66a8041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:14.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f66a803d560 0x7f66a8041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56380/0 (socket says 172.21.2.111:56380) 2021-04-03T03:16:14.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 -- 172.21.2.111:0/460085414 learned_addr learned my addr 172.21.2.111:0/460085414 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:14.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 -- 172.21.2.111:0/460085414 >> v2:172.21.2.115:3300/0 conn(0x7f66a80dbba0 msgr2=0x7f66a80dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:14.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 --2- 172.21.2.111:0/460085414 >> v2:172.21.2.115:3300/0 conn(0x7f66a80dbba0 0x7f66a80dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 -- 172.21.2.111:0/460085414 >> v1:172.21.2.115:6789/0 conn(0x7f66a8041060 legacy=0x7f66a803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:14.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 -- 172.21.2.111:0/460085414 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a803da30 con 0x7f66a803d560 2021-04-03T03:16:14.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66affff700 1 --2- 172.21.2.111:0/460085414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a803d560 0x7f66a8041650 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f66a80087d0 tx=0x7f66b005ad70).ready entity=mon.0 client_cookie=75ed7b6f9cd17df0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:14.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66ae7fc700 1 -- 172.21.2.111:0/460085414 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b005bda0 con 0x7f66a803d560 2021-04-03T03:16:14.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66ae7fc700 1 -- 172.21.2.111:0/460085414 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b005df60 con 0x7f66a803d560 2021-04-03T03:16:14.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66ae7fc700 1 -- 172.21.2.111:0/460085414 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b00699f0 con 0x7f66a803d560 2021-04-03T03:16:14.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 -- 172.21.2.111:0/460085414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a803d560 msgr2=0x7f66a8041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.924+0000 7f66b4e58700 1 --2- 172.21.2.111:0/460085414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a803d560 0x7f66a8041650 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f66a80087d0 tx=0x7f66b005ad70).stop 2021-04-03T03:16:14.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- 172.21.2.111:0/460085414 shutdown_connections 2021-04-03T03:16:14.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 --2- 172.21.2.111:0/460085414 >> v2:172.21.2.115:3300/0 conn(0x7f66a80dbba0 0x7f66a80dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 --2- 172.21.2.111:0/460085414 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a803d560 0x7f66a8041650 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:14.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- 172.21.2.111:0/460085414 >> 172.21.2.111:0/460085414 conn(0x7f66a80462b0 msgr2=0x7f66a80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:14.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- 172.21.2.111:0/460085414 shutdown_connections 2021-04-03T03:16:14.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- 172.21.2.111:0/460085414 wait complete. 2021-04-03T03:16:14.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 Processor -- start 2021-04-03T03:16:14.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- start start 2021-04-03T03:16:14.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66a803d560 0x7f66a8045a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a8041060 0x7f66a80d6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66a80dbba0 0x7f66a80d6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a803a450 con 0x7f66a8041060 2021-04-03T03:16:14.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a803a6b0 con 0x7f66a80dbba0 2021-04-03T03:16:14.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a803ab50 con 0x7f66a803d560 2021-04-03T03:16:14.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a8041060 0x7f66a80d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:14.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a8041060 0x7f66a80d6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56386/0 (socket says 172.21.2.111:56386) 2021-04-03T03:16:14.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 -- 172.21.2.111:0/2086268788 learned_addr learned my addr 172.21.2.111:0/2086268788 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:14.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 -- 172.21.2.111:0/2086268788 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66a803d560 msgr2=0x7f66a8045a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66a803d560 0x7f66a8045a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 -- 172.21.2.111:0/2086268788 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66a80dbba0 msgr2=0x7f66a80d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:14.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66a80dbba0 0x7f66a80d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:14.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 -- 172.21.2.111:0/2086268788 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a803b7c0 con 0x7f66a8041060 2021-04-03T03:16:14.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66aeffd700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a8041060 0x7f66a80d6270 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f66a0000f30 tx=0x7f66a0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:14.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66acff9700 1 -- 172.21.2.111:0/2086268788 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a000e4a0 con 0x7f66a8041060 2021-04-03T03:16:14.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66acff9700 1 -- 172.21.2.111:0/2086268788 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a000e600 con 0x7f66a8041060 2021-04-03T03:16:14.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66acff9700 1 -- 172.21.2.111:0/2086268788 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a000e960 con 0x7f66a8041060 2021-04-03T03:16:14.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a80d6c50 con 0x7f66a8041060 2021-04-03T03:16:14.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a8039e20 con 0x7f66a8041060 2021-04-03T03:16:14.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a80398a0 con 0x7f66a8041060 2021-04-03T03:16:14.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66acff9700 1 -- 172.21.2.111:0/2086268788 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f66a0010070 con 0x7f66a8041060 2021-04-03T03:16:14.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66acff9700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6690056f20 0x7f66900593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:14.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66acff9700 1 -- 172.21.2.111:0/2086268788 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f66a006fc00 con 0x7f66a8041060 2021-04-03T03:16:14.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.928+0000 7f66af7fe700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6690056f20 0x7f66900593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:14.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.932+0000 7f66af7fe700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6690056f20 0x7f66900593e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f66a4002f30 tx=0x7f66a40069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:14.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:14.932+0000 7f66acff9700 1 -- 172.21.2.111:0/2086268788 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f66a0044e80 con 0x7f66a8041060 2021-04-03T03:16:15.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.024+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66a8004db0 con 0x7f66a8041060 2021-04-03T03:16:15.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66acff9700 1 -- 172.21.2.111:0/2086268788 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 185+0+0 (secure 0 0 0) 0x7f66a00484a0 con 0x7f66a8041060 2021-04-03T03:16:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6690056f20 msgr2=0x7f66900593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6690056f20 0x7f66900593e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f66a4002f30 tx=0x7f66a40069f0).stop 2021-04-03T03:16:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a8041060 msgr2=0x7f66a80d6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a8041060 0x7f66a80d6270 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f66a0000f30 tx=0x7f66a0007c10).stop 2021-04-03T03:16:15.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 shutdown_connections 2021-04-03T03:16:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6690056f20 0x7f66900593e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66a80dbba0 0x7f66a80d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66a803d560 0x7f66a8045a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:15.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 --2- 172.21.2.111:0/2086268788 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66a8041060 0x7f66a80d6270 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 >> 172.21.2.111:0/2086268788 conn(0x7f66a80462b0 msgr2=0x7f66a8041570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 shutdown_connections 2021-04-03T03:16:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.861+0000 7f66b4e58700 1 -- 172.21.2.111:0/2086268788 wait complete. 2021-04-03T03:16:15.866 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:16:15.870 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:16:15.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8dfa421700 1 Processor -- start 2021-04-03T03:16:15.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8dfa421700 1 -- start start 2021-04-03T03:16:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8dfa421700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8df4144ef0 0x7f8df4147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8dfa421700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40607d0 con 0x7f8df40603f0 2021-04-03T03:16:15.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8dfa421700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df414b090 con 0x7f8df40add50 2021-04-03T03:16:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8dfa421700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df414b330 con 0x7f8df4144ef0 2021-04-03T03:16:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8df941f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8df40603f0 0x7f8df41447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57184/0 (socket says 172.21.2.111:57184) 2021-04-03T03:16:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8df941f700 1 -- 172.21.2.111:0/4221475062 learned_addr learned my addr 172.21.2.111:0/4221475062 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:15.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8df9c20700 1 --2- 172.21.2.111:0/4221475062 >> v2:172.21.2.115:3300/0 conn(0x7f8df4144ef0 0x7f8df4147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807676264 0 0) 0x7f8df40607d0 con 0x7f8df40603f0 2021-04-03T03:16:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.929+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc003240 con 0x7f8df40603f0 2021-04-03T03:16:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2537679432 0 0) 0x7f8ddc003240 con 0x7f8df40603f0 2021-04-03T03:16:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 >> v2:172.21.2.115:3300/0 conn(0x7f8df4144ef0 msgr2=0x7f8df4147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:15.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 --2- 172.21.2.111:0/4221475062 >> v2:172.21.2.115:3300/0 conn(0x7f8df4144ef0 0x7f8df4147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 >> v1:172.21.2.115:6789/0 conn(0x7f8df40add50 legacy=0x7f8df4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df414b5d0 con 0x7f8df40603f0 2021-04-03T03:16:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8df00016d0 con 0x7f8df40603f0 2021-04-03T03:16:15.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 >> v1:172.21.2.111:6789/0 conn(0x7f8df40603f0 legacy=0x7f8df41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ddc003780 0x7f8ddc003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc0044e0 0x7f8ddc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ddc004e50 0x7f8ddc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df414b330 con 0x7f8ddc003780 2021-04-03T03:16:15.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc005300 con 0x7f8ddc0044e0 2021-04-03T03:16:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc00bc20 con 0x7f8ddc004e50 2021-04-03T03:16:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ddc003780 0x7f8ddc003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 -- 172.21.2.111:0/4221475062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ddc004e50 msgr2=0x7f8ddc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8ddc004e50 0x7f8ddc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:15.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 -- 172.21.2.111:0/4221475062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc0044e0 msgr2=0x7f8ddc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc0044e0 0x7f8ddc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 -- 172.21.2.111:0/4221475062 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc003240 con 0x7f8ddc003780 2021-04-03T03:16:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df9c20700 1 -- 172.21.2.111:0/4221475062 reap_dead start 2021-04-03T03:16:15.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ddc003780 0x7f8ddc003ba0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f8df00013e0 tx=0x7f8df414b220).ready entity=mon.0 client_cookie=b744b794a3d11940 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df000ce30 con 0x7f8ddc003780 2021-04-03T03:16:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00070d0 con 0x7f8ddc003780 2021-04-03T03:16:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de7fff700 1 -- 172.21.2.111:0/4221475062 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df0007430 con 0x7f8ddc003780 2021-04-03T03:16:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- 172.21.2.111:0/4221475062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ddc003780 msgr2=0x7f8ddc003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:15.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ddc003780 0x7f8ddc003ba0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f8df00013e0 tx=0x7f8df414b220).stop 2021-04-03T03:16:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- 172.21.2.111:0/4221475062 shutdown_connections 2021-04-03T03:16:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 --2- 172.21.2.111:0/4221475062 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8ddc003780 0x7f8ddc003ba0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- 172.21.2.111:0/4221475062 >> 172.21.2.111:0/4221475062 conn(0x7f8df40b61c0 msgr2=0x7f8df405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- 172.21.2.111:0/4221475062 shutdown_connections 2021-04-03T03:16:15.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- 172.21.2.111:0/4221475062 wait complete. 2021-04-03T03:16:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 Processor -- start 2021-04-03T03:16:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- start start 2021-04-03T03:16:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc003780 0x7f8df41473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8df40a8e00 0x7f8df40b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8df41481d0 0x7f8df40ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df414a5b0 con 0x7f8df40a8e00 2021-04-03T03:16:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4148650 con 0x7f8ddc003780 2021-04-03T03:16:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40a8420 con 0x7f8df41481d0 2021-04-03T03:16:15.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8df40a8e00 0x7f8df40b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8df40a8e00 0x7f8df40b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56408/0 (socket says 172.21.2.111:56408) 2021-04-03T03:16:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 -- 172.21.2.111:0/57398034 learned_addr learned my addr 172.21.2.111:0/57398034 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df9c20700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8df41481d0 0x7f8df40ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df941f700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc003780 0x7f8df41473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:15.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 -- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8df41481d0 msgr2=0x7f8df40ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8df41481d0 0x7f8df40ab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 -- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc003780 msgr2=0x7f8df41473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc003780 0x7f8df41473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:15.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 -- 172.21.2.111:0/57398034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40a86c0 con 0x7f8df40a8e00 2021-04-03T03:16:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8df8c1e700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8df40a8e00 0x7f8df40b1e20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f8df40b5600 tx=0x7f8de8007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de67fc700 1 -- 172.21.2.111:0/57398034 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de8010510 con 0x7f8df40a8e00 2021-04-03T03:16:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de67fc700 1 -- 172.21.2.111:0/57398034 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de8010670 con 0x7f8df40a8e00 2021-04-03T03:16:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df40a8960 con 0x7f8df40a8e00 2021-04-03T03:16:15.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8de67fc700 1 -- 172.21.2.111:0/57398034 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de80107e0 con 0x7f8df40a8e00 2021-04-03T03:16:15.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.933+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df40abc70 con 0x7f8df40a8e00 2021-04-03T03:16:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.937+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df40a8af0 con 0x7f8df40a8e00 2021-04-03T03:16:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.937+0000 7f8de67fc700 1 -- 172.21.2.111:0/57398034 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8de8010070 con 0x7f8df40a8e00 2021-04-03T03:16:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.937+0000 7f8de67fc700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8dd4056f20 0x7f8dd40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:15.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.937+0000 7f8de67fc700 1 -- 172.21.2.111:0/57398034 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6064+0+0 (secure 0 0 0) 0x7f8de806fa60 con 0x7f8df40a8e00 2021-04-03T03:16:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.937+0000 7f8df941f700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8dd4056f20 0x7f8dd40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.937+0000 7f8df941f700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8dd4056f20 0x7f8dd40593e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f8df0019000 tx=0x7f8df0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:15.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:15.937+0000 7f8de67fc700 1 -- 172.21.2.111:0/57398034 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8de8044ea0 con 0x7f8df40a8e00 2021-04-03T03:16:16.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.033+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8df4057660 con 0x7f8df40a8e00 2021-04-03T03:16:16.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.869+0000 7f8de67fc700 1 -- 172.21.2.111:0/57398034 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8de80484c0 con 0x7f8df40a8e00 2021-04-03T03:16:16.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8dd4056f20 msgr2=0x7f8dd40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:16.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8dd4056f20 0x7f8dd40593e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f8df0019000 tx=0x7f8df0015000).stop 2021-04-03T03:16:16.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8df40a8e00 msgr2=0x7f8df40b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:16.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8df40a8e00 0x7f8df40b1e20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f8df40b5600 tx=0x7f8de8007bd0).stop 2021-04-03T03:16:16.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 shutdown_connections 2021-04-03T03:16:16.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8df41481d0 0x7f8df40ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:16.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8dd4056f20 0x7f8dd40593e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:16.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8ddc003780 0x7f8df41473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:16.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 --2- 172.21.2.111:0/57398034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8df40a8e00 0x7f8df40b1e20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:16.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 >> 172.21.2.111:0/57398034 conn(0x7f8df40b61c0 msgr2=0x7f8df41430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:16.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 shutdown_connections 2021-04-03T03:16:16.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.873+0000 7f8dfa421700 1 -- 172.21.2.111:0/57398034 wait complete. 2021-04-03T03:16:16.877 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:16:16.885 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:16:16.885 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:16:16.885 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:16:16.885 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:16:16.886 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:16:16.886 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:16:16.886 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:16:16.887 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:16:16.891 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:16:16.891 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:16:16.892 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:16:16.893 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:16:16.934 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:16:16.934 DEBUG:teuthology.parallel:result is None 2021-04-03T03:16:16.935 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:16:16.935 DEBUG:teuthology.parallel:result is None 2021-04-03T03:16:16.936 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:16:16.974 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:16:16.974 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:16:16.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b9a05700 1 Processor -- start 2021-04-03T03:16:16.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b9a05700 1 -- start start 2021-04-03T03:16:17.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b9a05700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 0x7fe4b4147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b9a05700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b40607d0 con 0x7fe4b40603f0 2021-04-03T03:16:17.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b9a05700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b414b090 con 0x7fe4b4144ef0 2021-04-03T03:16:17.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b9a05700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b414b330 con 0x7fe4b40add50 2021-04-03T03:16:17.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b8a03700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe4b40603f0 0x7fe4b41447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57216/0 (socket says 172.21.2.111:57216) 2021-04-03T03:16:17.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b8a03700 1 -- 172.21.2.111:0/3210753188 learned_addr learned my addr 172.21.2.111:0/3210753188 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:17.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b3fff700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 0x7fe4b4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628574767 0 0) 0x7fe4b40607d0 con 0x7fe4b40603f0 2021-04-03T03:16:17.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe494003240 con 0x7fe4b40603f0 2021-04-03T03:16:17.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1655758776 0 0) 0x7fe494003240 con 0x7fe4b40603f0 2021-04-03T03:16:17.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 msgr2=0x7fe4b4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:17.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 0x7fe4b4147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:17.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 >> v1:172.21.2.115:6789/0 conn(0x7fe4b4144ef0 legacy=0x7fe4b4147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:17.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b414b5d0 con 0x7fe4b40603f0 2021-04-03T03:16:17.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe4a40016d0 con 0x7fe4b40603f0 2021-04-03T03:16:17.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 >> v1:172.21.2.111:6789/0 conn(0x7fe4b40603f0 legacy=0x7fe4b41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:17.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe494003760 0x7fe494003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 0x7fe4940044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4940049d0 0x7fe494004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b414b330 con 0x7fe4940049d0 2021-04-03T03:16:17.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe494004200 con 0x7fe494003760 2021-04-03T03:16:17.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49400b730 con 0x7fe4b40add50 2021-04-03T03:16:17.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b9204700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4940049d0 0x7fe494004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b8a03700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe494003760 0x7fe494003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:16.997+0000 7fe4b3fff700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 0x7fe4940044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 msgr2=0x7fe4940044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:17.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 0x7fe4940044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:17.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe494003760 msgr2=0x7fe494003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:17.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe494003760 0x7fe494003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:17.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/3210753188 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494003240 con 0x7fe4940049d0 2021-04-03T03:16:17.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b8a03700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe494003760 0x7fe494003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:17.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b3fff700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40add50 0x7fe4940044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:17.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4940049d0 0x7fe494004e50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7fe4b40b5640 tx=0x7fe4ac007ef0).ready entity=mon.0 client_cookie=93328dd9eb35f18c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:17.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4ac001540 con 0x7fe4940049d0 2021-04-03T03:16:17.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac00c9f0 con 0x7fe4940049d0 2021-04-03T03:16:17.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b37fe700 1 -- 172.21.2.111:0/3210753188 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4ac00cd50 con 0x7fe4940049d0 2021-04-03T03:16:17.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/3210753188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4940049d0 msgr2=0x7fe494004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:17.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 --2- 172.21.2.111:0/3210753188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4940049d0 0x7fe494004e50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7fe4b40b5640 tx=0x7fe4ac007ef0).stop 2021-04-03T03:16:17.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/3210753188 reap_dead start 2021-04-03T03:16:17.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/3210753188 shutdown_connections 2021-04-03T03:16:17.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/3210753188 >> 172.21.2.111:0/3210753188 conn(0x7fe4b40b61c0 msgr2=0x7fe4b405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:17.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/3210753188 shutdown_connections 2021-04-03T03:16:17.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/3210753188 wait complete. 2021-04-03T03:16:17.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 Processor -- start 2021-04-03T03:16:17.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- start start 2021-04-03T03:16:17.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40b3f10 0x7fe4b4063580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe4b41569c0 0x7fe4b40a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4b415a3c0 0x7fe4b415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4159f90 con 0x7fe4b415a3c0 2021-04-03T03:16:17.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40b42f0 con 0x7fe4b41569c0 2021-04-03T03:16:17.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40a8e80 con 0x7fe4b40b3f10 2021-04-03T03:16:17.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4b415a3c0 0x7fe4b415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4b415a3c0 0x7fe4b415a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56440/0 (socket says 172.21.2.111:56440) 2021-04-03T03:16:17.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/2410841559 learned_addr learned my addr 172.21.2.111:0/2410841559 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:17.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b8a03700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40b3f10 0x7fe4b4063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b3fff700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe4b41569c0 0x7fe4b40a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40b3f10 msgr2=0x7fe4b4063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:17.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40b3f10 0x7fe4b4063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:17.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe4b41569c0 msgr2=0x7fe4b40a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:17.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe4b41569c0 0x7fe4b40a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:17.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 -- 172.21.2.111:0/2410841559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b40a9120 con 0x7fe4b415a3c0 2021-04-03T03:16:17.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b3fff700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe4b41569c0 0x7fe4b40a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:17.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9204700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4b415a3c0 0x7fe4b415a850 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fe4b40af1d0 tx=0x7fe4ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:17.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b1ffb700 1 -- 172.21.2.111:0/2410841559 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4ac001540 con 0x7fe4b415a3c0 2021-04-03T03:16:17.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b1ffb700 1 -- 172.21.2.111:0/2410841559 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac01d5a0 con 0x7fe4b415a3c0 2021-04-03T03:16:17.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b415ad40 con 0x7fe4b415a3c0 2021-04-03T03:16:17.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b1ffb700 1 -- 172.21.2.111:0/2410841559 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4ac01d900 con 0x7fe4b415a3c0 2021-04-03T03:16:17.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b40a70b0 con 0x7fe4b415a3c0 2021-04-03T03:16:17.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b415a100 con 0x7fe4b415a3c0 2021-04-03T03:16:17.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b1ffb700 1 -- 172.21.2.111:0/2410841559 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe4ac010070 con 0x7fe4b415a3c0 2021-04-03T03:16:17.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b1ffb700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe498056ea0 0x7fe498059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:17.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b1ffb700 1 -- 172.21.2.111:0/2410841559 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fe4ac070f90 con 0x7fe4b415a3c0 2021-04-03T03:16:17.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b8a03700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe498056ea0 0x7fe498059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:17.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.001+0000 7fe4b8a03700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe498056ea0 0x7fe498059360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fe4a400d000 tx=0x7fe4a400c040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:17.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.005+0000 7fe4b1ffb700 1 -- 172.21.2.111:0/2410841559 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe4b415a100 con 0x7fe4b415a3c0 2021-04-03T03:16:17.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.109+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fe4b40a8cd0 con 0x7fe4b415a3c0 2021-04-03T03:16:17.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.989+0000 7fe4b1ffb700 1 -- 172.21.2.111:0/2410841559 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v108) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe4ac017070 con 0x7fe4b415a3c0 2021-04-03T03:16:17.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe498056ea0 msgr2=0x7fe498059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:17.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe498056ea0 0x7fe498059360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fe4a400d000 tx=0x7fe4a400c040).stop 2021-04-03T03:16:17.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4b415a3c0 msgr2=0x7fe4b415a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:17.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4b415a3c0 0x7fe4b415a850 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fe4b40af1d0 tx=0x7fe4ac015000).stop 2021-04-03T03:16:17.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 shutdown_connections 2021-04-03T03:16:17.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe4b415a3c0 0x7fe4b415a850 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe4b40b3f10 0x7fe4b4063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe498056ea0 0x7fe498059360 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 --2- 172.21.2.111:0/2410841559 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe4b41569c0 0x7fe4b40a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 >> 172.21.2.111:0/2410841559 conn(0x7fe4b40b61c0 msgr2=0x7fe4b40b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 shutdown_connections 2021-04-03T03:16:18.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:17.997+0000 7fe4b9a05700 1 -- 172.21.2.111:0/2410841559 wait complete. 2021-04-03T03:16:18.001 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:16:18.006 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:16:18.007 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:16:18.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 Processor -- start 2021-04-03T03:16:18.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- start start 2021-04-03T03:16:18.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6a480603f0 0x7f6a480b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480af400 con 0x7f6a4814b700 2021-04-03T03:16:18.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480af6a0 con 0x7f6a480b0c90 2021-04-03T03:16:18.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a48060860 con 0x7f6a480603f0 2021-04-03T03:16:18.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4d6ab700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6a480603f0 0x7f6a480b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4d6ab700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6a480603f0 0x7f6a480b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56446/0 (socket says 172.21.2.111:56446) 2021-04-03T03:16:18.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4d6ab700 1 -- 172.21.2.111:0/2539675250 learned_addr learned my addr 172.21.2.111:0/2539675250 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4d6ab700 1 -- 172.21.2.111:0/2539675250 >> v1:172.21.2.115:6789/0 conn(0x7f6a480b0c90 legacy=0x7f6a48147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:18.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4d6ab700 1 -- 172.21.2.111:0/2539675250 >> v1:172.21.2.111:6789/0 conn(0x7f6a4814b700 legacy=0x7f6a480b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4d6ab700 1 -- 172.21.2.111:0/2539675250 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480afb40 con 0x7f6a480603f0 2021-04-03T03:16:18.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4d6ab700 1 --2- 172.21.2.111:0/2539675250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480603f0 0x7f6a480b0090 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f6a480620f0 tx=0x7f6a3c006e90).ready entity=mon.0 client_cookie=1f5918202719fcb4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a3bfff700 1 -- 172.21.2.111:0/2539675250 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3c009980 con 0x7f6a480603f0 2021-04-03T03:16:18.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a3bfff700 1 -- 172.21.2.111:0/2539675250 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c009ae0 con 0x7f6a480603f0 2021-04-03T03:16:18.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a3bfff700 1 -- 172.21.2.111:0/2539675250 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3c009e80 con 0x7f6a480603f0 2021-04-03T03:16:18.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/2539675250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480603f0 msgr2=0x7f6a480b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/2539675250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480603f0 0x7f6a480b0090 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f6a480620f0 tx=0x7f6a3c006e90).stop 2021-04-03T03:16:18.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/2539675250 shutdown_connections 2021-04-03T03:16:18.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/2539675250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480603f0 0x7f6a480b0090 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/2539675250 >> 172.21.2.111:0/2539675250 conn(0x7f6a480b61c0 msgr2=0x7f6a4805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/2539675250 shutdown_connections 2021-04-03T03:16:18.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.105+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/2539675250 wait complete. 2021-04-03T03:16:18.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 Processor -- start 2021-04-03T03:16:18.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 -- start start 2021-04-03T03:16:18.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a480603f0 0x7f6a48157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480b0c90 0x7f6a48062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a4814b700 0x7f6a480a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480ab750 con 0x7f6a480b0c90 2021-04-03T03:16:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480aad00 con 0x7f6a4814b700 2021-04-03T03:16:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480aaf60 con 0x7f6a480603f0 2021-04-03T03:16:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480b0c90 0x7f6a48062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480b0c90 0x7f6a48062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56452/0 (socket says 172.21.2.111:56452) 2021-04-03T03:16:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 -- 172.21.2.111:0/3129563158 learned_addr learned my addr 172.21.2.111:0/3129563158 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 -- 172.21.2.111:0/3129563158 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a480603f0 msgr2=0x7f6a48157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a480603f0 0x7f6a48157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 -- 172.21.2.111:0/3129563158 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a4814b700 msgr2=0x7f6a480a7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a4814b700 0x7f6a480a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 -- 172.21.2.111:0/3129563158 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480ab200 con 0x7f6a480b0c90 2021-04-03T03:16:18.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4ceaa700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480b0c90 0x7f6a48062bf0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f6a4806a3b0 tx=0x7f6a44007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a3a7fc700 1 -- 172.21.2.111:0/3129563158 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4400f470 con 0x7f6a480b0c90 2021-04-03T03:16:18.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a3a7fc700 1 -- 172.21.2.111:0/3129563158 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4400f5d0 con 0x7f6a480b0c90 2021-04-03T03:16:18.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a3a7fc700 1 -- 172.21.2.111:0/3129563158 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4400f930 con 0x7f6a480b0c90 2021-04-03T03:16:18.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a480a9b40 con 0x7f6a480b0c90 2021-04-03T03:16:18.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a480aa090 con 0x7f6a480b0c90 2021-04-03T03:16:18.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a48057660 con 0x7f6a480b0c90 2021-04-03T03:16:18.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a3a7fc700 1 -- 172.21.2.111:0/3129563158 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6a44019030 con 0x7f6a480b0c90 2021-04-03T03:16:18.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a3a7fc700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a28056ea0 0x7f6a28059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a3a7fc700 1 -- 172.21.2.111:0/3129563158 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f6a4406f520 con 0x7f6a480b0c90 2021-04-03T03:16:18.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4d6ab700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a28056ea0 0x7f6a28059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.109+0000 7f6a4d6ab700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a28056ea0 0x7f6a28059360 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f6a480620f0 tx=0x7f6a3c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.113+0000 7f6a3a7fc700 1 -- 172.21.2.111:0/3129563158 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6a44044b30 con 0x7f6a480b0c90 2021-04-03T03:16:18.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6a480aeec0 con 0x7f6a480b0c90 2021-04-03T03:16:18.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a3a7fc700 1 -- 172.21.2.111:0/3129563158 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f6a44048150 con 0x7f6a480b0c90 2021-04-03T03:16:18.241 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:18.242 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:18.242 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:16:18.242 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.243 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:16:18.243 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:16:18.243 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.243 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:18.243 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.244 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.244 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.244 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.244 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.245 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.245 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:16:18.245 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:16:18.245 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.245 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:18.246 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.246 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.246 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.246 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.247 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.247 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:16:18.247 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:16:18.247 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.248 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.248 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.248 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.248 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.249 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.249 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.249 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:16:18.249 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:16:18.250 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.250 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.250 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.250 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.250 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.251 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.251 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:16:18.251 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:16:18.252 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.252 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.252 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.253 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.253 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.253 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:16:18.253 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:16:18.254 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.254 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.254 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.255 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.255 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.255 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:16:18.255 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:16:18.256 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.256 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.256 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.257 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.257 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.257 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:16:18.257 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:16:18.257 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.258 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.258 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.259 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.259 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.259 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:16:18.259 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:16:18.259 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.260 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.260 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.261 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.261 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.261 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:16:18.261 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:16:18.261 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.262 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.262 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.262 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.263 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.263 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:16:18.263 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:16:18.263 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:18.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:16:18.264 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:16:18.264 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.264 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.265 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.265 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:16:18.265 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:16:18.265 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.266 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:18.266 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:16:18.266 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:16:18.266 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.266 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.267 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.267 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:16:18.267 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:16:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:16:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:16:18.271 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.272 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.272 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.272 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:16:18.272 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:16:18.273 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.273 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:16:18.273 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:18.273 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:18.273 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.274 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.274 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.274 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.274 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:16:18.275 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:16:18.275 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.275 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:16:18.275 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:16:18.275 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.276 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.276 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.276 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.276 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:16:18.276 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:16:18.277 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:18.277 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:16:18.277 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:16:18.277 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:18.278 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.278 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.278 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:18.278 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:18.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a28056ea0 msgr2=0x7f6a28059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a28056ea0 0x7f6a28059360 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f6a480620f0 tx=0x7f6a3c015000).stop 2021-04-03T03:16:18.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480b0c90 msgr2=0x7f6a48062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480b0c90 0x7f6a48062bf0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f6a4806a3b0 tx=0x7f6a44007bd0).stop 2021-04-03T03:16:18.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 shutdown_connections 2021-04-03T03:16:18.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6a28056ea0 0x7f6a28059360 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6a4814b700 0x7f6a480a7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6a480603f0 0x7f6a48157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 --2- 172.21.2.111:0/3129563158 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6a480b0c90 0x7f6a48062bf0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 >> 172.21.2.111:0/3129563158 conn(0x7f6a480b61c0 msgr2=0x7f6a4814de00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 shutdown_connections 2021-04-03T03:16:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.241+0000 7f6a4e6ad700 1 -- 172.21.2.111:0/3129563158 wait complete. 2021-04-03T03:16:18.283 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 Processor -- start 2021-04-03T03:16:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- start start 2021-04-03T03:16:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa594006bc0 0x7fa594042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa59403d360 0x7fa5940dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5940dc6e0 con 0x7fa5940dbb90 2021-04-03T03:16:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940d9560 con 0x7fa59403d360 2021-04-03T03:16:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59403dcb0 con 0x7fa594006bc0 2021-04-03T03:16:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa59403d360 0x7fa5940dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa59403d360 0x7fa5940dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56460/0 (socket says 172.21.2.111:56460) 2021-04-03T03:16:18.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 -- 172.21.2.111:0/145280913 learned_addr learned my addr 172.21.2.111:0/145280913 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 -- 172.21.2.111:0/145280913 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa594006bc0 msgr2=0x7fa594042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:18.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 --2- 172.21.2.111:0/145280913 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa594006bc0 0x7fa594042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 -- 172.21.2.111:0/145280913 >> v1:172.21.2.111:6789/0 conn(0x7fa5940dbb90 legacy=0x7fa5940064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 -- 172.21.2.111:0/145280913 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940dc460 con 0x7fa59403d360 2021-04-03T03:16:18.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59b7fe700 1 --2- 172.21.2.111:0/145280913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940dbf70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fa58c002f30 tx=0x7fa58c006e90).ready entity=mon.0 client_cookie=88b8f12528ddfb7b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa599ffb700 1 -- 172.21.2.111:0/145280913 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58c009970 con 0x7fa59403d360 2021-04-03T03:16:18.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa599ffb700 1 -- 172.21.2.111:0/145280913 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c009ad0 con 0x7fa59403d360 2021-04-03T03:16:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa599ffb700 1 -- 172.21.2.111:0/145280913 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58c009e30 con 0x7fa59403d360 2021-04-03T03:16:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- 172.21.2.111:0/145280913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 msgr2=0x7fa5940dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 --2- 172.21.2.111:0/145280913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940dbf70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fa58c002f30 tx=0x7fa58c006e90).stop 2021-04-03T03:16:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- 172.21.2.111:0/145280913 shutdown_connections 2021-04-03T03:16:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 --2- 172.21.2.111:0/145280913 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940dbf70 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 --2- 172.21.2.111:0/145280913 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa594006bc0 0x7fa594042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- 172.21.2.111:0/145280913 >> 172.21.2.111:0/145280913 conn(0x7fa5940462c0 msgr2=0x7fa5940466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- 172.21.2.111:0/145280913 shutdown_connections 2021-04-03T03:16:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- 172.21.2.111:0/145280913 wait complete. 2021-04-03T03:16:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 Processor -- start 2021-04-03T03:16:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.349+0000 7fa59bfff700 1 -- start start 2021-04-03T03:16:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5940dbb90 0x7fa5940e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5940e7460 0x7fa5940e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940d4510 con 0x7fa59403d360 2021-04-03T03:16:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59403c6d0 con 0x7fa5940e7460 2021-04-03T03:16:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940da0b0 con 0x7fa5940dbb90 2021-04-03T03:16:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940398b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56468/0 (socket says 172.21.2.111:56468) 2021-04-03T03:16:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 -- 172.21.2.111:0/3360689394 learned_addr learned my addr 172.21.2.111:0/3360689394 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59a7fc700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5940dbb90 0x7fa5940e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 -- 172.21.2.111:0/3360689394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5940dbb90 msgr2=0x7fa5940e7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5940dbb90 0x7fa5940e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 -- 172.21.2.111:0/3360689394 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5940e7460 msgr2=0x7fa5940e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5940e7460 0x7fa5940e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 -- 172.21.2.111:0/3360689394 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59403c900 con 0x7fa59403d360 2021-04-03T03:16:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59affd700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940398b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fa594046210 tx=0x7fa59c05ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa58bfff700 1 -- 172.21.2.111:0/3360689394 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa59c05df60 con 0x7fa59403d360 2021-04-03T03:16:18.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa58bfff700 1 -- 172.21.2.111:0/3360689394 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c0699f0 con 0x7fa59403d360 2021-04-03T03:16:18.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa58bfff700 1 -- 172.21.2.111:0/3360689394 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa59c05f4b0 con 0x7fa59403d360 2021-04-03T03:16:18.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5940e8560 con 0x7fa59403d360 2021-04-03T03:16:18.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5940db390 con 0x7fa59403d360 2021-04-03T03:16:18.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5940d9f10 con 0x7fa59403d360 2021-04-03T03:16:18.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa58bfff700 1 -- 172.21.2.111:0/3360689394 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa59c05f610 con 0x7fa59403d360 2021-04-03T03:16:18.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa58bfff700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578057050 0x7fa578059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa58bfff700 1 -- 172.21.2.111:0/3360689394 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fa59c0defc0 con 0x7fa59403d360 2021-04-03T03:16:18.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59a7fc700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578057050 0x7fa578059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.353+0000 7fa59a7fc700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578057050 0x7fa578059510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fa590007000 tx=0x7fa590008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.357+0000 7fa58bfff700 1 -- 172.21.2.111:0/3360689394 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa5940d9f10 con 0x7fa59403d360 2021-04-03T03:16:18.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5940d9f10 con 0x7fa59403d360 2021-04-03T03:16:18.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa58bfff700 1 -- 172.21.2.111:0/3360689394 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+13421 (secure 0 0 0) 0x7fa5940d9f10 con 0x7fa59403d360 2021-04-03T03:16:18.450 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:18.450 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":108,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:17.982187+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:18.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578057050 msgr2=0x7fa578059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578057050 0x7fa578059510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fa590007000 tx=0x7fa590008040).stop 2021-04-03T03:16:18.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 msgr2=0x7fa5940398b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940398b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fa594046210 tx=0x7fa59c05ad70).stop 2021-04-03T03:16:18.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 shutdown_connections 2021-04-03T03:16:18.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa578057050 0x7fa578059510 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5940e7460 0x7fa5940e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa59403d360 0x7fa5940398b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 --2- 172.21.2.111:0/3360689394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5940dbb90 0x7fa5940e7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 >> 172.21.2.111:0/3360689394 conn(0x7fa5940462c0 msgr2=0x7fa594042670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 shutdown_connections 2021-04-03T03:16:18.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.449+0000 7fa59bfff700 1 -- 172.21.2.111:0/3360689394 wait complete. 2021-04-03T03:16:18.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:16:18.459 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:18.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 Processor -- start 2021-04-03T03:16:18.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- start start 2021-04-03T03:16:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc0041060 0x7f3dc00dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3dc003d560 0x7f3dc0041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc00dc560 con 0x7f3dc00dbba0 2021-04-03T03:16:18.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc003de60 con 0x7f3dc003d560 2021-04-03T03:16:18.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00dc800 con 0x7f3dc0041060 2021-04-03T03:16:18.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3dc003d560 0x7f3dc0041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3dc003d560 0x7f3dc0041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56476/0 (socket says 172.21.2.111:56476) 2021-04-03T03:16:18.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 -- 172.21.2.111:0/3954181662 learned_addr learned my addr 172.21.2.111:0/3954181662 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 -- 172.21.2.111:0/3954181662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc0041060 msgr2=0x7f3dc00dbf80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:18.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 --2- 172.21.2.111:0/3954181662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc0041060 0x7f3dc00dbf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 -- 172.21.2.111:0/3954181662 >> v1:172.21.2.115:6789/0 conn(0x7f3dc00dbba0 legacy=0x7f3dc003fa80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:18.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 -- 172.21.2.111:0/3954181662 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc003da30 con 0x7f3dc003d560 2021-04-03T03:16:18.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dceb1f700 1 --2- 172.21.2.111:0/3954181662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc003d560 0x7f3dc0041650 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7f3dc003f960 tx=0x7f3dc805ad70).ready entity=mon.0 client_cookie=6517bac65d70ff2c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcd31c700 1 -- 172.21.2.111:0/3954181662 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc805bda0 con 0x7f3dc003d560 2021-04-03T03:16:18.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcd31c700 1 -- 172.21.2.111:0/3954181662 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc805df60 con 0x7f3dc003d560 2021-04-03T03:16:18.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcd31c700 1 -- 172.21.2.111:0/3954181662 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc80699f0 con 0x7f3dc003d560 2021-04-03T03:16:18.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- 172.21.2.111:0/3954181662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc003d560 msgr2=0x7f3dc0041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- 172.21.2.111:0/3954181662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc003d560 0x7f3dc0041650 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7f3dc003f960 tx=0x7f3dc805ad70).stop 2021-04-03T03:16:18.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- 172.21.2.111:0/3954181662 shutdown_connections 2021-04-03T03:16:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- 172.21.2.111:0/3954181662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc003d560 0x7f3dc0041650 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- 172.21.2.111:0/3954181662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc0041060 0x7f3dc00dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- 172.21.2.111:0/3954181662 >> 172.21.2.111:0/3954181662 conn(0x7f3dc00462b0 msgr2=0x7f3dc00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- 172.21.2.111:0/3954181662 shutdown_connections 2021-04-03T03:16:18.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- 172.21.2.111:0/3954181662 wait complete. 2021-04-03T03:16:18.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 Processor -- start 2021-04-03T03:16:18.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- start start 2021-04-03T03:16:18.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc003d560 0x7f3dc00eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc0041060 0x7f3dc00d6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc00dbba0 0x7f3dc00d6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00df000 con 0x7f3dc0041060 2021-04-03T03:16:18.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0039700 con 0x7f3dc00dbba0 2021-04-03T03:16:18.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcf320700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00399a0 con 0x7f3dc003d560 2021-04-03T03:16:18.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc0041060 0x7f3dc00d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc0041060 0x7f3dc00d6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56480/0 (socket says 172.21.2.111:56480) 2021-04-03T03:16:18.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 -- 172.21.2.111:0/476722765 learned_addr learned my addr 172.21.2.111:0/476722765 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 -- 172.21.2.111:0/476722765 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc003d560 msgr2=0x7f3dc00eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.529 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc003d560 0x7f3dc00eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 -- 172.21.2.111:0/476722765 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc00dbba0 msgr2=0x7f3dc00d6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc00dbba0 0x7f3dc00d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 -- 172.21.2.111:0/476722765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc009fd60 con 0x7f3dc0041060 2021-04-03T03:16:18.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3dcdb1d700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc0041060 0x7f3dc00d6360 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f3dc4002f70 tx=0x7f3dc4006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.517+0000 7f3db77fe700 1 -- 172.21.2.111:0/476722765 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc400d570 con 0x7f3dc0041060 2021-04-03T03:16:18.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3db77fe700 1 -- 172.21.2.111:0/476722765 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400d6d0 con 0x7f3dc0041060 2021-04-03T03:16:18.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc00a0010 con 0x7f3dc0041060 2021-04-03T03:16:18.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc00a04a0 con 0x7f3dc0041060 2021-04-03T03:16:18.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3db77fe700 1 -- 172.21.2.111:0/476722765 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc400da30 con 0x7f3dc0041060 2021-04-03T03:16:18.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc0004db0 con 0x7f3dc0041060 2021-04-03T03:16:18.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3db77fe700 1 -- 172.21.2.111:0/476722765 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3dc400f070 con 0x7f3dc0041060 2021-04-03T03:16:18.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3db77fe700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3da8056f20 0x7f3da80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3db77fe700 1 -- 172.21.2.111:0/476722765 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f3dc406fa00 con 0x7f3dc0041060 2021-04-03T03:16:18.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3dce31e700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3da8056f20 0x7f3da80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3dce31e700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3da8056f20 0x7f3da80593e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f3db8000e90 tx=0x7f3db8006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.521+0000 7f3db77fe700 1 -- 172.21.2.111:0/476722765 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3dc4045010 con 0x7f3dc0041060 2021-04-03T03:16:18.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.613+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3dc0039b60 con 0x7f3dc0041060 2021-04-03T03:16:18.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.613+0000 7f3db77fe700 1 -- 172.21.2.111:0/476722765 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v108) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3dc4048630 con 0x7f3dc0041060 2021-04-03T03:16:18.616 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:18.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.613+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3da8056f20 msgr2=0x7f3da80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.613+0000 7f3dcf320700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3da8056f20 0x7f3da80593e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f3db8000e90 tx=0x7f3db8006910).stop 2021-04-03T03:16:18.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.613+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc0041060 msgr2=0x7f3dc00d6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.613+0000 7f3dcf320700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc0041060 0x7f3dc00d6360 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f3dc4002f70 tx=0x7f3dc4006de0).stop 2021-04-03T03:16:18.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 shutdown_connections 2021-04-03T03:16:18.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3da8056f20 0x7f3da80593e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3dc00dbba0 0x7f3dc00d6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3dc003d560 0x7f3dc00eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 --2- 172.21.2.111:0/476722765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3dc0041060 0x7f3dc00d6360 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 >> 172.21.2.111:0/476722765 conn(0x7f3dc00462b0 msgr2=0x7f3dc0041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 shutdown_connections 2021-04-03T03:16:18.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.617+0000 7f3dcf320700 1 -- 172.21.2.111:0/476722765 wait complete. 2021-04-03T03:16:18.624 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:18.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 Processor -- start 2021-04-03T03:16:18.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- start start 2021-04-03T03:16:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11100603f0 0x7f11101447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f11100b17b0 0x7f1110144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11100607d0 con 0x7f11100add50 2021-04-03T03:16:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11101451c0 con 0x7f11100603f0 2021-04-03T03:16:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111014b090 con 0x7f11100b17b0 2021-04-03T03:16:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11100603f0 0x7f11101447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11100603f0 0x7f11101447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56488/0 (socket says 172.21.2.111:56488) 2021-04-03T03:16:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 -- 172.21.2.111:0/3656268109 learned_addr learned my addr 172.21.2.111:0/3656268109 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 -- 172.21.2.111:0/3656268109 >> v2:172.21.2.115:3300/0 conn(0x7f11100b17b0 msgr2=0x7f1110144cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 --2- 172.21.2.111:0/3656268109 >> v2:172.21.2.115:3300/0 conn(0x7f11100b17b0 0x7f1110144cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 -- 172.21.2.111:0/3656268109 >> v1:172.21.2.115:6789/0 conn(0x7f11100add50 legacy=0x7f1110147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 -- 172.21.2.111:0/3656268109 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111014b330 con 0x7f11100603f0 2021-04-03T03:16:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110effd700 1 --2- 172.21.2.111:0/3656268109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100603f0 0x7f11101447e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f11100ae520 tx=0x7f10f8006e90).ready entity=mon.0 client_cookie=fdc63d584cb5db87 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110dffb700 1 -- 172.21.2.111:0/3656268109 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10f8009930 con 0x7f11100603f0 2021-04-03T03:16:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110dffb700 1 -- 172.21.2.111:0/3656268109 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f8009a90 con 0x7f11100603f0 2021-04-03T03:16:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110dffb700 1 -- 172.21.2.111:0/3656268109 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10f8009e30 con 0x7f11100603f0 2021-04-03T03:16:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- 172.21.2.111:0/3656268109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100603f0 msgr2=0x7f11101447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 --2- 172.21.2.111:0/3656268109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100603f0 0x7f11101447e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f11100ae520 tx=0x7f10f8006e90).stop 2021-04-03T03:16:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- 172.21.2.111:0/3656268109 shutdown_connections 2021-04-03T03:16:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 --2- 172.21.2.111:0/3656268109 >> v2:172.21.2.115:3300/0 conn(0x7f11100b17b0 0x7f1110144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 --2- 172.21.2.111:0/3656268109 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100603f0 0x7f11101447e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- 172.21.2.111:0/3656268109 >> 172.21.2.111:0/3656268109 conn(0x7f11100b61c0 msgr2=0x7f111005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- 172.21.2.111:0/3656268109 shutdown_connections 2021-04-03T03:16:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- 172.21.2.111:0/3656268109 wait complete. 2021-04-03T03:16:18.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 Processor -- start 2021-04-03T03:16:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.685+0000 7f110ffff700 1 -- start start 2021-04-03T03:16:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11100603f0 0x7f1110143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100add50 0x7f1110141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11100b17b0 0x7f11101424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100a8430 con 0x7f11100add50 2021-04-03T03:16:18.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111014a240 con 0x7f11100b17b0 2021-04-03T03:16:18.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11100a6f20 con 0x7f11100603f0 2021-04-03T03:16:18.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100add50 0x7f1110141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100add50 0x7f1110141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56492/0 (socket says 172.21.2.111:56492) 2021-04-03T03:16:18.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 -- 172.21.2.111:0/570272668 learned_addr learned my addr 172.21.2.111:0/570272668 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 -- 172.21.2.111:0/570272668 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11100603f0 msgr2=0x7f1110143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11100603f0 0x7f1110143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 -- 172.21.2.111:0/570272668 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11100b17b0 msgr2=0x7f11101424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11100b17b0 0x7f11101424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 -- 172.21.2.111:0/570272668 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11101429c0 con 0x7f11100add50 2021-04-03T03:16:18.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110e7fc700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100add50 0x7f1110141fe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f1100002f70 tx=0x7f1100006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f1107fff700 1 -- 172.21.2.111:0/570272668 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f110000d4a0 con 0x7f11100add50 2021-04-03T03:16:18.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f1107fff700 1 -- 172.21.2.111:0/570272668 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110000d600 con 0x7f11100add50 2021-04-03T03:16:18.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11100a8b00 con 0x7f11100add50 2021-04-03T03:16:18.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1110143bc0 con 0x7f11100add50 2021-04-03T03:16:18.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f1107fff700 1 -- 172.21.2.111:0/570272668 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f110000d960 con 0x7f11100add50 2021-04-03T03:16:18.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11100a85a0 con 0x7f11100add50 2021-04-03T03:16:18.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f1107fff700 1 -- 172.21.2.111:0/570272668 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f110000f070 con 0x7f11100add50 2021-04-03T03:16:18.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f1107fff700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f10f4057000 0x7f10f40594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f1107fff700 1 -- 172.21.2.111:0/570272668 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f110006f9b0 con 0x7f11100add50 2021-04-03T03:16:18.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110effd700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f10f4057000 0x7f10f40594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.689+0000 7f110effd700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f10f4057000 0x7f10f40594c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f11100ae520 tx=0x7f10f8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.693+0000 7f1107fff700 1 -- 172.21.2.111:0/570272668 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f11100a85a0 con 0x7f11100add50 2021-04-03T03:16:18.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f11100a8780 con 0x7f11100add50 2021-04-03T03:16:18.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f1107fff700 1 -- 172.21.2.111:0/570272668 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1100044fc0 con 0x7f11100add50 2021-04-03T03:16:18.786 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:18.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f10f4057000 msgr2=0x7f10f40594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f10f4057000 0x7f10f40594c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f11100ae520 tx=0x7f10f8015000).stop 2021-04-03T03:16:18.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100add50 msgr2=0x7f1110141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100add50 0x7f1110141fe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f1100002f70 tx=0x7f1100006dd0).stop 2021-04-03T03:16:18.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 shutdown_connections 2021-04-03T03:16:18.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f10f4057000 0x7f10f40594c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11100b17b0 0x7f11101424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11100603f0 0x7f1110143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:18.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 --2- 172.21.2.111:0/570272668 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11100add50 0x7f1110141fe0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 >> 172.21.2.111:0/570272668 conn(0x7f11100b61c0 msgr2=0x7f111014cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 shutdown_connections 2021-04-03T03:16:18.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.785+0000 7f110ffff700 1 -- 172.21.2.111:0/570272668 wait complete. 2021-04-03T03:16:18.794 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:16:18.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1be146700 1 Processor -- start 2021-04-03T03:16:18.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1be146700 1 -- start start 2021-04-03T03:16:18.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1be146700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b0041a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1be146700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1b00d5e10 0x7fe1b00d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1be146700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b00492e0 con 0x7fe1b00431a0 2021-04-03T03:16:18.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1be146700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b003fac0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1be146700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b003fd60 con 0x7fe1b0048f00 2021-04-03T03:16:18.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bd945700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1b00d5e10 0x7fe1b00d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bc943700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe1b00431a0 0x7fe1b00d5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57296/0 (socket says 172.21.2.111:57296) 2021-04-03T03:16:18.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bc943700 1 -- 172.21.2.111:0/893767024 learned_addr learned my addr 172.21.2.111:0/893767024 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:18.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bd945700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1b00d5e10 0x7fe1b00d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56502/0 (socket says 172.21.2.111:56502) 2021-04-03T03:16:18.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bd144700 1 --2- 172.21.2.111:0/893767024 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b0041a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bd945700 1 -- 172.21.2.111:0/893767024 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 msgr2=0x7fe1b0041a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:18.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bd945700 1 --2- 172.21.2.111:0/893767024 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b0041a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.889+0000 7fe1bd945700 1 -- 172.21.2.111:0/893767024 >> v1:172.21.2.111:6789/0 conn(0x7fe1b00431a0 legacy=0x7fe1b00d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 -- 172.21.2.111:0/893767024 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00d5260 con 0x7fe1b00d5e10 2021-04-03T03:16:18.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd144700 1 --2- 172.21.2.111:0/893767024 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b0041a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:18.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 --2- 172.21.2.111:0/893767024 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d1bf0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7fe1b806b5d0 tx=0x7fe1b805ad70).ready entity=mon.0 client_cookie=b6657ca9a823cf6d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1a7fff700 1 -- 172.21.2.111:0/893767024 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1b805bda0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1a7fff700 1 -- 172.21.2.111:0/893767024 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b805df60 con 0x7fe1b00d5e10 2021-04-03T03:16:18.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1a7fff700 1 -- 172.21.2.111:0/893767024 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1b80699f0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- 172.21.2.111:0/893767024 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 msgr2=0x7fe1b00d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 --2- 172.21.2.111:0/893767024 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d1bf0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7fe1b806b5d0 tx=0x7fe1b805ad70).stop 2021-04-03T03:16:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- 172.21.2.111:0/893767024 shutdown_connections 2021-04-03T03:16:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 --2- 172.21.2.111:0/893767024 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d1bf0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 --2- 172.21.2.111:0/893767024 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b0041a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- 172.21.2.111:0/893767024 >> 172.21.2.111:0/893767024 conn(0x7fe1b000f2c0 msgr2=0x7fe1b00484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:18.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- 172.21.2.111:0/893767024 shutdown_connections 2021-04-03T03:16:18.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- 172.21.2.111:0/893767024 wait complete. 2021-04-03T03:16:18.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 Processor -- start 2021-04-03T03:16:18.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- start start 2021-04-03T03:16:18.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1b00431a0 0x7fe1b00d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b00d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00d92a0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00a4330 con 0x7fe1b00431a0 2021-04-03T03:16:18.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00d9000 con 0x7fe1b0048f00 2021-04-03T03:16:18.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56510/0 (socket says 172.21.2.111:56510) 2021-04-03T03:16:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 -- 172.21.2.111:0/2966127348 learned_addr learned my addr 172.21.2.111:0/2966127348 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd144700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1b00431a0 0x7fe1b00d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bc943700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b00d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 -- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 msgr2=0x7fe1b00d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b00d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 -- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1b00431a0 msgr2=0x7fe1b00d3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1b00431a0 0x7fe1b00d3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 -- 172.21.2.111:0/2966127348 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00d7b30 con 0x7fe1b00d5e10 2021-04-03T03:16:18.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd144700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1b00431a0 0x7fe1b00d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bc943700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b00d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1bd945700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d0f60 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fe1b806b5d0 tx=0x7fe1b8064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1a67fc700 1 -- 172.21.2.111:0/2966127348 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1b80441b0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1a67fc700 1 -- 172.21.2.111:0/2966127348 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b805f2c0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1a67fc700 1 -- 172.21.2.111:0/2966127348 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1b805f620 con 0x7fe1b00d5e10 2021-04-03T03:16:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b00d1450 con 0x7fe1b00d5e10 2021-04-03T03:16:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.893+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b00d8090 con 0x7fe1b00d5e10 2021-04-03T03:16:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.897+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b0004da0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.897+0000 7fe1a67fc700 1 -- 172.21.2.111:0/2966127348 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe1b8098060 con 0x7fe1b00d5e10 2021-04-03T03:16:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.897+0000 7fe1a67fc700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe198056fb0 0x7fe198059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.897+0000 7fe1a67fc700 1 -- 172.21.2.111:0/2966127348 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fe1b80e08e0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.897+0000 7fe1bd144700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe198056fb0 0x7fe198059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:18.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.897+0000 7fe1bd144700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe198056fb0 0x7fe198059470 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fe1b4008000 tx=0x7fe1b4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:18.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.897+0000 7fe1a67fc700 1 -- 172.21.2.111:0/2966127348 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe1b80b60b0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe1b0046bc0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1a67fc700 1 -- 172.21.2.111:0/2966127348 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe1b80b96d0 con 0x7fe1b00d5e10 2021-04-03T03:16:18.995 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:18.995 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:16:18.995 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.995 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:16:18.996 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:16:18.996 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:18.996 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:18.996 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:16:18.997 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:16:18.997 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:18.997 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:16:18.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe198056fb0 msgr2=0x7fe198059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe198056fb0 0x7fe198059470 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fe1b4008000 tx=0x7fe1b4008040).stop 2021-04-03T03:16:18.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 msgr2=0x7fe1b00d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:18.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d0f60 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fe1b806b5d0 tx=0x7fe1b8064030).stop 2021-04-03T03:16:18.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 shutdown_connections 2021-04-03T03:16:18.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1b00d5e10 0x7fe1b00d0f60 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:18.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe198056fb0 0x7fe198059470 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:19.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1b00431a0 0x7fe1b00d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:19.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 --2- 172.21.2.111:0/2966127348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1b0048f00 0x7fe1b00d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:19.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 >> 172.21.2.111:0/2966127348 conn(0x7fe1b000f2c0 msgr2=0x7fe1b00058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:19.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 shutdown_connections 2021-04-03T03:16:19.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:18.993+0000 7fe1be146700 1 -- 172.21.2.111:0/2966127348 wait complete. 2021-04-03T03:16:19.002 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:16:19.003 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:16:19.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 Processor -- start 2021-04-03T03:16:19.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- start start 2021-04-03T03:16:19.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f53580603f0 0x7f53581447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f53580b17b0 0x7f5358144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53580607d0 con 0x7f53580add50 2021-04-03T03:16:19.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53581451c0 con 0x7f53580b17b0 2021-04-03T03:16:19.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535814b090 con 0x7f53580603f0 2021-04-03T03:16:19.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f53580b17b0 0x7f5358144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:19.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f53580b17b0 0x7f5358144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56514/0 (socket says 172.21.2.111:56514) 2021-04-03T03:16:19.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 -- 172.21.2.111:0/3426887482 learned_addr learned my addr 172.21.2.111:0/3426887482 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:19.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 -- 172.21.2.111:0/3426887482 >> v2:172.21.2.115:3300/0 conn(0x7f53580603f0 msgr2=0x7f53581447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:19.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 --2- 172.21.2.111:0/3426887482 >> v2:172.21.2.115:3300/0 conn(0x7f53580603f0 0x7f53581447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 -- 172.21.2.111:0/3426887482 >> v1:172.21.2.111:6789/0 conn(0x7f53580add50 legacy=0x7f5358147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 -- 172.21.2.111:0/3426887482 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535814b330 con 0x7f53580b17b0 2021-04-03T03:16:19.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f5357fff700 1 --2- 172.21.2.111:0/3426887482 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580b17b0 0x7f5358144cd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f53500032b0 tx=0x7f5350009270).ready entity=mon.0 client_cookie=f28346c1472b4354 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:19.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f53567fc700 1 -- 172.21.2.111:0/3426887482 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535000bc00 con 0x7f53580b17b0 2021-04-03T03:16:19.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f53567fc700 1 -- 172.21.2.111:0/3426887482 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535000bd60 con 0x7f53580b17b0 2021-04-03T03:16:19.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f53567fc700 1 -- 172.21.2.111:0/3426887482 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5350011400 con 0x7f53580b17b0 2021-04-03T03:16:19.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- 172.21.2.111:0/3426887482 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580b17b0 msgr2=0x7f5358144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 --2- 172.21.2.111:0/3426887482 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580b17b0 0x7f5358144cd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f53500032b0 tx=0x7f5350009270).stop 2021-04-03T03:16:19.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- 172.21.2.111:0/3426887482 shutdown_connections 2021-04-03T03:16:19.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 --2- 172.21.2.111:0/3426887482 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580b17b0 0x7f5358144cd0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:19.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 --2- 172.21.2.111:0/3426887482 >> v2:172.21.2.115:3300/0 conn(0x7f53580603f0 0x7f53581447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- 172.21.2.111:0/3426887482 >> 172.21.2.111:0/3426887482 conn(0x7f53580b61c0 msgr2=0x7f535805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:19.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- 172.21.2.111:0/3426887482 shutdown_connections 2021-04-03T03:16:19.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.061+0000 7f535cf82700 1 -- 172.21.2.111:0/3426887482 wait complete. 2021-04-03T03:16:19.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 Processor -- start 2021-04-03T03:16:19.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 -- start start 2021-04-03T03:16:19.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580603f0 0x7f53580ab970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53580add50 0x7f5358062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53580b17b0 0x7f5358156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580658c0 con 0x7f53580603f0 2021-04-03T03:16:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580abe60 con 0x7f53580add50 2021-04-03T03:16:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53580ab290 con 0x7f53580b17b0 2021-04-03T03:16:19.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580603f0 0x7f53580ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580603f0 0x7f53580ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56524/0 (socket says 172.21.2.111:56524) 2021-04-03T03:16:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 -- 172.21.2.111:0/3068977897 learned_addr learned my addr 172.21.2.111:0/3068977897 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 -- 172.21.2.111:0/3068977897 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53580b17b0 msgr2=0x7f5358156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53580b17b0 0x7f5358156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 -- 172.21.2.111:0/3068977897 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53580add50 msgr2=0x7f5358062c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53580add50 0x7f5358062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 -- 172.21.2.111:0/3068977897 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53580ab400 con 0x7f53580603f0 2021-04-03T03:16:19.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f53577fe700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580603f0 0x7f53580ab970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f534c002f30 tx=0x7f534c006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:19.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5354ff9700 1 -- 172.21.2.111:0/3068977897 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f534c00d840 con 0x7f53580603f0 2021-04-03T03:16:19.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5354ff9700 1 -- 172.21.2.111:0/3068977897 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c000e90 con 0x7f53580603f0 2021-04-03T03:16:19.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5354ff9700 1 -- 172.21.2.111:0/3068977897 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f534c00db90 con 0x7f53580603f0 2021-04-03T03:16:19.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53580630f0 con 0x7f53580603f0 2021-04-03T03:16:19.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53580aad00 con 0x7f53580603f0 2021-04-03T03:16:19.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5358063400 con 0x7f53580603f0 2021-04-03T03:16:19.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5354ff9700 1 -- 172.21.2.111:0/3068977897 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f534c017030 con 0x7f53580603f0 2021-04-03T03:16:19.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5354ff9700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5338056ef0 0x7f53380593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5354ff9700 1 -- 172.21.2.111:0/3068977897 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f534c06f290 con 0x7f53580603f0 2021-04-03T03:16:19.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5356ffd700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5338056ef0 0x7f53380593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:19.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.065+0000 7f5356ffd700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5338056ef0 0x7f53380593b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f53580ae520 tx=0x7f53480077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:19.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.069+0000 7f5354ff9700 1 -- 172.21.2.111:0/3068977897 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f534c044a60 con 0x7f53580603f0 2021-04-03T03:16:19.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.161+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f5358057660 con 0x7f53580603f0 2021-04-03T03:16:19.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f5354ff9700 1 -- 172.21.2.111:0/3068977897 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v109) v1 ==== 133+0+0 (secure 0 0 0) 0x7f534c048080 con 0x7f53580603f0 2021-04-03T03:16:19.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5338056ef0 msgr2=0x7f53380593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5338056ef0 0x7f53380593b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f53580ae520 tx=0x7f53480077f0).stop 2021-04-03T03:16:19.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580603f0 msgr2=0x7f53580ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580603f0 0x7f53580ab970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f534c002f30 tx=0x7f534c006ea0).stop 2021-04-03T03:16:19.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 shutdown_connections 2021-04-03T03:16:19.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53580b17b0 0x7f5358156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53580603f0 0x7f53580ab970 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:19.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5338056ef0 0x7f53380593b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:19.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 --2- 172.21.2.111:0/3068977897 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53580add50 0x7f5358062c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 >> 172.21.2.111:0/3068977897 conn(0x7f53580b61c0 msgr2=0x7f5358144860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:19.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 shutdown_connections 2021-04-03T03:16:19.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.897+0000 7f535cf82700 1 -- 172.21.2.111:0/3068977897 wait complete. 2021-04-03T03:16:19.903 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:16:19.907 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:16:19.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 Processor -- start 2021-04-03T03:16:19.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 -- start start 2021-04-03T03:16:19.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb3e00b8350 0x7fb3e0141d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb3e00b24d0 0x7fb3e00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e00b8730 con 0x7fb3e00ad660 2021-04-03T03:16:19.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e0141350 con 0x7fb3e00b8350 2021-04-03T03:16:19.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e01415f0 con 0x7fb3e00b24d0 2021-04-03T03:16:19.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb3e00b8350 0x7fb3e0141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:19.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb3e00b8350 0x7fb3e0141d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56528/0 (socket says 172.21.2.111:56528) 2021-04-03T03:16:19.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 -- 172.21.2.111:0/498355083 learned_addr learned my addr 172.21.2.111:0/498355083 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:19.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 -- 172.21.2.111:0/498355083 >> v2:172.21.2.115:3300/0 conn(0x7fb3e00b24d0 msgr2=0x7fb3e00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:19.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 --2- 172.21.2.111:0/498355083 >> v2:172.21.2.115:3300/0 conn(0x7fb3e00b24d0 0x7fb3e00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 -- 172.21.2.111:0/498355083 >> v1:172.21.2.115:6789/0 conn(0x7fb3e00ad660 legacy=0x7fb3e0129ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:19.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 -- 172.21.2.111:0/498355083 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e0141890 con 0x7fb3e00b8350 2021-04-03T03:16:19.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3dffff700 1 --2- 172.21.2.111:0/498355083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b8350 0x7fb3e0141d30 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fb3e0064060 tx=0x7fb3d8009270).ready entity=mon.0 client_cookie=32a6fff61941965d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:19.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3deffd700 1 -- 172.21.2.111:0/498355083 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d800bbc0 con 0x7fb3e00b8350 2021-04-03T03:16:19.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3deffd700 1 -- 172.21.2.111:0/498355083 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d800bd20 con 0x7fb3e00b8350 2021-04-03T03:16:19.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3deffd700 1 -- 172.21.2.111:0/498355083 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d800be90 con 0x7fb3e00b8350 2021-04-03T03:16:19.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 -- 172.21.2.111:0/498355083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b8350 msgr2=0x7fb3e0141d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.965+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/498355083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b8350 0x7fb3e0141d30 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fb3e0064060 tx=0x7fb3d8009270).stop 2021-04-03T03:16:19.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- 172.21.2.111:0/498355083 shutdown_connections 2021-04-03T03:16:19.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/498355083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b8350 0x7fb3e0141d30 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:19.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/498355083 >> v2:172.21.2.115:3300/0 conn(0x7fb3e00b24d0 0x7fb3e00ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- 172.21.2.111:0/498355083 >> 172.21.2.111:0/498355083 conn(0x7fb3e0060490 msgr2=0x7fb3e00b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:19.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- 172.21.2.111:0/498355083 shutdown_connections 2021-04-03T03:16:19.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- 172.21.2.111:0/498355083 wait complete. 2021-04-03T03:16:19.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 Processor -- start 2021-04-03T03:16:19.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- start start 2021-04-03T03:16:19.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3e00ad660 0x7fb3e0121830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b24d0 0x7fb3e00b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb3e00b8350 0x7fb3e0066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e0065a70 con 0x7fb3e00b24d0 2021-04-03T03:16:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e0156a70 con 0x7fb3e00ad660 2021-04-03T03:16:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e0065cd0 con 0x7fb3e00b8350 2021-04-03T03:16:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b24d0 0x7fb3e00b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:19.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b24d0 0x7fb3e00b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56534/0 (socket says 172.21.2.111:56534) 2021-04-03T03:16:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 -- 172.21.2.111:0/600696096 learned_addr learned my addr 172.21.2.111:0/600696096 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 -- 172.21.2.111:0/600696096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb3e00b8350 msgr2=0x7fb3e0066ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb3e00b8350 0x7fb3e0066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 -- 172.21.2.111:0/600696096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3e00ad660 msgr2=0x7fb3e0121830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3e00ad660 0x7fb3e0121830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 -- 172.21.2.111:0/600696096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e0157990 con 0x7fb3e00b24d0 2021-04-03T03:16:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3df7fe700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b24d0 0x7fb3e00b5750 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fb3d4002f30 tx=0x7fb3d4006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dd7fa700 1 -- 172.21.2.111:0/600696096 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d400d7e0 con 0x7fb3e00b24d0 2021-04-03T03:16:19.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dd7fa700 1 -- 172.21.2.111:0/600696096 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d400d940 con 0x7fb3e00b24d0 2021-04-03T03:16:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e0156790 con 0x7fb3e00b24d0 2021-04-03T03:16:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dd7fa700 1 -- 172.21.2.111:0/600696096 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d400dca0 con 0x7fb3e00b24d0 2021-04-03T03:16:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e0067090 con 0x7fb3e00b24d0 2021-04-03T03:16:19.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e0065e90 con 0x7fb3e00b24d0 2021-04-03T03:16:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dd7fa700 1 -- 172.21.2.111:0/600696096 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb3d4017030 con 0x7fb3e00b24d0 2021-04-03T03:16:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dd7fa700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3c0057050 0x7fb3c0059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dd7fa700 1 -- 172.21.2.111:0/600696096 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5950+0+0 (secure 0 0 0) 0x7fb3d406f8c0 con 0x7fb3e00b24d0 2021-04-03T03:16:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dffff700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3c0057050 0x7fb3c0059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:19.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.969+0000 7fb3dffff700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3c0057050 0x7fb3c0059510 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fb3e0064060 tx=0x7fb3d8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:19.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:19.973+0000 7fb3dd7fa700 1 -- 172.21.2.111:0/600696096 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb3d4044da0 con 0x7fb3e00b24d0 2021-04-03T03:16:20.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.065+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb3e0057660 con 0x7fb3e00b24d0 2021-04-03T03:16:20.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.909+0000 7fb3dd7fa700 1 -- 172.21.2.111:0/600696096 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v110) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb3d4015030 con 0x7fb3e00b24d0 2021-04-03T03:16:20.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3c0057050 msgr2=0x7fb3c0059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:20.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3c0057050 0x7fb3c0059510 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fb3e0064060 tx=0x7fb3d8016000).stop 2021-04-03T03:16:20.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b24d0 msgr2=0x7fb3e00b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:20.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b24d0 0x7fb3e00b5750 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fb3d4002f30 tx=0x7fb3d4006e80).stop 2021-04-03T03:16:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 shutdown_connections 2021-04-03T03:16:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb3e00b8350 0x7fb3e0066ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3c0057050 0x7fb3c0059510 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb3e00ad660 0x7fb3e0121830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:20.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 --2- 172.21.2.111:0/600696096 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb3e00b24d0 0x7fb3e00b5750 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 >> 172.21.2.111:0/600696096 conn(0x7fb3e0060490 msgr2=0x7fb3e0061610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 shutdown_connections 2021-04-03T03:16:20.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.913+0000 7fb3e53aa700 1 -- 172.21.2.111:0/600696096 wait complete. 2021-04-03T03:16:20.917 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:16:20.921 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:16:20.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d927a700 1 Processor -- start 2021-04-03T03:16:20.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d927a700 1 -- start start 2021-04-03T03:16:20.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d927a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe6d41475a0 0x7fe6d405fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:20.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d927a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d40adae0 0x7fe6d414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:20.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d927a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d40b1cf0 con 0x7fe6d405f9c0 2021-04-03T03:16:20.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d927a700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d40ad7c0 con 0x7fe6d41475a0 2021-04-03T03:16:20.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d927a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d40ae160 con 0x7fe6d40adae0 2021-04-03T03:16:20.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d3fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe6d405f9c0 0x7fe6d4146e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57334/0 (socket says 172.21.2.111:57334) 2021-04-03T03:16:20.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d3fff700 1 -- 172.21.2.111:0/4112649244 learned_addr learned my addr 172.21.2.111:0/4112649244 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:20.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d8a79700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d40adae0 0x7fe6d414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:20.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576461745 0 0) 0x7fe6d40b1cf0 con 0x7fe6d405f9c0 2021-04-03T03:16:20.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b4003240 con 0x7fe6d405f9c0 2021-04-03T03:16:20.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6cbfff700 1 --2- 172.21.2.111:0/4112649244 >> v2:172.21.2.115:3300/0 conn(0x7fe6d41475a0 0x7fe6d405fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:20.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4126329244 0 0) 0x7fe6b4003240 con 0x7fe6d405f9c0 2021-04-03T03:16:20.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d40adae0 msgr2=0x7fe6d414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:20.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d40adae0 0x7fe6d414b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:20.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 >> v2:172.21.2.115:3300/0 conn(0x7fe6d41475a0 msgr2=0x7fe6d405fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:20.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 --2- 172.21.2.111:0/4112649244 >> v2:172.21.2.115:3300/0 conn(0x7fe6d41475a0 0x7fe6d405fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:20.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4060650 con 0x7fe6d405f9c0 2021-04-03T03:16:20.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe6bc0016d0 con 0x7fe6d405f9c0 2021-04-03T03:16:20.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 >> v1:172.21.2.111:6789/0 conn(0x7fe6d405f9c0 legacy=0x7fe6d4146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:20.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6b4003730 0x7fe6b4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:20.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d40adae0 0x7fe6b40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:20.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6b40049d0 0x7fe6b4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d40ae160 con 0x7fe6b4003730 2021-04-03T03:16:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40041d0 con 0x7fe6b40049d0 2021-04-03T03:16:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b400b730 con 0x7fe6d40adae0 2021-04-03T03:16:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d3fff700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6b4003730 0x7fe6b4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:20.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d3fff700 1 -- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d40adae0 msgr2=0x7fe6b40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.981+0000 7fe6d3fff700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d40adae0 0x7fe6b40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 -- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6b40049d0 msgr2=0x7fe6b4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6b40049d0 0x7fe6b4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:20.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 -- 172.21.2.111:0/4112649244 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4003240 con 0x7fe6b4003730 2021-04-03T03:16:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6b4003730 0x7fe6b4003b50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7fe6b40054b0 tx=0x7fe6bc0077d0).ready entity=mon.0 client_cookie=d31d304c762a84a7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6bc0070d0 con 0x7fe6b4003730 2021-04-03T03:16:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc007230 con 0x7fe6b4003730 2021-04-03T03:16:20.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d37fe700 1 -- 172.21.2.111:0/4112649244 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6bc0055f0 con 0x7fe6b4003730 2021-04-03T03:16:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/4112649244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6b4003730 msgr2=0x7fe6b4003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 --2- 172.21.2.111:0/4112649244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6b4003730 0x7fe6b4003b50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7fe6b40054b0 tx=0x7fe6bc0077d0).stop 2021-04-03T03:16:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d8a79700 1 -- 172.21.2.111:0/4112649244 reap_dead start 2021-04-03T03:16:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/4112649244 shutdown_connections 2021-04-03T03:16:20.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/4112649244 >> 172.21.2.111:0/4112649244 conn(0x7fe6d40b5fb0 msgr2=0x7fe6d40b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:20.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/4112649244 shutdown_connections 2021-04-03T03:16:20.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/4112649244 wait complete. 2021-04-03T03:16:20.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 Processor -- start 2021-04-03T03:16:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- start start 2021-04-03T03:16:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d40a7240 0x7fe6d4143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d4065c40 0x7fe6d4144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d4066590 0x7fe6d4063330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4144840 con 0x7fe6d40a7240 2021-04-03T03:16:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d40a7620 con 0x7fe6d4065c40 2021-04-03T03:16:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4064ee0 con 0x7fe6d4066590 2021-04-03T03:16:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d40a7240 0x7fe6d4143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d40a7240 0x7fe6d4143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56554/0 (socket says 172.21.2.111:56554) 2021-04-03T03:16:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 -- 172.21.2.111:0/1666707768 learned_addr learned my addr 172.21.2.111:0/1666707768 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6cbfff700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d4065c40 0x7fe6d4144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d8a79700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d4066590 0x7fe6d4063330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 -- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d4066590 msgr2=0x7fe6d4063330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d4066590 0x7fe6d4063330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 -- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d4065c40 msgr2=0x7fe6d4144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d4065c40 0x7fe6d4144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 -- 172.21.2.111:0/1666707768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4066df0 con 0x7fe6d40a7240 2021-04-03T03:16:21.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d3fff700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d40a7240 0x7fe6d4143e60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7fe6b40054b0 tx=0x7fe6bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d1ffb700 1 -- 172.21.2.111:0/1666707768 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6bc0016b0 con 0x7fe6d40a7240 2021-04-03T03:16:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d1ffb700 1 -- 172.21.2.111:0/1666707768 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc018070 con 0x7fe6d40a7240 2021-04-03T03:16:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d4156de0 con 0x7fe6d40a7240 2021-04-03T03:16:21.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d4067090 con 0x7fe6d40a7240 2021-04-03T03:16:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d1ffb700 1 -- 172.21.2.111:0/1666707768 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6bc013450 con 0x7fe6d40a7240 2021-04-03T03:16:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d4156f70 con 0x7fe6d40a7240 2021-04-03T03:16:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d1ffb700 1 -- 172.21.2.111:0/1666707768 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe6bc024020 con 0x7fe6d40a7240 2021-04-03T03:16:21.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6d1ffb700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6b8056ea0 0x7fe6b8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.985+0000 7fe6cbfff700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6b8056ea0 0x7fe6b8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.989+0000 7fe6cbfff700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6b8056ea0 0x7fe6b8059360 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fe6c400c000 tx=0x7fe6c4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.989+0000 7fe6d1ffb700 1 -- 172.21.2.111:0/1666707768 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6300+0+0 (secure 0 0 0) 0x7fe6bc049e00 con 0x7fe6d40a7240 2021-04-03T03:16:21.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:20.989+0000 7fe6d1ffb700 1 -- 172.21.2.111:0/1666707768 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe6bc0461a0 con 0x7fe6d40a7240 2021-04-03T03:16:21.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.121+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe6d4057660 con 0x7fe6d40a7240 2021-04-03T03:16:21.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.125+0000 7fe6d1ffb700 1 -- 172.21.2.111:0/1666707768 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v47) v1 ==== 189+0+0 (secure 0 0 0) 0x7fe6bc0497c0 con 0x7fe6d40a7240 2021-04-03T03:16:21.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6b8056ea0 msgr2=0x7fe6b8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6b8056ea0 0x7fe6b8059360 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fe6c400c000 tx=0x7fe6c4008040).stop 2021-04-03T03:16:21.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d40a7240 msgr2=0x7fe6d4143e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d40a7240 0x7fe6d4143e60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7fe6b40054b0 tx=0x7fe6bc015000).stop 2021-04-03T03:16:21.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 shutdown_connections 2021-04-03T03:16:21.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6d4066590 0x7fe6d4063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6d40a7240 0x7fe6d4143e60 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6b8056ea0 0x7fe6b8059360 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 --2- 172.21.2.111:0/1666707768 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6d4065c40 0x7fe6d4144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 >> 172.21.2.111:0/1666707768 conn(0x7fe6d40b5fb0 msgr2=0x7fe6d405fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:21.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 shutdown_connections 2021-04-03T03:16:21.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.129+0000 7fe6d927a700 1 -- 172.21.2.111:0/1666707768 wait complete. 2021-04-03T03:16:21.133 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 21 and data pool 22 2021-04-03T03:16:21.137 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:21.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 Processor -- start 2021-04-03T03:16:21.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- start start 2021-04-03T03:16:21.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40473f0 0x7f74b40d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f74b403dea0 0x7f74b40420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b40d06e0 con 0x7f74b40d0e20 2021-04-03T03:16:21.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b40d0980 con 0x7f74b403dea0 2021-04-03T03:16:21.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40d0410 con 0x7f74b40473f0 2021-04-03T03:16:21.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f74b403dea0 0x7f74b40420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f74b403dea0 0x7f74b40420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56578/0 (socket says 172.21.2.111:56578) 2021-04-03T03:16:21.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 -- 172.21.2.111:0/2557123586 learned_addr learned my addr 172.21.2.111:0/2557123586 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:21.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bbfff700 1 --2- 172.21.2.111:0/2557123586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40473f0 0x7f74b40d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 -- 172.21.2.111:0/2557123586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40473f0 msgr2=0x7f74b40d4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 --2- 172.21.2.111:0/2557123586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40473f0 0x7f74b40d4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 -- 172.21.2.111:0/2557123586 >> v1:172.21.2.115:6789/0 conn(0x7f74b40d0e20 legacy=0x7f74b403ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 -- 172.21.2.111:0/2557123586 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40425b0 con 0x7f74b403dea0 2021-04-03T03:16:21.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bbfff700 1 --2- 172.21.2.111:0/2557123586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40473f0 0x7f74b40d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:21.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74bb7fe700 1 --2- 172.21.2.111:0/2557123586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40420c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f74b400c6b0 tx=0x7f74ac006e90).ready entity=mon.0 client_cookie=f4e52cc7902cd581 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74baffd700 1 -- 172.21.2.111:0/2557123586 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ac0098e0 con 0x7f74b403dea0 2021-04-03T03:16:21.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74baffd700 1 -- 172.21.2.111:0/2557123586 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac009a40 con 0x7f74b403dea0 2021-04-03T03:16:21.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74baffd700 1 -- 172.21.2.111:0/2557123586 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ac009de0 con 0x7f74b403dea0 2021-04-03T03:16:21.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- 172.21.2.111:0/2557123586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 msgr2=0x7f74b40420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 --2- 172.21.2.111:0/2557123586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40420c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f74b400c6b0 tx=0x7f74ac006e90).stop 2021-04-03T03:16:21.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- 172.21.2.111:0/2557123586 shutdown_connections 2021-04-03T03:16:21.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 --2- 172.21.2.111:0/2557123586 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40473f0 0x7f74b40d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 --2- 172.21.2.111:0/2557123586 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40420c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- 172.21.2.111:0/2557123586 >> 172.21.2.111:0/2557123586 conn(0x7f74b400d9e0 msgr2=0x7f74b4046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:21.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- 172.21.2.111:0/2557123586 shutdown_connections 2021-04-03T03:16:21.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- 172.21.2.111:0/2557123586 wait complete. 2021-04-03T03:16:21.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 Processor -- start 2021-04-03T03:16:21.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.193+0000 7f74c12ee700 1 -- start start 2021-04-03T03:16:21.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74b40473f0 0x7f74b4006cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40d0e20 0x7f74b40cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40c9cf0 con 0x7f74b403dea0 2021-04-03T03:16:21.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40c8a80 con 0x7f74b40473f0 2021-04-03T03:16:21.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40c95b0 con 0x7f74b40d0e20 2021-04-03T03:16:21.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56584/0 (socket says 172.21.2.111:56584) 2021-04-03T03:16:21.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 -- 172.21.2.111:0/377910306 learned_addr learned my addr 172.21.2.111:0/377910306 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:21.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 -- 172.21.2.111:0/377910306 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40d0e20 msgr2=0x7f74b40cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40d0e20 0x7f74b40cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 -- 172.21.2.111:0/377910306 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74b40473f0 msgr2=0x7f74b4006cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74b40473f0 0x7f74b4006cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 -- 172.21.2.111:0/377910306 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40c9850 con 0x7f74b403dea0 2021-04-03T03:16:21.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bbfff700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40c8490 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f74b0007000 tx=0x7f74b0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74b97fa700 1 -- 172.21.2.111:0/377910306 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b0007750 con 0x7f74b403dea0 2021-04-03T03:16:21.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74b97fa700 1 -- 172.21.2.111:0/377910306 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b00078b0 con 0x7f74b403dea0 2021-04-03T03:16:21.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b40df920 con 0x7f74b403dea0 2021-04-03T03:16:21.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b40caaf0 con 0x7f74b403dea0 2021-04-03T03:16:21.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74b97fa700 1 -- 172.21.2.111:0/377910306 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b0007c10 con 0x7f74b403dea0 2021-04-03T03:16:21.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b4004da0 con 0x7f74b403dea0 2021-04-03T03:16:21.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74b97fa700 1 -- 172.21.2.111:0/377910306 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f74b0007d70 con 0x7f74b403dea0 2021-04-03T03:16:21.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74b97fa700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f749c057030 0x7f749c0594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74b97fa700 1 -- 172.21.2.111:0/377910306 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f74b00704c0 con 0x7f74b403dea0 2021-04-03T03:16:21.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bb7fe700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f749c057030 0x7f749c0594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.197+0000 7f74bb7fe700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f749c057030 0x7f749c0594f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f74b400c6b0 tx=0x7f74ac015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.201+0000 7f74b97fa700 1 -- 172.21.2.111:0/377910306 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f74b00457a0 con 0x7f74b403dea0 2021-04-03T03:16:21.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74b40c9a50 con 0x7f74b403dea0 2021-04-03T03:16:21.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74b97fa700 1 -- 172.21.2.111:0/377910306 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f74b40c9a50 con 0x7f74b403dea0 2021-04-03T03:16:21.299 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:21.299 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":112,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:21.125283+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:19.175402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:20.080209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:21.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f749c057030 msgr2=0x7f749c0594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f749c057030 0x7f749c0594f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f74b400c6b0 tx=0x7f74ac015000).stop 2021-04-03T03:16:21.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 msgr2=0x7f74b40c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40c8490 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f74b0007000 tx=0x7f74b0008040).stop 2021-04-03T03:16:21.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 shutdown_connections 2021-04-03T03:16:21.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74b40d0e20 0x7f74b40cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74b403dea0 0x7f74b40c8490 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f749c057030 0x7f749c0594f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 --2- 172.21.2.111:0/377910306 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74b40473f0 0x7f74b4006cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 >> 172.21.2.111:0/377910306 conn(0x7f74b400d9e0 msgr2=0x7f74b40cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:21.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 shutdown_connections 2021-04-03T03:16:21.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.297+0000 7f74c12ee700 1 -- 172.21.2.111:0/377910306 wait complete. 2021-04-03T03:16:21.307 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:21.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9d2ea700 1 Processor -- start 2021-04-03T03:16:21.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9d2ea700 1 -- start start 2021-04-03T03:16:21.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9d2ea700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b980603f0 0x7f3b980b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9d2ea700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b980b0c90 0x7f3b98147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9d2ea700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b980607d0 con 0x7f3b9814b700 2021-04-03T03:16:21.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9d2ea700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b980af400 con 0x7f3b980603f0 2021-04-03T03:16:21.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9d2ea700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980af6a0 con 0x7f3b980b0c90 2021-04-03T03:16:21.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b980603f0 0x7f3b980b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b980603f0 0x7f3b980b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56590/0 (socket says 172.21.2.111:56590) 2021-04-03T03:16:21.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 -- 172.21.2.111:0/2919756714 learned_addr learned my addr 172.21.2.111:0/2919756714 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:21.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b9cae9700 1 --2- 172.21.2.111:0/2919756714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b980b0c90 0x7f3b98147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 -- 172.21.2.111:0/2919756714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b980b0c90 msgr2=0x7f3b98147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 --2- 172.21.2.111:0/2919756714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b980b0c90 0x7f3b98147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 -- 172.21.2.111:0/2919756714 >> v1:172.21.2.115:6789/0 conn(0x7f3b9814b700 legacy=0x7f3b980b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 -- 172.21.2.111:0/2919756714 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980afb40 con 0x7f3b980603f0 2021-04-03T03:16:21.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b97fff700 1 --2- 172.21.2.111:0/2919756714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980b0090 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f3b9806a3b0 tx=0x7f3b88006e90).ready entity=mon.0 client_cookie=26480ae6ee680657 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b96ffd700 1 -- 172.21.2.111:0/2919756714 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b880098e0 con 0x7f3b980603f0 2021-04-03T03:16:21.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.361+0000 7f3b96ffd700 1 -- 172.21.2.111:0/2919756714 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b88009a40 con 0x7f3b980603f0 2021-04-03T03:16:21.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b96ffd700 1 -- 172.21.2.111:0/2919756714 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b88009de0 con 0x7f3b980603f0 2021-04-03T03:16:21.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/2919756714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 msgr2=0x7f3b980b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/2919756714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980b0090 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f3b9806a3b0 tx=0x7f3b88006e90).stop 2021-04-03T03:16:21.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/2919756714 shutdown_connections 2021-04-03T03:16:21.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/2919756714 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b980b0c90 0x7f3b98147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/2919756714 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980b0090 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/2919756714 >> 172.21.2.111:0/2919756714 conn(0x7f3b980b61c0 msgr2=0x7f3b9805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:21.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/2919756714 shutdown_connections 2021-04-03T03:16:21.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/2919756714 wait complete. 2021-04-03T03:16:21.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 Processor -- start 2021-04-03T03:16:21.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- start start 2021-04-03T03:16:21.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b980b0c90 0x7f3b98146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b9814b700 0x7f3b98146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980628d0 con 0x7f3b980603f0 2021-04-03T03:16:21.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b98062b70 con 0x7f3b980b0c90 2021-04-03T03:16:21.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b98062e10 con 0x7f3b9814b700 2021-04-03T03:16:21.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56594/0 (socket says 172.21.2.111:56594) 2021-04-03T03:16:21.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 -- 172.21.2.111:0/647300841 learned_addr learned my addr 172.21.2.111:0/647300841 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:21.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 -- 172.21.2.111:0/647300841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b9814b700 msgr2=0x7f3b98146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b9814b700 0x7f3b98146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 -- 172.21.2.111:0/647300841 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b980b0c90 msgr2=0x7f3b98146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b980b0c90 0x7f3b98146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 -- 172.21.2.111:0/647300841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980630b0 con 0x7f3b980603f0 2021-04-03T03:16:21.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b97fff700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980ada30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f3b9806a3b0 tx=0x7f3b88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b957fa700 1 -- 172.21.2.111:0/647300841 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b88011d90 con 0x7f3b980603f0 2021-04-03T03:16:21.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b957fa700 1 -- 172.21.2.111:0/647300841 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8801e460 con 0x7f3b980603f0 2021-04-03T03:16:21.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b980abd20 con 0x7f3b980603f0 2021-04-03T03:16:21.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b981564c0 con 0x7f3b980603f0 2021-04-03T03:16:21.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b957fa700 1 -- 172.21.2.111:0/647300841 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b8801e7c0 con 0x7f3b980603f0 2021-04-03T03:16:21.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b98063240 con 0x7f3b980603f0 2021-04-03T03:16:21.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.365+0000 7f3b957fa700 1 -- 172.21.2.111:0/647300841 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3b88025030 con 0x7f3b980603f0 2021-04-03T03:16:21.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.369+0000 7f3b957fa700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b7c0570a0 0x7f3b7c059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.369+0000 7f3b957fa700 1 -- 172.21.2.111:0/647300841 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3b880717c0 con 0x7f3b980603f0 2021-04-03T03:16:21.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.369+0000 7f3b977fe700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b7c0570a0 0x7f3b7c059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.369+0000 7f3b977fe700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b7c0570a0 0x7f3b7c059560 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f3b80002eb0 tx=0x7f3b80006950).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.369+0000 7f3b957fa700 1 -- 172.21.2.111:0/647300841 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3b88046aa0 con 0x7f3b980603f0 2021-04-03T03:16:21.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b98057660 con 0x7f3b980603f0 2021-04-03T03:16:21.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b957fa700 1 -- 172.21.2.111:0/647300841 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f3b8804a0c0 con 0x7f3b980603f0 2021-04-03T03:16:21.462 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:21.462 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":112,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:21.125283+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:19.175402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:20.080209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:21.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b7c0570a0 msgr2=0x7f3b7c059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b7c0570a0 0x7f3b7c059560 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f3b80002eb0 tx=0x7f3b80006950).stop 2021-04-03T03:16:21.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 msgr2=0x7f3b980ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980ada30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f3b9806a3b0 tx=0x7f3b88015000).stop 2021-04-03T03:16:21.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 shutdown_connections 2021-04-03T03:16:21.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b9814b700 0x7f3b98146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b980603f0 0x7f3b980ada30 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b7c0570a0 0x7f3b7c059560 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 --2- 172.21.2.111:0/647300841 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b980b0c90 0x7f3b98146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 >> 172.21.2.111:0/647300841 conn(0x7f3b980b61c0 msgr2=0x7f3b98147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:21.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 shutdown_connections 2021-04-03T03:16:21.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.461+0000 7f3b9d2ea700 1 -- 172.21.2.111:0/647300841 wait complete. 2021-04-03T03:16:21.470 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:16:21.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.525+0000 7f13494d2700 1 Processor -- start 2021-04-03T03:16:21.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- start start 2021-04-03T03:16:21.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f133c0dbba0 0x7f133c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c0dc560 con 0x7f133c03d560 2021-04-03T03:16:21.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c0dc800 con 0x7f133c041060 2021-04-03T03:16:21.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c03de60 con 0x7f133c0dbba0 2021-04-03T03:16:21.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1348cd1700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f133c03d560 0x7f133c041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57398/0 (socket says 172.21.2.111:57398) 2021-04-03T03:16:21.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1348cd1700 1 -- 172.21.2.111:0/428038571 learned_addr learned my addr 172.21.2.111:0/428038571 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:21.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484800746 0 0) 0x7f133c0dc560 con 0x7f133c03d560 2021-04-03T03:16:21.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1328003240 con 0x7f133c03d560 2021-04-03T03:16:21.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1383018763 0 0) 0x7f1328003240 con 0x7f133c03d560 2021-04-03T03:16:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 >> v2:172.21.2.115:3300/0 conn(0x7f133c0dbba0 msgr2=0x7f133c0dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 --2- 172.21.2.111:0/428038571 >> v2:172.21.2.115:3300/0 conn(0x7f133c0dbba0 0x7f133c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 >> v1:172.21.2.115:6789/0 conn(0x7f133c041060 legacy=0x7f133c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c03da30 con 0x7f133c03d560 2021-04-03T03:16:21.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f13380016d0 con 0x7f133c03d560 2021-04-03T03:16:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 >> v1:172.21.2.111:6789/0 conn(0x7f133c03d560 legacy=0x7f133c041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13280037b0 0x7f1328003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f13280044e0 0x7f1328004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1328004e50 0x7f132800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c03de60 con 0x7f13280037b0 2021-04-03T03:16:21.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328005300 con 0x7f13280044e0 2021-04-03T03:16:21.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132800bc20 con 0x7f1328004e50 2021-04-03T03:16:21.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1343fff700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13280037b0 0x7f1328003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1343fff700 1 -- 172.21.2.111:0/428038571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1328004e50 msgr2=0x7f132800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1343fff700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1328004e50 0x7f132800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1343fff700 1 -- 172.21.2.111:0/428038571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f13280044e0 msgr2=0x7f1328004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:21.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1343fff700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f13280044e0 0x7f1328004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1343fff700 1 -- 172.21.2.111:0/428038571 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c03da30 con 0x7f13280037b0 2021-04-03T03:16:21.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1348cd1700 1 -- 172.21.2.111:0/428038571 reap_dead start 2021-04-03T03:16:21.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1343fff700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13280037b0 0x7f1328003bb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f1330002f30 tx=0x7f1330006e10).ready entity=mon.0 client_cookie=3cf15c4ea001dda6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13300097f0 con 0x7f13280037b0 2021-04-03T03:16:21.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1330000e90 con 0x7f13280037b0 2021-04-03T03:16:21.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f1342ffd700 1 -- 172.21.2.111:0/428038571 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1330009b40 con 0x7f13280037b0 2021-04-03T03:16:21.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- 172.21.2.111:0/428038571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13280037b0 msgr2=0x7f1328003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13280037b0 0x7f1328003bb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f1330002f30 tx=0x7f1330006e10).stop 2021-04-03T03:16:21.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- 172.21.2.111:0/428038571 shutdown_connections 2021-04-03T03:16:21.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 --2- 172.21.2.111:0/428038571 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f13280037b0 0x7f1328003bb0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- 172.21.2.111:0/428038571 >> 172.21.2.111:0/428038571 conn(0x7f133c0462b0 msgr2=0x7f133c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:21.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- 172.21.2.111:0/428038571 shutdown_connections 2021-04-03T03:16:21.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- 172.21.2.111:0/428038571 wait complete. 2021-04-03T03:16:21.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 Processor -- start 2021-04-03T03:16:21.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.529+0000 7f13494d2700 1 -- start start 2021-04-03T03:16:21.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f13280037b0 0x7f133c0a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f133c0a2200 0x7f133c0d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f133c0d7110 0x7f133c0e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0ac670 con 0x7f133c0d7110 2021-04-03T03:16:21.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0a04d0 con 0x7f133c0a2200 2021-04-03T03:16:21.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0a13d0 con 0x7f13280037b0 2021-04-03T03:16:21.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f133c0d7110 0x7f133c0e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f133c0d7110 0x7f133c0e6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56616/0 (socket says 172.21.2.111:56616) 2021-04-03T03:16:21.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 -- 172.21.2.111:0/3809382806 learned_addr learned my addr 172.21.2.111:0/3809382806 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:21.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 -- 172.21.2.111:0/3809382806 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f13280037b0 msgr2=0x7f133c0a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1343fff700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f13280037b0 0x7f133c0a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f13280037b0 0x7f133c0a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:21.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 -- 172.21.2.111:0/3809382806 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f133c0a2200 msgr2=0x7f133c0d6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:21.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f133c0a2200 0x7f133c0d6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:21.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 -- 172.21.2.111:0/3809382806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0a0230 con 0x7f133c0d7110 2021-04-03T03:16:21.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1348cd1700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f133c0d7110 0x7f133c0e6cd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f133c0ac7b0 tx=0x7f1338001840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13417fa700 1 -- 172.21.2.111:0/3809382806 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13380070d0 con 0x7f133c0d7110 2021-04-03T03:16:21.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13417fa700 1 -- 172.21.2.111:0/3809382806 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1338007230 con 0x7f133c0d7110 2021-04-03T03:16:21.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c0a07b0 con 0x7f133c0d7110 2021-04-03T03:16:21.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c0d7720 con 0x7f133c0d7110 2021-04-03T03:16:21.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13417fa700 1 -- 172.21.2.111:0/3809382806 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1338004900 con 0x7f133c0d7110 2021-04-03T03:16:21.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c0ac360 con 0x7f133c0d7110 2021-04-03T03:16:21.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13417fa700 1 -- 172.21.2.111:0/3809382806 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f133801c070 con 0x7f133c0d7110 2021-04-03T03:16:21.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13417fa700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f70 0x7f132c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:21.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f13417fa700 1 -- 172.21.2.111:0/3809382806 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f133806f1f0 con 0x7f133c0d7110 2021-04-03T03:16:21.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1343fff700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f70 0x7f132c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:21.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.533+0000 7f1343fff700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f70 0x7f132c059430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f1330016000 tx=0x7f1330015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:21.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.537+0000 7f13417fa700 1 -- 172.21.2.111:0/3809382806 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f13380444d0 con 0x7f133c0d7110 2021-04-03T03:16:21.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:21.661+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f133c004db0 con 0x7f133c0d7110 2021-04-03T03:16:22.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.133+0000 7f13417fa700 1 -- 172.21.2.111:0/3809382806 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1338047af0 con 0x7f133c0d7110 2021-04-03T03:16:22.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f70 msgr2=0x7f132c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f70 0x7f132c059430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f1330016000 tx=0x7f1330015000).stop 2021-04-03T03:16:22.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f133c0d7110 msgr2=0x7f133c0e6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f133c0d7110 0x7f133c0e6cd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f133c0ac7b0 tx=0x7f1338001840).stop 2021-04-03T03:16:22.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 shutdown_connections 2021-04-03T03:16:22.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f133c0d7110 0x7f133c0e6cd0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f13280037b0 0x7f133c0a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f132c056f70 0x7f132c059430 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 --2- 172.21.2.111:0/3809382806 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f133c0a2200 0x7f133c0d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 >> 172.21.2.111:0/3809382806 conn(0x7f133c0462b0 msgr2=0x7f133c03fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 shutdown_connections 2021-04-03T03:16:22.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.137+0000 7f13494d2700 1 -- 172.21.2.111:0/3809382806 wait complete. 2021-04-03T03:16:22.144 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:22.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.197+0000 7f155544f700 1 Processor -- start 2021-04-03T03:16:22.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.197+0000 7f155544f700 1 -- start start 2021-04-03T03:16:22.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15500603f0 0x7f15500b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f155014b700 0x7f15500b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f15500b12b0 0x7f15500b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15500b0f60 con 0x7f155014b700 2021-04-03T03:16:22.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15500607d0 con 0x7f15500b12b0 2021-04-03T03:16:22.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15500af400 con 0x7f15500603f0 2021-04-03T03:16:22.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f155014b700 0x7f15500b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f155014b700 0x7f15500b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56622/0 (socket says 172.21.2.111:56622) 2021-04-03T03:16:22.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 -- 172.21.2.111:0/3184464260 learned_addr learned my addr 172.21.2.111:0/3184464260 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:22.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 -- 172.21.2.111:0/3184464260 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15500603f0 msgr2=0x7f15500b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 --2- 172.21.2.111:0/3184464260 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15500603f0 0x7f15500b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 -- 172.21.2.111:0/3184464260 >> v2:172.21.2.115:3300/0 conn(0x7f15500b12b0 msgr2=0x7f15500b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 --2- 172.21.2.111:0/3184464260 >> v2:172.21.2.115:3300/0 conn(0x7f15500b12b0 0x7f15500b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 -- 172.21.2.111:0/3184464260 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500af6a0 con 0x7f155014b700 2021-04-03T03:16:22.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154f7fe700 1 --2- 172.21.2.111:0/3184464260 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f155014b700 0x7f15500b0580 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f1550062740 tx=0x7f1548009270).ready entity=mon.0 client_cookie=a1ecc832f8e6ab07 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154effd700 1 -- 172.21.2.111:0/3184464260 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154800bbc0 con 0x7f155014b700 2021-04-03T03:16:22.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154effd700 1 -- 172.21.2.111:0/3184464260 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154800bd20 con 0x7f155014b700 2021-04-03T03:16:22.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154effd700 1 -- 172.21.2.111:0/3184464260 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154800be90 con 0x7f155014b700 2021-04-03T03:16:22.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- 172.21.2.111:0/3184464260 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f155014b700 msgr2=0x7f15500b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- 172.21.2.111:0/3184464260 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f155014b700 0x7f15500b0580 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f1550062740 tx=0x7f1548009270).stop 2021-04-03T03:16:22.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- 172.21.2.111:0/3184464260 shutdown_connections 2021-04-03T03:16:22.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- 172.21.2.111:0/3184464260 >> v2:172.21.2.115:3300/0 conn(0x7f15500b12b0 0x7f15500b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- 172.21.2.111:0/3184464260 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15500603f0 0x7f15500b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- 172.21.2.111:0/3184464260 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f155014b700 0x7f15500b0580 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- 172.21.2.111:0/3184464260 >> 172.21.2.111:0/3184464260 conn(0x7f15500b61c0 msgr2=0x7f155005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- 172.21.2.111:0/3184464260 shutdown_connections 2021-04-03T03:16:22.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- 172.21.2.111:0/3184464260 wait complete. 2021-04-03T03:16:22.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 Processor -- start 2021-04-03T03:16:22.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- start start 2021-04-03T03:16:22.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15500603f0 0x7f155014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15500b12b0 0x7f15500a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f155014b700 0x7f1550156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15501566b0 con 0x7f15500603f0 2021-04-03T03:16:22.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1550062f20 con 0x7f15500b12b0 2021-04-03T03:16:22.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1550063e80 con 0x7f155014b700 2021-04-03T03:16:22.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15500603f0 0x7f155014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15500603f0 0x7f155014e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56626/0 (socket says 172.21.2.111:56626) 2021-04-03T03:16:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 -- 172.21.2.111:0/4142607840 learned_addr learned my addr 172.21.2.111:0/4142607840 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 -- 172.21.2.111:0/4142607840 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f155014b700 msgr2=0x7f1550156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f155014b700 0x7f1550156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 -- 172.21.2.111:0/4142607840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15500b12b0 msgr2=0x7f15500a7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15500b12b0 0x7f15500a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 -- 172.21.2.111:0/4142607840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500a83a0 con 0x7f15500603f0 2021-04-03T03:16:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154ffff700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15500603f0 0x7f155014e460 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f1544002f30 tx=0x7f1544006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154d7fa700 1 -- 172.21.2.111:0/4142607840 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154400d840 con 0x7f15500603f0 2021-04-03T03:16:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154d7fa700 1 -- 172.21.2.111:0/4142607840 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1544000e90 con 0x7f15500603f0 2021-04-03T03:16:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15500aa280 con 0x7f15500603f0 2021-04-03T03:16:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15500a9a50 con 0x7f15500603f0 2021-04-03T03:16:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.201+0000 7f154d7fa700 1 -- 172.21.2.111:0/4142607840 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154400db90 con 0x7f15500603f0 2021-04-03T03:16:22.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.205+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1550063cb0 con 0x7f15500603f0 2021-04-03T03:16:22.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.205+0000 7f154d7fa700 1 -- 172.21.2.111:0/4142607840 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1544017030 con 0x7f15500603f0 2021-04-03T03:16:22.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.205+0000 7f154d7fa700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1530056ef0 0x7f15300593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.205+0000 7f154d7fa700 1 -- 172.21.2.111:0/4142607840 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f154406f590 con 0x7f15500603f0 2021-04-03T03:16:22.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.205+0000 7f154f7fe700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1530056ef0 0x7f15300593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.205+0000 7f154f7fe700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1530056ef0 0x7f15300593b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f1550062740 tx=0x7f1548016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.205+0000 7f154d7fa700 1 -- 172.21.2.111:0/4142607840 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1544044a30 con 0x7f15500603f0 2021-04-03T03:16:22.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.329+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1550057660 con 0x7f15500603f0 2021-04-03T03:16:22.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.329+0000 7f154d7fa700 1 -- 172.21.2.111:0/4142607840 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f1544048050 con 0x7f15500603f0 2021-04-03T03:16:22.332 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:22.333 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":48,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1453695075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1453695075},{"type":"v1","addr":"172.21.2.115:6833","nonce":1453695075}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:21.125389+0000","modified":"2021-04-03T03:16:22.128337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7503":{"gid":7503,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1395173601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1395173601},{"type":"v1","addr":"172.21.2.111:6835","nonce":1395173601}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-04-03T03:16:22.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1530056ef0 msgr2=0x7f15300593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1530056ef0 0x7f15300593b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f1550062740 tx=0x7f1548016000).stop 2021-04-03T03:16:22.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15500603f0 msgr2=0x7f155014e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15500603f0 0x7f155014e460 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f1544002f30 tx=0x7f1544006ea0).stop 2021-04-03T03:16:22.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 shutdown_connections 2021-04-03T03:16:22.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f155014b700 0x7f1550156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15500603f0 0x7f155014e460 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1530056ef0 0x7f15300593b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 --2- 172.21.2.111:0/4142607840 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15500b12b0 0x7f15500a7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 >> 172.21.2.111:0/4142607840 conn(0x7f15500b61c0 msgr2=0x7f1550067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 shutdown_connections 2021-04-03T03:16:22.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.333+0000 7f155544f700 1 -- 172.21.2.111:0/4142607840 wait complete. 2021-04-03T03:16:22.336 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 48 2021-04-03T03:16:22.340 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:22.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.393+0000 7fc8926fa700 1 Processor -- start 2021-04-03T03:16:22.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.393+0000 7fc8926fa700 1 -- start start 2021-04-03T03:16:22.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc8926fa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc8926fa700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc88c14b700 0x7fc88c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc8926fa700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c0607d0 con 0x7fc88c0b12b0 2021-04-03T03:16:22.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc8926fa700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c0af400 con 0x7fc88c14b700 2021-04-03T03:16:22.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc8926fa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0af6a0 con 0x7fc88c0603f0 2021-04-03T03:16:22.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc8916f8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc890ef7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc88c14b700 0x7fc88c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc891ef9700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fc88c0b12b0 0x7fc88c1465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:41734/0 (socket says 172.21.2.111:41734) 2021-04-03T03:16:22.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc891ef9700 1 -- 172.21.2.111:0/3625554359 learned_addr learned my addr 172.21.2.111:0/3625554359 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:22.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc890ef7700 1 -- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 msgr2=0x7fc88c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc890ef7700 1 --2- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc890ef7700 1 -- 172.21.2.111:0/3625554359 >> v1:172.21.2.115:6789/0 conn(0x7fc88c0b12b0 legacy=0x7fc88c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc87bfff700 1 -- 172.21.2.111:0/3625554359 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020339482 0 0) 0x7fc88c0607d0 con 0x7fc88c0b12b0 2021-04-03T03:16:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc890ef7700 1 -- 172.21.2.111:0/3625554359 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0afb40 con 0x7fc88c14b700 2021-04-03T03:16:22.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc8916f8700 1 --2- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:16:22.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.397+0000 7fc890ef7700 1 --2- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c14b700 0x7fc88c0b0580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fc87c002f30 tx=0x7fc87c006dd0).ready entity=mon.1 client_cookie=f68fb216cdaf0eb2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc87bfff700 1 -- 172.21.2.111:0/3625554359 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc87c009760 con 0x7fc88c14b700 2021-04-03T03:16:22.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc87bfff700 1 -- 172.21.2.111:0/3625554359 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c0098c0 con 0x7fc88c14b700 2021-04-03T03:16:22.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc87bfff700 1 -- 172.21.2.111:0/3625554359 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc87c009c20 con 0x7fc88c14b700 2021-04-03T03:16:22.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c14b700 msgr2=0x7fc88c0b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 --2- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c14b700 0x7fc88c0b0580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fc87c002f30 tx=0x7fc87c006dd0).stop 2021-04-03T03:16:22.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- 172.21.2.111:0/3625554359 shutdown_connections 2021-04-03T03:16:22.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 --2- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 --2- 172.21.2.111:0/3625554359 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c14b700 0x7fc88c0b0580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- 172.21.2.111:0/3625554359 >> 172.21.2.111:0/3625554359 conn(0x7fc88c0b61c0 msgr2=0x7fc88c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- 172.21.2.111:0/3625554359 shutdown_connections 2021-04-03T03:16:22.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- 172.21.2.111:0/3625554359 wait complete. 2021-04-03T03:16:22.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 Processor -- start 2021-04-03T03:16:22.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- start start 2021-04-03T03:16:22.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c0b12b0 0x7fc88c0639d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc88c14b700 0x7fc88c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c1599e0 con 0x7fc88c14b700 2021-04-03T03:16:22.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0a79d0 con 0x7fc88c0b12b0 2021-04-03T03:16:22.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c064200 con 0x7fc88c0603f0 2021-04-03T03:16:22.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc88c14b700 0x7fc88c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc88c14b700 0x7fc88c156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56642/0 (socket says 172.21.2.111:56642) 2021-04-03T03:16:22.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 -- 172.21.2.111:0/121529371 learned_addr learned my addr 172.21.2.111:0/121529371 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:22.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 -- 172.21.2.111:0/121529371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 msgr2=0x7fc88c14d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 -- 172.21.2.111:0/121529371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c0b12b0 msgr2=0x7fc88c0639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c0b12b0 0x7fc88c0639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 -- 172.21.2.111:0/121529371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c159c10 con 0x7fc88c14b700 2021-04-03T03:16:22.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc891ef9700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc88c14b700 0x7fc88c156400 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7fc884001210 tx=0x7fc884007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc87a7fc700 1 -- 172.21.2.111:0/121529371 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc884005c50 con 0x7fc88c14b700 2021-04-03T03:16:22.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc87a7fc700 1 -- 172.21.2.111:0/121529371 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc884005db0 con 0x7fc88c14b700 2021-04-03T03:16:22.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c14dae0 con 0x7fc88c14b700 2021-04-03T03:16:22.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c067440 con 0x7fc88c14b700 2021-04-03T03:16:22.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.401+0000 7fc87a7fc700 1 -- 172.21.2.111:0/121529371 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc884004a70 con 0x7fc88c14b700 2021-04-03T03:16:22.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.405+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c057660 con 0x7fc88c14b700 2021-04-03T03:16:22.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.405+0000 7fc87a7fc700 1 -- 172.21.2.111:0/121529371 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc88401b060 con 0x7fc88c14b700 2021-04-03T03:16:22.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.405+0000 7fc87a7fc700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc86c056ef0 0x7fc86c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.405+0000 7fc87a7fc700 1 -- 172.21.2.111:0/121529371 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc88406fed0 con 0x7fc88c14b700 2021-04-03T03:16:22.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.405+0000 7fc8916f8700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc86c056ef0 0x7fc86c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.405+0000 7fc8916f8700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc86c056ef0 0x7fc86c0593b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fc88800c000 tx=0x7fc888006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.405+0000 7fc87a7fc700 1 -- 172.21.2.111:0/121529371 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc8840451b0 con 0x7fc88c14b700 2021-04-03T03:16:22.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc88c0b3de0 con 0x7fc88c14b700 2021-04-03T03:16:22.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc87a7fc700 1 -- 172.21.2.111:0/121529371 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fc884019070 con 0x7fc88c14b700 2021-04-03T03:16:22.498 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:22.499 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":112,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:21.125283+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:19.175402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:20.080209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:22.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc86c056ef0 msgr2=0x7fc86c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc86c056ef0 0x7fc86c0593b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fc88800c000 tx=0x7fc888006dd0).stop 2021-04-03T03:16:22.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc88c14b700 msgr2=0x7fc88c156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc88c14b700 0x7fc88c156400 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7fc884001210 tx=0x7fc884007da0).stop 2021-04-03T03:16:22.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 shutdown_connections 2021-04-03T03:16:22.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc88c14b700 0x7fc88c156400 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc88c0603f0 0x7fc88c14d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc86c056ef0 0x7fc86c0593b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 --2- 172.21.2.111:0/121529371 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc88c0b12b0 0x7fc88c0639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 >> 172.21.2.111:0/121529371 conn(0x7fc88c0b61c0 msgr2=0x7fc88c0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 shutdown_connections 2021-04-03T03:16:22.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.497+0000 7fc8926fa700 1 -- 172.21.2.111:0/121529371 wait complete. 2021-04-03T03:16:22.506 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:22.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfdf14700 1 Processor -- start 2021-04-03T03:16:22.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfdf14700 1 -- start start 2021-04-03T03:16:22.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfdf14700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf0046ce0 0x7f0bf003f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfdf14700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0bf003d350 0x7f0bf0041c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfdf14700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf0041240 con 0x7f0bf0042160 2021-04-03T03:16:22.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfdf14700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf00f1850 con 0x7f0bf003d350 2021-04-03T03:16:22.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfdf14700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00470c0 con 0x7f0bf0046ce0 2021-04-03T03:16:22.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bff38e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0bf0042160 0x7f0bf00f51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57440/0 (socket says 172.21.2.111:57440) 2021-04-03T03:16:22.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bff38e700 1 -- 172.21.2.111:0/2226847674 learned_addr learned my addr 172.21.2.111:0/2226847674 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:22.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298844134 0 0) 0x7f0bf0041240 con 0x7f0bf0042160 2021-04-03T03:16:22.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be4003240 con 0x7f0bf0042160 2021-04-03T03:16:22.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfd713700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf0046ce0 0x7f0bf003f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bfcf12700 1 --2- 172.21.2.111:0/2226847674 >> v2:172.21.2.115:3300/0 conn(0x7f0bf003d350 0x7f0bf0041c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1010855209 0 0) 0x7f0be4003240 con 0x7f0bf0042160 2021-04-03T03:16:22.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf0046ce0 msgr2=0x7f0bf003f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf0046ce0 0x7f0bf003f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 >> v2:172.21.2.115:3300/0 conn(0x7f0bf003d350 msgr2=0x7f0bf0041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 --2- 172.21.2.111:0/2226847674 >> v2:172.21.2.115:3300/0 conn(0x7f0bf003d350 0x7f0bf0041c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00400d0 con 0x7f0bf0042160 2021-04-03T03:16:22.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f0bf80450f0 con 0x7f0bf0042160 2021-04-03T03:16:22.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 >> v1:172.21.2.111:6789/0 conn(0x7f0bf0042160 legacy=0x7f0bf00f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf0046ce0 0x7f0be40037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.565+0000 7f0bebfff700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0be4003cc0 0x7f0be40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bebfff700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0be40049d0 0x7f0be4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00f1850 con 0x7f0be40049d0 2021-04-03T03:16:22.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00470c0 con 0x7f0be4003cc0 2021-04-03T03:16:22.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40041e0 con 0x7f0bf0046ce0 2021-04-03T03:16:22.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0be40049d0 0x7f0be4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 -- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf0046ce0 msgr2=0x7f0be40037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf0046ce0 0x7f0be40037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 -- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0be4003cc0 msgr2=0x7f0be40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0be4003cc0 0x7f0be40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 -- 172.21.2.111:0/2226847674 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4003240 con 0x7f0be40049d0 2021-04-03T03:16:22.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0be40049d0 0x7f0be4004e50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f0bf8069660 tx=0x7f0bf003eaf0).ready entity=mon.0 client_cookie=5ca6fb0e3abe6354 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf805df60 con 0x7f0be40049d0 2021-04-03T03:16:22.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80699f0 con 0x7f0be40049d0 2021-04-03T03:16:22.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bebfff700 1 -- 172.21.2.111:0/2226847674 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf8061110 con 0x7f0be40049d0 2021-04-03T03:16:22.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- 172.21.2.111:0/2226847674 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0be40049d0 msgr2=0x7f0be4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 --2- 172.21.2.111:0/2226847674 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0be40049d0 0x7f0be4004e50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f0bf8069660 tx=0x7f0bf003eaf0).stop 2021-04-03T03:16:22.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bff38e700 1 -- 172.21.2.111:0/2226847674 reap_dead start 2021-04-03T03:16:22.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- 172.21.2.111:0/2226847674 shutdown_connections 2021-04-03T03:16:22.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- 172.21.2.111:0/2226847674 >> 172.21.2.111:0/2226847674 conn(0x7f0bf000d0c0 msgr2=0x7f0bf00462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- 172.21.2.111:0/2226847674 shutdown_connections 2021-04-03T03:16:22.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- 172.21.2.111:0/2226847674 wait complete. 2021-04-03T03:16:22.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 Processor -- start 2021-04-03T03:16:22.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- start start 2021-04-03T03:16:22.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bf0044a40 0x7f0bf003bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bf00d1470 0x7f0bf00d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf00c2e20 0x7f0bf00c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00d82f0 con 0x7f0bf00d1470 2021-04-03T03:16:22.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0044e20 con 0x7f0bf0044a40 2021-04-03T03:16:22.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf003b6a0 con 0x7f0bf00c2e20 2021-04-03T03:16:22.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bf00d1470 0x7f0bf00d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bf00d1470 0x7f0bf00d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56662/0 (socket says 172.21.2.111:56662) 2021-04-03T03:16:22.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 -- 172.21.2.111:0/1890583991 learned_addr learned my addr 172.21.2.111:0/1890583991 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:22.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 -- 172.21.2.111:0/1890583991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf00c2e20 msgr2=0x7f0bf00c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf00c2e20 0x7f0bf00c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 -- 172.21.2.111:0/1890583991 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bf0044a40 msgr2=0x7f0bf003bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bf0044a40 0x7f0bf003bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 -- 172.21.2.111:0/1890583991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf003b940 con 0x7f0bf00d1470 2021-04-03T03:16:22.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfcf12700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bf00d1470 0x7f0bf00d18f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f0bf400a000 tx=0x7f0bf4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0be9ffb700 1 -- 172.21.2.111:0/1890583991 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf400f680 con 0x7f0bf00d1470 2021-04-03T03:16:22.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0be9ffb700 1 -- 172.21.2.111:0/1890583991 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf400f040 con 0x7f0bf00d1470 2021-04-03T03:16:22.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0be9ffb700 1 -- 172.21.2.111:0/1890583991 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf400f9d0 con 0x7f0bf00d1470 2021-04-03T03:16:22.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf003b040 con 0x7f0bf00d1470 2021-04-03T03:16:22.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.569+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf00d7400 con 0x7f0bf00d1470 2021-04-03T03:16:22.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.573+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf0004d90 con 0x7f0bf00d1470 2021-04-03T03:16:22.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.573+0000 7f0be9ffb700 1 -- 172.21.2.111:0/1890583991 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0bf400fb30 con 0x7f0bf00d1470 2021-04-03T03:16:22.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.573+0000 7f0be9ffb700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bd8056e90 0x7f0bd8059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.573+0000 7f0be9ffb700 1 -- 172.21.2.111:0/1890583991 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0bf406fb10 con 0x7f0bf00d1470 2021-04-03T03:16:22.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.573+0000 7f0bfd713700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bd8056e90 0x7f0bd8059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.573+0000 7f0bfd713700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bd8056e90 0x7f0bd8059350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f0bec00a000 tx=0x7f0bec008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.573+0000 7f0be9ffb700 1 -- 172.21.2.111:0/1890583991 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0bf4042c50 con 0x7f0bf00d1470 2021-04-03T03:16:22.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.701+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0bf00d7710 con 0x7f0bf00d1470 2021-04-03T03:16:22.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.705+0000 7f0be9ffb700 1 -- 172.21.2.111:0/1890583991 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0bf4048930 con 0x7f0bf00d1470 2021-04-03T03:16:22.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bd8056e90 msgr2=0x7f0bd8059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bd8056e90 0x7f0bd8059350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f0bec00a000 tx=0x7f0bec008040).stop 2021-04-03T03:16:22.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bf00d1470 msgr2=0x7f0bf00d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bf00d1470 0x7f0bf00d18f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f0bf400a000 tx=0x7f0bf4008040).stop 2021-04-03T03:16:22.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 shutdown_connections 2021-04-03T03:16:22.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0bf00c2e20 0x7f0bf00c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0bd8056e90 0x7f0bd8059350 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0bf0044a40 0x7f0bf003bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 --2- 172.21.2.111:0/1890583991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0bf00d1470 0x7f0bf00d18f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 >> 172.21.2.111:0/1890583991 conn(0x7f0bf000d0c0 msgr2=0x7f0bf00f2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 shutdown_connections 2021-04-03T03:16:22.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.709+0000 7f0bfdf14700 1 -- 172.21.2.111:0/1890583991 wait complete. 2021-04-03T03:16:22.712 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:16:22.716 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:22.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 Processor -- start 2021-04-03T03:16:22.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- start start 2021-04-03T03:16:22.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78046ce0 0x7f7f7803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7f78042380 0x7f7f780f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f78041240 con 0x7f7f7803d350 2021-04-03T03:16:22.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f780f1850 con 0x7f7f78042380 2021-04-03T03:16:22.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780470c0 con 0x7f7f78046ce0 2021-04-03T03:16:22.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7f78042380 0x7f7f780f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8586b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7f7803d350 0x7f7f78041c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57460/0 (socket says 172.21.2.111:57460) 2021-04-03T03:16:22.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8586b700 1 -- 172.21.2.111:0/3472322691 learned_addr learned my addr 172.21.2.111:0/3472322691 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:22.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 -- 172.21.2.111:0/3472322691 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78046ce0 msgr2=0x7f7f7803f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- 172.21.2.111:0/3472322691 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78046ce0 0x7f7f7803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 -- 172.21.2.111:0/3472322691 >> v1:172.21.2.111:6789/0 conn(0x7f7f7803d350 legacy=0x7f7f78041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8506a700 1 -- 172.21.2.111:0/3472322691 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606859593 0 0) 0x7f7f78041240 con 0x7f7f7803d350 2021-04-03T03:16:22.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 -- 172.21.2.111:0/3472322691 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f780400d0 con 0x7f7f78042380 2021-04-03T03:16:22.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- 172.21.2.111:0/3472322691 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78042380 0x7f7f780f51b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f7f8006b5d0 tx=0x7f7f8005ad70).ready entity=mon.0 client_cookie=1d6d5237f2accc06 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8506a700 1 -- 172.21.2.111:0/3472322691 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8005bda0 con 0x7f7f78042380 2021-04-03T03:16:22.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8506a700 1 -- 172.21.2.111:0/3472322691 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8005df60 con 0x7f7f78042380 2021-04-03T03:16:22.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8506a700 1 -- 172.21.2.111:0/3472322691 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f800699f0 con 0x7f7f78042380 2021-04-03T03:16:22.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- 172.21.2.111:0/3472322691 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78042380 msgr2=0x7f7f780f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- 172.21.2.111:0/3472322691 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78042380 0x7f7f780f51b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f7f8006b5d0 tx=0x7f7f8005ad70).stop 2021-04-03T03:16:22.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- 172.21.2.111:0/3472322691 shutdown_connections 2021-04-03T03:16:22.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- 172.21.2.111:0/3472322691 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78042380 0x7f7f780f51b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- 172.21.2.111:0/3472322691 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78046ce0 0x7f7f7803f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- 172.21.2.111:0/3472322691 >> 172.21.2.111:0/3472322691 conn(0x7f7f7800d0c0 msgr2=0x7f7f780462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- 172.21.2.111:0/3472322691 shutdown_connections 2021-04-03T03:16:22.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- 172.21.2.111:0/3472322691 wait complete. 2021-04-03T03:16:22.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 Processor -- start 2021-04-03T03:16:22.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- start start 2021-04-03T03:16:22.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7f7803d350 0x7f7f780faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78042380 0x7f7f780d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78046ce0 0x7f7f780d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780d0d70 con 0x7f7f78046ce0 2021-04-03T03:16:22.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780d1010 con 0x7f7f7803d350 2021-04-03T03:16:22.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780d12b0 con 0x7f7f78042380 2021-04-03T03:16:22.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78046ce0 0x7f7f780d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78046ce0 0x7f7f780d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56676/0 (socket says 172.21.2.111:56676) 2021-04-03T03:16:22.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 -- 172.21.2.111:0/2404548806 learned_addr learned my addr 172.21.2.111:0/2404548806 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:22.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 -- 172.21.2.111:0/2404548806 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78042380 msgr2=0x7f7f780d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78042380 0x7f7f780d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 -- 172.21.2.111:0/2404548806 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7f7803d350 msgr2=0x7f7f780faaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7f7803d350 0x7f7f780faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 -- 172.21.2.111:0/2404548806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f780d1750 con 0x7f7f78046ce0 2021-04-03T03:16:22.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f87ce7700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78046ce0 0x7f7f780d30e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f7f8006b5d0 tx=0x7f7f80064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f72ffd700 1 -- 172.21.2.111:0/2404548806 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f800441b0 con 0x7f7f78046ce0 2021-04-03T03:16:22.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f72ffd700 1 -- 172.21.2.111:0/2404548806 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8005f2c0 con 0x7f7f78046ce0 2021-04-03T03:16:22.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f780cf5c0 con 0x7f7f78046ce0 2021-04-03T03:16:22.791 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f72ffd700 1 -- 172.21.2.111:0/2404548806 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8005f620 con 0x7f7f78046ce0 2021-04-03T03:16:22.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.777+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f780fb1f0 con 0x7f7f78046ce0 2021-04-03T03:16:22.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.781+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f780f16a0 con 0x7f7f78046ce0 2021-04-03T03:16:22.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.781+0000 7f7f72ffd700 1 -- 172.21.2.111:0/2404548806 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7f8007b020 con 0x7f7f78046ce0 2021-04-03T03:16:22.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.781+0000 7f7f72ffd700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7f68056e90 0x7f7f68059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.781+0000 7f7f72ffd700 1 -- 172.21.2.111:0/2404548806 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7f800cfa70 con 0x7f7f78046ce0 2021-04-03T03:16:22.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.781+0000 7f7f8606c700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7f68056e90 0x7f7f68059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.781+0000 7f7f8606c700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7f68056e90 0x7f7f68059350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f7f7c002f30 tx=0x7f7f7c0069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.781+0000 7f7f72ffd700 1 -- 172.21.2.111:0/2404548806 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7f800a4d50 con 0x7f7f78046ce0 2021-04-03T03:16:22.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.909+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7f780d1470 con 0x7f7f78046ce0 2021-04-03T03:16:22.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f72ffd700 1 -- 172.21.2.111:0/2404548806 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7f800a8370 con 0x7f7f78046ce0 2021-04-03T03:16:22.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7f68056e90 msgr2=0x7f7f68059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7f68056e90 0x7f7f68059350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f7f7c002f30 tx=0x7f7f7c0069f0).stop 2021-04-03T03:16:22.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78046ce0 msgr2=0x7f7f780d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78046ce0 0x7f7f780d30e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f7f8006b5d0 tx=0x7f7f80064030).stop 2021-04-03T03:16:22.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 shutdown_connections 2021-04-03T03:16:22.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7f78046ce0 0x7f7f780d30e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7f68056e90 0x7f7f68059350 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7f7803d350 0x7f7f780faaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 --2- 172.21.2.111:0/2404548806 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7f78042380 0x7f7f780d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 >> 172.21.2.111:0/2404548806 conn(0x7f7f7800d0c0 msgr2=0x7f7f78045370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 shutdown_connections 2021-04-03T03:16:22.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.917+0000 7f7f8686d700 1 -- 172.21.2.111:0/2404548806 wait complete. 2021-04-03T03:16:22.921 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:16:22.925 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:22.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 Processor -- start 2021-04-03T03:16:22.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- start start 2021-04-03T03:16:22.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdd6c153440 0x7fdd6c080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdd6c025480 0x7fdd6c0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c0b3ca0 con 0x7fdd6c0b38c0 2021-04-03T03:16:22.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c025b90 con 0x7fdd6c153440 2021-04-03T03:16:22.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c025e30 con 0x7fdd6c025480 2021-04-03T03:16:22.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdd6c153440 0x7fdd6c080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdd6c153440 0x7fdd6c080c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56686/0 (socket says 172.21.2.111:56686) 2021-04-03T03:16:22.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 -- 172.21.2.111:0/4020753472 learned_addr learned my addr 172.21.2.111:0/4020753472 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:22.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 -- 172.21.2.111:0/4020753472 >> v2:172.21.2.115:3300/0 conn(0x7fdd6c025480 msgr2=0x7fdd6c0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 --2- 172.21.2.111:0/4020753472 >> v2:172.21.2.115:3300/0 conn(0x7fdd6c025480 0x7fdd6c0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 -- 172.21.2.111:0/4020753472 >> v1:172.21.2.115:6789/0 conn(0x7fdd6c0b38c0 legacy=0x7fdd6c080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:22.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 -- 172.21.2.111:0/4020753472 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0260d0 con 0x7fdd6c153440 2021-04-03T03:16:22.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6bfff700 1 --2- 172.21.2.111:0/4020753472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c153440 0x7fdd6c080c60 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7fdd6c0af0b0 tx=0x7fdd5c006e90).ready entity=mon.0 client_cookie=ffbbdd6c8ea0a0f7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6b7fe700 1 -- 172.21.2.111:0/4020753472 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5c009930 con 0x7fdd6c153440 2021-04-03T03:16:22.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6b7fe700 1 -- 172.21.2.111:0/4020753472 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c009a90 con 0x7fdd6c153440 2021-04-03T03:16:22.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd6b7fe700 1 -- 172.21.2.111:0/4020753472 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5c009e30 con 0x7fdd6c153440 2021-04-03T03:16:22.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- 172.21.2.111:0/4020753472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c153440 msgr2=0x7fdd6c080c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- 172.21.2.111:0/4020753472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c153440 0x7fdd6c080c60 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7fdd6c0af0b0 tx=0x7fdd5c006e90).stop 2021-04-03T03:16:22.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- 172.21.2.111:0/4020753472 shutdown_connections 2021-04-03T03:16:22.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- 172.21.2.111:0/4020753472 >> v2:172.21.2.115:3300/0 conn(0x7fdd6c025480 0x7fdd6c0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- 172.21.2.111:0/4020753472 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c153440 0x7fdd6c080c60 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:22.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- 172.21.2.111:0/4020753472 >> 172.21.2.111:0/4020753472 conn(0x7fdd6c061610 msgr2=0x7fdd6c061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:22.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- 172.21.2.111:0/4020753472 shutdown_connections 2021-04-03T03:16:22.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- 172.21.2.111:0/4020753472 wait complete. 2021-04-03T03:16:22.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 Processor -- start 2021-04-03T03:16:22.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- start start 2021-04-03T03:16:22.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c025480 0x7fdd6c0aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd6c0b38c0 0x7fdd6c066a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd6c153440 0x7fdd6c064c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:22.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0659d0 con 0x7fdd6c025480 2021-04-03T03:16:22.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c067260 con 0x7fdd6c153440 2021-04-03T03:16:22.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd71555700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c067500 con 0x7fdd6c0b38c0 2021-04-03T03:16:22.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.985+0000 7fdd70d54700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c025480 0x7fdd6c0aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:22.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c025480 0x7fdd6c0aa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56690/0 (socket says 172.21.2.111:56690) 2021-04-03T03:16:22.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 -- 172.21.2.111:0/3409502573 learned_addr learned my addr 172.21.2.111:0/3409502573 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:22.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 -- 172.21.2.111:0/3409502573 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd6c0b38c0 msgr2=0x7fdd6c066a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd6c0b38c0 0x7fdd6c066a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 -- 172.21.2.111:0/3409502573 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd6c153440 msgr2=0x7fdd6c064c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:22.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd6c153440 0x7fdd6c064c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:22.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 -- 172.21.2.111:0/3409502573 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0677f0 con 0x7fdd6c025480 2021-04-03T03:16:22.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd70d54700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c025480 0x7fdd6c0aa0d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7fdd54000f30 tx=0x7fdd54007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:22.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd697fa700 1 -- 172.21.2.111:0/3409502573 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5400e4a0 con 0x7fdd6c025480 2021-04-03T03:16:23.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd697fa700 1 -- 172.21.2.111:0/3409502573 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400e600 con 0x7fdd6c025480 2021-04-03T03:16:23.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c066f50 con 0x7fdd6c025480 2021-04-03T03:16:23.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c0650f0 con 0x7fdd6c025480 2021-04-03T03:16:23.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd697fa700 1 -- 172.21.2.111:0/3409502573 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5400e960 con 0x7fdd6c025480 2021-04-03T03:16:23.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c0a9cb0 con 0x7fdd6c025480 2021-04-03T03:16:23.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd697fa700 1 -- 172.21.2.111:0/3409502573 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdd5401a060 con 0x7fdd6c025480 2021-04-03T03:16:23.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd697fa700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd50057000 0x7fdd500594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd697fa700 1 -- 172.21.2.111:0/3409502573 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fdd5406fd30 con 0x7fdd6c025480 2021-04-03T03:16:23.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd6bfff700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd50057000 0x7fdd500594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.989+0000 7fdd6bfff700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd50057000 0x7fdd500594c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fdd6c0af0b0 tx=0x7fdd5c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:22.993+0000 7fdd697fa700 1 -- 172.21.2.111:0/3409502573 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdd54045010 con 0x7fdd6c025480 2021-04-03T03:16:23.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.117+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fdd6c067980 con 0x7fdd6c025480 2021-04-03T03:16:23.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd697fa700 1 -- 172.21.2.111:0/3409502573 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7fdd54018070 con 0x7fdd6c025480 2021-04-03T03:16:23.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd50057000 msgr2=0x7fdd500594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd50057000 0x7fdd500594c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fdd6c0af0b0 tx=0x7fdd5c015000).stop 2021-04-03T03:16:23.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c025480 msgr2=0x7fdd6c0aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c025480 0x7fdd6c0aa0d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7fdd54000f30 tx=0x7fdd54007c10).stop 2021-04-03T03:16:23.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 shutdown_connections 2021-04-03T03:16:23.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd50057000 0x7fdd500594c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd6c153440 0x7fdd6c064c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd6c025480 0x7fdd6c0aa0d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 --2- 172.21.2.111:0/3409502573 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd6c0b38c0 0x7fdd6c066a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 >> 172.21.2.111:0/3409502573 conn(0x7fdd6c061610 msgr2=0x7fdd6c0b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:23.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 shutdown_connections 2021-04-03T03:16:23.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.125+0000 7fdd71555700 1 -- 172.21.2.111:0/3409502573 wait complete. 2021-04-03T03:16:23.130 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:16:23.134 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:23.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 Processor -- start 2021-04-03T03:16:23.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- start start 2021-04-03T03:16:23.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f805f9c0 0x7f53f814a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f80b1e50 con 0x7f53f80b0960 2021-04-03T03:16:23.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f80b02a0 con 0x7f53f80b14e0 2021-04-03T03:16:23.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80b18c0 con 0x7f53f805f9c0 2021-04-03T03:16:23.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f5d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f53f80b0960 0x7f53f80b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57490/0 (socket says 172.21.2.111:57490) 2021-04-03T03:16:23.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f5d9b700 1 -- 172.21.2.111:0/3128357384 learned_addr learned my addr 172.21.2.111:0/3128357384 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:23.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f6d9d700 1 --2- 172.21.2.111:0/3128357384 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f805f9c0 0x7f53f814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f559a700 1 -- 172.21.2.111:0/3128357384 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087320915 0 0) 0x7f53f80b1e50 con 0x7f53f80b0960 2021-04-03T03:16:23.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f559a700 1 -- 172.21.2.111:0/3128357384 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d8003240 con 0x7f53f80b0960 2021-04-03T03:16:23.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f559a700 1 -- 172.21.2.111:0/3128357384 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603394409 0 0) 0x7f53f80b02a0 con 0x7f53f80b14e0 2021-04-03T03:16:23.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f559a700 1 -- 172.21.2.111:0/3128357384 --> v1:172.21.2.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f80b1e50 con 0x7f53f80b14e0 2021-04-03T03:16:23.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f6d9d700 1 -- 172.21.2.111:0/3128357384 >> v1:172.21.2.115:6789/0 conn(0x7f53f80b14e0 legacy=0x7f53f814a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f6d9d700 1 -- 172.21.2.111:0/3128357384 >> v1:172.21.2.111:6789/0 conn(0x7f53f80b0960 legacy=0x7f53f80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f6d9d700 1 -- 172.21.2.111:0/3128357384 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8060260 con 0x7f53f805f9c0 2021-04-03T03:16:23.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f6d9d700 1 --2- 172.21.2.111:0/3128357384 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f805f9c0 0x7f53f814a920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f53f00032b0 tx=0x7f53f00091d0).ready entity=mon.2 client_cookie=6aab42ccf8b1ebeb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f559a700 1 -- 172.21.2.111:0/3128357384 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53f000b920 con 0x7f53f805f9c0 2021-04-03T03:16:23.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f559a700 1 -- 172.21.2.111:0/3128357384 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f000ba80 con 0x7f53f805f9c0 2021-04-03T03:16:23.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f559a700 1 -- 172.21.2.111:0/3128357384 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53f000bbf0 con 0x7f53f805f9c0 2021-04-03T03:16:23.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- 172.21.2.111:0/3128357384 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f805f9c0 msgr2=0x7f53f814a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 --2- 172.21.2.111:0/3128357384 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f805f9c0 0x7f53f814a920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f53f00032b0 tx=0x7f53f00091d0).stop 2021-04-03T03:16:23.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- 172.21.2.111:0/3128357384 shutdown_connections 2021-04-03T03:16:23.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 --2- 172.21.2.111:0/3128357384 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f805f9c0 0x7f53f814a920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- 172.21.2.111:0/3128357384 >> 172.21.2.111:0/3128357384 conn(0x7f53f8063d70 msgr2=0x7f53f8064180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:23.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- 172.21.2.111:0/3128357384 shutdown_connections 2021-04-03T03:16:23.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- 172.21.2.111:0/3128357384 wait complete. 2021-04-03T03:16:23.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 Processor -- start 2021-04-03T03:16:23.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.189+0000 7f53f759e700 1 -- start start 2021-04-03T03:16:23.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f805f9c0 0x7f53f8159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f80b0960 0x7f53f80676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53f80b14e0 0x7f53f809efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80b6860 con 0x7f53f805f9c0 2021-04-03T03:16:23.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80b48e0 con 0x7f53f80b14e0 2021-04-03T03:16:23.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80b4b80 con 0x7f53f80b0960 2021-04-03T03:16:23.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f805f9c0 0x7f53f8159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f805f9c0 0x7f53f8159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56704/0 (socket says 172.21.2.111:56704) 2021-04-03T03:16:23.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 -- 172.21.2.111:0/1122013105 learned_addr learned my addr 172.21.2.111:0/1122013105 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:23.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f5d9b700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f80b0960 0x7f53f80676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 -- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f80b0960 msgr2=0x7f53f80676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f80b0960 0x7f53f80676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 -- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53f80b14e0 msgr2=0x7f53f809efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53f80b14e0 0x7f53f809efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 -- 172.21.2.111:0/1122013105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f80b4360 con 0x7f53f805f9c0 2021-04-03T03:16:23.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f5d9b700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f80b0960 0x7f53f80676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:23.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f659c700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f805f9c0 0x7f53f8159b10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f53e80010d0 tx=0x7f53e8007a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53ef7fe700 1 -- 172.21.2.111:0/1122013105 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53e800e4a0 con 0x7f53f805f9c0 2021-04-03T03:16:23.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53ef7fe700 1 -- 172.21.2.111:0/1122013105 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e800e600 con 0x7f53f805f9c0 2021-04-03T03:16:23.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f80b4550 con 0x7f53f805f9c0 2021-04-03T03:16:23.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f80b6270 con 0x7f53f805f9c0 2021-04-03T03:16:23.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53ef7fe700 1 -- 172.21.2.111:0/1122013105 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53e800e960 con 0x7f53f805f9c0 2021-04-03T03:16:23.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f80b4d40 con 0x7f53f805f9c0 2021-04-03T03:16:23.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53ef7fe700 1 -- 172.21.2.111:0/1122013105 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f53e8010030 con 0x7f53f805f9c0 2021-04-03T03:16:23.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53ef7fe700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53dc056ea0 0x7f53dc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53ef7fe700 1 -- 172.21.2.111:0/1122013105 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f53e806ff80 con 0x7f53f805f9c0 2021-04-03T03:16:23.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f5d9b700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53dc056ea0 0x7f53dc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.193+0000 7f53f5d9b700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53dc056ea0 0x7f53dc059360 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f53e0008000 tx=0x7f53e0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.197+0000 7f53ef7fe700 1 -- 172.21.2.111:0/1122013105 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f53e8045260 con 0x7f53f805f9c0 2021-04-03T03:16:23.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53f80b4740 con 0x7f53f805f9c0 2021-04-03T03:16:23.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53ef7fe700 1 -- 172.21.2.111:0/1122013105 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f53f80b4740 con 0x7f53f805f9c0 2021-04-03T03:16:23.322 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:23.322 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":48,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1453695075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1453695075},{"type":"v1","addr":"172.21.2.115:6833","nonce":1453695075}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:21.125389+0000","modified":"2021-04-03T03:16:22.128337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7503":{"gid":7503,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1395173601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1395173601},{"type":"v1","addr":"172.21.2.111:6835","nonce":1395173601}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-04-03T03:16:23.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53dc056ea0 msgr2=0x7f53dc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53dc056ea0 0x7f53dc059360 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f53e0008000 tx=0x7f53e0009040).stop 2021-04-03T03:16:23.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f805f9c0 msgr2=0x7f53f8159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f805f9c0 0x7f53f8159b10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f53e80010d0 tx=0x7f53e8007a70).stop 2021-04-03T03:16:23.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 shutdown_connections 2021-04-03T03:16:23.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f53dc056ea0 0x7f53dc059360 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f53f80b14e0 0x7f53f809efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f53f805f9c0 0x7f53f8159b10 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 --2- 172.21.2.111:0/1122013105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f53f80b0960 0x7f53f80676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 >> 172.21.2.111:0/1122013105 conn(0x7f53f8063d70 msgr2=0x7f53f80ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:23.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 shutdown_connections 2021-04-03T03:16:23.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.321+0000 7f53f759e700 1 -- 172.21.2.111:0/1122013105 wait complete. 2021-04-03T03:16:23.326 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 48 2021-04-03T03:16:23.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:21.125389+0000', 'modified': '2021-04-03T03:16:22.128337+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7503}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7503': {'gid': 7503, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1395173601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1395173601}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1395173601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:23.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:23.532 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:23.604 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7503,"want_state":"up:active","state":"up:active","rank_uptime":2.4724266660000001,"mdsmap_epoch":48,"osdmap_epoch":113,"osdmap_epoch_barrier":112,"uptime":6.6241466280000001} 2021-04-03T03:16:23.604 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7503, "mdsmap_epoch": 48, "osdmap_epoch": 113, "osdmap_epoch_barrier": 112, "rank_uptime": 2.472426666, "state": "up:active", "uptime": 6.624146628, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:23.605 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:23.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 Processor -- start 2021-04-03T03:16:23.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- start start 2021-04-03T03:16:23.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f940c0603f0 0x7f940c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f940c14b700 0x7f940c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940c0607d0 con 0x7f940c0b12b0 2021-04-03T03:16:23.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940c0af400 con 0x7f940c14b700 2021-04-03T03:16:23.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940c0af6a0 con 0x7f940c0603f0 2021-04-03T03:16:23.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f940c14b700 0x7f940c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f940c14b700 0x7f940c0b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56714/0 (socket says 172.21.2.111:56714) 2021-04-03T03:16:23.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 -- 172.21.2.111:0/1725761328 learned_addr learned my addr 172.21.2.111:0/1725761328 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:23.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 -- 172.21.2.111:0/1725761328 >> v2:172.21.2.115:3300/0 conn(0x7f940c0603f0 msgr2=0x7f940c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:23.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 --2- 172.21.2.111:0/1725761328 >> v2:172.21.2.115:3300/0 conn(0x7f940c0603f0 0x7f940c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 -- 172.21.2.111:0/1725761328 >> v1:172.21.2.115:6789/0 conn(0x7f940c0b12b0 legacy=0x7f940c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:23.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 -- 172.21.2.111:0/1725761328 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c0afb40 con 0x7f940c14b700 2021-04-03T03:16:23.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9411474700 1 --2- 172.21.2.111:0/1725761328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0b0580 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f940c06a3b0 tx=0x7f9408006e90).ready entity=mon.0 client_cookie=f375d21e2a97e289 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9410c73700 1 -- 172.21.2.111:0/1725761328 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9408009930 con 0x7f940c14b700 2021-04-03T03:16:23.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9410c73700 1 -- 172.21.2.111:0/1725761328 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9408009a90 con 0x7f940c14b700 2021-04-03T03:16:23.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9410c73700 1 -- 172.21.2.111:0/1725761328 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9408009e30 con 0x7f940c14b700 2021-04-03T03:16:23.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- 172.21.2.111:0/1725761328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 msgr2=0x7f940c0b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- 172.21.2.111:0/1725761328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0b0580 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f940c06a3b0 tx=0x7f9408006e90).stop 2021-04-03T03:16:23.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- 172.21.2.111:0/1725761328 shutdown_connections 2021-04-03T03:16:23.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- 172.21.2.111:0/1725761328 >> v2:172.21.2.115:3300/0 conn(0x7f940c0603f0 0x7f940c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- 172.21.2.111:0/1725761328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0b0580 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- 172.21.2.111:0/1725761328 >> 172.21.2.111:0/1725761328 conn(0x7f940c0b61c0 msgr2=0x7f940c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:23.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- 172.21.2.111:0/1725761328 shutdown_connections 2021-04-03T03:16:23.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- 172.21.2.111:0/1725761328 wait complete. 2021-04-03T03:16:23.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 Processor -- start 2021-04-03T03:16:23.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- start start 2021-04-03T03:16:23.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f940c0603f0 0x7f940c0a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f940c0b12b0 0x7f940c067500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0aa120 con 0x7f940c14b700 2021-04-03T03:16:23.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0ab870 con 0x7f940c0603f0 2021-04-03T03:16:23.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412c77700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0a8360 con 0x7f940c0b12b0 2021-04-03T03:16:23.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412476700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412476700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56718/0 (socket says 172.21.2.111:56718) 2021-04-03T03:16:23.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.661+0000 7f9412476700 1 -- 172.21.2.111:0/3739921136 learned_addr learned my addr 172.21.2.111:0/3739921136 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:23.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412476700 1 -- 172.21.2.111:0/3739921136 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f940c0b12b0 msgr2=0x7f940c067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412476700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f940c0b12b0 0x7f940c067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412476700 1 -- 172.21.2.111:0/3739921136 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f940c0603f0 msgr2=0x7f940c0a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412476700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f940c0603f0 0x7f940c0a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412476700 1 -- 172.21.2.111:0/3739921136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c0ab170 con 0x7f940c14b700 2021-04-03T03:16:23.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412476700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0679f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f94040032d0 tx=0x7f9404006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f93feffd700 1 -- 172.21.2.111:0/3739921136 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f940400c960 con 0x7f940c14b700 2021-04-03T03:16:23.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f93feffd700 1 -- 172.21.2.111:0/3739921136 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940400cac0 con 0x7f940c14b700 2021-04-03T03:16:23.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f93feffd700 1 -- 172.21.2.111:0/3739921136 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f940400ce20 con 0x7f940c14b700 2021-04-03T03:16:23.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c067ee0 con 0x7f940c14b700 2021-04-03T03:16:23.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c0abcf0 con 0x7f940c14b700 2021-04-03T03:16:23.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c0a8730 con 0x7f940c14b700 2021-04-03T03:16:23.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f93feffd700 1 -- 172.21.2.111:0/3739921136 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f940401c070 con 0x7f940c14b700 2021-04-03T03:16:23.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f93feffd700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93ec056fb0 0x7f93ec059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f93feffd700 1 -- 172.21.2.111:0/3739921136 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f940406fd80 con 0x7f940c14b700 2021-04-03T03:16:23.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9411c75700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93ec056fb0 0x7f93ec059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.665+0000 7f9411c75700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93ec056fb0 0x7f93ec059470 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f9400002f30 tx=0x7f9400006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.669+0000 7f93feffd700 1 -- 172.21.2.111:0/3739921136 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f940c0a8730 con 0x7f940c14b700 2021-04-03T03:16:23.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.757+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f940c0a8730 con 0x7f940c14b700 2021-04-03T03:16:23.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f93feffd700 1 -- 172.21.2.111:0/3739921136 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f940c0a8730 con 0x7f940c14b700 2021-04-03T03:16:23.761 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:23.761 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":113,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:22.917071+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:19.175402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:20.080209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:23.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93ec056fb0 msgr2=0x7f93ec059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93ec056fb0 0x7f93ec059470 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f9400002f30 tx=0x7f9400006a60).stop 2021-04-03T03:16:23.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 msgr2=0x7f940c0679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0679f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f94040032d0 tx=0x7f9404006f90).stop 2021-04-03T03:16:23.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 shutdown_connections 2021-04-03T03:16:23.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f940c14b700 0x7f940c0679f0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f93ec056fb0 0x7f93ec059470 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f940c0603f0 0x7f940c0a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 --2- 172.21.2.111:0/3739921136 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f940c0b12b0 0x7f940c067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 >> 172.21.2.111:0/3739921136 conn(0x7f940c0b61c0 msgr2=0x7f940c148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:23.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 shutdown_connections 2021-04-03T03:16:23.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.761+0000 7f9412c77700 1 -- 172.21.2.111:0/3739921136 wait complete. 2021-04-03T03:16:23.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:16:19.175402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:16:20.080209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:16:23.770 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:23.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.825+0000 7f4601ad9700 1 Processor -- start 2021-04-03T03:16:23.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.825+0000 7f4601ad9700 1 -- start start 2021-04-03T03:16:23.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.825+0000 7f4601ad9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f45fc0add50 0x7f45fc147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.825+0000 7f4601ad9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45fc0607d0 con 0x7f45fc0603f0 2021-04-03T03:16:23.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.825+0000 7f4601ad9700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45fc14b090 con 0x7f45fc144ef0 2021-04-03T03:16:23.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.825+0000 7f4601ad9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45fc14b330 con 0x7f45fc0add50 2021-04-03T03:16:23.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.825+0000 7f45fbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f45fc0add50 0x7f45fc147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f45fc0603f0 0x7f45fc1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57516/0 (socket says 172.21.2.111:57516) 2021-04-03T03:16:23.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 -- 172.21.2.111:0/2876851255 learned_addr learned my addr 172.21.2.111:0/2876851255 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:23.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45fbfff700 1 -- 172.21.2.111:0/2876851255 >> v1:172.21.2.115:6789/0 conn(0x7f45fc144ef0 legacy=0x7f45fc147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:23.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45fbfff700 1 -- 172.21.2.111:0/2876851255 >> v1:172.21.2.111:6789/0 conn(0x7f45fc0603f0 legacy=0x7f45fc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45fbfff700 1 -- 172.21.2.111:0/2876851255 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc14b5d0 con 0x7f45fc0add50 2021-04-03T03:16:23.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45fbfff700 1 --2- 172.21.2.111:0/2876851255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0add50 0x7f45fc147740 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f45fc0b5600 tx=0x7f45e4006dd0).ready entity=mon.0 client_cookie=49df2b188d9150f4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45fb7fe700 1 -- 172.21.2.111:0/2876851255 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e40097b0 con 0x7f45fc0add50 2021-04-03T03:16:23.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45fb7fe700 1 -- 172.21.2.111:0/2876851255 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e4009910 con 0x7f45fc0add50 2021-04-03T03:16:23.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45fb7fe700 1 -- 172.21.2.111:0/2876851255 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e4009c70 con 0x7f45fc0add50 2021-04-03T03:16:23.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/2876851255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0add50 msgr2=0x7f45fc147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 --2- 172.21.2.111:0/2876851255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0add50 0x7f45fc147740 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f45fc0b5600 tx=0x7f45e4006dd0).stop 2021-04-03T03:16:23.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/2876851255 shutdown_connections 2021-04-03T03:16:23.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 --2- 172.21.2.111:0/2876851255 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0add50 0x7f45fc147740 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/2876851255 >> 172.21.2.111:0/2876851255 conn(0x7f45fc0b61c0 msgr2=0x7f45fc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:23.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/2876851255 shutdown_connections 2021-04-03T03:16:23.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/2876851255 wait complete. 2021-04-03T03:16:23.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 Processor -- start 2021-04-03T03:16:23.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- start start 2021-04-03T03:16:23.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0603f0 0x7f45fc0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45fc0add50 0x7f45fc15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45fc144ef0 0x7f45fc15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc15b180 con 0x7f45fc0603f0 2021-04-03T03:16:23.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc064da0 con 0x7f45fc144ef0 2021-04-03T03:16:23.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc064b00 con 0x7f45fc0add50 2021-04-03T03:16:23.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0603f0 0x7f45fc0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0603f0 0x7f45fc0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56730/0 (socket says 172.21.2.111:56730) 2021-04-03T03:16:23.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 -- 172.21.2.111:0/278748002 learned_addr learned my addr 172.21.2.111:0/278748002 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:23.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 -- 172.21.2.111:0/278748002 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45fc0add50 msgr2=0x7f45fc15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.840 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45fc0add50 0x7f45fc15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 -- 172.21.2.111:0/278748002 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45fc144ef0 msgr2=0x7f45fc15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45fc144ef0 0x7f45fc15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 -- 172.21.2.111:0/278748002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc15a300 con 0x7f45fc0603f0 2021-04-03T03:16:23.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4600ad7700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0603f0 0x7f45fc0b48b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f45fc0ae520 tx=0x7f45ec007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45f9ffb700 1 -- 172.21.2.111:0/278748002 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45ec007320 con 0x7f45fc0603f0 2021-04-03T03:16:23.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45f9ffb700 1 -- 172.21.2.111:0/278748002 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec007480 con 0x7f45fc0603f0 2021-04-03T03:16:23.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc0a9000 con 0x7f45fc0603f0 2021-04-03T03:16:23.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc0a9ae0 con 0x7f45fc0603f0 2021-04-03T03:16:23.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f45f9ffb700 1 -- 172.21.2.111:0/278748002 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45ec0057e0 con 0x7f45fc0603f0 2021-04-03T03:16:23.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.829+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d40033f0 con 0x7f45fc0603f0 2021-04-03T03:16:23.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.833+0000 7f45f9ffb700 1 -- 172.21.2.111:0/278748002 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f45ec010030 con 0x7f45fc0603f0 2021-04-03T03:16:23.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.833+0000 7f45f9ffb700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45e0056f70 0x7f45e0059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.833+0000 7f45f9ffb700 1 -- 172.21.2.111:0/278748002 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f45ec06ff10 con 0x7f45fc0603f0 2021-04-03T03:16:23.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.833+0000 7f45fbfff700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45e0056f70 0x7f45e0059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:23.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.833+0000 7f45fbfff700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45e0056f70 0x7f45e0059430 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f45fc0b5600 tx=0x7f45e4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:23.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.833+0000 7f45f9ffb700 1 -- 172.21.2.111:0/278748002 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f45ec045190 con 0x7f45fc0603f0 2021-04-03T03:16:23.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.925+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f45d4003c20 con 0x7f45fc0603f0 2021-04-03T03:16:23.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.925+0000 7f45f9ffb700 1 -- 172.21.2.111:0/278748002 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7f45ec0487b0 con 0x7f45fc0603f0 2021-04-03T03:16:23.928 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:23.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45e0056f70 msgr2=0x7f45e0059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45e0056f70 0x7f45e0059430 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f45fc0b5600 tx=0x7f45e4015000).stop 2021-04-03T03:16:23.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0603f0 msgr2=0x7f45fc0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:23.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0603f0 0x7f45fc0b48b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f45fc0ae520 tx=0x7f45ec007be0).stop 2021-04-03T03:16:23.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 shutdown_connections 2021-04-03T03:16:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45e0056f70 0x7f45e0059430 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45fc144ef0 0x7f45fc15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45fc0603f0 0x7f45fc0b48b0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 --2- 172.21.2.111:0/278748002 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45fc0add50 0x7f45fc15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:23.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 >> 172.21.2.111:0/278748002 conn(0x7f45fc0b61c0 msgr2=0x7f45fc0b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:23.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 shutdown_connections 2021-04-03T03:16:23.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.929+0000 7f4601ad9700 1 -- 172.21.2.111:0/278748002 wait complete. 2021-04-03T03:16:23.936 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:23.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 Processor -- start 2021-04-03T03:16:23.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- start start 2021-04-03T03:16:23.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 0x7f715c046850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f715c045d80 0x7f715c0d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:23.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c045680 con 0x7f715c0cdb60 2021-04-03T03:16:23.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c044df0 con 0x7f715c045d80 2021-04-03T03:16:23.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0d4820 con 0x7f715c046470 2021-04-03T03:16:24.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7162ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f715c045d80 0x7f715c0d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7163fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f715c0cdb60 0x7f715c0438a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57532/0 (socket says 172.21.2.111:57532) 2021-04-03T03:16:24.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7163fff700 1 -- 172.21.2.111:0/2180959606 learned_addr learned my addr 172.21.2.111:0/2180959606 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:24.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7162ffd700 1 -- 172.21.2.111:0/2180959606 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 msgr2=0x7f715c046850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:24.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7162ffd700 1 --2- 172.21.2.111:0/2180959606 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 0x7f715c046850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7162ffd700 1 -- 172.21.2.111:0/2180959606 >> v1:172.21.2.111:6789/0 conn(0x7f715c0cdb60 legacy=0x7f715c0438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f71627fc700 1 -- 172.21.2.111:0/2180959606 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960692827 0 0) 0x7f715c045680 con 0x7f715c0cdb60 2021-04-03T03:16:24.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7162ffd700 1 -- 172.21.2.111:0/2180959606 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c046e00 con 0x7f715c045d80 2021-04-03T03:16:24.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7162ffd700 1 --2- 172.21.2.111:0/2180959606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c045d80 0x7f715c0d4a70 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f7158002f30 tx=0x7f7158006dd0).ready entity=mon.0 client_cookie=b1ad3c5201f42925 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f71627fc700 1 -- 172.21.2.111:0/2180959606 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71580097f0 con 0x7f715c045d80 2021-04-03T03:16:24.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f71627fc700 1 -- 172.21.2.111:0/2180959606 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7158009950 con 0x7f715c045d80 2021-04-03T03:16:24.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f71627fc700 1 -- 172.21.2.111:0/2180959606 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7158009cb0 con 0x7f715c045d80 2021-04-03T03:16:24.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- 172.21.2.111:0/2180959606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c045d80 msgr2=0x7f715c0d4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 --2- 172.21.2.111:0/2180959606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c045d80 0x7f715c0d4a70 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f7158002f30 tx=0x7f7158006dd0).stop 2021-04-03T03:16:24.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- 172.21.2.111:0/2180959606 shutdown_connections 2021-04-03T03:16:24.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 --2- 172.21.2.111:0/2180959606 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 0x7f715c046850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 --2- 172.21.2.111:0/2180959606 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c045d80 0x7f715c0d4a70 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- 172.21.2.111:0/2180959606 >> 172.21.2.111:0/2180959606 conn(0x7f715c04a6d0 msgr2=0x7f715c04aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- 172.21.2.111:0/2180959606 shutdown_connections 2021-04-03T03:16:24.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.993+0000 7f7168c60700 1 -- 172.21.2.111:0/2180959606 wait complete. 2021-04-03T03:16:24.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 Processor -- start 2021-04-03T03:16:24.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 -- start start 2021-04-03T03:16:24.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f715c045d80 0x7f715c0cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 0x7f715c0e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c0cdb60 0x7f715c0e3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0ce050 con 0x7f715c0cdb60 2021-04-03T03:16:24.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0ceb60 con 0x7f715c045d80 2021-04-03T03:16:24.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0cee00 con 0x7f715c046470 2021-04-03T03:16:24.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c0cdb60 0x7f715c0e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c0cdb60 0x7f715c0e3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56746/0 (socket says 172.21.2.111:56746) 2021-04-03T03:16:24.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 -- 172.21.2.111:0/3166087078 learned_addr learned my addr 172.21.2.111:0/3166087078 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:24.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 -- 172.21.2.111:0/3166087078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 msgr2=0x7f715c0e2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 0x7f715c0e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 -- 172.21.2.111:0/3166087078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f715c045d80 msgr2=0x7f715c0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f715c045d80 0x7f715c0cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 -- 172.21.2.111:0/3166087078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c0ce480 con 0x7f715c0cdb60 2021-04-03T03:16:24.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7163fff700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c0cdb60 0x7f715c0e3190 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f7164069660 tx=0x7f716405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7160ff9700 1 -- 172.21.2.111:0/3166087078 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f716405df60 con 0x7f715c0cdb60 2021-04-03T03:16:24.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7160ff9700 1 -- 172.21.2.111:0/3166087078 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640699f0 con 0x7f715c0cdb60 2021-04-03T03:16:24.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c0d7e30 con 0x7f715c0cdb60 2021-04-03T03:16:24.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c03e580 con 0x7f715c0cdb60 2021-04-03T03:16:24.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7160ff9700 1 -- 172.21.2.111:0/3166087078 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f716406b5d0 con 0x7f715c0cdb60 2021-04-03T03:16:24.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71400033f0 con 0x7f715c0cdb60 2021-04-03T03:16:24.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7160ff9700 1 -- 172.21.2.111:0/3166087078 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7164061110 con 0x7f715c0cdb60 2021-04-03T03:16:24.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7160ff9700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7144057050 0x7f7144059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f7160ff9700 1 -- 172.21.2.111:0/3166087078 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f71640dedf0 con 0x7f715c0cdb60 2021-04-03T03:16:24.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f71637fe700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7144057050 0x7f7144059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:23.997+0000 7f71637fe700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7144057050 0x7f7144059510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f7154002f30 tx=0x7f7154006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.001+0000 7f7160ff9700 1 -- 172.21.2.111:0/3166087078 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f71640b41e0 con 0x7f715c0cdb60 2021-04-03T03:16:24.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7140003c20 con 0x7f715c0cdb60 2021-04-03T03:16:24.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7160ff9700 1 -- 172.21.2.111:0/3166087078 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7f71640b7800 con 0x7f715c0cdb60 2021-04-03T03:16:24.096 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:24.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7144057050 msgr2=0x7f7144059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7144057050 0x7f7144059510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f7154002f30 tx=0x7f7154006a60).stop 2021-04-03T03:16:24.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c0cdb60 msgr2=0x7f715c0e3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c0cdb60 0x7f715c0e3190 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f7164069660 tx=0x7f716405ad70).stop 2021-04-03T03:16:24.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 shutdown_connections 2021-04-03T03:16:24.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f715c0cdb60 0x7f715c0e3190 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7144057050 0x7f7144059510 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f715c045d80 0x7f715c0cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 --2- 172.21.2.111:0/3166087078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f715c046470 0x7f715c0e2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 >> 172.21.2.111:0/3166087078 conn(0x7f715c04a6d0 msgr2=0x7f715c048de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 shutdown_connections 2021-04-03T03:16:24.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.093+0000 7f7168c60700 1 -- 172.21.2.111:0/3166087078 wait complete. 2021-04-03T03:16:24.103 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:16:24.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 Processor -- start 2021-04-03T03:16:24.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- start start 2021-04-03T03:16:24.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f05180603f0 0x7f05181447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f05180add50 0x7f0518144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180b17b0 0x7f0518147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05181451c0 con 0x7f05180603f0 2021-04-03T03:16:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05180607d0 con 0x7f05180add50 2021-04-03T03:16:24.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051814b090 con 0x7f05180b17b0 2021-04-03T03:16:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f05180603f0 0x7f05181447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f05180603f0 0x7f05181447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56752/0 (socket says 172.21.2.111:56752) 2021-04-03T03:16:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 -- 172.21.2.111:0/2825547933 learned_addr learned my addr 172.21.2.111:0/2825547933 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 -- 172.21.2.111:0/2825547933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180b17b0 msgr2=0x7f0518147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:24.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 --2- 172.21.2.111:0/2825547933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180b17b0 0x7f0518147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 -- 172.21.2.111:0/2825547933 >> v2:172.21.2.115:3300/0 conn(0x7f05180add50 msgr2=0x7f0518144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 --2- 172.21.2.111:0/2825547933 >> v2:172.21.2.115:3300/0 conn(0x7f05180add50 0x7f0518144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 -- 172.21.2.111:0/2825547933 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051814b330 con 0x7f05180603f0 2021-04-03T03:16:24.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051d67e700 1 --2- 172.21.2.111:0/2825547933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05181447e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f050c002f30 tx=0x7f050c006e90).ready entity=mon.0 client_cookie=b1d85e3da0c779e4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f050bfff700 1 -- 172.21.2.111:0/2825547933 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f050c009920 con 0x7f05180603f0 2021-04-03T03:16:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f050bfff700 1 -- 172.21.2.111:0/2825547933 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c009a80 con 0x7f05180603f0 2021-04-03T03:16:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f050bfff700 1 -- 172.21.2.111:0/2825547933 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f050c009de0 con 0x7f05180603f0 2021-04-03T03:16:24.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- 172.21.2.111:0/2825547933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 msgr2=0x7f05181447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 --2- 172.21.2.111:0/2825547933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05181447e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f050c002f30 tx=0x7f050c006e90).stop 2021-04-03T03:16:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- 172.21.2.111:0/2825547933 shutdown_connections 2021-04-03T03:16:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 --2- 172.21.2.111:0/2825547933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180b17b0 0x7f0518147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 --2- 172.21.2.111:0/2825547933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05181447e0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 --2- 172.21.2.111:0/2825547933 >> v2:172.21.2.115:3300/0 conn(0x7f05180add50 0x7f0518144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- 172.21.2.111:0/2825547933 >> 172.21.2.111:0/2825547933 conn(0x7f05180b61c0 msgr2=0x7f051805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- 172.21.2.111:0/2825547933 shutdown_connections 2021-04-03T03:16:24.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- 172.21.2.111:0/2825547933 wait complete. 2021-04-03T03:16:24.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 Processor -- start 2021-04-03T03:16:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.161+0000 7f051e680700 1 -- start start 2021-04-03T03:16:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05180b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180add50 0x7f0518147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05180b17b0 0x7f05180a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05181482f0 con 0x7f05180603f0 2021-04-03T03:16:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05180ab5f0 con 0x7f05180b17b0 2021-04-03T03:16:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05180ab890 con 0x7f05180add50 2021-04-03T03:16:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05180b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05180b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56756/0 (socket says 172.21.2.111:56756) 2021-04-03T03:16:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 -- 172.21.2.111:0/2153363250 learned_addr learned my addr 172.21.2.111:0/2153363250 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 -- 172.21.2.111:0/2153363250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180add50 msgr2=0x7f0518147e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180add50 0x7f0518147e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 -- 172.21.2.111:0/2153363250 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05180b17b0 msgr2=0x7f05180a78a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05180b17b0 0x7f05180a78a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 -- 172.21.2.111:0/2153363250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05180a7360 con 0x7f05180603f0 2021-04-03T03:16:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051d67e700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05180b4850 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f050c002f30 tx=0x7f050c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f050a7fc700 1 -- 172.21.2.111:0/2153363250 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f050c011e50 con 0x7f05180603f0 2021-04-03T03:16:24.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f050a7fc700 1 -- 172.21.2.111:0/2153363250 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c01e460 con 0x7f05180603f0 2021-04-03T03:16:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05180a7550 con 0x7f05180603f0 2021-04-03T03:16:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f050a7fc700 1 -- 172.21.2.111:0/2153363250 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f050c01e7c0 con 0x7f05180603f0 2021-04-03T03:16:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05180a7fa0 con 0x7f05180603f0 2021-04-03T03:16:24.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05180aba50 con 0x7f05180603f0 2021-04-03T03:16:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f050a7fc700 1 -- 172.21.2.111:0/2153363250 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f050c01a040 con 0x7f05180603f0 2021-04-03T03:16:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f050a7fc700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f04f8056f20 0x7f04f80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f050a7fc700 1 -- 172.21.2.111:0/2153363250 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f050c071360 con 0x7f05180603f0 2021-04-03T03:16:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051ce7d700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f04f8056f20 0x7f04f80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.165+0000 7f051ce7d700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f04f8056f20 0x7f04f80593e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f0514002f30 tx=0x7f05140069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.169+0000 7f050a7fc700 1 -- 172.21.2.111:0/2153363250 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f050c046750 con 0x7f05180603f0 2021-04-03T03:16:24.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.261+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0518148430 con 0x7f05180603f0 2021-04-03T03:16:24.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.261+0000 7f050a7fc700 1 -- 172.21.2.111:0/2153363250 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+10 (secure 0 0 0) 0x7f050c023020 con 0x7f05180603f0 2021-04-03T03:16:24.264 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:24.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.261+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f04f8056f20 msgr2=0x7f04f80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.261+0000 7f051e680700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f04f8056f20 0x7f04f80593e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f0514002f30 tx=0x7f05140069f0).stop 2021-04-03T03:16:24.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 msgr2=0x7f05180b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05180b4850 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f050c002f30 tx=0x7f050c015000).stop 2021-04-03T03:16:24.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 shutdown_connections 2021-04-03T03:16:24.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f04f8056f20 0x7f04f80593e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05180b17b0 0x7f05180a78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05180603f0 0x7f05180b4850 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 --2- 172.21.2.111:0/2153363250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05180add50 0x7f0518147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 >> 172.21.2.111:0/2153363250 conn(0x7f05180b61c0 msgr2=0x7f051805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 shutdown_connections 2021-04-03T03:16:24.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.265+0000 7f051e680700 1 -- 172.21.2.111:0/2153363250 wait complete. 2021-04-03T03:16:24.272 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:16:24.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1823700 1 Processor -- start 2021-04-03T03:16:24.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1823700 1 -- start start 2021-04-03T03:16:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1823700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1823700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3aac144ef0 0x7f3aac147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1823700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0607d0 con 0x7f3aac0603f0 2021-04-03T03:16:24.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1823700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac14b090 con 0x7f3aac144ef0 2021-04-03T03:16:24.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1823700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac14b330 con 0x7f3aac0add50 2021-04-03T03:16:24.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3aabfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3aabfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:40116/0 (socket says 172.21.2.111:40116) 2021-04-03T03:16:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3aabfff700 1 -- 172.21.2.111:0/3283127608 learned_addr learned my addr 172.21.2.111:0/3283127608 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1022700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3aac144ef0 0x7f3aac147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab0821700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f3aac0603f0 0x7f3aac1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:41860/0 (socket says 172.21.2.111:41860) 2021-04-03T03:16:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1022700 1 -- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 msgr2=0x7f3aac147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1022700 1 --2- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1022700 1 -- 172.21.2.111:0/3283127608 >> v1:172.21.2.115:6789/0 conn(0x7f3aac0603f0 legacy=0x7f3aac1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.329+0000 7f3ab1022700 1 -- 172.21.2.111:0/3283127608 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac14b5d0 con 0x7f3aac144ef0 2021-04-03T03:16:24.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aabfff700 1 --2- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:16:24.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1022700 1 --2- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 0x7f3aac147c30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3aa00010d0 tx=0x7f3aa40091d0).ready entity=mon.1 client_cookie=51db1103c30951b8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aab7fe700 1 -- 172.21.2.111:0/3283127608 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa400b860 con 0x7f3aac144ef0 2021-04-03T03:16:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aab7fe700 1 -- 172.21.2.111:0/3283127608 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa400b9c0 con 0x7f3aac144ef0 2021-04-03T03:16:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aab7fe700 1 -- 172.21.2.111:0/3283127608 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa400bb30 con 0x7f3aac144ef0 2021-04-03T03:16:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 msgr2=0x7f3aac147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 0x7f3aac147c30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3aa00010d0 tx=0x7f3aa40091d0).stop 2021-04-03T03:16:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- 172.21.2.111:0/3283127608 shutdown_connections 2021-04-03T03:16:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 0x7f3aac147c30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3283127608 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- 172.21.2.111:0/3283127608 >> 172.21.2.111:0/3283127608 conn(0x7f3aac0b61c0 msgr2=0x7f3aac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- 172.21.2.111:0/3283127608 shutdown_connections 2021-04-03T03:16:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- 172.21.2.111:0/3283127608 wait complete. 2021-04-03T03:16:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 Processor -- start 2021-04-03T03:16:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- start start 2021-04-03T03:16:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3aac0603f0 0x7f3aac063df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac0642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 0x7f3aac0ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0a8da0 con 0x7f3aac0603f0 2021-04-03T03:16:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1573b0 con 0x7f3aac144ef0 2021-04-03T03:16:24.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac157620 con 0x7f3aac0add50 2021-04-03T03:16:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3aac0603f0 0x7f3aac063df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3aac0603f0 0x7f3aac063df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56770/0 (socket says 172.21.2.111:56770) 2021-04-03T03:16:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 -- 172.21.2.111:0/3781743399 learned_addr learned my addr 172.21.2.111:0/3781743399 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1022700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 0x7f3aac0ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aabfff700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac0642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 -- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 msgr2=0x7f3aac0642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac0642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 -- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 msgr2=0x7f3aac0ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 0x7f3aac0ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 -- 172.21.2.111:0/3781743399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac1578c0 con 0x7f3aac0603f0 2021-04-03T03:16:24.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab0821700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3aac0603f0 0x7f3aac063df0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f3aa0001190 tx=0x7f3aa0007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aa9ffb700 1 -- 172.21.2.111:0/3781743399 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa0007320 con 0x7f3aac0603f0 2021-04-03T03:16:24.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aa9ffb700 1 -- 172.21.2.111:0/3781743399 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0005530 con 0x7f3aac0603f0 2021-04-03T03:16:24.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac0a9d20 con 0x7f3aac0603f0 2021-04-03T03:16:24.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac0a99b0 con 0x7f3aac0603f0 2021-04-03T03:16:24.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.333+0000 7f3aa9ffb700 1 -- 172.21.2.111:0/3781743399 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa0005890 con 0x7f3aac0603f0 2021-04-03T03:16:24.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.337+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac157a50 con 0x7f3aac0603f0 2021-04-03T03:16:24.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.337+0000 7f3aa9ffb700 1 -- 172.21.2.111:0/3781743399 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3aa0010030 con 0x7f3aac0603f0 2021-04-03T03:16:24.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.337+0000 7f3aa9ffb700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3a8c057000 0x7f3a8c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.337+0000 7f3aa9ffb700 1 -- 172.21.2.111:0/3781743399 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3aa0070210 con 0x7f3aac0603f0 2021-04-03T03:16:24.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.337+0000 7f3aabfff700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3a8c057000 0x7f3a8c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.337+0000 7f3aabfff700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3a8c057000 0x7f3a8c0594c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f3a9c00b000 tx=0x7f3a9c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.337+0000 7f3aa9ffb700 1 -- 172.21.2.111:0/3781743399 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3aa00454f0 con 0x7f3aac0603f0 2021-04-03T03:16:24.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.429+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3aac057660 con 0x7f3aac0603f0 2021-04-03T03:16:24.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.429+0000 7f3aa9ffb700 1 -- 172.21.2.111:0/3781743399 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3aa0048b10 con 0x7f3aac0603f0 2021-04-03T03:16:24.432 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:24.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3a8c057000 msgr2=0x7f3a8c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3a8c057000 0x7f3a8c0594c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f3a9c00b000 tx=0x7f3a9c008040).stop 2021-04-03T03:16:24.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3aac0603f0 msgr2=0x7f3aac063df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3aac0603f0 0x7f3aac063df0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f3aa0001190 tx=0x7f3aa0007be0).stop 2021-04-03T03:16:24.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 shutdown_connections 2021-04-03T03:16:24.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3a8c057000 0x7f3a8c0594c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3aac144ef0 0x7f3aac0ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3aac0603f0 0x7f3aac063df0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 --2- 172.21.2.111:0/3781743399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3aac0add50 0x7f3aac0642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 >> 172.21.2.111:0/3781743399 conn(0x7f3aac0b61c0 msgr2=0x7f3aac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 shutdown_connections 2021-04-03T03:16:24.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.433+0000 7f3ab1823700 1 -- 172.21.2.111:0/3781743399 wait complete. 2021-04-03T03:16:24.440 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:16:24.440 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:24.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.493+0000 7f256b120700 1 Processor -- start 2021-04-03T03:16:24.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.493+0000 7f256b120700 1 -- start start 2021-04-03T03:16:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f25641430b0 0x7f2564143490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25640b1390 con 0x7f25640b03a0 2021-04-03T03:16:24.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2564143980 con 0x7f25640b0f40 2021-04-03T03:16:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25640607e0 con 0x7f25641430b0 2021-04-03T03:16:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a11e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f25641430b0 0x7f2564143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a11e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f25641430b0 0x7f2564143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56778/0 (socket says 172.21.2.111:56778) 2021-04-03T03:16:24.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a11e700 1 -- 172.21.2.111:0/3948159854 learned_addr learned my addr 172.21.2.111:0/3948159854 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:24.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a11e700 1 -- 172.21.2.111:0/3948159854 >> v1:172.21.2.115:6789/0 conn(0x7f25640b0f40 legacy=0x7f25640600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:24.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a11e700 1 -- 172.21.2.111:0/3948159854 >> v1:172.21.2.111:6789/0 conn(0x7f25640b03a0 legacy=0x7f25640b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a11e700 1 -- 172.21.2.111:0/3948159854 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640ad750 con 0x7f25641430b0 2021-04-03T03:16:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a11e700 1 --2- 172.21.2.111:0/3948159854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f2564143490 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f2564062370 tx=0x7f2560006e90).ready entity=mon.0 client_cookie=4cf83bd3c65fdb2f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256911c700 1 -- 172.21.2.111:0/3948159854 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2560009980 con 0x7f25641430b0 2021-04-03T03:16:24.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256911c700 1 -- 172.21.2.111:0/3948159854 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560009ae0 con 0x7f25641430b0 2021-04-03T03:16:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256911c700 1 -- 172.21.2.111:0/3948159854 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2560009e80 con 0x7f25641430b0 2021-04-03T03:16:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- 172.21.2.111:0/3948159854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 msgr2=0x7f2564143490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 --2- 172.21.2.111:0/3948159854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f2564143490 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f2564062370 tx=0x7f2560006e90).stop 2021-04-03T03:16:24.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- 172.21.2.111:0/3948159854 shutdown_connections 2021-04-03T03:16:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 --2- 172.21.2.111:0/3948159854 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f2564143490 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- 172.21.2.111:0/3948159854 >> 172.21.2.111:0/3948159854 conn(0x7f25640b61c0 msgr2=0x7f256405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- 172.21.2.111:0/3948159854 shutdown_connections 2021-04-03T03:16:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- 172.21.2.111:0/3948159854 wait complete. 2021-04-03T03:16:24.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 Processor -- start 2021-04-03T03:16:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- start start 2021-04-03T03:16:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25640b03a0 0x7f25640a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f25640b0f40 0x7f256413fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f25640aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640aa9b0 con 0x7f25641430b0 2021-04-03T03:16:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640b4ba0 con 0x7f25640b0f40 2021-04-03T03:16:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256b120700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25640b4e40 con 0x7f25640b03a0 2021-04-03T03:16:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a91f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f25640aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a91f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f25640aade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56786/0 (socket says 172.21.2.111:56786) 2021-04-03T03:16:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.497+0000 7f256a91f700 1 -- 172.21.2.111:0/2630851111 learned_addr learned my addr 172.21.2.111:0/2630851111 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a91f700 1 -- 172.21.2.111:0/2630851111 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25640b03a0 msgr2=0x7f25640a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a91f700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25640b03a0 0x7f25640a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a91f700 1 -- 172.21.2.111:0/2630851111 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f25640b0f40 msgr2=0x7f256413fb40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a91f700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f25640b0f40 0x7f256413fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a91f700 1 -- 172.21.2.111:0/2630851111 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564140030 con 0x7f25641430b0 2021-04-03T03:16:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a91f700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f25640aade0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f255c0012b0 tx=0x7f255c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f25577fe700 1 -- 172.21.2.111:0/2630851111 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f255c00a7b0 con 0x7f25641430b0 2021-04-03T03:16:24.508 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f25577fe700 1 -- 172.21.2.111:0/2630851111 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c00a910 con 0x7f25641430b0 2021-04-03T03:16:24.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25640ab2d0 con 0x7f25641430b0 2021-04-03T03:16:24.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25640b4890 con 0x7f25641430b0 2021-04-03T03:16:24.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f25577fe700 1 -- 172.21.2.111:0/2630851111 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f255c00ac70 con 0x7f25641430b0 2021-04-03T03:16:24.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25640aab20 con 0x7f25641430b0 2021-04-03T03:16:24.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f25577fe700 1 -- 172.21.2.111:0/2630851111 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f255c019030 con 0x7f25641430b0 2021-04-03T03:16:24.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f25577fe700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2544057050 0x7f2544059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:24.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f25577fe700 1 -- 172.21.2.111:0/2630851111 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f255c06fc10 con 0x7f25641430b0 2021-04-03T03:16:24.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a11e700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2544057050 0x7f2544059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:24.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.501+0000 7f256a11e700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2544057050 0x7f2544059510 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f2564062370 tx=0x7f2560015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:24.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.505+0000 7f25577fe700 1 -- 172.21.2.111:0/2630851111 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f255c042c50 con 0x7f25641430b0 2021-04-03T03:16:24.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2564057660 con 0x7f25641430b0 2021-04-03T03:16:24.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f25577fe700 1 -- 172.21.2.111:0/2630851111 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f25640aab20 con 0x7f25641430b0 2021-04-03T03:16:24.627 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:24.627 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":48,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1453695075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1453695075},{"type":"v1","addr":"172.21.2.115:6833","nonce":1453695075}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:21.125389+0000","modified":"2021-04-03T03:16:22.128337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7503":{"gid":7503,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1395173601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1395173601},{"type":"v1","addr":"172.21.2.111:6835","nonce":1395173601}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-04-03T03:16:24.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2544057050 msgr2=0x7f2544059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2544057050 0x7f2544059510 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f2564062370 tx=0x7f2560015000).stop 2021-04-03T03:16:24.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 msgr2=0x7f25640aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:24.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f25640aade0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f255c0012b0 tx=0x7f255c007f90).stop 2021-04-03T03:16:24.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 shutdown_connections 2021-04-03T03:16:24.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f25641430b0 0x7f25640aade0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f25640b03a0 0x7f25640a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2544057050 0x7f2544059510 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:24.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 --2- 172.21.2.111:0/2630851111 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f25640b0f40 0x7f256413fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:24.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 >> 172.21.2.111:0/2630851111 conn(0x7f25640b61c0 msgr2=0x7f25641417f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:24.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 shutdown_connections 2021-04-03T03:16:24.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:24.625+0000 7f256b120700 1 -- 172.21.2.111:0/2630851111 wait complete. 2021-04-03T03:16:24.631 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 48 2021-04-03T03:16:24.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:21.125389+0000', 'modified': '2021-04-03T03:16:22.128337+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7503}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7503': {'gid': 7503, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1395173601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1395173601}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1395173601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:24.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:24.838 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:24.910 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7503,"want_state":"up:active","state":"up:active","rank_uptime":3.7791366769999999,"mdsmap_epoch":48,"osdmap_epoch":113,"osdmap_epoch_barrier":112,"uptime":7.930856071} 2021-04-03T03:16:24.911 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7503, "mdsmap_epoch": 48, "osdmap_epoch": 113, "osdmap_epoch_barrier": 112, "rank_uptime": 3.779136677, "state": "up:active", "uptime": 7.930856071, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:24.911 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:16:24.912 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:16:24.912 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:16:24.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:24.913 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:24.923 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:16:24.974 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:24.975 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:25.023 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:25.023 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:25.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:25.025 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:16:25.076 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:16:25.076 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:25.122 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:16:25.129 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:25.129 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:25.162 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15872]: starting ceph client 2021-04-03T03:16:25.162 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:16:25.156+0000 7f33ee9c00c0 -1 init, newargv = 0x55eab920f350 newargc=9 2021-04-03T03:16:25.167 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:25.167 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:25.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:25.169 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:16:25.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15872]: starting fuse 2021-04-03T03:16:25.182 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:16:25.183 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:16:25.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:25.184 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:16:25.184 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:16:25.185 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:16:25.185 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:16:25.185 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:16:25.186 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:25.186 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:16:25.187 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:16:25.187 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:16:25.187 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:25.188 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:16:25.188 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:16:25.189 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:16:25.189 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:16:25.189 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:25.190 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:16:25.190 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:16:25.190 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:16:25.191 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:16:25.191 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:16:25.192 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:25.192 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:16:25.193 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:16:25.251 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15872.asok 2021-04-03T03:16:25.254 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15872.asok 2021-04-03T03:16:25.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:25.255 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15872.asok status 2021-04-03T03:16:25.387 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:16:25.387 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:16:25.388 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:16:25.388 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:16:25.388 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:16:25.388 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:16:25.388 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:16:25.389 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15872", 2021-04-03T03:16:25.389 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:16:25.389 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:25.389 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:16:25.390 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:16:25.390 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 5452, 2021-04-03T03:16:25.390 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:16:25.390 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:16:25.390 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:16:25.391 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 5452 2021-04-03T03:16:25.391 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:25.391 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:16:25.391 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:16:25.391 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:16:25.392 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3335740436 2021-04-03T03:16:25.392 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:16:25.392 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:25.392 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:16:25.392 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:16:25.392 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:16:25.393 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3335740436 2021-04-03T03:16:25.393 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:25.393 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.5452 172.21.2.126:0/3335740436", 2021-04-03T03:16:25.393 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/3335740436", 2021-04-03T03:16:25.393 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:16:25.394 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 48, 2021-04-03T03:16:25.394 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 113, 2021-04-03T03:16:25.394 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:16:25.394 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:16:25.394 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:16:25.395 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:16:25.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:25.396 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:25.396 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:25.404 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:16:25.405 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:25.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:25.406 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:25.472 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:25.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.529+0000 7fa44cebb700 1 Processor -- start 2021-04-03T03:16:25.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.529+0000 7fa44cebb700 1 -- start start 2021-04-03T03:16:25.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440047bb0 0x7fa4400d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa4400d7f70 0x7fa4400d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4400d73b0 con 0x7fa440046cf0 2021-04-03T03:16:25.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44003e950 con 0x7fa4400d7f70 2021-04-03T03:16:25.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa440047f90 con 0x7fa440047bb0 2021-04-03T03:16:25.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa4477fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440047bb0 0x7fa4400d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa4477fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440047bb0 0x7fa4400d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:40138/0 (socket says 172.21.2.111:40138) 2021-04-03T03:16:25.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa447fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa4400d7f70 0x7fa4400d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa4477fe700 1 -- 172.21.2.111:0/4015381058 learned_addr learned my addr 172.21.2.111:0/4015381058 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa447fff700 1 -- 172.21.2.111:0/4015381058 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440047bb0 msgr2=0x7fa4400d7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa447fff700 1 --2- 172.21.2.111:0/4015381058 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440047bb0 0x7fa4400d7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa447fff700 1 -- 172.21.2.111:0/4015381058 >> v1:172.21.2.115:6789/0 conn(0x7fa440046cf0 legacy=0x7fa440005770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa447fff700 1 -- 172.21.2.111:0/4015381058 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44003dec0 con 0x7fa4400d7f70 2021-04-03T03:16:25.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa447fff700 1 --2- 172.21.2.111:0/4015381058 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400d3d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fa44806b5d0 tx=0x7fa44805ad70).ready entity=mon.1 client_cookie=44fe024d6c9ba045 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa4467fc700 1 -- 172.21.2.111:0/4015381058 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa44805bda0 con 0x7fa4400d7f70 2021-04-03T03:16:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa4467fc700 1 -- 172.21.2.111:0/4015381058 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44805df60 con 0x7fa4400d7f70 2021-04-03T03:16:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa4467fc700 1 -- 172.21.2.111:0/4015381058 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4480699f0 con 0x7fa4400d7f70 2021-04-03T03:16:25.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 -- 172.21.2.111:0/4015381058 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 msgr2=0x7fa4400d3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 --2- 172.21.2.111:0/4015381058 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400d3d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fa44806b5d0 tx=0x7fa44805ad70).stop 2021-04-03T03:16:25.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 -- 172.21.2.111:0/4015381058 shutdown_connections 2021-04-03T03:16:25.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 --2- 172.21.2.111:0/4015381058 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400d3d70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 --2- 172.21.2.111:0/4015381058 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440047bb0 0x7fa4400d7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.533+0000 7fa44cebb700 1 -- 172.21.2.111:0/4015381058 >> 172.21.2.111:0/4015381058 conn(0x7fa440010de0 msgr2=0x7fa4400111f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:25.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- 172.21.2.111:0/4015381058 shutdown_connections 2021-04-03T03:16:25.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- 172.21.2.111:0/4015381058 wait complete. 2021-04-03T03:16:25.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 Processor -- start 2021-04-03T03:16:25.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- start start 2021-04-03T03:16:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440046cf0 0x7fa440009610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa440047bb0 0x7fa4400d5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa440009b00 con 0x7fa440047bb0 2021-04-03T03:16:25.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa440009d20 con 0x7fa4400d7f70 2021-04-03T03:16:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa440009fc0 con 0x7fa440046cf0 2021-04-03T03:16:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa440047bb0 0x7fa4400d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa440047bb0 0x7fa4400d5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56796/0 (socket says 172.21.2.111:56796) 2021-04-03T03:16:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 -- 172.21.2.111:0/3863553800 learned_addr learned my addr 172.21.2.111:0/3863553800 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:25.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa4477fe700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440046cf0 0x7fa440009610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa447fff700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 -- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440046cf0 msgr2=0x7fa440009610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440046cf0 0x7fa440009610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 -- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 msgr2=0x7fa4400ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 -- 172.21.2.111:0/3863553800 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44000a260 con 0x7fa440047bb0 2021-04-03T03:16:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa447fff700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa446ffd700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa440047bb0 0x7fa4400d5580 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7fa4400d7960 tx=0x7fa43c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa444ff9700 1 -- 172.21.2.111:0/3863553800 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43c00e4a0 con 0x7fa440047bb0 2021-04-03T03:16:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa444ff9700 1 -- 172.21.2.111:0/3863553800 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c00e600 con 0x7fa440047bb0 2021-04-03T03:16:25.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa444ff9700 1 -- 172.21.2.111:0/3863553800 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43c00e960 con 0x7fa440047bb0 2021-04-03T03:16:25.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4400ea670 con 0x7fa440047bb0 2021-04-03T03:16:25.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4400d5c80 con 0x7fa440047bb0 2021-04-03T03:16:25.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44000a3f0 con 0x7fa440047bb0 2021-04-03T03:16:25.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa444ff9700 1 -- 172.21.2.111:0/3863553800 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa43c019030 con 0x7fa440047bb0 2021-04-03T03:16:25.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.537+0000 7fa444ff9700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa4300570c0 0x7fa430059580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.541+0000 7fa444ff9700 1 -- 172.21.2.111:0/3863553800 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa43c06f880 con 0x7fa440047bb0 2021-04-03T03:16:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.541+0000 7fa4477fe700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa4300570c0 0x7fa430059580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.541+0000 7fa4477fe700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa4300570c0 0x7fa430059580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fa43400b000 tx=0x7fa434008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.541+0000 7fa444ff9700 1 -- 172.21.2.111:0/3863553800 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa43c044b60 con 0x7fa440047bb0 2021-04-03T03:16:25.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa440004da0 con 0x7fa440047bb0 2021-04-03T03:16:25.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa444ff9700 1 -- 172.21.2.111:0/3863553800 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fa43c048180 con 0x7fa440047bb0 2021-04-03T03:16:25.635 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:25.635 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":113,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:22.917071+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:19.175402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:20.080209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:25.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa4300570c0 msgr2=0x7fa430059580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa4300570c0 0x7fa430059580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fa43400b000 tx=0x7fa434008040).stop 2021-04-03T03:16:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa440047bb0 msgr2=0x7fa4400d5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa440047bb0 0x7fa4400d5580 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7fa4400d7960 tx=0x7fa43c007c00).stop 2021-04-03T03:16:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 shutdown_connections 2021-04-03T03:16:25.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa4300570c0 0x7fa430059580 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4400d7f70 0x7fa4400ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa440046cf0 0x7fa440009610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 --2- 172.21.2.111:0/3863553800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa440047bb0 0x7fa4400d5580 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 >> 172.21.2.111:0/3863553800 conn(0x7fa440010de0 msgr2=0x7fa4400d4830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:25.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 shutdown_connections 2021-04-03T03:16:25.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.633+0000 7fa44cebb700 1 -- 172.21.2.111:0/3863553800 wait complete. 2021-04-03T03:16:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:16:19.175402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:16:20.080209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:16:25.644 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:25.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 Processor -- start 2021-04-03T03:16:25.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 -- start start 2021-04-03T03:16:25.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f47140add50 0x7f4714147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140607d0 con 0x7f47140603f0 2021-04-03T03:16:25.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471414b090 con 0x7f4714144ef0 2021-04-03T03:16:25.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471414b330 con 0x7f47140add50 2021-04-03T03:16:25.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4718bf8700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f47140603f0 0x7f47141447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57596/0 (socket says 172.21.2.111:57596) 2021-04-03T03:16:25.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4718bf8700 1 -- 172.21.2.111:0/1406078258 learned_addr learned my addr 172.21.2.111:0/1406078258 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:25.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164438406 0 0) 0x7f47140607d0 con 0x7f47140603f0 2021-04-03T03:16:25.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f4003240 con 0x7f47140603f0 2021-04-03T03:16:25.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 --2- 172.21.2.111:0/1406078258 >> v2:172.21.2.115:3300/0 conn(0x7f47140add50 0x7f4714147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2437904060 0 0) 0x7f46f4003240 con 0x7f47140603f0 2021-04-03T03:16:25.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 >> v2:172.21.2.115:3300/0 conn(0x7f47140add50 msgr2=0x7f4714147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 --2- 172.21.2.111:0/1406078258 >> v2:172.21.2.115:3300/0 conn(0x7f47140add50 0x7f4714147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 >> v1:172.21.2.115:6789/0 conn(0x7f4714144ef0 legacy=0x7f4714147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471414b5d0 con 0x7f47140603f0 2021-04-03T03:16:25.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f46fc0016d0 con 0x7f47140603f0 2021-04-03T03:16:25.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 >> v1:172.21.2.111:6789/0 conn(0x7f47140603f0 legacy=0x7f47141447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f4003800 0x7f46f4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46f40044e0 0x7f46f4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46f4004e50 0x7f46f400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471414b330 con 0x7f46f40044e0 2021-04-03T03:16:25.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4005300 con 0x7f46f4004e50 2021-04-03T03:16:25.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f400bc20 con 0x7f46f4003800 2021-04-03T03:16:25.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46f40044e0 0x7f46f4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47193f9700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46f4004e50 0x7f46f400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4718bf8700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f4003800 0x7f46f4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 -- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f4003800 msgr2=0x7f46f4003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f4003800 0x7f46f4003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 -- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46f4004e50 msgr2=0x7f46f400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46f4004e50 0x7f46f400b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 -- 172.21.2.111:0/1406078258 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f4003240 con 0x7f46f40044e0 2021-04-03T03:16:25.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47193f9700 1 -- 172.21.2.111:0/1406078258 reap_dead start 2021-04-03T03:16:25.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4718bf8700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f4003800 0x7f46f4003c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:25.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4713fff700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46f40044e0 0x7f46f4004960 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f470400a000 tx=0x7f4704008040).ready entity=mon.0 client_cookie=a379e0671f1247a6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f470400da60 con 0x7f46f40044e0 2021-04-03T03:16:25.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470400dbc0 con 0x7f46f40044e0 2021-04-03T03:16:25.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f47137fe700 1 -- 172.21.2.111:0/1406078258 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4704013480 con 0x7f46f40044e0 2021-04-03T03:16:25.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 -- 172.21.2.111:0/1406078258 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46f40044e0 msgr2=0x7f46f4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.701+0000 7f4719bfa700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46f40044e0 0x7f46f4004960 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f470400a000 tx=0x7f4704008040).stop 2021-04-03T03:16:25.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- 172.21.2.111:0/1406078258 shutdown_connections 2021-04-03T03:16:25.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 --2- 172.21.2.111:0/1406078258 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46f40044e0 0x7f46f4004960 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- 172.21.2.111:0/1406078258 >> 172.21.2.111:0/1406078258 conn(0x7f47140b61c0 msgr2=0x7f471405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:25.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- 172.21.2.111:0/1406078258 shutdown_connections 2021-04-03T03:16:25.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- 172.21.2.111:0/1406078258 wait complete. 2021-04-03T03:16:25.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 Processor -- start 2021-04-03T03:16:25.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- start start 2021-04-03T03:16:25.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f40044e0 0x7f47140632e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4714065b70 0x7f47140637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4714066fc0 0x7f4714067450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47140644e0 con 0x7f4714065b70 2021-04-03T03:16:25.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714148160 con 0x7f4714066fc0 2021-04-03T03:16:25.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47141478e0 con 0x7f46f40044e0 2021-04-03T03:16:25.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4714065b70 0x7f47140637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4714065b70 0x7f47140637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56816/0 (socket says 172.21.2.111:56816) 2021-04-03T03:16:25.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 -- 172.21.2.111:0/3460600829 learned_addr learned my addr 172.21.2.111:0/3460600829 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:25.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 -- 172.21.2.111:0/3460600829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f40044e0 msgr2=0x7f47140632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f40044e0 0x7f47140632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:25.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 -- 172.21.2.111:0/3460600829 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4714066fc0 msgr2=0x7f4714067450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4714066fc0 0x7f4714067450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:25.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 -- 172.21.2.111:0/3460600829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714148390 con 0x7f4714065b70 2021-04-03T03:16:25.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4713fff700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4714065b70 0x7f47140637d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f470400a000 tx=0x7f4704008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4711ffb700 1 -- 172.21.2.111:0/3460600829 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4704021070 con 0x7f4714065b70 2021-04-03T03:16:25.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4711ffb700 1 -- 172.21.2.111:0/3460600829 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470401d630 con 0x7f4714065b70 2021-04-03T03:16:25.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47140b49a0 con 0x7f4714065b70 2021-04-03T03:16:25.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4714147360 con 0x7f4714065b70 2021-04-03T03:16:25.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4711ffb700 1 -- 172.21.2.111:0/3460600829 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f470401d990 con 0x7f4714065b70 2021-04-03T03:16:25.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47140659b0 con 0x7f4714065b70 2021-04-03T03:16:25.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4711ffb700 1 -- 172.21.2.111:0/3460600829 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f470401daf0 con 0x7f4714065b70 2021-04-03T03:16:25.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4711ffb700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f46f8056f70 0x7f46f8059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4711ffb700 1 -- 172.21.2.111:0/3460600829 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f4704071c10 con 0x7f4714065b70 2021-04-03T03:16:25.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.705+0000 7f4718bf8700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f46f8056f70 0x7f46f8059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.709+0000 7f4718bf8700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f46f8056f70 0x7f46f8059430 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f46fc008000 tx=0x7f46fc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.709+0000 7f4711ffb700 1 -- 172.21.2.111:0/3460600829 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4704046ef0 con 0x7f4714065b70 2021-04-03T03:16:25.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4714057660 con 0x7f4714065b70 2021-04-03T03:16:25.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4711ffb700 1 -- 172.21.2.111:0/3460600829 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7f470401dd80 con 0x7f4714065b70 2021-04-03T03:16:25.803 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:25.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f46f8056f70 msgr2=0x7f46f8059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f46f8056f70 0x7f46f8059430 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f46fc008000 tx=0x7f46fc009040).stop 2021-04-03T03:16:25.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4714065b70 msgr2=0x7f47140637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4714065b70 0x7f47140637d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f470400a000 tx=0x7f4704008040).stop 2021-04-03T03:16:25.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 shutdown_connections 2021-04-03T03:16:25.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f46f8056f70 0x7f46f8059430 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4714066fc0 0x7f4714067450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:25.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46f40044e0 0x7f47140632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:25.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 --2- 172.21.2.111:0/3460600829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4714065b70 0x7f47140637d0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 >> 172.21.2.111:0/3460600829 conn(0x7f47140b61c0 msgr2=0x7f47140b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:25.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 shutdown_connections 2021-04-03T03:16:25.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.801+0000 7f4719bfa700 1 -- 172.21.2.111:0/3460600829 wait complete. 2021-04-03T03:16:25.810 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:25.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a93fff700 1 Processor -- start 2021-04-03T03:16:25.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a93fff700 1 -- start start 2021-04-03T03:16:25.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a93fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a940603f0 0x7f1a941447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a93fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1a940add50 0x7f1a94144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a93fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a940607d0 con 0x7f1a940b17b0 2021-04-03T03:16:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a93fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a941451c0 con 0x7f1a940add50 2021-04-03T03:16:25.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a93fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9414b090 con 0x7f1a940603f0 2021-04-03T03:16:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a937fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1a940b17b0 0x7f1a94147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57618/0 (socket says 172.21.2.111:57618) 2021-04-03T03:16:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.865+0000 7f1a937fe700 1 -- 172.21.2.111:0/875363394 learned_addr learned my addr 172.21.2.111:0/875363394 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a92ffd700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a940603f0 0x7f1a941447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a927fc700 1 --2- 172.21.2.111:0/875363394 >> v2:172.21.2.115:3300/0 conn(0x7f1a940add50 0x7f1a94144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418106601 0 0) 0x7f1a940607d0 con 0x7f1a940b17b0 2021-04-03T03:16:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a74003240 con 0x7f1a940b17b0 2021-04-03T03:16:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1700427354 0 0) 0x7f1a74003240 con 0x7f1a940b17b0 2021-04-03T03:16:25.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a940603f0 msgr2=0x7f1a941447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a940603f0 0x7f1a941447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 >> v2:172.21.2.115:3300/0 conn(0x7f1a940add50 msgr2=0x7f1a94144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 --2- 172.21.2.111:0/875363394 >> v2:172.21.2.115:3300/0 conn(0x7f1a940add50 0x7f1a94144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9414b330 con 0x7f1a940b17b0 2021-04-03T03:16:25.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f1a8c001a50 con 0x7f1a940b17b0 2021-04-03T03:16:25.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 >> v1:172.21.2.111:6789/0 conn(0x7f1a940b17b0 legacy=0x7f1a94147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74003730 0x7f1a74003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a74004060 0x7f1a74004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a74004b40 0x7f1a74004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9414b090 con 0x7f1a74004b40 2021-04-03T03:16:25.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7400b860 con 0x7f1a74004060 2021-04-03T03:16:25.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7400ba20 con 0x7f1a74003730 2021-04-03T03:16:25.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a74004b40 0x7f1a74004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74003730 msgr2=0x7f1a74003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74003730 0x7f1a74003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:25.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a74004060 msgr2=0x7f1a74004540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:25.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a74004060 0x7f1a74004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:25.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/875363394 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9414b330 con 0x7f1a74004b40 2021-04-03T03:16:25.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a74004b40 0x7f1a74004fc0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f1a8c001760 tx=0x7f1a8c007be0).ready entity=mon.0 client_cookie=43276a7794e13886 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c005b80 con 0x7f1a74004b40 2021-04-03T03:16:25.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c005ce0 con 0x7f1a74004b40 2021-04-03T03:16:25.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a91ffb700 1 -- 172.21.2.111:0/875363394 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c007300 con 0x7f1a74004b40 2021-04-03T03:16:25.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- 172.21.2.111:0/875363394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a74004b40 msgr2=0x7f1a74004fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 --2- 172.21.2.111:0/875363394 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a74004b40 0x7f1a74004fc0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f1a8c001760 tx=0x7f1a8c007be0).stop 2021-04-03T03:16:25.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/875363394 reap_dead start 2021-04-03T03:16:25.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- 172.21.2.111:0/875363394 shutdown_connections 2021-04-03T03:16:25.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- 172.21.2.111:0/875363394 >> 172.21.2.111:0/875363394 conn(0x7f1a940b61c0 msgr2=0x7f1a9405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:25.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- 172.21.2.111:0/875363394 shutdown_connections 2021-04-03T03:16:25.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- 172.21.2.111:0/875363394 wait complete. 2021-04-03T03:16:25.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 Processor -- start 2021-04-03T03:16:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- start start 2021-04-03T03:16:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a94147090 0x7f1a94066d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a94063260 0x7f1a9415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a94062d20 0x7f1a9415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94067260 con 0x7f1a94062d20 2021-04-03T03:16:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94147470 con 0x7f1a94063260 2021-04-03T03:16:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a93fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9415abd0 con 0x7f1a94147090 2021-04-03T03:16:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a94062d20 0x7f1a9415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a94062d20 0x7f1a9415a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56838/0 (socket says 172.21.2.111:56838) 2021-04-03T03:16:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/2335572618 learned_addr learned my addr 172.21.2.111:0/2335572618 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a927fc700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a94063260 0x7f1a9415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a92ffd700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a94147090 0x7f1a94066d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a94147090 msgr2=0x7f1a94066d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a94147090 0x7f1a94066d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a94063260 msgr2=0x7f1a9415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a94063260 0x7f1a9415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 -- 172.21.2.111:0/2335572618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94066390 con 0x7f1a94062d20 2021-04-03T03:16:25.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a927fc700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a94063260 0x7f1a9415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:25.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a92ffd700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a94147090 0x7f1a94066d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:25.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a937fe700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a94062d20 0x7f1a9415a6e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f1a940af1d0 tx=0x7f1a8c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.869+0000 7f1a8bfff700 1 -- 172.21.2.111:0/2335572618 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c01f670 con 0x7f1a94062d20 2021-04-03T03:16:25.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a8bfff700 1 -- 172.21.2.111:0/2335572618 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c01f7d0 con 0x7f1a94062d20 2021-04-03T03:16:25.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a94066630 con 0x7f1a94062d20 2021-04-03T03:16:25.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a94067f00 con 0x7f1a94062d20 2021-04-03T03:16:25.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a8bfff700 1 -- 172.21.2.111:0/2335572618 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c01fb30 con 0x7f1a94062d20 2021-04-03T03:16:25.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c0033f0 con 0x7f1a94062d20 2021-04-03T03:16:25.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a8bfff700 1 -- 172.21.2.111:0/2335572618 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1a8c026030 con 0x7f1a94062d20 2021-04-03T03:16:25.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a8bfff700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a78057050 0x7f1a78059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:25.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a8bfff700 1 -- 172.21.2.111:0/2335572618 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1a8c071cb0 con 0x7f1a94062d20 2021-04-03T03:16:25.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a92ffd700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a78057050 0x7f1a78059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:25.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a92ffd700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a78057050 0x7f1a78059510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f1a7c00b000 tx=0x7f1a7c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:25.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.873+0000 7f1a8bfff700 1 -- 172.21.2.111:0/2335572618 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1a8c046f90 con 0x7f1a94062d20 2021-04-03T03:16:25.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a6c003c20 con 0x7f1a94062d20 2021-04-03T03:16:25.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a8bfff700 1 -- 172.21.2.111:0/2335572618 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a8c024020 con 0x7f1a94062d20 2021-04-03T03:16:25.970 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:25.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a78057050 msgr2=0x7f1a78059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a78057050 0x7f1a78059510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f1a7c00b000 tx=0x7f1a7c008040).stop 2021-04-03T03:16:25.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a94062d20 msgr2=0x7f1a9415a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:25.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a94062d20 0x7f1a9415a6e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f1a940af1d0 tx=0x7f1a8c016000).stop 2021-04-03T03:16:25.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 shutdown_connections 2021-04-03T03:16:25.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a94062d20 0x7f1a9415a6e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a94147090 0x7f1a94066d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a78057050 0x7f1a78059510 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 --2- 172.21.2.111:0/2335572618 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a94063260 0x7f1a9415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:25.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 >> 172.21.2.111:0/2335572618 conn(0x7f1a940b61c0 msgr2=0x7f1a9405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:25.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 shutdown_connections 2021-04-03T03:16:25.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:25.969+0000 7f1a93fff700 1 -- 172.21.2.111:0/2335572618 wait complete. 2021-04-03T03:16:25.978 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:16:26.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.033+0000 7f071ffff700 1 Processor -- start 2021-04-03T03:16:26.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.033+0000 7f071ffff700 1 -- start start 2021-04-03T03:16:26.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.033+0000 7f071ffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f07200add50 0x7f0720147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.033+0000 7f071ffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07200607d0 con 0x7f0720144ef0 2021-04-03T03:16:26.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.033+0000 7f071ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072014b090 con 0x7f07200603f0 2021-04-03T03:16:26.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.033+0000 7f071ffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072014b330 con 0x7f07200add50 2021-04-03T03:16:26.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071f7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0720144ef0 0x7f0720147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57638/0 (socket says 172.21.2.111:57638) 2021-04-03T03:16:26.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071f7fe700 1 -- 172.21.2.111:0/2479488053 learned_addr learned my addr 172.21.2.111:0/2479488053 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:26.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071e7fc700 1 --2- 172.21.2.111:0/2479488053 >> v2:172.21.2.115:3300/0 conn(0x7f07200add50 0x7f0720147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024535160 0 0) 0x7f07200607d0 con 0x7f0720144ef0 2021-04-03T03:16:26.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0700003240 con 0x7f0720144ef0 2021-04-03T03:16:26.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1780670100 0 0) 0x7f0700003240 con 0x7f0720144ef0 2021-04-03T03:16:26.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 >> v2:172.21.2.115:3300/0 conn(0x7f07200add50 msgr2=0x7f0720147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:26.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 --2- 172.21.2.111:0/2479488053 >> v2:172.21.2.115:3300/0 conn(0x7f07200add50 0x7f0720147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 >> v1:172.21.2.115:6789/0 conn(0x7f07200603f0 legacy=0x7f07201447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072014b5d0 con 0x7f0720144ef0 2021-04-03T03:16:26.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f0718001a50 con 0x7f0720144ef0 2021-04-03T03:16:26.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 >> v1:172.21.2.111:6789/0 conn(0x7f0720144ef0 legacy=0x7f0720147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f0700003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07000044e0 0x7f0700004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0700004e50 0x7f070000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072014b330 con 0x7f0700003730 2021-04-03T03:16:26.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700004260 con 0x7f0700004e50 2021-04-03T03:16:26.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070000bc20 con 0x7f07000044e0 2021-04-03T03:16:26.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f0700003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 -- 172.21.2.111:0/2479488053 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07000044e0 msgr2=0x7f0700004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:26.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07000044e0 0x7f0700004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 -- 172.21.2.111:0/2479488053 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0700004e50 msgr2=0x7f070000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:26.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0700004e50 0x7f070000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 -- 172.21.2.111:0/2479488053 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072014b5d0 con 0x7f0700003730 2021-04-03T03:16:26.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071f7fe700 1 -- 172.21.2.111:0/2479488053 reap_dead start 2021-04-03T03:16:26.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f0700003b50 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f07200b5600 tx=0x7f0708007b70).ready entity=mon.0 client_cookie=ef44e24569b950b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f070800c660 con 0x7f0700003730 2021-04-03T03:16:26.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070800c7c0 con 0x7f0700003730 2021-04-03T03:16:26.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071dffb700 1 -- 172.21.2.111:0/2479488053 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f070800cb20 con 0x7f0700003730 2021-04-03T03:16:26.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- 172.21.2.111:0/2479488053 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 msgr2=0x7f0700003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f0700003b50 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f07200b5600 tx=0x7f0708007b70).stop 2021-04-03T03:16:26.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- 172.21.2.111:0/2479488053 shutdown_connections 2021-04-03T03:16:26.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 --2- 172.21.2.111:0/2479488053 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f0700003b50 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- 172.21.2.111:0/2479488053 >> 172.21.2.111:0/2479488053 conn(0x7f07200b61c0 msgr2=0x7f072005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:26.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- 172.21.2.111:0/2479488053 shutdown_connections 2021-04-03T03:16:26.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- 172.21.2.111:0/2479488053 wait complete. 2021-04-03T03:16:26.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 Processor -- start 2021-04-03T03:16:26.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- start start 2021-04-03T03:16:26.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f07200b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07200a8570 0x7f07200a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07200abbd0 0x7f07200a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07200a8330 con 0x7f0700003730 2021-04-03T03:16:26.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720062af0 con 0x7f07200abbd0 2021-04-03T03:16:26.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0720062d90 con 0x7f07200a8570 2021-04-03T03:16:26.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f07200b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f07200b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56854/0 (socket says 172.21.2.111:56854) 2021-04-03T03:16:26.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 -- 172.21.2.111:0/1405721275 learned_addr learned my addr 172.21.2.111:0/1405721275 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:26.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 -- 172.21.2.111:0/1405721275 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07200a8570 msgr2=0x7f07200a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07200a8570 0x7f07200a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 -- 172.21.2.111:0/1405721275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07200abbd0 msgr2=0x7f07200a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07200abbd0 0x7f07200a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 -- 172.21.2.111:0/1405721275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720063030 con 0x7f0700003730 2021-04-03T03:16:26.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071effd700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f07200b1440 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f07200b5600 tx=0x7f0708017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f0717fff700 1 -- 172.21.2.111:0/1405721275 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0708001ea0 con 0x7f0700003730 2021-04-03T03:16:26.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f0717fff700 1 -- 172.21.2.111:0/1405721275 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0708012af0 con 0x7f0700003730 2021-04-03T03:16:26.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f0717fff700 1 -- 172.21.2.111:0/1405721275 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0708012e50 con 0x7f0700003730 2021-04-03T03:16:26.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07200634d0 con 0x7f0700003730 2021-04-03T03:16:26.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.037+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07200aa630 con 0x7f0700003730 2021-04-03T03:16:26.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.041+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07200631c0 con 0x7f0700003730 2021-04-03T03:16:26.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.041+0000 7f0717fff700 1 -- 172.21.2.111:0/1405721275 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0708010070 con 0x7f0700003730 2021-04-03T03:16:26.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.041+0000 7f0717fff700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0704056ea0 0x7f0704059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.041+0000 7f0717fff700 1 -- 172.21.2.111:0/1405721275 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0708070c80 con 0x7f0700003730 2021-04-03T03:16:26.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.041+0000 7f071e7fc700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0704056ea0 0x7f0704059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.041+0000 7f071e7fc700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0704056ea0 0x7f0704059360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f071000a000 tx=0x7f0710008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.041+0000 7f0717fff700 1 -- 172.21.2.111:0/1405721275 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0708077080 con 0x7f0700003730 2021-04-03T03:16:26.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0720057660 con 0x7f0700003730 2021-04-03T03:16:26.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f0717fff700 1 -- 172.21.2.111:0/1405721275 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0708045f60 con 0x7f0700003730 2021-04-03T03:16:26.138 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:26.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0704056ea0 msgr2=0x7f0704059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0704056ea0 0x7f0704059360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f071000a000 tx=0x7f0710008040).stop 2021-04-03T03:16:26.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 msgr2=0x7f07200b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f07200b1440 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f07200b5600 tx=0x7f0708017000).stop 2021-04-03T03:16:26.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 shutdown_connections 2021-04-03T03:16:26.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0704056ea0 0x7f0704059360 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f07200abbd0 0x7f07200a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0700003730 0x7f07200b1440 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 --2- 172.21.2.111:0/1405721275 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f07200a8570 0x7f07200a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 >> 172.21.2.111:0/1405721275 conn(0x7f07200b61c0 msgr2=0x7f07201444a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:26.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 shutdown_connections 2021-04-03T03:16:26.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.137+0000 7f071ffff700 1 -- 172.21.2.111:0/1405721275 wait complete. 2021-04-03T03:16:26.146 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:16:26.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 Processor -- start 2021-04-03T03:16:26.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 -- start start 2021-04-03T03:16:26.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2d540603f0 0x7f2d541447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 0x7f2d54144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d540607d0 con 0x7f2d540b17b0 2021-04-03T03:16:26.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d541451c0 con 0x7f2d540603f0 2021-04-03T03:16:26.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5414b090 con 0x7f2d540add50 2021-04-03T03:16:26.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2d540603f0 0x7f2d541447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2d540603f0 0x7f2d541447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56862/0 (socket says 172.21.2.111:56862) 2021-04-03T03:16:26.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 -- 172.21.2.111:0/208920078 learned_addr learned my addr 172.21.2.111:0/208920078 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:26.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 -- 172.21.2.111:0/208920078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 msgr2=0x7f2d54144cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:26.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 --2- 172.21.2.111:0/208920078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 0x7f2d54144cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 -- 172.21.2.111:0/208920078 >> v1:172.21.2.115:6789/0 conn(0x7f2d540b17b0 legacy=0x7f2d54147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:26.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 -- 172.21.2.111:0/208920078 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5414b330 con 0x7f2d540603f0 2021-04-03T03:16:26.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d58b9b700 1 --2- 172.21.2.111:0/208920078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540603f0 0x7f2d541447e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f2d540b5600 tx=0x7f2d3c006e90).ready entity=mon.0 client_cookie=aa8958f9143ddf18 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d537fe700 1 -- 172.21.2.111:0/208920078 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d3c0098e0 con 0x7f2d540603f0 2021-04-03T03:16:26.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d537fe700 1 -- 172.21.2.111:0/208920078 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c009a40 con 0x7f2d540603f0 2021-04-03T03:16:26.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d537fe700 1 -- 172.21.2.111:0/208920078 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d3c009de0 con 0x7f2d540603f0 2021-04-03T03:16:26.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 -- 172.21.2.111:0/208920078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540603f0 msgr2=0x7f2d541447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.201+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/208920078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540603f0 0x7f2d541447e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f2d540b5600 tx=0x7f2d3c006e90).stop 2021-04-03T03:16:26.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- 172.21.2.111:0/208920078 shutdown_connections 2021-04-03T03:16:26.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/208920078 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540603f0 0x7f2d541447e0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/208920078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 0x7f2d54144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- 172.21.2.111:0/208920078 >> 172.21.2.111:0/208920078 conn(0x7f2d540b61c0 msgr2=0x7f2d5405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:26.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- 172.21.2.111:0/208920078 shutdown_connections 2021-04-03T03:16:26.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- 172.21.2.111:0/208920078 wait complete. 2021-04-03T03:16:26.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 Processor -- start 2021-04-03T03:16:26.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- start start 2021-04-03T03:16:26.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d540603f0 0x7f2d54147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 0x7f2d540aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540b17b0 0x7f2d540aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54157790 con 0x7f2d540b17b0 2021-04-03T03:16:26.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54157a30 con 0x7f2d540603f0 2021-04-03T03:16:26.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540a8ca0 con 0x7f2d540add50 2021-04-03T03:16:26.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540b17b0 0x7f2d540aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540b17b0 0x7f2d540aa560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56872/0 (socket says 172.21.2.111:56872) 2021-04-03T03:16:26.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 -- 172.21.2.111:0/2254328646 learned_addr learned my addr 172.21.2.111:0/2254328646 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:26.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 -- 172.21.2.111:0/2254328646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 msgr2=0x7f2d540aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 0x7f2d540aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 -- 172.21.2.111:0/2254328646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d540603f0 msgr2=0x7f2d54147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d540603f0 0x7f2d54147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 -- 172.21.2.111:0/2254328646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54146f80 con 0x7f2d540b17b0 2021-04-03T03:16:26.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d5939c700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540b17b0 0x7f2d540aa560 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f2d4c001210 tx=0x7f2d4c006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d51ffb700 1 -- 172.21.2.111:0/2254328646 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4c00d8f0 con 0x7f2d540b17b0 2021-04-03T03:16:26.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d51ffb700 1 -- 172.21.2.111:0/2254328646 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c00da50 con 0x7f2d540b17b0 2021-04-03T03:16:26.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d51ffb700 1 -- 172.21.2.111:0/2254328646 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4c00ddb0 con 0x7f2d540b17b0 2021-04-03T03:16:26.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d540aadc0 con 0x7f2d540b17b0 2021-04-03T03:16:26.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d54156bc0 con 0x7f2d540b17b0 2021-04-03T03:16:26.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d54057660 con 0x7f2d540b17b0 2021-04-03T03:16:26.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d51ffb700 1 -- 172.21.2.111:0/2254328646 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2d4c013030 con 0x7f2d540b17b0 2021-04-03T03:16:26.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d51ffb700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2d38056fb0 0x7f2d38059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.205+0000 7f2d51ffb700 1 -- 172.21.2.111:0/2254328646 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f2d4c070330 con 0x7f2d540b17b0 2021-04-03T03:16:26.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.209+0000 7f2d58b9b700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2d38056fb0 0x7f2d38059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.209+0000 7f2d58b9b700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2d38056fb0 0x7f2d38059470 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f2d540b5600 tx=0x7f2d3c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.209+0000 7f2d51ffb700 1 -- 172.21.2.111:0/2254328646 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2d4c045720 con 0x7f2d540b17b0 2021-04-03T03:16:26.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d54148850 con 0x7f2d540b17b0 2021-04-03T03:16:26.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d51ffb700 1 -- 172.21.2.111:0/2254328646 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2d4c048d40 con 0x7f2d540b17b0 2021-04-03T03:16:26.303 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:26.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2d38056fb0 msgr2=0x7f2d38059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2d38056fb0 0x7f2d38059470 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f2d540b5600 tx=0x7f2d3c015000).stop 2021-04-03T03:16:26.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540b17b0 msgr2=0x7f2d540aa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540b17b0 0x7f2d540aa560 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f2d4c001210 tx=0x7f2d4c006f90).stop 2021-04-03T03:16:26.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 shutdown_connections 2021-04-03T03:16:26.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2d540b17b0 0x7f2d540aa560 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2d38056fb0 0x7f2d38059470 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2d540603f0 0x7f2d54147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 --2- 172.21.2.111:0/2254328646 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2d540add50 0x7f2d540aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:26.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 >> 172.21.2.111:0/2254328646 conn(0x7f2d540b61c0 msgr2=0x7f2d5405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:26.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 shutdown_connections 2021-04-03T03:16:26.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.301+0000 7f2d59b9d700 1 -- 172.21.2.111:0/2254328646 wait complete. 2021-04-03T03:16:26.311 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:16:26.311 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:26.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 Processor -- start 2021-04-03T03:16:26.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 -- start start 2021-04-03T03:16:26.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4ac40dbb90 0x7f4ac4042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac4006bc0 0x7f4ac40dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac40dc6e0 con 0x7f4ac403d360 2021-04-03T03:16:26.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac40d9560 con 0x7f4ac40dbb90 2021-04-03T03:16:26.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac403dcb0 con 0x7f4ac4006bc0 2021-04-03T03:16:26.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4ac40dbb90 0x7f4ac4042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4ac40dbb90 0x7f4ac4042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56876/0 (socket says 172.21.2.111:56876) 2021-04-03T03:16:26.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 -- 172.21.2.111:0/3577302871 learned_addr learned my addr 172.21.2.111:0/3577302871 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:26.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1416700 1 --2- 172.21.2.111:0/3577302871 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac4006bc0 0x7f4ac40dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 -- 172.21.2.111:0/3577302871 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac4006bc0 msgr2=0x7f4ac40dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:26.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 --2- 172.21.2.111:0/3577302871 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac4006bc0 0x7f4ac40dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 -- 172.21.2.111:0/3577302871 >> v1:172.21.2.111:6789/0 conn(0x7f4ac403d360 legacy=0x7f4ac40064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 -- 172.21.2.111:0/3577302871 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40dc460 con 0x7f4ac40dbb90 2021-04-03T03:16:26.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad0c15700 1 --2- 172.21.2.111:0/3577302871 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac4042750 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f4ac403e5f0 tx=0x7f4ac0006e90).ready entity=mon.0 client_cookie=6b93b7d22fc54f4b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4acb7fe700 1 -- 172.21.2.111:0/3577302871 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ac0009930 con 0x7f4ac40dbb90 2021-04-03T03:16:26.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4acb7fe700 1 -- 172.21.2.111:0/3577302871 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0009a90 con 0x7f4ac40dbb90 2021-04-03T03:16:26.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4acb7fe700 1 -- 172.21.2.111:0/3577302871 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ac0009df0 con 0x7f4ac40dbb90 2021-04-03T03:16:26.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 -- 172.21.2.111:0/3577302871 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 msgr2=0x7f4ac4042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.365+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/3577302871 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac4042750 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f4ac403e5f0 tx=0x7f4ac0006e90).stop 2021-04-03T03:16:26.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- 172.21.2.111:0/3577302871 shutdown_connections 2021-04-03T03:16:26.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/3577302871 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac4006bc0 0x7f4ac40dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/3577302871 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac4042750 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- 172.21.2.111:0/3577302871 >> 172.21.2.111:0/3577302871 conn(0x7f4ac40462c0 msgr2=0x7f4ac40466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:26.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- 172.21.2.111:0/3577302871 shutdown_connections 2021-04-03T03:16:26.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- 172.21.2.111:0/3577302871 wait complete. 2021-04-03T03:16:26.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 Processor -- start 2021-04-03T03:16:26.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- start start 2021-04-03T03:16:26.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac403d360 0x7f4ac40df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac40d5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ac40eb550 0x7f4ac40d5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40d5d40 con 0x7f4ac40dbb90 2021-04-03T03:16:26.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40d8400 con 0x7f4ac40eb550 2021-04-03T03:16:26.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40e73f0 con 0x7f4ac403d360 2021-04-03T03:16:26.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac40d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac40d5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56884/0 (socket says 172.21.2.111:56884) 2021-04-03T03:16:26.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 -- 172.21.2.111:0/2977120602 learned_addr learned my addr 172.21.2.111:0/2977120602 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:26.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad0c15700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac403d360 0x7f4ac40df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1416700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ac40eb550 0x7f4ac40d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 -- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac403d360 msgr2=0x7f4ac40df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac403d360 0x7f4ac40df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 -- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ac40eb550 msgr2=0x7f4ac40d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ac40eb550 0x7f4ac40d5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 -- 172.21.2.111:0/2977120602 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40397a0 con 0x7f4ac40dbb90 2021-04-03T03:16:26.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad0c15700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac403d360 0x7f4ac40df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:26.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4acbfff700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac40d5360 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f4ac4046210 tx=0x7f4abc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ac9ffb700 1 -- 172.21.2.111:0/2977120602 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4abc00e4a0 con 0x7f4ac40dbb90 2021-04-03T03:16:26.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ac9ffb700 1 -- 172.21.2.111:0/2977120602 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc00e600 con 0x7f4ac40dbb90 2021-04-03T03:16:26.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ac9ffb700 1 -- 172.21.2.111:0/2977120602 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4abc00e960 con 0x7f4ac40dbb90 2021-04-03T03:16:26.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac40d8630 con 0x7f4ac40dbb90 2021-04-03T03:16:26.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac40df7a0 con 0x7f4ac40dbb90 2021-04-03T03:16:26.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac403c300 con 0x7f4ac40dbb90 2021-04-03T03:16:26.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ac9ffb700 1 -- 172.21.2.111:0/2977120602 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4abc019030 con 0x7f4ac40dbb90 2021-04-03T03:16:26.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ac9ffb700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4aac057050 0x7f4aac059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:26.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ac9ffb700 1 -- 172.21.2.111:0/2977120602 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f4abc06f880 con 0x7f4ac40dbb90 2021-04-03T03:16:26.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.369+0000 7f4ad0c15700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4aac057050 0x7f4aac059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:26.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.373+0000 7f4ad0c15700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4aac057050 0x7f4aac059510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4ac0016000 tx=0x7f4ac0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:26.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.373+0000 7f4ac9ffb700 1 -- 172.21.2.111:0/2977120602 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4abc044c10 con 0x7f4ac40dbb90 2021-04-03T03:16:26.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.493+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ac40d8ad0 con 0x7f4ac40dbb90 2021-04-03T03:16:26.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.493+0000 7f4ac9ffb700 1 -- 172.21.2.111:0/2977120602 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f4abc017870 con 0x7f4ac40dbb90 2021-04-03T03:16:26.497 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:26.497 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":48,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/1453695075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":1453695075},{"type":"v1","addr":"172.21.2.115:6833","nonce":1453695075}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:21.125389+0000","modified":"2021-04-03T03:16:22.128337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7503":{"gid":7503,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1395173601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1395173601},{"type":"v1","addr":"172.21.2.111:6835","nonce":1395173601}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-04-03T03:16:26.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4aac057050 msgr2=0x7f4aac059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4aac057050 0x7f4aac059510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4ac0016000 tx=0x7f4ac0015000).stop 2021-04-03T03:16:26.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 msgr2=0x7f4ac40d5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:26.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac40d5360 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f4ac4046210 tx=0x7f4abc007c30).stop 2021-04-03T03:16:26.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 shutdown_connections 2021-04-03T03:16:26.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4aac057050 0x7f4aac059510 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ac40eb550 0x7f4ac40d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4ac403d360 0x7f4ac40df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 --2- 172.21.2.111:0/2977120602 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ac40dbb90 0x7f4ac40d5360 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:26.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 >> 172.21.2.111:0/2977120602 conn(0x7f4ac40462c0 msgr2=0x7f4ac40441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:26.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 shutdown_connections 2021-04-03T03:16:26.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:26.497+0000 7f4ad1c17700 1 -- 172.21.2.111:0/2977120602 wait complete. 2021-04-03T03:16:26.501 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 48 2021-04-03T03:16:26.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:21.125389+0000', 'modified': '2021-04-03T03:16:22.128337+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7503}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7503': {'gid': 7503, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1395173601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1395173601}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1395173601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:26.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:26.707 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:26.780 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7503,"want_state":"up:active","state":"up:active","rank_uptime":5.6480580390000004,"mdsmap_epoch":48,"osdmap_epoch":113,"osdmap_epoch_barrier":112,"uptime":9.7997773949999996} 2021-04-03T03:16:26.780 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7503, "mdsmap_epoch": 48, "osdmap_epoch": 113, "osdmap_epoch_barrier": 112, "rank_uptime": 5.648058039, "state": "up:active", "uptime": 9.799777395, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:26.781 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:16:26.781 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:16:26.781 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:16:26.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:26.782 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:26.792 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:16:26.842 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:26.843 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:26.892 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:26.892 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:26.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:26.894 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:16:26.945 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:16:26.945 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:26.990 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:16:26.998 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:26.999 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:27.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:16:27.024+0000 7f4b631a10c0 -1 init, newargv = 0x560bb3f42a20 newargc=9 2021-04-03T03:16:27.030 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15391]: starting ceph client 2021-04-03T03:16:27.034 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:27.035 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:27.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.037 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:16:27.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15391]: starting fuse 2021-04-03T03:16:27.043 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:16:27.043 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:16:27.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.044 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:16:27.045 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:16:27.045 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:16:27.046 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:16:27.046 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:16:27.046 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:27.047 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:16:27.047 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:16:27.048 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:16:27.048 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:27.048 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:16:27.049 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:16:27.049 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:16:27.050 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:16:27.050 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:27.051 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:16:27.051 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:16:27.051 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:16:27.052 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:16:27.052 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:16:27.053 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:27.053 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:16:27.053 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:16:27.107 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15391.asok 2021-04-03T03:16:27.110 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15391.asok 2021-04-03T03:16:27.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.111 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15391.asok status 2021-04-03T03:16:27.224 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:16:27.224 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:16:27.224 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:16:27.225 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:16:27.225 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:16:27.225 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:16:27.226 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:16:27.226 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15391", 2021-04-03T03:16:27.226 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:16:27.227 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:27.227 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:16:27.227 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:16:27.227 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 7713, 2021-04-03T03:16:27.228 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:16:27.228 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:16:27.228 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:16:27.228 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 7713 2021-04-03T03:16:27.229 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:27.229 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:16:27.229 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:16:27.229 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:16:27.230 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 2505511453 2021-04-03T03:16:27.230 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:16:27.230 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:27.230 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:16:27.231 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:16:27.231 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:16:27.231 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 2505511453 2021-04-03T03:16:27.231 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:27.231 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.7713 172.21.2.128:0/2505511453", 2021-04-03T03:16:27.232 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/2505511453", 2021-04-03T03:16:27.232 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:16:27.232 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 48, 2021-04-03T03:16:27.232 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 113, 2021-04-03T03:16:27.233 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:16:27.233 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:16:27.233 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:16:27.233 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:16:27.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.235 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:27.236 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:27.241 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:27.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:27.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.243 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:27.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.295 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:27.295 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:27.306 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:16:27.307 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:27.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.308 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:27.309 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:27.358 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:16:27.360 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:27.361 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:16:27.361 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:16:27.362 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:27.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15872]: fuse finished with error 0 and tester_r 0 2021-04-03T03:16:27.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.462 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:27.463 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:27.467 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:16:27.468 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:16:27.468 INFO:teuthology.orchestra.run:waiting for 900 2021-04-03T03:16:27.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:27.565 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:27.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.569 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:27.570 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:27.580 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:27.582 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:27.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.582 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:27.583 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:27.635 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:27.636 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:27.636 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:16:27.637 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:16:27.637 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:27.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15391]: fuse finished with error 0 and tester_r 0 2021-04-03T03:16:27.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:27.723 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:27.723 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:27.727 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:16:27.728 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:16:27.728 INFO:teuthology.orchestra.run:waiting for 900 2021-04-03T03:16:27.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:27.827 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:27.832 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:16:27.832 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:16:27.833 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:16:27.833 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:16:27.842 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:16:27.842 DEBUG:teuthology.parallel:result is None 2021-04-03T03:16:27.843 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:16:27.844 DEBUG:teuthology.parallel:result is None 2021-04-03T03:16:27.844 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail a 2021-04-03T03:16:27.845 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail b 2021-04-03T03:16:27.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 Processor -- start 2021-04-03T03:16:27.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f697cad8700 1 Processor -- start 2021-04-03T03:16:27.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f697cad8700 1 -- start start 2021-04-03T03:16:27.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- start start 2021-04-03T03:16:27.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff2c803db90 0x7ff2c803df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c8041070 0x7ff2c80d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f697cad8700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f697003f810 0x7f69700d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f697cad8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700d2790 0x7f697003e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f697cad8700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69700d2050 con 0x7f6970048d60 2021-04-03T03:16:27.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f697cad8700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700d22f0 con 0x7f697003f810 2021-04-03T03:16:27.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f697cad8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700d1d80 con 0x7f69700d2790 2021-04-03T03:16:27.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c803e460 con 0x7ff2c80d4750 2021-04-03T03:16:27.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c8042370 con 0x7ff2c803db90 2021-04-03T03:16:27.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8042610 con 0x7ff2c8041070 2021-04-03T03:16:27.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff2c803db90 0x7ff2c803df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff2c803db90 0x7ff2c803df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56890/0 (socket says 172.21.2.111:56890) 2021-04-03T03:16:27.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69777fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6970048d60 0x7f6970043a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57682/0 (socket says 172.21.2.111:57682) 2021-04-03T03:16:27.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69777fe700 1 -- 172.21.2.111:0/954539345 learned_addr learned my addr 172.21.2.111:0/954539345 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:27.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 -- 172.21.2.111:0/1439099596 learned_addr learned my addr 172.21.2.111:0/1439099596 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:27.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 -- 172.21.2.111:0/1439099596 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c8041070 msgr2=0x7ff2c80d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:27.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 --2- 172.21.2.111:0/1439099596 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c8041070 0x7ff2c80d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:27.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 -- 172.21.2.111:0/1439099596 >> v1:172.21.2.115:6789/0 conn(0x7ff2c80d4750 legacy=0x7ff2c80cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:27.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 -- 172.21.2.111:0/1439099596 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80416e0 con 0x7ff2c803db90 2021-04-03T03:16:27.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150686897 0 0) 0x7f69700d2050 con 0x7f6970048d60 2021-04-03T03:16:27.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f695c003240 con 0x7f6970048d60 2021-04-03T03:16:27.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 --2- 172.21.2.111:0/954539345 >> v2:172.21.2.115:3300/0 conn(0x7f697003f810 0x7f69700d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6977fff700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700d2790 0x7f697003e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2cf7fe700 1 --2- 172.21.2.111:0/1439099596 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c803db90 0x7ff2c803df70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7ff2c8041a60 tx=0x7ff2bc006e90).ready entity=mon.0 client_cookie=abebc2e556e48461 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:27.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4283692779 0 0) 0x7f695c003240 con 0x7f6970048d60 2021-04-03T03:16:27.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700d2790 msgr2=0x7f697003e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700d2790 0x7f697003e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 >> v2:172.21.2.115:3300/0 conn(0x7f697003f810 msgr2=0x7f69700d5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 --2- 172.21.2.111:0/954539345 >> v2:172.21.2.115:3300/0 conn(0x7f697003f810 0x7f69700d5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6970049140 con 0x7f6970048d60 2021-04-03T03:16:27.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f696c0016d0 con 0x7f6970048d60 2021-04-03T03:16:27.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2ce7fc700 1 -- 172.21.2.111:0/1439099596 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2bc0098e0 con 0x7ff2c803db90 2021-04-03T03:16:27.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2ce7fc700 1 -- 172.21.2.111:0/1439099596 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc009a40 con 0x7ff2c803db90 2021-04-03T03:16:27.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 >> v1:172.21.2.111:6789/0 conn(0x7f6970048d60 legacy=0x7f6970043a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:27.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2ce7fc700 1 -- 172.21.2.111:0/1439099596 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2bc009de0 con 0x7ff2c803db90 2021-04-03T03:16:27.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f695c0037b0 0x7f695c003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f695c0044e0 0x7f695c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/1439099596 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c803db90 msgr2=0x7ff2c803df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:27.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/1439099596 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c803db90 0x7ff2c803df70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7ff2c8041a60 tx=0x7ff2bc006e90).stop 2021-04-03T03:16:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f695c004e50 0x7f695c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700d22f0 con 0x7f695c0044e0 2021-04-03T03:16:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700d1d80 con 0x7f695c0037b0 2021-04-03T03:16:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c005350 con 0x7f695c004e50 2021-04-03T03:16:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6977fff700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700d2790 0x7f697003e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f695c0044e0 0x7f695c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 -- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f695c004e50 msgr2=0x7f695c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f695c004e50 0x7f695c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 -- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f695c0037b0 msgr2=0x7f695c003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f695c0037b0 0x7f695c003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 -- 172.21.2.111:0/954539345 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c003240 con 0x7f695c0044e0 2021-04-03T03:16:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7f6976ffd700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f695c0044e0 0x7f695c004960 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f696400c000 tx=0x7f6964008040).ready entity=mon.0 client_cookie=55f8764d7e5d39af server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:27.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/1439099596 shutdown_connections 2021-04-03T03:16:27.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/1439099596 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c803db90 0x7ff2c803df70 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:27.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/1439099596 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c8041070 0x7ff2c80d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:27.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/1439099596 >> 172.21.2.111:0/1439099596 conn(0x7ff2c80473c0 msgr2=0x7ff2c80477d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:27.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/1439099596 shutdown_connections 2021-04-03T03:16:27.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.945+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/1439099596 wait complete. 2021-04-03T03:16:27.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 Processor -- start 2021-04-03T03:16:27.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 -- start start 2021-04-03T03:16:27.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2c803db90 0x7ff2c8042b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c8041070 0x7ff2c80ead70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c80d4750 0x7ff2c80eb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c803cef0 con 0x7ff2c8041070 2021-04-03T03:16:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80a1f70 con 0x7ff2c803db90 2021-04-03T03:16:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80a21e0 con 0x7ff2c80d4750 2021-04-03T03:16:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c8041070 0x7ff2c80ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c8041070 0x7ff2c80ead70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56912/0 (socket says 172.21.2.111:56912) 2021-04-03T03:16:27.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 -- 172.21.2.111:0/752196798 learned_addr learned my addr 172.21.2.111:0/752196798 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f696400e970 con 0x7f695c0044e0 2021-04-03T03:16:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696400ead0 con 0x7f695c0044e0 2021-04-03T03:16:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f69767fc700 1 -- 172.21.2.111:0/954539345 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f696400ee30 con 0x7f695c0044e0 2021-04-03T03:16:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- 172.21.2.111:0/954539345 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f695c0044e0 msgr2=0x7f695c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 --2- 172.21.2.111:0/954539345 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f695c0044e0 0x7f695c004960 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f696400c000 tx=0x7f6964008040).stop 2021-04-03T03:16:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2cf7fe700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2c803db90 0x7ff2c8042b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6977fff700 1 -- 172.21.2.111:0/954539345 reap_dead start 2021-04-03T03:16:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 -- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c80d4750 msgr2=0x7ff2c80eb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c80d4750 0x7ff2c80eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 -- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2c803db90 msgr2=0x7ff2c8042b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2c803db90 0x7ff2c8042b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:27.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 -- 172.21.2.111:0/752196798 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8006ae0 con 0x7ff2c8041070 2021-04-03T03:16:27.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2cf7fe700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2c803db90 0x7ff2c8042b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:27.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ceffd700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c8041070 0x7ff2c80ead70 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7ff2c80a2100 tx=0x7ff2d005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:27.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ccff9700 1 -- 172.21.2.111:0/752196798 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d005f2c0 con 0x7ff2c8041070 2021-04-03T03:16:27.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ccff9700 1 -- 172.21.2.111:0/752196798 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d005f420 con 0x7ff2c8041070 2021-04-03T03:16:27.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c80a1050 con 0x7ff2c8041070 2021-04-03T03:16:27.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ccff9700 1 -- 172.21.2.111:0/752196798 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d005f780 con 0x7ff2c8041070 2021-04-03T03:16:27.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c80a0740 con 0x7ff2c8041070 2021-04-03T03:16:27.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- 172.21.2.111:0/954539345 shutdown_connections 2021-04-03T03:16:27.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- 172.21.2.111:0/954539345 >> 172.21.2.111:0/954539345 conn(0x7f697000f350 msgr2=0x7f69700484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:27.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- 172.21.2.111:0/954539345 shutdown_connections 2021-04-03T03:16:27.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- 172.21.2.111:0/954539345 wait complete. 2021-04-03T03:16:27.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c803d060 con 0x7ff2c8041070 2021-04-03T03:16:27.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 Processor -- start 2021-04-03T03:16:27.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- start start 2021-04-03T03:16:27.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ccff9700 1 -- 172.21.2.111:0/752196798 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff2d008b020 con 0x7ff2c8041070 2021-04-03T03:16:27.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ccff9700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2b8056ea0 0x7ff2b8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2ccff9700 1 -- 172.21.2.111:0/752196798 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff2d00df270 con 0x7ff2c8041070 2021-04-03T03:16:27.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2cf7fe700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2b8056ea0 0x7ff2b8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700c12b0 0x7f69700c1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69700d7520 0x7f69700d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69700d7f50 0x7f69700d8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700d88f0 con 0x7f69700d7520 2021-04-03T03:16:27.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700d7310 con 0x7f69700d7f50 2021-04-03T03:16:27.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f697cad8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700d7070 con 0x7f69700c12b0 2021-04-03T03:16:27.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69700d7520 0x7f69700d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69700d7520 0x7f69700d7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56918/0 (socket says 172.21.2.111:56918) 2021-04-03T03:16:27.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 -- 172.21.2.111:0/1374965234 learned_addr learned my addr 172.21.2.111:0/1374965234 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:27.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7ff2cf7fe700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2b8056ea0 0x7ff2b8059360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff2bc017000 tx=0x7ff2bc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:27.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 -- 172.21.2.111:0/1374965234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700c12b0 msgr2=0x7f69700c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:27.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700c12b0 0x7f69700c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:27.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 -- 172.21.2.111:0/1374965234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69700d7f50 msgr2=0x7f69700d8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:27.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69700d7f50 0x7f69700d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:27.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 -- 172.21.2.111:0/1374965234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700e3cb0 con 0x7f69700d7520 2021-04-03T03:16:27.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.949+0000 7f6976ffd700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69700d7520 0x7f69700d7a60 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f69700d8a30 tx=0x7f696401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:27.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f6974ff9700 1 -- 172.21.2.111:0/1374965234 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69640125e0 con 0x7f69700d7520 2021-04-03T03:16:27.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f6974ff9700 1 -- 172.21.2.111:0/1374965234 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964012740 con 0x7f69700d7520 2021-04-03T03:16:27.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69700c1b80 con 0x7f69700d7520 2021-04-03T03:16:27.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f6974ff9700 1 -- 172.21.2.111:0/1374965234 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6964012aa0 con 0x7f69700d7520 2021-04-03T03:16:27.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69700bfec0 con 0x7f69700d7520 2021-04-03T03:16:27.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7ff2ccff9700 1 -- 172.21.2.111:0/752196798 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff2c803d060 con 0x7ff2c8041070 2021-04-03T03:16:27.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6970004d30 con 0x7f69700d7520 2021-04-03T03:16:27.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f6974ff9700 1 -- 172.21.2.111:0/1374965234 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f696402b070 con 0x7f69700d7520 2021-04-03T03:16:27.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f6974ff9700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6960056ea0 0x7f6960059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:27.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f6974ff9700 1 -- 172.21.2.111:0/1374965234 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f6964071840 con 0x7f69700d7520 2021-04-03T03:16:27.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f69777fe700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6960056ea0 0x7f6960059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:27.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.953+0000 7f69777fe700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6960056ea0 0x7f6960059360 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f696c0013e0 tx=0x7f696c001280).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:27.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:27.957+0000 7f6974ff9700 1 -- 172.21.2.111:0/1374965234 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6964046c30 con 0x7f69700d7520 2021-04-03T03:16:28.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.085+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7ff2c80a1330 con 0x7ff2c8041070 2021-04-03T03:16:28.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.089+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f69700e3e40 con 0x7f69700d7520 2021-04-03T03:16:28.949 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:16:28.949+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:16:28.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7ff2ccff9700 1 -- 172.21.2.111:0/752196798 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 4976 v49) v1 ==== 95+0+0 (secure 0 0 0) 0x7ff2d00b45c0 con 0x7ff2c8041070 2021-04-03T03:16:28.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7f6974ff9700 1 -- 172.21.2.111:0/1374965234 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7503 v49) v1 ==== 95+0+0 (secure 0 0 0) 0x7f6964016020 con 0x7f69700d7520 2021-04-03T03:16:28.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6960056ea0 msgr2=0x7f6960059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:28.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7f697cad8700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6960056ea0 0x7f6960059360 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f696c0013e0 tx=0x7f696c001280).stop 2021-04-03T03:16:28.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69700d7520 msgr2=0x7f69700d7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:28.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7f697cad8700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69700d7520 0x7f69700d7a60 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f69700d8a30 tx=0x7f696401a000).stop 2021-04-03T03:16:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2b8056ea0 msgr2=0x7ff2b8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2b8056ea0 0x7ff2b8059360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff2bc017000 tx=0x7ff2bc015000).stop 2021-04-03T03:16:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c8041070 msgr2=0x7ff2c80ead70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:28.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.957+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c8041070 0x7ff2c80ead70 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7ff2c80a2100 tx=0x7ff2d005ad70).stop 2021-04-03T03:16:28.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 shutdown_connections 2021-04-03T03:16:28.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6960056ea0 0x7f6960059360 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:28.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69700d7f50 0x7f69700d8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69700c12b0 0x7f69700c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 --2- 172.21.2.111:0/1374965234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69700d7520 0x7f69700d7a60 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:28.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 >> 172.21.2.111:0/1374965234 conn(0x7f697000f350 msgr2=0x7f69700430c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 shutdown_connections 2021-04-03T03:16:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff2c80d4750 0x7ff2c80eb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff2b8056ea0 0x7ff2b8059360 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:28.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff2c803db90 0x7ff2c8042b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 --2- 172.21.2.111:0/752196798 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff2c8041070 0x7ff2c80ead70 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 >> 172.21.2.111:0/752196798 conn(0x7ff2c80473c0 msgr2=0x7ff2c80d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:28.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 shutdown_connections 2021-04-03T03:16:28.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 shutdown_connections 2021-04-03T03:16:28.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7f697cad8700 1 -- 172.21.2.111:0/1374965234 wait complete. 2021-04-03T03:16:28.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:28.961+0000 7ff2d4fc9700 1 -- 172.21.2.111:0/752196798 wait complete. 2021-04-03T03:16:28.968 INFO:teuthology.orchestra.run.gibba011.stderr:failed mds gid 7503 2021-04-03T03:16:28.969 INFO:teuthology.orchestra.run.gibba011.stderr:failed mds gid 4976 2021-04-03T03:16:28.970 DEBUG:teuthology.parallel:result is '' 2021-04-03T03:16:28.971 DEBUG:teuthology.parallel:result is '' 2021-04-03T03:16:28.971 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:16:29.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.085+0000 7f6055898700 1 Processor -- start 2021-04-03T03:16:29.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.085+0000 7f6055898700 1 -- start start 2021-04-03T03:16:29.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f604803f120 0x7f6048005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6048042470 0x7f60480d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604803d840 con 0x7f6048041fe0 2021-04-03T03:16:29.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604803faa0 con 0x7f604803f120 2021-04-03T03:16:29.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604803dca0 con 0x7f6048042470 2021-04-03T03:16:29.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6048041fe0 0x7f60480d89a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57718/0 (socket says 172.21.2.111:57718) 2021-04-03T03:16:29.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 -- 172.21.2.111:0/1656385468 learned_addr learned my addr 172.21.2.111:0/1656385468 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:29.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823775581 0 0) 0x7f604803d840 con 0x7f6048041fe0 2021-04-03T03:16:29.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f603c003240 con 0x7f6048041fe0 2021-04-03T03:16:29.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2748693897 0 0) 0x7f603c003240 con 0x7f6048041fe0 2021-04-03T03:16:29.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6048042470 msgr2=0x7f60480d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:29.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6048042470 0x7f60480d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 >> v2:172.21.2.115:3300/0 conn(0x7f604803f120 msgr2=0x7f6048005640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:29.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 --2- 172.21.2.111:0/1656385468 >> v2:172.21.2.115:3300/0 conn(0x7f604803f120 0x7f6048005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6048005b30 con 0x7f6048041fe0 2021-04-03T03:16:29.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f60400016d0 con 0x7f6048041fe0 2021-04-03T03:16:29.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 >> v1:172.21.2.111:6789/0 conn(0x7f6048041fe0 legacy=0x7f60480d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:29.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6048042470 0x7f603c0037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f603c003d10 0x7f603c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f603c0049d0 0x7f603c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604803faa0 con 0x7f603c003d10 2021-04-03T03:16:29.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604803dca0 con 0x7f603c0049d0 2021-04-03T03:16:29.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0042d0 con 0x7f6048042470 2021-04-03T03:16:29.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f603c003d10 0x7f603c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:29.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 -- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6048042470 msgr2=0x7f603c0037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:29.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6048042470 0x7f603c0037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 -- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f603c0049d0 msgr2=0x7f603c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:29.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f603c0049d0 0x7f603c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 -- 172.21.2.111:0/1656385468 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c003240 con 0x7f603c003d10 2021-04-03T03:16:29.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604ffff700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f603c003d10 0x7f603c0044e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f60400013e0 tx=0x7f604803ec70).ready entity=mon.0 client_cookie=7cadde3802faac52 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:29.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60400070d0 con 0x7f603c003d10 2021-04-03T03:16:29.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040007230 con 0x7f603c003d10 2021-04-03T03:16:29.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604f7fe700 1 -- 172.21.2.111:0/1656385468 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60400055f0 con 0x7f603c003d10 2021-04-03T03:16:29.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- 172.21.2.111:0/1656385468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f603c003d10 msgr2=0x7f603c0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:29.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 --2- 172.21.2.111:0/1656385468 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f603c003d10 0x7f603c0044e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f60400013e0 tx=0x7f604803ec70).stop 2021-04-03T03:16:29.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055097700 1 -- 172.21.2.111:0/1656385468 reap_dead start 2021-04-03T03:16:29.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- 172.21.2.111:0/1656385468 shutdown_connections 2021-04-03T03:16:29.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- 172.21.2.111:0/1656385468 >> 172.21.2.111:0/1656385468 conn(0x7f60480462b0 msgr2=0x7f60480466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:29.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- 172.21.2.111:0/1656385468 shutdown_connections 2021-04-03T03:16:29.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- 172.21.2.111:0/1656385468 wait complete. 2021-04-03T03:16:29.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 Processor -- start 2021-04-03T03:16:29.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- start start 2021-04-03T03:16:29.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60480d4660 0x7f60480e7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60480a20c0 0x7f604803c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60480a2540 0x7f604803c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60480a30a0 con 0x7f60480d4660 2021-04-03T03:16:29.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60480e8490 con 0x7f60480a2540 2021-04-03T03:16:29.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6055898700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60480d4a40 con 0x7f60480a20c0 2021-04-03T03:16:29.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60480d4660 0x7f60480e7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:29.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60480d4660 0x7f60480e7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56936/0 (socket says 172.21.2.111:56936) 2021-04-03T03:16:29.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 -- 172.21.2.111:0/3685297241 learned_addr learned my addr 172.21.2.111:0/3685297241 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:29.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 -- 172.21.2.111:0/3685297241 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60480a20c0 msgr2=0x7f604803c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:29.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60480a20c0 0x7f604803c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 -- 172.21.2.111:0/3685297241 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60480a2540 msgr2=0x7f604803c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:29.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60480a2540 0x7f604803c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 -- 172.21.2.111:0/3685297241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60480a1980 con 0x7f60480d4660 2021-04-03T03:16:29.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f6054896700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60480d4660 0x7f60480e7210 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f6044002f30 tx=0x7f6044006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:29.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.089+0000 7f604dffb700 1 -- 172.21.2.111:0/3685297241 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604400d7e0 con 0x7f60480d4660 2021-04-03T03:16:29.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604dffb700 1 -- 172.21.2.111:0/3685297241 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6044000e90 con 0x7f60480d4660 2021-04-03T03:16:29.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60480db100 con 0x7f60480d4660 2021-04-03T03:16:29.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60480a0a20 con 0x7f60480d4660 2021-04-03T03:16:29.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604dffb700 1 -- 172.21.2.111:0/3685297241 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604400db30 con 0x7f60480d4660 2021-04-03T03:16:29.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6048004d30 con 0x7f60480d4660 2021-04-03T03:16:29.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604dffb700 1 -- 172.21.2.111:0/3685297241 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f604400dc90 con 0x7f60480d4660 2021-04-03T03:16:29.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604dffb700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6030056ea0 0x7f6030059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:29.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604dffb700 1 -- 172.21.2.111:0/3685297241 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6044070490 con 0x7f60480d4660 2021-04-03T03:16:29.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604ffff700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6030056ea0 0x7f6030059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:29.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604ffff700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6030056ea0 0x7f6030059360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f60400013e0 tx=0x7f6040013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:29.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.093+0000 7f604dffb700 1 -- 172.21.2.111:0/3685297241 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6044045710 con 0x7f60480d4660 2021-04-03T03:16:29.217 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:29.219 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:29.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.217+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f60480a2a90 con 0x7f60480d4660 2021-04-03T03:16:29.222 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:29.225 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:29.228 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:29.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f604dffb700 1 -- 172.21.2.111:0/3685297241 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6044048d30 con 0x7f60480d4660 2021-04-03T03:16:29.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6030056ea0 msgr2=0x7f6030059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:29.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6030056ea0 0x7f6030059360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f60400013e0 tx=0x7f6040013000).stop 2021-04-03T03:16:29.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60480d4660 msgr2=0x7f60480e7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:29.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60480d4660 0x7f60480e7210 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f6044002f30 tx=0x7f6044006ea0).stop 2021-04-03T03:16:29.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 shutdown_connections 2021-04-03T03:16:29.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6030056ea0 0x7f6030059360 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:29.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f60480a2540 0x7f604803c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f60480d4660 0x7f60480e7210 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 --2- 172.21.2.111:0/3685297241 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f60480a20c0 0x7f604803c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 >> 172.21.2.111:0/3685297241 conn(0x7f60480462b0 msgr2=0x7f6048042950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 shutdown_connections 2021-04-03T03:16:29.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:29.973+0000 7f6055898700 1 -- 172.21.2.111:0/3685297241 wait complete. 2021-04-03T03:16:29.983 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:16:30.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 Processor -- start 2021-04-03T03:16:30.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- start start 2021-04-03T03:16:30.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f45d00add50 0x7f45d0147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:30.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f45d0144ef0 0x7f45d0147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:30.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00607d0 con 0x7f45d00603f0 2021-04-03T03:16:30.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d014b090 con 0x7f45d0144ef0 2021-04-03T03:16:30.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d014b330 con 0x7f45d00add50 2021-04-03T03:16:30.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f45d0144ef0 0x7f45d0147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:30.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f45d0144ef0 0x7f45d0147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56948/0 (socket says 172.21.2.111:56948) 2021-04-03T03:16:30.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 -- 172.21.2.111:0/896538774 learned_addr learned my addr 172.21.2.111:0/896538774 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:30.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 -- 172.21.2.111:0/896538774 >> v2:172.21.2.115:3300/0 conn(0x7f45d00add50 msgr2=0x7f45d0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:30.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 --2- 172.21.2.111:0/896538774 >> v2:172.21.2.115:3300/0 conn(0x7f45d00add50 0x7f45d0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 -- 172.21.2.111:0/896538774 >> v1:172.21.2.111:6789/0 conn(0x7f45d00603f0 legacy=0x7f45d01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 -- 172.21.2.111:0/896538774 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d014b5d0 con 0x7f45d0144ef0 2021-04-03T03:16:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d6998700 1 --2- 172.21.2.111:0/896538774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d0147c30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f45c80032b0 tx=0x7f45c8009270).ready entity=mon.0 client_cookie=578379dd935ef2e4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d5195700 1 -- 172.21.2.111:0/896538774 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c800bc00 con 0x7f45d0144ef0 2021-04-03T03:16:30.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d5195700 1 -- 172.21.2.111:0/896538774 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c800bd60 con 0x7f45d0144ef0 2021-04-03T03:16:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d5195700 1 -- 172.21.2.111:0/896538774 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c8011400 con 0x7f45d0144ef0 2021-04-03T03:16:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- 172.21.2.111:0/896538774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 msgr2=0x7f45d0147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 --2- 172.21.2.111:0/896538774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d0147c30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f45c80032b0 tx=0x7f45c8009270).stop 2021-04-03T03:16:30.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- 172.21.2.111:0/896538774 shutdown_connections 2021-04-03T03:16:30.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 --2- 172.21.2.111:0/896538774 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d0147c30 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:30.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 --2- 172.21.2.111:0/896538774 >> v2:172.21.2.115:3300/0 conn(0x7f45d00add50 0x7f45d0147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:30.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- 172.21.2.111:0/896538774 >> 172.21.2.111:0/896538774 conn(0x7f45d00b61c0 msgr2=0x7f45d005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:30.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- 172.21.2.111:0/896538774 shutdown_connections 2021-04-03T03:16:30.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.045+0000 7f45d7199700 1 -- 172.21.2.111:0/896538774 wait complete. 2021-04-03T03:16:30.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 Processor -- start 2021-04-03T03:16:30.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 -- start start 2021-04-03T03:16:30.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45d00603f0 0x7f45d00a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:30.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45d00add50 0x7f45d01452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:30.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d01573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:30.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00a7df0 con 0x7f45d0144ef0 2021-04-03T03:16:30.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0156c60 con 0x7f45d00603f0 2021-04-03T03:16:30.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0156f00 con 0x7f45d00add50 2021-04-03T03:16:30.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d01573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:30.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d01573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56952/0 (socket says 172.21.2.111:56952) 2021-04-03T03:16:30.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 -- 172.21.2.111:0/4078872093 learned_addr learned my addr 172.21.2.111:0/4078872093 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:30.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 -- 172.21.2.111:0/4078872093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45d00add50 msgr2=0x7f45d01452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:30.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45d00add50 0x7f45d01452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:30.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 -- 172.21.2.111:0/4078872093 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45d00603f0 msgr2=0x7f45d00a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:30.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45d00603f0 0x7f45d00a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:30.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 -- 172.21.2.111:0/4078872093 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00a7a70 con 0x7f45d0144ef0 2021-04-03T03:16:30.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6998700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d01573a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f45c80032b0 tx=0x7f45c8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:30.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45c37fe700 1 -- 172.21.2.111:0/4078872093 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c8020620 con 0x7f45d0144ef0 2021-04-03T03:16:30.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45c37fe700 1 -- 172.21.2.111:0/4078872093 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8020780 con 0x7f45d0144ef0 2021-04-03T03:16:30.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45c37fe700 1 -- 172.21.2.111:0/4078872093 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c8020ae0 con 0x7f45d0144ef0 2021-04-03T03:16:30.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d0147a70 con 0x7f45d0144ef0 2021-04-03T03:16:30.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d0145950 con 0x7f45d0144ef0 2021-04-03T03:16:30.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45c37fe700 1 -- 172.21.2.111:0/4078872093 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f45c8026020 con 0x7f45d0144ef0 2021-04-03T03:16:30.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d01570c0 con 0x7f45d0144ef0 2021-04-03T03:16:30.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45c37fe700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45b0056ea0 0x7f45b0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:30.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45c37fe700 1 -- 172.21.2.111:0/4078872093 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f45c80714b0 con 0x7f45d0144ef0 2021-04-03T03:16:30.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6197700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45b0056ea0 0x7f45b0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:30.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.049+0000 7f45d6197700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45b0056ea0 0x7f45b0059360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f45d00ae520 tx=0x7f45c40077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:30.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.053+0000 7f45c37fe700 1 -- 172.21.2.111:0/4078872093 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f45c80468f0 con 0x7f45d0144ef0 2021-04-03T03:16:30.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.145+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f45d0057660 con 0x7f45d0144ef0 2021-04-03T03:16:30.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45c37fe700 1 -- 172.21.2.111:0/4078872093 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 185+0+0 (secure 0 0 0) 0x7f45c8015020 con 0x7f45d0144ef0 2021-04-03T03:16:30.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45b0056ea0 msgr2=0x7f45b0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:30.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45b0056ea0 0x7f45b0059360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f45d00ae520 tx=0x7f45c40077f0).stop 2021-04-03T03:16:30.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 msgr2=0x7f45d01573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:30.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d01573a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f45c80032b0 tx=0x7f45c8016000).stop 2021-04-03T03:16:30.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 shutdown_connections 2021-04-03T03:16:30.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f45d0144ef0 0x7f45d01573a0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:30.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f45b0056ea0 0x7f45b0059360 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:30.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f45d00603f0 0x7f45d00a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:30.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 --2- 172.21.2.111:0/4078872093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f45d00add50 0x7f45d01452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:30.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 >> 172.21.2.111:0/4078872093 conn(0x7f45d00b61c0 msgr2=0x7f45d00b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:30.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 shutdown_connections 2021-04-03T03:16:30.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:30.977+0000 7f45d7199700 1 -- 172.21.2.111:0/4078872093 wait complete. 2021-04-03T03:16:30.984 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:16:30.988 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:16:31.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.041+0000 7f11dffff700 1 Processor -- start 2021-04-03T03:16:31.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.041+0000 7f11dffff700 1 -- start start 2021-04-03T03:16:31.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11d803d560 0x7f11d80dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:31.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d80dc560 con 0x7f11d8041060 2021-04-03T03:16:31.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d80dc800 con 0x7f11d80dbba0 2021-04-03T03:16:31.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d803de60 con 0x7f11d803d560 2021-04-03T03:16:31.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11d803d560 0x7f11d80dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:31.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11d803d560 0x7f11d80dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56958/0 (socket says 172.21.2.111:56958) 2021-04-03T03:16:31.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/1947185677 learned_addr learned my addr 172.21.2.111:0/1947185677 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:31.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/1947185677 >> v1:172.21.2.115:6789/0 conn(0x7f11d80dbba0 legacy=0x7f11d803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:31.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/1947185677 >> v1:172.21.2.111:6789/0 conn(0x7f11d8041060 legacy=0x7f11d8041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:31.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11ddffb700 1 -- 172.21.2.111:0/1947185677 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897215624 0 0) 0x7f11d80dc560 con 0x7f11d8041060 2021-04-03T03:16:31.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/1947185677 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d803da30 con 0x7f11d803d560 2021-04-03T03:16:31.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- 172.21.2.111:0/1947185677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d803d560 0x7f11d80dbf80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f11d8040a80 tx=0x7f11e005ad70).ready entity=mon.0 client_cookie=b81c103b946e2f7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:31.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11ddffb700 1 -- 172.21.2.111:0/1947185677 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e005bda0 con 0x7f11d803d560 2021-04-03T03:16:31.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11ddffb700 1 -- 172.21.2.111:0/1947185677 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e005df60 con 0x7f11d803d560 2021-04-03T03:16:31.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11ddffb700 1 -- 172.21.2.111:0/1947185677 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e00699f0 con 0x7f11d803d560 2021-04-03T03:16:31.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- 172.21.2.111:0/1947185677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d803d560 msgr2=0x7f11d80dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:31.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 --2- 172.21.2.111:0/1947185677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d803d560 0x7f11d80dbf80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f11d8040a80 tx=0x7f11e005ad70).stop 2021-04-03T03:16:31.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- 172.21.2.111:0/1947185677 shutdown_connections 2021-04-03T03:16:31.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 --2- 172.21.2.111:0/1947185677 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d803d560 0x7f11d80dbf80 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:31.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- 172.21.2.111:0/1947185677 >> 172.21.2.111:0/1947185677 conn(0x7f11d80462b0 msgr2=0x7f11d80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:31.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- 172.21.2.111:0/1947185677 shutdown_connections 2021-04-03T03:16:31.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- 172.21.2.111:0/1947185677 wait complete. 2021-04-03T03:16:31.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 Processor -- start 2021-04-03T03:16:31.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- start start 2021-04-03T03:16:31.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11d803d560 0x7f11d80ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:31.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11d8041060 0x7f11d80ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:31.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d80dbba0 0x7f11d80eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:31.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d803c010 con 0x7f11d80dbba0 2021-04-03T03:16:31.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80dfa00 con 0x7f11d803d560 2021-04-03T03:16:31.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80a05b0 con 0x7f11d8041060 2021-04-03T03:16:31.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d80dbba0 0x7f11d80eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:31.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d80dbba0 0x7f11d80eaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56964/0 (socket says 172.21.2.111:56964) 2021-04-03T03:16:31.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/3298882951 learned_addr learned my addr 172.21.2.111:0/3298882951 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:31.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11de7fc700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11d8041060 0x7f11d80ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:31.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11deffd700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11d803d560 0x7f11d80ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:31.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11d8041060 msgr2=0x7f11d80ea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:31.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11d8041060 0x7f11d80ea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:31.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11d803d560 msgr2=0x7f11d80ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:31.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11d803d560 0x7f11d80ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:31.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 -- 172.21.2.111:0/3298882951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d80a07e0 con 0x7f11d80dbba0 2021-04-03T03:16:31.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11deffd700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11d803d560 0x7f11d80ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:31.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11df7fe700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d80dbba0 0x7f11d80eaa90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f11d8040a80 tx=0x7f11e0064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:31.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11cffff700 1 -- 172.21.2.111:0/3298882951 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e00441b0 con 0x7f11d80dbba0 2021-04-03T03:16:31.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11cffff700 1 -- 172.21.2.111:0/3298882951 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e005f2c0 con 0x7f11d80dbba0 2021-04-03T03:16:31.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d8039b00 con 0x7f11d80dbba0 2021-04-03T03:16:31.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11cffff700 1 -- 172.21.2.111:0/3298882951 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e005f620 con 0x7f11d80dbba0 2021-04-03T03:16:31.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.045+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d8039580 con 0x7f11d80dbba0 2021-04-03T03:16:31.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.049+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d8039c90 con 0x7f11d80dbba0 2021-04-03T03:16:31.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.049+0000 7f11cffff700 1 -- 172.21.2.111:0/3298882951 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f11e008c020 con 0x7f11d80dbba0 2021-04-03T03:16:31.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.049+0000 7f11cffff700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11bc057050 0x7f11bc059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:31.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.049+0000 7f11cffff700 1 -- 172.21.2.111:0/3298882951 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6064+0+0 (secure 0 0 0) 0x7f11e00e04e0 con 0x7f11d80dbba0 2021-04-03T03:16:31.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.049+0000 7f11deffd700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11bc057050 0x7f11bc059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:31.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.049+0000 7f11deffd700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11bc057050 0x7f11bc059510 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f11d0007000 tx=0x7f11d0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:31.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.049+0000 7f11cffff700 1 -- 172.21.2.111:0/3298882951 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f11e00b5a30 con 0x7f11d80dbba0 2021-04-03T03:16:31.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.145+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f11d80a0970 con 0x7f11d80dbba0 2021-04-03T03:16:31.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11cffff700 1 -- 172.21.2.111:0/3298882951 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v117) v1 ==== 133+0+0 (secure 0 0 0) 0x7f11e00b9050 con 0x7f11d80dbba0 2021-04-03T03:16:31.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11bc057050 msgr2=0x7f11bc059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:31.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11bc057050 0x7f11bc059510 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f11d0007000 tx=0x7f11d0008040).stop 2021-04-03T03:16:31.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d80dbba0 msgr2=0x7f11d80eaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:31.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d80dbba0 0x7f11d80eaa90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f11d8040a80 tx=0x7f11e0064030).stop 2021-04-03T03:16:31.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 shutdown_connections 2021-04-03T03:16:31.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11d80dbba0 0x7f11d80eaa90 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:31.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11bc057050 0x7f11bc059510 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:31.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11d803d560 0x7f11d80ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:31.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 --2- 172.21.2.111:0/3298882951 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11d8041060 0x7f11d80ea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:31.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 >> 172.21.2.111:0/3298882951 conn(0x7f11d80462b0 msgr2=0x7f11d80055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:31.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 shutdown_connections 2021-04-03T03:16:31.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:31.993+0000 7f11dffff700 1 -- 172.21.2.111:0/3298882951 wait complete. 2021-04-03T03:16:31.997 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:16:32.001 DEBUG:teuthology.orchestra.run.gibba026:> sudo rados -p cephfs_metadata put key /etc/fstab 2021-04-03T03:16:32.024 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 -- 172.21.2.126:0/3139709888 wait complete. 2021-04-03T03:16:32.025 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 Processor -- start 2021-04-03T03:16:32.025 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 -- start start 2021-04-03T03:16:32.025 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 0x5582b276d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.026 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b2759f50 0x5582b275d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.026 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b276d9d0 0x5582b275d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.026 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b266bc40 con 0x5582b276d9d0 2021-04-03T03:16:32.026 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b264b620 con 0x5582b2759460 2021-04-03T03:16:32.026 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.016+0000 7f847d197bc0 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582b263bfd0 con 0x5582b2759f50 2021-04-03T03:16:32.027 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846f42f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 0x5582b276d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.027 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b276d9d0 0x5582b275d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.027 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b276d9d0 0x5582b275d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.126:37992/0 (socket says 172.21.2.126:37992) 2021-04-03T03:16:32.028 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 -- 172.21.2.126:0/2597409703 learned_addr learned my addr 172.21.2.126:0/2597409703 (peer_addr_for_me v2:172.21.2.126:0/0) 2021-04-03T03:16:32.028 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b2759f50 0x5582b275d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.028 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 -- 172.21.2.126:0/2597409703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b2759f50 msgr2=0x5582b275d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:32.028 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b2759f50 0x5582b275d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:32.029 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 -- 172.21.2.126:0/2597409703 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 msgr2=0x5582b276d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:32.029 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 0x5582b276d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:32.029 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 -- 172.21.2.126:0/2597409703 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582b2634950 con 0x5582b276d9d0 2021-04-03T03:16:32.029 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b276d9d0 0x5582b275d9b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f84640010d0 tx=0x7f8468015000).ready entity=mon.0 client_cookie=c7e1c8a9d0f61b03 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:32.029 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846d42b700 1 -- 172.21.2.126:0/2597409703 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8468005a10 con 0x5582b276d9d0 2021-04-03T03:16:32.030 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846d42b700 1 -- 172.21.2.126:0/2597409703 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846800eca0 con 0x5582b276d9d0 2021-04-03T03:16:32.030 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846d42b700 1 -- 172.21.2.126:0/2597409703 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8468005c50 con 0x5582b276d9d0 2021-04-03T03:16:32.030 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2597409703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b276d9d0 msgr2=0x5582b275d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:32.030 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b276d9d0 0x5582b275d9b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f84640010d0 tx=0x7f8468015000).stop 2021-04-03T03:16:32.034 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2597409703 shutdown_connections 2021-04-03T03:16:32.035 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b276d9d0 0x5582b275d9b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:32.035 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 0x5582b276d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:32.035 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2597409703 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b2759f50 0x5582b275d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:32.035 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2597409703 >> 172.21.2.126:0/2597409703 conn(0x5582b2755200 msgr2=0x5582b27558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:32.036 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2597409703 shutdown_connections 2021-04-03T03:16:32.036 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2597409703 wait complete. 2021-04-03T03:16:32.036 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 Processor -- start 2021-04-03T03:16:32.036 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- start start 2021-04-03T03:16:32.037 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 0x5582b276bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.037 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b2759f50 0x5582b276c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.037 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b276d9d0 0x5582b2776530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.037 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b2757900 con 0x5582b2759f50 2021-04-03T03:16:32.037 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b2776a20 con 0x5582b2759460 2021-04-03T03:16:32.038 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582b2776bb0 con 0x5582b276d9d0 2021-04-03T03:16:32.038 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846fc30700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b276d9d0 0x5582b2776530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.038 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b2759f50 0x5582b276c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.038 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b2759f50 0x5582b276c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.126:37996/0 (socket says 172.21.2.126:37996) 2021-04-03T03:16:32.039 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 -- 172.21.2.126:0/2092054161 learned_addr learned my addr 172.21.2.126:0/2092054161 (peer_addr_for_me v2:172.21.2.126:0/0) 2021-04-03T03:16:32.039 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846f42f700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 0x5582b276bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.039 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 -- 172.21.2.126:0/2092054161 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b276d9d0 msgr2=0x5582b2776530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:32.039 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x5582b276d9d0 0x5582b2776530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:32.040 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 -- 172.21.2.126:0/2092054161 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 msgr2=0x5582b276bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:32.040 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x5582b2759460 0x5582b276bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:32.040 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 -- 172.21.2.126:0/2092054161 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582b2776d70 con 0x5582b2759f50 2021-04-03T03:16:32.040 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846ec2e700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b2759f50 0x5582b276c2a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f846400c000 tx=0x7f8464001950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:32.041 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846cc2a700 1 -- 172.21.2.126:0/2092054161 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8464010840 con 0x5582b2759f50 2021-04-03T03:16:32.041 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846cc2a700 1 -- 172.21.2.126:0/2092054161 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84640109a0 con 0x5582b2759f50 2021-04-03T03:16:32.041 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5582b2776fc0 con 0x5582b2759f50 2021-04-03T03:16:32.041 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582b277d590 con 0x5582b2759f50 2021-04-03T03:16:32.042 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846cc2a700 1 -- 172.21.2.126:0/2092054161 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8464010d00 con 0x5582b2759f50 2021-04-03T03:16:32.042 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846cc2a700 1 -- 172.21.2.126:0/2092054161 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8464019030 con 0x5582b2759f50 2021-04-03T03:16:32.042 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846cc2a700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f845c055b90 0x7f845c058050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.042 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846cc2a700 1 -- 172.21.2.126:0/2092054161 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6418+0+0 (secure 0 0 0) 0x7f846400b030 con 0x5582b2759f50 2021-04-03T03:16:32.043 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x5582b277e370 0x5582b2780760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:32.043 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 --> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x5582b2780cf0 con 0x5582b277e370 2021-04-03T03:16:32.043 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.020+0000 7f846f42f700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f845c055b90 0x7f845c058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.043 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.024+0000 7f846fc30700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x5582b277e370 0x5582b2780760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:32.044 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.024+0000 7f846f42f700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f845c055b90 0x7f845c058050 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f8460009000 tx=0x7f8460000b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:32.044 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:32.024+0000 7f846fc30700 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x5582b277e370 0x5582b2780760 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:33.178 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.176+0000 7f846cc2a700 1 -- 172.21.2.126:0/2092054161 <== osd.0 v2:172.21.2.111:6812/15909 1 ==== osd_map(118..118 src has 1..118) v4 ==== 320+0+0 (crc 0 0 0) 0x7f846800ebe0 con 0x5582b277e370 2021-04-03T03:16:33.183 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f846fc30700 1 -- 172.21.2.126:0/2092054161 <== osd.0 v2:172.21.2.111:6812/15909 2 ==== osd_op_reply(1 key [writefull 0~617] v118'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f846800eda0 con 0x5582b277e370 2021-04-03T03:16:33.185 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x5582b277e370 msgr2=0x5582b2780760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.185 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x5582b277e370 0x5582b2780760 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:33.185 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f845c055b90 msgr2=0x7f845c058050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.186 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f845c055b90 0x7f845c058050 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f8460009000 tx=0x7f8460000b80).stop 2021-04-03T03:16:33.186 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b2759f50 msgr2=0x5582b276c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.186 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 --2- 172.21.2.126:0/2092054161 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x5582b2759f50 0x5582b276c2a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f846400c000 tx=0x7f8464001950).stop 2021-04-03T03:16:33.186 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f846fc30700 1 -- 172.21.2.126:0/2092054161 reap_dead start 2021-04-03T03:16:33.187 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 shutdown_connections 2021-04-03T03:16:33.187 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 >> 172.21.2.126:0/2092054161 conn(0x5582b2755200 msgr2=0x5582b27558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:33.187 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 shutdown_connections 2021-04-03T03:16:33.187 INFO:teuthology.orchestra.run.gibba026.stderr:2021-04-03T03:16:33.180+0000 7f847d197bc0 1 -- 172.21.2.126:0/2092054161 wait complete. 2021-04-03T03:16:33.192 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2021-04-03T03:16:33.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 Processor -- start 2021-04-03T03:16:33.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 -- start start 2021-04-03T03:16:33.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f02bc0b12b0 0x7f02bc1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc0607d0 con 0x7f02bc14b700 2021-04-03T03:16:33.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc0b0eb0 con 0x7f02bc0603f0 2021-04-03T03:16:33.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02bc0af400 con 0x7f02bc0b12b0 2021-04-03T03:16:33.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f02bc14b700 0x7f02bc0b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57760/0 (socket says 172.21.2.111:57760) 2021-04-03T03:16:33.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 -- 172.21.2.111:0/2745384173 learned_addr learned my addr 172.21.2.111:0/2745384173 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:33.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c20d1700 1 --2- 172.21.2.111:0/2745384173 >> v2:172.21.2.115:3300/0 conn(0x7f02bc0b12b0 0x7f02bc1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:33.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136320882 0 0) 0x7f02bc0607d0 con 0x7f02bc14b700 2021-04-03T03:16:33.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a4003240 con 0x7f02bc14b700 2021-04-03T03:16:33.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2038369811 0 0) 0x7f02a4003240 con 0x7f02bc14b700 2021-04-03T03:16:33.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 >> v2:172.21.2.115:3300/0 conn(0x7f02bc0b12b0 msgr2=0x7f02bc1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:33.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 --2- 172.21.2.111:0/2745384173 >> v2:172.21.2.115:3300/0 conn(0x7f02bc0b12b0 0x7f02bc1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:33.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 >> v1:172.21.2.115:6789/0 conn(0x7f02bc0603f0 legacy=0x7f02bc0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0af6a0 con 0x7f02bc14b700 2021-04-03T03:16:33.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f02b00016d0 con 0x7f02bc14b700 2021-04-03T03:16:33.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 >> v1:172.21.2.111:6789/0 conn(0x7f02bc14b700 legacy=0x7f02bc0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02a4003760 0x7f02a4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02a4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02a4004e50 0x7f02a400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0af400 con 0x7f02a40044e0 2021-04-03T03:16:33.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4004290 con 0x7f02a4004e50 2021-04-03T03:16:33.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a400bc20 con 0x7f02a4003760 2021-04-03T03:16:33.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02a4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:33.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 -- 172.21.2.111:0/2745384173 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02a4003760 msgr2=0x7f02a4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:33.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02a4003760 0x7f02a4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:33.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 -- 172.21.2.111:0/2745384173 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02a4004e50 msgr2=0x7f02a400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:33.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02a4004e50 0x7f02a400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:33.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 -- 172.21.2.111:0/2745384173 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0af6a0 con 0x7f02a40044e0 2021-04-03T03:16:33.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c20d1700 1 -- 172.21.2.111:0/2745384173 reap_dead start 2021-04-03T03:16:33.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c10cf700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02a4004960 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f02b00013e0 tx=0x7f02b0007750).ready entity=mon.0 client_cookie=1ccd733d755c2779 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:33.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b0007e10 con 0x7f02a40044e0 2021-04-03T03:16:33.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b00072e0 con 0x7f02a40044e0 2021-04-03T03:16:33.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c08ce700 1 -- 172.21.2.111:0/2745384173 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b00057e0 con 0x7f02a40044e0 2021-04-03T03:16:33.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 -- 172.21.2.111:0/2745384173 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 msgr2=0x7f02a4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.253+0000 7f02c28d2700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02a4004960 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f02b00013e0 tx=0x7f02b0007750).stop 2021-04-03T03:16:33.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- 172.21.2.111:0/2745384173 shutdown_connections 2021-04-03T03:16:33.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 --2- 172.21.2.111:0/2745384173 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02a4004960 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:33.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- 172.21.2.111:0/2745384173 >> 172.21.2.111:0/2745384173 conn(0x7f02bc0b61c0 msgr2=0x7f02bc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:33.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- 172.21.2.111:0/2745384173 shutdown_connections 2021-04-03T03:16:33.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- 172.21.2.111:0/2745384173 wait complete. 2021-04-03T03:16:33.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 Processor -- start 2021-04-03T03:16:33.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- start start 2021-04-03T03:16:33.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02bc062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02bc0aaa30 0x7f02bc0abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02bc159ae0 0x7f02bc159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0650d0 con 0x7f02a40044e0 2021-04-03T03:16:33.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc15a480 con 0x7f02bc0aaa30 2021-04-03T03:16:33.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0ab650 con 0x7f02bc159ae0 2021-04-03T03:16:33.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02bc062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:33.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02bc062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:56980/0 (socket says 172.21.2.111:56980) 2021-04-03T03:16:33.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 -- 172.21.2.111:0/595177880 learned_addr learned my addr 172.21.2.111:0/595177880 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:33.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 -- 172.21.2.111:0/595177880 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02bc159ae0 msgr2=0x7f02bc159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02bc159ae0 0x7f02bc159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:33.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 -- 172.21.2.111:0/595177880 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02bc0aaa30 msgr2=0x7f02bc0abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02bc0aaa30 0x7f02bc0abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:33.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 -- 172.21.2.111:0/595177880 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0aa390 con 0x7f02a40044e0 2021-04-03T03:16:33.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c18d0700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02bc062bf0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f02bc06a3b0 tx=0x7f02b8007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:33.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02aeffd700 1 -- 172.21.2.111:0/595177880 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b800e4a0 con 0x7f02a40044e0 2021-04-03T03:16:33.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02aeffd700 1 -- 172.21.2.111:0/595177880 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b800e600 con 0x7f02a40044e0 2021-04-03T03:16:33.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc064c60 con 0x7f02a40044e0 2021-04-03T03:16:33.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02aeffd700 1 -- 172.21.2.111:0/595177880 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b800e960 con 0x7f02a40044e0 2021-04-03T03:16:33.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc0676c0 con 0x7f02a40044e0 2021-04-03T03:16:33.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc057660 con 0x7f02a40044e0 2021-04-03T03:16:33.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02aeffd700 1 -- 172.21.2.111:0/595177880 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f02b8019030 con 0x7f02a40044e0 2021-04-03T03:16:33.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02aeffd700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f029c057000 0x7f029c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:33.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02aeffd700 1 -- 172.21.2.111:0/595177880 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6418+0+0 (secure 0 0 0) 0x7f02b806f870 con 0x7f02a40044e0 2021-04-03T03:16:33.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c10cf700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f029c057000 0x7f029c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:33.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.257+0000 7f02c10cf700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f029c057000 0x7f029c0594c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f02b00013e0 tx=0x7f02b0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:33.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.261+0000 7f02aeffd700 1 -- 172.21.2.111:0/595177880 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f02b8044c50 con 0x7f02a40044e0 2021-04-03T03:16:33.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.389+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f02bc06b370 con 0x7f02a40044e0 2021-04-03T03:16:33.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.389+0000 7f02aeffd700 1 -- 172.21.2.111:0/595177880 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1588 (secure 0 0 0) 0x7f02b8048270 con 0x7f02a40044e0 2021-04-03T03:16:33.393 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:33.393 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:33.393 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:33.393 INFO:teuthology.orchestra.run.gibba011.stdout: "total_bytes": 773094113280, 2021-04-03T03:16:33.393 INFO:teuthology.orchestra.run.gibba011.stdout: "total_avail_bytes": 764492480512, 2021-04-03T03:16:33.394 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_bytes": 11698176, 2021-04-03T03:16:33.394 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_bytes": 8601632768, 2021-04-03T03:16:33.394 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_ratio": 0.011126242578029633, 2021-04-03T03:16:33.394 INFO:teuthology.orchestra.run.gibba011.stdout: "num_osds": 8, 2021-04-03T03:16:33.395 INFO:teuthology.orchestra.run.gibba011.stdout: "num_per_pool_osds": 8, 2021-04-03T03:16:33.395 INFO:teuthology.orchestra.run.gibba011.stdout: "num_per_pool_omap_osds": 8 2021-04-03T03:16:33.395 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:33.395 INFO:teuthology.orchestra.run.gibba011.stdout: "stats_by_class": { 2021-04-03T03:16:33.395 INFO:teuthology.orchestra.run.gibba011.stdout: "ssd": { 2021-04-03T03:16:33.396 INFO:teuthology.orchestra.run.gibba011.stdout: "total_bytes": 773094113280, 2021-04-03T03:16:33.396 INFO:teuthology.orchestra.run.gibba011.stdout: "total_avail_bytes": 764492480512, 2021-04-03T03:16:33.396 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_bytes": 11698176, 2021-04-03T03:16:33.396 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_bytes": 8601632768, 2021-04-03T03:16:33.397 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_ratio": 0.011126242578029633 2021-04-03T03:16:33.397 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:33.397 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:33.397 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:16:33.397 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:33.398 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "device_health_metrics", 2021-04-03T03:16:33.398 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 1, 2021-04-03T03:16:33.398 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:33.398 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:33.399 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 2, 2021-04-03T03:16:33.399 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:33.399 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:33.399 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:33.399 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362918805504 2021-04-03T03:16:33.400 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:33.400 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:33.400 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:33.400 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "rbd", 2021-04-03T03:16:33.401 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 2, 2021-04-03T03:16:33.401 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:33.401 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:33.401 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 0, 2021-04-03T03:16:33.401 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:33.402 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:33.402 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:33.402 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362918805504 2021-04-03T03:16:33.402 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:33.402 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:33.403 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:33.403 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "cephfs_data", 2021-04-03T03:16:33.403 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 22, 2021-04-03T03:16:33.403 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:33.404 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:33.404 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 0, 2021-04-03T03:16:33.404 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:33.404 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:33.404 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:33.405 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362918805504 2021-04-03T03:16:33.405 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:33.405 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:33.405 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:33.405 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.389+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f029c057000 msgr2=0x7f029c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.389+0000 7f02c28d2700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f029c057000 0x7f029c0594c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f02b00013e0 tx=0x7f02b0015000).stop 2021-04-03T03:16:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.389+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 msgr2=0x7f02bc062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02bc062bf0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f02bc06a3b0 tx=0x7f02b8007c00).stop 2021-04-03T03:16:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 shutdown_connections 2021-04-03T03:16:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f02bc159ae0 0x7f02bc159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f02a40044e0 0x7f02bc062bf0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f029c057000 0x7f029c0594c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 --2- 172.21.2.111:0/595177880 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f02bc0aaa30 0x7f02bc0abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 >> 172.21.2.111:0/595177880 conn(0x7f02bc0b61c0 msgr2=0x7f02bc0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 shutdown_connections 2021-04-03T03:16:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:33.393+0000 7f02c28d2700 1 -- 172.21.2.111:0/595177880 wait complete. 2021-04-03T03:16:33.410 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2021-04-03T03:16:38.411 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2021-04-03T03:16:38.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1bd012700 1 Processor -- start 2021-04-03T03:16:38.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1bd012700 1 -- start start 2021-04-03T03:16:38.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1bd012700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b0005640 0x7ff1b00d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1bd012700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b003d840 con 0x7ff1b003f120 2021-04-03T03:16:38.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1bd012700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b003f800 con 0x7ff1b00d8e90 2021-04-03T03:16:38.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1bd012700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b003faa0 con 0x7ff1b0005640 2021-04-03T03:16:38.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1b7fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff1b003f120 0x7ff1b0041fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57780/0 (socket says 172.21.2.111:57780) 2021-04-03T03:16:38.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1b7fff700 1 -- 172.21.2.111:0/3046382858 learned_addr learned my addr 172.21.2.111:0/3046382858 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:38.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587268758 0 0) 0x7ff1b003d840 con 0x7ff1b003f120 2021-04-03T03:16:38.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.469+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a4003240 con 0x7ff1b003f120 2021-04-03T03:16:38.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2952647792 0 0) 0x7ff1a4003240 con 0x7ff1b003f120 2021-04-03T03:16:38.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b0005640 msgr2=0x7ff1b00d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b0005640 0x7ff1b00d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 >> v1:172.21.2.115:6789/0 conn(0x7ff1b00d8e90 legacy=0x7ff1b00e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b0005ad0 con 0x7ff1b003f120 2021-04-03T03:16:38.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ff1a80016d0 con 0x7ff1b003f120 2021-04-03T03:16:38.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 >> v1:172.21.2.111:6789/0 conn(0x7ff1b003f120 legacy=0x7ff1b0041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b0005640 0x7ff1a40037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a4003d20 0x7ff1a40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1a40049d0 0x7ff1a4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b003faa0 con 0x7ff1a4003d20 2021-04-03T03:16:38.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40042b0 con 0x7ff1a40049d0 2021-04-03T03:16:38.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a400b730 con 0x7ff1b0005640 2021-04-03T03:16:38.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a4003d20 0x7ff1a40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b0005640 0x7ff1a40037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 -- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b0005640 msgr2=0x7ff1a40037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b0005640 0x7ff1a40037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 -- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1a40049d0 msgr2=0x7ff1a4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1a40049d0 0x7ff1a4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 -- 172.21.2.111:0/3046382858 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4003240 con 0x7ff1a4003d20 2021-04-03T03:16:38.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a4003d20 0x7ff1a40044e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7ff1ac002f30 tx=0x7ff1ac006e10).ready entity=mon.0 client_cookie=38aaabd637452cee server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ac009790 con 0x7ff1a4003d20 2021-04-03T03:16:38.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac000e90 con 0x7ff1a4003d20 2021-04-03T03:16:38.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b6ffd700 1 -- 172.21.2.111:0/3046382858 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ac009ae0 con 0x7ff1a4003d20 2021-04-03T03:16:38.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- 172.21.2.111:0/3046382858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a4003d20 msgr2=0x7ff1a40044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 --2- 172.21.2.111:0/3046382858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1a4003d20 0x7ff1a40044e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7ff1ac002f30 tx=0x7ff1ac006e10).stop 2021-04-03T03:16:38.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bc811700 1 -- 172.21.2.111:0/3046382858 reap_dead start 2021-04-03T03:16:38.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- 172.21.2.111:0/3046382858 shutdown_connections 2021-04-03T03:16:38.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- 172.21.2.111:0/3046382858 >> 172.21.2.111:0/3046382858 conn(0x7ff1b00462b0 msgr2=0x7ff1b00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:38.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- 172.21.2.111:0/3046382858 shutdown_connections 2021-04-03T03:16:38.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- 172.21.2.111:0/3046382858 wait complete. 2021-04-03T03:16:38.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 Processor -- start 2021-04-03T03:16:38.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- start start 2021-04-03T03:16:38.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b009fab0 0x7ff1b00d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1b00d74a0 0x7ff1b00d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1b00a2dd0 0x7ff1b00d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00d9b30 con 0x7ff1b00d74a0 2021-04-03T03:16:38.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b009fe90 con 0x7ff1b00a2dd0 2021-04-03T03:16:38.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00dca80 con 0x7ff1b009fab0 2021-04-03T03:16:38.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1b00d74a0 0x7ff1b00d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1b00d74a0 0x7ff1b00d6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57004/0 (socket says 172.21.2.111:57004) 2021-04-03T03:16:38.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 -- 172.21.2.111:0/2942142036 learned_addr learned my addr 172.21.2.111:0/2942142036 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:38.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 -- 172.21.2.111:0/2942142036 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b009fab0 msgr2=0x7ff1b00d67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b009fab0 0x7ff1b00d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 -- 172.21.2.111:0/2942142036 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1b00a2dd0 msgr2=0x7ff1b00d9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1b00a2dd0 0x7ff1b00d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 -- 172.21.2.111:0/2942142036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00a2260 con 0x7ff1b00d74a0 2021-04-03T03:16:38.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b77fe700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1b00d74a0 0x7ff1b00d6cb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7ff1ac002f30 tx=0x7ff1ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b57fa700 1 -- 172.21.2.111:0/2942142036 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ac009ea0 con 0x7ff1b00d74a0 2021-04-03T03:16:38.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b57fa700 1 -- 172.21.2.111:0/2942142036 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac00bec0 con 0x7ff1b00d74a0 2021-04-03T03:16:38.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b00dcd20 con 0x7ff1b00d74a0 2021-04-03T03:16:38.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1b57fa700 1 -- 172.21.2.111:0/2942142036 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ac01f5e0 con 0x7ff1b00d74a0 2021-04-03T03:16:38.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.473+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b00a2700 con 0x7ff1b00d74a0 2021-04-03T03:16:38.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.477+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b00a23f0 con 0x7ff1b00d74a0 2021-04-03T03:16:38.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.477+0000 7ff1b57fa700 1 -- 172.21.2.111:0/2942142036 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff1ac01d070 con 0x7ff1b00d74a0 2021-04-03T03:16:38.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.477+0000 7ff1b57fa700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198056f70 0x7ff198059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.477+0000 7ff1b57fa700 1 -- 172.21.2.111:0/2942142036 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6418+0+0 (secure 0 0 0) 0x7ff1ac071270 con 0x7ff1b00d74a0 2021-04-03T03:16:38.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.477+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198056f70 0x7ff198059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.477+0000 7ff1b7fff700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198056f70 0x7ff198059430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7ff1a8008000 tx=0x7ff1a8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.477+0000 7ff1b57fa700 1 -- 172.21.2.111:0/2942142036 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff1ac046540 con 0x7ff1b00d74a0 2021-04-03T03:16:38.607 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.605+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7ff1b0004d10 con 0x7ff1b00d74a0 2021-04-03T03:16:38.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.605+0000 7ff1b57fa700 1 -- 172.21.2.111:0/2942142036 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1932 (secure 0 0 0) 0x7ff1ac01a020 con 0x7ff1b00d74a0 2021-04-03T03:16:38.608 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:38.608 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:38.609 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:38.609 INFO:teuthology.orchestra.run.gibba011.stdout: "total_bytes": 773094113280, 2021-04-03T03:16:38.609 INFO:teuthology.orchestra.run.gibba011.stdout: "total_avail_bytes": 764492173312, 2021-04-03T03:16:38.609 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_bytes": 12005376, 2021-04-03T03:16:38.609 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_bytes": 8601939968, 2021-04-03T03:16:38.610 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_ratio": 0.011126640252768993, 2021-04-03T03:16:38.610 INFO:teuthology.orchestra.run.gibba011.stdout: "num_osds": 8, 2021-04-03T03:16:38.610 INFO:teuthology.orchestra.run.gibba011.stdout: "num_per_pool_osds": 8, 2021-04-03T03:16:38.610 INFO:teuthology.orchestra.run.gibba011.stdout: "num_per_pool_omap_osds": 8 2021-04-03T03:16:38.610 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:38.611 INFO:teuthology.orchestra.run.gibba011.stdout: "stats_by_class": { 2021-04-03T03:16:38.611 INFO:teuthology.orchestra.run.gibba011.stdout: "ssd": { 2021-04-03T03:16:38.611 INFO:teuthology.orchestra.run.gibba011.stdout: "total_bytes": 773094113280, 2021-04-03T03:16:38.611 INFO:teuthology.orchestra.run.gibba011.stdout: "total_avail_bytes": 764492173312, 2021-04-03T03:16:38.612 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_bytes": 12005376, 2021-04-03T03:16:38.612 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_bytes": 8601939968, 2021-04-03T03:16:38.612 INFO:teuthology.orchestra.run.gibba011.stdout: "total_used_raw_ratio": 0.011126640252768993 2021-04-03T03:16:38.612 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:38.612 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:38.613 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:16:38.613 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:38.613 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "device_health_metrics", 2021-04-03T03:16:38.613 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 1, 2021-04-03T03:16:38.613 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:38.614 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:38.614 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 2, 2021-04-03T03:16:38.614 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:38.614 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:38.614 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:38.615 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362918707200 2021-04-03T03:16:38.615 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:38.615 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:38.615 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:38.616 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "rbd", 2021-04-03T03:16:38.616 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 2, 2021-04-03T03:16:38.616 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:38.616 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:38.616 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 0, 2021-04-03T03:16:38.617 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:38.617 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:38.617 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:38.617 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362918707200 2021-04-03T03:16:38.617 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:38.618 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:38.618 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:38.618 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "cephfs_data", 2021-04-03T03:16:38.618 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 22, 2021-04-03T03:16:38.618 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:38.619 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 0, 2021-04-03T03:16:38.619 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 0, 2021-04-03T03:16:38.619 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 0, 2021-04-03T03:16:38.619 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 0, 2021-04-03T03:16:38.619 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 0, 2021-04-03T03:16:38.620 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362918707200 2021-04-03T03:16:38.620 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:38.620 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:38.620 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:38.620 INFO:teuthology.orchestra.run.gibba011.stdout: "name": "cephfs_metadata", 2021-04-03T03:16:38.621 INFO:teuthology.orchestra.run.gibba011.stdout: "id": 23, 2021-04-03T03:16:38.621 INFO:teuthology.orchestra.run.gibba011.stdout: "stats": { 2021-04-03T03:16:38.621 INFO:teuthology.orchestra.run.gibba011.stdout: "stored": 617, 2021-04-03T03:16:38.621 INFO:teuthology.orchestra.run.gibba011.stdout: "objects": 1, 2021-04-03T03:16:38.622 INFO:teuthology.orchestra.run.gibba011.stdout: "kb_used": 8, 2021-04-03T03:16:38.622 INFO:teuthology.orchestra.run.gibba011.stdout: "bytes_used": 8192, 2021-04-03T03:16:38.622 INFO:teuthology.orchestra.run.gibba011.stdout: "percent_used": 1.1286274137489727e-08, 2021-04-03T03:16:38.622 INFO:teuthology.orchestra.run.gibba011.stdout: "max_avail": 362918707200 2021-04-03T03:16:38.622 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:38.623 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:38.623 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:38.623 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:38.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.605+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198056f70 msgr2=0x7ff198059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.605+0000 7ff1bd012700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198056f70 0x7ff198059430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7ff1a8008000 tx=0x7ff1a8009040).stop 2021-04-03T03:16:38.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.605+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1b00d74a0 msgr2=0x7ff1b00d6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.605+0000 7ff1bd012700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1b00d74a0 0x7ff1b00d6cb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7ff1ac002f30 tx=0x7ff1ac015000).stop 2021-04-03T03:16:38.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 shutdown_connections 2021-04-03T03:16:38.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff198056f70 0x7ff198059430 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff1b00a2dd0 0x7ff1b00d9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff1b009fab0 0x7ff1b00d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 --2- 172.21.2.111:0/2942142036 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff1b00d74a0 0x7ff1b00d6cb0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 >> 172.21.2.111:0/2942142036 conn(0x7ff1b00462b0 msgr2=0x7ff1b0041930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:38.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 shutdown_connections 2021-04-03T03:16:38.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.609+0000 7ff1bd012700 1 -- 172.21.2.111:0/2942142036 wait complete. 2021-04-03T03:16:38.627 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2021-04-03T03:16:38.628 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data 2021-04-03T03:16:38.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 Processor -- start 2021-04-03T03:16:38.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- start start 2021-04-03T03:16:38.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c380603f0 0x7f8c381447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8c380add50 0x7f8c38144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c380607d0 con 0x7f8c380b17b0 2021-04-03T03:16:38.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c381451c0 con 0x7f8c380add50 2021-04-03T03:16:38.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3814b090 con 0x7f8c380603f0 2021-04-03T03:16:38.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36b85700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8c380b17b0 0x7f8c38147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57804/0 (socket says 172.21.2.111:57804) 2021-04-03T03:16:38.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36b85700 1 -- 172.21.2.111:0/1315407426 learned_addr learned my addr 172.21.2.111:0/1315407426 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35b83700 1 --2- 172.21.2.111:0/1315407426 >> v2:172.21.2.115:3300/0 conn(0x7f8c380add50 0x7f8c38144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c380603f0 0x7f8c381447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442281058 0 0) 0x7f8c380607d0 con 0x7f8c380b17b0 2021-04-03T03:16:38.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c18003240 con 0x7f8c380b17b0 2021-04-03T03:16:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1410409019 0 0) 0x7f8c18003240 con 0x7f8c380b17b0 2021-04-03T03:16:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c380603f0 msgr2=0x7f8c381447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c380603f0 0x7f8c381447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 >> v2:172.21.2.115:3300/0 conn(0x7f8c380add50 msgr2=0x7f8c38144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 --2- 172.21.2.111:0/1315407426 >> v2:172.21.2.115:3300/0 conn(0x7f8c380add50 0x7f8c38144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3814b330 con 0x7f8c380b17b0 2021-04-03T03:16:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8c30001a50 con 0x7f8c380b17b0 2021-04-03T03:16:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 >> v1:172.21.2.111:6789/0 conn(0x7f8c380b17b0 legacy=0x7f8c38147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c180037b0 0x7f8c18003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c180044e0 0x7f8c18004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c380603f0 0x7f8c18004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3814b090 con 0x7f8c180037b0 2021-04-03T03:16:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18005340 con 0x7f8c180044e0 2021-04-03T03:16:38.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1800b730 con 0x7f8c380603f0 2021-04-03T03:16:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c180037b0 0x7f8c18003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35b83700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c180044e0 0x7f8c18004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 -- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c380603f0 msgr2=0x7f8c18004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c380603f0 0x7f8c18004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 -- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c180044e0 msgr2=0x7f8c18004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c180044e0 0x7f8c18004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 -- 172.21.2.111:0/1315407426 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18003240 con 0x7f8c180037b0 2021-04-03T03:16:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35b83700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c180044e0 0x7f8c18004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:38.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36384700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c180037b0 0x7f8c18003bd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f8c2000c000 tx=0x7f8c20008040).ready entity=mon.0 client_cookie=d4407607d70ba085 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c2000ebc0 con 0x7f8c180037b0 2021-04-03T03:16:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2000ed20 con 0x7f8c180037b0 2021-04-03T03:16:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c35382700 1 -- 172.21.2.111:0/1315407426 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c200125e0 con 0x7f8c180037b0 2021-04-03T03:16:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- 172.21.2.111:0/1315407426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c180037b0 msgr2=0x7f8c18003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.697 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 --2- 172.21.2.111:0/1315407426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c180037b0 0x7f8c18003bd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f8c2000c000 tx=0x7f8c20008040).stop 2021-04-03T03:16:38.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c36b85700 1 -- 172.21.2.111:0/1315407426 reap_dead start 2021-04-03T03:16:38.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- 172.21.2.111:0/1315407426 shutdown_connections 2021-04-03T03:16:38.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- 172.21.2.111:0/1315407426 >> 172.21.2.111:0/1315407426 conn(0x7f8c380b61c0 msgr2=0x7f8c3805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:38.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- 172.21.2.111:0/1315407426 shutdown_connections 2021-04-03T03:16:38.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.685+0000 7f8c37386700 1 -- 172.21.2.111:0/1315407426 wait complete. 2021-04-03T03:16:38.698 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 Processor -- start 2021-04-03T03:16:38.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 -- start start 2021-04-03T03:16:38.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c38062f10 0x7f8c380632f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c380abb60 0x7f8c380662f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c38156780 0x7f8c38156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380aadd0 con 0x7f8c38062f10 2021-04-03T03:16:38.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c38064da0 con 0x7f8c380abb60 2021-04-03T03:16:38.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c38065bb0 con 0x7f8c38156780 2021-04-03T03:16:38.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c38062f10 0x7f8c380632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c38062f10 0x7f8c380632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57020/0 (socket says 172.21.2.111:57020) 2021-04-03T03:16:38.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 -- 172.21.2.111:0/1398510231 learned_addr learned my addr 172.21.2.111:0/1398510231 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:38.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 -- 172.21.2.111:0/1398510231 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c38156780 msgr2=0x7f8c38156c00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:38.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 --2- 172.21.2.111:0/1398510231 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8c38156780 0x7f8c38156c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 -- 172.21.2.111:0/1398510231 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c380abb60 msgr2=0x7f8c380662f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:38.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 --2- 172.21.2.111:0/1398510231 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8c380abb60 0x7f8c380662f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 -- 172.21.2.111:0/1398510231 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c38065e50 con 0x7f8c38062f10 2021-04-03T03:16:38.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c36384700 1 --2- 172.21.2.111:0/1398510231 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8c38062f10 0x7f8c380632f0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f8c2000c000 tx=0x7f8c20019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c2f7fe700 1 -- 172.21.2.111:0/1398510231 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c2001c450 con 0x7f8c38062f10 2021-04-03T03:16:38.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c2f7fe700 1 -- 172.21.2.111:0/1398510231 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2001c5b0 con 0x7f8c38062f10 2021-04-03T03:16:38.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 -- 172.21.2.111:0/1398510231 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c38063950 con 0x7f8c38062f10 2021-04-03T03:16:38.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c2f7fe700 1 -- 172.21.2.111:0/1398510231 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c2001c910 con 0x7f8c38062f10 2021-04-03T03:16:38.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 -- 172.21.2.111:0/1398510231 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c380667e0 con 0x7f8c38062f10 2021-04-03T03:16:38.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c37386700 1 -- 172.21.2.111:0/1398510231 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c38065fe0 con 0x7f8c38062f10 2021-04-03T03:16:38.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c2f7fe700 1 -- 172.21.2.111:0/1398510231 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8c2001ca70 con 0x7f8c38062f10 2021-04-03T03:16:38.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c2f7fe700 1 --2- 172.21.2.111:0/1398510231 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c1c056f70 0x7f8c1c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c2f7fe700 1 -- 172.21.2.111:0/1398510231 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6418+0+0 (secure 0 0 0) 0x7f8c20072080 con 0x7f8c38062f10 2021-04-03T03:16:38.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c35b83700 1 --2- 172.21.2.111:0/1398510231 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c1c056f70 0x7f8c1c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.689+0000 7f8c35b83700 1 --2- 172.21.2.111:0/1398510231 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8c1c056f70 0x7f8c1c059430 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f8c28008000 tx=0x7f8c28008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.693+0000 7f8c2f7fe700 1 -- 172.21.2.111:0/1398510231 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8c20047350 con 0x7f8c38062f10 2021-04-03T03:16:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.817+0000 7f8c37386700 1 -- 172.21.2.111:0/1398510231 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8c38057660 con 0x7f8c38062f10 2021-04-03T03:16:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.817+0000 7f8c2f7fe700 1 -- 172.21.2.111:0/1398510231 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v50) v1 ==== 209+0+0 (secure 0 0 0) 0x7f8c2004a970 con 0x7f8c38062f10 2021-04-03T03:16:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2021-04-03T03:16:38.820 DEBUG:teuthology.orchestra.run:got remote process result: 22 2021-04-03T03:16:38.821 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:16:38.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.877+0000 7fef51a70700 1 Processor -- start 2021-04-03T03:16:38.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.877+0000 7fef51a70700 1 -- start start 2021-04-03T03:16:38.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fef4c0603f0 0x7fef4c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fef4c0add50 0x7fef4c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c0607d0 con 0x7fef4c0b17b0 2021-04-03T03:16:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c1451c0 con 0x7fef4c0603f0 2021-04-03T03:16:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c14b090 con 0x7fef4c0add50 2021-04-03T03:16:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fef4c0603f0 0x7fef4c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fef4c0603f0 0x7fef4c1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57028/0 (socket says 172.21.2.111:57028) 2021-04-03T03:16:38.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 -- 172.21.2.111:0/1798542538 learned_addr learned my addr 172.21.2.111:0/1798542538 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:38.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef5126f700 1 --1- 172.21.2.111:0/1798542538 >> v1:172.21.2.111:6789/0 conn(0x7fef4c0b17b0 0x7fef4c147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57822/0 (socket says 172.21.2.111:57822) 2021-04-03T03:16:38.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 --2- 172.21.2.111:0/1798542538 >> v2:172.21.2.115:3300/0 conn(0x7fef4c0add50 0x7fef4c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 -- 172.21.2.111:0/1798542538 >> v2:172.21.2.115:3300/0 conn(0x7fef4c0add50 msgr2=0x7fef4c144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:38.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 --2- 172.21.2.111:0/1798542538 >> v2:172.21.2.115:3300/0 conn(0x7fef4c0add50 0x7fef4c144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 -- 172.21.2.111:0/1798542538 >> v1:172.21.2.111:6789/0 conn(0x7fef4c0b17b0 legacy=0x7fef4c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4b7fe700 1 -- 172.21.2.111:0/1798542538 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611246990 0 0) 0x7fef4c0607d0 con 0x7fef4c0b17b0 2021-04-03T03:16:38.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 -- 172.21.2.111:0/1798542538 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c14b330 con 0x7fef4c0603f0 2021-04-03T03:16:38.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef50a6e700 1 --2- 172.21.2.111:0/1798542538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0603f0 0x7fef4c1447e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fef3c002f30 tx=0x7fef3c006e90).ready entity=mon.0 client_cookie=12e93ac6507ee5ee server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4b7fe700 1 -- 172.21.2.111:0/1798542538 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef3c0099c0 con 0x7fef4c0603f0 2021-04-03T03:16:38.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4b7fe700 1 -- 172.21.2.111:0/1798542538 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c009b20 con 0x7fef4c0603f0 2021-04-03T03:16:38.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4b7fe700 1 -- 172.21.2.111:0/1798542538 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef3c009e80 con 0x7fef4c0603f0 2021-04-03T03:16:38.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- 172.21.2.111:0/1798542538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0603f0 msgr2=0x7fef4c1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- 172.21.2.111:0/1798542538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0603f0 0x7fef4c1447e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fef3c002f30 tx=0x7fef3c006e90).stop 2021-04-03T03:16:38.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- 172.21.2.111:0/1798542538 shutdown_connections 2021-04-03T03:16:38.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- 172.21.2.111:0/1798542538 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0603f0 0x7fef4c1447e0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- 172.21.2.111:0/1798542538 >> v2:172.21.2.115:3300/0 conn(0x7fef4c0add50 0x7fef4c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:38.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- 172.21.2.111:0/1798542538 >> 172.21.2.111:0/1798542538 conn(0x7fef4c0b61c0 msgr2=0x7fef4c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:38.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- 172.21.2.111:0/1798542538 shutdown_connections 2021-04-03T03:16:38.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- 172.21.2.111:0/1798542538 wait complete. 2021-04-03T03:16:38.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 Processor -- start 2021-04-03T03:16:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- start start 2021-04-03T03:16:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef4c0603f0 0x7fef4c1469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0add50 0x7fef4c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef4c0b17b0 0x7fef4c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1464a0 con 0x7fef4c0add50 2021-04-03T03:16:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c147b90 con 0x7fef4c0603f0 2021-04-03T03:16:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef51a70700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c147e30 con 0x7fef4c0b17b0 2021-04-03T03:16:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0add50 0x7fef4c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0add50 0x7fef4c156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57036/0 (socket says 172.21.2.111:57036) 2021-04-03T03:16:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 -- 172.21.2.111:0/3377740447 learned_addr learned my addr 172.21.2.111:0/3377740447 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 -- 172.21.2.111:0/3377740447 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef4c0b17b0 msgr2=0x7fef4c1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef4c0b17b0 0x7fef4c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 -- 172.21.2.111:0/3377740447 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef4c0603f0 msgr2=0x7fef4c1469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef4c0603f0 0x7fef4c1469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 -- 172.21.2.111:0/3377740447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0a7a40 con 0x7fef4c0add50 2021-04-03T03:16:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.881+0000 7fef4bfff700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0add50 0x7fef4c156840 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fef3400a000 tx=0x7fef34008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef49ffb700 1 -- 172.21.2.111:0/3377740447 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef340097d0 con 0x7fef4c0add50 2021-04-03T03:16:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef49ffb700 1 -- 172.21.2.111:0/3377740447 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef34009930 con 0x7fef4c0add50 2021-04-03T03:16:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c156d30 con 0x7fef4c0add50 2021-04-03T03:16:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef49ffb700 1 -- 172.21.2.111:0/3377740447 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef34009c90 con 0x7fef4c0add50 2021-04-03T03:16:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c067e60 con 0x7fef4c0add50 2021-04-03T03:16:38.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c1479e0 con 0x7fef4c0add50 2021-04-03T03:16:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef49ffb700 1 -- 172.21.2.111:0/3377740447 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fef3401c460 con 0x7fef4c0add50 2021-04-03T03:16:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef49ffb700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef30056f70 0x7fef30059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef49ffb700 1 -- 172.21.2.111:0/3377740447 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6418+0+0 (secure 0 0 0) 0x7fef340706d0 con 0x7fef4c0add50 2021-04-03T03:16:38.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef50a6e700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef30056f70 0x7fef30059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:38.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.885+0000 7fef50a6e700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef30056f70 0x7fef30059430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fef3c002f30 tx=0x7fef3c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:38.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:38.889+0000 7fef49ffb700 1 -- 172.21.2.111:0/3377740447 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fef34045940 con 0x7fef4c0add50 2021-04-03T03:16:39.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.013+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fef4c145ee0 con 0x7fef4c0add50 2021-04-03T03:16:39.015 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:16:39.013+0000 7f88ca3f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:16:39.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.017+0000 7fef49ffb700 1 -- 172.21.2.111:0/3377740447 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v51) v1 ==== 189+0+0 (secure 0 0 0) 0x7fef34048f60 con 0x7fef4c0add50 2021-04-03T03:16:39.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef30056f70 msgr2=0x7fef30059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:39.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef30056f70 0x7fef30059430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fef3c002f30 tx=0x7fef3c015000).stop 2021-04-03T03:16:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0add50 msgr2=0x7fef4c156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0add50 0x7fef4c156840 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fef3400a000 tx=0x7fef34008040).stop 2021-04-03T03:16:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 shutdown_connections 2021-04-03T03:16:39.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef4c0b17b0 0x7fef4c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef30056f70 0x7fef30059430 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef4c0603f0 0x7fef4c1469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 --2- 172.21.2.111:0/3377740447 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef4c0add50 0x7fef4c156840 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:39.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 >> 172.21.2.111:0/3377740447 conn(0x7fef4c0b61c0 msgr2=0x7fef4c0b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 shutdown_connections 2021-04-03T03:16:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.021+0000 7fef51a70700 1 -- 172.21.2.111:0/3377740447 wait complete. 2021-04-03T03:16:39.024 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 23 and data pool 22 2021-04-03T03:16:39.028 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:16:39.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 Processor -- start 2021-04-03T03:16:39.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- start start 2021-04-03T03:16:39.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f05b803e220 0x7f05b803ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:39.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b8042670 0x7f05b80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:39.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f05b8040c50 0x7f05b80d4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:39.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b8039580 con 0x7f05b803e220 2021-04-03T03:16:39.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b803e600 con 0x7f05b8040c50 2021-04-03T03:16:39.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8039770 con 0x7f05b8042670 2021-04-03T03:16:39.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f05b803e220 0x7f05b803ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:39.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f05b803e220 0x7f05b803ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57042/0 (socket says 172.21.2.111:57042) 2021-04-03T03:16:39.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 -- 172.21.2.111:0/21441094 learned_addr learned my addr 172.21.2.111:0/21441094 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:39.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 -- 172.21.2.111:0/21441094 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b8042670 msgr2=0x7f05b80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:39.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 --2- 172.21.2.111:0/21441094 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b8042670 0x7f05b80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 -- 172.21.2.111:0/21441094 >> v2:172.21.2.115:3300/0 conn(0x7f05b8040c50 msgr2=0x7f05b80d4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:39.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 --2- 172.21.2.111:0/21441094 >> v2:172.21.2.115:3300/0 conn(0x7f05b8040c50 0x7f05b80d4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 -- 172.21.2.111:0/21441094 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8039a10 con 0x7f05b803e220 2021-04-03T03:16:39.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05be59c700 1 --2- 172.21.2.111:0/21441094 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b803e220 0x7f05b803ff50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f05a8002f30 tx=0x7f05a8006e90).ready entity=mon.0 client_cookie=ff829fb9b38231fe server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:39.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bd59a700 1 -- 172.21.2.111:0/21441094 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a8009920 con 0x7f05b803e220 2021-04-03T03:16:39.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bd59a700 1 -- 172.21.2.111:0/21441094 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a8009a80 con 0x7f05b803e220 2021-04-03T03:16:39.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bd59a700 1 -- 172.21.2.111:0/21441094 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a8009de0 con 0x7f05b803e220 2021-04-03T03:16:39.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- 172.21.2.111:0/21441094 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b803e220 msgr2=0x7f05b803ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:39.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 --2- 172.21.2.111:0/21441094 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b803e220 0x7f05b803ff50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f05a8002f30 tx=0x7f05a8006e90).stop 2021-04-03T03:16:39.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- 172.21.2.111:0/21441094 shutdown_connections 2021-04-03T03:16:39.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 --2- 172.21.2.111:0/21441094 >> v2:172.21.2.115:3300/0 conn(0x7f05b8040c50 0x7f05b80d4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 --2- 172.21.2.111:0/21441094 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b803e220 0x7f05b803ff50 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:39.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 --2- 172.21.2.111:0/21441094 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b8042670 0x7f05b80ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- 172.21.2.111:0/21441094 >> 172.21.2.111:0/21441094 conn(0x7f05b80462b0 msgr2=0x7f05b80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:39.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- 172.21.2.111:0/21441094 shutdown_connections 2021-04-03T03:16:39.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.085+0000 7f05bf59e700 1 -- 172.21.2.111:0/21441094 wait complete. 2021-04-03T03:16:39.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 Processor -- start 2021-04-03T03:16:39.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 -- start start 2021-04-03T03:16:39.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b803e220 0x7f05b80cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:39.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05b8040c50 0x7f05b80d3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:39.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b8042670 0x7f05b80cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:39.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b803a4a0 con 0x7f05b8042670 2021-04-03T03:16:39.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b803a6d0 con 0x7f05b8040c50 2021-04-03T03:16:39.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b803caf0 con 0x7f05b803e220 2021-04-03T03:16:39.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b8042670 0x7f05b80cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:39.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b8042670 0x7f05b80cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57052/0 (socket says 172.21.2.111:57052) 2021-04-03T03:16:39.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 -- 172.21.2.111:0/4047378331 learned_addr learned my addr 172.21.2.111:0/4047378331 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:39.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 -- 172.21.2.111:0/4047378331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b803e220 msgr2=0x7f05b80cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:39.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b803e220 0x7f05b80cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 -- 172.21.2.111:0/4047378331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05b8040c50 msgr2=0x7f05b80d3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:39.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05b8040c50 0x7f05b80d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:39.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 -- 172.21.2.111:0/4047378331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b803ba20 con 0x7f05b8042670 2021-04-03T03:16:39.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bed9d700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b8042670 0x7f05b80cd100 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f05c006b5d0 tx=0x7f05c005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:39.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05b77fe700 1 -- 172.21.2.111:0/4047378331 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05c005f2c0 con 0x7f05b8042670 2021-04-03T03:16:39.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05b77fe700 1 -- 172.21.2.111:0/4047378331 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c005f420 con 0x7f05b8042670 2021-04-03T03:16:39.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b803bc10 con 0x7f05b8042670 2021-04-03T03:16:39.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b803af40 con 0x7f05b8042670 2021-04-03T03:16:39.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05b77fe700 1 -- 172.21.2.111:0/4047378331 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05c005f780 con 0x7f05b8042670 2021-04-03T03:16:39.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b803bda0 con 0x7f05b8042670 2021-04-03T03:16:39.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05b77fe700 1 -- 172.21.2.111:0/4047378331 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f05c006adb0 con 0x7f05b8042670 2021-04-03T03:16:39.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05b77fe700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05a4056f20 0x7f05a40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:39.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05b77fe700 1 -- 172.21.2.111:0/4047378331 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f05c00df270 con 0x7f05b8042670 2021-04-03T03:16:39.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05be59c700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05a4056f20 0x7f05a40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:39.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.089+0000 7f05be59c700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05a4056f20 0x7f05a40593e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f05a8002f30 tx=0x7f05a8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:39.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.093+0000 7f05b77fe700 1 -- 172.21.2.111:0/4047378331 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f05c00b4650 con 0x7f05b8042670 2021-04-03T03:16:39.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:39.217+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05b8004d30 con 0x7f05b8042670 2021-04-03T03:16:40.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05b77fe700 1 -- 172.21.2.111:0/4047378331 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05c00b7c70 con 0x7f05b8042670 2021-04-03T03:16:40.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05a4056f20 msgr2=0x7f05a40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:40.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05a4056f20 0x7f05a40593e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f05a8002f30 tx=0x7f05a8015000).stop 2021-04-03T03:16:40.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b8042670 msgr2=0x7f05b80cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:40.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b8042670 0x7f05b80cd100 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f05c006b5d0 tx=0x7f05c005ad70).stop 2021-04-03T03:16:40.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 shutdown_connections 2021-04-03T03:16:40.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05b8042670 0x7f05b80cd100 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:40.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05b803e220 0x7f05b80cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:40.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05a4056f20 0x7f05a40593e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:40.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 --2- 172.21.2.111:0/4047378331 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05b8040c50 0x7f05b80d3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:40.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 >> 172.21.2.111:0/4047378331 conn(0x7f05b80462b0 msgr2=0x7f05b80cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:40.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 shutdown_connections 2021-04-03T03:16:40.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.021+0000 7f05bf59e700 1 -- 172.21.2.111:0/4047378331 wait complete. 2021-04-03T03:16:40.031 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:16:40.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 Processor -- start 2021-04-03T03:16:40.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- start start 2021-04-03T03:16:40.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09380dbba0 0x7f09380dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:40.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 0x7f093803fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:40.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09380dc560 con 0x7f093803d560 2021-04-03T03:16:40.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093803de60 con 0x7f09380dbba0 2021-04-03T03:16:40.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09380dc800 con 0x7f0938041060 2021-04-03T03:16:40.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09380dbba0 0x7f09380dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:40.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f09380dbba0 0x7f09380dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57058/0 (socket says 172.21.2.111:57058) 2021-04-03T03:16:40.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 -- 172.21.2.111:0/2097417075 learned_addr learned my addr 172.21.2.111:0/2097417075 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:40.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 -- 172.21.2.111:0/2097417075 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 msgr2=0x7f093803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:40.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 --2- 172.21.2.111:0/2097417075 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 0x7f093803fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:40.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 -- 172.21.2.111:0/2097417075 >> v1:172.21.2.115:6789/0 conn(0x7f093803d560 legacy=0x7f0938041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:40.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 -- 172.21.2.111:0/2097417075 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093803da30 con 0x7f09380dbba0 2021-04-03T03:16:40.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093ede5700 1 --2- 172.21.2.111:0/2097417075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09380dbba0 0x7f09380dbf80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f093803f960 tx=0x7f0928006e90).ready entity=mon.0 client_cookie=65357ebfa15451ea server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:40.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093dde3700 1 -- 172.21.2.111:0/2097417075 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09280098e0 con 0x7f09380dbba0 2021-04-03T03:16:40.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093dde3700 1 -- 172.21.2.111:0/2097417075 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928009a40 con 0x7f09380dbba0 2021-04-03T03:16:40.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093dde3700 1 -- 172.21.2.111:0/2097417075 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0928009de0 con 0x7f09380dbba0 2021-04-03T03:16:40.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- 172.21.2.111:0/2097417075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09380dbba0 msgr2=0x7f09380dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:40.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- 172.21.2.111:0/2097417075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09380dbba0 0x7f09380dbf80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f093803f960 tx=0x7f0928006e90).stop 2021-04-03T03:16:40.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- 172.21.2.111:0/2097417075 shutdown_connections 2021-04-03T03:16:40.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- 172.21.2.111:0/2097417075 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09380dbba0 0x7f09380dbf80 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:40.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- 172.21.2.111:0/2097417075 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 0x7f093803fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:40.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- 172.21.2.111:0/2097417075 >> 172.21.2.111:0/2097417075 conn(0x7f09380462b0 msgr2=0x7f09380466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:40.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- 172.21.2.111:0/2097417075 shutdown_connections 2021-04-03T03:16:40.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- 172.21.2.111:0/2097417075 wait complete. 2021-04-03T03:16:40.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 Processor -- start 2021-04-03T03:16:40.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- start start 2021-04-03T03:16:40.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f093803d560 0x7f09380dca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:40.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 0x7f09380ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:40.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09380dbba0 0x7f09380d75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:40.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380df950 con 0x7f093803d560 2021-04-03T03:16:40.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093803b250 con 0x7f09380dbba0 2021-04-03T03:16:40.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.089+0000 7f093fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093803b4f0 con 0x7f0938041060 2021-04-03T03:16:40.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f093803d560 0x7f09380dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:40.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f093803d560 0x7f09380dca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57062/0 (socket says 172.21.2.111:57062) 2021-04-03T03:16:40.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 -- 172.21.2.111:0/952413124 learned_addr learned my addr 172.21.2.111:0/952413124 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:40.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 -- 172.21.2.111:0/952413124 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 msgr2=0x7f09380ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:40.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 0x7f09380ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:40.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 -- 172.21.2.111:0/952413124 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09380dbba0 msgr2=0x7f09380d75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:40.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09380dbba0 0x7f09380d75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:40.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 -- 172.21.2.111:0/952413124 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380de2d0 con 0x7f093803d560 2021-04-03T03:16:40.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093ede5700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f093803d560 0x7f09380dca00 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f093803f960 tx=0x7f0928015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:40.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f0937fff700 1 -- 172.21.2.111:0/952413124 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0928011ce0 con 0x7f093803d560 2021-04-03T03:16:40.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f0937fff700 1 -- 172.21.2.111:0/952413124 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928011e40 con 0x7f093803d560 2021-04-03T03:16:40.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f0937fff700 1 -- 172.21.2.111:0/952413124 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f092801e650 con 0x7f093803d560 2021-04-03T03:16:40.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09380d7a90 con 0x7f093803d560 2021-04-03T03:16:40.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093803ae60 con 0x7f093803d560 2021-04-03T03:16:40.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093803b0b0 con 0x7f093803d560 2021-04-03T03:16:40.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f0937fff700 1 -- 172.21.2.111:0/952413124 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0928025030 con 0x7f093803d560 2021-04-03T03:16:40.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f0937fff700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0924056f20 0x7f09240593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:40.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f0937fff700 1 -- 172.21.2.111:0/952413124 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0928071890 con 0x7f093803d560 2021-04-03T03:16:40.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093e5e4700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0924056f20 0x7f09240593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:40.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.093+0000 7f093e5e4700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0924056f20 0x7f09240593e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f0930002f30 tx=0x7f0930006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:40.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.097+0000 7f0937fff700 1 -- 172.21.2.111:0/952413124 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f093803b0b0 con 0x7f093803d560 2021-04-03T03:16:40.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:40.189+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f093803b0b0 con 0x7f093803d560 2021-04-03T03:16:41.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.029+0000 7f0937fff700 1 -- 172.21.2.111:0/952413124 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 185+0+0 (secure 0 0 0) 0x7f093803b0b0 con 0x7f093803d560 2021-04-03T03:16:41.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0924056f20 msgr2=0x7f09240593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:41.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0924056f20 0x7f09240593e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f0930002f30 tx=0x7f0930006a60).stop 2021-04-03T03:16:41.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f093803d560 msgr2=0x7f09380dca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:41.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f093803d560 0x7f09380dca00 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f093803f960 tx=0x7f0928015000).stop 2021-04-03T03:16:41.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 shutdown_connections 2021-04-03T03:16:41.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0924056f20 0x7f09240593e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:41.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09380dbba0 0x7f09380d75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:41.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f093803d560 0x7f09380dca00 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:41.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 --2- 172.21.2.111:0/952413124 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0938041060 0x7f09380ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:41.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 >> 172.21.2.111:0/952413124 conn(0x7f09380462b0 msgr2=0x7f0938041fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:41.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 shutdown_connections 2021-04-03T03:16:41.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.033+0000 7f093fde7700 1 -- 172.21.2.111:0/952413124 wait complete. 2021-04-03T03:16:41.040 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:16:41.047 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:16:41.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf93fff700 1 Processor -- start 2021-04-03T03:16:41.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf93fff700 1 -- start start 2021-04-03T03:16:41.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf93fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fcf940603f0 0x7fcf941447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:41.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf93fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940add50 0x7fcf94144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:41.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf93fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf940607d0 con 0x7fcf940b17b0 2021-04-03T03:16:41.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf93fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf941451c0 con 0x7fcf940603f0 2021-04-03T03:16:41.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf93fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9414b090 con 0x7fcf940add50 2021-04-03T03:16:41.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf927fc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940add50 0x7fcf94144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:41.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf937fe700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fcf940b17b0 0x7fcf94147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:42168/0 (socket says 172.21.2.111:42168) 2021-04-03T03:16:41.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf937fe700 1 -- 172.21.2.111:0/2105194923 learned_addr learned my addr 172.21.2.111:0/2105194923 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:41.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf92ffd700 1 --2- 172.21.2.111:0/2105194923 >> v2:172.21.2.115:3300/0 conn(0x7fcf940603f0 0x7fcf941447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:41.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf92ffd700 1 -- 172.21.2.111:0/2105194923 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940add50 msgr2=0x7fcf94144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:41.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf92ffd700 1 --2- 172.21.2.111:0/2105194923 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940add50 0x7fcf94144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:41.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf92ffd700 1 -- 172.21.2.111:0/2105194923 >> v1:172.21.2.115:6789/0 conn(0x7fcf940b17b0 legacy=0x7fcf94147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:41.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf92ffd700 1 -- 172.21.2.111:0/2105194923 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9414b330 con 0x7fcf940603f0 2021-04-03T03:16:41.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.105+0000 7fcf92ffd700 1 --2- 172.21.2.111:0/2105194923 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940603f0 0x7fcf941447e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fcf84002f30 tx=0x7fcf84006dd0).ready entity=mon.1 client_cookie=c78025cbbe656c73 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:41.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf91ffb700 1 -- 172.21.2.111:0/2105194923 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf84009760 con 0x7fcf940603f0 2021-04-03T03:16:41.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf91ffb700 1 -- 172.21.2.111:0/2105194923 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf840098c0 con 0x7fcf940603f0 2021-04-03T03:16:41.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf91ffb700 1 -- 172.21.2.111:0/2105194923 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf84009c20 con 0x7fcf940603f0 2021-04-03T03:16:41.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/2105194923 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940603f0 msgr2=0x7fcf941447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:41.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 --2- 172.21.2.111:0/2105194923 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940603f0 0x7fcf941447e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fcf84002f30 tx=0x7fcf84006dd0).stop 2021-04-03T03:16:41.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/2105194923 shutdown_connections 2021-04-03T03:16:41.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 --2- 172.21.2.111:0/2105194923 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940603f0 0x7fcf941447e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:41.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 --2- 172.21.2.111:0/2105194923 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940add50 0x7fcf94144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:41.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/2105194923 >> 172.21.2.111:0/2105194923 conn(0x7fcf940b61c0 msgr2=0x7fcf9405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:41.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/2105194923 shutdown_connections 2021-04-03T03:16:41.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/2105194923 wait complete. 2021-04-03T03:16:41.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 Processor -- start 2021-04-03T03:16:41.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- start start 2021-04-03T03:16:41.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf940603f0 0x7fcf94149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:41.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940add50 0x7fcf940b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:41.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940b17b0 0x7fcf940b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:41.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94062a70 con 0x7fcf940603f0 2021-04-03T03:16:41.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940b0ff0 con 0x7fcf940add50 2021-04-03T03:16:41.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940b3ff0 con 0x7fcf940b17b0 2021-04-03T03:16:41.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf940603f0 0x7fcf94149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:41.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf940603f0 0x7fcf94149910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57078/0 (socket says 172.21.2.111:57078) 2021-04-03T03:16:41.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 -- 172.21.2.111:0/3742383971 learned_addr learned my addr 172.21.2.111:0/3742383971 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:41.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf927fc700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940add50 0x7fcf940b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:41.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 -- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940b17b0 msgr2=0x7fcf940b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:41.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940b17b0 0x7fcf940b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:41.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 -- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940add50 msgr2=0x7fcf940b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:41.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940add50 0x7fcf940b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:41.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 -- 172.21.2.111:0/3742383971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf94062ca0 con 0x7fcf940603f0 2021-04-03T03:16:41.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf927fc700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940add50 0x7fcf940b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:41.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf92ffd700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf940603f0 0x7fcf94149910 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7fcf84002f30 tx=0x7fcf84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:41.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf8bfff700 1 -- 172.21.2.111:0/3742383971 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf84011e80 con 0x7fcf940603f0 2021-04-03T03:16:41.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf8bfff700 1 -- 172.21.2.111:0/3742383971 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8401c5c0 con 0x7fcf940603f0 2021-04-03T03:16:41.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf8bfff700 1 -- 172.21.2.111:0/3742383971 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf8401c920 con 0x7fcf940603f0 2021-04-03T03:16:41.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf940b3a70 con 0x7fcf940603f0 2021-04-03T03:16:41.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf940affc0 con 0x7fcf940603f0 2021-04-03T03:16:41.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.109+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf94057660 con 0x7fcf940603f0 2021-04-03T03:16:41.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.113+0000 7fcf8bfff700 1 -- 172.21.2.111:0/3742383971 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fcf8400f070 con 0x7fcf940603f0 2021-04-03T03:16:41.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.113+0000 7fcf8bfff700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf780570a0 0x7fcf78059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:41.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.113+0000 7fcf8bfff700 1 -- 172.21.2.111:0/3742383971 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6064+0+0 (secure 0 0 0) 0x7fcf84071f90 con 0x7fcf940603f0 2021-04-03T03:16:41.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.113+0000 7fcf927fc700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf780570a0 0x7fcf78059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:41.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.113+0000 7fcf927fc700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf780570a0 0x7fcf78059560 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fcf7c00b000 tx=0x7fcf7c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:41.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.113+0000 7fcf8bfff700 1 -- 172.21.2.111:0/3742383971 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fcf840473d0 con 0x7fcf940603f0 2021-04-03T03:16:41.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:41.209+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fcf94148850 con 0x7fcf940603f0 2021-04-03T03:16:42.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.037+0000 7fcf8bfff700 1 -- 172.21.2.111:0/3742383971 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v122) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcf8404a9f0 con 0x7fcf940603f0 2021-04-03T03:16:42.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf780570a0 msgr2=0x7fcf78059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf780570a0 0x7fcf78059560 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fcf7c00b000 tx=0x7fcf7c008040).stop 2021-04-03T03:16:42.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf940603f0 msgr2=0x7fcf94149910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf940603f0 0x7fcf94149910 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7fcf84002f30 tx=0x7fcf84015000).stop 2021-04-03T03:16:42.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 shutdown_connections 2021-04-03T03:16:42.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf940b17b0 0x7fcf940b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf940603f0 0x7fcf94149910 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf780570a0 0x7fcf78059560 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 --2- 172.21.2.111:0/3742383971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf940add50 0x7fcf940b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 >> 172.21.2.111:0/3742383971 conn(0x7fcf940b61c0 msgr2=0x7fcf940b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:42.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 shutdown_connections 2021-04-03T03:16:42.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.041+0000 7fcf93fff700 1 -- 172.21.2.111:0/3742383971 wait complete. 2021-04-03T03:16:42.045 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:16:42.050 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data 2021-04-03T03:16:42.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 Processor -- start 2021-04-03T03:16:42.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- start start 2021-04-03T03:16:42.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9d8002cbe0 0x7f9d800469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800d7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d80044e00 con 0x7f9d800059b0 2021-04-03T03:16:42.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8002d8c0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8002cfc0 con 0x7f9d800d7a90 2021-04-03T03:16:42.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85b83700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800d7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d86384700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9d8002cbe0 0x7f9d800469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d86b85700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f9d800059b0 0x7f9d80005e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:42184/0 (socket says 172.21.2.111:42184) 2021-04-03T03:16:42.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d86b85700 1 -- 172.21.2.111:0/208147407 learned_addr learned my addr 172.21.2.111:0/208147407 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:42.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85b83700 1 -- 172.21.2.111:0/208147407 >> v2:172.21.2.115:3300/0 conn(0x7f9d8002cbe0 msgr2=0x7f9d800469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:42.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85b83700 1 --2- 172.21.2.111:0/208147407 >> v2:172.21.2.115:3300/0 conn(0x7f9d8002cbe0 0x7f9d800469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85b83700 1 -- 172.21.2.111:0/208147407 >> v1:172.21.2.115:6789/0 conn(0x7f9d800059b0 legacy=0x7f9d80005e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85b83700 1 -- 172.21.2.111:0/208147407 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d80046e90 con 0x7f9d800d7a90 2021-04-03T03:16:42.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85b83700 1 --2- 172.21.2.111:0/208147407 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800d7ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f9d78002f30 tx=0x7f9d78006dd0).ready entity=mon.2 client_cookie=cc9ee9477752678c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85382700 1 -- 172.21.2.111:0/208147407 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d78009760 con 0x7f9d800d7a90 2021-04-03T03:16:42.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85382700 1 -- 172.21.2.111:0/208147407 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d780098c0 con 0x7f9d800d7a90 2021-04-03T03:16:42.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d85382700 1 -- 172.21.2.111:0/208147407 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d78009c20 con 0x7f9d800d7a90 2021-04-03T03:16:42.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- 172.21.2.111:0/208147407 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 msgr2=0x7f9d800d7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 --2- 172.21.2.111:0/208147407 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800d7ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f9d78002f30 tx=0x7f9d78006dd0).stop 2021-04-03T03:16:42.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- 172.21.2.111:0/208147407 shutdown_connections 2021-04-03T03:16:42.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 --2- 172.21.2.111:0/208147407 >> v2:172.21.2.115:3300/0 conn(0x7f9d8002cbe0 0x7f9d800469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 --2- 172.21.2.111:0/208147407 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800d7ef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- 172.21.2.111:0/208147407 >> 172.21.2.111:0/208147407 conn(0x7f9d80047160 msgr2=0x7f9d80047570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:42.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- 172.21.2.111:0/208147407 shutdown_connections 2021-04-03T03:16:42.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.109+0000 7f9d87386700 1 -- 172.21.2.111:0/208147407 wait complete. 2021-04-03T03:16:42.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 Processor -- start 2021-04-03T03:16:42.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 -- start start 2021-04-03T03:16:42.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d800059b0 0x7f9d8000cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d8002cbe0 0x7f9d800daa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800daef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800291c0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800d91b0 con 0x7f9d800059b0 2021-04-03T03:16:42.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800d82a0 con 0x7f9d800d7a90 2021-04-03T03:16:42.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d8002cbe0 0x7f9d800daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d8002cbe0 0x7f9d800daa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57094/0 (socket says 172.21.2.111:57094) 2021-04-03T03:16:42.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 -- 172.21.2.111:0/3811118599 learned_addr learned my addr 172.21.2.111:0/3811118599 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:42.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 -- 172.21.2.111:0/3811118599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 msgr2=0x7f9d800daef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 -- 172.21.2.111:0/3811118599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d800059b0 msgr2=0x7f9d8000cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d800059b0 0x7f9d8000cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 -- 172.21.2.111:0/3811118599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800095d0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d85b83700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d8002cbe0 0x7f9d800daa00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f9d78002f30 tx=0x7f9d78015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d7f7fe700 1 -- 172.21.2.111:0/3811118599 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d7801d450 con 0x7f9d8002cbe0 2021-04-03T03:16:42.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d7f7fe700 1 -- 172.21.2.111:0/3811118599 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7801d5b0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d7f7fe700 1 -- 172.21.2.111:0/3811118599 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d7801d910 con 0x7f9d8002cbe0 2021-04-03T03:16:42.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d80028de0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8000d2b0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d800d8ff0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d7f7fe700 1 -- 172.21.2.111:0/3811118599 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9d7800f070 con 0x7f9d8002cbe0 2021-04-03T03:16:42.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d7f7fe700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d6c0570a0 0x7f9d6c059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d7f7fe700 1 -- 172.21.2.111:0/3811118599 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6418+0+0 (secure 0 0 0) 0x7f9d780720f0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d86384700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d6c0570a0 0x7f9d6c059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.113+0000 7f9d86384700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d6c0570a0 0x7f9d6c059560 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f9d7000c000 tx=0x7f9d70006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.117+0000 7f9d7f7fe700 1 -- 172.21.2.111:0/3811118599 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9d780473c0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.241+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9d8000d4d0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.244 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:16:42.241+0000 7f88ca3f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:16:42.248 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d7f7fe700 1 -- 172.21.2.111:0/3811118599 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9d7804a9e0 con 0x7f9d8002cbe0 2021-04-03T03:16:42.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d6c0570a0 msgr2=0x7f9d6c059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d6c0570a0 0x7f9d6c059560 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f9d7000c000 tx=0x7f9d70006dd0).stop 2021-04-03T03:16:42.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d8002cbe0 msgr2=0x7f9d800daa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d8002cbe0 0x7f9d800daa00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f9d78002f30 tx=0x7f9d78015000).stop 2021-04-03T03:16:42.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 shutdown_connections 2021-04-03T03:16:42.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9d800d7a90 0x7f9d800daef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9d6c0570a0 0x7f9d6c059560 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9d800059b0 0x7f9d8000cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 --2- 172.21.2.111:0/3811118599 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9d8002cbe0 0x7f9d800daa00 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 >> 172.21.2.111:0/3811118599 conn(0x7f9d80047160 msgr2=0x7f9d8002a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:42.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 shutdown_connections 2021-04-03T03:16:42.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.249+0000 7f9d87386700 1 -- 172.21.2.111:0/3811118599 wait complete. 2021-04-03T03:16:42.253 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 24 and data pool 22 2021-04-03T03:16:42.258 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:16:42.258 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:42.264 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:42.271 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:42.284 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:42.296 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:42.308 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:16:42.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:42.309 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:42.310 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:42.331 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:16:42.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:42.333 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:42.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:42.334 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:42.384 INFO:teuthology.orchestra.run.gibba026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:16:42.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:42.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:42.386 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:42.386 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:42.395 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:16:42.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:42.397 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:42.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:42.398 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:42.448 INFO:teuthology.orchestra.run.gibba028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:16:42.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:42.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:42.450 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:42.451 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:16:42.462 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:16:42.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:42.464 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:16:42.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:42.465 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:16:42.513 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:16:42.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:42.515 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:16:42.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.573+0000 7fb8159e0700 1 Processor -- start 2021-04-03T03:16:42.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.573+0000 7fb8159e0700 1 -- start start 2021-04-03T03:16:42.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb8100604b0 0x7fb810142d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8100ade10 con 0x7fb8100600d0 2021-04-03T03:16:42.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8100ad8d0 con 0x7fb810143230 2021-04-03T03:16:42.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8100adb70 con 0x7fb8100604b0 2021-04-03T03:16:42.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb8100604b0 0x7fb810142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb8100604b0 0x7fb810142d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57100/0 (socket says 172.21.2.111:57100) 2021-04-03T03:16:42.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80ffff700 1 -- 172.21.2.111:0/3151532728 learned_addr learned my addr 172.21.2.111:0/3151532728 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:42.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8149de700 1 --1- 172.21.2.111:0/3151532728 >> v1:172.21.2.111:6789/0 conn(0x7fb8100600d0 0x7fb810139500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57890/0 (socket says 172.21.2.111:57890) 2021-04-03T03:16:42.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80ffff700 1 -- 172.21.2.111:0/3151532728 >> v1:172.21.2.115:6789/0 conn(0x7fb810143230 legacy=0x7fb810143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:42.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80ffff700 1 -- 172.21.2.111:0/3151532728 >> v1:172.21.2.111:6789/0 conn(0x7fb8100600d0 legacy=0x7fb810139500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80ffff700 1 -- 172.21.2.111:0/3151532728 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100ae630 con 0x7fb8100604b0 2021-04-03T03:16:42.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80ffff700 1 --2- 172.21.2.111:0/3151532728 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb8100604b0 0x7fb810142d40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7fb7f80010d0 tx=0x7fb800006f20).ready entity=mon.0 client_cookie=a2bda79897982fb4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80f7fe700 1 -- 172.21.2.111:0/3151532728 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb80000dd10 con 0x7fb8100604b0 2021-04-03T03:16:42.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80f7fe700 1 -- 172.21.2.111:0/3151532728 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80000de70 con 0x7fb8100604b0 2021-04-03T03:16:42.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb80f7fe700 1 -- 172.21.2.111:0/3151532728 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb800010660 con 0x7fb8100604b0 2021-04-03T03:16:42.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- 172.21.2.111:0/3151532728 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb8100604b0 msgr2=0x7fb810142d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 --2- 172.21.2.111:0/3151532728 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb8100604b0 0x7fb810142d40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7fb7f80010d0 tx=0x7fb800006f20).stop 2021-04-03T03:16:42.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- 172.21.2.111:0/3151532728 shutdown_connections 2021-04-03T03:16:42.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 --2- 172.21.2.111:0/3151532728 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb8100604b0 0x7fb810142d40 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- 172.21.2.111:0/3151532728 >> 172.21.2.111:0/3151532728 conn(0x7fb8100b61c0 msgr2=0x7fb81005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:42.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- 172.21.2.111:0/3151532728 shutdown_connections 2021-04-03T03:16:42.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- 172.21.2.111:0/3151532728 wait complete. 2021-04-03T03:16:42.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 Processor -- start 2021-04-03T03:16:42.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- start start 2021-04-03T03:16:42.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb8100600d0 0x7fb81013f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb8100604b0 0x7fb81013c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb810143230 0x7fb8101375a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81013d0b0 con 0x7fb810143230 2021-04-03T03:16:42.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81013bfe0 con 0x7fb8100600d0 2021-04-03T03:16:42.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8159e0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81013c280 con 0x7fb8100604b0 2021-04-03T03:16:42.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb810143230 0x7fb8101375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb810143230 0x7fb8101375a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57106/0 (socket says 172.21.2.111:57106) 2021-04-03T03:16:42.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 -- 172.21.2.111:0/1750199357 learned_addr learned my addr 172.21.2.111:0/1750199357 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:42.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 -- 172.21.2.111:0/1750199357 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb8100604b0 msgr2=0x7fb81013c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb8100604b0 0x7fb81013c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 -- 172.21.2.111:0/1750199357 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb8100600d0 msgr2=0x7fb81013f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb8100600d0 0x7fb81013f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 -- 172.21.2.111:0/1750199357 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81013fec0 con 0x7fb810143230 2021-04-03T03:16:42.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.577+0000 7fb8151df700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb810143230 0x7fb8101375a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7fb808003270 tx=0x7fb8080090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb80dffb700 1 -- 172.21.2.111:0/1750199357 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb80800c900 con 0x7fb810143230 2021-04-03T03:16:42.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb80dffb700 1 -- 172.21.2.111:0/1750199357 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80800ca60 con 0x7fb810143230 2021-04-03T03:16:42.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb80dffb700 1 -- 172.21.2.111:0/1750199357 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb80800cdc0 con 0x7fb810143230 2021-04-03T03:16:42.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81013cd30 con 0x7fb810143230 2021-04-03T03:16:42.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb810131a40 con 0x7fb810143230 2021-04-03T03:16:42.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb810057660 con 0x7fb810143230 2021-04-03T03:16:42.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb80dffb700 1 -- 172.21.2.111:0/1750199357 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb808015470 con 0x7fb810143230 2021-04-03T03:16:42.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb80dffb700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb7f4057000 0x7fb7f40594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb80dffb700 1 -- 172.21.2.111:0/1750199357 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fb8080702d0 con 0x7fb810143230 2021-04-03T03:16:42.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb8149de700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb7f4057000 0x7fb7f40594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb8149de700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb7f4057000 0x7fb7f40594c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fb7f8001190 tx=0x7fb7f80077a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.581+0000 7fb80dffb700 1 -- 172.21.2.111:0/1750199357 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb8080454f0 con 0x7fb810143230 2021-04-03T03:16:42.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.673+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb81013cec0 con 0x7fb810143230 2021-04-03T03:16:42.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb80dffb700 1 -- 172.21.2.111:0/1750199357 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb808048b10 con 0x7fb810143230 2021-04-03T03:16:42.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb7f4057000 msgr2=0x7fb7f40594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb7f4057000 0x7fb7f40594c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fb7f8001190 tx=0x7fb7f80077a0).stop 2021-04-03T03:16:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb810143230 msgr2=0x7fb8101375a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb810143230 0x7fb8101375a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7fb808003270 tx=0x7fb8080090f0).stop 2021-04-03T03:16:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 shutdown_connections 2021-04-03T03:16:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb810143230 0x7fb8101375a0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb7f4057000 0x7fb7f40594c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb8100600d0 0x7fb81013f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 --2- 172.21.2.111:0/1750199357 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb8100604b0 0x7fb81013c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 >> 172.21.2.111:0/1750199357 conn(0x7fb8100b61c0 msgr2=0x7fb810142610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:42.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 shutdown_connections 2021-04-03T03:16:42.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.681+0000 7fb8159e0700 1 -- 172.21.2.111:0/1750199357 wait complete. 2021-04-03T03:16:42.690 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2021-04-03T03:16:42.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 Processor -- start 2021-04-03T03:16:42.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- start start 2021-04-03T03:16:42.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe74014b700 0x7fe7400b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7400607d0 con 0x7fe7400603f0 2021-04-03T03:16:42.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7400af400 con 0x7fe7400b0c90 2021-04-03T03:16:42.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400af6a0 con 0x7fe74014b700 2021-04-03T03:16:42.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe744d1f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe7400603f0 0x7fe7400b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57904/0 (socket says 172.21.2.111:57904) 2021-04-03T03:16:42.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe744d1f700 1 -- 172.21.2.111:0/3099685298 learned_addr learned my addr 172.21.2.111:0/3099685298 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:42.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606770308 0 0) 0x7fe7400607d0 con 0x7fe7400603f0 2021-04-03T03:16:42.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe724003240 con 0x7fe7400603f0 2021-04-03T03:16:42.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73ffff700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe74014b700 0x7fe7400b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3593028089 0 0) 0x7fe724003240 con 0x7fe7400603f0 2021-04-03T03:16:42.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe74014b700 msgr2=0x7fe7400b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:42.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe74014b700 0x7fe7400b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 >> v1:172.21.2.115:6789/0 conn(0x7fe7400b0c90 legacy=0x7fe740147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:42.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400afb40 con 0x7fe7400603f0 2021-04-03T03:16:42.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe7340016d0 con 0x7fe7400603f0 2021-04-03T03:16:42.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 >> v1:172.21.2.111:6789/0 conn(0x7fe7400603f0 legacy=0x7fe7400b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe724003760 0x7fe724003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7240044e0 0x7fe724004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe724004e50 0x7fe72400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400af6a0 con 0x7fe724004e50 2021-04-03T03:16:42.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240041f0 con 0x7fe7240044e0 2021-04-03T03:16:42.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72400bc20 con 0x7fe724003760 2021-04-03T03:16:42.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73ffff700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe74014b700 0x7fe7400b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:42.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe724004e50 0x7fe72400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73ffff700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7240044e0 0x7fe724004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe744d1f700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe724003760 0x7fe724003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 -- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe724003760 msgr2=0x7fe724003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:42.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe724003760 0x7fe724003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 -- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7240044e0 msgr2=0x7fe724004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:42.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe7240044e0 0x7fe724004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 -- 172.21.2.111:0/3099685298 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724003240 con 0x7fe724004e50 2021-04-03T03:16:42.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe724004e50 0x7fe72400b730 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7fe7400b5640 tx=0x7fe738007ef0).ready entity=mon.0 client_cookie=b7f312d78eea82ab server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe738001540 con 0x7fe724004e50 2021-04-03T03:16:42.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73800ca10 con 0x7fe724004e50 2021-04-03T03:16:42.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe73f7fe700 1 -- 172.21.2.111:0/3099685298 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73800cd70 con 0x7fe724004e50 2021-04-03T03:16:42.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- 172.21.2.111:0/3099685298 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe724004e50 msgr2=0x7fe72400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 --2- 172.21.2.111:0/3099685298 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe724004e50 0x7fe72400b730 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7fe7400b5640 tx=0x7fe738007ef0).stop 2021-04-03T03:16:42.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745520700 1 -- 172.21.2.111:0/3099685298 reap_dead start 2021-04-03T03:16:42.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- 172.21.2.111:0/3099685298 shutdown_connections 2021-04-03T03:16:42.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- 172.21.2.111:0/3099685298 >> 172.21.2.111:0/3099685298 conn(0x7fe7400b61c0 msgr2=0x7fe74005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:42.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- 172.21.2.111:0/3099685298 shutdown_connections 2021-04-03T03:16:42.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.745+0000 7fe745d21700 1 -- 172.21.2.111:0/3099685298 wait complete. 2021-04-03T03:16:42.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 Processor -- start 2021-04-03T03:16:42.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 -- start start 2021-04-03T03:16:42.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe740149ed0 0x7fe7400a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7400a98d0 0x7fe740062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe740062ff0 0x7fe7401457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400a8a50 con 0x7fe7400a98d0 2021-04-03T03:16:42.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400a96f0 con 0x7fe740062ff0 2021-04-03T03:16:42.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74014a2b0 con 0x7fe740149ed0 2021-04-03T03:16:42.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7400a98d0 0x7fe740062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7400a98d0 0x7fe740062b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57126/0 (socket says 172.21.2.111:57126) 2021-04-03T03:16:42.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 -- 172.21.2.111:0/448421283 learned_addr learned my addr 172.21.2.111:0/448421283 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:42.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe744d1f700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe740149ed0 0x7fe7400a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 -- 172.21.2.111:0/448421283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe740149ed0 msgr2=0x7fe7400a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe740149ed0 0x7fe7400a6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:42.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 -- 172.21.2.111:0/448421283 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe740062ff0 msgr2=0x7fe7401457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:42.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe740062ff0 0x7fe7401457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:42.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 -- 172.21.2.111:0/448421283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400a87b0 con 0x7fe7400a98d0 2021-04-03T03:16:42.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73ffff700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7400a98d0 0x7fe740062b00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fe7400a8b90 tx=0x7fe72c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73dffb700 1 -- 172.21.2.111:0/448421283 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe72c0138d0 con 0x7fe7400a98d0 2021-04-03T03:16:42.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73dffb700 1 -- 172.21.2.111:0/448421283 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c013a30 con 0x7fe7400a98d0 2021-04-03T03:16:42.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7400a7410 con 0x7fe7400a98d0 2021-04-03T03:16:42.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73dffb700 1 -- 172.21.2.111:0/448421283 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe72c013d90 con 0x7fe7400a98d0 2021-04-03T03:16:42.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7400a93e0 con 0x7fe7400a98d0 2021-04-03T03:16:42.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe740057660 con 0x7fe7400a98d0 2021-04-03T03:16:42.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73dffb700 1 -- 172.21.2.111:0/448421283 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe72c01b3b0 con 0x7fe7400a98d0 2021-04-03T03:16:42.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73dffb700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe728056f20 0x7fe7280593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:42.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe73dffb700 1 -- 172.21.2.111:0/448421283 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fe72c0704c0 con 0x7fe7400a98d0 2021-04-03T03:16:42.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe744d1f700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe728056f20 0x7fe7280593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:42.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.749+0000 7fe744d1f700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe728056f20 0x7fe7280593e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fe73400d000 tx=0x7fe734009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:42.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.753+0000 7fe73dffb700 1 -- 172.21.2.111:0/448421283 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe72c0457f0 con 0x7fe7400a98d0 2021-04-03T03:16:42.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:42.881+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7fe740062780 con 0x7fe7400a98d0 2021-04-03T03:16:43.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe73dffb700 1 -- 172.21.2.111:0/448421283 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v353) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe72c017020 con 0x7fe7400a98d0 2021-04-03T03:16:43.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe728056f20 msgr2=0x7fe7280593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:43.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe728056f20 0x7fe7280593e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fe73400d000 tx=0x7fe734009040).stop 2021-04-03T03:16:43.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7400a98d0 msgr2=0x7fe740062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:43.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7400a98d0 0x7fe740062b00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fe7400a8b90 tx=0x7fe72c008040).stop 2021-04-03T03:16:43.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 shutdown_connections 2021-04-03T03:16:43.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe728056f20 0x7fe7280593e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:43.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe740062ff0 0x7fe7401457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:43.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe740149ed0 0x7fe7400a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:43.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 --2- 172.21.2.111:0/448421283 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe7400a98d0 0x7fe740062b00 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:43.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 >> 172.21.2.111:0/448421283 conn(0x7fe7400b61c0 msgr2=0x7fe74014ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:43.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 shutdown_connections 2021-04-03T03:16:43.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.053+0000 7fe745d21700 1 -- 172.21.2.111:0/448421283 wait complete. 2021-04-03T03:16:43.061 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2021-04-03T03:16:43.062 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:16:43.063 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2021-04-03T03:16:43.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 Processor -- start 2021-04-03T03:16:43.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- start start 2021-04-03T03:16:43.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3ed00603f0 0x7f3ed00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:43.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3ed00b0c90 0x7f3ed0147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:43.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed00af400 con 0x7f3ed014b700 2021-04-03T03:16:43.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed0060860 con 0x7f3ed00b0c90 2021-04-03T03:16:43.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00af6a0 con 0x7f3ed00603f0 2021-04-03T03:16:43.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3ed00b0c90 0x7f3ed0147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:43.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3ed00b0c90 0x7f3ed0147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57134/0 (socket says 172.21.2.111:57134) 2021-04-03T03:16:43.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 -- 172.21.2.111:0/2963410688 learned_addr learned my addr 172.21.2.111:0/2963410688 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:43.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 -- 172.21.2.111:0/2963410688 >> v2:172.21.2.115:3300/0 conn(0x7f3ed00603f0 msgr2=0x7f3ed00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:43.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 --2- 172.21.2.111:0/2963410688 >> v2:172.21.2.115:3300/0 conn(0x7f3ed00603f0 0x7f3ed00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:43.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 -- 172.21.2.111:0/2963410688 >> v1:172.21.2.111:6789/0 conn(0x7f3ed014b700 legacy=0x7f3ed00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:43.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 -- 172.21.2.111:0/2963410688 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00afb40 con 0x7f3ed00b0c90 2021-04-03T03:16:43.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed4b82700 1 --2- 172.21.2.111:0/2963410688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed00b0c90 0x7f3ed0147680 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f3ec80032b0 tx=0x7f3ec8009270).ready entity=mon.0 client_cookie=412cacc533540965 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:43.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3eceffd700 1 -- 172.21.2.111:0/2963410688 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec800bc00 con 0x7f3ed00b0c90 2021-04-03T03:16:43.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3eceffd700 1 -- 172.21.2.111:0/2963410688 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec800bd60 con 0x7f3ed00b0c90 2021-04-03T03:16:43.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3eceffd700 1 -- 172.21.2.111:0/2963410688 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec8011400 con 0x7f3ed00b0c90 2021-04-03T03:16:43.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- 172.21.2.111:0/2963410688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed00b0c90 msgr2=0x7f3ed0147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:43.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- 172.21.2.111:0/2963410688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed00b0c90 0x7f3ed0147680 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f3ec80032b0 tx=0x7f3ec8009270).stop 2021-04-03T03:16:43.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- 172.21.2.111:0/2963410688 shutdown_connections 2021-04-03T03:16:43.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- 172.21.2.111:0/2963410688 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed00b0c90 0x7f3ed0147680 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:43.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- 172.21.2.111:0/2963410688 >> v2:172.21.2.115:3300/0 conn(0x7f3ed00603f0 0x7f3ed00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:43.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- 172.21.2.111:0/2963410688 >> 172.21.2.111:0/2963410688 conn(0x7f3ed00b61c0 msgr2=0x7f3ed005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:43.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- 172.21.2.111:0/2963410688 shutdown_connections 2021-04-03T03:16:43.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- 172.21.2.111:0/2963410688 wait complete. 2021-04-03T03:16:43.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 Processor -- start 2021-04-03T03:16:43.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- start start 2021-04-03T03:16:43.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ed00603f0 0x7f3ed00ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:43.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ed00b0c90 0x7f3ed00675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:43.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed014b700 0x7f3ed0067a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:43.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0157910 con 0x7f3ed014b700 2021-04-03T03:16:43.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00a9ac0 con 0x7f3ed00b0c90 2021-04-03T03:16:43.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.129+0000 7f3ed5383700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0064340 con 0x7f3ed00603f0 2021-04-03T03:16:43.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed014b700 0x7f3ed0067a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:43.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed014b700 0x7f3ed0067a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57142/0 (socket says 172.21.2.111:57142) 2021-04-03T03:16:43.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 -- 172.21.2.111:0/3511524935 learned_addr learned my addr 172.21.2.111:0/3511524935 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:43.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 -- 172.21.2.111:0/3511524935 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ed00603f0 msgr2=0x7f3ed00ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:43.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ed00603f0 0x7f3ed00ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:43.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 -- 172.21.2.111:0/3511524935 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ed00b0c90 msgr2=0x7f3ed00675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:43.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ed00b0c90 0x7f3ed00675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:43.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 -- 172.21.2.111:0/3511524935 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0064570 con 0x7f3ed014b700 2021-04-03T03:16:43.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed4b82700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed014b700 0x7f3ed0067a90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f3ec80032b0 tx=0x7f3ec8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:43.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecd7fa700 1 -- 172.21.2.111:0/3511524935 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec8020620 con 0x7f3ed014b700 2021-04-03T03:16:43.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecd7fa700 1 -- 172.21.2.111:0/3511524935 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec800bed0 con 0x7f3ed014b700 2021-04-03T03:16:43.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed0064810 con 0x7f3ed014b700 2021-04-03T03:16:43.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed00abed0 con 0x7f3ed014b700 2021-04-03T03:16:43.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecd7fa700 1 -- 172.21.2.111:0/3511524935 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec8020970 con 0x7f3ed014b700 2021-04-03T03:16:43.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed0057660 con 0x7f3ed014b700 2021-04-03T03:16:43.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecd7fa700 1 -- 172.21.2.111:0/3511524935 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3ec8026020 con 0x7f3ed014b700 2021-04-03T03:16:43.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecd7fa700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3eb4056ef0 0x7f3eb40593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:43.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecd7fa700 1 -- 172.21.2.111:0/3511524935 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f3ec8071050 con 0x7f3ed014b700 2021-04-03T03:16:43.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecffff700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3eb4056ef0 0x7f3eb40593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:43.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.133+0000 7f3ecffff700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3eb4056ef0 0x7f3eb40593b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f3eb8002f30 tx=0x7f3eb8006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:43.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.137+0000 7f3ecd7fa700 1 -- 172.21.2.111:0/3511524935 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3ec80463e0 con 0x7f3ed014b700 2021-04-03T03:16:43.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:43.269+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f3ed00b1550 con 0x7f3ed014b700 2021-04-03T03:16:44.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ecd7fa700 1 -- 172.21.2.111:0/3511524935 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v354) v1 ==== 131+0+0 (secure 0 0 0) 0x7f3ec8015020 con 0x7f3ed014b700 2021-04-03T03:16:44.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3eb4056ef0 msgr2=0x7f3eb40593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3eb4056ef0 0x7f3eb40593b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f3eb8002f30 tx=0x7f3eb8006a60).stop 2021-04-03T03:16:44.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed014b700 msgr2=0x7f3ed0067a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed014b700 0x7f3ed0067a90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f3ec80032b0 tx=0x7f3ec8016000).stop 2021-04-03T03:16:44.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 shutdown_connections 2021-04-03T03:16:44.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3ed014b700 0x7f3ed0067a90 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3ed00603f0 0x7f3ed00ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3eb4056ef0 0x7f3eb40593b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 --2- 172.21.2.111:0/3511524935 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3ed00b0c90 0x7f3ed00675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 >> 172.21.2.111:0/3511524935 conn(0x7f3ed00b61c0 msgr2=0x7f3ed00acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:44.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 shutdown_connections 2021-04-03T03:16:44.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.057+0000 7f3ed5383700 1 -- 172.21.2.111:0/3511524935 wait complete. 2021-04-03T03:16:44.066 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:16:44.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.129+0000 7fcfe5ddc700 1 Processor -- start 2021-04-03T03:16:44.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- start start 2021-04-03T03:16:44.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 0x7fcfd8042360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcfd803ccc0 0x7fcfd803eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd80411e0 con 0x7fcfd803d860 2021-04-03T03:16:44.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80d4b80 con 0x7fcfd803ccc0 2021-04-03T03:16:44.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd803d130 con 0x7fcfd8041f80 2021-04-03T03:16:44.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcfd803ccc0 0x7fcfd803eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcfd803ccc0 0x7fcfd803eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57148/0 (socket says 172.21.2.111:57148) 2021-04-03T03:16:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 -- 172.21.2.111:0/3147774123 learned_addr learned my addr 172.21.2.111:0/3147774123 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 -- 172.21.2.111:0/3147774123 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 msgr2=0x7fcfd8042360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 --2- 172.21.2.111:0/3147774123 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 0x7fcfd8042360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 -- 172.21.2.111:0/3147774123 >> v1:172.21.2.115:6789/0 conn(0x7fcfd803d860 legacy=0x7fcfd803c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:44.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 -- 172.21.2.111:0/3147774123 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8041480 con 0x7fcfd803ccc0 2021-04-03T03:16:44.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe55db700 1 --2- 172.21.2.111:0/3147774123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd803eee0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7fcfd80d4f00 tx=0x7fcfe005ad70).ready entity=mon.0 client_cookie=d299f79b2a394c51 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfdf7fe700 1 -- 172.21.2.111:0/3147774123 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe005bda0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfdf7fe700 1 -- 172.21.2.111:0/3147774123 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe005df60 con 0x7fcfd803ccc0 2021-04-03T03:16:44.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfdf7fe700 1 -- 172.21.2.111:0/3147774123 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfe00699f0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/3147774123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 msgr2=0x7fcfd803eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/3147774123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd803eee0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7fcfd80d4f00 tx=0x7fcfe005ad70).stop 2021-04-03T03:16:44.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/3147774123 shutdown_connections 2021-04-03T03:16:44.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/3147774123 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd803eee0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/3147774123 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 0x7fcfd8042360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/3147774123 >> 172.21.2.111:0/3147774123 conn(0x7fcfd80462b0 msgr2=0x7fcfd80466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:44.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/3147774123 shutdown_connections 2021-04-03T03:16:44.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/3147774123 wait complete. 2021-04-03T03:16:44.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 Processor -- start 2021-04-03T03:16:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- start start 2021-04-03T03:16:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd80e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfd803d860 0x7fcfd80dada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 0x7fcfd80d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80a3ed0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80d8ff0 con 0x7fcfd803d860 2021-04-03T03:16:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe5ddc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80d7010 con 0x7fcfd8041f80 2021-04-03T03:16:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd80e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd80e2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57150/0 (socket says 172.21.2.111:57150) 2021-04-03T03:16:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 -- 172.21.2.111:0/2399525001 learned_addr learned my addr 172.21.2.111:0/2399525001 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 -- 172.21.2.111:0/2399525001 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 msgr2=0x7fcfd80d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 0x7fcfd80d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 -- 172.21.2.111:0/2399525001 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfd803d860 msgr2=0x7fcfd80dada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfd803d860 0x7fcfd80dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 -- 172.21.2.111:0/2399525001 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80d72b0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfe4dda700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd80e2fb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7fcfd4002f30 tx=0x7fcfd4006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.133+0000 7fcfddffb700 1 -- 172.21.2.111:0/2399525001 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfd400d7c0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfddffb700 1 -- 172.21.2.111:0/2399525001 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd400d920 con 0x7fcfd803ccc0 2021-04-03T03:16:44.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd80d7550 con 0x7fcfd803ccc0 2021-04-03T03:16:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd80d7b30 con 0x7fcfd803ccc0 2021-04-03T03:16:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfddffb700 1 -- 172.21.2.111:0/2399525001 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfd400dc80 con 0x7fcfd803ccc0 2021-04-03T03:16:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd80d94a0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfddffb700 1 -- 172.21.2.111:0/2399525001 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fcfd40103c0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfddffb700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfc0057070 0x7fcfc0059530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfddffb700 1 -- 172.21.2.111:0/2399525001 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fcfd406fbf0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfdffff700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfc0057070 0x7fcfc0059530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfdffff700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfc0057070 0x7fcfc0059530 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fcfd0000f30 tx=0x7fcfd00077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.137+0000 7fcfddffb700 1 -- 172.21.2.111:0/2399525001 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fcfd4044e10 con 0x7fcfd803ccc0 2021-04-03T03:16:44.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.233+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcfd8004db0 con 0x7fcfd803ccc0 2021-04-03T03:16:44.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.237+0000 7fcfddffb700 1 -- 172.21.2.111:0/2399525001 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcfd406f020 con 0x7fcfd803ccc0 2021-04-03T03:16:44.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfc0057070 msgr2=0x7fcfc0059530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfc0057070 0x7fcfc0059530 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fcfd0000f30 tx=0x7fcfd00077d0).stop 2021-04-03T03:16:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 msgr2=0x7fcfd80e2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd80e2fb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7fcfd4002f30 tx=0x7fcfd4006e30).stop 2021-04-03T03:16:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 shutdown_connections 2021-04-03T03:16:44.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcfd8041f80 0x7fcfd80d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcfd803ccc0 0x7fcfd80e2fb0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcfc0057070 0x7fcfc0059530 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 --2- 172.21.2.111:0/2399525001 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcfd803d860 0x7fcfd80dada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 >> 172.21.2.111:0/2399525001 conn(0x7fcfd80462b0 msgr2=0x7fcfd80064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:44.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 shutdown_connections 2021-04-03T03:16:44.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.241+0000 7fcfe5ddc700 1 -- 172.21.2.111:0/2399525001 wait complete. 2021-04-03T03:16:44.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:16:44.249 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:44.256 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:44.263 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:44.275 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:44.287 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:16:44.299 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:16:44.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:44.300 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:44.300 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:44.319 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:16:44.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:44.321 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:44.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:44.322 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:44.322 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:44.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:44.333 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:16:44.333 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:44.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:44.334 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:44.335 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:16:44.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:16:44.346 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:16:44.346 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:16:44.346 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:16:44.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 Processor -- start 2021-04-03T03:16:44.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 -- start start 2021-04-03T03:16:44.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f5ff80603f0 0x7f5ff80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5ff80b0c90 0x7f5ff8147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff80af400 con 0x7f5ff814b700 2021-04-03T03:16:44.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff8060860 con 0x7f5ff80b0c90 2021-04-03T03:16:44.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80af6a0 con 0x7f5ff80603f0 2021-04-03T03:16:44.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff6b85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5ff80b0c90 0x7f5ff8147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff5b83700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5ff814b700 0x7f5ff80b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57950/0 (socket says 172.21.2.111:57950) 2021-04-03T03:16:44.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff5b83700 1 -- 172.21.2.111:0/3327368572 learned_addr learned my addr 172.21.2.111:0/3327368572 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:44.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff6b85700 1 --2- 172.21.2.111:0/3327368572 >> v2:172.21.2.111:3300/0 conn(0x7f5ff80b0c90 0x7f5ff8147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57158/0 (socket says 172.21.2.111:57158) 2021-04-03T03:16:44.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff6b85700 1 -- 172.21.2.111:0/3327368572 >> v2:172.21.2.115:3300/0 conn(0x7f5ff80603f0 msgr2=0x7f5ff80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:44.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff5382700 1 -- 172.21.2.111:0/3327368572 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299165659 0 0) 0x7f5ff80af400 con 0x7f5ff814b700 2021-04-03T03:16:44.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff6b85700 1 --2- 172.21.2.111:0/3327368572 >> v2:172.21.2.115:3300/0 conn(0x7f5ff80603f0 0x7f5ff80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff6b85700 1 -- 172.21.2.111:0/3327368572 >> v1:172.21.2.111:6789/0 conn(0x7f5ff814b700 legacy=0x7f5ff80b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff6b85700 1 -- 172.21.2.111:0/3327368572 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80afb40 con 0x7f5ff80b0c90 2021-04-03T03:16:44.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff6b85700 1 --2- 172.21.2.111:0/3327368572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8147680 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f5ff00032b0 tx=0x7f5ff00091d0).ready entity=mon.0 client_cookie=7fde6874c5bfdc3a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff5382700 1 -- 172.21.2.111:0/3327368572 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff000ba50 con 0x7f5ff80b0c90 2021-04-03T03:16:44.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff5382700 1 -- 172.21.2.111:0/3327368572 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff000bbb0 con 0x7f5ff80b0c90 2021-04-03T03:16:44.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff5382700 1 -- 172.21.2.111:0/3327368572 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff000bd20 con 0x7f5ff80b0c90 2021-04-03T03:16:44.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 -- 172.21.2.111:0/3327368572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 msgr2=0x7f5ff8147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.405+0000 7f5ff7386700 1 --2- 172.21.2.111:0/3327368572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8147680 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f5ff00032b0 tx=0x7f5ff00091d0).stop 2021-04-03T03:16:44.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- 172.21.2.111:0/3327368572 shutdown_connections 2021-04-03T03:16:44.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 --2- 172.21.2.111:0/3327368572 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8147680 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 --2- 172.21.2.111:0/3327368572 >> v2:172.21.2.115:3300/0 conn(0x7f5ff80603f0 0x7f5ff80b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- 172.21.2.111:0/3327368572 >> 172.21.2.111:0/3327368572 conn(0x7f5ff80b61c0 msgr2=0x7f5ff805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:44.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- 172.21.2.111:0/3327368572 shutdown_connections 2021-04-03T03:16:44.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- 172.21.2.111:0/3327368572 wait complete. 2021-04-03T03:16:44.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 Processor -- start 2021-04-03T03:16:44.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- start start 2021-04-03T03:16:44.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ff80603f0 0x7f5ff8064820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ff814b700 0x7f5ff81598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80640e0 con 0x7f5ff80b0c90 2021-04-03T03:16:44.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8064380 con 0x7f5ff80603f0 2021-04-03T03:16:44.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8064d10 con 0x7f5ff814b700 2021-04-03T03:16:44.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57164/0 (socket says 172.21.2.111:57164) 2021-04-03T03:16:44.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 -- 172.21.2.111:0/2902612765 learned_addr learned my addr 172.21.2.111:0/2902612765 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:44.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 -- 172.21.2.111:0/2902612765 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ff814b700 msgr2=0x7f5ff81598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ff814b700 0x7f5ff81598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 -- 172.21.2.111:0/2902612765 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ff80603f0 msgr2=0x7f5ff8064820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ff80603f0 0x7f5ff8064820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 -- 172.21.2.111:0/2902612765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80a9ef0 con 0x7f5ff80b0c90 2021-04-03T03:16:44.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff5b83700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8062bf0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f5fe0000b80 tx=0x7f5fe0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5fef7fe700 1 -- 172.21.2.111:0/2902612765 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fe00072e0 con 0x7f5ff80b0c90 2021-04-03T03:16:44.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5fef7fe700 1 -- 172.21.2.111:0/2902612765 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe0007440 con 0x7f5ff80b0c90 2021-04-03T03:16:44.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff80a9b10 con 0x7f5ff80b0c90 2021-04-03T03:16:44.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff80632f0 con 0x7f5ff80b0c90 2021-04-03T03:16:44.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5fef7fe700 1 -- 172.21.2.111:0/2902612765 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fe00057e0 con 0x7f5ff80b0c90 2021-04-03T03:16:44.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8063f10 con 0x7f5ff80b0c90 2021-04-03T03:16:44.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.409+0000 7f5fef7fe700 1 -- 172.21.2.111:0/2902612765 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5fe0010070 con 0x7f5ff80b0c90 2021-04-03T03:16:44.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.413+0000 7f5fef7fe700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5fdc056ef0 0x7f5fdc0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.413+0000 7f5fef7fe700 1 -- 172.21.2.111:0/2902612765 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f5fe006f960 con 0x7f5ff80b0c90 2021-04-03T03:16:44.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.413+0000 7f5ff6384700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5fdc056ef0 0x7f5fdc0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.413+0000 7f5ff6384700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5fdc056ef0 0x7f5fdc0593b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f5fe8002f30 tx=0x7f5fe8006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.413+0000 7f5fef7fe700 1 -- 172.21.2.111:0/2902612765 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5fe0044be0 con 0x7f5ff80b0c90 2021-04-03T03:16:44.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5ff8057660 con 0x7f5ff80b0c90 2021-04-03T03:16:44.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5fef7fe700 1 -- 172.21.2.111:0/2902612765 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v125) v1 ==== 81+0+34205 (secure 0 0 0) 0x7f5fe0048200 con 0x7f5ff80b0c90 2021-04-03T03:16:44.508 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:44.508 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:44.509 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 125, 2021-04-03T03:16:44.509 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:16:44.509 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:16:44.509 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:16:44.250824+0000", 2021-04-03T03:16:44.510 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:16:44.510 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:16:44.510 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:16:44.510 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:16:44.510 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:16:44.511 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:16:44.511 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:16:44.511 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:16:44.511 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:16:44.512 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:44.512 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:16:44.512 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:16:44.512 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:16:44.512 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:16:44.513 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:16:44.513 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 24, 2021-04-03T03:16:44.513 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:16:44.513 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:16:44.513 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:16:44.514 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:16:44.514 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:16:44.514 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.514 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:16:44.514 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:16:44.515 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:16:44.515 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:44.515 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:44.515 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:44.516 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:44.516 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:44.516 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:44.516 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:44.516 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:44.517 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:16:44.517 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:16:44.517 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:16:44.517 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:16:44.517 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:16:44.518 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:44.518 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:44.518 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:44.518 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:44.518 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:44.519 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:44.519 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:44.519 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.519 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:16:44.519 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:44.520 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:44.520 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:44.520 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:44.520 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:44.520 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:44.521 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:44.521 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:44.521 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:44.521 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:44.521 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:44.522 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:44.522 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:44.522 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:44.522 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:44.522 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:44.523 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:44.523 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:44.523 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:44.523 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:44.524 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:44.524 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:44.524 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:44.524 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:44.524 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:44.525 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:44.525 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.525 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:44.525 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:44.525 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:44.526 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:44.526 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:44.526 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:44.526 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:44.526 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:44.527 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:44.527 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:44.527 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:44.527 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:16:44.528 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:16:44.528 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.528 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:44.528 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:16:44.528 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.529 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.529 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.529 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:16:44.529 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:16:44.529 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:16:44.530 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:44.530 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:44.530 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:44.530 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:44.530 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:44.531 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:44.531 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:44.531 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:44.531 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:16:44.531 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:16:44.532 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:16:44.532 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:16:44.532 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:16:44.532 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:44.532 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:44.533 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:44.533 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:44.533 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:44.533 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:44.533 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:44.534 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.534 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:16:44.534 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:44.534 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:44.535 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:44.535 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:44.535 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:44.535 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:44.536 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:44.536 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:44.536 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:44.536 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:44.536 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:44.537 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:44.537 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:44.537 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:44.537 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:44.537 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:44.538 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:44.538 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:44.538 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:44.538 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:44.539 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:44.539 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:44.539 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:44.539 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:44.539 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:44.540 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:44.540 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.540 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:44.540 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:44.541 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:44.541 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:44.541 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:44.541 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:44.541 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:44.542 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:44.542 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:44.542 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:44.542 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:44.542 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:16:44.543 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:44.543 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:16:44.543 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.543 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.544 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.544 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 22, 2021-04-03T03:16:44.544 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:16:44.544 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:16:20.080209+0000", 2021-04-03T03:16:44.544 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:44.545 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:44.545 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:44.545 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:44.545 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:44.545 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:44.546 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:44.546 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:44.546 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:16:44.546 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:16:44.546 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:16:44.547 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:16:44.547 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:16:44.547 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:44.547 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:44.548 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:44.548 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:44.548 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:44.548 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:44.548 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:44.549 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.549 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "123", 2021-04-03T03:16:44.549 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:44.549 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:44.549 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:44.550 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:44.550 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:44.550 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:44.550 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:44.550 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:44.551 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:44.551 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:44.551 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:44.551 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:44.551 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:44.552 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:44.552 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:44.552 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:44.552 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:44.552 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:44.553 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:44.553 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:44.553 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:44.553 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:44.554 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:44.554 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:44.554 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:44.554 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:44.554 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.555 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:44.555 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:44.555 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:44.555 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:44.555 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:44.556 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:44.556 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:44.556 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:44.556 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:44.556 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:44.557 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:44.557 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:16:44.557 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:44.557 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:16:44.557 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:16:44.558 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.558 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.558 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.558 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.558 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 24, 2021-04-03T03:16:44.559 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:16:44.559 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:16:41.225539+0000", 2021-04-03T03:16:44.559 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:16:44.559 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:16:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:16:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:16:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:16:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:16:44.560 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:16:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:16:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:16:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:16:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:16:44.561 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:16:44.562 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:16:44.562 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:16:44.562 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:16:44.562 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:16:44.563 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:16:44.563 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:16:44.563 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:16:44.563 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:16:44.563 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.564 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "125", 2021-04-03T03:16:44.564 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:16:44.564 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:16:44.564 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:16:44.564 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:16:44.565 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:16:44.565 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:16:44.565 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:16:44.565 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:16:44.565 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:16:44.566 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:16:44.566 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:16:44.566 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:16:44.566 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:16:44.566 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:16:44.567 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:16:44.567 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:16:44.567 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:16:44.567 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:16:44.568 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:16:44.568 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:16:44.568 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:16:44.568 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:16:44.568 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:16:44.569 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:16:44.569 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:16:44.569 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:16:44.569 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.569 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:16:44.570 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:16:44.570 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:16:44.570 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:16:44.570 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:16:44.570 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:16:44.571 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:16:44.571 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:16:44.571 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:16:44.571 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:16:44.571 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:16:44.572 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:16:44.572 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:16:44.572 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:16:44.572 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:16:44.572 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.573 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:16:44.573 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:16:44.573 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:16:44.573 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.573 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.574 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.574 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:44.574 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:16:44.574 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.574 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:16:44.575 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:16:44.575 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.575 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.575 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.576 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.576 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.576 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.576 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.576 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 122, 2021-04-03T03:16:44.577 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.577 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.577 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.577 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.577 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.578 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.578 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:16:44.578 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.578 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.578 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.579 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.579 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:16:44.579 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.579 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.580 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.580 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.580 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.580 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.580 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.581 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.581 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:16:44.581 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.581 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.581 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.582 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.582 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:16:44.582 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.582 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.582 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.583 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.583 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.583 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.583 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.583 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.584 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:16:44.584 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.584 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.584 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.585 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.585 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:16:44.585 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.585 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.585 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.586 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.586 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.586 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.586 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.587 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.587 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:16:44.587 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.587 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.587 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.588 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.588 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:16:44.588 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:16:44.588 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.588 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.589 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.589 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:16:44.589 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:16:44.589 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:16:44.589 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:16:44.590 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.590 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.590 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.590 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:16:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:16:44.591 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.592 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.593 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.593 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 122, 2021-04-03T03:16:44.593 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.593 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.594 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.594 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.594 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.594 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.594 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:16:44.595 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.595 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.595 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.595 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.596 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:16:44.596 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.596 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.596 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.597 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.597 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.597 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.597 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.597 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.598 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:16:44.598 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.598 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.598 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.598 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.599 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:16:44.599 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.599 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.599 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.600 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.600 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.600 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.600 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.600 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:16:44.601 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.601 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.601 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.601 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.602 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:16:44.602 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.602 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.602 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.602 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.603 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.603 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.603 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.603 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.603 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:16:44.604 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.604 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.604 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.604 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.604 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:16:44.605 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:16:44.605 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.605 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.605 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.605 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:16:44.606 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:16:44.606 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:16:44.606 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:16:44.606 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.606 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.607 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.607 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.607 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.607 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.608 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:16:44.608 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:16:44.608 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.608 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.608 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.609 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.609 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.609 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.609 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.610 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 117, 2021-04-03T03:16:44.610 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.610 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.610 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.610 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.611 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.611 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.611 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:16:44.611 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.611 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.612 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.612 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.612 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:16:44.612 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.612 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.613 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.613 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.613 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.613 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.613 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.614 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.614 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:16:44.614 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.614 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.614 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.615 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.615 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:16:44.615 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.615 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.616 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.616 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.616 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.616 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.616 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.617 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.617 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:16:44.617 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.617 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.617 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.618 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.618 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:16:44.618 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.618 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.619 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.619 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.619 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.619 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.619 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.620 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.620 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:16:44.620 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.620 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.620 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.621 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.621 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:16:44.621 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:16:44.621 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.622 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.622 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.622 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:16:44.622 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:16:44.622 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:16:44.623 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:16:44.623 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.623 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.623 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.623 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.624 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.624 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.624 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:16:44.624 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:16:44.625 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.625 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.625 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.625 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.625 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.626 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.626 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.626 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 123, 2021-04-03T03:16:44.626 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.626 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.627 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.627 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.627 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.627 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.628 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:16:44.628 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.628 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.628 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.628 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.629 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:16:44.629 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.629 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.629 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.629 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.630 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.630 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.630 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.630 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.630 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:16:44.631 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.631 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.631 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.631 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.632 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:16:44.632 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.632 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.632 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.632 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.633 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.633 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.633 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.633 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.633 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:16:44.634 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.634 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.634 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.634 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.634 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:16:44.635 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.635 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.635 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.635 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.636 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.636 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.636 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.636 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.636 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:16:44.637 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.637 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.637 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.637 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.638 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:16:44.638 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:16:44.638 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.638 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.638 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.639 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:16:44.639 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:16:44.639 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:16:44.639 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:16:44.640 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.640 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.640 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.640 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.640 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.641 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.641 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:16:44.641 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:16:44.641 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.641 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.642 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.642 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.642 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.642 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.642 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.643 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 117, 2021-04-03T03:16:44.643 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.643 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.643 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.644 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.644 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.644 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.644 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:16:44.644 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.645 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.645 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.645 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.645 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:16:44.645 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.646 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.646 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.646 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.646 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.646 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.647 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.647 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.647 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:16:44.647 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.648 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.648 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.648 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.648 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:16:44.648 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.649 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.649 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.649 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.649 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.649 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.650 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.650 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.650 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:16:44.650 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.650 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.651 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.651 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.651 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:16:44.651 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.652 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.652 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.652 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.652 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.652 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.653 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.653 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.653 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:16:44.653 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.653 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.654 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.654 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.654 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:16:44.654 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:16:44.654 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.655 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.655 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.655 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:16:44.655 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:16:44.655 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:16:44.656 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:16:44.656 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.656 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.656 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.657 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.657 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.657 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.657 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:16:44.657 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:16:44.658 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.658 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.658 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.658 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.659 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.659 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.659 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.659 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 122, 2021-04-03T03:16:44.660 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.660 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.660 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.660 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.660 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.661 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.661 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:16:44.661 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.661 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.662 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.662 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.662 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:16:44.662 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.662 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.663 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.663 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.663 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.663 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.664 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.664 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.664 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:16:44.664 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.664 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.665 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.665 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.665 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:16:44.665 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.665 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.666 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.666 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.666 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.666 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.667 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.667 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.667 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:16:44.667 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.667 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.668 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.668 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.668 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:16:44.668 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.668 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.669 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.669 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.669 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.669 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.670 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.670 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.670 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:16:44.670 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.670 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.671 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.671 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.671 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:16:44.671 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:16:44.671 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:16:44.672 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:16:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:16:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:16:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.673 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:16:44.674 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:16:44.675 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.675 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.675 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.675 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.675 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.676 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.676 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.676 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 123, 2021-04-03T03:16:44.676 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.676 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.677 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.677 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.677 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.677 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.678 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:16:44.678 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.678 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.678 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.678 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.679 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:16:44.679 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.679 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.679 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.679 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.680 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.680 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.680 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.680 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.680 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:16:44.681 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.681 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.681 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.681 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.682 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:16:44.682 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.682 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.682 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.682 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.683 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.683 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.683 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.683 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.683 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:16:44.684 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.684 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.684 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.684 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.685 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:16:44.685 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.685 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.685 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.685 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.686 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.686 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.686 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.686 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.686 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:16:44.687 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.687 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.687 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.687 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.688 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:16:44.688 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:16:44.688 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.688 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.688 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.689 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:16:44.689 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:16:44.689 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:16:44.689 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:16:44.689 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.690 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.690 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.690 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.690 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.690 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.691 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:16:44.691 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:16:44.691 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:16:44.691 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:16:44.692 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:16:44.692 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:16:44.692 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:16:44.692 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:16:44.692 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:16:44.693 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 123, 2021-04-03T03:16:44.693 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:16:44.693 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:16:44.693 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:16:44.693 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.694 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.694 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.694 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:16:44.694 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.694 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.695 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.695 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.695 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:16:44.695 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.696 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.696 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.696 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.696 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:16:44.696 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.697 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.697 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.697 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:16:44.697 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.697 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.698 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.698 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.698 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:16:44.698 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.698 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.699 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.699 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.699 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:16:44.699 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.700 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.700 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.700 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:16:44.700 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.700 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.701 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.701 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.701 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:16:44.701 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.701 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.702 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.702 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.702 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:16:44.702 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:16:44.702 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.703 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:16:44.703 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:16:44.703 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.703 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.704 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.704 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:16:44.704 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:16:44.704 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:16:44.704 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.705 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.705 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.705 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:16:44.705 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:16:44.705 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:16:44.706 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:16:44.706 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:16:44.706 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:16:44.706 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:16:44.706 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:44.707 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.707 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:44.707 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:16:44.707 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.708 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:16:44.708 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.708 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.708 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.708 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.709 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.709 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:16:44.709 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.709 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.709 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.710 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:16:44.710 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.710 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.710 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.710 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.711 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.711 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:16:44.711 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.711 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.712 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.712 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:16:44.712 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.712 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.712 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.713 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.713 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.713 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:16:44.713 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.713 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.714 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.714 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:16:44.714 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.714 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.715 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.715 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.715 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.715 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:16:44.715 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.716 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.716 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.716 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:16:44.716 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.716 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.717 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.717 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.717 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.717 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:16:44.717 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.718 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.718 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.718 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:16:44.718 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.718 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.719 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.719 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.719 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.719 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:16:44.720 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.720 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.720 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.720 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:16:44.720 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.721 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.721 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.721 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.721 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.721 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:16:44.722 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.722 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.722 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:44.722 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:16:44.722 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:16:44.723 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:16:44.723 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:16:44.723 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:16:44.723 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:16:44.724 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:16:44.724 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:16:44.724 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.724 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:16:44.724 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:16:44.725 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:16:44.725 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:16:44.725 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:16:44.725 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": { 2021-04-03T03:16:44.726 INFO:teuthology.orchestra.run.gibba011.stdout: "172.21.2.111:6835/1395173601": "2021-04-04T03:16:28.094805+0000", 2021-04-03T03:16:44.726 INFO:teuthology.orchestra.run.gibba011.stdout: "172.21.2.111:6834/1395173601": "2021-04-04T03:16:28.094805+0000" 2021-04-03T03:16:44.726 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.726 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:16:44.726 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:16:44.727 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:16:44.727 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:16:44.727 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:16:44.727 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:16:44.727 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:16:44.728 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:16:44.728 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:16:44.728 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:16:44.728 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:16:44.728 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:16:44.729 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.729 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:16:44.729 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:16:44.729 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:16:44.730 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:16:44.730 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:16:44.730 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:16:44.730 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:16:44.730 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:44.731 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:44.731 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:16:44.731 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:16:44.731 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:16:44.731 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:16:44.732 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:16:44.732 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:44.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5fdc056ef0 msgr2=0x7f5fdc0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5fdc056ef0 0x7f5fdc0593b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f5fe8002f30 tx=0x7f5fe8006a60).stop 2021-04-03T03:16:44.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 msgr2=0x7f5ff8062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8062bf0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f5fe0000b80 tx=0x7f5fe0007c10).stop 2021-04-03T03:16:44.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 shutdown_connections 2021-04-03T03:16:44.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5ff814b700 0x7f5ff81598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5fdc056ef0 0x7f5fdc0593b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5ff80603f0 0x7f5ff8064820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 --2- 172.21.2.111:0/2902612765 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5ff80b0c90 0x7f5ff8062bf0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 >> 172.21.2.111:0/2902612765 conn(0x7f5ff80b61c0 msgr2=0x7f5ff80ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:44.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 shutdown_connections 2021-04-03T03:16:44.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.505+0000 7f5ff7386700 1 -- 172.21.2.111:0/2902612765 wait complete. 2021-04-03T03:16:44.736 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:44.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.809+0000 7f8a525e5700 1 Processor -- start 2021-04-03T03:16:44.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- start start 2021-04-03T03:16:44.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0b5d60 0x7f8a4c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c0b4bc0 con 0x7f8a4c0603f0 2021-04-03T03:16:44.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c0607d0 con 0x7f8a4c0b44a0 2021-04-03T03:16:44.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0b4df0 con 0x7f8a4c0b5d60 2021-04-03T03:16:44.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a515e3700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8a4c0603f0 0x7f8a4c0b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:57962/0 (socket says 172.21.2.111:57962) 2021-04-03T03:16:44.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a515e3700 1 -- 172.21.2.111:0/4283850658 learned_addr learned my addr 172.21.2.111:0/4283850658 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:44.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740575274 0 0) 0x7f8a4c0b4bc0 con 0x7f8a4c0603f0 2021-04-03T03:16:44.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a34003240 con 0x7f8a4c0603f0 2021-04-03T03:16:44.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4248676788 0 0) 0x7f8a34003240 con 0x7f8a4c0603f0 2021-04-03T03:16:44.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0b5d60 msgr2=0x7f8a4c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:44.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0b5d60 0x7f8a4c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 >> v1:172.21.2.115:6789/0 conn(0x7f8a4c0b44a0 legacy=0x7f8a4c142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:44.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0b5090 con 0x7f8a4c0603f0 2021-04-03T03:16:44.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8a480016d0 con 0x7f8a4c0603f0 2021-04-03T03:16:44.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 >> v1:172.21.2.111:6789/0 conn(0x7f8a4c0603f0 legacy=0x7f8a4c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a34003760 0x7f8a34003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0b5d60 0x7f8a340044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a340049d0 0x7f8a34004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0b4df0 con 0x7f8a340049d0 2021-04-03T03:16:44.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340041e0 con 0x7f8a34003760 2021-04-03T03:16:44.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3400b730 con 0x7f8a4c0b5d60 2021-04-03T03:16:44.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a340049d0 0x7f8a34004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 -- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0b5d60 msgr2=0x7f8a340044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:44.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0b5d60 0x7f8a340044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 -- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a34003760 msgr2=0x7f8a34003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:44.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a34003760 0x7f8a34003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 -- 172.21.2.111:0/4283850658 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34003240 con 0x7f8a340049d0 2021-04-03T03:16:44.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a340049d0 0x7f8a34004e50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f8a44002330 tx=0x7f8a44007f40).ready entity=mon.0 client_cookie=58c2efaf690178e0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4400c940 con 0x7f8a340049d0 2021-04-03T03:16:44.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4400caa0 con 0x7f8a340049d0 2021-04-03T03:16:44.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a3bfff700 1 -- 172.21.2.111:0/4283850658 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4400ce00 con 0x7f8a340049d0 2021-04-03T03:16:44.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- 172.21.2.111:0/4283850658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a340049d0 msgr2=0x7f8a34004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 --2- 172.21.2.111:0/4283850658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a340049d0 0x7f8a34004e50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f8a44002330 tx=0x7f8a44007f40).stop 2021-04-03T03:16:44.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a51de4700 1 -- 172.21.2.111:0/4283850658 reap_dead start 2021-04-03T03:16:44.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- 172.21.2.111:0/4283850658 shutdown_connections 2021-04-03T03:16:44.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- 172.21.2.111:0/4283850658 >> 172.21.2.111:0/4283850658 conn(0x7f8a4c05c7d0 msgr2=0x7f8a4c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:44.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- 172.21.2.111:0/4283850658 shutdown_connections 2021-04-03T03:16:44.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- 172.21.2.111:0/4283850658 wait complete. 2021-04-03T03:16:44.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 Processor -- start 2021-04-03T03:16:44.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 -- start start 2021-04-03T03:16:44.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0aef50 0x7f8a4c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.813+0000 7f8a525e5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a4c0af590 0x7f8a4c155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a525e5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a4c0ae580 0x7f8a4c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a525e5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c156270 con 0x7f8a4c0af590 2021-04-03T03:16:44.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a525e5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0ad1b0 con 0x7f8a4c0ae580 2021-04-03T03:16:44.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a525e5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0ad880 con 0x7f8a4c0aef50 2021-04-03T03:16:44.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a4c0af590 0x7f8a4c155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a4c0af590 0x7f8a4c155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57184/0 (socket says 172.21.2.111:57184) 2021-04-03T03:16:44.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 -- 172.21.2.111:0/3449355310 learned_addr learned my addr 172.21.2.111:0/3449355310 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:44.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 -- 172.21.2.111:0/3449355310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0aef50 msgr2=0x7f8a4c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0aef50 0x7f8a4c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 -- 172.21.2.111:0/3449355310 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a4c0ae580 msgr2=0x7f8a4c155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a4c0ae580 0x7f8a4c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 -- 172.21.2.111:0/3449355310 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0af330 con 0x7f8a4c0af590 2021-04-03T03:16:44.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a50de2700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a4c0af590 0x7f8a4c155890 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f8a3c002f30 tx=0x7f8a3c006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a3a7fc700 1 -- 172.21.2.111:0/3449355310 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00d7e0 con 0x7f8a4c0af590 2021-04-03T03:16:44.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a3a7fc700 1 -- 172.21.2.111:0/3449355310 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c00d940 con 0x7f8a4c0af590 2021-04-03T03:16:44.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a3a7fc700 1 -- 172.21.2.111:0/3449355310 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00dce0 con 0x7f8a4c0af590 2021-04-03T03:16:44.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c113930 con 0x7f8a4c0af590 2021-04-03T03:16:44.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c0ad3e0 con 0x7f8a4c0af590 2021-04-03T03:16:44.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c0ad5a0 con 0x7f8a4c0af590 2021-04-03T03:16:44.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a3a7fc700 1 -- 172.21.2.111:0/3449355310 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8a3c011480 con 0x7f8a4c0af590 2021-04-03T03:16:44.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a3a7fc700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a2c056f60 0x7f8a2c059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a3a7fc700 1 -- 172.21.2.111:0/3449355310 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f8a3c070030 con 0x7f8a4c0af590 2021-04-03T03:16:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a515e3700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a2c056f60 0x7f8a2c059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.817+0000 7f8a515e3700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a2c056f60 0x7f8a2c059420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f8a4c0b78f0 tx=0x7f8a4c0b22f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:44.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.821+0000 7f8a3a7fc700 1 -- 172.21.2.111:0/3449355310 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8a3c045250 con 0x7f8a4c0af590 2021-04-03T03:16:44.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.945+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a4c069270 con 0x7f8a4c0af590 2021-04-03T03:16:44.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.945+0000 7f8a3a7fc700 1 -- 172.21.2.111:0/3449355310 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f8a3c048870 con 0x7f8a4c0af590 2021-04-03T03:16:44.948 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:44.949 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":53,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:42.245973+0000","modified":"2021-04-03T03:16:42.246018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-04-03T03:16:44.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.945+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a2c056f60 msgr2=0x7f8a2c059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.945+0000 7f8a525e5700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a2c056f60 0x7f8a2c059420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f8a4c0b78f0 tx=0x7f8a4c0b22f0).stop 2021-04-03T03:16:44.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.945+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a4c0af590 msgr2=0x7f8a4c155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:44.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.945+0000 7f8a525e5700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a4c0af590 0x7f8a4c155890 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f8a3c002f30 tx=0x7f8a3c006e50).stop 2021-04-03T03:16:44.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 shutdown_connections 2021-04-03T03:16:44.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a2c056f60 0x7f8a2c059420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a4c0ae580 0x7f8a4c155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a4c0aef50 0x7f8a4c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:44.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 --2- 172.21.2.111:0/3449355310 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a4c0af590 0x7f8a4c155890 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:44.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 >> 172.21.2.111:0/3449355310 conn(0x7f8a4c05c7d0 msgr2=0x7f8a4c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:44.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 shutdown_connections 2021-04-03T03:16:44.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:44.949+0000 7f8a525e5700 1 -- 172.21.2.111:0/3449355310 wait complete. 2021-04-03T03:16:44.952 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 53 2021-04-03T03:16:44.956 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:16:45.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.021+0000 7feb850ee700 1 Processor -- start 2021-04-03T03:16:45.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.021+0000 7feb850ee700 1 -- start start 2021-04-03T03:16:45.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.021+0000 7feb850ee700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7feb78005640 0x7feb780d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:45.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.021+0000 7feb850ee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb780d8e90 0x7feb780e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:45.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.021+0000 7feb850ee700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7803d840 con 0x7feb7803f120 2021-04-03T03:16:45.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.021+0000 7feb850ee700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7803faa0 con 0x7feb78005640 2021-04-03T03:16:45.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.021+0000 7feb850ee700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7803dca0 con 0x7feb780d8e90 2021-04-03T03:16:45.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb848ed700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb780d8e90 0x7feb780e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:45.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7f7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7feb78005640 0x7feb780d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:45.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7ffff700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7feb7803f120 0x7feb78041fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:42288/0 (socket says 172.21.2.111:42288) 2021-04-03T03:16:45.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7ffff700 1 -- 172.21.2.111:0/3027254283 learned_addr learned my addr 172.21.2.111:0/3027254283 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:45.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7effd700 1 -- 172.21.2.111:0/3027254283 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79173829 0 0) 0x7feb7803d840 con 0x7feb7803f120 2021-04-03T03:16:45.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7effd700 1 -- 172.21.2.111:0/3027254283 --> v1:172.21.2.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb6c003240 con 0x7feb7803f120 2021-04-03T03:16:45.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb848ed700 1 -- 172.21.2.111:0/3027254283 >> v2:172.21.2.115:3300/0 conn(0x7feb78005640 msgr2=0x7feb780d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:45.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb848ed700 1 --2- 172.21.2.111:0/3027254283 >> v2:172.21.2.115:3300/0 conn(0x7feb78005640 0x7feb780d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:45.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb848ed700 1 -- 172.21.2.111:0/3027254283 >> v1:172.21.2.115:6789/0 conn(0x7feb7803f120 legacy=0x7feb78041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:45.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb848ed700 1 -- 172.21.2.111:0/3027254283 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7803f800 con 0x7feb780d8e90 2021-04-03T03:16:45.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7f7fe700 1 --2- 172.21.2.111:0/3027254283 >> v2:172.21.2.115:3300/0 conn(0x7feb78005640 0x7feb780d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:16:45.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb848ed700 1 --2- 172.21.2.111:0/3027254283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb780d8e90 0x7feb780e2bc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7feb8006b5d0 tx=0x7feb8005ad70).ready entity=mon.2 client_cookie=dcfd7f391964a64b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:45.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7effd700 1 -- 172.21.2.111:0/3027254283 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8005bda0 con 0x7feb780d8e90 2021-04-03T03:16:45.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7effd700 1 -- 172.21.2.111:0/3027254283 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb80026620 con 0x7feb780d8e90 2021-04-03T03:16:45.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb7effd700 1 -- 172.21.2.111:0/3027254283 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8005df60 con 0x7feb780d8e90 2021-04-03T03:16:45.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- 172.21.2.111:0/3027254283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb780d8e90 msgr2=0x7feb780e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:45.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 --2- 172.21.2.111:0/3027254283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb780d8e90 0x7feb780e2bc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7feb8006b5d0 tx=0x7feb8005ad70).stop 2021-04-03T03:16:45.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- 172.21.2.111:0/3027254283 shutdown_connections 2021-04-03T03:16:45.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 --2- 172.21.2.111:0/3027254283 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb780d8e90 0x7feb780e2bc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:45.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 --2- 172.21.2.111:0/3027254283 >> v2:172.21.2.115:3300/0 conn(0x7feb78005640 0x7feb780d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:45.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- 172.21.2.111:0/3027254283 >> 172.21.2.111:0/3027254283 conn(0x7feb780462b0 msgr2=0x7feb780466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:45.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- 172.21.2.111:0/3027254283 shutdown_connections 2021-04-03T03:16:45.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- 172.21.2.111:0/3027254283 wait complete. 2021-04-03T03:16:45.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 Processor -- start 2021-04-03T03:16:45.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- start start 2021-04-03T03:16:45.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb78005640 0x7feb780e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:45.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb7803f120 0x7feb780dae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:45.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb780d8e90 0x7feb7809f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:45.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb78039f90 con 0x7feb780d8e90 2021-04-03T03:16:45.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7803a1c0 con 0x7feb78005640 2021-04-03T03:16:45.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.025+0000 7feb850ee700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7803a460 con 0x7feb7803f120 2021-04-03T03:16:45.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb780d8e90 0x7feb7809f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:45.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb780d8e90 0x7feb7809f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57200/0 (socket says 172.21.2.111:57200) 2021-04-03T03:16:45.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 -- 172.21.2.111:0/1156268483 learned_addr learned my addr 172.21.2.111:0/1156268483 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:45.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7ffff700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb78005640 0x7feb780e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:45.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7f7fe700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb7803f120 0x7feb780dae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:45.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 -- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb7803f120 msgr2=0x7feb780dae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:45.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb7803f120 0x7feb780dae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:45.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 -- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb78005640 msgr2=0x7feb780e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:45.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb78005640 0x7feb780e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:45.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 -- 172.21.2.111:0/1156268483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7803c510 con 0x7feb780d8e90 2021-04-03T03:16:45.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb848ed700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb780d8e90 0x7feb7809f6a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7feb8006b5d0 tx=0x7feb80064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:45.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7d7fa700 1 -- 172.21.2.111:0/1156268483 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb800699f0 con 0x7feb780d8e90 2021-04-03T03:16:45.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7d7fa700 1 -- 172.21.2.111:0/1156268483 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8005df60 con 0x7feb780d8e90 2021-04-03T03:16:45.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7803b350 con 0x7feb780d8e90 2021-04-03T03:16:45.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7d7fa700 1 -- 172.21.2.111:0/1156268483 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8005f2c0 con 0x7feb780d8e90 2021-04-03T03:16:45.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7803bab0 con 0x7feb780d8e90 2021-04-03T03:16:45.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb78004d30 con 0x7feb780d8e90 2021-04-03T03:16:45.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7d7fa700 1 -- 172.21.2.111:0/1156268483 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7feb80098060 con 0x7feb780d8e90 2021-04-03T03:16:45.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7d7fa700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb600570a0 0x7feb60059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:45.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7d7fa700 1 -- 172.21.2.111:0/1156268483 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6502+0+0 (secure 0 0 0) 0x7feb800e0a00 con 0x7feb780d8e90 2021-04-03T03:16:45.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7ffff700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb600570a0 0x7feb60059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:45.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.029+0000 7feb7ffff700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb600570a0 0x7feb60059560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7feb7400b000 tx=0x7feb74009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:45.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.033+0000 7feb7d7fa700 1 -- 172.21.2.111:0/1156268483 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7feb800b5d30 con 0x7feb780d8e90 2021-04-03T03:16:45.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:45.157+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feb78040670 con 0x7feb780d8e90 2021-04-03T03:16:46.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb7d7fa700 1 -- 172.21.2.111:0/1156268483 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7feb800b9350 con 0x7feb780d8e90 2021-04-03T03:16:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb600570a0 msgr2=0x7feb60059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb600570a0 0x7feb60059560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7feb7400b000 tx=0x7feb74009040).stop 2021-04-03T03:16:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb780d8e90 msgr2=0x7feb7809f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb780d8e90 0x7feb7809f6a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7feb8006b5d0 tx=0x7feb80064030).stop 2021-04-03T03:16:46.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 shutdown_connections 2021-04-03T03:16:46.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7feb780d8e90 0x7feb7809f6a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb600570a0 0x7feb60059560 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7feb78005640 0x7feb780e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 --2- 172.21.2.111:0/1156268483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7feb7803f120 0x7feb780dae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 >> 172.21.2.111:0/1156268483 conn(0x7feb780462b0 msgr2=0x7feb78044580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 shutdown_connections 2021-04-03T03:16:46.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.069+0000 7feb850ee700 1 -- 172.21.2.111:0/1156268483 wait complete. 2021-04-03T03:16:46.074 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:16:46.078 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:46.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 Processor -- start 2021-04-03T03:16:46.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- start start 2021-04-03T03:16:46.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbde00dbb90 0x7fbde0042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde003d360 0x7fbde00dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde00dc6e0 con 0x7fbde00064b0 2021-04-03T03:16:46.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde00d9560 con 0x7fbde00dbb90 2021-04-03T03:16:46.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde003dcb0 con 0x7fbde003d360 2021-04-03T03:16:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdee574700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fbde00064b0 0x7fbde00056b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:42302/0 (socket says 172.21.2.111:42302) 2021-04-03T03:16:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbded572700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde003d360 0x7fbde00dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdee574700 1 -- 172.21.2.111:0/1671489553 learned_addr learned my addr 172.21.2.111:0/1671489553 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdedd73700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fbde00dbb90 0x7fbde0042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdedd73700 1 -- 172.21.2.111:0/1671489553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde003d360 msgr2=0x7fbde00dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdedd73700 1 --2- 172.21.2.111:0/1671489553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde003d360 0x7fbde00dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdedd73700 1 -- 172.21.2.111:0/1671489553 >> v1:172.21.2.115:6789/0 conn(0x7fbde00064b0 legacy=0x7fbde00056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdedd73700 1 -- 172.21.2.111:0/1671489553 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00dc460 con 0x7fbde00dbb90 2021-04-03T03:16:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdedd73700 1 --2- 172.21.2.111:0/1671489553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde00dbb90 0x7fbde0042750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fbde4002f30 tx=0x7fbde4006dd0).ready entity=mon.1 client_cookie=3fd26c9e20694615 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdecd71700 1 -- 172.21.2.111:0/1671489553 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde4009760 con 0x7fbde00dbb90 2021-04-03T03:16:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdecd71700 1 -- 172.21.2.111:0/1671489553 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde40098c0 con 0x7fbde00dbb90 2021-04-03T03:16:46.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdecd71700 1 -- 172.21.2.111:0/1671489553 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde4009c20 con 0x7fbde00dbb90 2021-04-03T03:16:46.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- 172.21.2.111:0/1671489553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde00dbb90 msgr2=0x7fbde0042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 --2- 172.21.2.111:0/1671489553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde00dbb90 0x7fbde0042750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fbde4002f30 tx=0x7fbde4006dd0).stop 2021-04-03T03:16:46.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- 172.21.2.111:0/1671489553 shutdown_connections 2021-04-03T03:16:46.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 --2- 172.21.2.111:0/1671489553 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde00dbb90 0x7fbde0042750 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 --2- 172.21.2.111:0/1671489553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde003d360 0x7fbde00dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- 172.21.2.111:0/1671489553 >> 172.21.2.111:0/1671489553 conn(0x7fbde00462c0 msgr2=0x7fbde00466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:46.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- 172.21.2.111:0/1671489553 shutdown_connections 2021-04-03T03:16:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.133+0000 7fbdeed75700 1 -- 172.21.2.111:0/1671489553 wait complete. 2021-04-03T03:16:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 Processor -- start 2021-04-03T03:16:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 -- start start 2021-04-03T03:16:46.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde00064b0 0x7fbde009f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde003d360 0x7fbde00ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde00dbb90 0x7fbde00eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0039fa0 con 0x7fbde00dbb90 2021-04-03T03:16:46.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde003b610 con 0x7fbde003d360 2021-04-03T03:16:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde003b8b0 con 0x7fbde00064b0 2021-04-03T03:16:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde00dbb90 0x7fbde00eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde00dbb90 0x7fbde00eacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57214/0 (socket says 172.21.2.111:57214) 2021-04-03T03:16:46.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 -- 172.21.2.111:0/3188136507 learned_addr learned my addr 172.21.2.111:0/3188136507 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbded572700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde003d360 0x7fbde00ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdedd73700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde00064b0 0x7fbde009f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 -- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde00064b0 msgr2=0x7fbde009f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde00064b0 0x7fbde009f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 -- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde003d360 msgr2=0x7fbde00ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde003d360 0x7fbde00ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 -- 172.21.2.111:0/3188136507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde003a1d0 con 0x7fbde00dbb90 2021-04-03T03:16:46.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdee574700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde00dbb90 0x7fbde00eacb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fbde0047210 tx=0x7fbde805ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdd6ffd700 1 -- 172.21.2.111:0/3188136507 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde805df60 con 0x7fbde00dbb90 2021-04-03T03:16:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdd6ffd700 1 -- 172.21.2.111:0/3188136507 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde80699f0 con 0x7fbde00dbb90 2021-04-03T03:16:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde003bbe0 con 0x7fbde00dbb90 2021-04-03T03:16:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdd6ffd700 1 -- 172.21.2.111:0/3188136507 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde806b5d0 con 0x7fbde00dbb90 2021-04-03T03:16:46.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde003ad20 con 0x7fbde00dbb90 2021-04-03T03:16:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde0004da0 con 0x7fbde00dbb90 2021-04-03T03:16:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdd6ffd700 1 -- 172.21.2.111:0/3188136507 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbde8061110 con 0x7fbde00dbb90 2021-04-03T03:16:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdd6ffd700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc80570a0 0x7fbdc8059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdd6ffd700 1 -- 172.21.2.111:0/3188136507 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fbde80df1a0 con 0x7fbde00dbb90 2021-04-03T03:16:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdedd73700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc80570a0 0x7fbdc8059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.137+0000 7fbdedd73700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc80570a0 0x7fbdc8059560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fbde401b000 tx=0x7fbde4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:46.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.141+0000 7fbdd6ffd700 1 -- 172.21.2.111:0/3188136507 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbde80b4530 con 0x7fbde00dbb90 2021-04-03T03:16:46.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.265+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbde0046ff0 con 0x7fbde00dbb90 2021-04-03T03:16:46.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.265+0000 7fbdd6ffd700 1 -- 172.21.2.111:0/3188136507 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fbde80b7b50 con 0x7fbde00dbb90 2021-04-03T03:16:46.269 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:46.269 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":54,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":54,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:42.245973+0000","modified":"2021-04-03T03:16:46.058198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-04-03T03:16:46.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc80570a0 msgr2=0x7fbdc8059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc80570a0 0x7fbdc8059560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fbde401b000 tx=0x7fbde4015000).stop 2021-04-03T03:16:46.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde00dbb90 msgr2=0x7fbde00eacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde00dbb90 0x7fbde00eacb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fbde0047210 tx=0x7fbde805ad70).stop 2021-04-03T03:16:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 shutdown_connections 2021-04-03T03:16:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde00dbb90 0x7fbde00eacb0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde00064b0 0x7fbde009f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc80570a0 0x7fbdc8059560 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 --2- 172.21.2.111:0/3188136507 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde003d360 0x7fbde00ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 >> 172.21.2.111:0/3188136507 conn(0x7fbde00462c0 msgr2=0x7fbde0041610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:46.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 shutdown_connections 2021-04-03T03:16:46.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.269+0000 7fbdeed75700 1 -- 172.21.2.111:0/3188136507 wait complete. 2021-04-03T03:16:46.273 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 54 2021-04-03T03:16:46.277 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:16:46.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 Processor -- start 2021-04-03T03:16:46.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- start start 2021-04-03T03:16:46.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40add50 0x7fbde4147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbde4144ef0 0x7fbde4147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde40607d0 con 0x7fbde40603f0 2021-04-03T03:16:46.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde414b090 con 0x7fbde4144ef0 2021-04-03T03:16:46.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde414b330 con 0x7fbde40add50 2021-04-03T03:16:46.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbde4144ef0 0x7fbde4147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fbde4144ef0 0x7fbde4147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57222/0 (socket says 172.21.2.111:57222) 2021-04-03T03:16:46.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 -- 172.21.2.111:0/2646124908 learned_addr learned my addr 172.21.2.111:0/2646124908 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:46.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 -- 172.21.2.111:0/2646124908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40add50 msgr2=0x7fbde4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:46.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 --2- 172.21.2.111:0/2646124908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40add50 0x7fbde4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:46.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 -- 172.21.2.111:0/2646124908 >> v1:172.21.2.111:6789/0 conn(0x7fbde40603f0 legacy=0x7fbde41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 -- 172.21.2.111:0/2646124908 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde414b5d0 con 0x7fbde4144ef0 2021-04-03T03:16:46.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde37fe700 1 --2- 172.21.2.111:0/2646124908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde4144ef0 0x7fbde4147c30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7fbdd40010d0 tx=0x7fbddc009270).ready entity=mon.0 client_cookie=e8d0b82b1a96fe60 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:46.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde1ffb700 1 -- 172.21.2.111:0/2646124908 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbddc00bac0 con 0x7fbde4144ef0 2021-04-03T03:16:46.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde1ffb700 1 -- 172.21.2.111:0/2646124908 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc00bc20 con 0x7fbde4144ef0 2021-04-03T03:16:46.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde1ffb700 1 -- 172.21.2.111:0/2646124908 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbddc00bd90 con 0x7fbde4144ef0 2021-04-03T03:16:46.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- 172.21.2.111:0/2646124908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde4144ef0 msgr2=0x7fbde4147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:46.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2646124908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde4144ef0 0x7fbde4147c30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7fbdd40010d0 tx=0x7fbddc009270).stop 2021-04-03T03:16:46.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- 172.21.2.111:0/2646124908 shutdown_connections 2021-04-03T03:16:46.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2646124908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde4144ef0 0x7fbde4147c30 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:46.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2646124908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40add50 0x7fbde4147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:46.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- 172.21.2.111:0/2646124908 >> 172.21.2.111:0/2646124908 conn(0x7fbde40b61c0 msgr2=0x7fbde405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:46.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- 172.21.2.111:0/2646124908 shutdown_connections 2021-04-03T03:16:46.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- 172.21.2.111:0/2646124908 wait complete. 2021-04-03T03:16:46.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 Processor -- start 2021-04-03T03:16:46.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- start start 2021-04-03T03:16:46.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40603f0 0x7fbde40a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde40add50 0x7fbde40aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde4144ef0 0x7fbde40ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40a9380 con 0x7fbde40add50 2021-04-03T03:16:46.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40ab710 con 0x7fbde4144ef0 2021-04-03T03:16:46.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40a72e0 con 0x7fbde40603f0 2021-04-03T03:16:46.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde40add50 0x7fbde40aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde40add50 0x7fbde40aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57224/0 (socket says 172.21.2.111:57224) 2021-04-03T03:16:46.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.333+0000 7fbde27fc700 1 -- 172.21.2.111:0/2981577900 learned_addr learned my addr 172.21.2.111:0/2981577900 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:46.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde27fc700 1 -- 172.21.2.111:0/2981577900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40603f0 msgr2=0x7fbde40a80b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:46.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde27fc700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40603f0 0x7fbde40a80b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:46.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde27fc700 1 -- 172.21.2.111:0/2981577900 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde4144ef0 msgr2=0x7fbde40ab220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:46.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde27fc700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde4144ef0 0x7fbde40ab220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:46.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde27fc700 1 -- 172.21.2.111:0/2981577900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40a74a0 con 0x7fbde40add50 2021-04-03T03:16:46.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde27fc700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde40add50 0x7fbde40aad30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fbdcc002f30 tx=0x7fbdcc006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:46.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbddbfff700 1 -- 172.21.2.111:0/2981577900 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdcc00d710 con 0x7fbde40add50 2021-04-03T03:16:46.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbddbfff700 1 -- 172.21.2.111:0/2981577900 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc000e90 con 0x7fbde40add50 2021-04-03T03:16:46.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde40a6f20 con 0x7fbde40add50 2021-04-03T03:16:46.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbddbfff700 1 -- 172.21.2.111:0/2981577900 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdcc00da60 con 0x7fbde40add50 2021-04-03T03:16:46.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde40abb50 con 0x7fbde40add50 2021-04-03T03:16:46.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde4057660 con 0x7fbde40add50 2021-04-03T03:16:46.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbddbfff700 1 -- 172.21.2.111:0/2981577900 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fbdcc00dbc0 con 0x7fbde40add50 2021-04-03T03:16:46.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbddbfff700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc8056fb0 0x7fbdc8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:46.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbddbfff700 1 -- 172.21.2.111:0/2981577900 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fbdcc06feb0 con 0x7fbde40add50 2021-04-03T03:16:46.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde2ffd700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc8056fb0 0x7fbdc8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:46.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.337+0000 7fbde2ffd700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc8056fb0 0x7fbdc8059470 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fbdd4001190 tx=0x7fbdd4007730).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:46.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.341+0000 7fbddbfff700 1 -- 172.21.2.111:0/2981577900 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fbdcc0450d0 con 0x7fbde40add50 2021-04-03T03:16:46.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:46.465+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbde40a9550 con 0x7fbde40add50 2021-04-03T03:16:47.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.085+0000 7fbddbfff700 1 -- 172.21.2.111:0/2981577900 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdcc0486f0 con 0x7fbde40add50 2021-04-03T03:16:47.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc8056fb0 msgr2=0x7fbdc8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:47.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc8056fb0 0x7fbdc8059470 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fbdd4001190 tx=0x7fbdd4007730).stop 2021-04-03T03:16:47.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde40add50 msgr2=0x7fbde40aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:47.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde40add50 0x7fbde40aad30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fbdcc002f30 tx=0x7fbdcc006e70).stop 2021-04-03T03:16:47.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 shutdown_connections 2021-04-03T03:16:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fbdc8056fb0 0x7fbdc8059470 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fbde4144ef0 0x7fbde40ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fbde40603f0 0x7fbde40a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:47.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 --2- 172.21.2.111:0/2981577900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fbde40add50 0x7fbde40aad30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 >> 172.21.2.111:0/2981577900 conn(0x7fbde40b61c0 msgr2=0x7fbde405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 shutdown_connections 2021-04-03T03:16:47.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.089+0000 7fbde3fff700 1 -- 172.21.2.111:0/2981577900 wait complete. 2021-04-03T03:16:47.097 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:16:47.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 Processor -- start 2021-04-03T03:16:47.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- start start 2021-04-03T03:16:47.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0cb4047580 0x7f0cb403bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:47.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0cb40423a0 0x7f0cb403c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:47.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb4043400 con 0x7f0cb40d38b0 2021-04-03T03:16:47.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb4047960 con 0x7f0cb4047580 2021-04-03T03:16:47.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb4042ec0 con 0x7f0cb40423a0 2021-04-03T03:16:47.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0cb4047580 0x7f0cb403bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:47.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0cb4047580 0x7f0cb403bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57232/0 (socket says 172.21.2.111:57232) 2021-04-03T03:16:47.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 -- 172.21.2.111:0/2739926083 learned_addr learned my addr 172.21.2.111:0/2739926083 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:47.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 -- 172.21.2.111:0/2739926083 >> v2:172.21.2.115:3300/0 conn(0x7f0cb40423a0 msgr2=0x7f0cb403c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:47.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 --2- 172.21.2.111:0/2739926083 >> v2:172.21.2.115:3300/0 conn(0x7f0cb40423a0 0x7f0cb403c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:47.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 -- 172.21.2.111:0/2739926083 >> v1:172.21.2.111:6789/0 conn(0x7f0cb40d38b0 legacy=0x7f0cb40cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:47.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 -- 172.21.2.111:0/2739926083 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4043160 con 0x7f0cb4047580 2021-04-03T03:16:47.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbbfff700 1 --2- 172.21.2.111:0/2739926083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb403bf30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f0cb40d0310 tx=0x7f0cb0006e90).ready entity=mon.0 client_cookie=4455211150888e9a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:47.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbaffd700 1 -- 172.21.2.111:0/2739926083 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cb0009980 con 0x7f0cb4047580 2021-04-03T03:16:47.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbaffd700 1 -- 172.21.2.111:0/2739926083 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0009ae0 con 0x7f0cb4047580 2021-04-03T03:16:47.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cbaffd700 1 -- 172.21.2.111:0/2739926083 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cb0009e40 con 0x7f0cb4047580 2021-04-03T03:16:47.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- 172.21.2.111:0/2739926083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 msgr2=0x7f0cb403bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:47.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/2739926083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb403bf30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f0cb40d0310 tx=0x7f0cb0006e90).stop 2021-04-03T03:16:47.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- 172.21.2.111:0/2739926083 shutdown_connections 2021-04-03T03:16:47.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/2739926083 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb403bf30 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:47.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/2739926083 >> v2:172.21.2.115:3300/0 conn(0x7f0cb40423a0 0x7f0cb403c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:47.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- 172.21.2.111:0/2739926083 >> 172.21.2.111:0/2739926083 conn(0x7f0cb400d9a0 msgr2=0x7f0cb4046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:47.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- 172.21.2.111:0/2739926083 shutdown_connections 2021-04-03T03:16:47.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- 172.21.2.111:0/2739926083 wait complete. 2021-04-03T03:16:47.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 Processor -- start 2021-04-03T03:16:47.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.161+0000 7f0cc17c5700 1 -- start start 2021-04-03T03:16:47.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cb40423a0 0x7f0cb40d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:47.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb40439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:47.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cb40d38b0 0x7f0cb40cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:47.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40d4a10 con 0x7f0cb4047580 2021-04-03T03:16:47.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40d4cb0 con 0x7f0cb40423a0 2021-04-03T03:16:47.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40d4f50 con 0x7f0cb40d38b0 2021-04-03T03:16:47.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb40439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:47.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb40439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57240/0 (socket says 172.21.2.111:57240) 2021-04-03T03:16:47.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 -- 172.21.2.111:0/3010150429 learned_addr learned my addr 172.21.2.111:0/3010150429 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:47.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 -- 172.21.2.111:0/3010150429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cb40d38b0 msgr2=0x7f0cb40cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:47.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cb40d38b0 0x7f0cb40cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:47.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 -- 172.21.2.111:0/3010150429 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cb40423a0 msgr2=0x7f0cb40d6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:47.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cb40423a0 0x7f0cb40d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:47.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 -- 172.21.2.111:0/3010150429 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb403e440 con 0x7f0cb4047580 2021-04-03T03:16:47.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbb7fe700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb40439e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f0cac002f30 tx=0x7f0cac006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:47.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cb97fa700 1 -- 172.21.2.111:0/3010150429 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cac00d840 con 0x7f0cb4047580 2021-04-03T03:16:47.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cb97fa700 1 -- 172.21.2.111:0/3010150429 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac000e90 con 0x7f0cb4047580 2021-04-03T03:16:47.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb403eeb0 con 0x7f0cb4047580 2021-04-03T03:16:47.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cb97fa700 1 -- 172.21.2.111:0/3010150429 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cac00db90 con 0x7f0cb4047580 2021-04-03T03:16:47.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb40ccaa0 con 0x7f0cb4047580 2021-04-03T03:16:47.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb403f120 con 0x7f0cb4047580 2021-04-03T03:16:47.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cb97fa700 1 -- 172.21.2.111:0/3010150429 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0cac017030 con 0x7f0cb4047580 2021-04-03T03:16:47.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cb97fa700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c9c056f60 0x7f0c9c059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:47.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cb97fa700 1 -- 172.21.2.111:0/3010150429 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0cac06f560 con 0x7f0cb4047580 2021-04-03T03:16:47.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbbfff700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c9c056f60 0x7f0c9c059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:47.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.165+0000 7f0cbbfff700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c9c056f60 0x7f0c9c059420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f0cb40d0310 tx=0x7f0cb0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:47.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.169+0000 7f0cb97fa700 1 -- 172.21.2.111:0/3010150429 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0cac0449a0 con 0x7f0cb4047580 2021-04-03T03:16:47.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:47.261+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0cb4004db0 con 0x7f0cb4047580 2021-04-03T03:16:48.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cb97fa700 1 -- 172.21.2.111:0/3010150429 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 185+0+0 (secure 0 0 0) 0x7f0cac047fc0 con 0x7f0cb4047580 2021-04-03T03:16:48.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c9c056f60 msgr2=0x7f0c9c059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:48.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c9c056f60 0x7f0c9c059420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f0cb40d0310 tx=0x7f0cb0015000).stop 2021-04-03T03:16:48.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 msgr2=0x7f0cb40439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:48.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb40439e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f0cac002f30 tx=0x7f0cac006ea0).stop 2021-04-03T03:16:48.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 shutdown_connections 2021-04-03T03:16:48.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0cb40d38b0 0x7f0cb40cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:48.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c9c056f60 0x7f0c9c059420 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:48.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0cb40423a0 0x7f0cb40d6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:48.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 --2- 172.21.2.111:0/3010150429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0cb4047580 0x7f0cb40439e0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:48.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.085+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 >> 172.21.2.111:0/3010150429 conn(0x7f0cb400d9a0 msgr2=0x7f0cb4044a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:48.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.089+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 shutdown_connections 2021-04-03T03:16:48.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.089+0000 7f0cc17c5700 1 -- 172.21.2.111:0/3010150429 wait complete. 2021-04-03T03:16:48.092 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:16:48.096 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:16:48.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 Processor -- start 2021-04-03T03:16:48.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- start start 2021-04-03T03:16:48.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fafc40d7c20 0x7fafc4005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:48.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fafc40060b0 0x7fafc4006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:48.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc40445f0 con 0x7fafc40d7840 2021-04-03T03:16:48.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc4044890 con 0x7fafc40d7c20 2021-04-03T03:16:48.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc4005720 con 0x7fafc40060b0 2021-04-03T03:16:48.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fafc40d7c20 0x7fafc4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:48.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fafc40d7c20 0x7fafc4005bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57248/0 (socket says 172.21.2.111:57248) 2021-04-03T03:16:48.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 -- 172.21.2.111:0/1284932419 learned_addr learned my addr 172.21.2.111:0/1284932419 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:48.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 -- 172.21.2.111:0/1284932419 >> v2:172.21.2.115:3300/0 conn(0x7fafc40060b0 msgr2=0x7fafc4006540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:48.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 --2- 172.21.2.111:0/1284932419 >> v2:172.21.2.115:3300/0 conn(0x7fafc40060b0 0x7fafc4006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:48.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 -- 172.21.2.111:0/1284932419 >> v1:172.21.2.115:6789/0 conn(0x7fafc40d7840 legacy=0x7fafc40cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:48.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 -- 172.21.2.111:0/1284932419 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40d80a0 con 0x7fafc40d7c20 2021-04-03T03:16:48.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafcaffd700 1 --2- 172.21.2.111:0/1284932419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40d7c20 0x7fafc4005bc0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7fafc4047130 tx=0x7fafb8006e90).ready entity=mon.0 client_cookie=1fd7a46d8d7a341d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:48.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafca7fc700 1 -- 172.21.2.111:0/1284932419 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb8009930 con 0x7fafc40d7c20 2021-04-03T03:16:48.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafca7fc700 1 -- 172.21.2.111:0/1284932419 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb8009a90 con 0x7fafc40d7c20 2021-04-03T03:16:48.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafca7fc700 1 -- 172.21.2.111:0/1284932419 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb8009e30 con 0x7fafc40d7c20 2021-04-03T03:16:48.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- 172.21.2.111:0/1284932419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40d7c20 msgr2=0x7fafc4005bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:48.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 --2- 172.21.2.111:0/1284932419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40d7c20 0x7fafc4005bc0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7fafc4047130 tx=0x7fafb8006e90).stop 2021-04-03T03:16:48.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- 172.21.2.111:0/1284932419 shutdown_connections 2021-04-03T03:16:48.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 --2- 172.21.2.111:0/1284932419 >> v2:172.21.2.115:3300/0 conn(0x7fafc40060b0 0x7fafc4006540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:48.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 --2- 172.21.2.111:0/1284932419 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40d7c20 0x7fafc4005bc0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:48.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- 172.21.2.111:0/1284932419 >> 172.21.2.111:0/1284932419 conn(0x7fafc404a6d0 msgr2=0x7fafc404aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:48.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- 172.21.2.111:0/1284932419 shutdown_connections 2021-04-03T03:16:48.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- 172.21.2.111:0/1284932419 wait complete. 2021-04-03T03:16:48.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 Processor -- start 2021-04-03T03:16:48.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.153+0000 7fafd0aff700 1 -- start start 2021-04-03T03:16:48.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40060b0 0x7fafc40459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:48.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fafc40d7840 0x7fafc40e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:48.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fafc40d7c20 0x7fafc40e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:48.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40e7640 con 0x7fafc40060b0 2021-04-03T03:16:48.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40d3b40 con 0x7fafc40d7840 2021-04-03T03:16:48.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40d3660 con 0x7fafc40d7c20 2021-04-03T03:16:48.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40060b0 0x7fafc40459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:48.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40060b0 0x7fafc40459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57252/0 (socket says 172.21.2.111:57252) 2021-04-03T03:16:48.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 -- 172.21.2.111:0/2008675627 learned_addr learned my addr 172.21.2.111:0/2008675627 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:48.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 -- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fafc40d7c20 msgr2=0x7fafc40e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:48.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcbfff700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fafc40d7c20 0x7fafc40e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:48.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcaffd700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fafc40d7840 0x7fafc40e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:48.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fafc40d7c20 0x7fafc40e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:48.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 -- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fafc40d7840 msgr2=0x7fafc40e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:48.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fafc40d7840 0x7fafc40e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:48.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 -- 172.21.2.111:0/2008675627 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40d3060 con 0x7fafc40060b0 2021-04-03T03:16:48.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcb7fe700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40060b0 0x7fafc40459e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fafc0001f60 tx=0x7fafc0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:48.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafc8ff9700 1 -- 172.21.2.111:0/2008675627 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc000e4a0 con 0x7fafc40060b0 2021-04-03T03:16:48.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafc8ff9700 1 -- 172.21.2.111:0/2008675627 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000e600 con 0x7fafc40060b0 2021-04-03T03:16:48.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafc8ff9700 1 -- 172.21.2.111:0/2008675627 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc000e960 con 0x7fafc40060b0 2021-04-03T03:16:48.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc40e8540 con 0x7fafc40060b0 2021-04-03T03:16:48.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc40de3c0 con 0x7fafc40060b0 2021-04-03T03:16:48.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc40decb0 con 0x7fafc40060b0 2021-04-03T03:16:48.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafc8ff9700 1 -- 172.21.2.111:0/2008675627 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fafc0019030 con 0x7fafc40060b0 2021-04-03T03:16:48.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafc8ff9700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fafb4056ef0 0x7fafb40593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:48.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafc8ff9700 1 -- 172.21.2.111:0/2008675627 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6064+0+0 (secure 0 0 0) 0x7fafc006f5d0 con 0x7fafc40060b0 2021-04-03T03:16:48.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcaffd700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fafb4056ef0 0x7fafb40593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:48.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.157+0000 7fafcaffd700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fafb4056ef0 0x7fafb40593b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fafb8017000 tx=0x7fafb8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:48.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.161+0000 7fafc8ff9700 1 -- 172.21.2.111:0/2008675627 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fafc0042c50 con 0x7fafc40060b0 2021-04-03T03:16:48.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:48.257+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fafc4004db0 con 0x7fafc40060b0 2021-04-03T03:16:49.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.097+0000 7fafc8ff9700 1 -- 172.21.2.111:0/2008675627 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 173+0+0 (secure 0 0 0) 0x7fafc0044ac0 con 0x7fafc40060b0 2021-04-03T03:16:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fafb4056ef0 msgr2=0x7fafb40593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fafb4056ef0 0x7fafb40593b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fafb8017000 tx=0x7fafb8015000).stop 2021-04-03T03:16:49.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40060b0 msgr2=0x7fafc40459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40060b0 0x7fafc40459e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fafc0001f60 tx=0x7fafc0007c10).stop 2021-04-03T03:16:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 shutdown_connections 2021-04-03T03:16:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fafc40d7c20 0x7fafc40e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fafc40060b0 0x7fafc40459e0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fafb4056ef0 0x7fafb40593b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 --2- 172.21.2.111:0/2008675627 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fafc40d7840 0x7fafc40e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 >> 172.21.2.111:0/2008675627 conn(0x7fafc404a6d0 msgr2=0x7fafc40ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:49.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 shutdown_connections 2021-04-03T03:16:49.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.101+0000 7fafd0aff700 1 -- 172.21.2.111:0/2008675627 wait complete. 2021-04-03T03:16:49.105 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:16:49.109 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:16:49.110 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:16:49.110 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:16:49.110 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:16:49.112 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:16:49.112 DEBUG:teuthology.parallel:result is None 2021-04-03T03:16:49.113 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:16:49.114 DEBUG:teuthology.parallel:result is None 2021-04-03T03:16:49.114 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:16:49.146 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:16:49.158 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:16:49.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.169+0000 7f0b9ffff700 1 Processor -- start 2021-04-03T03:16:49.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.169+0000 7f0b9ffff700 1 -- start start 2021-04-03T03:16:49.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0ba00603f0 0x7f0ba0144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:49.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0ba00b0290 0x7f0ba01400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:49.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba00b0be0 con 0x7f0ba00b17f0 2021-04-03T03:16:49.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba0145300 con 0x7f0ba00b0290 2021-04-03T03:16:49.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba00607d0 con 0x7f0ba00603f0 2021-04-03T03:16:49.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0ba00b0290 0x7f0ba01400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:49.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0ba00603f0 0x7f0ba0144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:49.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0ba00b0290 0x7f0ba01400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57274/0 (socket says 172.21.2.111:57274) 2021-04-03T03:16:49.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 -- 172.21.2.111:0/2126685945 learned_addr learned my addr 172.21.2.111:0/2126685945 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:49.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 -- 172.21.2.111:0/2126685945 >> v2:172.21.2.115:3300/0 conn(0x7f0ba00603f0 msgr2=0x7f0ba0144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:49.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 --2- 172.21.2.111:0/2126685945 >> v2:172.21.2.115:3300/0 conn(0x7f0ba00603f0 0x7f0ba0144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 -- 172.21.2.111:0/2126685945 >> v1:172.21.2.115:6789/0 conn(0x7f0ba00b17f0 legacy=0x7f0ba0144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:49.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 -- 172.21.2.111:0/2126685945 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00b0e10 con 0x7f0ba00b0290 2021-04-03T03:16:49.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 --2- 172.21.2.111:0/2126685945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00b0290 0x7f0ba01400e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f0b980032b0 tx=0x7f0b98009270).ready entity=mon.0 client_cookie=9612040e994afd2b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:49.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9dffb700 1 -- 172.21.2.111:0/2126685945 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b98009e60 con 0x7f0ba00b0290 2021-04-03T03:16:49.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9dffb700 1 -- 172.21.2.111:0/2126685945 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9800bc00 con 0x7f0ba00b0290 2021-04-03T03:16:49.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9dffb700 1 -- 172.21.2.111:0/2126685945 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b9800bd70 con 0x7f0ba00b0290 2021-04-03T03:16:49.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2126685945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00b0290 msgr2=0x7f0ba01400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:49.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2126685945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00b0290 0x7f0ba01400e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f0b980032b0 tx=0x7f0b98009270).stop 2021-04-03T03:16:49.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2126685945 shutdown_connections 2021-04-03T03:16:49.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2126685945 >> v2:172.21.2.115:3300/0 conn(0x7f0ba00603f0 0x7f0ba0144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2126685945 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00b0290 0x7f0ba01400e0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2126685945 >> 172.21.2.111:0/2126685945 conn(0x7f0ba00b61c0 msgr2=0x7f0ba005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:49.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2126685945 shutdown_connections 2021-04-03T03:16:49.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2126685945 wait complete. 2021-04-03T03:16:49.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 Processor -- start 2021-04-03T03:16:49.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- start start 2021-04-03T03:16:49.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00603f0 0x7f0ba00accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:49.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ba00b0290 0x7f0ba00add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:49.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ba00b17f0 0x7f0ba00ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:49.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0064270 con 0x7f0ba00603f0 2021-04-03T03:16:49.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0064510 con 0x7f0ba00b0290 2021-04-03T03:16:49.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00647b0 con 0x7f0ba00b17f0 2021-04-03T03:16:49.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ba00b17f0 0x7f0ba00ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:49.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ba00b17f0 0x7f0ba00ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:40628/0 (socket says 172.21.2.111:40628) 2021-04-03T03:16:49.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 -- 172.21.2.111:0/2888996615 learned_addr learned my addr 172.21.2.111:0/2888996615 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:49.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00603f0 0x7f0ba00accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:49.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9e7fc700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ba00b0290 0x7f0ba00add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:49.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 -- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ba00b17f0 msgr2=0x7f0ba00ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:49.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ba00b17f0 0x7f0ba00ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 -- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ba00b0290 msgr2=0x7f0ba00add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:49.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ba00b0290 0x7f0ba00add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:49.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 -- 172.21.2.111:0/2888996615 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00ad1b0 con 0x7f0ba00603f0 2021-04-03T03:16:49.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9f7fe700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ba00b17f0 0x7f0ba00ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:16:49.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9effd700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00603f0 0x7f0ba00accc0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f0b8800c000 tx=0x7f0b88006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:49.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b97fff700 1 -- 172.21.2.111:0/2888996615 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b8800ca10 con 0x7f0ba00603f0 2021-04-03T03:16:49.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b97fff700 1 -- 172.21.2.111:0/2888996615 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8800cb70 con 0x7f0ba00603f0 2021-04-03T03:16:49.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba00ad960 con 0x7f0ba00603f0 2021-04-03T03:16:49.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b97fff700 1 -- 172.21.2.111:0/2888996615 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b8800ee80 con 0x7f0ba00603f0 2021-04-03T03:16:49.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba0136170 con 0x7f0ba00603f0 2021-04-03T03:16:49.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba0057660 con 0x7f0ba00603f0 2021-04-03T03:16:49.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.173+0000 7f0b97fff700 1 -- 172.21.2.111:0/2888996615 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0b880123f0 con 0x7f0ba00603f0 2021-04-03T03:16:49.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.177+0000 7f0b97fff700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0b84056f20 0x7f0b840593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:49.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.177+0000 7f0b97fff700 1 -- 172.21.2.111:0/2888996615 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f0b88070210 con 0x7f0ba00603f0 2021-04-03T03:16:49.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.177+0000 7f0b9e7fc700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0b84056f20 0x7f0b840593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:49.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.177+0000 7f0b9e7fc700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0b84056f20 0x7f0b840593e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f0b9000d000 tx=0x7f0b90007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:49.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.177+0000 7f0b97fff700 1 -- 172.21.2.111:0/2888996615 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0b88045870 con 0x7f0ba00603f0 2021-04-03T03:16:49.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:49.281+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0ba00686d0 con 0x7f0ba00603f0 2021-04-03T03:16:50.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b97fff700 1 -- 172.21.2.111:0/2888996615 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0b88048e90 con 0x7f0ba00603f0 2021-04-03T03:16:50.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0b84056f20 msgr2=0x7f0b840593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0b84056f20 0x7f0b840593e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f0b9000d000 tx=0x7f0b90007b70).stop 2021-04-03T03:16:50.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00603f0 msgr2=0x7f0ba00accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00603f0 0x7f0ba00accc0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f0b8800c000 tx=0x7f0b88006dd0).stop 2021-04-03T03:16:50.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 shutdown_connections 2021-04-03T03:16:50.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ba00b17f0 0x7f0ba00ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ba00603f0 0x7f0ba00accc0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0b84056f20 0x7f0b840593e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 --2- 172.21.2.111:0/2888996615 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ba00b0290 0x7f0ba00add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 >> 172.21.2.111:0/2888996615 conn(0x7f0ba00b61c0 msgr2=0x7f0ba0142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 shutdown_connections 2021-04-03T03:16:50.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.173+0000 7f0b9ffff700 1 -- 172.21.2.111:0/2888996615 wait complete. 2021-04-03T03:16:50.179 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:16:50.184 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:16:50.184 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:16:50.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 Processor -- start 2021-04-03T03:16:50.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 -- start start 2021-04-03T03:16:50.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15780d4330 0x7f157803ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780d06e0 con 0x7f157803dea0 2021-04-03T03:16:50.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780d0980 con 0x7f15780473f0 2021-04-03T03:16:50.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15780d47c0 con 0x7f15780d4330 2021-04-03T03:16:50.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1585c5f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15780d4330 0x7f157803ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1585c5f700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15780d4330 0x7f157803ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57288/0 (socket says 172.21.2.111:57288) 2021-04-03T03:16:50.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1585c5f700 1 -- 172.21.2.111:0/3069905368 learned_addr learned my addr 172.21.2.111:0/3069905368 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:50.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1585c5f700 1 -- 172.21.2.111:0/3069905368 >> v1:172.21.2.115:6789/0 conn(0x7f15780473f0 legacy=0x7f15780420c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:16:50.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1585c5f700 1 -- 172.21.2.111:0/3069905368 >> v1:172.21.2.111:6789/0 conn(0x7f157803dea0 legacy=0x7f15780d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1585c5f700 1 -- 172.21.2.111:0/3069905368 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780d0410 con 0x7f15780d4330 2021-04-03T03:16:50.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1585c5f700 1 --2- 172.21.2.111:0/3069905368 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f157803ce50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f15780d49a0 tx=0x7f158005ad70).ready entity=mon.0 client_cookie=4a85077e9d4b391a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1573fff700 1 -- 172.21.2.111:0/3069905368 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158005dfc0 con 0x7f15780d4330 2021-04-03T03:16:50.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1573fff700 1 -- 172.21.2.111:0/3069905368 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15800699f0 con 0x7f15780d4330 2021-04-03T03:16:50.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1573fff700 1 -- 172.21.2.111:0/3069905368 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158006b610 con 0x7f15780d4330 2021-04-03T03:16:50.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 -- 172.21.2.111:0/3069905368 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 msgr2=0x7f157803ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.337+0000 7f1586460700 1 --2- 172.21.2.111:0/3069905368 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f157803ce50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f15780d49a0 tx=0x7f158005ad70).stop 2021-04-03T03:16:50.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- 172.21.2.111:0/3069905368 shutdown_connections 2021-04-03T03:16:50.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 --2- 172.21.2.111:0/3069905368 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f157803ce50 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- 172.21.2.111:0/3069905368 >> 172.21.2.111:0/3069905368 conn(0x7f157800d9e0 msgr2=0x7f1578046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- 172.21.2.111:0/3069905368 shutdown_connections 2021-04-03T03:16:50.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- 172.21.2.111:0/3069905368 wait complete. 2021-04-03T03:16:50.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 Processor -- start 2021-04-03T03:16:50.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- start start 2021-04-03T03:16:50.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f157803dea0 0x7f15780a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780473f0 0x7f15780a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f15780a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157803c930 con 0x7f15780d4330 2021-04-03T03:16:50.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780e3b30 con 0x7f157803dea0 2021-04-03T03:16:50.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157803ba20 con 0x7f15780473f0 2021-04-03T03:16:50.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f15780a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f15780a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57296/0 (socket says 172.21.2.111:57296) 2021-04-03T03:16:50.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 -- 172.21.2.111:0/1922169169 learned_addr learned my addr 172.21.2.111:0/1922169169 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:50.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 -- 172.21.2.111:0/1922169169 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780473f0 msgr2=0x7f15780a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780473f0 0x7f15780a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 -- 172.21.2.111:0/1922169169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f157803dea0 msgr2=0x7f15780a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f157803dea0 0x7f15780a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 -- 172.21.2.111:0/1922169169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780e3dd0 con 0x7f15780d4330 2021-04-03T03:16:50.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1585c5f700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f15780a4a70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f15780d49a0 tx=0x7f1580087000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f15727fc700 1 -- 172.21.2.111:0/1922169169 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1580068960 con 0x7f15780d4330 2021-04-03T03:16:50.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f15727fc700 1 -- 172.21.2.111:0/1922169169 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158005f2c0 con 0x7f15780d4330 2021-04-03T03:16:50.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157803cd60 con 0x7f15780d4330 2021-04-03T03:16:50.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15780a42e0 con 0x7f15780d4330 2021-04-03T03:16:50.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f15727fc700 1 -- 172.21.2.111:0/1922169169 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f158005f620 con 0x7f15780d4330 2021-04-03T03:16:50.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157803ca70 con 0x7f15780d4330 2021-04-03T03:16:50.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f15727fc700 1 -- 172.21.2.111:0/1922169169 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1580096030 con 0x7f15780d4330 2021-04-03T03:16:50.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f15727fc700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1560057000 0x7f15600594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f15727fc700 1 -- 172.21.2.111:0/1922169169 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f15800e0320 con 0x7f15780d4330 2021-04-03T03:16:50.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f158545e700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1560057000 0x7f15600594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.341+0000 7f158545e700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1560057000 0x7f15600594c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f157c002f70 tx=0x7f157c006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.345+0000 7f15727fc700 1 -- 172.21.2.111:0/1922169169 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f15800b5a20 con 0x7f15780d4330 2021-04-03T03:16:50.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1578004da0 con 0x7f15780d4330 2021-04-03T03:16:50.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f15727fc700 1 -- 172.21.2.111:0/1922169169 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f158008a020 con 0x7f15780d4330 2021-04-03T03:16:50.474 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:50.474 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:16:50.475 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:16:50.475 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.475 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:16:50.475 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:16:50.476 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.476 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:50.476 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.476 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.476 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.477 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.477 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.477 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.477 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:16:50.478 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:16:50.478 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.478 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:50.478 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.478 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.479 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.479 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.479 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.479 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.480 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:16:50.480 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:16:50.480 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.480 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.480 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.481 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.481 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.481 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.481 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.482 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:16:50.482 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:16:50.482 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.482 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.482 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.483 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.483 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.483 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.483 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.483 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:16:50.484 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:16:50.484 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.484 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.484 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.485 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.485 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.485 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.485 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.485 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:16:50.486 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:16:50.486 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.486 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.486 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.486 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.487 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.487 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.487 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.487 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:16:50.487 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:16:50.488 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.488 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.488 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.488 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.489 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.489 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.489 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.489 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:16:50.489 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:16:50.490 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.490 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.490 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.490 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.491 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.491 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.491 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:16:50.491 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:16:50.492 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.492 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.492 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.492 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.493 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.493 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.493 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.493 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:16:50.493 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:16:50.494 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.494 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.494 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.494 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.495 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.495 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.495 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.495 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:16:50.495 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:16:50.496 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.496 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:50.496 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:16:50.496 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:16:50.497 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.497 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.497 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.497 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:16:50.497 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:16:50.498 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.498 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:50.498 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:16:50.498 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:16:50.499 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.499 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.499 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.499 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:16:50.499 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:16:50.500 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.500 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:16:50.500 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:16:50.500 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:16:50.500 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.501 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.501 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.501 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:16:50.501 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:16:50.502 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.502 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:16:50.502 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:16:50.502 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:16:50.502 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.503 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.503 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.503 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.503 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:16:50.503 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:16:50.504 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.504 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:16:50.504 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:16:50.504 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.504 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.505 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:50.505 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:50.505 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:16:50.505 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:16:50.505 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:16:50.506 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:16:50.506 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:16:50.506 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:16:50.506 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.507 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:50.507 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:16:50.507 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:16:50.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1560057000 msgr2=0x7f15600594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1560057000 0x7f15600594c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f157c002f70 tx=0x7f157c006990).stop 2021-04-03T03:16:50.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 msgr2=0x7f15780a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f15780a4a70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f15780d49a0 tx=0x7f1580087000).stop 2021-04-03T03:16:50.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 shutdown_connections 2021-04-03T03:16:50.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15780d4330 0x7f15780a4a70 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1560057000 0x7f15600594c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f157803dea0 0x7f15780a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 --2- 172.21.2.111:0/1922169169 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15780473f0 0x7f15780a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 >> 172.21.2.111:0/1922169169 conn(0x7f157800d9e0 msgr2=0x7f1578045310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 shutdown_connections 2021-04-03T03:16:50.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.473+0000 7f1586460700 1 -- 172.21.2.111:0/1922169169 wait complete. 2021-04-03T03:16:50.512 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:50.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19bfff700 1 Processor -- start 2021-04-03T03:16:50.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19bfff700 1 -- start start 2021-04-03T03:16:50.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd19c0b12b0 0x7fd19c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0607d0 con 0x7fd19c14b700 2021-04-03T03:16:50.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0b0eb0 con 0x7fd19c0603f0 2021-04-03T03:16:50.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19bfff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c0af400 con 0x7fd19c0b12b0 2021-04-03T03:16:50.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd19c0b12b0 0x7fd19c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19a7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd19c14b700 0x7fd19c0b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58094/0 (socket says 172.21.2.111:58094) 2021-04-03T03:16:50.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd19c0b12b0 0x7fd19c1522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57300/0 (socket says 172.21.2.111:57300) 2021-04-03T03:16:50.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.573+0000 7fd19a7fc700 1 -- 172.21.2.111:0/3617611451 learned_addr learned my addr 172.21.2.111:0/3617611451 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:50.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd199ffb700 1 -- 172.21.2.111:0/3617611451 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930601535 0 0) 0x7fd19c0607d0 con 0x7fd19c14b700 2021-04-03T03:16:50.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd199ffb700 1 -- 172.21.2.111:0/3617611451 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd17c003240 con 0x7fd19c14b700 2021-04-03T03:16:50.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19b7fe700 1 -- 172.21.2.111:0/3617611451 >> v1:172.21.2.115:6789/0 conn(0x7fd19c0603f0 legacy=0x7fd19c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:50.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19b7fe700 1 -- 172.21.2.111:0/3617611451 >> v1:172.21.2.111:6789/0 conn(0x7fd19c14b700 legacy=0x7fd19c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19b7fe700 1 -- 172.21.2.111:0/3617611451 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0af6a0 con 0x7fd19c0b12b0 2021-04-03T03:16:50.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19b7fe700 1 --2- 172.21.2.111:0/3617611451 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0b12b0 0x7fd19c1522d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7fd19c1486c0 tx=0x7fd1940091d0).ready entity=mon.0 client_cookie=b10fab72b345e968 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd199ffb700 1 -- 172.21.2.111:0/3617611451 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd19400b990 con 0x7fd19c0b12b0 2021-04-03T03:16:50.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd199ffb700 1 -- 172.21.2.111:0/3617611451 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400baf0 con 0x7fd19c0b12b0 2021-04-03T03:16:50.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd199ffb700 1 -- 172.21.2.111:0/3617611451 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd19400bc60 con 0x7fd19c0b12b0 2021-04-03T03:16:50.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/3617611451 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0b12b0 msgr2=0x7fd19c1522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 --2- 172.21.2.111:0/3617611451 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0b12b0 0x7fd19c1522d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7fd19c1486c0 tx=0x7fd1940091d0).stop 2021-04-03T03:16:50.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/3617611451 shutdown_connections 2021-04-03T03:16:50.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 --2- 172.21.2.111:0/3617611451 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0b12b0 0x7fd19c1522d0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/3617611451 >> 172.21.2.111:0/3617611451 conn(0x7fd19c0b61c0 msgr2=0x7fd19c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/3617611451 shutdown_connections 2021-04-03T03:16:50.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/3617611451 wait complete. 2021-04-03T03:16:50.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 Processor -- start 2021-04-03T03:16:50.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- start start 2021-04-03T03:16:50.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0603f0 0x7fd19c0a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd19c0b12b0 0x7fd19c063b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd19c14b700 0x7fd19c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0646f0 con 0x7fd19c0603f0 2021-04-03T03:16:50.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c15a940 con 0x7fd19c14b700 2021-04-03T03:16:50.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0641b0 con 0x7fd19c0b12b0 2021-04-03T03:16:50.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0603f0 0x7fd19c0a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0603f0 0x7fd19c0a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57306/0 (socket says 172.21.2.111:57306) 2021-04-03T03:16:50.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 -- 172.21.2.111:0/973332876 learned_addr learned my addr 172.21.2.111:0/973332876 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:50.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 -- 172.21.2.111:0/973332876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd19c0b12b0 msgr2=0x7fd19c063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd19c0b12b0 0x7fd19c063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 -- 172.21.2.111:0/973332876 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd19c14b700 msgr2=0x7fd19c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd19c14b700 0x7fd19c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 -- 172.21.2.111:0/973332876 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c064450 con 0x7fd19c0603f0 2021-04-03T03:16:50.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19affd700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0603f0 0x7fd19c0a99c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7fd184002f50 tx=0x7fd184006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd193fff700 1 -- 172.21.2.111:0/973332876 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18400d4a0 con 0x7fd19c0603f0 2021-04-03T03:16:50.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd193fff700 1 -- 172.21.2.111:0/973332876 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18400d600 con 0x7fd19c0603f0 2021-04-03T03:16:50.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c0aab30 con 0x7fd19c0603f0 2021-04-03T03:16:50.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c156670 con 0x7fd19c0603f0 2021-04-03T03:16:50.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd193fff700 1 -- 172.21.2.111:0/973332876 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18400d960 con 0x7fd19c0603f0 2021-04-03T03:16:50.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.577+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c064000 con 0x7fd19c0603f0 2021-04-03T03:16:50.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.581+0000 7fd193fff700 1 -- 172.21.2.111:0/973332876 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd18400f070 con 0x7fd19c0603f0 2021-04-03T03:16:50.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.581+0000 7fd193fff700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd180056f70 0x7fd180059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.581+0000 7fd193fff700 1 -- 172.21.2.111:0/973332876 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fd18406f940 con 0x7fd19c0603f0 2021-04-03T03:16:50.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.581+0000 7fd19a7fc700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd180056f70 0x7fd180059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.581+0000 7fd19a7fc700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd180056f70 0x7fd180059430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fd18c0010d0 tx=0x7fd18c0076a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.581+0000 7fd193fff700 1 -- 172.21.2.111:0/973332876 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd184045060 con 0x7fd19c0603f0 2021-04-03T03:16:50.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.673+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd19c057660 con 0x7fd19c0603f0 2021-04-03T03:16:50.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.673+0000 7fd193fff700 1 -- 172.21.2.111:0/973332876 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13424 (secure 0 0 0) 0x7fd184048680 con 0x7fd19c0603f0 2021-04-03T03:16:50.677 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:50.677 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":128,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:50.164747+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:50.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd180056f70 msgr2=0x7fd180059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd180056f70 0x7fd180059430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fd18c0010d0 tx=0x7fd18c0076a0).stop 2021-04-03T03:16:50.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0603f0 msgr2=0x7fd19c0a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0603f0 0x7fd19c0a99c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7fd184002f50 tx=0x7fd184006dd0).stop 2021-04-03T03:16:50.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 shutdown_connections 2021-04-03T03:16:50.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd180056f70 0x7fd180059430 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd19c14b700 0x7fd19c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd19c0603f0 0x7fd19c0a99c0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 --2- 172.21.2.111:0/973332876 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd19c0b12b0 0x7fd19c063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 >> 172.21.2.111:0/973332876 conn(0x7fd19c0b61c0 msgr2=0x7fd19c0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 shutdown_connections 2021-04-03T03:16:50.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.677+0000 7fd19bfff700 1 -- 172.21.2.111:0/973332876 wait complete. 2021-04-03T03:16:50.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:16:50.686 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:50.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.741+0000 7fa43bfff700 1 Processor -- start 2021-04-03T03:16:50.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.741+0000 7fa43bfff700 1 -- start start 2021-04-03T03:16:50.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa434041f80 0x7fa434042360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340411e0 con 0x7fa43403d860 2021-04-03T03:16:50.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340d4b80 con 0x7fa43403ccc0 2021-04-03T03:16:50.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43403d130 con 0x7fa434041f80 2021-04-03T03:16:50.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43a7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa43403d860 0x7fa43403c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58108/0 (socket says 172.21.2.111:58108) 2021-04-03T03:16:50.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43a7fc700 1 -- 172.21.2.111:0/4233955360 learned_addr learned my addr 172.21.2.111:0/4233955360 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:50.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160471398 0 0) 0x7fa4340411e0 con 0x7fa43403d860 2021-04-03T03:16:50.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41c003240 con 0x7fa43403d860 2021-04-03T03:16:50.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa434041f80 0x7fa434042360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 438145818 0 0) 0x7fa41c003240 con 0x7fa43403d860 2021-04-03T03:16:50.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa434041f80 msgr2=0x7fa434042360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:50.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa434041f80 0x7fa434042360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 >> v1:172.21.2.115:6789/0 conn(0x7fa43403ccc0 legacy=0x7fa43403eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:50.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434041480 con 0x7fa43403d860 2021-04-03T03:16:50.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa42c0016d0 con 0x7fa43403d860 2021-04-03T03:16:50.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 >> v1:172.21.2.111:6789/0 conn(0x7fa43403d860 legacy=0x7fa43403c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa41c0037e0 0x7fa41c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa41c0044e0 0x7fa41c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa434041f80 0x7fa41c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43403d130 con 0x7fa41c0037e0 2021-04-03T03:16:50.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c004350 con 0x7fa41c0044e0 2021-04-03T03:16:50.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c00b730 con 0x7fa434041f80 2021-04-03T03:16:50.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa41c0037e0 0x7fa41c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43a7fc700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa41c0044e0 0x7fa41c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 -- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa434041f80 msgr2=0x7fa41c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:50.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa434041f80 0x7fa41c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 -- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa41c0044e0 msgr2=0x7fa41c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:50.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa41c0044e0 0x7fa41c004960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 -- 172.21.2.111:0/4233955360 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434041480 con 0x7fa41c0037e0 2021-04-03T03:16:50.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa41c0037e0 0x7fa41c003be0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7fa42400a000 tx=0x7fa424008040).ready entity=mon.0 client_cookie=3e53faec7dae5118 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42400da60 con 0x7fa41c0037e0 2021-04-03T03:16:50.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42400dbc0 con 0x7fa41c0037e0 2021-04-03T03:16:50.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa439ffb700 1 -- 172.21.2.111:0/4233955360 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa424013480 con 0x7fa41c0037e0 2021-04-03T03:16:50.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- 172.21.2.111:0/4233955360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa41c0037e0 msgr2=0x7fa41c003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 --2- 172.21.2.111:0/4233955360 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa41c0037e0 0x7fa41c003be0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7fa42400a000 tx=0x7fa424008040).stop 2021-04-03T03:16:50.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43b7fe700 1 -- 172.21.2.111:0/4233955360 reap_dead start 2021-04-03T03:16:50.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- 172.21.2.111:0/4233955360 shutdown_connections 2021-04-03T03:16:50.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- 172.21.2.111:0/4233955360 >> 172.21.2.111:0/4233955360 conn(0x7fa4340462b0 msgr2=0x7fa4340466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- 172.21.2.111:0/4233955360 shutdown_connections 2021-04-03T03:16:50.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- 172.21.2.111:0/4233955360 wait complete. 2021-04-03T03:16:50.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 Processor -- start 2021-04-03T03:16:50.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- start start 2021-04-03T03:16:50.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4340d9e70 0x7fa4340d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4340da250 0x7fa4340d2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa4340e5dd0 0x7fa4340e6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340dab80 con 0x7fa4340d9e70 2021-04-03T03:16:50.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434006da0 con 0x7fa4340da250 2021-04-03T03:16:50.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43bfff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340d9640 con 0x7fa4340e5dd0 2021-04-03T03:16:50.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4340d9e70 0x7fa4340d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4340d9e70 0x7fa4340d1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57328/0 (socket says 172.21.2.111:57328) 2021-04-03T03:16:50.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 -- 172.21.2.111:0/4047843549 learned_addr learned my addr 172.21.2.111:0/4047843549 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:50.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 -- 172.21.2.111:0/4047843549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa4340e5dd0 msgr2=0x7fa4340e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa4340e5dd0 0x7fa4340e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 -- 172.21.2.111:0/4047843549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4340da250 msgr2=0x7fa4340d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.745+0000 7fa43affd700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4340da250 0x7fa4340d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa43affd700 1 -- 172.21.2.111:0/4047843549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340a4cd0 con 0x7fa4340d9e70 2021-04-03T03:16:50.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa43affd700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4340d9e70 0x7fa4340d1b10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fa42400a000 tx=0x7fa424008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa433fff700 1 -- 172.21.2.111:0/4047843549 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa424002f30 con 0x7fa4340d9e70 2021-04-03T03:16:50.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa433fff700 1 -- 172.21.2.111:0/4047843549 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42401d410 con 0x7fa4340d9e70 2021-04-03T03:16:50.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4340d8540 con 0x7fa4340d9e70 2021-04-03T03:16:50.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa433fff700 1 -- 172.21.2.111:0/4047843549 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42401d770 con 0x7fa4340d9e70 2021-04-03T03:16:50.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4340d82d0 con 0x7fa4340d9e70 2021-04-03T03:16:50.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa434004db0 con 0x7fa4340d9e70 2021-04-03T03:16:50.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa433fff700 1 -- 172.21.2.111:0/4047843549 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa42401d8d0 con 0x7fa4340d9e70 2021-04-03T03:16:50.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa433fff700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa420056fe0 0x7fa4200594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa433fff700 1 -- 172.21.2.111:0/4047843549 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fa4240717e0 con 0x7fa4340d9e70 2021-04-03T03:16:50.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa43a7fc700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa420056fe0 0x7fa4200594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa43a7fc700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa420056fe0 0x7fa4200594a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fa42c008000 tx=0x7fa42c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.749+0000 7fa433fff700 1 -- 172.21.2.111:0/4047843549 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa424046f00 con 0x7fa4340d9e70 2021-04-03T03:16:50.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa4340d8760 con 0x7fa4340d9e70 2021-04-03T03:16:50.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa433fff700 1 -- 172.21.2.111:0/4047843549 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v128) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa42404a520 con 0x7fa4340d9e70 2021-04-03T03:16:50.848 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:50.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa420056fe0 msgr2=0x7fa4200594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa420056fe0 0x7fa4200594a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fa42c008000 tx=0x7fa42c009040).stop 2021-04-03T03:16:50.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4340d9e70 msgr2=0x7fa4340d1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4340d9e70 0x7fa4340d1b10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fa42400a000 tx=0x7fa424008040).stop 2021-04-03T03:16:50.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 shutdown_connections 2021-04-03T03:16:50.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa4340e5dd0 0x7fa4340e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa4340d9e70 0x7fa4340d1b10 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa420056fe0 0x7fa4200594a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 --2- 172.21.2.111:0/4047843549 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa4340da250 0x7fa4340d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 >> 172.21.2.111:0/4047843549 conn(0x7fa4340462b0 msgr2=0x7fa434005a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 shutdown_connections 2021-04-03T03:16:50.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.845+0000 7fa43bfff700 1 -- 172.21.2.111:0/4047843549 wait complete. 2021-04-03T03:16:50.855 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:50.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb5947700 1 Processor -- start 2021-04-03T03:16:50.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb5947700 1 -- start start 2021-04-03T03:16:50.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb5947700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7febb00add50 0x7febb0147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb5947700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb00607d0 con 0x7febb00603f0 2021-04-03T03:16:50.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb5947700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb014b090 con 0x7febb0144ef0 2021-04-03T03:16:50.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb5947700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb014b330 con 0x7febb00add50 2021-04-03T03:16:50.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb4945700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7febb00603f0 0x7febb01447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58124/0 (socket says 172.21.2.111:58124) 2021-04-03T03:16:50.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febb4945700 1 -- 172.21.2.111:0/640601091 learned_addr learned my addr 172.21.2.111:0/640601091 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:50.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.909+0000 7febaffff700 1 --2- 172.21.2.111:0/640601091 >> v2:172.21.2.111:3300/0 conn(0x7febb00add50 0x7febb0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaffff700 1 -- 172.21.2.111:0/640601091 >> v1:172.21.2.115:6789/0 conn(0x7febb0144ef0 legacy=0x7febb0147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:50.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaffff700 1 -- 172.21.2.111:0/640601091 >> v1:172.21.2.111:6789/0 conn(0x7febb00603f0 legacy=0x7febb01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaf7fe700 1 -- 172.21.2.111:0/640601091 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440819537 0 0) 0x7febb00607d0 con 0x7febb00603f0 2021-04-03T03:16:50.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaffff700 1 -- 172.21.2.111:0/640601091 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb014b5d0 con 0x7febb00add50 2021-04-03T03:16:50.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaffff700 1 --2- 172.21.2.111:0/640601091 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb00add50 0x7febb0147740 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7febb00b5600 tx=0x7feb98006dd0).ready entity=mon.0 client_cookie=a943052c3907bd87 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaf7fe700 1 -- 172.21.2.111:0/640601091 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb980097b0 con 0x7febb00add50 2021-04-03T03:16:50.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaf7fe700 1 -- 172.21.2.111:0/640601091 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb98009910 con 0x7febb00add50 2021-04-03T03:16:50.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febaf7fe700 1 -- 172.21.2.111:0/640601091 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb98009c70 con 0x7febb00add50 2021-04-03T03:16:50.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/640601091 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb00add50 msgr2=0x7febb0147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 --2- 172.21.2.111:0/640601091 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb00add50 0x7febb0147740 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7febb00b5600 tx=0x7feb98006dd0).stop 2021-04-03T03:16:50.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/640601091 shutdown_connections 2021-04-03T03:16:50.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 --2- 172.21.2.111:0/640601091 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb00add50 0x7febb0147740 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:50.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/640601091 >> 172.21.2.111:0/640601091 conn(0x7febb00b61c0 msgr2=0x7febb005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:50.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/640601091 shutdown_connections 2021-04-03T03:16:50.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/640601091 wait complete. 2021-04-03T03:16:50.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 Processor -- start 2021-04-03T03:16:50.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- start start 2021-04-03T03:16:50.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febb00603f0 0x7febb00aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febb00add50 0x7febb015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb0144ef0 0x7febb00a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb015ae30 con 0x7febb0144ef0 2021-04-03T03:16:50.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb015ab90 con 0x7febb00add50 2021-04-03T03:16:50.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb00ab590 con 0x7febb00603f0 2021-04-03T03:16:50.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb0144ef0 0x7febb00a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb0144ef0 0x7febb00a6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57342/0 (socket says 172.21.2.111:57342) 2021-04-03T03:16:50.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 -- 172.21.2.111:0/2164533557 learned_addr learned my addr 172.21.2.111:0/2164533557 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:50.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 -- 172.21.2.111:0/2164533557 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febb00603f0 msgr2=0x7febb00aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febb00603f0 0x7febb00aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 -- 172.21.2.111:0/2164533557 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febb00add50 msgr2=0x7febb015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:50.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febb00add50 0x7febb015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:50.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 -- 172.21.2.111:0/2164533557 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb00abf20 con 0x7febb0144ef0 2021-04-03T03:16:50.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5146700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb0144ef0 0x7febb00a6f20 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7feba8003250 tx=0x7feba8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febadffb700 1 -- 172.21.2.111:0/2164533557 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba800c960 con 0x7febb0144ef0 2021-04-03T03:16:50.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febadffb700 1 -- 172.21.2.111:0/2164533557 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba800cac0 con 0x7febb0144ef0 2021-04-03T03:16:50.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febadffb700 1 -- 172.21.2.111:0/2164533557 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba800ce20 con 0x7febb0144ef0 2021-04-03T03:16:50.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb00a98b0 con 0x7febb0144ef0 2021-04-03T03:16:50.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb015a940 con 0x7febb0144ef0 2021-04-03T03:16:50.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb00a9fb0 con 0x7febb0144ef0 2021-04-03T03:16:50.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.913+0000 7febadffb700 1 -- 172.21.2.111:0/2164533557 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7feba801f3f0 con 0x7febb0144ef0 2021-04-03T03:16:50.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.917+0000 7febadffb700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb94056fb0 0x7feb94059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:50.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.917+0000 7febadffb700 1 -- 172.21.2.111:0/2164533557 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5596+0+0 (secure 0 0 0) 0x7feba806fe80 con 0x7febb0144ef0 2021-04-03T03:16:50.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.917+0000 7febb4945700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb94056fb0 0x7feb94059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:50.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.917+0000 7febb4945700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb94056fb0 0x7feb94059470 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7feba0000b80 tx=0x7feba0001160).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:50.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:50.917+0000 7febadffb700 1 -- 172.21.2.111:0/2164533557 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7feba8045430 con 0x7febb0144ef0 2021-04-03T03:16:51.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7febb00ab750 con 0x7febb0144ef0 2021-04-03T03:16:51.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febadffb700 1 -- 172.21.2.111:0/2164533557 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v128) v1 ==== 92+0+10 (secure 0 0 0) 0x7feba801a030 con 0x7febb0144ef0 2021-04-03T03:16:51.014 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:51.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb94056fb0 msgr2=0x7feb94059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb94056fb0 0x7feb94059470 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7feba0000b80 tx=0x7feba0001160).stop 2021-04-03T03:16:51.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb0144ef0 msgr2=0x7febb00a6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb0144ef0 0x7febb00a6f20 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7feba8003250 tx=0x7feba8009060).stop 2021-04-03T03:16:51.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 shutdown_connections 2021-04-03T03:16:51.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7febb0144ef0 0x7febb00a6f20 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7febb00603f0 0x7febb00aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7feb94056fb0 0x7feb94059470 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 --2- 172.21.2.111:0/2164533557 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7febb00add50 0x7febb015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 >> 172.21.2.111:0/2164533557 conn(0x7febb00b61c0 msgr2=0x7febb00b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:51.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 shutdown_connections 2021-04-03T03:16:51.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.013+0000 7febb5947700 1 -- 172.21.2.111:0/2164533557 wait complete. 2021-04-03T03:16:51.022 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:16:51.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 Processor -- start 2021-04-03T03:16:51.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- start start 2021-04-03T03:16:51.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4b8414b700 0x7f4b840b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4b840b0c90 0x7f4b84147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b840af400 con 0x7f4b840603f0 2021-04-03T03:16:51.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b84060860 con 0x7f4b840b0c90 2021-04-03T03:16:51.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b840af6a0 con 0x7f4b8414b700 2021-04-03T03:16:51.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b88fbc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4b840603f0 0x7f4b840b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58138/0 (socket says 172.21.2.111:58138) 2021-04-03T03:16:51.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b88fbc700 1 -- 172.21.2.111:0/3094783878 learned_addr learned my addr 172.21.2.111:0/3094783878 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:51.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b897bd700 1 --2- 172.21.2.111:0/3094783878 >> v2:172.21.2.111:3300/0 conn(0x7f4b840b0c90 0x7f4b84147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b897bd700 1 -- 172.21.2.111:0/3094783878 >> v2:172.21.2.115:3300/0 conn(0x7f4b8414b700 msgr2=0x7f4b840b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:51.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b897bd700 1 --2- 172.21.2.111:0/3094783878 >> v2:172.21.2.115:3300/0 conn(0x7f4b8414b700 0x7f4b840b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b897bd700 1 -- 172.21.2.111:0/3094783878 >> v1:172.21.2.111:6789/0 conn(0x7f4b840603f0 legacy=0x7f4b840b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b897bd700 1 -- 172.21.2.111:0/3094783878 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840afb40 con 0x7f4b840b0c90 2021-04-03T03:16:51.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b897bd700 1 --2- 172.21.2.111:0/3094783878 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b840b0c90 0x7f4b84147680 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f4b740010d0 tx=0x7f4b7c0091d0).ready entity=mon.0 client_cookie=ef7c7e856ebeaae1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:51.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b837fe700 1 -- 172.21.2.111:0/3094783878 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b7c00b910 con 0x7f4b840b0c90 2021-04-03T03:16:51.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b837fe700 1 -- 172.21.2.111:0/3094783878 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c00ba70 con 0x7f4b840b0c90 2021-04-03T03:16:51.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b837fe700 1 -- 172.21.2.111:0/3094783878 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b7c00bbe0 con 0x7f4b840b0c90 2021-04-03T03:16:51.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- 172.21.2.111:0/3094783878 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b840b0c90 msgr2=0x7f4b84147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/3094783878 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b840b0c90 0x7f4b84147680 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f4b740010d0 tx=0x7f4b7c0091d0).stop 2021-04-03T03:16:51.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- 172.21.2.111:0/3094783878 shutdown_connections 2021-04-03T03:16:51.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/3094783878 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b840b0c90 0x7f4b84147680 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/3094783878 >> v2:172.21.2.115:3300/0 conn(0x7f4b8414b700 0x7f4b840b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- 172.21.2.111:0/3094783878 >> 172.21.2.111:0/3094783878 conn(0x7f4b840b61c0 msgr2=0x7f4b8405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:51.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- 172.21.2.111:0/3094783878 shutdown_connections 2021-04-03T03:16:51.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.077+0000 7f4b89fbe700 1 -- 172.21.2.111:0/3094783878 wait complete. 2021-04-03T03:16:51.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 Processor -- start 2021-04-03T03:16:51.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 -- start start 2021-04-03T03:16:51.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b840603f0 0x7f4b84063390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b840b0c90 0x7f4b840a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.088 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b8414b700 0x7f4b84067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8415aa30 con 0x7f4b8414b700 2021-04-03T03:16:51.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840a9d20 con 0x7f4b840b0c90 2021-04-03T03:16:51.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8415a050 con 0x7f4b840603f0 2021-04-03T03:16:51.089 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b8414b700 0x7f4b84067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b8414b700 0x7f4b84067630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57356/0 (socket says 172.21.2.111:57356) 2021-04-03T03:16:51.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 -- 172.21.2.111:0/2561777356 learned_addr learned my addr 172.21.2.111:0/2561777356 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:51.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 -- 172.21.2.111:0/2561777356 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b840603f0 msgr2=0x7f4b84063390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b840603f0 0x7f4b84063390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.090 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 -- 172.21.2.111:0/2561777356 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b840b0c90 msgr2=0x7f4b840a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b840b0c90 0x7f4b840a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 -- 172.21.2.111:0/2561777356 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8415a2f0 con 0x7f4b8414b700 2021-04-03T03:16:51.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b897bd700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b8414b700 0x7f4b84067630 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f4b740010d0 tx=0x7f4b7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:51.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b81ffb700 1 -- 172.21.2.111:0/2561777356 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b7c00be90 con 0x7f4b8414b700 2021-04-03T03:16:51.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b81ffb700 1 -- 172.21.2.111:0/2561777356 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c006ae0 con 0x7f4b8414b700 2021-04-03T03:16:51.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8415a590 con 0x7f4b8414b700 2021-04-03T03:16:51.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b81ffb700 1 -- 172.21.2.111:0/2561777356 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b7c01f5e0 con 0x7f4b8414b700 2021-04-03T03:16:51.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b84159c60 con 0x7f4b8414b700 2021-04-03T03:16:51.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8415a720 con 0x7f4b8414b700 2021-04-03T03:16:51.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b81ffb700 1 -- 172.21.2.111:0/2561777356 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4b7c026030 con 0x7f4b8414b700 2021-04-03T03:16:51.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b81ffb700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b68056f10 0x7f4b680593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b81ffb700 1 -- 172.21.2.111:0/2561777356 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f4b7c070aa0 con 0x7f4b8414b700 2021-04-03T03:16:51.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b88fbc700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b68056f10 0x7f4b680593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.081+0000 7f4b88fbc700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b68056f10 0x7f4b680593d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f4b74001190 tx=0x7f4b740077b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:51.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.085+0000 7f4b81ffb700 1 -- 172.21.2.111:0/2561777356 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4b7c0461c0 con 0x7f4b8414b700 2021-04-03T03:16:51.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4b84057660 con 0x7f4b8414b700 2021-04-03T03:16:51.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b81ffb700 1 -- 172.21.2.111:0/2561777356 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4b7c0240a0 con 0x7f4b8414b700 2021-04-03T03:16:51.184 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:51.184 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:16:51.184 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:51.184 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:16:51.185 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:16:51.185 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:16:51.185 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:16:51.185 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:16:51.186 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:16:51.186 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:16:51.186 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:16:51.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b68056f10 msgr2=0x7f4b680593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b68056f10 0x7f4b680593d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f4b74001190 tx=0x7f4b740077b0).stop 2021-04-03T03:16:51.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b8414b700 msgr2=0x7f4b84067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b8414b700 0x7f4b84067630 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f4b740010d0 tx=0x7f4b7c016000).stop 2021-04-03T03:16:51.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 shutdown_connections 2021-04-03T03:16:51.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b8414b700 0x7f4b84067630 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b840603f0 0x7f4b84063390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b68056f10 0x7f4b680593d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 --2- 172.21.2.111:0/2561777356 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b840b0c90 0x7f4b840a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 >> 172.21.2.111:0/2561777356 conn(0x7f4b840b61c0 msgr2=0x7f4b8405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:51.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 shutdown_connections 2021-04-03T03:16:51.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.181+0000 7f4b89fbe700 1 -- 172.21.2.111:0/2561777356 wait complete. 2021-04-03T03:16:51.191 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:16:51.192 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:16:51.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 Processor -- start 2021-04-03T03:16:51.250 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- start start 2021-04-03T03:16:51.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 0x7f71280dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71280dc560 con 0x7f712803d560 2021-04-03T03:16:51.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71280dc800 con 0x7f7128041060 2021-04-03T03:16:51.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712803de60 con 0x7f71280dbba0 2021-04-03T03:16:51.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712f5e6700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f712803d560 0x7f7128041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58152/0 (socket says 172.21.2.111:58152) 2021-04-03T03:16:51.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712f5e6700 1 -- 172.21.2.111:0/3936032245 learned_addr learned my addr 172.21.2.111:0/3936032245 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:51.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512456137 0 0) 0x7f71280dc560 con 0x7f712803d560 2021-04-03T03:16:51.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7110003240 con 0x7f712803d560 2021-04-03T03:16:51.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1935791951 0 0) 0x7f7110003240 con 0x7f712803d560 2021-04-03T03:16:51.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 0x7f71280dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 msgr2=0x7f71280dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:51.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 0x7f71280dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 >> v1:172.21.2.115:6789/0 conn(0x7f7128041060 legacy=0x7f712803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:51.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712803da30 con 0x7f712803d560 2021-04-03T03:16:51.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f71300450f0 con 0x7f712803d560 2021-04-03T03:16:51.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 >> v1:172.21.2.111:6789/0 conn(0x7f712803d560 legacy=0x7f7128041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71100037e0 0x7f7110003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71100044e0 0x7f7110004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 0x7f7110004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712803de60 con 0x7f71100037e0 2021-04-03T03:16:51.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110005390 con 0x7f71100044e0 2021-04-03T03:16:51.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71100042e0 con 0x7f71280dbba0 2021-04-03T03:16:51.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71100037e0 0x7f7110003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712f5e6700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 0x7f7110004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 -- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 msgr2=0x7f7110004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:51.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280dbba0 0x7f7110004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 -- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71100044e0 msgr2=0x7f7110004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:51.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712e5e4700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71100044e0 0x7f7110004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71100044e0 0x7f7110004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:51.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 -- 172.21.2.111:0/3936032245 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110003240 con 0x7f71100037e0 2021-04-03T03:16:51.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712e5e4700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f71100044e0 0x7f7110004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:51.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712ede5700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71100037e0 0x7f7110003be0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f7118007000 tx=0x7f7118008040).ready entity=mon.0 client_cookie=17ebaaf4029aa0b8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:51.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f711800a950 con 0x7f71100037e0 2021-04-03T03:16:51.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711800c030 con 0x7f71100037e0 2021-04-03T03:16:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712dde3700 1 -- 172.21.2.111:0/3936032245 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f711800aca0 con 0x7f71100037e0 2021-04-03T03:16:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- 172.21.2.111:0/3936032245 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71100037e0 msgr2=0x7f7110003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 --2- 172.21.2.111:0/3936032245 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71100037e0 0x7f7110003be0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f7118007000 tx=0x7f7118008040).stop 2021-04-03T03:16:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712f5e6700 1 -- 172.21.2.111:0/3936032245 reap_dead start 2021-04-03T03:16:51.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- 172.21.2.111:0/3936032245 shutdown_connections 2021-04-03T03:16:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- 172.21.2.111:0/3936032245 >> 172.21.2.111:0/3936032245 conn(0x7f71280462b0 msgr2=0x7f71280466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- 172.21.2.111:0/3936032245 shutdown_connections 2021-04-03T03:16:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.249+0000 7f712fde7700 1 -- 172.21.2.111:0/3936032245 wait complete. 2021-04-03T03:16:51.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 Processor -- start 2021-04-03T03:16:51.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 -- start start 2021-04-03T03:16:51.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280a2bc0 0x7f71280a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71280df590 0x7f712809f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f712809fd10 0x7f71280e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71280a0220 con 0x7f71280df590 2021-04-03T03:16:51.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128039da0 con 0x7f712809fd10 2021-04-03T03:16:51.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7128039a90 con 0x7f71280a2bc0 2021-04-03T03:16:51.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71280df590 0x7f712809f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71280df590 0x7f712809f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57374/0 (socket says 172.21.2.111:57374) 2021-04-03T03:16:51.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 -- 172.21.2.111:0/2449552055 learned_addr learned my addr 172.21.2.111:0/2449552055 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:51.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 -- 172.21.2.111:0/2449552055 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280a2bc0 msgr2=0x7f71280a2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280a2bc0 0x7f71280a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 -- 172.21.2.111:0/2449552055 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f712809fd10 msgr2=0x7f71280e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f712809fd10 0x7f71280e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 -- 172.21.2.111:0/2449552055 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280eaf20 con 0x7f71280df590 2021-04-03T03:16:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712e5e4700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71280df590 0x7f712809f820 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f7120008000 tx=0x7f7120009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f7127fff700 1 -- 172.21.2.111:0/2449552055 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f712000f510 con 0x7f71280df590 2021-04-03T03:16:51.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f7127fff700 1 -- 172.21.2.111:0/2449552055 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712000f670 con 0x7f71280df590 2021-04-03T03:16:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f7127fff700 1 -- 172.21.2.111:0/2449552055 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f712000f9d0 con 0x7f71280df590 2021-04-03T03:16:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7128039580 con 0x7f71280df590 2021-04-03T03:16:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71280e7310 con 0x7f71280df590 2021-04-03T03:16:51.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71280a03e0 con 0x7f71280df590 2021-04-03T03:16:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f7127fff700 1 -- 172.21.2.111:0/2449552055 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f712001a030 con 0x7f71280df590 2021-04-03T03:16:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f7127fff700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7114056f70 0x7f7114059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f7127fff700 1 -- 172.21.2.111:0/2449552055 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f712006f920 con 0x7f71280df590 2021-04-03T03:16:51.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712ede5700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7114056f70 0x7f7114059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:51.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.253+0000 7f712ede5700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7114056f70 0x7f7114059430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f7118007000 tx=0x7f7118008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:51.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.257+0000 7f7127fff700 1 -- 172.21.2.111:0/2449552055 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7120044f30 con 0x7f71280df590 2021-04-03T03:16:51.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:51.349+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f7128004db0 con 0x7f71280df590 2021-04-03T03:16:52.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f7127fff700 1 -- 172.21.2.111:0/2449552055 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v129) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7120048550 con 0x7f71280df590 2021-04-03T03:16:52.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7114056f70 msgr2=0x7f7114059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:52.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7114056f70 0x7f7114059430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f7118007000 tx=0x7f7118008040).stop 2021-04-03T03:16:52.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71280df590 msgr2=0x7f712809f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:52.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71280df590 0x7f712809f820 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f7120008000 tx=0x7f7120009040).stop 2021-04-03T03:16:52.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 shutdown_connections 2021-04-03T03:16:52.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7114056f70 0x7f7114059430 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:52.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f712809fd10 0x7f71280e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:52.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f71280a2bc0 0x7f71280a2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:52.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 --2- 172.21.2.111:0/2449552055 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f71280df590 0x7f712809f820 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:52.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 >> 172.21.2.111:0/2449552055 conn(0x7f71280462b0 msgr2=0x7f71280055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:52.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 shutdown_connections 2021-04-03T03:16:52.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.125+0000 7f712fde7700 1 -- 172.21.2.111:0/2449552055 wait complete. 2021-04-03T03:16:52.131 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:16:52.135 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:16:52.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 Processor -- start 2021-04-03T03:16:52.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- start start 2021-04-03T03:16:52.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fdd0c0dbba0 0x7fdd0c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c041060 0x7fdd0c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c0dc560 con 0x7fdd0c03d560 2021-04-03T03:16:52.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c03de60 con 0x7fdd0c0dbba0 2021-04-03T03:16:52.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0dc800 con 0x7fdd0c041060 2021-04-03T03:16:52.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd135e6700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fdd0c03d560 0x7fdd0c041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58176/0 (socket says 172.21.2.111:58176) 2021-04-03T03:16:52.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd135e6700 1 -- 172.21.2.111:0/3003670933 learned_addr learned my addr 172.21.2.111:0/3003670933 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:52.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c041060 0x7fdd0c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:52.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd12de5700 1 --2- 172.21.2.111:0/3003670933 >> v2:172.21.2.115:3300/0 conn(0x7fdd0c0dbba0 0x7fdd0c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:52.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616898156 0 0) 0x7fdd0c0dc560 con 0x7fdd0c03d560 2021-04-03T03:16:52.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf4003240 con 0x7fdd0c03d560 2021-04-03T03:16:52.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3938364659 0 0) 0x7fdcf4003240 con 0x7fdd0c03d560 2021-04-03T03:16:52.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c041060 msgr2=0x7fdd0c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:52.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c041060 0x7fdd0c03fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:52.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 >> v2:172.21.2.115:3300/0 conn(0x7fdd0c0dbba0 msgr2=0x7fdd0c0dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:52.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 --2- 172.21.2.111:0/3003670933 >> v2:172.21.2.115:3300/0 conn(0x7fdd0c0dbba0 0x7fdd0c0dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:52.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c03da30 con 0x7fdd0c03d560 2021-04-03T03:16:52.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fdd140450f0 con 0x7fdd0c03d560 2021-04-03T03:16:52.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 >> v1:172.21.2.111:6789/0 conn(0x7fdd0c03d560 legacy=0x7fdd0c041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:52.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdcf40037b0 0x7fdcf4003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdcf40044e0 0x7fdcf4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c041060 0x7fdcf4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c03de60 con 0x7fdcf40044e0 2021-04-03T03:16:52.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0dc800 con 0x7fdcf40037b0 2021-04-03T03:16:52.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4004280 con 0x7fdd0c041060 2021-04-03T03:16:52.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdcf40044e0 0x7fdcf4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:52.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 -- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c041060 msgr2=0x7fdcf4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:52.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c041060 0x7fdcf4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:52.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 -- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdcf40037b0 msgr2=0x7fdcf4003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:52.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdcf40037b0 0x7fdcf4003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:52.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 -- 172.21.2.111:0/3003670933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4003240 con 0x7fdcf40044e0 2021-04-03T03:16:52.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd125e4700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdcf40044e0 0x7fdcf4004960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fdcfc00c000 tx=0x7fdcfc008040).ready entity=mon.0 client_cookie=ce05cdfc8619244e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:52.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcfc00ebc0 con 0x7fdcf40044e0 2021-04-03T03:16:52.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc00ed20 con 0x7fdcf40044e0 2021-04-03T03:16:52.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd11de3700 1 -- 172.21.2.111:0/3003670933 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcfc0125e0 con 0x7fdcf40044e0 2021-04-03T03:16:52.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- 172.21.2.111:0/3003670933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdcf40044e0 msgr2=0x7fdcf4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:52.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 --2- 172.21.2.111:0/3003670933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdcf40044e0 0x7fdcf4004960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fdcfc00c000 tx=0x7fdcfc008040).stop 2021-04-03T03:16:52.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd135e6700 1 -- 172.21.2.111:0/3003670933 reap_dead start 2021-04-03T03:16:52.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- 172.21.2.111:0/3003670933 shutdown_connections 2021-04-03T03:16:52.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- 172.21.2.111:0/3003670933 >> 172.21.2.111:0/3003670933 conn(0x7fdd0c0462b0 msgr2=0x7fdd0c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:52.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- 172.21.2.111:0/3003670933 shutdown_connections 2021-04-03T03:16:52.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- 172.21.2.111:0/3003670933 wait complete. 2021-04-03T03:16:52.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 Processor -- start 2021-04-03T03:16:52.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.197+0000 7fdd13de7700 1 -- start start 2021-04-03T03:16:52.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd0c0dcd20 0x7fdd0c0d6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c0a30a0 0x7fdd0c0d6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd0c039580 0x7fdd0c0e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0eb170 con 0x7fdd0c039580 2021-04-03T03:16:52.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c03a040 con 0x7fdd0c0dcd20 2021-04-03T03:16:52.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0eac50 con 0x7fdd0c0a30a0 2021-04-03T03:16:52.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd0c039580 0x7fdd0c0e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:52.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd0c039580 0x7fdd0c0e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57396/0 (socket says 172.21.2.111:57396) 2021-04-03T03:16:52.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 -- 172.21.2.111:0/3834681071 learned_addr learned my addr 172.21.2.111:0/3834681071 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:52.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 -- 172.21.2.111:0/3834681071 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c0a30a0 msgr2=0x7fdd0c0d6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:52.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c0a30a0 0x7fdd0c0d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:52.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 -- 172.21.2.111:0/3834681071 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd0c0dcd20 msgr2=0x7fdd0c0d6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:52.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd0c0dcd20 0x7fdd0c0d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:52.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 -- 172.21.2.111:0/3834681071 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0dd100 con 0x7fdd0c039580 2021-04-03T03:16:52.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd135e6700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd0c039580 0x7fdd0c0e70a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fdd0c09f8b0 tx=0x7fdd1405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:52.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd0bfff700 1 -- 172.21.2.111:0/3834681071 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd1405bda0 con 0x7fdd0c039580 2021-04-03T03:16:52.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd0bfff700 1 -- 172.21.2.111:0/3834681071 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1405df60 con 0x7fdd0c039580 2021-04-03T03:16:52.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd0bfff700 1 -- 172.21.2.111:0/3834681071 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd140699f0 con 0x7fdd0c039580 2021-04-03T03:16:52.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c0d6a60 con 0x7fdd0c039580 2021-04-03T03:16:52.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c039be0 con 0x7fdd0c039580 2021-04-03T03:16:52.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c004db0 con 0x7fdd0c039580 2021-04-03T03:16:52.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd0bfff700 1 -- 172.21.2.111:0/3834681071 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdd14061070 con 0x7fdd0c039580 2021-04-03T03:16:52.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd0bfff700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcf8056f20 0x7fdcf80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:52.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd0bfff700 1 -- 172.21.2.111:0/3834681071 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5950+0+0 (secure 0 0 0) 0x7fdd140de4d0 con 0x7fdd0c039580 2021-04-03T03:16:52.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd12de5700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcf8056f20 0x7fdcf80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:52.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.201+0000 7fdd12de5700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcf8056f20 0x7fdcf80593e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fdd0400a000 tx=0x7fdd04008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:52.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.205+0000 7fdd0bfff700 1 -- 172.21.2.111:0/3834681071 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdd140b3a90 con 0x7fdd0c039580 2021-04-03T03:16:52.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:52.297+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fdd0c0083c0 con 0x7fdd0c039580 2021-04-03T03:16:53.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd0bfff700 1 -- 172.21.2.111:0/3834681071 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v130) v1 ==== 125+0+0 (secure 0 0 0) 0x7fdd140b70b0 con 0x7fdd0c039580 2021-04-03T03:16:53.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcf8056f20 msgr2=0x7fdcf80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcf8056f20 0x7fdcf80593e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fdd0400a000 tx=0x7fdd04008040).stop 2021-04-03T03:16:53.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd0c039580 msgr2=0x7fdd0c0e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd0c039580 0x7fdd0c0e70a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fdd0c09f8b0 tx=0x7fdd1405ad70).stop 2021-04-03T03:16:53.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 shutdown_connections 2021-04-03T03:16:53.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd0c039580 0x7fdd0c0e70a0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdcf8056f20 0x7fdcf80593e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd0c0dcd20 0x7fdd0c0d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 --2- 172.21.2.111:0/3834681071 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd0c0a30a0 0x7fdd0c0d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 >> 172.21.2.111:0/3834681071 conn(0x7fdd0c0462b0 msgr2=0x7fdd0c041690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 shutdown_connections 2021-04-03T03:16:53.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.137+0000 7fdd13de7700 1 -- 172.21.2.111:0/3834681071 wait complete. 2021-04-03T03:16:53.142 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:16:53.146 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:16:53.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 Processor -- start 2021-04-03T03:16:53.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- start start 2021-04-03T03:16:53.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6948039dd0 0x7f69480d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480d1c70 con 0x7f694803f880 2021-04-03T03:16:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480400d0 con 0x7f694803fc60 2021-04-03T03:16:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69480d2f00 con 0x7f6948039dd0 2021-04-03T03:16:53.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f694803f880 0x7f69480ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58192/0 (socket says 172.21.2.111:58192) 2021-04-03T03:16:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694ffff700 1 -- 172.21.2.111:0/2935423202 learned_addr learned my addr 172.21.2.111:0/2935423202 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802735737 0 0) 0x7f69480d1c70 con 0x7f694803f880 2021-04-03T03:16:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f693c003240 con 0x7f694803f880 2021-04-03T03:16:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694f7fe700 1 --2- 172.21.2.111:0/2935423202 >> v2:172.21.2.115:3300/0 conn(0x7f6948039dd0 0x7f69480d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4227630758 0 0) 0x7f693c003240 con 0x7f694803f880 2021-04-03T03:16:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 >> v2:172.21.2.115:3300/0 conn(0x7f6948039dd0 msgr2=0x7f69480d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 --2- 172.21.2.111:0/2935423202 >> v2:172.21.2.115:3300/0 conn(0x7f6948039dd0 0x7f69480d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 >> v1:172.21.2.115:6789/0 conn(0x7f694803fc60 legacy=0x7f6948005680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480d1ea0 con 0x7f694803f880 2021-04-03T03:16:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f69440016d0 con 0x7f694803f880 2021-04-03T03:16:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 >> v1:172.21.2.111:6789/0 conn(0x7f694803f880 legacy=0x7f69480ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f693c0037b0 0x7f693c003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f693c0044e0 0x7f693c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f693c004e50 0x7f693c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480d2f00 con 0x7f693c004e50 2021-04-03T03:16:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c005300 con 0x7f693c0037b0 2021-04-03T03:16:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693c00bc20 con 0x7f693c0044e0 2021-04-03T03:16:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f693c004e50 0x7f693c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 -- 172.21.2.111:0/2935423202 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f693c0044e0 msgr2=0x7f693c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f693c0044e0 0x7f693c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 -- 172.21.2.111:0/2935423202 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f693c0037b0 msgr2=0x7f693c003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f693c0037b0 0x7f693c003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 -- 172.21.2.111:0/2935423202 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480d1ea0 con 0x7f693c004e50 2021-04-03T03:16:53.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 -- 172.21.2.111:0/2935423202 reap_dead start 2021-04-03T03:16:53.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6954c45700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f693c004e50 0x7f693c00b730 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f695006b5d0 tx=0x7f695005ad70).ready entity=mon.0 client_cookie=2b2fb74c6697d136 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f695005bda0 con 0x7f693c004e50 2021-04-03T03:16:53.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695005df60 con 0x7f693c004e50 2021-04-03T03:16:53.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f694effd700 1 -- 172.21.2.111:0/2935423202 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69500699f0 con 0x7f693c004e50 2021-04-03T03:16:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- 172.21.2.111:0/2935423202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f693c004e50 msgr2=0x7f693c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f693c004e50 0x7f693c00b730 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f695006b5d0 tx=0x7f695005ad70).stop 2021-04-03T03:16:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- 172.21.2.111:0/2935423202 shutdown_connections 2021-04-03T03:16:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 --2- 172.21.2.111:0/2935423202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f693c004e50 0x7f693c00b730 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- 172.21.2.111:0/2935423202 >> 172.21.2.111:0/2935423202 conn(0x7f69480462b0 msgr2=0x7f69480466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- 172.21.2.111:0/2935423202 shutdown_connections 2021-04-03T03:16:53.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- 172.21.2.111:0/2935423202 wait complete. 2021-04-03T03:16:53.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 Processor -- start 2021-04-03T03:16:53.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.205+0000 7f6955446700 1 -- start start 2021-04-03T03:16:53.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f693c004e50 0x7f69480b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69480d5360 0x7f69480b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69480d4420 0x7f69480b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480d5e70 con 0x7f69480d5360 2021-04-03T03:16:53.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480d3cd0 con 0x7f693c004e50 2021-04-03T03:16:53.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69480d34c0 con 0x7f69480d4420 2021-04-03T03:16:53.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69480d5360 0x7f69480b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69480d5360 0x7f69480b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57414/0 (socket says 172.21.2.111:57414) 2021-04-03T03:16:53.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 -- 172.21.2.111:0/30768202 learned_addr learned my addr 172.21.2.111:0/30768202 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:53.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 -- 172.21.2.111:0/30768202 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69480d4420 msgr2=0x7f69480b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69480d4420 0x7f69480b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 -- 172.21.2.111:0/30768202 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f693c004e50 msgr2=0x7f69480b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f693c004e50 0x7f69480b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 -- 172.21.2.111:0/30768202 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480d3f00 con 0x7f69480d5360 2021-04-03T03:16:53.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694f7fe700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69480d5360 0x7f69480b3f50 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f69480e08c0 tx=0x7f6940008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694d7fa700 1 -- 172.21.2.111:0/30768202 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f694000f680 con 0x7f69480d5360 2021-04-03T03:16:53.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694d7fa700 1 -- 172.21.2.111:0/30768202 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694000f040 con 0x7f69480d5360 2021-04-03T03:16:53.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694d7fa700 1 -- 172.21.2.111:0/30768202 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f694000f9d0 con 0x7f69480d5360 2021-04-03T03:16:53.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69480d3220 con 0x7f69480d5360 2021-04-03T03:16:53.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69480b4930 con 0x7f69480d5360 2021-04-03T03:16:53.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6948004db0 con 0x7f69480d5360 2021-04-03T03:16:53.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694d7fa700 1 -- 172.21.2.111:0/30768202 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f694000fb30 con 0x7f69480d5360 2021-04-03T03:16:53.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694d7fa700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69300570a0 0x7f6930059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694d7fa700 1 -- 172.21.2.111:0/30768202 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f694006fb90 con 0x7f69480d5360 2021-04-03T03:16:53.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694ffff700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69300570a0 0x7f6930059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.209+0000 7f694ffff700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69300570a0 0x7f6930059560 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f6944000fe0 tx=0x7f6944001aa0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.213+0000 7f694d7fa700 1 -- 172.21.2.111:0/30768202 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6940044ee0 con 0x7f69480d5360 2021-04-03T03:16:53.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.337+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f694803aba0 con 0x7f69480d5360 2021-04-03T03:16:53.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f694d7fa700 1 -- 172.21.2.111:0/30768202 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v57) v1 ==== 189+0+0 (secure 0 0 0) 0x7f6940048500 con 0x7f69480d5360 2021-04-03T03:16:53.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69300570a0 msgr2=0x7f6930059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69300570a0 0x7f6930059560 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f6944000fe0 tx=0x7f6944001aa0).stop 2021-04-03T03:16:53.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69480d5360 msgr2=0x7f69480b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69480d5360 0x7f69480b3f50 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f69480e08c0 tx=0x7f6940008040).stop 2021-04-03T03:16:53.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 shutdown_connections 2021-04-03T03:16:53.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69480d4420 0x7f69480b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69300570a0 0x7f6930059560 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f693c004e50 0x7f69480b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 --2- 172.21.2.111:0/30768202 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69480d5360 0x7f69480b3f50 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 >> 172.21.2.111:0/30768202 conn(0x7f69480462b0 msgr2=0x7f6948005e90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 shutdown_connections 2021-04-03T03:16:53.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.341+0000 7f6955446700 1 -- 172.21.2.111:0/30768202 wait complete. 2021-04-03T03:16:53.348 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 25 and data pool 26 2021-04-03T03:16:53.352 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:53.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.405+0000 7f0ecd854700 1 Processor -- start 2021-04-03T03:16:53.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.405+0000 7f0ecd854700 1 -- start start 2021-04-03T03:16:53.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec0006a30 0x7f0ec0006260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec003fb20 con 0x7f0ec0005b30 2021-04-03T03:16:53.414 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec003fdc0 con 0x7f0ec00056b0 2021-04-03T03:16:53.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0040060 con 0x7f0ec0006a30 2021-04-03T03:16:53.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd053700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0ec0005b30 0x7f0ec00dac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58224/0 (socket says 172.21.2.111:58224) 2021-04-03T03:16:53.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd053700 1 -- 172.21.2.111:0/3600537617 learned_addr learned my addr 172.21.2.111:0/3600537617 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:53.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec0006a30 0x7f0ec0006260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608751031 0 0) 0x7f0ec003fb20 con 0x7f0ec0005b30 2021-04-03T03:16:53.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eac003240 con 0x7f0ec0005b30 2021-04-03T03:16:53.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1154502685 0 0) 0x7f0eac003240 con 0x7f0ec0005b30 2021-04-03T03:16:53.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec0006a30 msgr2=0x7f0ec0006260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec0006a30 0x7f0ec0006260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 >> v1:172.21.2.115:6789/0 conn(0x7f0ec00056b0 legacy=0x7f0ec00d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0040500 con 0x7f0ec0005b30 2021-04-03T03:16:53.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f0ebc0016d0 con 0x7f0ec0005b30 2021-04-03T03:16:53.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 >> v1:172.21.2.111:6789/0 conn(0x7f0ec0005b30 legacy=0x7f0ec00dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0eac003760 0x7f0eac003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec0006a30 0x7f0eac0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0eac0049d0 0x7f0eac004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0040060 con 0x7f0eac003760 2021-04-03T03:16:53.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac004250 con 0x7f0eac0049d0 2021-04-03T03:16:53.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac00d940 con 0x7f0ec0006a30 2021-04-03T03:16:53.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0eac003760 0x7f0eac003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 -- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec0006a30 msgr2=0x7f0eac0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec0006a30 0x7f0eac0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 -- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0eac0049d0 msgr2=0x7f0eac004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0eac0049d0 0x7f0eac004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 -- 172.21.2.111:0/3600537617 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac003240 con 0x7f0eac003760 2021-04-03T03:16:53.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0eac003760 0x7f0eac003b80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4008040).ready entity=mon.0 client_cookie=cbccf18048d5b089 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb400ebc0 con 0x7f0eac003760 2021-04-03T03:16:53.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb400ed20 con 0x7f0eac003760 2021-04-03T03:16:53.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec77fe700 1 -- 172.21.2.111:0/3600537617 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb40125e0 con 0x7f0eac003760 2021-04-03T03:16:53.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- 172.21.2.111:0/3600537617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0eac003760 msgr2=0x7f0eac003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 --2- 172.21.2.111:0/3600537617 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0eac003760 0x7f0eac003b80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4008040).stop 2021-04-03T03:16:53.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd053700 1 -- 172.21.2.111:0/3600537617 reap_dead start 2021-04-03T03:16:53.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- 172.21.2.111:0/3600537617 shutdown_connections 2021-04-03T03:16:53.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- 172.21.2.111:0/3600537617 >> 172.21.2.111:0/3600537617 conn(0x7f0ec00462c0 msgr2=0x7f0ec00466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- 172.21.2.111:0/3600537617 shutdown_connections 2021-04-03T03:16:53.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- 172.21.2.111:0/3600537617 wait complete. 2021-04-03T03:16:53.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 Processor -- start 2021-04-03T03:16:53.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- start start 2021-04-03T03:16:53.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ec00d0430 0x7f0ec00c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ec009fb20 0x7f0ec00e2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec00a0070 0x7f0ec00e2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00c1910 con 0x7f0ec00d0430 2021-04-03T03:16:53.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00a1010 con 0x7f0ec009fb20 2021-04-03T03:16:53.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00a1520 con 0x7f0ec00a0070 2021-04-03T03:16:53.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ec00d0430 0x7f0ec00c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ec00d0430 0x7f0ec00c1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57440/0 (socket says 172.21.2.111:57440) 2021-04-03T03:16:53.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 -- 172.21.2.111:0/4035948333 learned_addr learned my addr 172.21.2.111:0/4035948333 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:53.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 -- 172.21.2.111:0/4035948333 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec00a0070 msgr2=0x7f0ec00e2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec00a0070 0x7f0ec00e2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 -- 172.21.2.111:0/4035948333 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ec009fb20 msgr2=0x7f0ec00e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ec009fb20 0x7f0ec00e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 -- 172.21.2.111:0/4035948333 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00d0810 con 0x7f0ec00d0430 2021-04-03T03:16:53.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecc852700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ec00d0430 0x7f0ec00c1d70 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec5ffb700 1 -- 172.21.2.111:0/4035948333 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb401c450 con 0x7f0ec00d0430 2021-04-03T03:16:53.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ec5ffb700 1 -- 172.21.2.111:0/4035948333 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb401c5b0 con 0x7f0ec00d0430 2021-04-03T03:16:53.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.409+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec00a1280 con 0x7f0ec00d0430 2021-04-03T03:16:53.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec00c0c00 con 0x7f0ec00d0430 2021-04-03T03:16:53.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ec5ffb700 1 -- 172.21.2.111:0/4035948333 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb401c910 con 0x7f0ec00d0430 2021-04-03T03:16:53.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec0004da0 con 0x7f0ec00d0430 2021-04-03T03:16:53.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ec5ffb700 1 -- 172.21.2.111:0/4035948333 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0eb402b070 con 0x7f0ec00d0430 2021-04-03T03:16:53.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ec5ffb700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0eb0056f70 0x7f0eb0059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ec5ffb700 1 -- 172.21.2.111:0/4035948333 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0eb4071d50 con 0x7f0ec00d0430 2021-04-03T03:16:53.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ec7fff700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0eb0056f70 0x7f0eb0059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ec7fff700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0eb0056f70 0x7f0eb0059430 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f0ec000c360 tx=0x7f0ec805ad70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.413+0000 7f0ec5ffb700 1 -- 172.21.2.111:0/4035948333 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0eb4047030 con 0x7f0ec00d0430 2021-04-03T03:16:53.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ec0046030 con 0x7f0ec00d0430 2021-04-03T03:16:53.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ec5ffb700 1 -- 172.21.2.111:0/4035948333 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f0eb404a650 con 0x7f0ec00d0430 2021-04-03T03:16:53.512 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:53.512 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":131,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:53.339226+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:51.364989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:52.312016+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:53.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0eb0056f70 msgr2=0x7f0eb0059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0eb0056f70 0x7f0eb0059430 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f0ec000c360 tx=0x7f0ec805ad70).stop 2021-04-03T03:16:53.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ec00d0430 msgr2=0x7f0ec00c1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ec00d0430 0x7f0ec00c1d70 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4019000).stop 2021-04-03T03:16:53.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 shutdown_connections 2021-04-03T03:16:53.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ec00a0070 0x7f0ec00e2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ec00d0430 0x7f0ec00c1d70 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0eb0056f70 0x7f0eb0059430 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 --2- 172.21.2.111:0/4035948333 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ec009fb20 0x7f0ec00e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 >> 172.21.2.111:0/4035948333 conn(0x7f0ec00462c0 msgr2=0x7f0ec00ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 shutdown_connections 2021-04-03T03:16:53.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.509+0000 7f0ecd854700 1 -- 172.21.2.111:0/4035948333 wait complete. 2021-04-03T03:16:53.520 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:53.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.577+0000 7f74e4f06700 1 Processor -- start 2021-04-03T03:16:53.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.577+0000 7f74e4f06700 1 -- start start 2021-04-03T03:16:53.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e00603f0 0x7f74e01450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f74e00af450 0x7f74e00b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e00677d0 con 0x7f74e0062ad0 2021-04-03T03:16:53.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e01455d0 con 0x7f74e00af450 2021-04-03T03:16:53.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0141070 con 0x7f74e00603f0 2021-04-03T03:16:53.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74df7fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e00603f0 0x7f74e01450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74dffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f74e00af450 0x7f74e00b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74deffd700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f74e0062ad0 0x7f74e00aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:42548/0 (socket says 172.21.2.111:42548) 2021-04-03T03:16:53.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74deffd700 1 -- 172.21.2.111:0/1860187263 learned_addr learned my addr 172.21.2.111:0/1860187263 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:53.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74dffff700 1 -- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e00603f0 msgr2=0x7f74e01450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74dffff700 1 --2- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e00603f0 0x7f74e01450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74dffff700 1 -- 172.21.2.111:0/1860187263 >> v1:172.21.2.115:6789/0 conn(0x7f74e0062ad0 legacy=0x7f74e00aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74dffff700 1 -- 172.21.2.111:0/1860187263 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00607d0 con 0x7f74e00af450 2021-04-03T03:16:53.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74df7fe700 1 --2- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e00603f0 0x7f74e01450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:16:53.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74dffff700 1 --2- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 0x7f74e00b09f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f74d80032b0 tx=0x7f74d800d660).ready entity=mon.1 client_cookie=f2d0490f459bd82f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74de7fc700 1 -- 172.21.2.111:0/1860187263 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d8012070 con 0x7f74e00af450 2021-04-03T03:16:53.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74de7fc700 1 -- 172.21.2.111:0/1860187263 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800fd20 con 0x7f74e00af450 2021-04-03T03:16:53.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74de7fc700 1 -- 172.21.2.111:0/1860187263 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d8017660 con 0x7f74e00af450 2021-04-03T03:16:53.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 msgr2=0x7f74e00b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 0x7f74e00b09f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f74d80032b0 tx=0x7f74d800d660).stop 2021-04-03T03:16:53.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- 172.21.2.111:0/1860187263 shutdown_connections 2021-04-03T03:16:53.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 0x7f74e00b09f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1860187263 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e00603f0 0x7f74e01450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- 172.21.2.111:0/1860187263 >> 172.21.2.111:0/1860187263 conn(0x7f74e00b61c0 msgr2=0x7f74e005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- 172.21.2.111:0/1860187263 shutdown_connections 2021-04-03T03:16:53.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- 172.21.2.111:0/1860187263 wait complete. 2021-04-03T03:16:53.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 Processor -- start 2021-04-03T03:16:53.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.581+0000 7f74e4f06700 1 -- start start 2021-04-03T03:16:53.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74e00603f0 0x7f74e009eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e0062ad0 0x7f74e00abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 0x7f74e00ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0064440 con 0x7f74e00603f0 2021-04-03T03:16:53.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00646e0 con 0x7f74e00af450 2021-04-03T03:16:53.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00656c0 con 0x7f74e0062ad0 2021-04-03T03:16:53.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74e00603f0 0x7f74e009eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74e00603f0 0x7f74e009eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57456/0 (socket says 172.21.2.111:57456) 2021-04-03T03:16:53.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 -- 172.21.2.111:0/1171662908 learned_addr learned my addr 172.21.2.111:0/1171662908 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:53.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74dffff700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 0x7f74e00ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74deffd700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e0062ad0 0x7f74e00abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 -- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e0062ad0 msgr2=0x7f74e00abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e0062ad0 0x7f74e00abc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 -- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 msgr2=0x7f74e00ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 0x7f74e00ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 -- 172.21.2.111:0/1171662908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00ac5e0 con 0x7f74e00603f0 2021-04-03T03:16:53.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74df7fe700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74e00603f0 0x7f74e009eb50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f74c800c000 tx=0x7f74c8006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74dcff9700 1 -- 172.21.2.111:0/1171662908 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c800cda0 con 0x7f74e00603f0 2021-04-03T03:16:53.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74dcff9700 1 -- 172.21.2.111:0/1171662908 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c80123f0 con 0x7f74e00603f0 2021-04-03T03:16:53.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e00640c0 con 0x7f74e00603f0 2021-04-03T03:16:53.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74dcff9700 1 -- 172.21.2.111:0/1171662908 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c8012750 con 0x7f74e00603f0 2021-04-03T03:16:53.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e009f420 con 0x7f74e00603f0 2021-04-03T03:16:53.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e0144f80 con 0x7f74e00603f0 2021-04-03T03:16:53.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74dcff9700 1 -- 172.21.2.111:0/1171662908 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f74c80128b0 con 0x7f74e00603f0 2021-04-03T03:16:53.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74dcff9700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f74c4056fe0 0x7f74c40594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74dcff9700 1 -- 172.21.2.111:0/1171662908 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f74c8070900 con 0x7f74e00603f0 2021-04-03T03:16:53.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74deffd700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f74c4056fe0 0x7f74c40594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.585+0000 7f74deffd700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f74c4056fe0 0x7f74c40594a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f74d0008000 tx=0x7f74d0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.589+0000 7f74dcff9700 1 -- 172.21.2.111:0/1171662908 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f74c8045cf0 con 0x7f74e00603f0 2021-04-03T03:16:53.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.677+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74e0065880 con 0x7f74e00603f0 2021-04-03T03:16:53.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.677+0000 7f74dcff9700 1 -- 172.21.2.111:0/1171662908 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f74c8049310 con 0x7f74e00603f0 2021-04-03T03:16:53.681 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:53.682 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":131,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:53.339226+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:51.364989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:52.312016+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:53.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f74c4056fe0 msgr2=0x7f74c40594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f74c4056fe0 0x7f74c40594a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f74d0008000 tx=0x7f74d0009040).stop 2021-04-03T03:16:53.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74e00603f0 msgr2=0x7f74e009eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74e00603f0 0x7f74e009eb50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f74c800c000 tx=0x7f74c8006dd0).stop 2021-04-03T03:16:53.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 shutdown_connections 2021-04-03T03:16:53.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f74c4056fe0 0x7f74c40594a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f74e00af450 0x7f74e00ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f74e00603f0 0x7f74e009eb50 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 --2- 172.21.2.111:0/1171662908 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f74e0062ad0 0x7f74e00abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 >> 172.21.2.111:0/1171662908 conn(0x7f74e00b61c0 msgr2=0x7f74e00b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 shutdown_connections 2021-04-03T03:16:53.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.681+0000 7f74e4f06700 1 -- 172.21.2.111:0/1171662908 wait complete. 2021-04-03T03:16:53.689 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:16:53.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 Processor -- start 2021-04-03T03:16:53.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- start start 2021-04-03T03:16:53.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 0x7f26801447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f26800b17b0 0x7f2680144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26800607d0 con 0x7f26800add50 2021-04-03T03:16:53.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26801451c0 con 0x7f26800b17b0 2021-04-03T03:16:53.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268014b090 con 0x7f26800603f0 2021-04-03T03:16:53.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f26800b17b0 0x7f2680144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f26800b17b0 0x7f2680144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57464/0 (socket says 172.21.2.111:57464) 2021-04-03T03:16:53.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 -- 172.21.2.111:0/2544645975 learned_addr learned my addr 172.21.2.111:0/2544645975 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:53.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 -- 172.21.2.111:0/2544645975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 msgr2=0x7f26801447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 --2- 172.21.2.111:0/2544645975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 0x7f26801447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 -- 172.21.2.111:0/2544645975 >> v1:172.21.2.115:6789/0 conn(0x7f26800add50 legacy=0x7f2680147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:53.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 -- 172.21.2.111:0/2544645975 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268014b330 con 0x7f26800b17b0 2021-04-03T03:16:53.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267eb85700 1 --2- 172.21.2.111:0/2544645975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800b17b0 0x7f2680144cd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f26800b5600 tx=0x7f2678009270).ready entity=mon.0 client_cookie=4d517fa20de020bc server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267d382700 1 -- 172.21.2.111:0/2544645975 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f267800bbc0 con 0x7f26800b17b0 2021-04-03T03:16:53.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267d382700 1 -- 172.21.2.111:0/2544645975 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267800bd20 con 0x7f26800b17b0 2021-04-03T03:16:53.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267d382700 1 -- 172.21.2.111:0/2544645975 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2678011400 con 0x7f26800b17b0 2021-04-03T03:16:53.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- 172.21.2.111:0/2544645975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800b17b0 msgr2=0x7f2680144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 --2- 172.21.2.111:0/2544645975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800b17b0 0x7f2680144cd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f26800b5600 tx=0x7f2678009270).stop 2021-04-03T03:16:53.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- 172.21.2.111:0/2544645975 shutdown_connections 2021-04-03T03:16:53.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 --2- 172.21.2.111:0/2544645975 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800b17b0 0x7f2680144cd0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:53.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 --2- 172.21.2.111:0/2544645975 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 0x7f26801447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- 172.21.2.111:0/2544645975 >> 172.21.2.111:0/2544645975 conn(0x7f26800b61c0 msgr2=0x7f268005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:53.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- 172.21.2.111:0/2544645975 shutdown_connections 2021-04-03T03:16:53.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.745+0000 7f267f386700 1 -- 172.21.2.111:0/2544645975 wait complete. 2021-04-03T03:16:53.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 Processor -- start 2021-04-03T03:16:53.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 -- start start 2021-04-03T03:16:53.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 0x7f268014a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800add50 0x7f2680143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26800b17b0 0x7f2680143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800ab950 con 0x7f26800add50 2021-04-03T03:16:53.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800aa7c0 con 0x7f26800b17b0 2021-04-03T03:16:53.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26800aac60 con 0x7f26800603f0 2021-04-03T03:16:53.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800add50 0x7f2680143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800add50 0x7f2680143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57468/0 (socket says 172.21.2.111:57468) 2021-04-03T03:16:53.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 -- 172.21.2.111:0/3222322683 learned_addr learned my addr 172.21.2.111:0/3222322683 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:53.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 -- 172.21.2.111:0/3222322683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 msgr2=0x7f268014a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 0x7f268014a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 -- 172.21.2.111:0/3222322683 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26800b17b0 msgr2=0x7f2680143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:53.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26800b17b0 0x7f2680143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:53.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 -- 172.21.2.111:0/3222322683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26800ab070 con 0x7f26800add50 2021-04-03T03:16:53.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267db83700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800add50 0x7f2680143a20 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f2668002f50 tx=0x7f2668006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f26777fe700 1 -- 172.21.2.111:0/3222322683 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266800d4a0 con 0x7f26800add50 2021-04-03T03:16:53.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f26777fe700 1 -- 172.21.2.111:0/3222322683 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266800d600 con 0x7f26800add50 2021-04-03T03:16:53.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f26777fe700 1 -- 172.21.2.111:0/3222322683 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266800d960 con 0x7f26800add50 2021-04-03T03:16:53.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2680144400 con 0x7f26800add50 2021-04-03T03:16:53.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26800ab6c0 con 0x7f26800add50 2021-04-03T03:16:53.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26800ac070 con 0x7f26800add50 2021-04-03T03:16:53.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f26777fe700 1 -- 172.21.2.111:0/3222322683 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f266801a060 con 0x7f26800add50 2021-04-03T03:16:53.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f26777fe700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2664056f10 0x7f26640593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:53.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f26777fe700 1 -- 172.21.2.111:0/3222322683 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f266806fbc0 con 0x7f26800add50 2021-04-03T03:16:53.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267e384700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2664056f10 0x7f26640593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:53.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.749+0000 7f267e384700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2664056f10 0x7f26640593d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f2670002f30 tx=0x7f2670006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:53.763 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.753+0000 7f26777fe700 1 -- 172.21.2.111:0/3222322683 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2668044ea0 con 0x7f26800add50 2021-04-03T03:16:53.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:53.877+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f26800aa980 con 0x7f26800add50 2021-04-03T03:16:54.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f26777fe700 1 -- 172.21.2.111:0/3222322683 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2668018020 con 0x7f26800add50 2021-04-03T03:16:54.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2664056f10 msgr2=0x7f26640593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2664056f10 0x7f26640593d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f2670002f30 tx=0x7f2670006a60).stop 2021-04-03T03:16:54.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800add50 msgr2=0x7f2680143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800add50 0x7f2680143a20 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f2668002f50 tx=0x7f2668006dc0).stop 2021-04-03T03:16:54.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 shutdown_connections 2021-04-03T03:16:54.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2664056f10 0x7f26640593d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f26800b17b0 0x7f2680143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f26800603f0 0x7f268014a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 --2- 172.21.2.111:0/3222322683 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f26800add50 0x7f2680143a20 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 >> 172.21.2.111:0/3222322683 conn(0x7f26800b61c0 msgr2=0x7f2680149830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:54.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 shutdown_connections 2021-04-03T03:16:54.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.353+0000 7f267f386700 1 -- 172.21.2.111:0/3222322683 wait complete. 2021-04-03T03:16:54.362 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:54.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 Processor -- start 2021-04-03T03:16:54.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- start start 2021-04-03T03:16:54.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00603f0 0x7f2af00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f2af00b0c90 0x7f2af0147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af00607d0 con 0x7f2af014b700 2021-04-03T03:16:54.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00af400 con 0x7f2af00b0c90 2021-04-03T03:16:54.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00af6a0 con 0x7f2af00603f0 2021-04-03T03:16:54.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aee7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f2af014b700 0x7f2af00b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58272/0 (socket says 172.21.2.111:58272) 2021-04-03T03:16:54.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aee7fc700 1 -- 172.21.2.111:0/3197337800 learned_addr learned my addr 172.21.2.111:0/3197337800 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:54.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00603f0 0x7f2af00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746266400 0 0) 0x7f2af00607d0 con 0x7f2af014b700 2021-04-03T03:16:54.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aef7fe700 1 --2- 172.21.2.111:0/3197337800 >> v2:172.21.2.115:3300/0 conn(0x7f2af00b0c90 0x7f2af0147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad0003240 con 0x7f2af014b700 2021-04-03T03:16:54.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 564670687 0 0) 0x7f2ad0003240 con 0x7f2af014b700 2021-04-03T03:16:54.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00603f0 msgr2=0x7f2af00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:54.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00603f0 0x7f2af00b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 >> v2:172.21.2.115:3300/0 conn(0x7f2af00b0c90 msgr2=0x7f2af0147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:54.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 --2- 172.21.2.111:0/3197337800 >> v2:172.21.2.115:3300/0 conn(0x7f2af00b0c90 0x7f2af0147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00afb40 con 0x7f2af014b700 2021-04-03T03:16:54.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f2ae8001a50 con 0x7f2af014b700 2021-04-03T03:16:54.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 >> v1:172.21.2.111:6789/0 conn(0x7f2af014b700 legacy=0x7f2af00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ad0003730 0x7f2ad0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ad00044e0 0x7f2ad0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2ad0004e50 0x7f2ad000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00af400 con 0x7f2ad0003730 2021-04-03T03:16:54.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00af6a0 con 0x7f2ad0004e50 2021-04-03T03:16:54.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00041c0 con 0x7f2ad00044e0 2021-04-03T03:16:54.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ad0003730 0x7f2ad0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aee7fc700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ad00044e0 0x7f2ad0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 -- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ad00044e0 msgr2=0x7f2ad0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:54.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ad00044e0 0x7f2ad0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 -- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2ad0004e50 msgr2=0x7f2ad000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:54.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2ad0004e50 0x7f2ad000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 -- 172.21.2.111:0/3197337800 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0003240 con 0x7f2ad0003730 2021-04-03T03:16:54.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aee7fc700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2ad00044e0 0x7f2ad0004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:54.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ad0003730 0x7f2ad0003b50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f2ae000a000 tx=0x7f2ae0008040).ready entity=mon.0 client_cookie=68a512b5384bbae7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae000b9e0 con 0x7f2ad0003730 2021-04-03T03:16:54.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae000bb40 con 0x7f2ad0003730 2021-04-03T03:16:54.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aedffb700 1 -- 172.21.2.111:0/3197337800 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae000bea0 con 0x7f2ad0003730 2021-04-03T03:16:54.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- 172.21.2.111:0/3197337800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ad0003730 msgr2=0x7f2ad0003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 --2- 172.21.2.111:0/3197337800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2ad0003730 0x7f2ad0003b50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f2ae000a000 tx=0x7f2ae0008040).stop 2021-04-03T03:16:54.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aef7fe700 1 -- 172.21.2.111:0/3197337800 reap_dead start 2021-04-03T03:16:54.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- 172.21.2.111:0/3197337800 shutdown_connections 2021-04-03T03:16:54.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- 172.21.2.111:0/3197337800 >> 172.21.2.111:0/3197337800 conn(0x7f2af00b61c0 msgr2=0x7f2af005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:54.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- 172.21.2.111:0/3197337800 shutdown_connections 2021-04-03T03:16:54.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.421+0000 7f2aeffff700 1 -- 172.21.2.111:0/3197337800 wait complete. 2021-04-03T03:16:54.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 Processor -- start 2021-04-03T03:16:54.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 -- start start 2021-04-03T03:16:54.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2af014a630 0x7f2af0156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00abac0 0x7f2af0064030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2af0064520 0x7f2af0066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af014aa10 con 0x7f2af0064520 2021-04-03T03:16:54.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00aa050 con 0x7f2af014a630 2021-04-03T03:16:54.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00aa2f0 con 0x7f2af00abac0 2021-04-03T03:16:54.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2af0064520 0x7f2af0066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2af0064520 0x7f2af0066d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57492/0 (socket says 172.21.2.111:57492) 2021-04-03T03:16:54.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 -- 172.21.2.111:0/2103042531 learned_addr learned my addr 172.21.2.111:0/2103042531 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:54.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2af014a630 0x7f2af0156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 -- 172.21.2.111:0/2103042531 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00abac0 msgr2=0x7f2af0064030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00abac0 0x7f2af0064030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 -- 172.21.2.111:0/2103042531 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2af014a630 msgr2=0x7f2af0156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2af014a630 0x7f2af0156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 -- 172.21.2.111:0/2103042531 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00a9bb0 con 0x7f2af0064520 2021-04-03T03:16:54.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aef7fe700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2af0064520 0x7f2af0066d50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f2ad8000b20 tx=0x7f2ad8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2ae7fff700 1 -- 172.21.2.111:0/2103042531 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ad80106e0 con 0x7f2af0064520 2021-04-03T03:16:54.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2ae7fff700 1 -- 172.21.2.111:0/2103042531 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8010840 con 0x7f2af0064520 2021-04-03T03:16:54.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af0062740 con 0x7f2af0064520 2021-04-03T03:16:54.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2ae7fff700 1 -- 172.21.2.111:0/2103042531 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ad8010ba0 con 0x7f2af0064520 2021-04-03T03:16:54.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af0064b30 con 0x7f2af0064520 2021-04-03T03:16:54.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af00a9d40 con 0x7f2af0064520 2021-04-03T03:16:54.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2ae7fff700 1 -- 172.21.2.111:0/2103042531 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f2ad801a030 con 0x7f2af0064520 2021-04-03T03:16:54.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2ae7fff700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ad4056fe0 0x7f2ad40594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2ae7fff700 1 -- 172.21.2.111:0/2103042531 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f2ad806fa90 con 0x7f2af0064520 2021-04-03T03:16:54.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ad4056fe0 0x7f2ad40594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.425+0000 7f2aeeffd700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ad4056fe0 0x7f2ad40594a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f2ae0014000 tx=0x7f2ae0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.429+0000 7f2ae7fff700 1 -- 172.21.2.111:0/2103042531 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2ad8044e80 con 0x7f2af0064520 2021-04-03T03:16:54.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.553+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2af0063d20 con 0x7f2af0064520 2021-04-03T03:16:54.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2ae7fff700 1 -- 172.21.2.111:0/2103042531 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f2ad80484a0 con 0x7f2af0064520 2021-04-03T03:16:54.560 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:54.560 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":58,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3599494763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3599494763},{"type":"v1","addr":"172.21.2.115:6833","nonce":3599494763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:53.339332+0000","modified":"2021-04-03T03:16:54.343270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7890":{"gid":7890,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/233883066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":233883066},{"type":"v1","addr":"172.21.2.111:6835","nonce":233883066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-04-03T03:16:54.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ad4056fe0 msgr2=0x7f2ad40594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ad4056fe0 0x7f2ad40594a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f2ae0014000 tx=0x7f2ae0008040).stop 2021-04-03T03:16:54.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2af0064520 msgr2=0x7f2af0066d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2af0064520 0x7f2af0066d50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f2ad8000b20 tx=0x7f2ad8008040).stop 2021-04-03T03:16:54.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 shutdown_connections 2021-04-03T03:16:54.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2af0064520 0x7f2af0066d50 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2ad4056fe0 0x7f2ad40594a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2af014a630 0x7f2af0156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 --2- 172.21.2.111:0/2103042531 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f2af00abac0 0x7f2af0064030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 >> 172.21.2.111:0/2103042531 conn(0x7f2af00b61c0 msgr2=0x7f2af00b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:54.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 shutdown_connections 2021-04-03T03:16:54.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.557+0000 7f2aeffff700 1 -- 172.21.2.111:0/2103042531 wait complete. 2021-04-03T03:16:54.564 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 58 2021-04-03T03:16:54.566 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:54.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.621+0000 7fe3bc930700 1 Processor -- start 2021-04-03T03:16:54.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.621+0000 7fe3bc930700 1 -- start start 2021-04-03T03:16:54.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe3b80603f0 0x7fe3b80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b8147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80607d0 con 0x7fe3b814b700 2021-04-03T03:16:54.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80af400 con 0x7fe3b80603f0 2021-04-03T03:16:54.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80af6a0 con 0x7fe3b80b0c90 2021-04-03T03:16:54.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe3b80603f0 0x7fe3b80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe3b80603f0 0x7fe3b80b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57496/0 (socket says 172.21.2.111:57496) 2021-04-03T03:16:54.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 -- 172.21.2.111:0/1609974829 learned_addr learned my addr 172.21.2.111:0/1609974829 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:54.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b6ffd700 1 --1- 172.21.2.111:0/1609974829 >> v1:172.21.2.111:6789/0 conn(0x7fe3b814b700 0x7fe3b80b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58290/0 (socket says 172.21.2.111:58290) 2021-04-03T03:16:54.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 --2- 172.21.2.111:0/1609974829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b8147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 -- 172.21.2.111:0/1609974829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 msgr2=0x7fe3b8147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:54.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 --2- 172.21.2.111:0/1609974829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b8147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 -- 172.21.2.111:0/1609974829 >> v1:172.21.2.111:6789/0 conn(0x7fe3b814b700 legacy=0x7fe3b80b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 -- 172.21.2.111:0/1609974829 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80afb40 con 0x7fe3b80603f0 2021-04-03T03:16:54.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b77fe700 1 --2- 172.21.2.111:0/1609974829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b80603f0 0x7fe3b80b0090 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7fe3a0002f30 tx=0x7fe3a0006e90).ready entity=mon.0 client_cookie=1f3c133cc53770a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b67fc700 1 -- 172.21.2.111:0/1609974829 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3a0009970 con 0x7fe3b80603f0 2021-04-03T03:16:54.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b67fc700 1 -- 172.21.2.111:0/1609974829 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0009ad0 con 0x7fe3b80603f0 2021-04-03T03:16:54.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b67fc700 1 -- 172.21.2.111:0/1609974829 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3a0009e30 con 0x7fe3b80603f0 2021-04-03T03:16:54.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- 172.21.2.111:0/1609974829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b80603f0 msgr2=0x7fe3b80b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- 172.21.2.111:0/1609974829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b80603f0 0x7fe3b80b0090 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7fe3a0002f30 tx=0x7fe3a0006e90).stop 2021-04-03T03:16:54.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- 172.21.2.111:0/1609974829 shutdown_connections 2021-04-03T03:16:54.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- 172.21.2.111:0/1609974829 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b8147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- 172.21.2.111:0/1609974829 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b80603f0 0x7fe3b80b0090 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- 172.21.2.111:0/1609974829 >> 172.21.2.111:0/1609974829 conn(0x7fe3b80b61c0 msgr2=0x7fe3b805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:54.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- 172.21.2.111:0/1609974829 shutdown_connections 2021-04-03T03:16:54.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- 172.21.2.111:0/1609974829 wait complete. 2021-04-03T03:16:54.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 Processor -- start 2021-04-03T03:16:54.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- start start 2021-04-03T03:16:54.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe3b80603f0 0x7fe3b80a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b80aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b814b700 0x7fe3b81565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80ab200 con 0x7fe3b814b700 2021-04-03T03:16:54.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8064320 con 0x7fe3b80603f0 2021-04-03T03:16:54.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3bc930700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80ab3f0 con 0x7fe3b80b0c90 2021-04-03T03:16:54.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b814b700 0x7fe3b81565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b814b700 0x7fe3b81565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57506/0 (socket says 172.21.2.111:57506) 2021-04-03T03:16:54.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 -- 172.21.2.111:0/173473692 learned_addr learned my addr 172.21.2.111:0/173473692 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:54.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b6ffd700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b80aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 -- 172.21.2.111:0/173473692 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 msgr2=0x7fe3b80aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b80aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 -- 172.21.2.111:0/173473692 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe3b80603f0 msgr2=0x7fe3b80a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe3b80603f0 0x7fe3b80a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 -- 172.21.2.111:0/173473692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80ab690 con 0x7fe3b814b700 2021-04-03T03:16:54.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b7fff700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b814b700 0x7fe3b81565f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7fe3b8063380 tx=0x7fe3b0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b4ff9700 1 -- 172.21.2.111:0/173473692 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b000bad0 con 0x7fe3b814b700 2021-04-03T03:16:54.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.625+0000 7fe3b4ff9700 1 -- 172.21.2.111:0/173473692 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b000bc30 con 0x7fe3b814b700 2021-04-03T03:16:54.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3b4ff9700 1 -- 172.21.2.111:0/173473692 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b0012420 con 0x7fe3b814b700 2021-04-03T03:16:54.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b80a9170 con 0x7fe3b814b700 2021-04-03T03:16:54.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b8064000 con 0x7fe3b814b700 2021-04-03T03:16:54.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b80aa770 con 0x7fe3b814b700 2021-04-03T03:16:54.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3b4ff9700 1 -- 172.21.2.111:0/173473692 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe3b0012580 con 0x7fe3b814b700 2021-04-03T03:16:54.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3b4ff9700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe39c056ec0 0x7fe39c059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3b4ff9700 1 -- 172.21.2.111:0/173473692 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe3b0070960 con 0x7fe3b814b700 2021-04-03T03:16:54.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3b77fe700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe39c056ec0 0x7fe39c059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3b77fe700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe39c056ec0 0x7fe39c059380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fe3a0002f30 tx=0x7fe3a0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.629+0000 7fe3b4ff9700 1 -- 172.21.2.111:0/173473692 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe3b0045cf0 con 0x7fe3b814b700 2021-04-03T03:16:54.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.721+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3b8057660 con 0x7fe3b814b700 2021-04-03T03:16:54.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3b4ff9700 1 -- 172.21.2.111:0/173473692 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fe3b0049310 con 0x7fe3b814b700 2021-04-03T03:16:54.725 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:54.725 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":132,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:54.343386+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:51.364989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:52.312016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:54.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe39c056ec0 msgr2=0x7fe39c059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe39c056ec0 0x7fe39c059380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fe3a0002f30 tx=0x7fe3a0015000).stop 2021-04-03T03:16:54.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b814b700 msgr2=0x7fe3b81565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b814b700 0x7fe3b81565f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7fe3b8063380 tx=0x7fe3b0008040).stop 2021-04-03T03:16:54.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 shutdown_connections 2021-04-03T03:16:54.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe3b814b700 0x7fe3b81565f0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe39c056ec0 0x7fe39c059380 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe3b80603f0 0x7fe3b80a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 --2- 172.21.2.111:0/173473692 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe3b80b0c90 0x7fe3b80aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 >> 172.21.2.111:0/173473692 conn(0x7fe3b80b61c0 msgr2=0x7fe3b8147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:54.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 shutdown_connections 2021-04-03T03:16:54.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.725+0000 7fe3bc930700 1 -- 172.21.2.111:0/173473692 wait complete. 2021-04-03T03:16:54.733 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:54.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 Processor -- start 2021-04-03T03:16:54.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- start start 2021-04-03T03:16:54.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a00b38c0 0x7f97a0025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f97a0153440 0x7f97a0080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97a0080a40 0x7f97a014feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a0025970 con 0x7f97a0080a40 2021-04-03T03:16:54.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a00b3ca0 con 0x7f97a0153440 2021-04-03T03:16:54.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0025b90 con 0x7f97a00b38c0 2021-04-03T03:16:54.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97a0080a40 0x7f97a014feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97a0080a40 0x7f97a014feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57514/0 (socket says 172.21.2.111:57514) 2021-04-03T03:16:54.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 -- 172.21.2.111:0/2693634188 learned_addr learned my addr 172.21.2.111:0/2693634188 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:54.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 -- 172.21.2.111:0/2693634188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a00b38c0 msgr2=0x7f97a0025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:54.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 --2- 172.21.2.111:0/2693634188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a00b38c0 0x7f97a0025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 -- 172.21.2.111:0/2693634188 >> v2:172.21.2.115:3300/0 conn(0x7f97a0153440 msgr2=0x7f97a0080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:54.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 --2- 172.21.2.111:0/2693634188 >> v2:172.21.2.115:3300/0 conn(0x7f97a0153440 0x7f97a0080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 -- 172.21.2.111:0/2693634188 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0025e30 con 0x7f97a0080a40 2021-04-03T03:16:54.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f97a4879700 1 --2- 172.21.2.111:0/2693634188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a014feb0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f97980032b0 tx=0x7f9798009270).ready entity=mon.0 client_cookie=a20d72bbb99bd01d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979db83700 1 -- 172.21.2.111:0/2693634188 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979800bc00 con 0x7f97a0080a40 2021-04-03T03:16:54.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979db83700 1 -- 172.21.2.111:0/2693634188 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979800bd60 con 0x7f97a0080a40 2021-04-03T03:16:54.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979db83700 1 -- 172.21.2.111:0/2693634188 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9798011400 con 0x7f97a0080a40 2021-04-03T03:16:54.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- 172.21.2.111:0/2693634188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 msgr2=0x7f97a014feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 --2- 172.21.2.111:0/2693634188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a014feb0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f97980032b0 tx=0x7f9798009270).stop 2021-04-03T03:16:54.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- 172.21.2.111:0/2693634188 shutdown_connections 2021-04-03T03:16:54.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 --2- 172.21.2.111:0/2693634188 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a014feb0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 --2- 172.21.2.111:0/2693634188 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a00b38c0 0x7f97a0025480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 --2- 172.21.2.111:0/2693634188 >> v2:172.21.2.115:3300/0 conn(0x7f97a0153440 0x7f97a0080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- 172.21.2.111:0/2693634188 >> 172.21.2.111:0/2693634188 conn(0x7f97a0061610 msgr2=0x7f97a0061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:54.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- 172.21.2.111:0/2693634188 shutdown_connections 2021-04-03T03:16:54.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- 172.21.2.111:0/2693634188 wait complete. 2021-04-03T03:16:54.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 Processor -- start 2021-04-03T03:16:54.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.793+0000 7f979f386700 1 -- start start 2021-04-03T03:16:54.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a0151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97a00b38c0 0x7f97a0150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a0153440 0x7f97a00ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00ac2d0 con 0x7f97a0080a40 2021-04-03T03:16:54.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00a7030 con 0x7f97a00b38c0 2021-04-03T03:16:54.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a012baa0 con 0x7f97a0153440 2021-04-03T03:16:54.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a0151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a0151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57516/0 (socket says 172.21.2.111:57516) 2021-04-03T03:16:54.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 -- 172.21.2.111:0/4221232956 learned_addr learned my addr 172.21.2.111:0/4221232956 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:54.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 -- 172.21.2.111:0/4221232956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a0153440 msgr2=0x7f97a00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a0153440 0x7f97a00ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 -- 172.21.2.111:0/4221232956 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97a00b38c0 msgr2=0x7f97a0150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97a00b38c0 0x7f97a0150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 -- 172.21.2.111:0/4221232956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00a7260 con 0x7f97a0080a40 2021-04-03T03:16:54.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979eb85700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a0151cd0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f9788002f30 tx=0x7f9788006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f97977fe700 1 -- 172.21.2.111:0/4221232956 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978800d840 con 0x7f97a0080a40 2021-04-03T03:16:54.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f97977fe700 1 -- 172.21.2.111:0/4221232956 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788000e90 con 0x7f97a0080a40 2021-04-03T03:16:54.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a012bd00 con 0x7f97a0080a40 2021-04-03T03:16:54.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a012ea20 con 0x7f97a0080a40 2021-04-03T03:16:54.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f97977fe700 1 -- 172.21.2.111:0/4221232956 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978800db90 con 0x7f97a0080a40 2021-04-03T03:16:54.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a012f120 con 0x7f97a0080a40 2021-04-03T03:16:54.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f97977fe700 1 -- 172.21.2.111:0/4221232956 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9788017030 con 0x7f97a0080a40 2021-04-03T03:16:54.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f97977fe700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9784057090 0x7f9784059550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:54.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f97977fe700 1 -- 172.21.2.111:0/4221232956 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f978806f6f0 con 0x7f97a0080a40 2021-04-03T03:16:54.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979e384700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9784057090 0x7f9784059550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:54.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.797+0000 7f979e384700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9784057090 0x7f9784059550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f9790002f30 tx=0x7f9790006940).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:54.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.801+0000 7f97977fe700 1 -- 172.21.2.111:0/4221232956 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9788044a80 con 0x7f97a0080a40 2021-04-03T03:16:54.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.929+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f97a00ae570 con 0x7f97a0080a40 2021-04-03T03:16:54.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f97977fe700 1 -- 172.21.2.111:0/4221232956 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v38) v1 ==== 184+0+0 (secure 0 0 0) 0x7f97880480a0 con 0x7f97a0080a40 2021-04-03T03:16:54.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9784057090 msgr2=0x7f9784059550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9784057090 0x7f9784059550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f9790002f30 tx=0x7f9790006940).stop 2021-04-03T03:16:54.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 msgr2=0x7f97a0151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:54.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a0151cd0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f9788002f30 tx=0x7f9788006ea0).stop 2021-04-03T03:16:54.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 shutdown_connections 2021-04-03T03:16:54.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97a0153440 0x7f97a00ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97a0080a40 0x7f97a0151cd0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9784057090 0x7f9784059550 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:54.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 --2- 172.21.2.111:0/4221232956 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97a00b38c0 0x7f97a0150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:54.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 >> 172.21.2.111:0/4221232956 conn(0x7f97a0061610 msgr2=0x7f97a00afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:54.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 shutdown_connections 2021-04-03T03:16:54.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:54.933+0000 7f979f386700 1 -- 172.21.2.111:0/4221232956 wait complete. 2021-04-03T03:16:54.939 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:16:54.943 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:55.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 Processor -- start 2021-04-03T03:16:55.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- start start 2021-04-03T03:16:55.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa904046ce0 0x7fa90403f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 0x7fa904041c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa904041240 con 0x7fa904042160 2021-04-03T03:16:55.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040f1850 con 0x7fa904046ce0 2021-04-03T03:16:55.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040470c0 con 0x7fa90403d350 2021-04-03T03:16:55.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa914138700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa904042160 0x7fa9040f51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58318/0 (socket says 172.21.2.111:58318) 2021-04-03T03:16:55.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa914138700 1 -- 172.21.2.111:0/2432877958 learned_addr learned my addr 172.21.2.111:0/2432877958 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:55.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa911cbc700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 0x7fa904041c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573128743 0 0) 0x7fa904041240 con 0x7fa904042160 2021-04-03T03:16:55.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 --2- 172.21.2.111:0/2432877958 >> v2:172.21.2.115:3300/0 conn(0x7fa904046ce0 0x7fa90403f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f8003240 con 0x7fa904042160 2021-04-03T03:16:55.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3301722826 0 0) 0x7fa8f8003240 con 0x7fa904042160 2021-04-03T03:16:55.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 msgr2=0x7fa904041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 0x7fa904041c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 >> v2:172.21.2.115:3300/0 conn(0x7fa904046ce0 msgr2=0x7fa90403f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 --2- 172.21.2.111:0/2432877958 >> v2:172.21.2.115:3300/0 conn(0x7fa904046ce0 0x7fa90403f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040400d0 con 0x7fa904042160 2021-04-03T03:16:55.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fa90c0450f0 con 0x7fa904042160 2021-04-03T03:16:55.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 >> v1:172.21.2.111:6789/0 conn(0x7fa904042160 legacy=0x7fa9040f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f8003730 0x7fa8f8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 0x7fa8f80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f80049d0 0x7fa8f8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040f1850 con 0x7fa8f8003730 2021-04-03T03:16:55.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040470c0 con 0x7fa8f80049d0 2021-04-03T03:16:55.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8004160 con 0x7fa90403d350 2021-04-03T03:16:55.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f8003730 0x7fa8f8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa911cbc700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 0x7fa8f80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa914138700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f80049d0 0x7fa8f8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 -- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 msgr2=0x7fa8f80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa90403d350 0x7fa8f80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 -- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f80049d0 msgr2=0x7fa8f8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8f80049d0 0x7fa8f8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 -- 172.21.2.111:0/2432877958 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8003240 con 0x7fa8f8003730 2021-04-03T03:16:55.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9124bd700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f8003730 0x7fa8f8003b50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7fa90000c000 tx=0x7fa900008040).ready entity=mon.0 client_cookie=2b0ce5bf0b4bd5ab server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa90000eb60 con 0x7fa8f8003730 2021-04-03T03:16:55.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000ecc0 con 0x7fa8f8003730 2021-04-03T03:16:55.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa9114bb700 1 -- 172.21.2.111:0/2432877958 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9000125e0 con 0x7fa8f8003730 2021-04-03T03:16:55.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- 172.21.2.111:0/2432877958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f8003730 msgr2=0x7fa8f8003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 --2- 172.21.2.111:0/2432877958 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8f8003730 0x7fa8f8003b50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7fa90000c000 tx=0x7fa900008040).stop 2021-04-03T03:16:55.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa914138700 1 -- 172.21.2.111:0/2432877958 reap_dead start 2021-04-03T03:16:55.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- 172.21.2.111:0/2432877958 shutdown_connections 2021-04-03T03:16:55.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- 172.21.2.111:0/2432877958 >> 172.21.2.111:0/2432877958 conn(0x7fa90400d0c0 msgr2=0x7fa9040462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- 172.21.2.111:0/2432877958 shutdown_connections 2021-04-03T03:16:55.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.001+0000 7fa912cbe700 1 -- 172.21.2.111:0/2432877958 wait complete. 2021-04-03T03:16:55.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 Processor -- start 2021-04-03T03:16:55.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 -- start start 2021-04-03T03:16:55.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9040d00f0 0x7fa9040d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9040af070 0x7fa9040d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9040d1390 0x7fa904045350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040aeb60 con 0x7fa9040d1390 2021-04-03T03:16:55.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040f1a30 con 0x7fa9040d00f0 2021-04-03T03:16:55.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040f1c90 con 0x7fa9040af070 2021-04-03T03:16:55.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9040d1390 0x7fa904045350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9040d1390 0x7fa904045350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57538/0 (socket says 172.21.2.111:57538) 2021-04-03T03:16:55.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 -- 172.21.2.111:0/485544693 learned_addr learned my addr 172.21.2.111:0/485544693 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:55.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa911cbc700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9040af070 0x7fa9040d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa9124bd700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9040d00f0 0x7fa9040d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 -- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9040af070 msgr2=0x7fa9040d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9040af070 0x7fa9040d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 -- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9040d00f0 msgr2=0x7fa9040d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9040d00f0 0x7fa9040d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 -- 172.21.2.111:0/485544693 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040ce490 con 0x7fa9040d1390 2021-04-03T03:16:55.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa9124bd700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9040d00f0 0x7fa9040d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:55.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa911cbc700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9040af070 0x7fa9040d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:16:55.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa914138700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9040d1390 0x7fa904045350 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fa90c05aec0 tx=0x7fa90403eaf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa8feffd700 1 -- 172.21.2.111:0/485544693 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa90c0699f0 con 0x7fa9040d1390 2021-04-03T03:16:55.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa8feffd700 1 -- 172.21.2.111:0/485544693 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c06a730 con 0x7fa9040d1390 2021-04-03T03:16:55.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9040ce760 con 0x7fa9040d1390 2021-04-03T03:16:55.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa8feffd700 1 -- 172.21.2.111:0/485544693 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa90c061260 con 0x7fa9040d1390 2021-04-03T03:16:55.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9040d1d70 con 0x7fa9040d1390 2021-04-03T03:16:55.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa904004d90 con 0x7fa9040d1390 2021-04-03T03:16:55.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa8feffd700 1 -- 172.21.2.111:0/485544693 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa90c05f9c0 con 0x7fa9040d1390 2021-04-03T03:16:55.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa8feffd700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8f4056fb0 0x7fa8f4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa8feffd700 1 -- 172.21.2.111:0/485544693 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa90c0cdc00 con 0x7fa9040d1390 2021-04-03T03:16:55.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa9124bd700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8f4056fb0 0x7fa8f4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.005+0000 7fa9124bd700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8f4056fb0 0x7fa8f4059470 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa90001a000 tx=0x7fa900019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.009+0000 7fa8feffd700 1 -- 172.21.2.111:0/485544693 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa90c0a2f90 con 0x7fa9040d1390 2021-04-03T03:16:55.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.133+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa9040083c0 con 0x7fa9040d1390 2021-04-03T03:16:55.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa8feffd700 1 -- 172.21.2.111:0/485544693 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v39) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa90c0a65b0 con 0x7fa9040d1390 2021-04-03T03:16:55.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8f4056fb0 msgr2=0x7fa8f4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8f4056fb0 0x7fa8f4059470 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa90001a000 tx=0x7fa900019000).stop 2021-04-03T03:16:55.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9040d1390 msgr2=0x7fa904045350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9040d1390 0x7fa904045350 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fa90c05aec0 tx=0x7fa90403eaf0).stop 2021-04-03T03:16:55.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 shutdown_connections 2021-04-03T03:16:55.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa9040d1390 0x7fa904045350 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8f4056fb0 0x7fa8f4059470 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa9040d00f0 0x7fa9040d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 --2- 172.21.2.111:0/485544693 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa9040af070 0x7fa9040d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 >> 172.21.2.111:0/485544693 conn(0x7fa90400d0c0 msgr2=0x7fa9040f3900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 shutdown_connections 2021-04-03T03:16:55.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.141+0000 7fa912cbe700 1 -- 172.21.2.111:0/485544693 wait complete. 2021-04-03T03:16:55.146 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:16:55.150 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:16:55.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 Processor -- start 2021-04-03T03:16:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- start start 2021-04-03T03:16:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f07500b38c0 0x7f0750025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750153440 0x7f0750080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0750080a40 0x7f075014feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0750025970 con 0x7f0750080a40 2021-04-03T03:16:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07500b3ca0 con 0x7f07500b38c0 2021-04-03T03:16:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0750025b90 con 0x7f0750153440 2021-04-03T03:16:55.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0750080a40 0x7f075014feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0750080a40 0x7f075014feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57546/0 (socket says 172.21.2.111:57546) 2021-04-03T03:16:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 -- 172.21.2.111:0/3999972459 learned_addr learned my addr 172.21.2.111:0/3999972459 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0755a94700 1 --2- 172.21.2.111:0/3999972459 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750153440 0x7f0750080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 -- 172.21.2.111:0/3999972459 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750153440 msgr2=0x7f0750080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 --2- 172.21.2.111:0/3999972459 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750153440 0x7f0750080550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 -- 172.21.2.111:0/3999972459 >> v2:172.21.2.115:3300/0 conn(0x7f07500b38c0 msgr2=0x7f0750025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 --2- 172.21.2.111:0/3999972459 >> v2:172.21.2.115:3300/0 conn(0x7f07500b38c0 0x7f0750025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 -- 172.21.2.111:0/3999972459 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750025e30 con 0x7f0750080a40 2021-04-03T03:16:55.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0757f10700 1 --2- 172.21.2.111:0/3999972459 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0750080a40 0x7f075014feb0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f07480032b0 tx=0x7f0748009270).ready entity=mon.0 client_cookie=c2e185ae3c9c06af server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0755293700 1 -- 172.21.2.111:0/3999972459 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074800bc00 con 0x7f0750080a40 2021-04-03T03:16:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0755293700 1 -- 172.21.2.111:0/3999972459 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074800bd60 con 0x7f0750080a40 2021-04-03T03:16:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0755293700 1 -- 172.21.2.111:0/3999972459 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0748011400 con 0x7f0750080a40 2021-04-03T03:16:55.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- 172.21.2.111:0/3999972459 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0750080a40 msgr2=0x7f075014feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 --2- 172.21.2.111:0/3999972459 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0750080a40 0x7f075014feb0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f07480032b0 tx=0x7f0748009270).stop 2021-04-03T03:16:55.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- 172.21.2.111:0/3999972459 shutdown_connections 2021-04-03T03:16:55.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 --2- 172.21.2.111:0/3999972459 >> v2:172.21.2.115:3300/0 conn(0x7f07500b38c0 0x7f0750025480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 --2- 172.21.2.111:0/3999972459 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0750080a40 0x7f075014feb0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 --2- 172.21.2.111:0/3999972459 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750153440 0x7f0750080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- 172.21.2.111:0/3999972459 >> 172.21.2.111:0/3999972459 conn(0x7f0750061610 msgr2=0x7f0750061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- 172.21.2.111:0/3999972459 shutdown_connections 2021-04-03T03:16:55.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- 172.21.2.111:0/3999972459 wait complete. 2021-04-03T03:16:55.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 Processor -- start 2021-04-03T03:16:55.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.209+0000 7f0756a96700 1 -- start start 2021-04-03T03:16:55.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750080a40 0x7f07500abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07500b38c0 0x7f07500664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0750153440 0x7f07500669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0750064020 con 0x7f07500b38c0 2021-04-03T03:16:55.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0750150e40 con 0x7f0750153440 2021-04-03T03:16:55.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0750064ff0 con 0x7f0750080a40 2021-04-03T03:16:55.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07500b38c0 0x7f07500664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07500b38c0 0x7f07500664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57548/0 (socket says 172.21.2.111:57548) 2021-04-03T03:16:55.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 -- 172.21.2.111:0/672129022 learned_addr learned my addr 172.21.2.111:0/672129022 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:55.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756295700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750080a40 0x7f07500abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 -- 172.21.2.111:0/672129022 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750080a40 msgr2=0x7f07500abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750080a40 0x7f07500abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 -- 172.21.2.111:0/672129022 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0750153440 msgr2=0x7f07500669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0750153440 0x7f07500669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 -- 172.21.2.111:0/672129022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750065290 con 0x7f07500b38c0 2021-04-03T03:16:55.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0755a94700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07500b38c0 0x7f07500664c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f074c007000 tx=0x7f074c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0746ffd700 1 -- 172.21.2.111:0/672129022 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074c00e5e0 con 0x7f07500b38c0 2021-04-03T03:16:55.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0746ffd700 1 -- 172.21.2.111:0/672129022 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c00c030 con 0x7f07500b38c0 2021-04-03T03:16:55.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0750064a90 con 0x7f07500b38c0 2021-04-03T03:16:55.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0746ffd700 1 -- 172.21.2.111:0/672129022 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074c00e930 con 0x7f07500b38c0 2021-04-03T03:16:55.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075009ede0 con 0x7f07500b38c0 2021-04-03T03:16:55.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0750057660 con 0x7f07500b38c0 2021-04-03T03:16:55.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0746ffd700 1 -- 172.21.2.111:0/672129022 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f074c019030 con 0x7f07500b38c0 2021-04-03T03:16:55.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0746ffd700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0738056e90 0x7f0738059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0746ffd700 1 -- 172.21.2.111:0/672129022 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f074c06f9e0 con 0x7f07500b38c0 2021-04-03T03:16:55.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756295700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0738056e90 0x7f0738059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.213+0000 7f0756295700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0738056e90 0x7f0738059350 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f0740007000 tx=0x7f0740008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.217+0000 7f0746ffd700 1 -- 172.21.2.111:0/672129022 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f074c042c50 con 0x7f07500b38c0 2021-04-03T03:16:55.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.341+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0750069270 con 0x7f07500b38c0 2021-04-03T03:16:55.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.345+0000 7f0746ffd700 1 -- 172.21.2.111:0/672129022 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v40) v1 ==== 184+0+0 (secure 0 0 0) 0x7f074c044e80 con 0x7f07500b38c0 2021-04-03T03:16:55.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.345+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0738056e90 msgr2=0x7f0738059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0738056e90 0x7f0738059350 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f0740007000 tx=0x7f0740008040).stop 2021-04-03T03:16:55.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07500b38c0 msgr2=0x7f07500664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07500b38c0 0x7f07500664c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f074c007000 tx=0x7f074c008040).stop 2021-04-03T03:16:55.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 shutdown_connections 2021-04-03T03:16:55.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0738056e90 0x7f0738059350 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0750153440 0x7f07500669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0750080a40 0x7f07500abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 --2- 172.21.2.111:0/672129022 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f07500b38c0 0x7f07500664c0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 >> 172.21.2.111:0/672129022 conn(0x7f0750061610 msgr2=0x7f07500b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 shutdown_connections 2021-04-03T03:16:55.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.349+0000 7f0756a96700 1 -- 172.21.2.111:0/672129022 wait complete. 2021-04-03T03:16:55.352 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:16:55.357 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:55.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 Processor -- start 2021-04-03T03:16:55.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 -- start start 2021-04-03T03:16:55.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c440603f0 0x7f0c440b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0c440b0c90 0x7f0c44147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c440607d0 con 0x7f0c4414b700 2021-04-03T03:16:55.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440af400 con 0x7f0c440b0c90 2021-04-03T03:16:55.413 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440af6a0 con 0x7f0c440603f0 2021-04-03T03:16:55.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0c440b0c90 0x7f0c44147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0c440b0c90 0x7f0c44147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57560/0 (socket says 172.21.2.111:57560) 2021-04-03T03:16:55.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 -- 172.21.2.111:0/668351718 learned_addr learned my addr 172.21.2.111:0/668351718 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:55.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c42ffd700 1 --2- 172.21.2.111:0/668351718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c440603f0 0x7f0c440b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 -- 172.21.2.111:0/668351718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c440603f0 msgr2=0x7f0c440b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 --2- 172.21.2.111:0/668351718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c440603f0 0x7f0c440b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 -- 172.21.2.111:0/668351718 >> v1:172.21.2.115:6789/0 conn(0x7f0c4414b700 legacy=0x7f0c440b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 -- 172.21.2.111:0/668351718 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440afb40 con 0x7f0c440b0c90 2021-04-03T03:16:55.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c437fe700 1 --2- 172.21.2.111:0/668351718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c44147680 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f0c440628d0 tx=0x7f0c3c009270).ready entity=mon.0 client_cookie=7fa9f1135d7e7555 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c41ffb700 1 -- 172.21.2.111:0/668351718 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c3c00bbc0 con 0x7f0c440b0c90 2021-04-03T03:16:55.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c41ffb700 1 -- 172.21.2.111:0/668351718 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c00bd20 con 0x7f0c440b0c90 2021-04-03T03:16:55.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c41ffb700 1 -- 172.21.2.111:0/668351718 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c3c011400 con 0x7f0c440b0c90 2021-04-03T03:16:55.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 -- 172.21.2.111:0/668351718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 msgr2=0x7f0c44147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.409+0000 7f0c43fff700 1 --2- 172.21.2.111:0/668351718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c44147680 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f0c440628d0 tx=0x7f0c3c009270).stop 2021-04-03T03:16:55.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- 172.21.2.111:0/668351718 shutdown_connections 2021-04-03T03:16:55.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 --2- 172.21.2.111:0/668351718 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c44147680 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 --2- 172.21.2.111:0/668351718 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c440603f0 0x7f0c440b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- 172.21.2.111:0/668351718 >> 172.21.2.111:0/668351718 conn(0x7f0c440b61c0 msgr2=0x7f0c4405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- 172.21.2.111:0/668351718 shutdown_connections 2021-04-03T03:16:55.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- 172.21.2.111:0/668351718 wait complete. 2021-04-03T03:16:55.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 Processor -- start 2021-04-03T03:16:55.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- start start 2021-04-03T03:16:55.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c440603f0 0x7f0c440ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c4414a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.422 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c4414b700 0x7f0c44064310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440aa3f0 con 0x7f0c440b0c90 2021-04-03T03:16:55.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440a9eb0 con 0x7f0c440603f0 2021-04-03T03:16:55.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440aa150 con 0x7f0c4414b700 2021-04-03T03:16:55.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c4414a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c4414a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57564/0 (socket says 172.21.2.111:57564) 2021-04-03T03:16:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 -- 172.21.2.111:0/3571138108 learned_addr learned my addr 172.21.2.111:0/3571138108 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 -- 172.21.2.111:0/3571138108 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c4414b700 msgr2=0x7f0c44064310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c4414b700 0x7f0c44064310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 -- 172.21.2.111:0/3571138108 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c440603f0 msgr2=0x7f0c440ab430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:16:55.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c440603f0 0x7f0c440ab430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 -- 172.21.2.111:0/3571138108 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c441599f0 con 0x7f0c440b0c90 2021-04-03T03:16:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c427fc700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c4414a8c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f0c34002ef0 tx=0x7f0c34006d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c3bfff700 1 -- 172.21.2.111:0/3571138108 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c3400d4a0 con 0x7f0c440b0c90 2021-04-03T03:16:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c3bfff700 1 -- 172.21.2.111:0/3571138108 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34000e90 con 0x7f0c440b0c90 2021-04-03T03:16:55.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c44159c40 con 0x7f0c440b0c90 2021-04-03T03:16:55.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c440ab070 con 0x7f0c440b0c90 2021-04-03T03:16:55.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c3bfff700 1 -- 172.21.2.111:0/3571138108 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c3400d7f0 con 0x7f0c440b0c90 2021-04-03T03:16:55.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c44057660 con 0x7f0c440b0c90 2021-04-03T03:16:55.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c3bfff700 1 -- 172.21.2.111:0/3571138108 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0c3400f030 con 0x7f0c440b0c90 2021-04-03T03:16:55.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c3bfff700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c28057000 0x7f0c280594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c3bfff700 1 -- 172.21.2.111:0/3571138108 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0c3406fee0 con 0x7f0c440b0c90 2021-04-03T03:16:55.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c42ffd700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c28057000 0x7f0c280594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.413+0000 7f0c42ffd700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c28057000 0x7f0c280594c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f0c2c00a000 tx=0x7f0c2c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.417+0000 7f0c3bfff700 1 -- 172.21.2.111:0/3571138108 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0c340451c0 con 0x7f0c440b0c90 2021-04-03T03:16:55.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c44064090 con 0x7f0c440b0c90 2021-04-03T03:16:55.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c3bfff700 1 -- 172.21.2.111:0/3571138108 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f0c340487e0 con 0x7f0c440b0c90 2021-04-03T03:16:55.542 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:55.542 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":58,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3599494763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3599494763},{"type":"v1","addr":"172.21.2.115:6833","nonce":3599494763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:53.339332+0000","modified":"2021-04-03T03:16:54.343270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7890":{"gid":7890,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/233883066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":233883066},{"type":"v1","addr":"172.21.2.111:6835","nonce":233883066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-04-03T03:16:55.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c28057000 msgr2=0x7f0c280594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c28057000 0x7f0c280594c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f0c2c00a000 tx=0x7f0c2c008040).stop 2021-04-03T03:16:55.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 msgr2=0x7f0c4414a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c4414a8c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f0c34002ef0 tx=0x7f0c34006d20).stop 2021-04-03T03:16:55.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 shutdown_connections 2021-04-03T03:16:55.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c4414b700 0x7f0c44064310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0c28057000 0x7f0c280594c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c440603f0 0x7f0c440ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 --2- 172.21.2.111:0/3571138108 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c440b0c90 0x7f0c4414a8c0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 >> 172.21.2.111:0/3571138108 conn(0x7f0c440b61c0 msgr2=0x7f0c4405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 shutdown_connections 2021-04-03T03:16:55.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.541+0000 7f0c43fff700 1 -- 172.21.2.111:0/3571138108 wait complete. 2021-04-03T03:16:55.546 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 58 2021-04-03T03:16:55.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:53.339332+0000', 'modified': '2021-04-03T03:16:54.343270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7890}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7890': {'gid': 7890, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/233883066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 233883066}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 233883066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:55.752 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:55.752 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:55.838 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7890,"want_state":"up:active","state":"up:active","rank_uptime":2.4919868169999999,"mdsmap_epoch":58,"osdmap_epoch":132,"osdmap_epoch_barrier":131,"uptime":6.6856050460000001} 2021-04-03T03:16:55.838 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7890, "mdsmap_epoch": 58, "osdmap_epoch": 132, "osdmap_epoch_barrier": 131, "rank_uptime": 2.491986817, "state": "up:active", "uptime": 6.685605046, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:55.839 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:55.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 Processor -- start 2021-04-03T03:16:55.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 -- start start 2021-04-03T03:16:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fecac14b700 0x7fecac0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac0af400 con 0x7fecac0603f0 2021-04-03T03:16:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac0af6a0 con 0x7fecac0b0c90 2021-04-03T03:16:55.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecac060860 con 0x7fecac14b700 2021-04-03T03:16:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fecac0603f0 0x7fecac0b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58362/0 (socket says 172.21.2.111:58362) 2021-04-03T03:16:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 -- 172.21.2.111:0/964708796 learned_addr learned my addr 172.21.2.111:0/964708796 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651189577 0 0) 0x7fecac0af400 con 0x7fecac0603f0 2021-04-03T03:16:55.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec8c003240 con 0x7fecac0603f0 2021-04-03T03:16:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaa5e4700 1 --2- 172.21.2.111:0/964708796 >> v2:172.21.2.115:3300/0 conn(0x7fecac14b700 0x7fecac0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2187246169 0 0) 0x7fec8c003240 con 0x7fecac0603f0 2021-04-03T03:16:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 >> v2:172.21.2.115:3300/0 conn(0x7fecac14b700 msgr2=0x7fecac0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 --2- 172.21.2.111:0/964708796 >> v2:172.21.2.115:3300/0 conn(0x7fecac14b700 0x7fecac0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 >> v1:172.21.2.115:6789/0 conn(0x7fecac0b0c90 legacy=0x7fecac147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0afb40 con 0x7fecac0603f0 2021-04-03T03:16:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fec940016d0 con 0x7fecac0603f0 2021-04-03T03:16:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 >> v1:172.21.2.111:6789/0 conn(0x7fecac0603f0 legacy=0x7fecac0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fec8c003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fec8c0044e0 0x7fec8c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fec8c004e50 0x7fec8c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac060860 con 0x7fec8c0037b0 2021-04-03T03:16:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c005300 con 0x7fec8c0044e0 2021-04-03T03:16:55.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c00bc20 con 0x7fec8c004e50 2021-04-03T03:16:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fec8c003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecab5e6700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fec8c004e50 0x7fec8c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 -- 172.21.2.111:0/964708796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fec8c004e50 msgr2=0x7fec8c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fec8c004e50 0x7fec8c00b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 -- 172.21.2.111:0/964708796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fec8c0044e0 msgr2=0x7fec8c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fec8c0044e0 0x7fec8c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 -- 172.21.2.111:0/964708796 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0afb40 con 0x7fec8c0037b0 2021-04-03T03:16:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecab5e6700 1 -- 172.21.2.111:0/964708796 reap_dead start 2021-04-03T03:16:55.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecaade5700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fec8c003bb0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7fec94000fe0 tx=0x7fec94007780).ready entity=mon.0 client_cookie=247286b02407a78f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec9400ce50 con 0x7fec8c0037b0 2021-04-03T03:16:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec940072e0 con 0x7fec8c0037b0 2021-04-03T03:16:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7feca9de3700 1 -- 172.21.2.111:0/964708796 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec940057e0 con 0x7fec8c0037b0 2021-04-03T03:16:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 -- 172.21.2.111:0/964708796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 msgr2=0x7fec8c003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.905 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.893+0000 7fecabde7700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fec8c003bb0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7fec94000fe0 tx=0x7fec94007780).stop 2021-04-03T03:16:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- 172.21.2.111:0/964708796 shutdown_connections 2021-04-03T03:16:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 --2- 172.21.2.111:0/964708796 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fec8c003bb0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- 172.21.2.111:0/964708796 >> 172.21.2.111:0/964708796 conn(0x7fecac0b61c0 msgr2=0x7fecac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- 172.21.2.111:0/964708796 shutdown_connections 2021-04-03T03:16:55.906 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- 172.21.2.111:0/964708796 wait complete. 2021-04-03T03:16:55.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 Processor -- start 2021-04-03T03:16:55.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- start start 2021-04-03T03:16:55.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fecac0aa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fecac0ad820 0x7fecac146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fecac146cf0 0x7fecac064070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0adc80 con 0x7fec8c0037b0 2021-04-03T03:16:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0ab160 con 0x7fecac0ad820 2021-04-03T03:16:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0ab400 con 0x7fecac146cf0 2021-04-03T03:16:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fecac0aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fecac0aa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57582/0 (socket says 172.21.2.111:57582) 2021-04-03T03:16:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 -- 172.21.2.111:0/198278560 learned_addr learned my addr 172.21.2.111:0/198278560 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 -- 172.21.2.111:0/198278560 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fecac146cf0 msgr2=0x7fecac064070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fecac146cf0 0x7fecac064070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 -- 172.21.2.111:0/198278560 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fecac0ad820 msgr2=0x7fecac146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fecac0ad820 0x7fecac146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 -- 172.21.2.111:0/198278560 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0ab6a0 con 0x7fec8c0037b0 2021-04-03T03:16:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaade5700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fecac0aa030 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fec94000fe0 tx=0x7fec94015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7feca3fff700 1 -- 172.21.2.111:0/198278560 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec94004b50 con 0x7fec8c0037b0 2021-04-03T03:16:55.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7feca3fff700 1 -- 172.21.2.111:0/198278560 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94004cb0 con 0x7fec8c0037b0 2021-04-03T03:16:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac159580 con 0x7fec8c0037b0 2021-04-03T03:16:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac0a9c30 con 0x7fec8c0037b0 2021-04-03T03:16:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7feca3fff700 1 -- 172.21.2.111:0/198278560 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec9401f660 con 0x7fec8c0037b0 2021-04-03T03:16:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac0ab830 con 0x7fec8c0037b0 2021-04-03T03:16:55.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7feca3fff700 1 -- 172.21.2.111:0/198278560 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fec94010070 con 0x7fec8c0037b0 2021-04-03T03:16:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7feca3fff700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fec90057030 0x7fec900594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7feca3fff700 1 -- 172.21.2.111:0/198278560 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fec94070fe0 con 0x7fec8c0037b0 2021-04-03T03:16:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaa5e4700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fec90057030 0x7fec900594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:55.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.897+0000 7fecaa5e4700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fec90057030 0x7fec900594f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fec9c00a000 tx=0x7fec9c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:55.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.901+0000 7feca3fff700 1 -- 172.21.2.111:0/198278560 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fec94046320 con 0x7fec8c0037b0 2021-04-03T03:16:55.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecac057660 con 0x7fec8c0037b0 2021-04-03T03:16:55.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7feca3fff700 1 -- 172.21.2.111:0/198278560 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fec94049940 con 0x7fec8c0037b0 2021-04-03T03:16:55.995 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:55.996 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":132,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:54.343386+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:51.364989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:52.312016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fec90057030 msgr2=0x7fec900594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fec90057030 0x7fec900594f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fec9c00a000 tx=0x7fec9c008040).stop 2021-04-03T03:16:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 msgr2=0x7fecac0aa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fecac0aa030 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fec94000fe0 tx=0x7fec94015000).stop 2021-04-03T03:16:55.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 shutdown_connections 2021-04-03T03:16:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fecac146cf0 0x7fecac064070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fec8c0037b0 0x7fecac0aa030 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fec90057030 0x7fec900594f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:55.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 --2- 172.21.2.111:0/198278560 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fecac0ad820 0x7fecac146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:55.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 >> 172.21.2.111:0/198278560 conn(0x7fecac0b61c0 msgr2=0x7fecac0b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:55.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 shutdown_connections 2021-04-03T03:16:55.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:55.993+0000 7fecabde7700 1 -- 172.21.2.111:0/198278560 wait complete. 2021-04-03T03:16:56.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:16:51.364989+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:16:52.312016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:16:56.005 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:56.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41307b700 1 Processor -- start 2021-04-03T03:16:56.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41307b700 1 -- start start 2021-04-03T03:16:56.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41307b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb40c0add50 0x7fb40c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41307b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb40c144ef0 0x7fb40c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41307b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c0607d0 con 0x7fb40c0603f0 2021-04-03T03:16:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41307b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c14b090 con 0x7fb40c144ef0 2021-04-03T03:16:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41307b700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c14b330 con 0x7fb40c0add50 2021-04-03T03:16:56.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb40c144ef0 0x7fb40c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb40c144ef0 0x7fb40c147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57590/0 (socket says 172.21.2.111:57590) 2021-04-03T03:16:56.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 -- 172.21.2.111:0/3580565647 learned_addr learned my addr 172.21.2.111:0/3580565647 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:56.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb412079700 1 --1- 172.21.2.111:0/3580565647 >> v1:172.21.2.111:6789/0 conn(0x7fb40c0603f0 0x7fb40c1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58382/0 (socket says 172.21.2.111:58382) 2021-04-03T03:16:56.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb411878700 1 --2- 172.21.2.111:0/3580565647 >> v2:172.21.2.115:3300/0 conn(0x7fb40c0add50 0x7fb40c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 -- 172.21.2.111:0/3580565647 >> v2:172.21.2.115:3300/0 conn(0x7fb40c0add50 msgr2=0x7fb40c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 --2- 172.21.2.111:0/3580565647 >> v2:172.21.2.115:3300/0 conn(0x7fb40c0add50 0x7fb40c147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 -- 172.21.2.111:0/3580565647 >> v1:172.21.2.111:6789/0 conn(0x7fb40c0603f0 legacy=0x7fb40c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 -- 172.21.2.111:0/3580565647 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c14b5d0 con 0x7fb40c144ef0 2021-04-03T03:16:56.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb411077700 1 -- 172.21.2.111:0/3580565647 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651897444 0 0) 0x7fb40c0607d0 con 0x7fb40c0603f0 2021-04-03T03:16:56.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.061+0000 7fb41287a700 1 --2- 172.21.2.111:0/3580565647 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c144ef0 0x7fb40c147c30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7fb4040032b0 tx=0x7fb404009270).ready entity=mon.0 client_cookie=eea650e1977a8c9d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb411077700 1 -- 172.21.2.111:0/3580565647 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40400bc00 con 0x7fb40c144ef0 2021-04-03T03:16:56.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb411077700 1 -- 172.21.2.111:0/3580565647 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40400bd60 con 0x7fb40c144ef0 2021-04-03T03:16:56.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb411077700 1 -- 172.21.2.111:0/3580565647 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb404011400 con 0x7fb40c144ef0 2021-04-03T03:16:56.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/3580565647 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c144ef0 msgr2=0x7fb40c147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 --2- 172.21.2.111:0/3580565647 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c144ef0 0x7fb40c147c30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7fb4040032b0 tx=0x7fb404009270).stop 2021-04-03T03:16:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/3580565647 shutdown_connections 2021-04-03T03:16:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 --2- 172.21.2.111:0/3580565647 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c144ef0 0x7fb40c147c30 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 --2- 172.21.2.111:0/3580565647 >> v2:172.21.2.115:3300/0 conn(0x7fb40c0add50 0x7fb40c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/3580565647 >> 172.21.2.111:0/3580565647 conn(0x7fb40c0b61c0 msgr2=0x7fb40c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/3580565647 shutdown_connections 2021-04-03T03:16:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/3580565647 wait complete. 2021-04-03T03:16:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 Processor -- start 2021-04-03T03:16:56.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- start start 2021-04-03T03:16:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c0603f0 0x7fb40c0a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0add50 0x7fb40c147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c144ef0 0x7fb40c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0a8de0 con 0x7fb40c0603f0 2021-04-03T03:16:56.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0aa9d0 con 0x7fb40c144ef0 2021-04-03T03:16:56.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c156ec0 con 0x7fb40c0add50 2021-04-03T03:16:56.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c0603f0 0x7fb40c0a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c0603f0 0x7fb40c0a9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57598/0 (socket says 172.21.2.111:57598) 2021-04-03T03:16:56.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 -- 172.21.2.111:0/637931566 learned_addr learned my addr 172.21.2.111:0/637931566 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:56.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 -- 172.21.2.111:0/637931566 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0add50 msgr2=0x7fb40c147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0add50 0x7fb40c147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 -- 172.21.2.111:0/637931566 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c144ef0 msgr2=0x7fb40c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41287a700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c144ef0 0x7fb40c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c144ef0 0x7fb40c1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 -- 172.21.2.111:0/637931566 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c157160 con 0x7fb40c0603f0 2021-04-03T03:16:56.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb412079700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c0603f0 0x7fb40c0a9ae0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7fb40c156de0 tx=0x7fb400007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb3ff7fe700 1 -- 172.21.2.111:0/637931566 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb400007320 con 0x7fb40c0603f0 2021-04-03T03:16:56.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb3ff7fe700 1 -- 172.21.2.111:0/637931566 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400005530 con 0x7fb40c0603f0 2021-04-03T03:16:56.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb3ff7fe700 1 -- 172.21.2.111:0/637931566 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb400005890 con 0x7fb40c0603f0 2021-04-03T03:16:56.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c157400 con 0x7fb40c0603f0 2021-04-03T03:16:56.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c157d90 con 0x7fb40c0603f0 2021-04-03T03:16:56.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c057660 con 0x7fb40c0603f0 2021-04-03T03:16:56.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.065+0000 7fb3ff7fe700 1 -- 172.21.2.111:0/637931566 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb400010030 con 0x7fb40c0603f0 2021-04-03T03:16:56.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.069+0000 7fb3ff7fe700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec056ea0 0x7fb3ec059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.069+0000 7fb3ff7fe700 1 -- 172.21.2.111:0/637931566 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb40006fe60 con 0x7fb40c0603f0 2021-04-03T03:16:56.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.069+0000 7fb411878700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec056ea0 0x7fb3ec059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.069+0000 7fb411878700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec056ea0 0x7fb3ec059360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fb40800a000 tx=0x7fb408008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.069+0000 7fb3ff7fe700 1 -- 172.21.2.111:0/637931566 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb400045250 con 0x7fb40c0603f0 2021-04-03T03:16:56.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb40c06b370 con 0x7fb40c0603f0 2021-04-03T03:16:56.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb3ff7fe700 1 -- 172.21.2.111:0/637931566 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v132) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb400048870 con 0x7fb40c0603f0 2021-04-03T03:16:56.163 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:56.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec056ea0 msgr2=0x7fb3ec059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec056ea0 0x7fb3ec059360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fb40800a000 tx=0x7fb408008040).stop 2021-04-03T03:16:56.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c0603f0 msgr2=0x7fb40c0a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c0603f0 0x7fb40c0a9ae0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7fb40c156de0 tx=0x7fb400007a40).stop 2021-04-03T03:16:56.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 shutdown_connections 2021-04-03T03:16:56.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb3ec056ea0 0x7fb3ec059360 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb40c144ef0 0x7fb40c1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb40c0603f0 0x7fb40c0a9ae0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 --2- 172.21.2.111:0/637931566 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb40c0add50 0x7fb40c147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 >> 172.21.2.111:0/637931566 conn(0x7fb40c0b61c0 msgr2=0x7fb40c0b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 shutdown_connections 2021-04-03T03:16:56.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.161+0000 7fb41307b700 1 -- 172.21.2.111:0/637931566 wait complete. 2021-04-03T03:16:56.171 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:56.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2eb2a700 1 Processor -- start 2021-04-03T03:16:56.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2eb2a700 1 -- start start 2021-04-03T03:16:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2eb2a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9a280603f0 0x7f9a281447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2eb2a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280b17b0 0x7f9a28144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2eb2a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a280607d0 con 0x7f9a280add50 2021-04-03T03:16:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2eb2a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a281451c0 con 0x7f9a280603f0 2021-04-03T03:16:56.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2eb2a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2814b090 con 0x7f9a280b17b0 2021-04-03T03:16:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2db28700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9a280603f0 0x7f9a281447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2d327700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f9a280add50 0x7f9a28147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58400/0 (socket says 172.21.2.111:58400) 2021-04-03T03:16:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.225+0000 7f9a2d327700 1 -- 172.21.2.111:0/4131441127 learned_addr learned my addr 172.21.2.111:0/4131441127 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:56.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2cb26700 1 -- 172.21.2.111:0/4131441127 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528675824 0 0) 0x7f9a280607d0 con 0x7f9a280add50 2021-04-03T03:16:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2cb26700 1 -- 172.21.2.111:0/4131441127 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a10003240 con 0x7f9a280add50 2021-04-03T03:16:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2db28700 1 -- 172.21.2.111:0/4131441127 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280b17b0 msgr2=0x7f9a28144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2db28700 1 --2- 172.21.2.111:0/4131441127 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280b17b0 0x7f9a28144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2db28700 1 -- 172.21.2.111:0/4131441127 >> v1:172.21.2.111:6789/0 conn(0x7f9a280add50 legacy=0x7f9a28147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2db28700 1 -- 172.21.2.111:0/4131441127 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2814b330 con 0x7f9a280603f0 2021-04-03T03:16:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2db28700 1 --2- 172.21.2.111:0/4131441127 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280603f0 0x7f9a281447e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f9a18002f30 tx=0x7f9a18006dd0).ready entity=mon.0 client_cookie=4df22a2f7397d6c8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2cb26700 1 -- 172.21.2.111:0/4131441127 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a18009760 con 0x7f9a280603f0 2021-04-03T03:16:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2cb26700 1 -- 172.21.2.111:0/4131441127 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a180098c0 con 0x7f9a280603f0 2021-04-03T03:16:56.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2cb26700 1 -- 172.21.2.111:0/4131441127 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a18009c20 con 0x7f9a280603f0 2021-04-03T03:16:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/4131441127 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280603f0 msgr2=0x7f9a281447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/4131441127 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280603f0 0x7f9a281447e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f9a18002f30 tx=0x7f9a18006dd0).stop 2021-04-03T03:16:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/4131441127 shutdown_connections 2021-04-03T03:16:56.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/4131441127 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280b17b0 0x7f9a28144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/4131441127 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280603f0 0x7f9a281447e0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/4131441127 >> 172.21.2.111:0/4131441127 conn(0x7f9a280b61c0 msgr2=0x7f9a2805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/4131441127 shutdown_connections 2021-04-03T03:16:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/4131441127 wait complete. 2021-04-03T03:16:56.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 Processor -- start 2021-04-03T03:16:56.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- start start 2021-04-03T03:16:56.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280603f0 0x7f9a280b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280add50 0x7f9a280b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280b17b0 0x7f9a281462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28068060 con 0x7f9a280b17b0 2021-04-03T03:16:56.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280aa9d0 con 0x7f9a280603f0 2021-04-03T03:16:56.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28146790 con 0x7f9a280add50 2021-04-03T03:16:56.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280b17b0 0x7f9a281462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280b17b0 0x7f9a281462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57614/0 (socket says 172.21.2.111:57614) 2021-04-03T03:16:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 -- 172.21.2.111:0/1304400755 learned_addr learned my addr 172.21.2.111:0/1304400755 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 -- 172.21.2.111:0/1304400755 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280add50 msgr2=0x7f9a280b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280add50 0x7f9a280b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 -- 172.21.2.111:0/1304400755 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280603f0 msgr2=0x7f9a280b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280603f0 0x7f9a280b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 -- 172.21.2.111:0/1304400755 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a280aa730 con 0x7f9a280b17b0 2021-04-03T03:16:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2e329700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280b17b0 0x7f9a281462a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f9a200032b0 tx=0x7f9a200091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a16ffd700 1 -- 172.21.2.111:0/1304400755 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a2000c960 con 0x7f9a280b17b0 2021-04-03T03:16:56.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a16ffd700 1 -- 172.21.2.111:0/1304400755 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2000cac0 con 0x7f9a280b17b0 2021-04-03T03:16:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a28156b00 con 0x7f9a280b17b0 2021-04-03T03:16:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a16ffd700 1 -- 172.21.2.111:0/1304400755 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a2000ce20 con 0x7f9a280b17b0 2021-04-03T03:16:56.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a28147bb0 con 0x7f9a280b17b0 2021-04-03T03:16:56.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28156c90 con 0x7f9a280b17b0 2021-04-03T03:16:56.243 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a16ffd700 1 -- 172.21.2.111:0/1304400755 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9a20014420 con 0x7f9a280b17b0 2021-04-03T03:16:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a16ffd700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a08056f20 0x7f9a080593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a16ffd700 1 -- 172.21.2.111:0/1304400755 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9a20070080 con 0x7f9a280b17b0 2021-04-03T03:16:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.229+0000 7f9a2db28700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a08056f20 0x7f9a080593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.244 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.233+0000 7f9a2db28700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a08056f20 0x7f9a080593e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9a18002f30 tx=0x7f9a18015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.245 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.233+0000 7f9a16ffd700 1 -- 172.21.2.111:0/1304400755 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9a20045370 con 0x7f9a280b17b0 2021-04-03T03:16:56.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.325+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a28057660 con 0x7f9a280b17b0 2021-04-03T03:16:56.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.325+0000 7f9a16ffd700 1 -- 172.21.2.111:0/1304400755 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a20019070 con 0x7f9a280b17b0 2021-04-03T03:16:56.329 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:56.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a08056f20 msgr2=0x7f9a080593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a08056f20 0x7f9a080593e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9a18002f30 tx=0x7f9a18015000).stop 2021-04-03T03:16:56.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280b17b0 msgr2=0x7f9a281462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280b17b0 0x7f9a281462a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f9a200032b0 tx=0x7f9a200091a0).stop 2021-04-03T03:16:56.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 shutdown_connections 2021-04-03T03:16:56.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9a280b17b0 0x7f9a281462a0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9a08056f20 0x7f9a080593e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9a280603f0 0x7f9a280b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 --2- 172.21.2.111:0/1304400755 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9a280add50 0x7f9a280b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 >> 172.21.2.111:0/1304400755 conn(0x7f9a280b61c0 msgr2=0x7f9a28143600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 shutdown_connections 2021-04-03T03:16:56.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.329+0000 7f9a2eb2a700 1 -- 172.21.2.111:0/1304400755 wait complete. 2021-04-03T03:16:56.337 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:16:56.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 Processor -- start 2021-04-03T03:16:56.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 -- start start 2021-04-03T03:16:56.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0c080603f0 0x7f0c081447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080add50 0x7f0c08144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c080607d0 con 0x7f0c080b17b0 2021-04-03T03:16:56.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c081451c0 con 0x7f0c080603f0 2021-04-03T03:16:56.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0814b090 con 0x7f0c080add50 2021-04-03T03:16:56.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d1e1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0c080603f0 0x7f0c081447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d9e2700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0c080b17b0 0x7f0c08147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58414/0 (socket says 172.21.2.111:58414) 2021-04-03T03:16:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d9e2700 1 -- 172.21.2.111:0/3639759553 learned_addr learned my addr 172.21.2.111:0/3639759553 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0c9e0700 1 --2- 172.21.2.111:0/3639759553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080add50 0x7f0c08144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d1e1700 1 -- 172.21.2.111:0/3639759553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080add50 msgr2=0x7f0c08144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d1e1700 1 --2- 172.21.2.111:0/3639759553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080add50 0x7f0c08144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d1e1700 1 -- 172.21.2.111:0/3639759553 >> v1:172.21.2.111:6789/0 conn(0x7f0c080b17b0 legacy=0x7f0c08147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d1e1700 1 -- 172.21.2.111:0/3639759553 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0814b330 con 0x7f0c080603f0 2021-04-03T03:16:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0c9e0700 1 --2- 172.21.2.111:0/3639759553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080add50 0x7f0c08144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:56.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0d1e1700 1 --2- 172.21.2.111:0/3639759553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080603f0 0x7f0c081447e0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7f0c04002f30 tx=0x7f0c04006dd0).ready entity=mon.0 client_cookie=cbffec246dd451cb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0bf7fff700 1 -- 172.21.2.111:0/3639759553 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c040097f0 con 0x7f0c080603f0 2021-04-03T03:16:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0bf7fff700 1 -- 172.21.2.111:0/3639759553 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04009950 con 0x7f0c080603f0 2021-04-03T03:16:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0bf7fff700 1 -- 172.21.2.111:0/3639759553 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c04009cb0 con 0x7f0c080603f0 2021-04-03T03:16:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3639759553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080603f0 msgr2=0x7f0c081447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.393+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3639759553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080603f0 0x7f0c081447e0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7f0c04002f30 tx=0x7f0c04006dd0).stop 2021-04-03T03:16:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3639759553 shutdown_connections 2021-04-03T03:16:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3639759553 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080603f0 0x7f0c081447e0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3639759553 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080add50 0x7f0c08144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3639759553 >> 172.21.2.111:0/3639759553 conn(0x7f0c080b61c0 msgr2=0x7f0c0805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3639759553 shutdown_connections 2021-04-03T03:16:56.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3639759553 wait complete. 2021-04-03T03:16:56.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 Processor -- start 2021-04-03T03:16:56.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- start start 2021-04-03T03:16:56.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080603f0 0x7f0c081568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080add50 0x7f0c08147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c080b17b0 0x7f0c08157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080aa720 con 0x7f0c080add50 2021-04-03T03:16:56.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080aa460 con 0x7f0c080b17b0 2021-04-03T03:16:56.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080b0c50 con 0x7f0c080603f0 2021-04-03T03:16:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080add50 0x7f0c08147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080add50 0x7f0c08147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57626/0 (socket says 172.21.2.111:57626) 2021-04-03T03:16:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 -- 172.21.2.111:0/3103879609 learned_addr learned my addr 172.21.2.111:0/3103879609 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:56.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0d1e1700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080603f0 0x7f0c081568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 -- 172.21.2.111:0/3103879609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080603f0 msgr2=0x7f0c081568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080603f0 0x7f0c081568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 -- 172.21.2.111:0/3103879609 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c080b17b0 msgr2=0x7f0c08157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c080b17b0 0x7f0c08157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 -- 172.21.2.111:0/3103879609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080b0660 con 0x7f0c080add50 2021-04-03T03:16:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0c9e0700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080add50 0x7f0c08147b50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f0bf8007000 tx=0x7f0bf8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0bf67fc700 1 -- 172.21.2.111:0/3103879609 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf800ce60 con 0x7f0c080add50 2021-04-03T03:16:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0bf67fc700 1 -- 172.21.2.111:0/3103879609 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf8007750 con 0x7f0c080add50 2021-04-03T03:16:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c080b08b0 con 0x7f0c080add50 2021-04-03T03:16:56.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c08067ee0 con 0x7f0c080add50 2021-04-03T03:16:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0bf67fc700 1 -- 172.21.2.111:0/3103879609 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bf8007ab0 con 0x7f0c080add50 2021-04-03T03:16:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c08057660 con 0x7f0c080add50 2021-04-03T03:16:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0bf67fc700 1 -- 172.21.2.111:0/3103879609 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0bf8007c10 con 0x7f0c080add50 2021-04-03T03:16:56.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.397+0000 7f0bf67fc700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0be8057000 0x7f0be80594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.401+0000 7f0bf67fc700 1 -- 172.21.2.111:0/3103879609 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0bf80704e0 con 0x7f0c080add50 2021-04-03T03:16:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.401+0000 7f0c0d1e1700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0be8057000 0x7f0be80594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.401+0000 7f0c0d1e1700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0be8057000 0x7f0be80594c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f0c04017000 tx=0x7f0c04015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.401+0000 7f0bf67fc700 1 -- 172.21.2.111:0/3103879609 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0bf8045760 con 0x7f0c080add50 2021-04-03T03:16:56.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.493+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c08148850 con 0x7f0c080add50 2021-04-03T03:16:56.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.493+0000 7f0bf67fc700 1 -- 172.21.2.111:0/3103879609 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0bf8048d80 con 0x7f0c080add50 2021-04-03T03:16:56.497 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:56.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0be8057000 msgr2=0x7f0be80594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0be8057000 0x7f0be80594c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f0c04017000 tx=0x7f0c04015000).stop 2021-04-03T03:16:56.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080add50 msgr2=0x7f0c08147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080add50 0x7f0c08147b50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f0bf8007000 tx=0x7f0bf8008040).stop 2021-04-03T03:16:56.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 shutdown_connections 2021-04-03T03:16:56.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0be8057000 0x7f0be80594c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0c080b17b0 0x7f0c08157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0c080603f0 0x7f0c081568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 --2- 172.21.2.111:0/3103879609 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0c080add50 0x7f0c08147b50 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 >> 172.21.2.111:0/3103879609 conn(0x7f0c080b61c0 msgr2=0x7f0c080b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 shutdown_connections 2021-04-03T03:16:56.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.497+0000 7f0c0e1e3700 1 -- 172.21.2.111:0/3103879609 wait complete. 2021-04-03T03:16:56.505 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:16:56.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c8d5f700 1 Processor -- start 2021-04-03T03:16:56.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c8d5f700 1 -- start start 2021-04-03T03:16:56.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c8d5f700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f86c4144ef0 0x7f86c4147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c8d5f700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c40607d0 con 0x7f86c40add50 2021-04-03T03:16:56.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c8d5f700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c414b090 con 0x7f86c40603f0 2021-04-03T03:16:56.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c8d5f700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c414b330 con 0x7f86c4144ef0 2021-04-03T03:16:56.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c2ffd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f86c40add50 0x7f86c4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58428/0 (socket says 172.21.2.111:58428) 2021-04-03T03:16:56.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c2ffd700 1 -- 172.21.2.111:0/1369693209 learned_addr learned my addr 172.21.2.111:0/1369693209 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:56.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110494492 0 0) 0x7f86c40607d0 con 0x7f86c40add50 2021-04-03T03:16:56.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86a4003240 con 0x7f86c40add50 2021-04-03T03:16:56.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1139545495 0 0) 0x7f86a4003240 con 0x7f86c40add50 2021-04-03T03:16:56.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 >> v2:172.21.2.115:3300/0 conn(0x7f86c4144ef0 msgr2=0x7f86c4147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c3fff700 1 --2- 172.21.2.111:0/1369693209 >> v2:172.21.2.115:3300/0 conn(0x7f86c4144ef0 0x7f86c4147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 --2- 172.21.2.111:0/1369693209 >> v2:172.21.2.115:3300/0 conn(0x7f86c4144ef0 0x7f86c4147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 >> v1:172.21.2.115:6789/0 conn(0x7f86c40603f0 legacy=0x7f86c41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c414b5d0 con 0x7f86c40add50 2021-04-03T03:16:56.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f86b40016d0 con 0x7f86c40add50 2021-04-03T03:16:56.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 >> v1:172.21.2.111:6789/0 conn(0x7f86c40add50 legacy=0x7f86c4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86a4003800 0x7f86a4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86a40044e0 0x7f86a4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86a4004e50 0x7f86a400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c414b330 con 0x7f86a4003800 2021-04-03T03:16:56.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4005300 con 0x7f86a40044e0 2021-04-03T03:16:56.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a400bc20 con 0x7f86a4004e50 2021-04-03T03:16:56.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c3fff700 1 --2- 172.21.2.111:0/1369693209 >> v2:172.21.2.115:3300/0 conn(0x7f86c4144ef0 0x7f86c4147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:16:56.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.561+0000 7f86c37fe700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86a4003800 0x7f86a4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c37fe700 1 -- 172.21.2.111:0/1369693209 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86a4004e50 msgr2=0x7f86a400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c37fe700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86a4004e50 0x7f86a400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c37fe700 1 -- 172.21.2.111:0/1369693209 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86a40044e0 msgr2=0x7f86a4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c37fe700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86a40044e0 0x7f86a4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c37fe700 1 -- 172.21.2.111:0/1369693209 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a4003240 con 0x7f86a4003800 2021-04-03T03:16:56.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c3fff700 1 -- 172.21.2.111:0/1369693209 reap_dead start 2021-04-03T03:16:56.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c37fe700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86a4003800 0x7f86a4003c00 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f86ac001f60 tx=0x7f86ac007b70).ready entity=mon.0 client_cookie=d4e35231dcc8c350 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86ac00c870 con 0x7f86a4003800 2021-04-03T03:16:56.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac00c9d0 con 0x7f86a4003800 2021-04-03T03:16:56.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c27fc700 1 -- 172.21.2.111:0/1369693209 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86ac00cd30 con 0x7f86a4003800 2021-04-03T03:16:56.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/1369693209 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86a4003800 msgr2=0x7f86a4003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86a4003800 0x7f86a4003c00 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f86ac001f60 tx=0x7f86ac007b70).stop 2021-04-03T03:16:56.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/1369693209 shutdown_connections 2021-04-03T03:16:56.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/1369693209 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86a4003800 0x7f86a4003c00 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/1369693209 >> 172.21.2.111:0/1369693209 conn(0x7f86c40b61c0 msgr2=0x7f86c405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/1369693209 shutdown_connections 2021-04-03T03:16:56.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/1369693209 wait complete. 2021-04-03T03:16:56.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 Processor -- start 2021-04-03T03:16:56.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- start start 2021-04-03T03:16:56.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86a4003800 0x7f86c41480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86c40aa6a0 0x7f86c4157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86c41575f0 0x7f86c4159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4156770 con 0x7f86c40aa6a0 2021-04-03T03:16:56.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c41569c0 con 0x7f86a4003800 2021-04-03T03:16:56.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4156c60 con 0x7f86c41575f0 2021-04-03T03:16:56.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86c40aa6a0 0x7f86c4157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86c40aa6a0 0x7f86c4157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57646/0 (socket says 172.21.2.111:57646) 2021-04-03T03:16:56.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 -- 172.21.2.111:0/2112229141 learned_addr learned my addr 172.21.2.111:0/2112229141 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:56.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 -- 172.21.2.111:0/2112229141 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86c41575f0 msgr2=0x7f86c4159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86c41575f0 0x7f86c4159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 -- 172.21.2.111:0/2112229141 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86a4003800 msgr2=0x7f86c41480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86a4003800 0x7f86c41480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 -- 172.21.2.111:0/2112229141 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4067e50 con 0x7f86c40aa6a0 2021-04-03T03:16:56.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c2ffd700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86c40aa6a0 0x7f86c4157100 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f86c40b5600 tx=0x7f86c414b220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c0ff9700 1 -- 172.21.2.111:0/2112229141 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86b4007170 con 0x7f86c40aa6a0 2021-04-03T03:16:56.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c0ff9700 1 -- 172.21.2.111:0/2112229141 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b40072d0 con 0x7f86c40aa6a0 2021-04-03T03:16:56.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c40a82c0 con 0x7f86c40aa6a0 2021-04-03T03:16:56.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c415a210 con 0x7f86c40aa6a0 2021-04-03T03:16:56.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c0ff9700 1 -- 172.21.2.111:0/2112229141 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86b400a9a0 con 0x7f86c40aa6a0 2021-04-03T03:16:56.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c40aaaa0 con 0x7f86c40aa6a0 2021-04-03T03:16:56.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c0ff9700 1 -- 172.21.2.111:0/2112229141 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f86b401b060 con 0x7f86c40aa6a0 2021-04-03T03:16:56.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c0ff9700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f86a8056ea0 0x7f86a8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.565+0000 7f86c0ff9700 1 -- 172.21.2.111:0/2112229141 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f86b406fca0 con 0x7f86c40aa6a0 2021-04-03T03:16:56.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.569+0000 7f86c37fe700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f86a8056ea0 0x7f86a8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.569+0000 7f86c37fe700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f86a8056ea0 0x7f86a8059360 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f86ac001f60 tx=0x7f86ac016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.569+0000 7f86c0ff9700 1 -- 172.21.2.111:0/2112229141 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f86b4074070 con 0x7f86c40aa6a0 2021-04-03T03:16:56.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86c4156e20 con 0x7f86c40aa6a0 2021-04-03T03:16:56.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c0ff9700 1 -- 172.21.2.111:0/2112229141 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+10 (secure 0 0 0) 0x7f86b4019030 con 0x7f86c40aa6a0 2021-04-03T03:16:56.664 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:56.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f86a8056ea0 msgr2=0x7f86a8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f86a8056ea0 0x7f86a8059360 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f86ac001f60 tx=0x7f86ac016000).stop 2021-04-03T03:16:56.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86c40aa6a0 msgr2=0x7f86c4157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86c40aa6a0 0x7f86c4157100 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f86c40b5600 tx=0x7f86c414b220).stop 2021-04-03T03:16:56.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 shutdown_connections 2021-04-03T03:16:56.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f86c41575f0 0x7f86c4159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f86a8056ea0 0x7f86a8059360 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f86a4003800 0x7f86c41480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 --2- 172.21.2.111:0/2112229141 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f86c40aa6a0 0x7f86c4157100 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 >> 172.21.2.111:0/2112229141 conn(0x7f86c40b61c0 msgr2=0x7f86c40b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 shutdown_connections 2021-04-03T03:16:56.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.661+0000 7f86c8d5f700 1 -- 172.21.2.111:0/2112229141 wait complete. 2021-04-03T03:16:56.671 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:16:56.672 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:56.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a74a6a700 1 Processor -- start 2021-04-03T03:16:56.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a74a6a700 1 -- start start 2021-04-03T03:16:56.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a74a6a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1a700603f0 0x7f1a700b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a74a6a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f1a7014b700 0x7f1a700b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a74a6a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a700607d0 con 0x7f1a700b12b0 2021-04-03T03:16:56.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a74a6a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a700af400 con 0x7f1a7014b700 2021-04-03T03:16:56.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a74a6a700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a700af6a0 con 0x7f1a700603f0 2021-04-03T03:16:56.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a6ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1a700b12b0 0x7f1a701465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58446/0 (socket says 172.21.2.111:58446) 2021-04-03T03:16:56.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a6ffff700 1 -- 172.21.2.111:0/2814590132 learned_addr learned my addr 172.21.2.111:0/2814590132 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:56.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.729+0000 7f1a6effd700 1 --2- 172.21.2.111:0/2814590132 >> v2:172.21.2.111:3300/0 conn(0x7f1a7014b700 0x7f1a700b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6effd700 1 -- 172.21.2.111:0/2814590132 >> v2:172.21.2.115:3300/0 conn(0x7f1a700603f0 msgr2=0x7f1a700b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:56.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6effd700 1 --2- 172.21.2.111:0/2814590132 >> v2:172.21.2.115:3300/0 conn(0x7f1a700603f0 0x7f1a700b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6effd700 1 -- 172.21.2.111:0/2814590132 >> v1:172.21.2.111:6789/0 conn(0x7f1a700b12b0 legacy=0x7f1a701465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.737 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6effd700 1 -- 172.21.2.111:0/2814590132 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700afb40 con 0x7f1a7014b700 2021-04-03T03:16:56.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6effd700 1 --2- 172.21.2.111:0/2814590132 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a7014b700 0x7f1a700b0580 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f1a60002f30 tx=0x7f1a60006dd0).ready entity=mon.0 client_cookie=3aaf0d098a4660bd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6e7fc700 1 -- 172.21.2.111:0/2814590132 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a600097f0 con 0x7f1a7014b700 2021-04-03T03:16:56.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6e7fc700 1 -- 172.21.2.111:0/2814590132 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60009950 con 0x7f1a7014b700 2021-04-03T03:16:56.738 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6e7fc700 1 -- 172.21.2.111:0/2814590132 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a60009cb0 con 0x7f1a7014b700 2021-04-03T03:16:56.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- 172.21.2.111:0/2814590132 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a7014b700 msgr2=0x7f1a700b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/2814590132 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a7014b700 0x7f1a700b0580 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f1a60002f30 tx=0x7f1a60006dd0).stop 2021-04-03T03:16:56.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- 172.21.2.111:0/2814590132 shutdown_connections 2021-04-03T03:16:56.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/2814590132 >> v2:172.21.2.115:3300/0 conn(0x7f1a700603f0 0x7f1a700b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/2814590132 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a7014b700 0x7f1a700b0580 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- 172.21.2.111:0/2814590132 >> 172.21.2.111:0/2814590132 conn(0x7f1a700b61c0 msgr2=0x7f1a7005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- 172.21.2.111:0/2814590132 shutdown_connections 2021-04-03T03:16:56.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- 172.21.2.111:0/2814590132 wait complete. 2021-04-03T03:16:56.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 Processor -- start 2021-04-03T03:16:56.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- start start 2021-04-03T03:16:56.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a700603f0 0x7f1a70064c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a700b12b0 0x7f1a70062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a7014b700 0x7f1a70067590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700642a0 con 0x7f1a700603f0 2021-04-03T03:16:56.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a70064540 con 0x7f1a7014b700 2021-04-03T03:16:56.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700ab9e0 con 0x7f1a700b12b0 2021-04-03T03:16:56.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a700603f0 0x7f1a70064c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a700603f0 0x7f1a70064c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57660/0 (socket says 172.21.2.111:57660) 2021-04-03T03:16:56.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 -- 172.21.2.111:0/16445933 learned_addr learned my addr 172.21.2.111:0/16445933 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:56.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 -- 172.21.2.111:0/16445933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a700b12b0 msgr2=0x7f1a70062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a700b12b0 0x7f1a70062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 -- 172.21.2.111:0/16445933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a7014b700 msgr2=0x7f1a70067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a7014b700 0x7f1a70067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 -- 172.21.2.111:0/16445933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700a9e20 con 0x7f1a700603f0 2021-04-03T03:16:56.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6f7fe700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a700603f0 0x7f1a70064c80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f1a58002f30 tx=0x7f1a58006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6cff9700 1 -- 172.21.2.111:0/16445933 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a5800d840 con 0x7f1a700603f0 2021-04-03T03:16:56.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6cff9700 1 -- 172.21.2.111:0/16445933 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58000e90 con 0x7f1a700603f0 2021-04-03T03:16:56.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a700aa200 con 0x7f1a700603f0 2021-04-03T03:16:56.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a70067a80 con 0x7f1a700603f0 2021-04-03T03:16:56.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.733+0000 7f1a6cff9700 1 -- 172.21.2.111:0/16445933 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a5800db90 con 0x7f1a700603f0 2021-04-03T03:16:56.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.737+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a700aa010 con 0x7f1a700603f0 2021-04-03T03:16:56.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.737+0000 7f1a6cff9700 1 -- 172.21.2.111:0/16445933 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1a58017030 con 0x7f1a700603f0 2021-04-03T03:16:56.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.737+0000 7f1a6cff9700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a54056f70 0x7f1a54059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:56.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.737+0000 7f1a6cff9700 1 -- 172.21.2.111:0/16445933 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1a5806f6e0 con 0x7f1a700603f0 2021-04-03T03:16:56.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.737+0000 7f1a6effd700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a54056f70 0x7f1a54059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:56.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.737+0000 7f1a6effd700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a54056f70 0x7f1a54059430 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f1a60002f30 tx=0x7f1a60015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:56.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.737+0000 7f1a6cff9700 1 -- 172.21.2.111:0/16445933 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1a58044a70 con 0x7f1a700603f0 2021-04-03T03:16:56.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a70057660 con 0x7f1a700603f0 2021-04-03T03:16:56.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a6cff9700 1 -- 172.21.2.111:0/16445933 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f1a58048090 con 0x7f1a700603f0 2021-04-03T03:16:56.868 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:56.868 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":58,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3599494763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3599494763},{"type":"v1","addr":"172.21.2.115:6833","nonce":3599494763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:53.339332+0000","modified":"2021-04-03T03:16:54.343270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7890":{"gid":7890,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/233883066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":233883066},{"type":"v1","addr":"172.21.2.111:6835","nonce":233883066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-04-03T03:16:56.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a54056f70 msgr2=0x7f1a54059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a54056f70 0x7f1a54059430 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f1a60002f30 tx=0x7f1a60015000).stop 2021-04-03T03:16:56.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a700603f0 msgr2=0x7f1a70064c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:56.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a700603f0 0x7f1a70064c80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f1a58002f30 tx=0x7f1a58006ea0).stop 2021-04-03T03:16:56.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 shutdown_connections 2021-04-03T03:16:56.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a54056f70 0x7f1a54059430 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a7014b700 0x7f1a70067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a700603f0 0x7f1a70064c80 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:56.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 --2- 172.21.2.111:0/16445933 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a700b12b0 0x7f1a70062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:56.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 >> 172.21.2.111:0/16445933 conn(0x7f1a700b61c0 msgr2=0x7f1a70147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:56.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 shutdown_connections 2021-04-03T03:16:56.872 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:56.865+0000 7f1a74a6a700 1 -- 172.21.2.111:0/16445933 wait complete. 2021-04-03T03:16:56.872 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 58 2021-04-03T03:16:57.077 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:53.339332+0000', 'modified': '2021-04-03T03:16:54.343270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7890}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7890': {'gid': 7890, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/233883066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 233883066}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 233883066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:57.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:57.078 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:57.164 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7890,"want_state":"up:active","state":"up:active","rank_uptime":3.8179839420000001,"mdsmap_epoch":58,"osdmap_epoch":132,"osdmap_epoch_barrier":131,"uptime":8.011600133} 2021-04-03T03:16:57.165 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7890, "mdsmap_epoch": 58, "osdmap_epoch": 132, "osdmap_epoch_barrier": 131, "rank_uptime": 3.817983942, "state": "up:active", "uptime": 8.011600133, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:57.165 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:16:57.166 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:16:57.166 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:16:57.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:57.167 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:57.177 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:16:57.230 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:57.231 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:57.278 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:57.279 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:57.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:57.281 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:16:57.332 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:16:57.332 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:57.378 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:16:57.389 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:57.389 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:57.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:16:57.412+0000 7f8db13df0c0 -1 init, newargv = 0x5560ca220180 newargc=9 2021-04-03T03:16:57.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15983]: starting ceph client 2021-04-03T03:16:57.425 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:57.425 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:57.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15983]: starting fuse 2021-04-03T03:16:57.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:57.427 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:16:57.471 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:16:57.472 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:16:57.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:57.473 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:16:57.473 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:16:57.474 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:16:57.474 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:16:57.475 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:16:57.475 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:57.476 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:16:57.476 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:16:57.476 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:16:57.477 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:57.477 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:16:57.478 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:16:57.478 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:16:57.478 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:16:57.479 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:57.479 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:16:57.480 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:16:57.480 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:16:57.480 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:16:57.481 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:16:57.481 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:16:57.482 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:16:57.482 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:16:57.544 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.15983.asok 2021-04-03T03:16:57.546 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15983.asok 2021-04-03T03:16:57.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:57.547 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15983.asok status 2021-04-03T03:16:57.644 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:16:57.645 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:16:57.645 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:16:57.645 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:16:57.645 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:16:57.645 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:16:57.646 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:16:57.646 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "15983", 2021-04-03T03:16:57.646 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:16:57.646 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:57.647 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:16:57.647 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:16:57.647 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 5108, 2021-04-03T03:16:57.647 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:16:57.647 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:16:57.648 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:16:57.648 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 5108 2021-04-03T03:16:57.648 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:57.648 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:16:57.649 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:16:57.649 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:16:57.649 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3957335890 2021-04-03T03:16:57.649 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:16:57.649 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:57.650 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:16:57.650 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:16:57.650 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:16:57.650 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3957335890 2021-04-03T03:16:57.650 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:16:57.651 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.5108 172.21.2.126:0/3957335890", 2021-04-03T03:16:57.651 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/3957335890", 2021-04-03T03:16:57.651 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:16:57.652 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 58, 2021-04-03T03:16:57.652 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 132, 2021-04-03T03:16:57.652 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:16:57.652 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:16:57.652 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:16:57.653 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:16:57.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:57.654 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:57.655 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:57.660 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:16:57.660 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:16:57.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:57.661 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:16:57.713 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:16:57.774 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 Processor -- start 2021-04-03T03:16:57.775 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- start start 2021-04-03T03:16:57.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0603f0 0x7f163c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.776 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f163c14b700 0x7f163c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163c0607d0 con 0x7f163c0b12b0 2021-04-03T03:16:57.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c0af400 con 0x7f163c14b700 2021-04-03T03:16:57.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0af6a0 con 0x7f163c0603f0 2021-04-03T03:16:57.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163bfff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f163c0b12b0 0x7f163c1465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58458/0 (socket says 172.21.2.111:58458) 2021-04-03T03:16:57.777 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163affd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f163c14b700 0x7f163c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:57.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163bfff700 1 -- 172.21.2.111:0/4077136725 learned_addr learned my addr 172.21.2.111:0/4077136725 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:57.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163affd700 1 -- 172.21.2.111:0/4077136725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0603f0 msgr2=0x7f163c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:57.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163affd700 1 --2- 172.21.2.111:0/4077136725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0603f0 0x7f163c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163affd700 1 -- 172.21.2.111:0/4077136725 >> v1:172.21.2.111:6789/0 conn(0x7f163c0b12b0 legacy=0x7f163c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163affd700 1 -- 172.21.2.111:0/4077136725 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0afb40 con 0x7f163c14b700 2021-04-03T03:16:57.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163affd700 1 --2- 172.21.2.111:0/4077136725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c14b700 0x7f163c0b0580 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f1624002f30 tx=0x7f1624006dd0).ready entity=mon.0 client_cookie=735ecffaddd74e24 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:57.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163a7fc700 1 -- 172.21.2.111:0/4077136725 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16240097f0 con 0x7f163c14b700 2021-04-03T03:16:57.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163a7fc700 1 -- 172.21.2.111:0/4077136725 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624009950 con 0x7f163c14b700 2021-04-03T03:16:57.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f163a7fc700 1 -- 172.21.2.111:0/4077136725 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1624009cb0 con 0x7f163c14b700 2021-04-03T03:16:57.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- 172.21.2.111:0/4077136725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c14b700 msgr2=0x7f163c0b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 --2- 172.21.2.111:0/4077136725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c14b700 0x7f163c0b0580 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f1624002f30 tx=0x7f1624006dd0).stop 2021-04-03T03:16:57.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- 172.21.2.111:0/4077136725 shutdown_connections 2021-04-03T03:16:57.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 --2- 172.21.2.111:0/4077136725 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0603f0 0x7f163c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 --2- 172.21.2.111:0/4077136725 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c14b700 0x7f163c0b0580 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:57.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- 172.21.2.111:0/4077136725 >> 172.21.2.111:0/4077136725 conn(0x7f163c0b61c0 msgr2=0x7f163c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:57.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- 172.21.2.111:0/4077136725 shutdown_connections 2021-04-03T03:16:57.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.773+0000 7f1640b81700 1 -- 172.21.2.111:0/4077136725 wait complete. 2021-04-03T03:16:57.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 Processor -- start 2021-04-03T03:16:57.784 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 -- start start 2021-04-03T03:16:57.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c0603f0 0x7f163c14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0b12b0 0x7f163c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f163c14b700 0x7f163c0a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0aef30 con 0x7f163c0603f0 2021-04-03T03:16:57.785 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0a6f20 con 0x7f163c14b700 2021-04-03T03:16:57.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0a8e40 con 0x7f163c0b12b0 2021-04-03T03:16:57.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c0603f0 0x7f163c14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:57.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c0603f0 0x7f163c14e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57674/0 (socket says 172.21.2.111:57674) 2021-04-03T03:16:57.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 -- 172.21.2.111:0/3004072235 learned_addr learned my addr 172.21.2.111:0/3004072235 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:57.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 -- 172.21.2.111:0/3004072235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0b12b0 msgr2=0x7f163c156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0b12b0 0x7f163c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 -- 172.21.2.111:0/3004072235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f163c14b700 msgr2=0x7f163c0a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f163c14b700 0x7f163c0a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.787 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 -- 172.21.2.111:0/3004072235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0a7150 con 0x7f163c0603f0 2021-04-03T03:16:57.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163b7fe700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c0603f0 0x7f163c14e730 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f162c002f30 tx=0x7f162c006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:57.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1638ff9700 1 -- 172.21.2.111:0/3004072235 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f162c00d840 con 0x7f163c0603f0 2021-04-03T03:16:57.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1638ff9700 1 -- 172.21.2.111:0/3004072235 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c000e90 con 0x7f163c0603f0 2021-04-03T03:16:57.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1638ff9700 1 -- 172.21.2.111:0/3004072235 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f162c00db90 con 0x7f163c0603f0 2021-04-03T03:16:57.788 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c1568f0 con 0x7f163c0603f0 2021-04-03T03:16:57.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c0a8ad0 con 0x7f163c0603f0 2021-04-03T03:16:57.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c0a9d00 con 0x7f163c0603f0 2021-04-03T03:16:57.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1638ff9700 1 -- 172.21.2.111:0/3004072235 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f162c017030 con 0x7f163c0603f0 2021-04-03T03:16:57.789 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1638ff9700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1620056f70 0x7f1620059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f1638ff9700 1 -- 172.21.2.111:0/3004072235 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f162c06f710 con 0x7f163c0603f0 2021-04-03T03:16:57.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163affd700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1620056f70 0x7f1620059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:57.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.777+0000 7f163affd700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1620056f70 0x7f1620059430 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1624002f30 tx=0x7f1624015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:57.790 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.781+0000 7f1638ff9700 1 -- 172.21.2.111:0/3004072235 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f162c044aa0 con 0x7f163c0603f0 2021-04-03T03:16:57.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f163c0a72e0 con 0x7f163c0603f0 2021-04-03T03:16:57.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1638ff9700 1 -- 172.21.2.111:0/3004072235 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f162c0480c0 con 0x7f163c0603f0 2021-04-03T03:16:57.874 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:57.875 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":132,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:54.343386+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:51.364989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:52.312016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:16:57.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1620056f70 msgr2=0x7f1620059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1620056f70 0x7f1620059430 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1624002f30 tx=0x7f1624015000).stop 2021-04-03T03:16:57.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c0603f0 msgr2=0x7f163c14e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c0603f0 0x7f163c14e730 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f162c002f30 tx=0x7f162c006ea0).stop 2021-04-03T03:16:57.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 shutdown_connections 2021-04-03T03:16:57.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1620056f70 0x7f1620059430 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:57.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f163c14b700 0x7f163c0a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f163c0603f0 0x7f163c14e730 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:57.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 --2- 172.21.2.111:0/3004072235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f163c0b12b0 0x7f163c156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 >> 172.21.2.111:0/3004072235 conn(0x7f163c0b61c0 msgr2=0x7f163c0acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:57.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 shutdown_connections 2021-04-03T03:16:57.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.873+0000 7f1640b81700 1 -- 172.21.2.111:0/3004072235 wait complete. 2021-04-03T03:16:57.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:16:51.364989+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:16:52.312016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:16:57.883 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:16:57.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 Processor -- start 2021-04-03T03:16:57.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- start start 2021-04-03T03:16:57.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44440dbba0 0x7f44440dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4444041060 0x7f444403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44440dc560 con 0x7f444403d560 2021-04-03T03:16:57.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444403de60 con 0x7f4444041060 2021-04-03T03:16:57.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44440dc800 con 0x7f44440dbba0 2021-04-03T03:16:57.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4444041060 0x7f444403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:57.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4444041060 0x7f444403fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57684/0 (socket says 172.21.2.111:57684) 2021-04-03T03:16:57.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 -- 172.21.2.111:0/2475723649 learned_addr learned my addr 172.21.2.111:0/2475723649 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:57.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44512e5700 1 --2- 172.21.2.111:0/2475723649 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44440dbba0 0x7f44440dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:57.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 -- 172.21.2.111:0/2475723649 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44440dbba0 msgr2=0x7f44440dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:57.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 --2- 172.21.2.111:0/2475723649 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44440dbba0 0x7f44440dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:57.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 -- 172.21.2.111:0/2475723649 >> v1:172.21.2.115:6789/0 conn(0x7f444403d560 legacy=0x7f4444041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:57.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 -- 172.21.2.111:0/2475723649 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444403da30 con 0x7f4444041060 2021-04-03T03:16:57.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4450ae4700 1 --2- 172.21.2.111:0/2475723649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4444041060 0x7f444403fa80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f444403f960 tx=0x7f4440006e90).ready entity=mon.0 client_cookie=dd812c54210981a0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:57.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f443ffff700 1 -- 172.21.2.111:0/2475723649 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44400098e0 con 0x7f4444041060 2021-04-03T03:16:57.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f443ffff700 1 -- 172.21.2.111:0/2475723649 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440009a40 con 0x7f4444041060 2021-04-03T03:16:57.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f443ffff700 1 -- 172.21.2.111:0/2475723649 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4440009de0 con 0x7f4444041060 2021-04-03T03:16:57.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- 172.21.2.111:0/2475723649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4444041060 msgr2=0x7f444403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- 172.21.2.111:0/2475723649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4444041060 0x7f444403fa80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f444403f960 tx=0x7f4440006e90).stop 2021-04-03T03:16:57.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- 172.21.2.111:0/2475723649 shutdown_connections 2021-04-03T03:16:57.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- 172.21.2.111:0/2475723649 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f44440dbba0 0x7f44440dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:57.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- 172.21.2.111:0/2475723649 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4444041060 0x7f444403fa80 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:57.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- 172.21.2.111:0/2475723649 >> 172.21.2.111:0/2475723649 conn(0x7f44440462b0 msgr2=0x7f44440466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:57.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- 172.21.2.111:0/2475723649 shutdown_connections 2021-04-03T03:16:57.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- 172.21.2.111:0/2475723649 wait complete. 2021-04-03T03:16:57.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 Processor -- start 2021-04-03T03:16:57.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- start start 2021-04-03T03:16:57.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f444403d560 0x7f44440dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4444041060 0x7f444403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44440dbba0 0x7f444403a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440397b0 con 0x7f44440dbba0 2021-04-03T03:16:57.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444403ade0 con 0x7f444403d560 2021-04-03T03:16:57.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f44522e7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444403b080 con 0x7f4444041060 2021-04-03T03:16:57.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4451ae6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44440dbba0 0x7f444403a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:57.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4451ae6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44440dbba0 0x7f444403a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57688/0 (socket says 172.21.2.111:57688) 2021-04-03T03:16:57.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4451ae6700 1 -- 172.21.2.111:0/1911333119 learned_addr learned my addr 172.21.2.111:0/1911333119 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:57.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4451ae6700 1 -- 172.21.2.111:0/1911333119 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4444041060 msgr2=0x7f444403b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.941+0000 7f4451ae6700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4444041060 0x7f444403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f4451ae6700 1 -- 172.21.2.111:0/1911333119 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f444403d560 msgr2=0x7f44440dd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:57.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f4451ae6700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f444403d560 0x7f44440dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:57.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f4451ae6700 1 -- 172.21.2.111:0/1911333119 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444403aa70 con 0x7f44440dbba0 2021-04-03T03:16:57.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f4451ae6700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44440dbba0 0x7f444403a580 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f444c06b5d0 tx=0x7f444c05ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:57.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f443e7fc700 1 -- 172.21.2.111:0/1911333119 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f444c0699f0 con 0x7f44440dbba0 2021-04-03T03:16:57.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f443e7fc700 1 -- 172.21.2.111:0/1911333119 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c05f2c0 con 0x7f44440dbba0 2021-04-03T03:16:57.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f443e7fc700 1 -- 172.21.2.111:0/1911333119 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f444c05f620 con 0x7f44440dbba0 2021-04-03T03:16:57.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4444039c10 con 0x7f44440dbba0 2021-04-03T03:16:57.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444403a1b0 con 0x7f44440dbba0 2021-04-03T03:16:57.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4444039970 con 0x7f44440dbba0 2021-04-03T03:16:57.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f443e7fc700 1 -- 172.21.2.111:0/1911333119 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f444c08b020 con 0x7f44440dbba0 2021-04-03T03:16:57.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f443e7fc700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f442c057050 0x7f442c059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:57.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f443e7fc700 1 -- 172.21.2.111:0/1911333119 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f444c0def40 con 0x7f44440dbba0 2021-04-03T03:16:57.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f44512e5700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f442c057050 0x7f442c059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:57.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.945+0000 7f44512e5700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f442c057050 0x7f442c059510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f444800a000 tx=0x7f4448008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:57.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:57.949+0000 7f443e7fc700 1 -- 172.21.2.111:0/1911333119 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4444039970 con 0x7f44440dbba0 2021-04-03T03:16:58.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4444004db0 con 0x7f44440dbba0 2021-04-03T03:16:58.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f443e7fc700 1 -- 172.21.2.111:0/1911333119 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v132) v1 ==== 110+0+10 (secure 0 0 0) 0x7f444c0b43e0 con 0x7f44440dbba0 2021-04-03T03:16:58.043 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:16:58.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f442c057050 msgr2=0x7f442c059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f442c057050 0x7f442c059510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f444800a000 tx=0x7f4448008040).stop 2021-04-03T03:16:58.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44440dbba0 msgr2=0x7f444403a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44440dbba0 0x7f444403a580 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f444c06b5d0 tx=0x7f444c05ad70).stop 2021-04-03T03:16:58.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 shutdown_connections 2021-04-03T03:16:58.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f44440dbba0 0x7f444403a580 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f442c057050 0x7f442c059510 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f444403d560 0x7f44440dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 --2- 172.21.2.111:0/1911333119 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4444041060 0x7f444403b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 >> 172.21.2.111:0/1911333119 conn(0x7f44440462b0 msgr2=0x7f4444041860 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 shutdown_connections 2021-04-03T03:16:58.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.041+0000 7f44522e7700 1 -- 172.21.2.111:0/1911333119 wait complete. 2021-04-03T03:16:58.050 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:16:58.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 Processor -- start 2021-04-03T03:16:58.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 -- start start 2021-04-03T03:16:58.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc0740603f0 0x7fc0741447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc0740add50 0x7fc074144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0740607d0 con 0x7fc0740b17b0 2021-04-03T03:16:58.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0741451c0 con 0x7fc0740add50 2021-04-03T03:16:58.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07414b090 con 0x7fc0740603f0 2021-04-03T03:16:58.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc0740add50 0x7fc074144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc0740add50 0x7fc074144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57696/0 (socket says 172.21.2.111:57696) 2021-04-03T03:16:58.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 -- 172.21.2.111:0/135766702 learned_addr learned my addr 172.21.2.111:0/135766702 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:58.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 -- 172.21.2.111:0/135766702 >> v2:172.21.2.115:3300/0 conn(0x7fc0740603f0 msgr2=0x7fc0741447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 --2- 172.21.2.111:0/135766702 >> v2:172.21.2.115:3300/0 conn(0x7fc0740603f0 0x7fc0741447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 -- 172.21.2.111:0/135766702 >> v1:172.21.2.115:6789/0 conn(0x7fc0740b17b0 legacy=0x7fc074147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 -- 172.21.2.111:0/135766702 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07414b330 con 0x7fc0740add50 2021-04-03T03:16:58.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc0727fc700 1 --2- 172.21.2.111:0/135766702 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740add50 0x7fc074144cd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fc0740ae520 tx=0x7fc064006e90).ready entity=mon.0 client_cookie=ac000c863e6eee5a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc071ffb700 1 -- 172.21.2.111:0/135766702 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc064009930 con 0x7fc0740add50 2021-04-03T03:16:58.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc071ffb700 1 -- 172.21.2.111:0/135766702 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc064009a90 con 0x7fc0740add50 2021-04-03T03:16:58.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc071ffb700 1 -- 172.21.2.111:0/135766702 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc064009e30 con 0x7fc0740add50 2021-04-03T03:16:58.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 -- 172.21.2.111:0/135766702 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740add50 msgr2=0x7fc074144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.105+0000 7fc073fff700 1 --2- 172.21.2.111:0/135766702 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740add50 0x7fc074144cd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fc0740ae520 tx=0x7fc064006e90).stop 2021-04-03T03:16:58.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- 172.21.2.111:0/135766702 shutdown_connections 2021-04-03T03:16:58.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 --2- 172.21.2.111:0/135766702 >> v2:172.21.2.115:3300/0 conn(0x7fc0740603f0 0x7fc0741447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 --2- 172.21.2.111:0/135766702 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740add50 0x7fc074144cd0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- 172.21.2.111:0/135766702 >> 172.21.2.111:0/135766702 conn(0x7fc0740b61c0 msgr2=0x7fc07405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- 172.21.2.111:0/135766702 shutdown_connections 2021-04-03T03:16:58.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- 172.21.2.111:0/135766702 wait complete. 2021-04-03T03:16:58.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 Processor -- start 2021-04-03T03:16:58.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- start start 2021-04-03T03:16:58.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740603f0 0x7fc0740a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0740add50 0x7fc074062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0740b17b0 0x7fc074156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc074063670 con 0x7fc0740603f0 2021-04-03T03:16:58.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740aa960 con 0x7fc0740b17b0 2021-04-03T03:16:58.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740aab50 con 0x7fc0740add50 2021-04-03T03:16:58.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740603f0 0x7fc0740a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740603f0 0x7fc0740a7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57704/0 (socket says 172.21.2.111:57704) 2021-04-03T03:16:58.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 -- 172.21.2.111:0/3929981957 learned_addr learned my addr 172.21.2.111:0/3929981957 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:58.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 -- 172.21.2.111:0/3929981957 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0740add50 msgr2=0x7fc074062d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0740add50 0x7fc074062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 -- 172.21.2.111:0/3929981957 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0740b17b0 msgr2=0x7fc074156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0740b17b0 0x7fc074156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 -- 172.21.2.111:0/3929981957 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc074156a70 con 0x7fc0740603f0 2021-04-03T03:16:58.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc072ffd700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740603f0 0x7fc0740a7eb0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fc05c002f30 tx=0x7fc05c006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc06bfff700 1 -- 172.21.2.111:0/3929981957 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05c00d840 con 0x7fc0740603f0 2021-04-03T03:16:58.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc06bfff700 1 -- 172.21.2.111:0/3929981957 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c000e90 con 0x7fc0740603f0 2021-04-03T03:16:58.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc06bfff700 1 -- 172.21.2.111:0/3929981957 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05c00db90 con 0x7fc0740603f0 2021-04-03T03:16:58.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0740a9a70 con 0x7fc0740603f0 2021-04-03T03:16:58.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0740abd80 con 0x7fc0740603f0 2021-04-03T03:16:58.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc074156c60 con 0x7fc0740603f0 2021-04-03T03:16:58.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.109+0000 7fc06bfff700 1 -- 172.21.2.111:0/3929981957 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc05c017030 con 0x7fc0740603f0 2021-04-03T03:16:58.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.113+0000 7fc06bfff700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc058056ea0 0x7fc058059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.113+0000 7fc06bfff700 1 -- 172.21.2.111:0/3929981957 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc05c06f530 con 0x7fc0740603f0 2021-04-03T03:16:58.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.113+0000 7fc0727fc700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc058056ea0 0x7fc058059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.113+0000 7fc0727fc700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc058056ea0 0x7fc058059360 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fc0740ae520 tx=0x7fc064015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.113+0000 7fc06bfff700 1 -- 172.21.2.111:0/3929981957 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc05c0449d0 con 0x7fc0740603f0 2021-04-03T03:16:58.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc074057660 con 0x7fc0740603f0 2021-04-03T03:16:58.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc06bfff700 1 -- 172.21.2.111:0/3929981957 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc05c047ff0 con 0x7fc0740603f0 2021-04-03T03:16:58.208 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:58.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc058056ea0 msgr2=0x7fc058059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc058056ea0 0x7fc058059360 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fc0740ae520 tx=0x7fc064015000).stop 2021-04-03T03:16:58.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740603f0 msgr2=0x7fc0740a7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740603f0 0x7fc0740a7eb0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fc05c002f30 tx=0x7fc05c006ea0).stop 2021-04-03T03:16:58.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 shutdown_connections 2021-04-03T03:16:58.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc058056ea0 0x7fc058059360 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0740b17b0 0x7fc074156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0740603f0 0x7fc0740a7eb0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 --2- 172.21.2.111:0/3929981957 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0740add50 0x7fc074062d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 >> 172.21.2.111:0/3929981957 conn(0x7fc0740b61c0 msgr2=0x7fc0741447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 shutdown_connections 2021-04-03T03:16:58.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.205+0000 7fc073fff700 1 -- 172.21.2.111:0/3929981957 wait complete. 2021-04-03T03:16:58.216 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:16:58.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 Processor -- start 2021-04-03T03:16:58.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 -- start start 2021-04-03T03:16:58.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80603f0 0x7fadf81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadf80add50 0x7fadf8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf80607d0 con 0x7fadf80b17b0 2021-04-03T03:16:58.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf81451c0 con 0x7fadf80add50 2021-04-03T03:16:58.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf814b090 con 0x7fadf80603f0 2021-04-03T03:16:58.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadf80add50 0x7fadf8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fadf80add50 0x7fadf8144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57710/0 (socket says 172.21.2.111:57710) 2021-04-03T03:16:58.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 -- 172.21.2.111:0/3514074018 learned_addr learned my addr 172.21.2.111:0/3514074018 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:58.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 -- 172.21.2.111:0/3514074018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80603f0 msgr2=0x7fadf81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 --2- 172.21.2.111:0/3514074018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80603f0 0x7fadf81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 -- 172.21.2.111:0/3514074018 >> v1:172.21.2.115:6789/0 conn(0x7fadf80b17b0 legacy=0x7fadf8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 -- 172.21.2.111:0/3514074018 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf814b330 con 0x7fadf80add50 2021-04-03T03:16:58.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf7fff700 1 --2- 172.21.2.111:0/3514074018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80add50 0x7fadf8144cd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7fadf80b5600 tx=0x7fade8006e90).ready entity=mon.0 client_cookie=9f9f9d3d519c5b8e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf77fe700 1 -- 172.21.2.111:0/3514074018 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fade80098e0 con 0x7fadf80add50 2021-04-03T03:16:58.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf77fe700 1 -- 172.21.2.111:0/3514074018 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade8009a40 con 0x7fadf80add50 2021-04-03T03:16:58.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadf77fe700 1 -- 172.21.2.111:0/3514074018 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fade8009de0 con 0x7fadf80add50 2021-04-03T03:16:58.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 -- 172.21.2.111:0/3514074018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80add50 msgr2=0x7fadf8144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.281+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3514074018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80add50 0x7fadf8144cd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7fadf80b5600 tx=0x7fade8006e90).stop 2021-04-03T03:16:58.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- 172.21.2.111:0/3514074018 shutdown_connections 2021-04-03T03:16:58.289 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3514074018 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80603f0 0x7fadf81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3514074018 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80add50 0x7fadf8144cd0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- 172.21.2.111:0/3514074018 >> 172.21.2.111:0/3514074018 conn(0x7fadf80b61c0 msgr2=0x7fadf805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- 172.21.2.111:0/3514074018 shutdown_connections 2021-04-03T03:16:58.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- 172.21.2.111:0/3514074018 wait complete. 2021-04-03T03:16:58.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 Processor -- start 2021-04-03T03:16:58.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- start start 2021-04-03T03:16:58.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80603f0 0x7fadf8062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80add50 0x7fadf81439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadf80b17b0 0x7fadf8143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8157c20 con 0x7fadf80603f0 2021-04-03T03:16:58.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8157450 con 0x7fadf80b17b0 2021-04-03T03:16:58.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf81443c0 con 0x7fadf80add50 2021-04-03T03:16:58.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80603f0 0x7fadf8062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80603f0 0x7fadf8062d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57714/0 (socket says 172.21.2.111:57714) 2021-04-03T03:16:58.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 -- 172.21.2.111:0/3314087863 learned_addr learned my addr 172.21.2.111:0/3314087863 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:58.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 -- 172.21.2.111:0/3314087863 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80add50 msgr2=0x7fadf81439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80add50 0x7fadf81439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 -- 172.21.2.111:0/3314087863 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadf80b17b0 msgr2=0x7fadf8143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadf80b17b0 0x7fadf8143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 -- 172.21.2.111:0/3314087863 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80aa9e0 con 0x7fadf80603f0 2021-04-03T03:16:58.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfce4a700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80603f0 0x7fadf8062d90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7fade0002f30 tx=0x7fade0006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadf5ffb700 1 -- 172.21.2.111:0/3314087863 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fade000d770 con 0x7fadf80603f0 2021-04-03T03:16:58.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadf5ffb700 1 -- 172.21.2.111:0/3314087863 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade0000e90 con 0x7fadf80603f0 2021-04-03T03:16:58.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadf5ffb700 1 -- 172.21.2.111:0/3314087863 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fade000dac0 con 0x7fadf80603f0 2021-04-03T03:16:58.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf8157680 con 0x7fadf80603f0 2021-04-03T03:16:58.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf80a9910 con 0x7fadf80603f0 2021-04-03T03:16:58.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8157810 con 0x7fadf80603f0 2021-04-03T03:16:58.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadf5ffb700 1 -- 172.21.2.111:0/3314087863 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fade0017030 con 0x7fadf80603f0 2021-04-03T03:16:58.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadf5ffb700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faddc056ea0 0x7faddc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadf5ffb700 1 -- 172.21.2.111:0/3314087863 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fade006f500 con 0x7fadf80603f0 2021-04-03T03:16:58.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.285+0000 7fadf7fff700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faddc056ea0 0x7faddc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.289+0000 7fadf7fff700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faddc056ea0 0x7faddc059360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fadf80b5600 tx=0x7fade8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.289+0000 7fadf5ffb700 1 -- 172.21.2.111:0/3314087863 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fade00449a0 con 0x7fadf80603f0 2021-04-03T03:16:58.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadf8057660 con 0x7fadf80603f0 2021-04-03T03:16:58.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadf5ffb700 1 -- 172.21.2.111:0/3314087863 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+10 (secure 0 0 0) 0x7fade0047fc0 con 0x7fadf80603f0 2021-04-03T03:16:58.383 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:58.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faddc056ea0 msgr2=0x7faddc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faddc056ea0 0x7faddc059360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fadf80b5600 tx=0x7fade8015000).stop 2021-04-03T03:16:58.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80603f0 msgr2=0x7fadf8062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80603f0 0x7fadf8062d90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7fade0002f30 tx=0x7fade0006ea0).stop 2021-04-03T03:16:58.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 shutdown_connections 2021-04-03T03:16:58.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faddc056ea0 0x7faddc059360 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fadf80b17b0 0x7fadf8143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fadf80603f0 0x7fadf8062d90 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 --2- 172.21.2.111:0/3314087863 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fadf80add50 0x7fadf81439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 >> 172.21.2.111:0/3314087863 conn(0x7fadf80b61c0 msgr2=0x7fadf8144a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 shutdown_connections 2021-04-03T03:16:58.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.381+0000 7fadfde4c700 1 -- 172.21.2.111:0/3314087863 wait complete. 2021-04-03T03:16:58.391 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:16:58.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 Processor -- start 2021-04-03T03:16:58.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 -- start start 2021-04-03T03:16:58.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc0fc0603f0 0x7fc0fc1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0add50 0x7fc0fc144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc0fc0b17b0 0x7fc0fc147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc1451c0 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0607d0 con 0x7fc0fc0603f0 2021-04-03T03:16:58.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc14b090 con 0x7fc0fc0add50 2021-04-03T03:16:58.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc0fc0b17b0 0x7fc0fc147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc0fc0b17b0 0x7fc0fc147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57722/0 (socket says 172.21.2.111:57722) 2021-04-03T03:16:58.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.445+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1510515558 learned_addr learned my addr 172.21.2.111:0/1510515558 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:58.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1510515558 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0add50 msgr2=0x7fc0fc144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.452 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- 172.21.2.111:0/1510515558 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0add50 0x7fc0fc144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1510515558 >> v2:172.21.2.115:3300/0 conn(0x7fc0fc0603f0 msgr2=0x7fc0fc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- 172.21.2.111:0/1510515558 >> v2:172.21.2.115:3300/0 conn(0x7fc0fc0603f0 0x7fc0fc1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1510515558 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc14b330 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- 172.21.2.111:0/1510515558 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc147740 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7fc0f40032b0 tx=0x7fc0f4009270).ready entity=mon.0 client_cookie=3e9a560092a022ce server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0f959a700 1 -- 172.21.2.111:0/1510515558 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0f400bc00 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0f959a700 1 -- 172.21.2.111:0/1510515558 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f400bd60 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0f959a700 1 -- 172.21.2.111:0/1510515558 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0f4011400 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1510515558 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 msgr2=0x7fc0fc147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1510515558 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc147740 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7fc0f40032b0 tx=0x7fc0f4009270).stop 2021-04-03T03:16:58.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1510515558 shutdown_connections 2021-04-03T03:16:58.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1510515558 >> v2:172.21.2.115:3300/0 conn(0x7fc0fc0603f0 0x7fc0fc1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1510515558 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc147740 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1510515558 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0add50 0x7fc0fc144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1510515558 >> 172.21.2.111:0/1510515558 conn(0x7fc0fc0b61c0 msgr2=0x7fc0fc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1510515558 shutdown_connections 2021-04-03T03:16:58.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1510515558 wait complete. 2021-04-03T03:16:58.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 Processor -- start 2021-04-03T03:16:58.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- start start 2021-04-03T03:16:58.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0603f0 0x7fc0fc147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0fc0add50 0x7fc0fc0649c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc064eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0ab710 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0ab9b0 con 0x7fc0fc0add50 2021-04-03T03:16:58.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc063560 con 0x7fc0fc0603f0 2021-04-03T03:16:58.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc064eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc064eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57728/0 (socket says 172.21.2.111:57728) 2021-04-03T03:16:58.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1323675497 learned_addr learned my addr 172.21.2.111:0/1323675497 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:58.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1323675497 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0603f0 msgr2=0x7fc0fc147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0603f0 0x7fc0fc147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1323675497 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0fc0add50 msgr2=0x7fc0fc0649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0fc0add50 0x7fc0fc0649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 -- 172.21.2.111:0/1323675497 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc147d40 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fad9d700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc064eb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7fc0f40032b0 tx=0x7fc0f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0f37fe700 1 -- 172.21.2.111:0/1323675497 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0f401f420 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0f37fe700 1 -- 172.21.2.111:0/1323675497 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f401f580 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc0aaaf0 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc0abe50 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0f37fe700 1 -- 172.21.2.111:0/1323675497 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0f401f8e0 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.449+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc0ab570 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.453+0000 7fc0f37fe700 1 -- 172.21.2.111:0/1323675497 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc0f4026030 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.453+0000 7fc0f37fe700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc0e0056ef0 0x7fc0e00593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.453+0000 7fc0f37fe700 1 -- 172.21.2.111:0/1323675497 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc0f4070f10 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.453+0000 7fc0fa59c700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc0e0056ef0 0x7fc0e00593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.453+0000 7fc0fa59c700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc0e0056ef0 0x7fc0e00593b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fc0ec002f30 tx=0x7fc0ec006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.453+0000 7fc0f37fe700 1 -- 172.21.2.111:0/1323675497 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc0fc0ab570 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.545+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0fc0ab570 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0f37fe700 1 -- 172.21.2.111:0/1323675497 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc0fc0ab570 con 0x7fc0fc0b17b0 2021-04-03T03:16:58.550 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:16:58.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc0e0056ef0 msgr2=0x7fc0e00593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc0e0056ef0 0x7fc0e00593b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fc0ec002f30 tx=0x7fc0ec006a60).stop 2021-04-03T03:16:58.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 msgr2=0x7fc0fc064eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc064eb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7fc0f40032b0 tx=0x7fc0f4016000).stop 2021-04-03T03:16:58.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 shutdown_connections 2021-04-03T03:16:58.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc0fc0b17b0 0x7fc0fc064eb0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc0fc0603f0 0x7fc0fc147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc0e0056ef0 0x7fc0e00593b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 --2- 172.21.2.111:0/1323675497 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc0fc0add50 0x7fc0fc0649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 >> 172.21.2.111:0/1323675497 conn(0x7fc0fc0b61c0 msgr2=0x7fc0fc149c10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 shutdown_connections 2021-04-03T03:16:58.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.549+0000 7fc0fb59e700 1 -- 172.21.2.111:0/1323675497 wait complete. 2021-04-03T03:16:58.557 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:16:58.558 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:16:58.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.609+0000 7fc46359e700 1 Processor -- start 2021-04-03T03:16:58.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.609+0000 7fc46359e700 1 -- start start 2021-04-03T03:16:58.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4640b12b0 0x7fc4641522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4640607d0 con 0x7fc46414b700 2021-04-03T03:16:58.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4640b0eb0 con 0x7fc4640603f0 2021-04-03T03:16:58.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640af400 con 0x7fc4640b12b0 2021-04-03T03:16:58.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc46414b700 0x7fc4640b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58530/0 (socket says 172.21.2.111:58530) 2021-04-03T03:16:58.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 -- 172.21.2.111:0/1760202098 learned_addr learned my addr 172.21.2.111:0/1760202098 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:16:58.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749744164 0 0) 0x7fc4640607d0 con 0x7fc46414b700 2021-04-03T03:16:58.617 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc444003240 con 0x7fc46414b700 2021-04-03T03:16:58.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc462d9d700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4640b12b0 0x7fc4641522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3713680697 0 0) 0x7fc444003240 con 0x7fc46414b700 2021-04-03T03:16:58.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4640b12b0 msgr2=0x7fc4641522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4640b12b0 0x7fc4641522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 >> v1:172.21.2.115:6789/0 conn(0x7fc4640603f0 legacy=0x7fc4640b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640af6a0 con 0x7fc46414b700 2021-04-03T03:16:58.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fc4540016d0 con 0x7fc46414b700 2021-04-03T03:16:58.619 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 >> v1:172.21.2.111:6789/0 conn(0x7fc46414b700 legacy=0x7fc4640b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc444003760 0x7fc444003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4440044e0 0x7fc444004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4640b12b0 0x7fc444004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640af400 con 0x7fc4440044e0 2021-04-03T03:16:58.620 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc444005340 con 0x7fc444003760 2021-04-03T03:16:58.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44400b730 con 0x7fc4640b12b0 2021-04-03T03:16:58.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4440044e0 0x7fc444004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 -- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4640b12b0 msgr2=0x7fc444004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc4640b12b0 0x7fc444004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.621 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 -- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc444003760 msgr2=0x7fc444003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:16:58.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc444003760 0x7fc444003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 -- 172.21.2.111:0/1760202098 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444003240 con 0x7fc4440044e0 2021-04-03T03:16:58.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc461d9b700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4440044e0 0x7fc444004960 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7fc4540013e0 tx=0x7fc4540076a0).ready entity=mon.0 client_cookie=4da898e038f82701 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc45400ce80 con 0x7fc4440044e0 2021-04-03T03:16:58.622 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4540070d0 con 0x7fc4440044e0 2021-04-03T03:16:58.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46159a700 1 -- 172.21.2.111:0/1760202098 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc454007430 con 0x7fc4440044e0 2021-04-03T03:16:58.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- 172.21.2.111:0/1760202098 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4440044e0 msgr2=0x7fc444004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 --2- 172.21.2.111:0/1760202098 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4440044e0 0x7fc444004960 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7fc4540013e0 tx=0x7fc4540076a0).stop 2021-04-03T03:16:58.623 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc462d9d700 1 -- 172.21.2.111:0/1760202098 reap_dead start 2021-04-03T03:16:58.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- 172.21.2.111:0/1760202098 shutdown_connections 2021-04-03T03:16:58.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- 172.21.2.111:0/1760202098 >> 172.21.2.111:0/1760202098 conn(0x7fc4640b61c0 msgr2=0x7fc46405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- 172.21.2.111:0/1760202098 shutdown_connections 2021-04-03T03:16:58.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- 172.21.2.111:0/1760202098 wait complete. 2021-04-03T03:16:58.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 Processor -- start 2021-04-03T03:16:58.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.613+0000 7fc46359e700 1 -- start start 2021-04-03T03:16:58.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc46414cde0 0x7fc4640ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4640ab0f0 0x7fc464062ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc464159e50 0x7fc46415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640aae80 con 0x7fc4640ab0f0 2021-04-03T03:16:58.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4641599f0 con 0x7fc464159e50 2021-04-03T03:16:58.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc464065330 con 0x7fc46414cde0 2021-04-03T03:16:58.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4640ab0f0 0x7fc464062ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4640ab0f0 0x7fc464062ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57748/0 (socket says 172.21.2.111:57748) 2021-04-03T03:16:58.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 -- 172.21.2.111:0/2407001652 learned_addr learned my addr 172.21.2.111:0/2407001652 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:16:58.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 -- 172.21.2.111:0/2407001652 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc46414cde0 msgr2=0x7fc4640ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc46414cde0 0x7fc4640ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 -- 172.21.2.111:0/2407001652 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc464159e50 msgr2=0x7fc46415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc464159e50 0x7fc46415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 -- 172.21.2.111:0/2407001652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46414d1c0 con 0x7fc4640ab0f0 2021-04-03T03:16:58.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc461d9b700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4640ab0f0 0x7fc464062ae0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fc4540013e0 tx=0x7fc454013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc45b7fe700 1 -- 172.21.2.111:0/2407001652 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc454007420 con 0x7fc4640ab0f0 2021-04-03T03:16:58.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc45b7fe700 1 -- 172.21.2.111:0/2407001652 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454006e00 con 0x7fc4640ab0f0 2021-04-03T03:16:58.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc45b7fe700 1 -- 172.21.2.111:0/2407001652 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc454006300 con 0x7fc4640ab0f0 2021-04-03T03:16:58.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc464065090 con 0x7fc4640ab0f0 2021-04-03T03:16:58.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4640a90c0 con 0x7fc4640ab0f0 2021-04-03T03:16:58.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc464159b60 con 0x7fc4640ab0f0 2021-04-03T03:16:58.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc45b7fe700 1 -- 172.21.2.111:0/2407001652 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc45401c070 con 0x7fc4640ab0f0 2021-04-03T03:16:58.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc45b7fe700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc448056ea0 0x7fc448059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:16:58.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc45b7fe700 1 -- 172.21.2.111:0/2407001652 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc454070af0 con 0x7fc4640ab0f0 2021-04-03T03:16:58.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46259c700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc448056ea0 0x7fc448059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:16:58.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.617+0000 7fc46259c700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc448056ea0 0x7fc448059360 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fc4640620f0 tx=0x7fc44c0077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:16:58.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.621+0000 7fc45b7fe700 1 -- 172.21.2.111:0/2407001652 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc454045ee0 con 0x7fc4640ab0f0 2021-04-03T03:16:58.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.741+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc464057660 con 0x7fc4640ab0f0 2021-04-03T03:16:58.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc45b7fe700 1 -- 172.21.2.111:0/2407001652 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fc454049500 con 0x7fc4640ab0f0 2021-04-03T03:16:58.746 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:58.746 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":58,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3599494763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3599494763},{"type":"v1","addr":"172.21.2.115:6833","nonce":3599494763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:53.339332+0000","modified":"2021-04-03T03:16:54.343270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7890":{"gid":7890,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/233883066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":233883066},{"type":"v1","addr":"172.21.2.111:6835","nonce":233883066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-04-03T03:16:58.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc448056ea0 msgr2=0x7fc448059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc448056ea0 0x7fc448059360 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fc4640620f0 tx=0x7fc44c0077f0).stop 2021-04-03T03:16:58.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4640ab0f0 msgr2=0x7fc464062ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:16:58.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4640ab0f0 0x7fc464062ae0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fc4540013e0 tx=0x7fc454013000).stop 2021-04-03T03:16:58.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 shutdown_connections 2021-04-03T03:16:58.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc448056ea0 0x7fc448059360 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc464159e50 0x7fc46415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc46414cde0 0x7fc4640ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:16:58.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 --2- 172.21.2.111:0/2407001652 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc4640ab0f0 0x7fc464062ae0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:16:58.749 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 >> 172.21.2.111:0/2407001652 conn(0x7fc4640b61c0 msgr2=0x7fc4641494a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:16:58.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 shutdown_connections 2021-04-03T03:16:58.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:16:58.745+0000 7fc46359e700 1 -- 172.21.2.111:0/2407001652 wait complete. 2021-04-03T03:16:58.750 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 58 2021-04-03T03:16:58.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:53.339332+0000', 'modified': '2021-04-03T03:16:54.343270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7890}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7890': {'gid': 7890, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/233883066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 233883066}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 233883066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:16:58.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:16:58.956 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:16:59.028 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7890,"want_state":"up:active","state":"up:active","rank_uptime":5.6815137709999997,"mdsmap_epoch":58,"osdmap_epoch":132,"osdmap_epoch_barrier":131,"uptime":9.875131927} 2021-04-03T03:16:59.028 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7890, "mdsmap_epoch": 58, "osdmap_epoch": 132, "osdmap_epoch_barrier": 131, "rank_uptime": 5.681513771, "state": "up:active", "uptime": 9.875131927, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:16:59.028 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:16:59.029 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:16:59.029 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:16:59.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.030 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.040 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:16:59.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:59.091 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:59.139 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:59.140 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:59.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.141 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:16:59.193 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:16:59.193 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.239 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:16:59.246 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:16:59.246 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:16:59.260 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:59.263 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:59.266 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:59.269 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:59.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:16:59.272+0000 7fb353c1e0c0 -1 init, newargv = 0x5559d02bf9d0 newargc=9 2021-04-03T03:16:59.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15478]: starting ceph client 2021-04-03T03:16:59.279 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:16:59.283 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:16:59.284 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:16:59.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.285 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:16:59.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15478]: starting fuse 2021-04-03T03:16:59.289 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:16:59.289 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:16:59.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.290 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:16:59.291 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:16:59.291 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:16:59.291 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:16:59.292 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:16:59.292 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:59.293 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:16:59.293 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:16:59.293 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:16:59.294 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:59.294 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:16:59.295 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:16:59.295 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:16:59.295 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:16:59.296 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:59.296 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:16:59.297 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:16:59.297 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:16:59.297 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:16:59.298 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:16:59.298 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:16:59.299 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:16:59.299 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:16:59.370 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15478.asok 2021-04-03T03:16:59.373 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15478.asok 2021-04-03T03:16:59.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.374 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15478.asok status 2021-04-03T03:16:59.476 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:16:59.477 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:16:59.477 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:16:59.477 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:16:59.477 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:16:59.478 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:16:59.478 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:16:59.478 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15478", 2021-04-03T03:16:59.478 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:16:59.478 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:59.479 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:16:59.479 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:16:59.479 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 5590, 2021-04-03T03:16:59.479 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:16:59.480 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:16:59.480 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:16:59.480 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 5590 2021-04-03T03:16:59.480 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:59.481 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:16:59.481 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:16:59.481 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:16:59.481 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 3147115186 2021-04-03T03:16:59.481 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:16:59.482 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:59.482 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:16:59.482 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:16:59.482 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:16:59.482 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 3147115186 2021-04-03T03:16:59.483 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:16:59.483 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.5590 172.21.2.128:0/3147115186", 2021-04-03T03:16:59.483 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/3147115186", 2021-04-03T03:16:59.483 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:16:59.483 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 58, 2021-04-03T03:16:59.484 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 132, 2021-04-03T03:16:59.484 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:16:59.484 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:16:59.484 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:16:59.484 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:16:59.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.486 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:59.486 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.491 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:59.492 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:59.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.493 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.542 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:59.542 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.593 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:59.594 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:59.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.595 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:59.595 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.647 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:16:59.648 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:16:59.648 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:16:59.649 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:16:59.649 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15478]: fuse finished with error 0 and tester_r 0 2021-04-03T03:16:59.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:16:59.728 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:16:59.729 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.777 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:16:59.778 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:16:59.778 INFO:teuthology.orchestra.run:waiting for 900 2021-04-03T03:16:59.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:16:59.875 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:16:59.880 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:16:59.883 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:16:59.886 DEBUG:teuthology.orchestra.run.gibba026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:16:59.889 DEBUG:teuthology.orchestra.run.gibba028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:16:59.923 DEBUG:teuthology.orchestra.run.gibba031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:16:59.928 INFO:teuthology.orchestra.run.gibba026.stdout:[global] 2021-04-03T03:16:59.928 INFO:teuthology.orchestra.run.gibba026.stdout: chdir = "" 2021-04-03T03:16:59.928 INFO:teuthology.orchestra.run.gibba026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:16:59.929 INFO:teuthology.orchestra.run.gibba026.stdout: auth supported = cephx 2021-04-03T03:16:59.929 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.929 INFO:teuthology.orchestra.run.gibba026.stdout: filestore xattr use omap = true 2021-04-03T03:16:59.929 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.929 INFO:teuthology.orchestra.run.gibba026.stdout: mon clock drift allowed = 1.000 2021-04-03T03:16:59.930 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.930 INFO:teuthology.orchestra.run.gibba026.stdout: osd crush chooseleaf type = 0 2021-04-03T03:16:59.930 INFO:teuthology.orchestra.run.gibba026.stdout: auth debug = true 2021-04-03T03:16:59.930 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.931 INFO:teuthology.orchestra.run.gibba026.stdout: ms die on old message = true 2021-04-03T03:16:59.931 INFO:teuthology.orchestra.run.gibba026.stdout: ms die on bug = true 2021-04-03T03:16:59.931 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.931 INFO:teuthology.orchestra.run.gibba026.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:16:59.931 INFO:teuthology.orchestra.run.gibba026.stdout: mon pg warn max object skew = 0 2021-04-03T03:16:59.932 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.932 INFO:teuthology.orchestra.run.gibba026.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:16:59.932 INFO:teuthology.orchestra.run.gibba026.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:16:59.932 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.933 INFO:teuthology.orchestra.run.gibba026.stdout: osd pool default size = 2 2021-04-03T03:16:59.933 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.933 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd allow primary affinity = true 2021-04-03T03:16:59.933 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd allow pg remap = true 2021-04-03T03:16:59.933 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:16:59.934 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:16:59.934 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on no sortbitwise = false 2021-04-03T03:16:59.934 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:16:59.934 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on too few osds = false 2021-04-03T03:16:59.935 INFO:teuthology.orchestra.run.gibba026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:16:59.935 INFO:teuthology.orchestra.run.gibba026.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:16:59.935 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.935 INFO:teuthology.orchestra.run.gibba026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:16:59.935 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.936 INFO:teuthology.orchestra.run.gibba026.stdout: osd default data pool replay window = 5 2021-04-03T03:16:59.936 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.936 INFO:teuthology.orchestra.run.gibba026.stdout: mon allow pool delete = true 2021-04-03T03:16:59.936 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.937 INFO:teuthology.orchestra.run.gibba026.stdout: mon cluster log file level = debug 2021-04-03T03:16:59.937 INFO:teuthology.orchestra.run.gibba026.stdout: debug asserts on shutdown = true 2021-04-03T03:16:59.937 INFO:teuthology.orchestra.run.gibba026.stdout: mon health detail to clog = false 2021-04-03T03:16:59.937 INFO:teuthology.orchestra.run.gibba026.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:16:59.938 INFO:teuthology.orchestra.run.gibba026.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:16:59.938 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.938 INFO:teuthology.orchestra.run.gibba026.stdout:[osd] 2021-04-03T03:16:59.938 INFO:teuthology.orchestra.run.gibba026.stdout: osd journal size = 100 2021-04-03T03:16:59.938 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.939 INFO:teuthology.orchestra.run.gibba026.stdout: osd scrub load threshold = 5.0 2021-04-03T03:16:59.939 INFO:teuthology.orchestra.run.gibba026.stdout: osd scrub max interval = 600 2021-04-03T03:16:59.939 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.939 INFO:teuthology.orchestra.run.gibba026.stdout: osd recover clone overlap = true 2021-04-03T03:16:59.939 INFO:teuthology.orchestra.run.gibba026.stdout: osd recovery max chunk = 1048576 2021-04-03T03:16:59.940 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.940 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug shutdown = true 2021-04-03T03:16:59.940 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug op order = true 2021-04-03T03:16:59.940 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug verify stray on activate = true 2021-04-03T03:16:59.940 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.941 INFO:teuthology.orchestra.run.gibba026.stdout: osd open classes on start = true 2021-04-03T03:16:59.941 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug pg log writeout = true 2021-04-03T03:16:59.941 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.941 INFO:teuthology.orchestra.run.gibba026.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:16:59.942 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.942 INFO:teuthology.orchestra.run.gibba026.stdout: osd map max advance = 10 2021-04-03T03:16:59.942 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.942 INFO:teuthology.orchestra.run.gibba026.stdout: journal zero on create = true 2021-04-03T03:16:59.942 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.942 INFO:teuthology.orchestra.run.gibba026.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:16:59.943 INFO:teuthology.orchestra.run.gibba026.stdout: filestore apply finisher threads = 3 2021-04-03T03:16:59.943 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.943 INFO:teuthology.orchestra.run.gibba026.stdout: bdev debug aio = true 2021-04-03T03:16:59.943 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug misdirected ops = true 2021-04-03T03:16:59.944 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore block size = 96636764160 2021-04-03T03:16:59.944 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore compression mode = aggressive 2021-04-03T03:16:59.944 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore fsck on mount = True 2021-04-03T03:16:59.944 INFO:teuthology.orchestra.run.gibba026.stdout: debug bluefs = 20 2021-04-03T03:16:59.944 INFO:teuthology.orchestra.run.gibba026.stdout: debug bluestore = 20 2021-04-03T03:16:59.945 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:16:59.945 INFO:teuthology.orchestra.run.gibba026.stdout: debug osd = 20 2021-04-03T03:16:59.945 INFO:teuthology.orchestra.run.gibba026.stdout: debug rocksdb = 10 2021-04-03T03:16:59.945 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:16:59.945 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd full ratio = 0.9 2021-04-03T03:16:59.946 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:16:59.946 INFO:teuthology.orchestra.run.gibba026.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:16:59.946 INFO:teuthology.orchestra.run.gibba026.stdout: osd objectstore = bluestore 2021-04-03T03:16:59.946 INFO:teuthology.orchestra.run.gibba026.stdout: osd op complaint time = 180 2021-04-03T03:16:59.946 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.947 INFO:teuthology.orchestra.run.gibba026.stdout:[mgr] 2021-04-03T03:16:59.947 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:16:59.947 INFO:teuthology.orchestra.run.gibba026.stdout: debug mgr = 20 2021-04-03T03:16:59.947 INFO:teuthology.orchestra.run.gibba026.stdout: debug mon = 20 2021-04-03T03:16:59.948 INFO:teuthology.orchestra.run.gibba026.stdout: debug auth = 20 2021-04-03T03:16:59.948 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:16:59.948 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:16:59.948 INFO:teuthology.orchestra.run.gibba026.stdout: mgr/telemetry/nag = false 2021-04-03T03:16:59.949 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.949 INFO:teuthology.orchestra.run.gibba026.stdout:[mon] 2021-04-03T03:16:59.949 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:16:59.949 INFO:teuthology.orchestra.run.gibba026.stdout: debug mon = 20 2021-04-03T03:16:59.949 INFO:teuthology.orchestra.run.gibba026.stdout: debug paxos = 20 2021-04-03T03:16:59.950 INFO:teuthology.orchestra.run.gibba026.stdout: debug auth = 20 2021-04-03T03:16:59.950 INFO:teuthology.orchestra.run.gibba026.stdout: mon data avail warn = 5 2021-04-03T03:16:59.950 INFO:teuthology.orchestra.run.gibba026.stdout: mon mgr mkfs grace = 240 2021-04-03T03:16:59.950 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:16:59.950 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd reporter subtree level = osd 2021-04-03T03:16:59.951 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd prime pg temp = true 2021-04-03T03:16:59.951 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:16:59.951 INFO:teuthology.orchestra.run.gibba026.stdout: mon op complaint time = 120 2021-04-03T03:16:59.951 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:16:59.951 INFO:teuthology.orchestra.run.gibba026.stdout:[client] 2021-04-03T03:16:59.952 INFO:teuthology.orchestra.run.gibba026.stdout: rgw cache enabled = true 2021-04-03T03:16:59.952 INFO:teuthology.orchestra.run.gibba026.stdout: rgw enable ops log = true 2021-04-03T03:16:59.952 INFO:teuthology.orchestra.run.gibba026.stdout: rgw enable usage log = true 2021-04-03T03:16:59.952 INFO:teuthology.orchestra.run.gibba026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:16:59.952 INFO:teuthology.orchestra.run.gibba026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:16:59.953 INFO:teuthology.orchestra.run.gibba026.stdout: client mount timeout = 600 2021-04-03T03:16:59.953 INFO:teuthology.orchestra.run.gibba026.stdout: debug client = 20 2021-04-03T03:16:59.953 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:16:59.953 INFO:teuthology.orchestra.run.gibba026.stdout: rados mon op timeout = 15m 2021-04-03T03:16:59.953 INFO:teuthology.orchestra.run.gibba026.stdout: rados osd op timeout = 15m 2021-04-03T03:16:59.954 INFO:teuthology.orchestra.run.gibba026.stdout: client debug getattr caps = true 2021-04-03T03:16:59.954 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.a] 2021-04-03T03:16:59.954 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.b] 2021-04-03T03:16:59.954 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.c] 2021-04-03T03:16:59.954 INFO:teuthology.orchestra.run.gibba026.stdout:[mds.a] 2021-04-03T03:16:59.955 INFO:teuthology.orchestra.run.gibba026.stdout:[mds.b] 2021-04-03T03:16:59.955 INFO:teuthology.orchestra.run.gibba026.stdout:[mds] 2021-04-03T03:16:59.955 INFO:teuthology.orchestra.run.gibba026.stdout: debug mds = 20 2021-04-03T03:16:59.955 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:16:59.955 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal frag = True 2021-04-03T03:16:59.956 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal fragment size max = 10000 2021-04-03T03:16:59.956 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal merge size = 5 2021-04-03T03:16:59.956 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal split bits = 3 2021-04-03T03:16:59.956 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal split size = 100 2021-04-03T03:16:59.956 INFO:teuthology.orchestra.run.gibba026.stdout: mds debug frag = True 2021-04-03T03:16:59.956 INFO:teuthology.orchestra.run.gibba026.stdout: mds debug scatterstat = True 2021-04-03T03:16:59.957 INFO:teuthology.orchestra.run.gibba026.stdout: mds op complaint time = 180 2021-04-03T03:16:59.957 INFO:teuthology.orchestra.run.gibba026.stdout: mds verify scatter = True 2021-04-03T03:16:59.957 INFO:teuthology.orchestra.run.gibba026.stdout: osd op complaint time = 180 2021-04-03T03:16:59.957 INFO:teuthology.orchestra.run.gibba026.stdout: rados mon op timeout = 15m 2021-04-03T03:16:59.958 INFO:teuthology.orchestra.run.gibba026.stdout: rados osd op timeout = 15m 2021-04-03T03:16:59.958 INFO:teuthology.orchestra.run.gibba015.stdout:[global] 2021-04-03T03:16:59.958 INFO:teuthology.orchestra.run.gibba015.stdout: chdir = "" 2021-04-03T03:16:59.958 INFO:teuthology.orchestra.run.gibba015.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:16:59.958 INFO:teuthology.orchestra.run.gibba015.stdout: auth supported = cephx 2021-04-03T03:16:59.959 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.959 INFO:teuthology.orchestra.run.gibba015.stdout: filestore xattr use omap = true 2021-04-03T03:16:59.959 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.959 INFO:teuthology.orchestra.run.gibba015.stdout: mon clock drift allowed = 1.000 2021-04-03T03:16:59.959 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.960 INFO:teuthology.orchestra.run.gibba015.stdout: osd crush chooseleaf type = 0 2021-04-03T03:16:59.960 INFO:teuthology.orchestra.run.gibba015.stdout: auth debug = true 2021-04-03T03:16:59.960 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.960 INFO:teuthology.orchestra.run.gibba015.stdout: ms die on old message = true 2021-04-03T03:16:59.960 INFO:teuthology.orchestra.run.gibba015.stdout: ms die on bug = true 2021-04-03T03:16:59.961 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.961 INFO:teuthology.orchestra.run.gibba015.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:16:59.961 INFO:teuthology.orchestra.run.gibba015.stdout: mon pg warn max object skew = 0 2021-04-03T03:16:59.961 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.961 INFO:teuthology.orchestra.run.gibba015.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:16:59.962 INFO:teuthology.orchestra.run.gibba015.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:16:59.962 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.962 INFO:teuthology.orchestra.run.gibba015.stdout: osd pool default size = 2 2021-04-03T03:16:59.962 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.962 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd allow primary affinity = true 2021-04-03T03:16:59.963 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd allow pg remap = true 2021-04-03T03:16:59.963 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:16:59.963 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:16:59.963 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on no sortbitwise = false 2021-04-03T03:16:59.963 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:16:59.964 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on too few osds = false 2021-04-03T03:16:59.964 INFO:teuthology.orchestra.run.gibba015.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:16:59.964 INFO:teuthology.orchestra.run.gibba015.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:16:59.964 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.964 INFO:teuthology.orchestra.run.gibba015.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:16:59.964 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.965 INFO:teuthology.orchestra.run.gibba015.stdout: osd default data pool replay window = 5 2021-04-03T03:16:59.965 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.965 INFO:teuthology.orchestra.run.gibba015.stdout: mon allow pool delete = true 2021-04-03T03:16:59.965 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.965 INFO:teuthology.orchestra.run.gibba015.stdout: mon cluster log file level = debug 2021-04-03T03:16:59.966 INFO:teuthology.orchestra.run.gibba015.stdout: debug asserts on shutdown = true 2021-04-03T03:16:59.966 INFO:teuthology.orchestra.run.gibba015.stdout: mon health detail to clog = false 2021-04-03T03:16:59.966 INFO:teuthology.orchestra.run.gibba015.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:16:59.966 INFO:teuthology.orchestra.run.gibba015.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:16:59.967 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.967 INFO:teuthology.orchestra.run.gibba015.stdout:[osd] 2021-04-03T03:16:59.967 INFO:teuthology.orchestra.run.gibba015.stdout: osd journal size = 100 2021-04-03T03:16:59.967 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.967 INFO:teuthology.orchestra.run.gibba015.stdout: osd scrub load threshold = 5.0 2021-04-03T03:16:59.968 INFO:teuthology.orchestra.run.gibba015.stdout: osd scrub max interval = 600 2021-04-03T03:16:59.968 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.968 INFO:teuthology.orchestra.run.gibba015.stdout: osd recover clone overlap = true 2021-04-03T03:16:59.968 INFO:teuthology.orchestra.run.gibba015.stdout: osd recovery max chunk = 1048576 2021-04-03T03:16:59.968 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.969 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug shutdown = true 2021-04-03T03:16:59.969 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug op order = true 2021-04-03T03:16:59.969 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug verify stray on activate = true 2021-04-03T03:16:59.969 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.969 INFO:teuthology.orchestra.run.gibba015.stdout: osd open classes on start = true 2021-04-03T03:16:59.970 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug pg log writeout = true 2021-04-03T03:16:59.970 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.970 INFO:teuthology.orchestra.run.gibba015.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:16:59.970 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.970 INFO:teuthology.orchestra.run.gibba015.stdout: osd map max advance = 10 2021-04-03T03:16:59.971 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.971 INFO:teuthology.orchestra.run.gibba015.stdout: journal zero on create = true 2021-04-03T03:16:59.971 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.971 INFO:teuthology.orchestra.run.gibba015.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:16:59.971 INFO:teuthology.orchestra.run.gibba015.stdout: filestore apply finisher threads = 3 2021-04-03T03:16:59.972 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.972 INFO:teuthology.orchestra.run.gibba015.stdout: bdev debug aio = true 2021-04-03T03:16:59.972 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug misdirected ops = true 2021-04-03T03:16:59.972 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore block size = 96636764160 2021-04-03T03:16:59.972 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore compression mode = aggressive 2021-04-03T03:16:59.973 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore fsck on mount = True 2021-04-03T03:16:59.973 INFO:teuthology.orchestra.run.gibba015.stdout: debug bluefs = 20 2021-04-03T03:16:59.973 INFO:teuthology.orchestra.run.gibba015.stdout: debug bluestore = 20 2021-04-03T03:16:59.973 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:16:59.973 INFO:teuthology.orchestra.run.gibba015.stdout: debug osd = 20 2021-04-03T03:16:59.974 INFO:teuthology.orchestra.run.gibba015.stdout: debug rocksdb = 10 2021-04-03T03:16:59.974 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:16:59.974 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd full ratio = 0.9 2021-04-03T03:16:59.974 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:16:59.974 INFO:teuthology.orchestra.run.gibba015.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:16:59.974 INFO:teuthology.orchestra.run.gibba015.stdout: osd objectstore = bluestore 2021-04-03T03:16:59.975 INFO:teuthology.orchestra.run.gibba015.stdout: osd op complaint time = 180 2021-04-03T03:16:59.975 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.975 INFO:teuthology.orchestra.run.gibba015.stdout:[mgr] 2021-04-03T03:16:59.975 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:16:59.976 INFO:teuthology.orchestra.run.gibba015.stdout: debug mgr = 20 2021-04-03T03:16:59.976 INFO:teuthology.orchestra.run.gibba015.stdout: debug mon = 20 2021-04-03T03:16:59.976 INFO:teuthology.orchestra.run.gibba015.stdout: debug auth = 20 2021-04-03T03:16:59.976 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:16:59.976 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:16:59.977 INFO:teuthology.orchestra.run.gibba015.stdout: mgr/telemetry/nag = false 2021-04-03T03:16:59.977 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.977 INFO:teuthology.orchestra.run.gibba015.stdout:[mon] 2021-04-03T03:16:59.977 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:16:59.977 INFO:teuthology.orchestra.run.gibba015.stdout: debug mon = 20 2021-04-03T03:16:59.977 INFO:teuthology.orchestra.run.gibba015.stdout: debug paxos = 20 2021-04-03T03:16:59.978 INFO:teuthology.orchestra.run.gibba015.stdout: debug auth = 20 2021-04-03T03:16:59.978 INFO:teuthology.orchestra.run.gibba015.stdout: mon data avail warn = 5 2021-04-03T03:16:59.978 INFO:teuthology.orchestra.run.gibba015.stdout: mon mgr mkfs grace = 240 2021-04-03T03:16:59.978 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:16:59.978 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd reporter subtree level = osd 2021-04-03T03:16:59.979 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd prime pg temp = true 2021-04-03T03:16:59.979 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:16:59.979 INFO:teuthology.orchestra.run.gibba015.stdout: mon op complaint time = 120 2021-04-03T03:16:59.979 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:16:59.980 INFO:teuthology.orchestra.run.gibba015.stdout:[client] 2021-04-03T03:16:59.980 INFO:teuthology.orchestra.run.gibba015.stdout: rgw cache enabled = true 2021-04-03T03:16:59.980 INFO:teuthology.orchestra.run.gibba015.stdout: rgw enable ops log = true 2021-04-03T03:16:59.980 INFO:teuthology.orchestra.run.gibba015.stdout: rgw enable usage log = true 2021-04-03T03:16:59.980 INFO:teuthology.orchestra.run.gibba015.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:16:59.980 INFO:teuthology.orchestra.run.gibba015.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:16:59.981 INFO:teuthology.orchestra.run.gibba015.stdout: client mount timeout = 600 2021-04-03T03:16:59.981 INFO:teuthology.orchestra.run.gibba015.stdout: debug client = 20 2021-04-03T03:16:59.981 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:16:59.981 INFO:teuthology.orchestra.run.gibba015.stdout: rados mon op timeout = 15m 2021-04-03T03:16:59.981 INFO:teuthology.orchestra.run.gibba015.stdout: rados osd op timeout = 15m 2021-04-03T03:16:59.982 INFO:teuthology.orchestra.run.gibba015.stdout: client debug getattr caps = true 2021-04-03T03:16:59.982 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.a] 2021-04-03T03:16:59.982 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.b] 2021-04-03T03:16:59.982 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.c] 2021-04-03T03:16:59.982 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.a] 2021-04-03T03:16:59.983 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.b] 2021-04-03T03:16:59.983 INFO:teuthology.orchestra.run.gibba015.stdout:[mds] 2021-04-03T03:16:59.983 INFO:teuthology.orchestra.run.gibba015.stdout: debug mds = 20 2021-04-03T03:16:59.983 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:16:59.983 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal frag = True 2021-04-03T03:16:59.984 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal fragment size max = 10000 2021-04-03T03:16:59.984 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal merge size = 5 2021-04-03T03:16:59.984 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal split bits = 3 2021-04-03T03:16:59.984 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal split size = 100 2021-04-03T03:16:59.985 INFO:teuthology.orchestra.run.gibba015.stdout: mds debug frag = True 2021-04-03T03:16:59.985 INFO:teuthology.orchestra.run.gibba015.stdout: mds debug scatterstat = True 2021-04-03T03:16:59.985 INFO:teuthology.orchestra.run.gibba015.stdout: mds op complaint time = 180 2021-04-03T03:16:59.985 INFO:teuthology.orchestra.run.gibba015.stdout: mds verify scatter = True 2021-04-03T03:16:59.985 INFO:teuthology.orchestra.run.gibba015.stdout: osd op complaint time = 180 2021-04-03T03:16:59.986 INFO:teuthology.orchestra.run.gibba015.stdout: rados mon op timeout = 15m 2021-04-03T03:16:59.986 INFO:teuthology.orchestra.run.gibba015.stdout: rados osd op timeout = 15m 2021-04-03T03:16:59.986 INFO:teuthology.orchestra.run.gibba011.stdout:[global] 2021-04-03T03:16:59.986 INFO:teuthology.orchestra.run.gibba011.stdout: chdir = "" 2021-04-03T03:16:59.986 INFO:teuthology.orchestra.run.gibba011.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:16:59.987 INFO:teuthology.orchestra.run.gibba011.stdout: auth supported = cephx 2021-04-03T03:16:59.987 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.987 INFO:teuthology.orchestra.run.gibba011.stdout: filestore xattr use omap = true 2021-04-03T03:16:59.987 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.988 INFO:teuthology.orchestra.run.gibba011.stdout: mon clock drift allowed = 1.000 2021-04-03T03:16:59.988 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.988 INFO:teuthology.orchestra.run.gibba011.stdout: osd crush chooseleaf type = 0 2021-04-03T03:16:59.988 INFO:teuthology.orchestra.run.gibba011.stdout: auth debug = true 2021-04-03T03:16:59.988 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.989 INFO:teuthology.orchestra.run.gibba011.stdout: ms die on old message = true 2021-04-03T03:16:59.989 INFO:teuthology.orchestra.run.gibba011.stdout: ms die on bug = true 2021-04-03T03:16:59.989 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.989 INFO:teuthology.orchestra.run.gibba011.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:16:59.989 INFO:teuthology.orchestra.run.gibba011.stdout: mon pg warn max object skew = 0 2021-04-03T03:16:59.989 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.990 INFO:teuthology.orchestra.run.gibba011.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:16:59.990 INFO:teuthology.orchestra.run.gibba011.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:16:59.990 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.990 INFO:teuthology.orchestra.run.gibba011.stdout: osd pool default size = 2 2021-04-03T03:16:59.990 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.991 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd allow primary affinity = true 2021-04-03T03:16:59.991 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd allow pg remap = true 2021-04-03T03:16:59.991 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:16:59.991 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:16:59.992 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on no sortbitwise = false 2021-04-03T03:16:59.992 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:16:59.992 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on too few osds = false 2021-04-03T03:16:59.992 INFO:teuthology.orchestra.run.gibba011.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:16:59.992 INFO:teuthology.orchestra.run.gibba011.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:16:59.993 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.993 INFO:teuthology.orchestra.run.gibba011.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:16:59.993 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.993 INFO:teuthology.orchestra.run.gibba011.stdout: osd default data pool replay window = 5 2021-04-03T03:16:59.993 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.994 INFO:teuthology.orchestra.run.gibba011.stdout: mon allow pool delete = true 2021-04-03T03:16:59.994 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.994 INFO:teuthology.orchestra.run.gibba011.stdout: mon cluster log file level = debug 2021-04-03T03:16:59.994 INFO:teuthology.orchestra.run.gibba011.stdout: debug asserts on shutdown = true 2021-04-03T03:16:59.995 INFO:teuthology.orchestra.run.gibba011.stdout: mon health detail to clog = false 2021-04-03T03:16:59.995 INFO:teuthology.orchestra.run.gibba011.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:16:59.995 INFO:teuthology.orchestra.run.gibba011.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:16:59.995 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.995 INFO:teuthology.orchestra.run.gibba011.stdout:[osd] 2021-04-03T03:16:59.996 INFO:teuthology.orchestra.run.gibba011.stdout: osd journal size = 100 2021-04-03T03:16:59.996 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.996 INFO:teuthology.orchestra.run.gibba011.stdout: osd scrub load threshold = 5.0 2021-04-03T03:16:59.996 INFO:teuthology.orchestra.run.gibba011.stdout: osd scrub max interval = 600 2021-04-03T03:16:59.996 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.997 INFO:teuthology.orchestra.run.gibba011.stdout: osd recover clone overlap = true 2021-04-03T03:16:59.997 INFO:teuthology.orchestra.run.gibba011.stdout: osd recovery max chunk = 1048576 2021-04-03T03:16:59.997 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.997 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug shutdown = true 2021-04-03T03:16:59.997 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug op order = true 2021-04-03T03:16:59.998 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug verify stray on activate = true 2021-04-03T03:16:59.998 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.998 INFO:teuthology.orchestra.run.gibba011.stdout: osd open classes on start = true 2021-04-03T03:16:59.998 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug pg log writeout = true 2021-04-03T03:16:59.999 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.999 INFO:teuthology.orchestra.run.gibba011.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:16:59.999 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:16:59.999 INFO:teuthology.orchestra.run.gibba011.stdout: osd map max advance = 10 2021-04-03T03:16:59.999 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:00.000 INFO:teuthology.orchestra.run.gibba011.stdout: journal zero on create = true 2021-04-03T03:17:00.000 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:00.000 INFO:teuthology.orchestra.run.gibba011.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:00.000 INFO:teuthology.orchestra.run.gibba011.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:00.000 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:00.001 INFO:teuthology.orchestra.run.gibba011.stdout: bdev debug aio = true 2021-04-03T03:17:00.001 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug misdirected ops = true 2021-04-03T03:17:00.001 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore block size = 96636764160 2021-04-03T03:17:00.001 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:00.001 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore fsck on mount = True 2021-04-03T03:17:00.002 INFO:teuthology.orchestra.run.gibba011.stdout: debug bluefs = 20 2021-04-03T03:17:00.002 INFO:teuthology.orchestra.run.gibba011.stdout: debug bluestore = 20 2021-04-03T03:17:00.002 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:00.002 INFO:teuthology.orchestra.run.gibba011.stdout: debug osd = 20 2021-04-03T03:17:00.002 INFO:teuthology.orchestra.run.gibba011.stdout: debug rocksdb = 10 2021-04-03T03:17:00.002 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:00.003 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:00.003 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:00.003 INFO:teuthology.orchestra.run.gibba011.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:00.003 INFO:teuthology.orchestra.run.gibba011.stdout: osd objectstore = bluestore 2021-04-03T03:17:00.004 INFO:teuthology.orchestra.run.gibba011.stdout: osd op complaint time = 180 2021-04-03T03:17:00.004 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:00.004 INFO:teuthology.orchestra.run.gibba011.stdout:[mgr] 2021-04-03T03:17:00.004 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:00.004 INFO:teuthology.orchestra.run.gibba011.stdout: debug mgr = 20 2021-04-03T03:17:00.005 INFO:teuthology.orchestra.run.gibba011.stdout: debug mon = 20 2021-04-03T03:17:00.005 INFO:teuthology.orchestra.run.gibba011.stdout: debug auth = 20 2021-04-03T03:17:00.005 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:00.005 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:00.005 INFO:teuthology.orchestra.run.gibba011.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:00.006 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:00.006 INFO:teuthology.orchestra.run.gibba011.stdout:[mon] 2021-04-03T03:17:00.006 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:00.006 INFO:teuthology.orchestra.run.gibba011.stdout: debug mon = 20 2021-04-03T03:17:00.006 INFO:teuthology.orchestra.run.gibba011.stdout: debug paxos = 20 2021-04-03T03:17:00.007 INFO:teuthology.orchestra.run.gibba011.stdout: debug auth = 20 2021-04-03T03:17:00.007 INFO:teuthology.orchestra.run.gibba011.stdout: mon data avail warn = 5 2021-04-03T03:17:00.007 INFO:teuthology.orchestra.run.gibba011.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:00.007 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:00.008 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:00.008 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd prime pg temp = true 2021-04-03T03:17:00.008 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:00.008 INFO:teuthology.orchestra.run.gibba011.stdout: mon op complaint time = 120 2021-04-03T03:17:00.008 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:00.009 INFO:teuthology.orchestra.run.gibba011.stdout:[client] 2021-04-03T03:17:00.009 INFO:teuthology.orchestra.run.gibba011.stdout: rgw cache enabled = true 2021-04-03T03:17:00.009 INFO:teuthology.orchestra.run.gibba011.stdout: rgw enable ops log = true 2021-04-03T03:17:00.009 INFO:teuthology.orchestra.run.gibba011.stdout: rgw enable usage log = true 2021-04-03T03:17:00.009 INFO:teuthology.orchestra.run.gibba011.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:00.010 INFO:teuthology.orchestra.run.gibba011.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:00.010 INFO:teuthology.orchestra.run.gibba011.stdout: client mount timeout = 600 2021-04-03T03:17:00.010 INFO:teuthology.orchestra.run.gibba011.stdout: debug client = 20 2021-04-03T03:17:00.010 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:00.010 INFO:teuthology.orchestra.run.gibba011.stdout: rados mon op timeout = 15m 2021-04-03T03:17:00.010 INFO:teuthology.orchestra.run.gibba011.stdout: rados osd op timeout = 15m 2021-04-03T03:17:00.011 INFO:teuthology.orchestra.run.gibba011.stdout: client debug getattr caps = true 2021-04-03T03:17:00.011 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.a] 2021-04-03T03:17:00.011 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.b] 2021-04-03T03:17:00.011 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.c] 2021-04-03T03:17:00.012 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.a] 2021-04-03T03:17:00.012 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.b] 2021-04-03T03:17:00.012 INFO:teuthology.orchestra.run.gibba011.stdout:[mds] 2021-04-03T03:17:00.012 INFO:teuthology.orchestra.run.gibba011.stdout: debug mds = 20 2021-04-03T03:17:00.012 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:00.012 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal frag = True 2021-04-03T03:17:00.013 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:00.013 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal merge size = 5 2021-04-03T03:17:00.013 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal split bits = 3 2021-04-03T03:17:00.013 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal split size = 100 2021-04-03T03:17:00.013 INFO:teuthology.orchestra.run.gibba011.stdout: mds debug frag = True 2021-04-03T03:17:00.014 INFO:teuthology.orchestra.run.gibba011.stdout: mds debug scatterstat = True 2021-04-03T03:17:00.014 INFO:teuthology.orchestra.run.gibba011.stdout: mds op complaint time = 180 2021-04-03T03:17:00.014 INFO:teuthology.orchestra.run.gibba011.stdout: mds verify scatter = True 2021-04-03T03:17:00.014 INFO:teuthology.orchestra.run.gibba011.stdout: osd op complaint time = 180 2021-04-03T03:17:00.015 INFO:teuthology.orchestra.run.gibba011.stdout: rados mon op timeout = 15m 2021-04-03T03:17:00.015 INFO:teuthology.orchestra.run.gibba011.stdout: rados osd op timeout = 15m 2021-04-03T03:17:00.018 INFO:teuthology.orchestra.run.gibba031.stdout:[global] 2021-04-03T03:17:00.018 INFO:teuthology.orchestra.run.gibba031.stdout: chdir = "" 2021-04-03T03:17:00.018 INFO:teuthology.orchestra.run.gibba031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:17:00.019 INFO:teuthology.orchestra.run.gibba031.stdout: auth supported = cephx 2021-04-03T03:17:00.019 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.019 INFO:teuthology.orchestra.run.gibba031.stdout: filestore xattr use omap = true 2021-04-03T03:17:00.019 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.019 INFO:teuthology.orchestra.run.gibba031.stdout: mon clock drift allowed = 1.000 2021-04-03T03:17:00.020 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.020 INFO:teuthology.orchestra.run.gibba031.stdout: osd crush chooseleaf type = 0 2021-04-03T03:17:00.020 INFO:teuthology.orchestra.run.gibba031.stdout: auth debug = true 2021-04-03T03:17:00.020 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.020 INFO:teuthology.orchestra.run.gibba031.stdout: ms die on old message = true 2021-04-03T03:17:00.021 INFO:teuthology.orchestra.run.gibba031.stdout: ms die on bug = true 2021-04-03T03:17:00.021 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.021 INFO:teuthology.orchestra.run.gibba031.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:17:00.021 INFO:teuthology.orchestra.run.gibba031.stdout: mon pg warn max object skew = 0 2021-04-03T03:17:00.021 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.022 INFO:teuthology.orchestra.run.gibba031.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:17:00.022 INFO:teuthology.orchestra.run.gibba031.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:17:00.022 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.022 INFO:teuthology.orchestra.run.gibba031.stdout: osd pool default size = 2 2021-04-03T03:17:00.022 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.023 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd allow primary affinity = true 2021-04-03T03:17:00.023 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd allow pg remap = true 2021-04-03T03:17:00.023 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:17:00.023 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:17:00.024 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on no sortbitwise = false 2021-04-03T03:17:00.024 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:17:00.024 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on too few osds = false 2021-04-03T03:17:00.024 INFO:teuthology.orchestra.run.gibba031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:17:00.024 INFO:teuthology.orchestra.run.gibba031.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:17:00.025 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.025 INFO:teuthology.orchestra.run.gibba031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:17:00.025 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.025 INFO:teuthology.orchestra.run.gibba031.stdout: osd default data pool replay window = 5 2021-04-03T03:17:00.025 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.026 INFO:teuthology.orchestra.run.gibba031.stdout: mon allow pool delete = true 2021-04-03T03:17:00.026 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.026 INFO:teuthology.orchestra.run.gibba031.stdout: mon cluster log file level = debug 2021-04-03T03:17:00.026 INFO:teuthology.orchestra.run.gibba031.stdout: debug asserts on shutdown = true 2021-04-03T03:17:00.026 INFO:teuthology.orchestra.run.gibba031.stdout: mon health detail to clog = false 2021-04-03T03:17:00.027 INFO:teuthology.orchestra.run.gibba031.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:17:00.027 INFO:teuthology.orchestra.run.gibba031.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:17:00.027 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.027 INFO:teuthology.orchestra.run.gibba031.stdout:[osd] 2021-04-03T03:17:00.027 INFO:teuthology.orchestra.run.gibba031.stdout: osd journal size = 100 2021-04-03T03:17:00.028 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.028 INFO:teuthology.orchestra.run.gibba031.stdout: osd scrub load threshold = 5.0 2021-04-03T03:17:00.028 INFO:teuthology.orchestra.run.gibba031.stdout: osd scrub max interval = 600 2021-04-03T03:17:00.028 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.028 INFO:teuthology.orchestra.run.gibba031.stdout: osd recover clone overlap = true 2021-04-03T03:17:00.028 INFO:teuthology.orchestra.run.gibba031.stdout: osd recovery max chunk = 1048576 2021-04-03T03:17:00.029 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.029 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug shutdown = true 2021-04-03T03:17:00.029 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug op order = true 2021-04-03T03:17:00.029 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug verify stray on activate = true 2021-04-03T03:17:00.029 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.030 INFO:teuthology.orchestra.run.gibba031.stdout: osd open classes on start = true 2021-04-03T03:17:00.030 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug pg log writeout = true 2021-04-03T03:17:00.030 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.030 INFO:teuthology.orchestra.run.gibba031.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:17:00.030 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.031 INFO:teuthology.orchestra.run.gibba031.stdout: osd map max advance = 10 2021-04-03T03:17:00.031 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.031 INFO:teuthology.orchestra.run.gibba031.stdout: journal zero on create = true 2021-04-03T03:17:00.031 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.031 INFO:teuthology.orchestra.run.gibba031.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:00.032 INFO:teuthology.orchestra.run.gibba031.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:00.032 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.032 INFO:teuthology.orchestra.run.gibba031.stdout: bdev debug aio = true 2021-04-03T03:17:00.032 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug misdirected ops = true 2021-04-03T03:17:00.032 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore block size = 96636764160 2021-04-03T03:17:00.032 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:00.033 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore fsck on mount = True 2021-04-03T03:17:00.033 INFO:teuthology.orchestra.run.gibba031.stdout: debug bluefs = 20 2021-04-03T03:17:00.033 INFO:teuthology.orchestra.run.gibba031.stdout: debug bluestore = 20 2021-04-03T03:17:00.033 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:00.034 INFO:teuthology.orchestra.run.gibba031.stdout: debug osd = 20 2021-04-03T03:17:00.034 INFO:teuthology.orchestra.run.gibba031.stdout: debug rocksdb = 10 2021-04-03T03:17:00.034 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:00.034 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:00.034 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:00.035 INFO:teuthology.orchestra.run.gibba031.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:00.035 INFO:teuthology.orchestra.run.gibba031.stdout: osd objectstore = bluestore 2021-04-03T03:17:00.035 INFO:teuthology.orchestra.run.gibba031.stdout: osd op complaint time = 180 2021-04-03T03:17:00.035 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.036 INFO:teuthology.orchestra.run.gibba031.stdout:[mgr] 2021-04-03T03:17:00.036 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:00.036 INFO:teuthology.orchestra.run.gibba031.stdout: debug mgr = 20 2021-04-03T03:17:00.036 INFO:teuthology.orchestra.run.gibba031.stdout: debug mon = 20 2021-04-03T03:17:00.036 INFO:teuthology.orchestra.run.gibba031.stdout: debug auth = 20 2021-04-03T03:17:00.037 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:00.037 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:00.037 INFO:teuthology.orchestra.run.gibba031.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:00.037 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.037 INFO:teuthology.orchestra.run.gibba031.stdout:[mon] 2021-04-03T03:17:00.038 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:00.038 INFO:teuthology.orchestra.run.gibba031.stdout: debug mon = 20 2021-04-03T03:17:00.038 INFO:teuthology.orchestra.run.gibba031.stdout: debug paxos = 20 2021-04-03T03:17:00.038 INFO:teuthology.orchestra.run.gibba031.stdout: debug auth = 20 2021-04-03T03:17:00.038 INFO:teuthology.orchestra.run.gibba031.stdout: mon data avail warn = 5 2021-04-03T03:17:00.039 INFO:teuthology.orchestra.run.gibba031.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:00.039 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:00.039 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:00.039 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd prime pg temp = true 2021-04-03T03:17:00.039 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:00.039 INFO:teuthology.orchestra.run.gibba031.stdout: mon op complaint time = 120 2021-04-03T03:17:00.040 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:00.040 INFO:teuthology.orchestra.run.gibba031.stdout:[client] 2021-04-03T03:17:00.040 INFO:teuthology.orchestra.run.gibba031.stdout: rgw cache enabled = true 2021-04-03T03:17:00.040 INFO:teuthology.orchestra.run.gibba031.stdout: rgw enable ops log = true 2021-04-03T03:17:00.040 INFO:teuthology.orchestra.run.gibba031.stdout: rgw enable usage log = true 2021-04-03T03:17:00.041 INFO:teuthology.orchestra.run.gibba031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:00.041 INFO:teuthology.orchestra.run.gibba031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:00.041 INFO:teuthology.orchestra.run.gibba031.stdout: client mount timeout = 600 2021-04-03T03:17:00.041 INFO:teuthology.orchestra.run.gibba031.stdout: debug client = 20 2021-04-03T03:17:00.041 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:00.042 INFO:teuthology.orchestra.run.gibba031.stdout: rados mon op timeout = 15m 2021-04-03T03:17:00.042 INFO:teuthology.orchestra.run.gibba031.stdout: rados osd op timeout = 15m 2021-04-03T03:17:00.042 INFO:teuthology.orchestra.run.gibba031.stdout: client debug getattr caps = true 2021-04-03T03:17:00.042 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.a] 2021-04-03T03:17:00.042 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.b] 2021-04-03T03:17:00.043 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.c] 2021-04-03T03:17:00.043 INFO:teuthology.orchestra.run.gibba031.stdout:[mds.a] 2021-04-03T03:17:00.043 INFO:teuthology.orchestra.run.gibba031.stdout:[mds.b] 2021-04-03T03:17:00.043 INFO:teuthology.orchestra.run.gibba031.stdout:[mds] 2021-04-03T03:17:00.043 INFO:teuthology.orchestra.run.gibba031.stdout: debug mds = 20 2021-04-03T03:17:00.044 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:00.044 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal frag = True 2021-04-03T03:17:00.044 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:00.044 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal merge size = 5 2021-04-03T03:17:00.044 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal split bits = 3 2021-04-03T03:17:00.045 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal split size = 100 2021-04-03T03:17:00.045 INFO:teuthology.orchestra.run.gibba031.stdout: mds debug frag = True 2021-04-03T03:17:00.045 INFO:teuthology.orchestra.run.gibba031.stdout: mds debug scatterstat = True 2021-04-03T03:17:00.045 INFO:teuthology.orchestra.run.gibba031.stdout: mds op complaint time = 180 2021-04-03T03:17:00.045 INFO:teuthology.orchestra.run.gibba031.stdout: mds verify scatter = True 2021-04-03T03:17:00.046 INFO:teuthology.orchestra.run.gibba031.stdout: osd op complaint time = 180 2021-04-03T03:17:00.046 INFO:teuthology.orchestra.run.gibba031.stdout: rados mon op timeout = 15m 2021-04-03T03:17:00.046 INFO:teuthology.orchestra.run.gibba031.stdout: rados osd op timeout = 15m 2021-04-03T03:17:00.046 INFO:teuthology.orchestra.run.gibba028.stdout:[global] 2021-04-03T03:17:00.046 INFO:teuthology.orchestra.run.gibba028.stdout: chdir = "" 2021-04-03T03:17:00.046 INFO:teuthology.orchestra.run.gibba028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:17:00.047 INFO:teuthology.orchestra.run.gibba028.stdout: auth supported = cephx 2021-04-03T03:17:00.047 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.047 INFO:teuthology.orchestra.run.gibba028.stdout: filestore xattr use omap = true 2021-04-03T03:17:00.047 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.048 INFO:teuthology.orchestra.run.gibba028.stdout: mon clock drift allowed = 1.000 2021-04-03T03:17:00.048 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.048 INFO:teuthology.orchestra.run.gibba028.stdout: osd crush chooseleaf type = 0 2021-04-03T03:17:00.048 INFO:teuthology.orchestra.run.gibba028.stdout: auth debug = true 2021-04-03T03:17:00.048 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.049 INFO:teuthology.orchestra.run.gibba028.stdout: ms die on old message = true 2021-04-03T03:17:00.049 INFO:teuthology.orchestra.run.gibba028.stdout: ms die on bug = true 2021-04-03T03:17:00.049 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.049 INFO:teuthology.orchestra.run.gibba028.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:17:00.049 INFO:teuthology.orchestra.run.gibba028.stdout: mon pg warn max object skew = 0 2021-04-03T03:17:00.049 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.050 INFO:teuthology.orchestra.run.gibba028.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:17:00.050 INFO:teuthology.orchestra.run.gibba028.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:17:00.050 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.050 INFO:teuthology.orchestra.run.gibba028.stdout: osd pool default size = 2 2021-04-03T03:17:00.051 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.051 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd allow primary affinity = true 2021-04-03T03:17:00.051 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd allow pg remap = true 2021-04-03T03:17:00.051 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:17:00.051 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:17:00.052 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on no sortbitwise = false 2021-04-03T03:17:00.052 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:17:00.052 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on too few osds = false 2021-04-03T03:17:00.052 INFO:teuthology.orchestra.run.gibba028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:17:00.052 INFO:teuthology.orchestra.run.gibba028.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:17:00.053 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.053 INFO:teuthology.orchestra.run.gibba028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:17:00.053 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.053 INFO:teuthology.orchestra.run.gibba028.stdout: osd default data pool replay window = 5 2021-04-03T03:17:00.053 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.054 INFO:teuthology.orchestra.run.gibba028.stdout: mon allow pool delete = true 2021-04-03T03:17:00.054 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.054 INFO:teuthology.orchestra.run.gibba028.stdout: mon cluster log file level = debug 2021-04-03T03:17:00.054 INFO:teuthology.orchestra.run.gibba028.stdout: debug asserts on shutdown = true 2021-04-03T03:17:00.054 INFO:teuthology.orchestra.run.gibba028.stdout: mon health detail to clog = false 2021-04-03T03:17:00.055 INFO:teuthology.orchestra.run.gibba028.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:17:00.055 INFO:teuthology.orchestra.run.gibba028.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:17:00.055 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.055 INFO:teuthology.orchestra.run.gibba028.stdout:[osd] 2021-04-03T03:17:00.055 INFO:teuthology.orchestra.run.gibba028.stdout: osd journal size = 100 2021-04-03T03:17:00.056 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.056 INFO:teuthology.orchestra.run.gibba028.stdout: osd scrub load threshold = 5.0 2021-04-03T03:17:00.056 INFO:teuthology.orchestra.run.gibba028.stdout: osd scrub max interval = 600 2021-04-03T03:17:00.056 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.056 INFO:teuthology.orchestra.run.gibba028.stdout: osd recover clone overlap = true 2021-04-03T03:17:00.057 INFO:teuthology.orchestra.run.gibba028.stdout: osd recovery max chunk = 1048576 2021-04-03T03:17:00.057 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.057 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug shutdown = true 2021-04-03T03:17:00.057 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug op order = true 2021-04-03T03:17:00.057 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug verify stray on activate = true 2021-04-03T03:17:00.057 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.058 INFO:teuthology.orchestra.run.gibba028.stdout: osd open classes on start = true 2021-04-03T03:17:00.058 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug pg log writeout = true 2021-04-03T03:17:00.058 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.058 INFO:teuthology.orchestra.run.gibba028.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:17:00.058 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.059 INFO:teuthology.orchestra.run.gibba028.stdout: osd map max advance = 10 2021-04-03T03:17:00.059 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.059 INFO:teuthology.orchestra.run.gibba028.stdout: journal zero on create = true 2021-04-03T03:17:00.059 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.059 INFO:teuthology.orchestra.run.gibba028.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:00.060 INFO:teuthology.orchestra.run.gibba028.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:00.060 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.060 INFO:teuthology.orchestra.run.gibba028.stdout: bdev debug aio = true 2021-04-03T03:17:00.060 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug misdirected ops = true 2021-04-03T03:17:00.061 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore block size = 96636764160 2021-04-03T03:17:00.061 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:00.061 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore fsck on mount = True 2021-04-03T03:17:00.061 INFO:teuthology.orchestra.run.gibba028.stdout: debug bluefs = 20 2021-04-03T03:17:00.061 INFO:teuthology.orchestra.run.gibba028.stdout: debug bluestore = 20 2021-04-03T03:17:00.062 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:00.062 INFO:teuthology.orchestra.run.gibba028.stdout: debug osd = 20 2021-04-03T03:17:00.062 INFO:teuthology.orchestra.run.gibba028.stdout: debug rocksdb = 10 2021-04-03T03:17:00.062 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:00.062 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:00.063 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:00.063 INFO:teuthology.orchestra.run.gibba028.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:00.063 INFO:teuthology.orchestra.run.gibba028.stdout: osd objectstore = bluestore 2021-04-03T03:17:00.063 INFO:teuthology.orchestra.run.gibba028.stdout: osd op complaint time = 180 2021-04-03T03:17:00.063 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.064 INFO:teuthology.orchestra.run.gibba028.stdout:[mgr] 2021-04-03T03:17:00.064 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:00.064 INFO:teuthology.orchestra.run.gibba028.stdout: debug mgr = 20 2021-04-03T03:17:00.064 INFO:teuthology.orchestra.run.gibba028.stdout: debug mon = 20 2021-04-03T03:17:00.064 INFO:teuthology.orchestra.run.gibba028.stdout: debug auth = 20 2021-04-03T03:17:00.065 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:00.065 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:00.065 INFO:teuthology.orchestra.run.gibba028.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:00.065 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.065 INFO:teuthology.orchestra.run.gibba028.stdout:[mon] 2021-04-03T03:17:00.066 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:00.066 INFO:teuthology.orchestra.run.gibba028.stdout: debug mon = 20 2021-04-03T03:17:00.066 INFO:teuthology.orchestra.run.gibba028.stdout: debug paxos = 20 2021-04-03T03:17:00.066 INFO:teuthology.orchestra.run.gibba028.stdout: debug auth = 20 2021-04-03T03:17:00.066 INFO:teuthology.orchestra.run.gibba028.stdout: mon data avail warn = 5 2021-04-03T03:17:00.066 INFO:teuthology.orchestra.run.gibba028.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:00.067 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:00.067 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:00.067 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd prime pg temp = true 2021-04-03T03:17:00.067 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:00.067 INFO:teuthology.orchestra.run.gibba028.stdout: mon op complaint time = 120 2021-04-03T03:17:00.068 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:00.068 INFO:teuthology.orchestra.run.gibba028.stdout:[client] 2021-04-03T03:17:00.068 INFO:teuthology.orchestra.run.gibba028.stdout: rgw cache enabled = true 2021-04-03T03:17:00.068 INFO:teuthology.orchestra.run.gibba028.stdout: rgw enable ops log = true 2021-04-03T03:17:00.068 INFO:teuthology.orchestra.run.gibba028.stdout: rgw enable usage log = true 2021-04-03T03:17:00.069 INFO:teuthology.orchestra.run.gibba028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:00.069 INFO:teuthology.orchestra.run.gibba028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:00.069 INFO:teuthology.orchestra.run.gibba028.stdout: client mount timeout = 600 2021-04-03T03:17:00.069 INFO:teuthology.orchestra.run.gibba028.stdout: debug client = 20 2021-04-03T03:17:00.069 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:00.070 INFO:teuthology.orchestra.run.gibba028.stdout: rados mon op timeout = 15m 2021-04-03T03:17:00.070 INFO:teuthology.orchestra.run.gibba028.stdout: rados osd op timeout = 15m 2021-04-03T03:17:00.070 INFO:teuthology.orchestra.run.gibba028.stdout: client debug getattr caps = true 2021-04-03T03:17:00.070 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.a] 2021-04-03T03:17:00.070 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.b] 2021-04-03T03:17:00.071 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.c] 2021-04-03T03:17:00.071 INFO:teuthology.orchestra.run.gibba028.stdout:[mds.a] 2021-04-03T03:17:00.071 INFO:teuthology.orchestra.run.gibba028.stdout:[mds.b] 2021-04-03T03:17:00.071 INFO:teuthology.orchestra.run.gibba028.stdout:[mds] 2021-04-03T03:17:00.071 INFO:teuthology.orchestra.run.gibba028.stdout: debug mds = 20 2021-04-03T03:17:00.072 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:00.072 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal frag = True 2021-04-03T03:17:00.072 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:00.072 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal merge size = 5 2021-04-03T03:17:00.072 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal split bits = 3 2021-04-03T03:17:00.072 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal split size = 100 2021-04-03T03:17:00.073 INFO:teuthology.orchestra.run.gibba028.stdout: mds debug frag = True 2021-04-03T03:17:00.073 INFO:teuthology.orchestra.run.gibba028.stdout: mds debug scatterstat = True 2021-04-03T03:17:00.073 INFO:teuthology.orchestra.run.gibba028.stdout: mds op complaint time = 180 2021-04-03T03:17:00.073 INFO:teuthology.orchestra.run.gibba028.stdout: mds verify scatter = True 2021-04-03T03:17:00.073 INFO:teuthology.orchestra.run.gibba028.stdout: osd op complaint time = 180 2021-04-03T03:17:00.074 INFO:teuthology.orchestra.run.gibba028.stdout: rados mon op timeout = 15m 2021-04-03T03:17:00.074 INFO:teuthology.orchestra.run.gibba028.stdout: rados osd op timeout = 15m 2021-04-03T03:17:00.074 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:00.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 Processor -- start 2021-04-03T03:17:00.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- start start 2021-04-03T03:17:00.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f811405cab0 0x7f81140b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f81140b4750 0x7f81140b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81140b5590 con 0x7f81140b2f60 2021-04-03T03:17:00.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81141433d0 con 0x7f811405cab0 2021-04-03T03:17:00.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811405ce90 con 0x7f81140b4750 2021-04-03T03:17:00.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f811405cab0 0x7f81140b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f811405cab0 0x7f81140b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57756/0 (socket says 172.21.2.111:57756) 2021-04-03T03:17:00.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 -- 172.21.2.111:0/3357160056 learned_addr learned my addr 172.21.2.111:0/3357160056 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 -- 172.21.2.111:0/3357160056 >> v2:172.21.2.115:3300/0 conn(0x7f81140b4750 msgr2=0x7f81140b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:00.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 --2- 172.21.2.111:0/3357160056 >> v2:172.21.2.115:3300/0 conn(0x7f81140b4750 0x7f81140b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 -- 172.21.2.111:0/3357160056 >> v1:172.21.2.115:6789/0 conn(0x7f81140b2f60 legacy=0x7f81140b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:00.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 -- 172.21.2.111:0/3357160056 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81140b3630 con 0x7f811405cab0 2021-04-03T03:17:00.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8112ffd700 1 --2- 172.21.2.111:0/3357160056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f811405cab0 0x7f81140b2580 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f81140b2420 tx=0x7f8104006e90).ready entity=mon.0 client_cookie=2b1e8f699a392086 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8111ffb700 1 -- 172.21.2.111:0/3357160056 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8104009930 con 0x7f811405cab0 2021-04-03T03:17:00.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8111ffb700 1 -- 172.21.2.111:0/3357160056 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8104009a90 con 0x7f811405cab0 2021-04-03T03:17:00.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8111ffb700 1 -- 172.21.2.111:0/3357160056 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8104009e30 con 0x7f811405cab0 2021-04-03T03:17:00.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- 172.21.2.111:0/3357160056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f811405cab0 msgr2=0x7f81140b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- 172.21.2.111:0/3357160056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f811405cab0 0x7f81140b2580 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f81140b2420 tx=0x7f8104006e90).stop 2021-04-03T03:17:00.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- 172.21.2.111:0/3357160056 shutdown_connections 2021-04-03T03:17:00.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- 172.21.2.111:0/3357160056 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f811405cab0 0x7f81140b2580 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- 172.21.2.111:0/3357160056 >> v2:172.21.2.115:3300/0 conn(0x7f81140b4750 0x7f81140b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- 172.21.2.111:0/3357160056 >> 172.21.2.111:0/3357160056 conn(0x7f81140633b0 msgr2=0x7f811405c080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- 172.21.2.111:0/3357160056 shutdown_connections 2021-04-03T03:17:00.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- 172.21.2.111:0/3357160056 wait complete. 2021-04-03T03:17:00.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 Processor -- start 2021-04-03T03:17:00.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- start start 2021-04-03T03:17:00.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f811405cab0 0x7f81140b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81140b2f60 0x7f81141471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81140b4750 0x7f81140ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8114146dc0 con 0x7f81140b2f60 2021-04-03T03:17:00.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141457d0 con 0x7f811405cab0 2021-04-03T03:17:00.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f8113fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8114145a70 con 0x7f81140b4750 2021-04-03T03:17:00.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81140b2f60 0x7f81141471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81140b2f60 0x7f81141471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57762/0 (socket says 172.21.2.111:57762) 2021-04-03T03:17:00.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 -- 172.21.2.111:0/3092582532 learned_addr learned my addr 172.21.2.111:0/3092582532 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 -- 172.21.2.111:0/3092582532 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81140b4750 msgr2=0x7f81140ae7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:00.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81140b4750 0x7f81140ae7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 -- 172.21.2.111:0/3092582532 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f811405cab0 msgr2=0x7f81140b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f811405cab0 0x7f81140b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.133+0000 7f81127fc700 1 -- 172.21.2.111:0/3092582532 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81141476e0 con 0x7f81140b2f60 2021-04-03T03:17:00.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f81127fc700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81140b2f60 0x7f81141471f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f80fc002f30 tx=0x7f80fc006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f810bfff700 1 -- 172.21.2.111:0/3092582532 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80fc00d7c0 con 0x7f81140b2f60 2021-04-03T03:17:00.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f810bfff700 1 -- 172.21.2.111:0/3092582532 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc00d920 con 0x7f81140b2f60 2021-04-03T03:17:00.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8114144900 con 0x7f81140b2f60 2021-04-03T03:17:00.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8114146600 con 0x7f81140b2f60 2021-04-03T03:17:00.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f810bfff700 1 -- 172.21.2.111:0/3092582532 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80fc00dc80 con 0x7f81140b2f60 2021-04-03T03:17:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8114146f30 con 0x7f81140b2f60 2021-04-03T03:17:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f810bfff700 1 -- 172.21.2.111:0/3092582532 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f80fc0103c0 con 0x7f81140b2f60 2021-04-03T03:17:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f810bfff700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f80f80570a0 0x7f80f8059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f810bfff700 1 -- 172.21.2.111:0/3092582532 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f80fc06fc30 con 0x7f81140b2f60 2021-04-03T03:17:00.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f8112ffd700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f80f80570a0 0x7f80f8059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.137+0000 7f8112ffd700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f80f80570a0 0x7f80f8059560 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f81140b2420 tx=0x7f8104015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.141+0000 7f810bfff700 1 -- 172.21.2.111:0/3092582532 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8114146f30 con 0x7f81140b2f60 2021-04-03T03:17:00.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8114146f00 con 0x7f81140b2f60 2021-04-03T03:17:00.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f810bfff700 1 -- 172.21.2.111:0/3092582532 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f80fc044eb0 con 0x7f81140b2f60 2021-04-03T03:17:00.238 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:00.238 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":132,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:16:54.343386+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:16:51.364989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-04-03T03:16:52.312016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:00.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f80f80570a0 msgr2=0x7f80f8059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f80f80570a0 0x7f80f8059560 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f81140b2420 tx=0x7f8104015000).stop 2021-04-03T03:17:00.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81140b2f60 msgr2=0x7f81141471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81140b2f60 0x7f81141471f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f80fc002f30 tx=0x7f80fc006e30).stop 2021-04-03T03:17:00.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 shutdown_connections 2021-04-03T03:17:00.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f81140b4750 0x7f81140ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f80f80570a0 0x7f80f8059560 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f811405cab0 0x7f81140b9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 --2- 172.21.2.111:0/3092582532 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f81140b2f60 0x7f81141471f0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 >> 172.21.2.111:0/3092582532 conn(0x7f81140633b0 msgr2=0x7f81140b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 shutdown_connections 2021-04-03T03:17:00.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.237+0000 7f8113fff700 1 -- 172.21.2.111:0/3092582532 wait complete. 2021-04-03T03:17:00.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:16:51.364989+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:16:52.312016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:17:00.247 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:17:00.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 Processor -- start 2021-04-03T03:17:00.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- start start 2021-04-03T03:17:00.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 0x7fa5d0146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa5d0060290 0x7fa5d01474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d00ad210 con 0x7fa5d00b6a90 2021-04-03T03:17:00.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d0148c40 con 0x7fa5d0060290 2021-04-03T03:17:00.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00ad440 con 0x7fa5d00ad810 2021-04-03T03:17:00.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa5d0060290 0x7fa5d01474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa5d0060290 0x7fa5d01474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57770/0 (socket says 172.21.2.111:57770) 2021-04-03T03:17:00.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cf7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa5d00b6a90 0x7fa5d014b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58560/0 (socket says 172.21.2.111:58560) 2021-04-03T03:17:00.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 -- 172.21.2.111:0/778856795 learned_addr learned my addr 172.21.2.111:0/778856795 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5ceffd700 1 --2- 172.21.2.111:0/778856795 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 0x7fa5d0146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 -- 172.21.2.111:0/778856795 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 msgr2=0x7fa5d0146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:00.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 --2- 172.21.2.111:0/778856795 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 0x7fa5d0146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 -- 172.21.2.111:0/778856795 >> v1:172.21.2.111:6789/0 conn(0x7fa5d00b6a90 legacy=0x7fa5d014b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 -- 172.21.2.111:0/778856795 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d014b0d0 con 0x7fa5d0060290 2021-04-03T03:17:00.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5ce7fc700 1 -- 172.21.2.111:0/778856795 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131189281 0 0) 0x7fa5d00ad210 con 0x7fa5d00b6a90 2021-04-03T03:17:00.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 --2- 172.21.2.111:0/778856795 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d0060290 0x7fa5d01474c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fa5c8003300 tx=0x7fa5c8009270).ready entity=mon.0 client_cookie=e3db8f8c37efde54 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5ce7fc700 1 -- 172.21.2.111:0/778856795 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c800bc00 con 0x7fa5d0060290 2021-04-03T03:17:00.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5ce7fc700 1 -- 172.21.2.111:0/778856795 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c800bd60 con 0x7fa5d0060290 2021-04-03T03:17:00.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5ce7fc700 1 -- 172.21.2.111:0/778856795 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c8011400 con 0x7fa5d0060290 2021-04-03T03:17:00.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/778856795 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d0060290 msgr2=0x7fa5d01474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/778856795 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d0060290 0x7fa5d01474c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fa5c8003300 tx=0x7fa5c8009270).stop 2021-04-03T03:17:00.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/778856795 shutdown_connections 2021-04-03T03:17:00.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/778856795 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d0060290 0x7fa5d01474c0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/778856795 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 0x7fa5d0146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/778856795 >> 172.21.2.111:0/778856795 conn(0x7fa5d0060c60 msgr2=0x7fa5d00616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/778856795 shutdown_connections 2021-04-03T03:17:00.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/778856795 wait complete. 2021-04-03T03:17:00.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 Processor -- start 2021-04-03T03:17:00.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- start start 2021-04-03T03:17:00.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d0060290 0x7fa5d014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 0x7fa5d0156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d00b6a90 0x7fa5d0156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d009bff0 con 0x7fa5d00b6a90 2021-04-03T03:17:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00ae830 con 0x7fa5d0060290 2021-04-03T03:17:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5d4e3a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00aead0 con 0x7fa5d00ad810 2021-04-03T03:17:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d00b6a90 0x7fa5d0156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d00b6a90 0x7fa5d0156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57774/0 (socket says 172.21.2.111:57774) 2021-04-03T03:17:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.305+0000 7fa5cffff700 1 -- 172.21.2.111:0/3616735794 learned_addr learned my addr 172.21.2.111:0/3616735794 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cffff700 1 -- 172.21.2.111:0/3616735794 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 msgr2=0x7fa5d0156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cffff700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 0x7fa5d0156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cffff700 1 -- 172.21.2.111:0/3616735794 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d0060290 msgr2=0x7fa5d014d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cffff700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d0060290 0x7fa5d014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cffff700 1 -- 172.21.2.111:0/3616735794 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d009b4a0 con 0x7fa5d00b6a90 2021-04-03T03:17:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cffff700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d00b6a90 0x7fa5d0156c30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fa5c8003300 tx=0x7fa5c8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5ccff9700 1 -- 172.21.2.111:0/3616735794 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c8011400 con 0x7fa5d00b6a90 2021-04-03T03:17:00.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5ccff9700 1 -- 172.21.2.111:0/3616735794 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8006040 con 0x7fa5d00b6a90 2021-04-03T03:17:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5ccff9700 1 -- 172.21.2.111:0/3616735794 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c80063a0 con 0x7fa5d00b6a90 2021-04-03T03:17:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d0157120 con 0x7fa5d00b6a90 2021-04-03T03:17:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d009c630 con 0x7fa5d00b6a90 2021-04-03T03:17:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d009c130 con 0x7fa5d00b6a90 2021-04-03T03:17:00.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5ccff9700 1 -- 172.21.2.111:0/3616735794 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa5c8026020 con 0x7fa5d00b6a90 2021-04-03T03:17:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5ccff9700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b4056f20 0x7fa5b40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5ccff9700 1 -- 172.21.2.111:0/3616735794 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa5c8071230 con 0x7fa5d00b6a90 2021-04-03T03:17:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cf7fe700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b4056f20 0x7fa5b40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.309+0000 7fa5cf7fe700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b4056f20 0x7fa5b40593e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fa5b8000b80 tx=0x7fa5b8007800).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.313+0000 7fa5ccff9700 1 -- 172.21.2.111:0/3616735794 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa5c8046510 con 0x7fa5d00b6a90 2021-04-03T03:17:00.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.405+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa5d0057660 con 0x7fa5d00b6a90 2021-04-03T03:17:00.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.405+0000 7fa5ccff9700 1 -- 172.21.2.111:0/3616735794 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v132) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa5c80150b0 con 0x7fa5d00b6a90 2021-04-03T03:17:00.408 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:17:00.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.405+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b4056f20 msgr2=0x7fa5b40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.405+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b4056f20 0x7fa5b40593e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fa5b8000b80 tx=0x7fa5b8007800).stop 2021-04-03T03:17:00.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.405+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d00b6a90 msgr2=0x7fa5d0156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.405+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d00b6a90 0x7fa5d0156c30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fa5c8003300 tx=0x7fa5c8016000).stop 2021-04-03T03:17:00.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 shutdown_connections 2021-04-03T03:17:00.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d00b6a90 0x7fa5d0156c30 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b4056f20 0x7fa5b40593e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d0060290 0x7fa5d014d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 --2- 172.21.2.111:0/3616735794 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d00ad810 0x7fa5d0156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.411 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 >> 172.21.2.111:0/3616735794 conn(0x7fa5d0060c60 msgr2=0x7fa5d00aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 shutdown_connections 2021-04-03T03:17:00.412 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.409+0000 7fa5d4e3a700 1 -- 172.21.2.111:0/3616735794 wait complete. 2021-04-03T03:17:00.416 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:17:00.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 Processor -- start 2021-04-03T03:17:00.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- start start 2021-04-03T03:17:00.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f65d00dbba0 0x7f65d00dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f65d0041060 0x7f65d003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d0041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d003de60 con 0x7f65d0041060 2021-04-03T03:17:00.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00dc560 con 0x7f65d00dbba0 2021-04-03T03:17:00.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00dc800 con 0x7f65d003d560 2021-04-03T03:17:00.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f65d0041060 0x7f65d003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f65d0041060 0x7f65d003fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57782/0 (socket says 172.21.2.111:57782) 2021-04-03T03:17:00.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 -- 172.21.2.111:0/2281197920 learned_addr learned my addr 172.21.2.111:0/2281197920 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 -- 172.21.2.111:0/2281197920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 msgr2=0x7f65d0041650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:00.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 --2- 172.21.2.111:0/2281197920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d0041650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 -- 172.21.2.111:0/2281197920 >> v2:172.21.2.115:3300/0 conn(0x7f65d00dbba0 msgr2=0x7f65d00dbf80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:00.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 --2- 172.21.2.111:0/2281197920 >> v2:172.21.2.115:3300/0 conn(0x7f65d00dbba0 0x7f65d00dbf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 -- 172.21.2.111:0/2281197920 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d003da30 con 0x7f65d0041060 2021-04-03T03:17:00.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d5d9b700 1 --2- 172.21.2.111:0/2281197920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d0041060 0x7f65d003fa80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f65d806b5d0 tx=0x7f65d8068230).ready entity=mon.0 client_cookie=812377fbfaa8b134 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d559a700 1 -- 172.21.2.111:0/2281197920 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65d8088070 con 0x7f65d0041060 2021-04-03T03:17:00.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d559a700 1 -- 172.21.2.111:0/2281197920 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d805be40 con 0x7f65d0041060 2021-04-03T03:17:00.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d559a700 1 -- 172.21.2.111:0/2281197920 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65d805df60 con 0x7f65d0041060 2021-04-03T03:17:00.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- 172.21.2.111:0/2281197920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d0041060 msgr2=0x7f65d003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 --2- 172.21.2.111:0/2281197920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d0041060 0x7f65d003fa80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f65d806b5d0 tx=0x7f65d8068230).stop 2021-04-03T03:17:00.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- 172.21.2.111:0/2281197920 shutdown_connections 2021-04-03T03:17:00.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 --2- 172.21.2.111:0/2281197920 >> v2:172.21.2.115:3300/0 conn(0x7f65d00dbba0 0x7f65d00dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 --2- 172.21.2.111:0/2281197920 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d0041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 --2- 172.21.2.111:0/2281197920 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d0041060 0x7f65d003fa80 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- 172.21.2.111:0/2281197920 >> 172.21.2.111:0/2281197920 conn(0x7f65d00462b0 msgr2=0x7f65d00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- 172.21.2.111:0/2281197920 shutdown_connections 2021-04-03T03:17:00.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- 172.21.2.111:0/2281197920 wait complete. 2021-04-03T03:17:00.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 Processor -- start 2021-04-03T03:17:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.473+0000 7f65d759e700 1 -- start start 2021-04-03T03:17:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d00e7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f65d0041060 0x7f65d009ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d00dbba0 0x7f65d00457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d003c1d0 con 0x7f65d00dbba0 2021-04-03T03:17:00.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00a0410 con 0x7f65d0041060 2021-04-03T03:17:00.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00a0b70 con 0x7f65d003d560 2021-04-03T03:17:00.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d00dbba0 0x7f65d00457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d00dbba0 0x7f65d00457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57790/0 (socket says 172.21.2.111:57790) 2021-04-03T03:17:00.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 -- 172.21.2.111:0/140496253 learned_addr learned my addr 172.21.2.111:0/140496253 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d659c700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d00e7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 -- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 msgr2=0x7f65d00e7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d00e7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 -- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f65d0041060 msgr2=0x7f65d009ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f65d0041060 0x7f65d009ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 -- 172.21.2.111:0/140496253 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00a0630 con 0x7f65d00dbba0 2021-04-03T03:17:00.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d659c700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d00e7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:00.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d6d9d700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d00dbba0 0x7f65d00457a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f65d00a0550 tx=0x7f65c8006d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65c77fe700 1 -- 172.21.2.111:0/140496253 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c800d4a0 con 0x7f65d00dbba0 2021-04-03T03:17:00.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65c77fe700 1 -- 172.21.2.111:0/140496253 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800d600 con 0x7f65d00dbba0 2021-04-03T03:17:00.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00a08d0 con 0x7f65d00dbba0 2021-04-03T03:17:00.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65c77fe700 1 -- 172.21.2.111:0/140496253 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c800d9a0 con 0x7f65d00dbba0 2021-04-03T03:17:00.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d0045ea0 con 0x7f65d00dbba0 2021-04-03T03:17:00.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d00ea810 con 0x7f65d00dbba0 2021-04-03T03:17:00.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65c77fe700 1 -- 172.21.2.111:0/140496253 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f65c800f030 con 0x7f65d00dbba0 2021-04-03T03:17:00.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65c77fe700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65b4056ef0 0x7f65b40593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65c77fe700 1 -- 172.21.2.111:0/140496253 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f65c806fef0 con 0x7f65d00dbba0 2021-04-03T03:17:00.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d659c700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65b4056ef0 0x7f65b40593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.477+0000 7f65d659c700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65b4056ef0 0x7f65b40593b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f65cc007000 tx=0x7f65cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.481+0000 7f65c77fe700 1 -- 172.21.2.111:0/140496253 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f65d00ea810 con 0x7f65d00dbba0 2021-04-03T03:17:00.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65d00ea810 con 0x7f65d00dbba0 2021-04-03T03:17:00.575 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65c77fe700 1 -- 172.21.2.111:0/140496253 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65d00ea810 con 0x7f65d00dbba0 2021-04-03T03:17:00.576 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:00.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65b4056ef0 msgr2=0x7f65b40593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65b4056ef0 0x7f65b40593b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f65cc007000 tx=0x7f65cc008040).stop 2021-04-03T03:17:00.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d00dbba0 msgr2=0x7f65d00457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d00dbba0 0x7f65d00457a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f65d00a0550 tx=0x7f65c8006d20).stop 2021-04-03T03:17:00.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 shutdown_connections 2021-04-03T03:17:00.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f65d00dbba0 0x7f65d00457a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f65d003d560 0x7f65d00e7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f65b4056ef0 0x7f65b40593b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 --2- 172.21.2.111:0/140496253 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f65d0041060 0x7f65d009ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 >> 172.21.2.111:0/140496253 conn(0x7f65d00462b0 msgr2=0x7f65d003f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 shutdown_connections 2021-04-03T03:17:00.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.573+0000 7f65d759e700 1 -- 172.21.2.111:0/140496253 wait complete. 2021-04-03T03:17:00.584 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:17:00.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3b59e700 1 Processor -- start 2021-04-03T03:17:00.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3b59e700 1 -- start start 2021-04-03T03:17:00.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 0x7f8b3c146c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8b3c0aeea0 0x7f8b3c147170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3b59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c0b1960 con 0x7f8b3c0b1030 2021-04-03T03:17:00.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c0aded0 con 0x7f8b3c0aeea0 2021-04-03T03:17:00.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0b6680 con 0x7f8b3c0b62a0 2021-04-03T03:17:00.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3ad9d700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f8b3c0b1030 0x7f8b3c14aeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:42894/0 (socket says 172.21.2.111:42894) 2021-04-03T03:17:00.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b39d9b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8b3c0aeea0 0x7f8b3c147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3a59c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 0x7f8b3c146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.645+0000 7f8b3ad9d700 1 -- 172.21.2.111:0/2260003078 learned_addr learned my addr 172.21.2.111:0/2260003078 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:00.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 -- 172.21.2.111:0/2260003078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 msgr2=0x7f8b3c146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:00.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 --2- 172.21.2.111:0/2260003078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 0x7f8b3c146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 -- 172.21.2.111:0/2260003078 >> v1:172.21.2.115:6789/0 conn(0x7f8b3c0b1030 legacy=0x7f8b3c14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 -- 172.21.2.111:0/2260003078 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0af370 con 0x7f8b3c0aeea0 2021-04-03T03:17:00.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 --2- 172.21.2.111:0/2260003078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 0x7f8b3c147170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f8b24002f30 tx=0x7f8b24006dd0).ready entity=mon.1 client_cookie=8a97b193afbaac3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3959a700 1 -- 172.21.2.111:0/2260003078 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b24009760 con 0x7f8b3c0aeea0 2021-04-03T03:17:00.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3959a700 1 -- 172.21.2.111:0/2260003078 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240098c0 con 0x7f8b3c0aeea0 2021-04-03T03:17:00.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3959a700 1 -- 172.21.2.111:0/2260003078 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b24009c20 con 0x7f8b3c0aeea0 2021-04-03T03:17:00.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/2260003078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 msgr2=0x7f8b3c147170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/2260003078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 0x7f8b3c147170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f8b24002f30 tx=0x7f8b24006dd0).stop 2021-04-03T03:17:00.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/2260003078 shutdown_connections 2021-04-03T03:17:00.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/2260003078 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 0x7f8b3c146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/2260003078 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 0x7f8b3c147170 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/2260003078 >> 172.21.2.111:0/2260003078 conn(0x7f8b3c05c080 msgr2=0x7f8b3c0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/2260003078 shutdown_connections 2021-04-03T03:17:00.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/2260003078 wait complete. 2021-04-03T03:17:00.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 Processor -- start 2021-04-03T03:17:00.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- start start 2021-04-03T03:17:00.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 0x7f8b3c062d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8b3c0b1030 0x7f8b3c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 0x7f8b3c065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c064090 con 0x7f8b3c0b1030 2021-04-03T03:17:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c064530 con 0x7f8b3c0aeea0 2021-04-03T03:17:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c064290 con 0x7f8b3c0b62a0 2021-04-03T03:17:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8b3c0b1030 0x7f8b3c1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8b3c0b1030 0x7f8b3c1569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57804/0 (socket says 172.21.2.111:57804) 2021-04-03T03:17:00.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 -- 172.21.2.111:0/3325635234 learned_addr learned my addr 172.21.2.111:0/3325635234 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 -- 172.21.2.111:0/3325635234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 msgr2=0x7f8b3c065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 0x7f8b3c065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 -- 172.21.2.111:0/3325635234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 msgr2=0x7f8b3c062d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 0x7f8b3c062d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 -- 172.21.2.111:0/3325635234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c065a40 con 0x7f8b3c0b1030 2021-04-03T03:17:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b39d9b700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8b3c0b1030 0x7f8b3c1569c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f8b3c0646c0 tx=0x7f8b24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b337fe700 1 -- 172.21.2.111:0/3325635234 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b24011e20 con 0x7f8b3c0b1030 2021-04-03T03:17:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b337fe700 1 -- 172.21.2.111:0/3325635234 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2401c5c0 con 0x7f8b3c0b1030 2021-04-03T03:17:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c063270 con 0x7f8b3c0b1030 2021-04-03T03:17:00.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c066e30 con 0x7f8b3c0b1030 2021-04-03T03:17:00.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b337fe700 1 -- 172.21.2.111:0/3325635234 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b2401c920 con 0x7f8b3c0b1030 2021-04-03T03:17:00.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.649+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c065bd0 con 0x7f8b3c0b1030 2021-04-03T03:17:00.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.653+0000 7f8b337fe700 1 -- 172.21.2.111:0/3325635234 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8b2400f070 con 0x7f8b3c0b1030 2021-04-03T03:17:00.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.653+0000 7f8b337fe700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8b20057000 0x7f8b200594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.653+0000 7f8b337fe700 1 -- 172.21.2.111:0/3325635234 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8b24071f80 con 0x7f8b3c0b1030 2021-04-03T03:17:00.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.653+0000 7f8b3a59c700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8b20057000 0x7f8b200594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.653+0000 7f8b3a59c700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8b20057000 0x7f8b200594c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f8b2c00c000 tx=0x7f8b2c006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.653+0000 7f8b337fe700 1 -- 172.21.2.111:0/3325635234 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8b24047370 con 0x7f8b3c0b1030 2021-04-03T03:17:00.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.745+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b3c057660 con 0x7f8b3c0b1030 2021-04-03T03:17:00.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.745+0000 7f8b337fe700 1 -- 172.21.2.111:0/3325635234 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8b2404a990 con 0x7f8b3c0b1030 2021-04-03T03:17:00.749 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:00.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8b20057000 msgr2=0x7f8b200594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8b20057000 0x7f8b200594c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f8b2c00c000 tx=0x7f8b2c006dd0).stop 2021-04-03T03:17:00.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8b3c0b1030 msgr2=0x7f8b3c1569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.750 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8b3c0b1030 0x7f8b3c1569c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f8b3c0646c0 tx=0x7f8b24015000).stop 2021-04-03T03:17:00.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 shutdown_connections 2021-04-03T03:17:00.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8b3c0b62a0 0x7f8b3c065ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.751 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8b20057000 0x7f8b200594c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8b3c0aeea0 0x7f8b3c062d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 --2- 172.21.2.111:0/3325635234 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8b3c0b1030 0x7f8b3c1569c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 >> 172.21.2.111:0/3325635234 conn(0x7f8b3c05c080 msgr2=0x7f8b3c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 shutdown_connections 2021-04-03T03:17:00.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.749+0000 7f8b3b59e700 1 -- 172.21.2.111:0/3325635234 wait complete. 2021-04-03T03:17:00.756 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:17:00.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6beea6700 1 Processor -- start 2021-04-03T03:17:00.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6beea6700 1 -- start start 2021-04-03T03:17:00.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6beea6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b8063480 0x7ff6b81435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6beea6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff6b8143aa0 0x7ff6b805c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6beea6700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b8142ee0 con 0x7ff6b80603f0 2021-04-03T03:17:00.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6beea6700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b80b31a0 con 0x7ff6b8143aa0 2021-04-03T03:17:00.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6beea6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b80607d0 con 0x7ff6b8063480 2021-04-03T03:17:00.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6be6a5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff6b8143aa0 0x7ff6b805c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bdea4700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7ff6b80603f0 0x7ff6b80b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:42906/0 (socket says 172.21.2.111:42906) 2021-04-03T03:17:00.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bd6a3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b8063480 0x7ff6b81435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bdea4700 1 -- 172.21.2.111:0/4092448792 learned_addr learned my addr 172.21.2.111:0/4092448792 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:00.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bd6a3700 1 -- 172.21.2.111:0/4092448792 >> v2:172.21.2.115:3300/0 conn(0x7ff6b8143aa0 msgr2=0x7ff6b805c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:00.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bd6a3700 1 --2- 172.21.2.111:0/4092448792 >> v2:172.21.2.115:3300/0 conn(0x7ff6b8143aa0 0x7ff6b805c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bd6a3700 1 -- 172.21.2.111:0/4092448792 >> v1:172.21.2.115:6789/0 conn(0x7ff6b80603f0 legacy=0x7ff6b80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bd6a3700 1 -- 172.21.2.111:0/4092448792 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b80b1b10 con 0x7ff6b8063480 2021-04-03T03:17:00.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.813+0000 7ff6bd6a3700 1 --2- 172.21.2.111:0/4092448792 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b8063480 0x7ff6b81435b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7ff6b4002f30 tx=0x7ff6b4006dd0).ready entity=mon.2 client_cookie=c6e63bf24f3d8676 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bcea2700 1 -- 172.21.2.111:0/4092448792 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b4009760 con 0x7ff6b8063480 2021-04-03T03:17:00.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bcea2700 1 -- 172.21.2.111:0/4092448792 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b40098c0 con 0x7ff6b8063480 2021-04-03T03:17:00.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- 172.21.2.111:0/4092448792 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b8063480 msgr2=0x7ff6b81435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4092448792 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b8063480 0x7ff6b81435b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7ff6b4002f30 tx=0x7ff6b4006dd0).stop 2021-04-03T03:17:00.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- 172.21.2.111:0/4092448792 shutdown_connections 2021-04-03T03:17:00.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4092448792 >> v2:172.21.2.115:3300/0 conn(0x7ff6b8143aa0 0x7ff6b805c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4092448792 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b8063480 0x7ff6b81435b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- 172.21.2.111:0/4092448792 >> 172.21.2.111:0/4092448792 conn(0x7ff6b805c910 msgr2=0x7ff6b805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- 172.21.2.111:0/4092448792 shutdown_connections 2021-04-03T03:17:00.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- 172.21.2.111:0/4092448792 wait complete. 2021-04-03T03:17:00.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 Processor -- start 2021-04-03T03:17:00.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- start start 2021-04-03T03:17:00.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b80603f0 0x7ff6b80643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff6b8063480 0x7ff6b8065340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff6b8143aa0 0x7ff6b8065830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8131260 con 0x7ff6b8063480 2021-04-03T03:17:00.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8067070 con 0x7ff6b8143aa0 2021-04-03T03:17:00.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b812df30 con 0x7ff6b80603f0 2021-04-03T03:17:00.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff6b8063480 0x7ff6b8065340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff6b8063480 0x7ff6b8065340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57816/0 (socket says 172.21.2.111:57816) 2021-04-03T03:17:00.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 -- 172.21.2.111:0/4248948125 learned_addr learned my addr 172.21.2.111:0/4248948125 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6be6a5700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff6b8143aa0 0x7ff6b8065830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bdea4700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b80603f0 0x7ff6b80643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 -- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b80603f0 msgr2=0x7ff6b80643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b80603f0 0x7ff6b80643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 -- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff6b8143aa0 msgr2=0x7ff6b8065830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff6b8143aa0 0x7ff6b8065830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 -- 172.21.2.111:0/4248948125 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b812e0f0 con 0x7ff6b8063480 2021-04-03T03:17:00.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6bd6a3700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff6b8063480 0x7ff6b8065340 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7ff6b4002f30 tx=0x7ff6b4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6aaffd700 1 -- 172.21.2.111:0/4248948125 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b4010740 con 0x7ff6b8063480 2021-04-03T03:17:00.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6aaffd700 1 -- 172.21.2.111:0/4248948125 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b40108a0 con 0x7ff6b8063480 2021-04-03T03:17:00.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6aaffd700 1 -- 172.21.2.111:0/4248948125 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b4010c00 con 0x7ff6b8063480 2021-04-03T03:17:00.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b8067580 con 0x7ff6b8063480 2021-04-03T03:17:00.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.817+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b8065eb0 con 0x7ff6b8063480 2021-04-03T03:17:00.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.821+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b81313d0 con 0x7ff6b8063480 2021-04-03T03:17:00.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.821+0000 7ff6aaffd700 1 -- 172.21.2.111:0/4248948125 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff6b401a040 con 0x7ff6b8063480 2021-04-03T03:17:00.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.821+0000 7ff6aaffd700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff698056ea0 0x7ff698059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.821+0000 7ff6aaffd700 1 -- 172.21.2.111:0/4248948125 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff6b40717e0 con 0x7ff6b8063480 2021-04-03T03:17:00.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.821+0000 7ff6bdea4700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff698056ea0 0x7ff698059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.821+0000 7ff6bdea4700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff698056ea0 0x7ff698059360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7ff6ac00b000 tx=0x7ff6ac009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.821+0000 7ff6aaffd700 1 -- 172.21.2.111:0/4248948125 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff6b81313d0 con 0x7ff6b8063480 2021-04-03T03:17:00.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6b81313a0 con 0x7ff6b8063480 2021-04-03T03:17:00.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6aaffd700 1 -- 172.21.2.111:0/4248948125 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff6b40230a0 con 0x7ff6b8063480 2021-04-03T03:17:00.916 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:00.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff698056ea0 msgr2=0x7ff698059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff698056ea0 0x7ff698059360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7ff6ac00b000 tx=0x7ff6ac009040).stop 2021-04-03T03:17:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff6b8063480 msgr2=0x7ff6b8065340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff6b8063480 0x7ff6b8065340 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7ff6b4002f30 tx=0x7ff6b4015000).stop 2021-04-03T03:17:00.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 shutdown_connections 2021-04-03T03:17:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff698056ea0 0x7ff698059360 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff6b8143aa0 0x7ff6b8065830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff6b80603f0 0x7ff6b80643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 --2- 172.21.2.111:0/4248948125 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff6b8063480 0x7ff6b8065340 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 >> 172.21.2.111:0/4248948125 conn(0x7ff6b805c910 msgr2=0x7ff6b80b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 shutdown_connections 2021-04-03T03:17:00.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.913+0000 7ff6beea6700 1 -- 172.21.2.111:0/4248948125 wait complete. 2021-04-03T03:17:00.924 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:17:00.924 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:00.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97effff700 1 Processor -- start 2021-04-03T03:17:00.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97effff700 1 -- start start 2021-04-03T03:17:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97effff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97f00abf00 0x7f97f00b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97effff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f97f0143940 0x7f97f0143df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97effff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00b01d0 con 0x7f97f00b6260 2021-04-03T03:17:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97effff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f00b6640 con 0x7f97f00abf00 2021-04-03T03:17:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97effff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f00b0400 con 0x7f97f0143940 2021-04-03T03:17:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97ee7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97f00abf00 0x7f97f00b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97ee7fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97f00abf00 0x7f97f00b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57824/0 (socket says 172.21.2.111:57824) 2021-04-03T03:17:00.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.977+0000 7f97ee7fc700 1 -- 172.21.2.111:0/460206849 learned_addr learned my addr 172.21.2.111:0/460206849 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ee7fc700 1 -- 172.21.2.111:0/460206849 >> v2:172.21.2.115:3300/0 conn(0x7f97f0143940 msgr2=0x7f97f0143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ee7fc700 1 --2- 172.21.2.111:0/460206849 >> v2:172.21.2.115:3300/0 conn(0x7f97f0143940 0x7f97f0143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ee7fc700 1 -- 172.21.2.111:0/460206849 >> v1:172.21.2.115:6789/0 conn(0x7f97f00b6260 legacy=0x7f97f00b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:00.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ee7fc700 1 -- 172.21.2.111:0/460206849 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f01474f0 con 0x7f97f00abf00 2021-04-03T03:17:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ee7fc700 1 --2- 172.21.2.111:0/460206849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f00abf00 0x7f97f00b10b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f97f00acc80 tx=0x7f97e4006e90).ready entity=mon.0 client_cookie=e87df10b26c40ecb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97edffb700 1 -- 172.21.2.111:0/460206849 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e40098f0 con 0x7f97f00abf00 2021-04-03T03:17:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97edffb700 1 -- 172.21.2.111:0/460206849 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4009a50 con 0x7f97f00abf00 2021-04-03T03:17:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97edffb700 1 -- 172.21.2.111:0/460206849 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e4009df0 con 0x7f97f00abf00 2021-04-03T03:17:00.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/460206849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f00abf00 msgr2=0x7f97f00b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 --2- 172.21.2.111:0/460206849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f00abf00 0x7f97f00b10b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7f97f00acc80 tx=0x7f97e4006e90).stop 2021-04-03T03:17:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/460206849 shutdown_connections 2021-04-03T03:17:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 --2- 172.21.2.111:0/460206849 >> v2:172.21.2.115:3300/0 conn(0x7f97f0143940 0x7f97f0143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 --2- 172.21.2.111:0/460206849 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f00abf00 0x7f97f00b10b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/460206849 >> 172.21.2.111:0/460206849 conn(0x7f97f00637e0 msgr2=0x7f97f00b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/460206849 shutdown_connections 2021-04-03T03:17:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/460206849 wait complete. 2021-04-03T03:17:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 Processor -- start 2021-04-03T03:17:00.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- start start 2021-04-03T03:17:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97f00abf00 0x7f97f0144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97f00b6260 0x7f97f0152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f0143940 0x7f97f0155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00647f0 con 0x7f97f0143940 2021-04-03T03:17:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0064550 con 0x7f97f00b6260 2021-04-03T03:17:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0155880 con 0x7f97f00abf00 2021-04-03T03:17:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f0143940 0x7f97f0155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f0143940 0x7f97f0155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57830/0 (socket says 172.21.2.111:57830) 2021-04-03T03:17:00.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 -- 172.21.2.111:0/2001150551 learned_addr learned my addr 172.21.2.111:0/2001150551 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:00.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97eeffd700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97f00abf00 0x7f97f0144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 -- 172.21.2.111:0/2001150551 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97f00abf00 msgr2=0x7f97f0144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97f00abf00 0x7f97f0144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 -- 172.21.2.111:0/2001150551 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97f00b6260 msgr2=0x7f97f0152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97f00b6260 0x7f97f0152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:00.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 -- 172.21.2.111:0/2001150551 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0152650 con 0x7f97f0143940 2021-04-03T03:17:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97ef7fe700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f0143940 0x7f97f0155d20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f97e80032b0 tx=0x7f97e80091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97dffff700 1 -- 172.21.2.111:0/2001150551 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e800c9e0 con 0x7f97f0143940 2021-04-03T03:17:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97dffff700 1 -- 172.21.2.111:0/2001150551 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e800cb40 con 0x7f97f0143940 2021-04-03T03:17:00.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f00641d0 con 0x7f97f0143940 2021-04-03T03:17:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f0067860 con 0x7f97f0143940 2021-04-03T03:17:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97dffff700 1 -- 172.21.2.111:0/2001150551 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e800cea0 con 0x7f97f0143940 2021-04-03T03:17:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f0057660 con 0x7f97f0143940 2021-04-03T03:17:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.981+0000 7f97dffff700 1 -- 172.21.2.111:0/2001150551 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f97e8015470 con 0x7f97f0143940 2021-04-03T03:17:00.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.985+0000 7f97dffff700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f97cc057000 0x7f97cc0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.985+0000 7f97dffff700 1 -- 172.21.2.111:0/2001150551 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f97e8070270 con 0x7f97f0143940 2021-04-03T03:17:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.985+0000 7f97eeffd700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f97cc057000 0x7f97cc0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.985+0000 7f97eeffd700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f97cc057000 0x7f97cc0594c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f97e0007000 tx=0x7f97e0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:00.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:00.985+0000 7f97dffff700 1 -- 172.21.2.111:0/2001150551 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f97e80454f0 con 0x7f97f0143940 2021-04-03T03:17:01.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f01559f0 con 0x7f97f0143940 2021-04-03T03:17:01.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97dffff700 1 -- 172.21.2.111:0/2001150551 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f97e8048b10 con 0x7f97f0143940 2021-04-03T03:17:01.111 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:01.111 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":58,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3599494763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3599494763},{"type":"v1","addr":"172.21.2.115:6833","nonce":3599494763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:53.339332+0000","modified":"2021-04-03T03:16:54.343270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7890":{"gid":7890,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/233883066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":233883066},{"type":"v1","addr":"172.21.2.111:6835","nonce":233883066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-04-03T03:17:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f97cc057000 msgr2=0x7f97cc0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f97cc057000 0x7f97cc0594c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f97e0007000 tx=0x7f97e0008040).stop 2021-04-03T03:17:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f0143940 msgr2=0x7f97f0155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:01.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f0143940 0x7f97f0155d20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f97e80032b0 tx=0x7f97e80091a0).stop 2021-04-03T03:17:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 shutdown_connections 2021-04-03T03:17:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97f0143940 0x7f97f0155d20 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97f00abf00 0x7f97f0144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:01.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f97cc057000 0x7f97cc0594c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 --2- 172.21.2.111:0/2001150551 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97f00b6260 0x7f97f0152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 >> 172.21.2.111:0/2001150551 conn(0x7f97f00637e0 msgr2=0x7f97f0060360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 shutdown_connections 2021-04-03T03:17:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:01.109+0000 7f97effff700 1 -- 172.21.2.111:0/2001150551 wait complete. 2021-04-03T03:17:01.114 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 58 2021-04-03T03:17:01.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:16:53.339332+0000', 'modified': '2021-04-03T03:16:54.343270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7890}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7890': {'gid': 7890, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/233883066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 233883066}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 233883066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:17:01.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:17:01.321 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:17:01.410 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":7890,"want_state":"up:active","state":"up:active","rank_uptime":8.0629974519999994,"mdsmap_epoch":58,"osdmap_epoch":132,"osdmap_epoch_barrier":131,"uptime":12.25661468} 2021-04-03T03:17:01.410 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 7890, "mdsmap_epoch": 58, "osdmap_epoch": 132, "osdmap_epoch_barrier": 131, "rank_uptime": 8.062997452, "state": "up:active", "uptime": 12.25661468, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:17:01.411 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:17:01.411 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:17:01.411 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:17:01.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.412 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:01.422 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:17:01.475 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:01.475 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:01.523 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:01.524 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:01.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.525 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:17:01.577 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:17:01.577 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:01.623 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:17:01.630 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:01.631 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:01.663 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:17:01.656+0000 7fb443ff90c0 -1 init, newargv = 0x555f28239770 newargc=9 2021-04-03T03:17:01.663 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15561]: starting ceph client 2021-04-03T03:17:01.668 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:01.668 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:01.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.670 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:17:01.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15561]: starting fuse 2021-04-03T03:17:01.681 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:17:01.682 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:17:01.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.683 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:17:01.683 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:17:01.683 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:17:01.684 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:17:01.684 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:17:01.685 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:01.685 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:17:01.685 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:17:01.686 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:17:01.686 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:01.687 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:17:01.687 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:17:01.687 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:17:01.688 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:17:01.688 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:01.689 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:17:01.689 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:17:01.689 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:17:01.690 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:17:01.690 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:17:01.691 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:01.691 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:17:01.691 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:17:01.776 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15561.asok 2021-04-03T03:17:01.780 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15561.asok 2021-04-03T03:17:01.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.781 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15561.asok status 2021-04-03T03:17:01.846 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:17:01.847 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:17:01.847 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:17:01.847 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:17:01.848 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:17:01.848 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:17:01.848 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:17:01.848 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15561", 2021-04-03T03:17:01.848 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:17:01.849 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:01.849 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:17:01.849 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:17:01.849 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 5135, 2021-04-03T03:17:01.849 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:17:01.850 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:17:01.850 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:17:01.850 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 5135 2021-04-03T03:17:01.850 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:01.851 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:17:01.851 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:17:01.851 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:17:01.851 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 3971867857 2021-04-03T03:17:01.852 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:17:01.852 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:01.852 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:17:01.852 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:17:01.853 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:17:01.853 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 3971867857 2021-04-03T03:17:01.853 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:01.853 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.5135 172.21.2.128:0/3971867857", 2021-04-03T03:17:01.853 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/3971867857", 2021-04-03T03:17:01.854 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:17:01.854 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 58, 2021-04-03T03:17:01.854 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 132, 2021-04-03T03:17:01.854 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:17:01.855 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:17:01.855 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:17:01.855 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:17:01.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.857 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:01.857 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:01.862 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:17:01.863 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:01.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.864 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:01.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:01.914 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:01.915 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:01.926 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:17:01.927 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:17:01.928 DEBUG:teuthology.orchestra.run.gibba026:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-04-03T03:17:01.928 DEBUG:teuthology.orchestra.run.gibba026:> import time 2021-04-03T03:17:01.929 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:01.929 DEBUG:teuthology.orchestra.run.gibba026:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2021-04-03T03:17:01.929 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content'"'"') 2021-04-03T03:17:01.930 DEBUG:teuthology.orchestra.run.gibba026:> f.flush() 2021-04-03T03:17:01.930 DEBUG:teuthology.orchestra.run.gibba026:> f.write('"'"'content2'"'"') 2021-04-03T03:17:01.931 DEBUG:teuthology.orchestra.run.gibba026:> while True: 2021-04-03T03:17:01.931 DEBUG:teuthology.orchestra.run.gibba026:> time.sleep(1) 2021-04-03T03:17:01.931 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:17:01.970 DEBUG:teuthology.orchestra.run.gibba026:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2021-04-03T03:17:01.986 INFO:teuthology.orchestra.run.gibba026.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2021-04-03T03:17:01.987 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-04-03T03:17:02.988 DEBUG:teuthology.orchestra.run.gibba026:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2021-04-03T03:17:03.000 INFO:teuthology.orchestra.run.gibba026.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2021-04-03T03:17:03.001 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2021-04-03T03:17:03.001 DEBUG:teuthology.orchestra.run.gibba028:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2021-04-03T03:17:03.054 INFO:teuthology.orchestra.run.gibba028.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2021-04-03T03:17:03.055 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2021-04-03T03:17:03.055 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-04-03T03:17:03.056 DEBUG:teuthology.orchestra.run.gibba028:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2021-04-03T03:17:03.111 INFO:teuthology.orchestra.run.gibba028.stdout:content 2021-04-03T03:17:03.122 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:17:03.122 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:03.134 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:03.159 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:03.173 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:03.185 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:03.198 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:17:03.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:03.198 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:03.199 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:03.226 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:17:03.227 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:17:03.227 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:17:03.228 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:17:03.228 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:03.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[15983]: fuse finished with error 0 and tester_r 0 2021-04-03T03:17:03.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:03.314 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:03.314 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:03.319 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:17:03.319 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:17:03.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:17:03.430 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:03.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:03.476 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:03.477 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:03.487 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:17:03.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:03.489 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:17:03.490 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:17:03.490 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:03.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15561]: fuse finished with error 0 and tester_r 0 2021-04-03T03:17:03.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:03.583 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:03.583 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:03.587 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:17:03.587 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:17:03.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:17:03.663 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:03.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:03.668 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:03.668 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:17:03.677 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:17:03.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:03.679 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:17:03.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:17:03.680 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:17:03.728 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:17:03.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:03.731 DEBUG:teuthology.orchestra.run.gibba011:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:17:03.734 DEBUG:teuthology.orchestra.run.gibba015:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:17:03.737 DEBUG:teuthology.orchestra.run.gibba026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:17:03.740 DEBUG:teuthology.orchestra.run.gibba028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:17:03.743 DEBUG:teuthology.orchestra.run.gibba031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-04-03T03:17:03.777 INFO:teuthology.orchestra.run.gibba028.stdout:[global] 2021-04-03T03:17:03.778 INFO:teuthology.orchestra.run.gibba028.stdout: chdir = "" 2021-04-03T03:17:03.778 INFO:teuthology.orchestra.run.gibba028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:17:03.778 INFO:teuthology.orchestra.run.gibba028.stdout: auth supported = cephx 2021-04-03T03:17:03.778 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.778 INFO:teuthology.orchestra.run.gibba028.stdout: filestore xattr use omap = true 2021-04-03T03:17:03.779 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.779 INFO:teuthology.orchestra.run.gibba028.stdout: mon clock drift allowed = 1.000 2021-04-03T03:17:03.779 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.779 INFO:teuthology.orchestra.run.gibba028.stdout: osd crush chooseleaf type = 0 2021-04-03T03:17:03.780 INFO:teuthology.orchestra.run.gibba028.stdout: auth debug = true 2021-04-03T03:17:03.780 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.780 INFO:teuthology.orchestra.run.gibba028.stdout: ms die on old message = true 2021-04-03T03:17:03.780 INFO:teuthology.orchestra.run.gibba028.stdout: ms die on bug = true 2021-04-03T03:17:03.780 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.781 INFO:teuthology.orchestra.run.gibba028.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:17:03.781 INFO:teuthology.orchestra.run.gibba028.stdout: mon pg warn max object skew = 0 2021-04-03T03:17:03.781 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.781 INFO:teuthology.orchestra.run.gibba028.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:17:03.782 INFO:teuthology.orchestra.run.gibba028.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:17:03.782 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.782 INFO:teuthology.orchestra.run.gibba028.stdout: osd pool default size = 2 2021-04-03T03:17:03.782 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.782 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd allow primary affinity = true 2021-04-03T03:17:03.783 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd allow pg remap = true 2021-04-03T03:17:03.783 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:17:03.783 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:17:03.783 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on no sortbitwise = false 2021-04-03T03:17:03.783 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:17:03.784 INFO:teuthology.orchestra.run.gibba028.stdout: mon warn on too few osds = false 2021-04-03T03:17:03.784 INFO:teuthology.orchestra.run.gibba028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:17:03.784 INFO:teuthology.orchestra.run.gibba028.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:17:03.784 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.784 INFO:teuthology.orchestra.run.gibba028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:17:03.785 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.785 INFO:teuthology.orchestra.run.gibba028.stdout: osd default data pool replay window = 5 2021-04-03T03:17:03.785 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.785 INFO:teuthology.orchestra.run.gibba028.stdout: mon allow pool delete = true 2021-04-03T03:17:03.786 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.786 INFO:teuthology.orchestra.run.gibba028.stdout: mon cluster log file level = debug 2021-04-03T03:17:03.786 INFO:teuthology.orchestra.run.gibba028.stdout: debug asserts on shutdown = true 2021-04-03T03:17:03.786 INFO:teuthology.orchestra.run.gibba028.stdout: mon health detail to clog = false 2021-04-03T03:17:03.787 INFO:teuthology.orchestra.run.gibba028.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:17:03.787 INFO:teuthology.orchestra.run.gibba028.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:17:03.787 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.787 INFO:teuthology.orchestra.run.gibba028.stdout:[osd] 2021-04-03T03:17:03.787 INFO:teuthology.orchestra.run.gibba028.stdout: osd journal size = 100 2021-04-03T03:17:03.788 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.788 INFO:teuthology.orchestra.run.gibba028.stdout: osd scrub load threshold = 5.0 2021-04-03T03:17:03.788 INFO:teuthology.orchestra.run.gibba028.stdout: osd scrub max interval = 600 2021-04-03T03:17:03.788 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.789 INFO:teuthology.orchestra.run.gibba028.stdout: osd recover clone overlap = true 2021-04-03T03:17:03.789 INFO:teuthology.orchestra.run.gibba028.stdout: osd recovery max chunk = 1048576 2021-04-03T03:17:03.789 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.789 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug shutdown = true 2021-04-03T03:17:03.789 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug op order = true 2021-04-03T03:17:03.790 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug verify stray on activate = true 2021-04-03T03:17:03.790 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.790 INFO:teuthology.orchestra.run.gibba028.stdout: osd open classes on start = true 2021-04-03T03:17:03.790 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug pg log writeout = true 2021-04-03T03:17:03.790 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.791 INFO:teuthology.orchestra.run.gibba028.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:17:03.791 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.791 INFO:teuthology.orchestra.run.gibba028.stdout: osd map max advance = 10 2021-04-03T03:17:03.791 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.792 INFO:teuthology.orchestra.run.gibba028.stdout: journal zero on create = true 2021-04-03T03:17:03.792 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.792 INFO:teuthology.orchestra.run.gibba028.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:03.792 INFO:teuthology.orchestra.run.gibba028.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:03.792 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.793 INFO:teuthology.orchestra.run.gibba028.stdout: bdev debug aio = true 2021-04-03T03:17:03.793 INFO:teuthology.orchestra.run.gibba028.stdout: osd debug misdirected ops = true 2021-04-03T03:17:03.793 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore block size = 96636764160 2021-04-03T03:17:03.793 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:03.794 INFO:teuthology.orchestra.run.gibba028.stdout: bluestore fsck on mount = True 2021-04-03T03:17:03.794 INFO:teuthology.orchestra.run.gibba028.stdout: debug bluefs = 20 2021-04-03T03:17:03.794 INFO:teuthology.orchestra.run.gibba028.stdout: debug bluestore = 20 2021-04-03T03:17:03.794 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:03.794 INFO:teuthology.orchestra.run.gibba028.stdout: debug osd = 20 2021-04-03T03:17:03.795 INFO:teuthology.orchestra.run.gibba028.stdout: debug rocksdb = 10 2021-04-03T03:17:03.795 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:03.795 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:03.795 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:03.795 INFO:teuthology.orchestra.run.gibba028.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:03.796 INFO:teuthology.orchestra.run.gibba028.stdout: osd objectstore = bluestore 2021-04-03T03:17:03.796 INFO:teuthology.orchestra.run.gibba028.stdout: osd op complaint time = 180 2021-04-03T03:17:03.796 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.796 INFO:teuthology.orchestra.run.gibba028.stdout:[mgr] 2021-04-03T03:17:03.797 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:03.797 INFO:teuthology.orchestra.run.gibba028.stdout: debug mgr = 20 2021-04-03T03:17:03.797 INFO:teuthology.orchestra.run.gibba028.stdout: debug mon = 20 2021-04-03T03:17:03.797 INFO:teuthology.orchestra.run.gibba028.stdout: debug auth = 20 2021-04-03T03:17:03.797 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.798 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.798 INFO:teuthology.orchestra.run.gibba028.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:03.798 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.798 INFO:teuthology.orchestra.run.gibba028.stdout:[mon] 2021-04-03T03:17:03.798 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:03.799 INFO:teuthology.orchestra.run.gibba028.stdout: debug mon = 20 2021-04-03T03:17:03.799 INFO:teuthology.orchestra.run.gibba028.stdout: debug paxos = 20 2021-04-03T03:17:03.799 INFO:teuthology.orchestra.run.gibba028.stdout: debug auth = 20 2021-04-03T03:17:03.799 INFO:teuthology.orchestra.run.gibba028.stdout: mon data avail warn = 5 2021-04-03T03:17:03.800 INFO:teuthology.orchestra.run.gibba028.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:03.800 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.800 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:03.800 INFO:teuthology.orchestra.run.gibba028.stdout: mon osd prime pg temp = true 2021-04-03T03:17:03.800 INFO:teuthology.orchestra.run.gibba028.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.801 INFO:teuthology.orchestra.run.gibba028.stdout: mon op complaint time = 120 2021-04-03T03:17:03.801 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-04-03T03:17:03.801 INFO:teuthology.orchestra.run.gibba028.stdout:[client] 2021-04-03T03:17:03.801 INFO:teuthology.orchestra.run.gibba028.stdout: rgw cache enabled = true 2021-04-03T03:17:03.801 INFO:teuthology.orchestra.run.gibba028.stdout: rgw enable ops log = true 2021-04-03T03:17:03.802 INFO:teuthology.orchestra.run.gibba028.stdout: rgw enable usage log = true 2021-04-03T03:17:03.802 INFO:teuthology.orchestra.run.gibba028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:03.802 INFO:teuthology.orchestra.run.gibba028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:03.802 INFO:teuthology.orchestra.run.gibba028.stdout: client mount timeout = 600 2021-04-03T03:17:03.803 INFO:teuthology.orchestra.run.gibba028.stdout: debug client = 20 2021-04-03T03:17:03.803 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:03.803 INFO:teuthology.orchestra.run.gibba028.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.803 INFO:teuthology.orchestra.run.gibba028.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.803 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.a] 2021-04-03T03:17:03.804 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.b] 2021-04-03T03:17:03.804 INFO:teuthology.orchestra.run.gibba028.stdout:[mon.c] 2021-04-03T03:17:03.804 INFO:teuthology.orchestra.run.gibba028.stdout:[mds.a] 2021-04-03T03:17:03.804 INFO:teuthology.orchestra.run.gibba028.stdout:[mds.b] 2021-04-03T03:17:03.805 INFO:teuthology.orchestra.run.gibba028.stdout:[mds] 2021-04-03T03:17:03.805 INFO:teuthology.orchestra.run.gibba028.stdout: debug mds = 20 2021-04-03T03:17:03.805 INFO:teuthology.orchestra.run.gibba028.stdout: debug ms = 1 2021-04-03T03:17:03.805 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal frag = True 2021-04-03T03:17:03.805 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:03.806 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal merge size = 5 2021-04-03T03:17:03.806 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal split bits = 3 2021-04-03T03:17:03.806 INFO:teuthology.orchestra.run.gibba028.stdout: mds bal split size = 100 2021-04-03T03:17:03.806 INFO:teuthology.orchestra.run.gibba028.stdout: mds debug frag = True 2021-04-03T03:17:03.806 INFO:teuthology.orchestra.run.gibba028.stdout: mds debug scatterstat = True 2021-04-03T03:17:03.807 INFO:teuthology.orchestra.run.gibba028.stdout: mds op complaint time = 180 2021-04-03T03:17:03.807 INFO:teuthology.orchestra.run.gibba028.stdout: mds verify scatter = True 2021-04-03T03:17:03.807 INFO:teuthology.orchestra.run.gibba028.stdout: osd op complaint time = 180 2021-04-03T03:17:03.807 INFO:teuthology.orchestra.run.gibba028.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.808 INFO:teuthology.orchestra.run.gibba028.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.808 INFO:teuthology.orchestra.run.gibba015.stdout:[global] 2021-04-03T03:17:03.808 INFO:teuthology.orchestra.run.gibba015.stdout: chdir = "" 2021-04-03T03:17:03.808 INFO:teuthology.orchestra.run.gibba015.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:17:03.809 INFO:teuthology.orchestra.run.gibba015.stdout: auth supported = cephx 2021-04-03T03:17:03.809 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.809 INFO:teuthology.orchestra.run.gibba015.stdout: filestore xattr use omap = true 2021-04-03T03:17:03.809 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.809 INFO:teuthology.orchestra.run.gibba015.stdout: mon clock drift allowed = 1.000 2021-04-03T03:17:03.810 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.810 INFO:teuthology.orchestra.run.gibba015.stdout: osd crush chooseleaf type = 0 2021-04-03T03:17:03.810 INFO:teuthology.orchestra.run.gibba015.stdout: auth debug = true 2021-04-03T03:17:03.810 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.810 INFO:teuthology.orchestra.run.gibba015.stdout: ms die on old message = true 2021-04-03T03:17:03.811 INFO:teuthology.orchestra.run.gibba015.stdout: ms die on bug = true 2021-04-03T03:17:03.811 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.811 INFO:teuthology.orchestra.run.gibba015.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:17:03.811 INFO:teuthology.orchestra.run.gibba015.stdout: mon pg warn max object skew = 0 2021-04-03T03:17:03.811 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.812 INFO:teuthology.orchestra.run.gibba015.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:17:03.812 INFO:teuthology.orchestra.run.gibba015.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:17:03.812 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.812 INFO:teuthology.orchestra.run.gibba015.stdout: osd pool default size = 2 2021-04-03T03:17:03.812 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.813 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd allow primary affinity = true 2021-04-03T03:17:03.813 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd allow pg remap = true 2021-04-03T03:17:03.813 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:17:03.813 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:17:03.813 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on no sortbitwise = false 2021-04-03T03:17:03.814 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:17:03.814 INFO:teuthology.orchestra.run.gibba015.stdout: mon warn on too few osds = false 2021-04-03T03:17:03.814 INFO:teuthology.orchestra.run.gibba015.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:17:03.814 INFO:teuthology.orchestra.run.gibba015.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:17:03.814 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.815 INFO:teuthology.orchestra.run.gibba015.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:17:03.815 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.815 INFO:teuthology.orchestra.run.gibba015.stdout: osd default data pool replay window = 5 2021-04-03T03:17:03.815 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.815 INFO:teuthology.orchestra.run.gibba015.stdout: mon allow pool delete = true 2021-04-03T03:17:03.816 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.816 INFO:teuthology.orchestra.run.gibba015.stdout: mon cluster log file level = debug 2021-04-03T03:17:03.816 INFO:teuthology.orchestra.run.gibba015.stdout: debug asserts on shutdown = true 2021-04-03T03:17:03.816 INFO:teuthology.orchestra.run.gibba015.stdout: mon health detail to clog = false 2021-04-03T03:17:03.816 INFO:teuthology.orchestra.run.gibba015.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:17:03.817 INFO:teuthology.orchestra.run.gibba015.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:17:03.817 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.817 INFO:teuthology.orchestra.run.gibba015.stdout:[osd] 2021-04-03T03:17:03.817 INFO:teuthology.orchestra.run.gibba015.stdout: osd journal size = 100 2021-04-03T03:17:03.817 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.818 INFO:teuthology.orchestra.run.gibba015.stdout: osd scrub load threshold = 5.0 2021-04-03T03:17:03.818 INFO:teuthology.orchestra.run.gibba015.stdout: osd scrub max interval = 600 2021-04-03T03:17:03.818 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.818 INFO:teuthology.orchestra.run.gibba015.stdout: osd recover clone overlap = true 2021-04-03T03:17:03.818 INFO:teuthology.orchestra.run.gibba015.stdout: osd recovery max chunk = 1048576 2021-04-03T03:17:03.819 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.819 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug shutdown = true 2021-04-03T03:17:03.819 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug op order = true 2021-04-03T03:17:03.819 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug verify stray on activate = true 2021-04-03T03:17:03.819 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.820 INFO:teuthology.orchestra.run.gibba015.stdout: osd open classes on start = true 2021-04-03T03:17:03.820 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug pg log writeout = true 2021-04-03T03:17:03.820 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.820 INFO:teuthology.orchestra.run.gibba015.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:17:03.821 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.821 INFO:teuthology.orchestra.run.gibba015.stdout: osd map max advance = 10 2021-04-03T03:17:03.821 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.821 INFO:teuthology.orchestra.run.gibba015.stdout: journal zero on create = true 2021-04-03T03:17:03.821 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.822 INFO:teuthology.orchestra.run.gibba015.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:03.822 INFO:teuthology.orchestra.run.gibba015.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:03.822 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.822 INFO:teuthology.orchestra.run.gibba015.stdout: bdev debug aio = true 2021-04-03T03:17:03.823 INFO:teuthology.orchestra.run.gibba015.stdout: osd debug misdirected ops = true 2021-04-03T03:17:03.823 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore block size = 96636764160 2021-04-03T03:17:03.823 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:03.823 INFO:teuthology.orchestra.run.gibba015.stdout: bluestore fsck on mount = True 2021-04-03T03:17:03.823 INFO:teuthology.orchestra.run.gibba015.stdout: debug bluefs = 20 2021-04-03T03:17:03.824 INFO:teuthology.orchestra.run.gibba015.stdout: debug bluestore = 20 2021-04-03T03:17:03.824 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:17:03.824 INFO:teuthology.orchestra.run.gibba015.stdout: debug osd = 20 2021-04-03T03:17:03.824 INFO:teuthology.orchestra.run.gibba015.stdout: debug rocksdb = 10 2021-04-03T03:17:03.825 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:03.825 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:03.825 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:03.825 INFO:teuthology.orchestra.run.gibba015.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:03.825 INFO:teuthology.orchestra.run.gibba015.stdout: osd objectstore = bluestore 2021-04-03T03:17:03.826 INFO:teuthology.orchestra.run.gibba015.stdout: osd op complaint time = 180 2021-04-03T03:17:03.826 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.826 INFO:teuthology.orchestra.run.gibba015.stdout:[mgr] 2021-04-03T03:17:03.826 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:17:03.826 INFO:teuthology.orchestra.run.gibba015.stdout: debug mgr = 20 2021-04-03T03:17:03.827 INFO:teuthology.orchestra.run.gibba015.stdout: debug mon = 20 2021-04-03T03:17:03.827 INFO:teuthology.orchestra.run.gibba015.stdout: debug auth = 20 2021-04-03T03:17:03.827 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.827 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.828 INFO:teuthology.orchestra.run.gibba015.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:03.828 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.828 INFO:teuthology.orchestra.run.gibba015.stdout:[mon] 2021-04-03T03:17:03.828 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:17:03.828 INFO:teuthology.orchestra.run.gibba015.stdout: debug mon = 20 2021-04-03T03:17:03.829 INFO:teuthology.orchestra.run.gibba015.stdout: debug paxos = 20 2021-04-03T03:17:03.829 INFO:teuthology.orchestra.run.gibba015.stdout: debug auth = 20 2021-04-03T03:17:03.829 INFO:teuthology.orchestra.run.gibba015.stdout: mon data avail warn = 5 2021-04-03T03:17:03.829 INFO:teuthology.orchestra.run.gibba015.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:03.829 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.830 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:03.830 INFO:teuthology.orchestra.run.gibba015.stdout: mon osd prime pg temp = true 2021-04-03T03:17:03.830 INFO:teuthology.orchestra.run.gibba015.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.830 INFO:teuthology.orchestra.run.gibba015.stdout: mon op complaint time = 120 2021-04-03T03:17:03.830 INFO:teuthology.orchestra.run.gibba015.stdout: 2021-04-03T03:17:03.831 INFO:teuthology.orchestra.run.gibba015.stdout:[client] 2021-04-03T03:17:03.831 INFO:teuthology.orchestra.run.gibba015.stdout: rgw cache enabled = true 2021-04-03T03:17:03.831 INFO:teuthology.orchestra.run.gibba015.stdout: rgw enable ops log = true 2021-04-03T03:17:03.831 INFO:teuthology.orchestra.run.gibba015.stdout: rgw enable usage log = true 2021-04-03T03:17:03.832 INFO:teuthology.orchestra.run.gibba015.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:03.832 INFO:teuthology.orchestra.run.gibba015.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:03.832 INFO:teuthology.orchestra.run.gibba015.stdout: client mount timeout = 600 2021-04-03T03:17:03.832 INFO:teuthology.orchestra.run.gibba015.stdout: debug client = 20 2021-04-03T03:17:03.832 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:17:03.833 INFO:teuthology.orchestra.run.gibba015.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.833 INFO:teuthology.orchestra.run.gibba015.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.833 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.a] 2021-04-03T03:17:03.833 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.b] 2021-04-03T03:17:03.833 INFO:teuthology.orchestra.run.gibba015.stdout:[mon.c] 2021-04-03T03:17:03.834 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.a] 2021-04-03T03:17:03.834 INFO:teuthology.orchestra.run.gibba015.stdout:[mds.b] 2021-04-03T03:17:03.834 INFO:teuthology.orchestra.run.gibba015.stdout:[mds] 2021-04-03T03:17:03.834 INFO:teuthology.orchestra.run.gibba015.stdout: debug mds = 20 2021-04-03T03:17:03.834 INFO:teuthology.orchestra.run.gibba015.stdout: debug ms = 1 2021-04-03T03:17:03.835 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal frag = True 2021-04-03T03:17:03.835 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:03.835 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal merge size = 5 2021-04-03T03:17:03.835 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal split bits = 3 2021-04-03T03:17:03.836 INFO:teuthology.orchestra.run.gibba015.stdout: mds bal split size = 100 2021-04-03T03:17:03.836 INFO:teuthology.orchestra.run.gibba015.stdout: mds debug frag = True 2021-04-03T03:17:03.836 INFO:teuthology.orchestra.run.gibba015.stdout: mds debug scatterstat = True 2021-04-03T03:17:03.836 INFO:teuthology.orchestra.run.gibba015.stdout: mds op complaint time = 180 2021-04-03T03:17:03.836 INFO:teuthology.orchestra.run.gibba015.stdout: mds verify scatter = True 2021-04-03T03:17:03.837 INFO:teuthology.orchestra.run.gibba015.stdout: osd op complaint time = 180 2021-04-03T03:17:03.837 INFO:teuthology.orchestra.run.gibba015.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.837 INFO:teuthology.orchestra.run.gibba015.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.837 INFO:teuthology.orchestra.run.gibba026.stdout:[global] 2021-04-03T03:17:03.838 INFO:teuthology.orchestra.run.gibba026.stdout: chdir = "" 2021-04-03T03:17:03.838 INFO:teuthology.orchestra.run.gibba026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:17:03.838 INFO:teuthology.orchestra.run.gibba026.stdout: auth supported = cephx 2021-04-03T03:17:03.838 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.838 INFO:teuthology.orchestra.run.gibba026.stdout: filestore xattr use omap = true 2021-04-03T03:17:03.839 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.839 INFO:teuthology.orchestra.run.gibba026.stdout: mon clock drift allowed = 1.000 2021-04-03T03:17:03.839 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.839 INFO:teuthology.orchestra.run.gibba026.stdout: osd crush chooseleaf type = 0 2021-04-03T03:17:03.840 INFO:teuthology.orchestra.run.gibba026.stdout: auth debug = true 2021-04-03T03:17:03.840 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.840 INFO:teuthology.orchestra.run.gibba026.stdout: ms die on old message = true 2021-04-03T03:17:03.840 INFO:teuthology.orchestra.run.gibba026.stdout: ms die on bug = true 2021-04-03T03:17:03.841 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.841 INFO:teuthology.orchestra.run.gibba026.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:17:03.841 INFO:teuthology.orchestra.run.gibba026.stdout: mon pg warn max object skew = 0 2021-04-03T03:17:03.841 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.841 INFO:teuthology.orchestra.run.gibba026.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:17:03.842 INFO:teuthology.orchestra.run.gibba026.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:17:03.842 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.842 INFO:teuthology.orchestra.run.gibba026.stdout: osd pool default size = 2 2021-04-03T03:17:03.842 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.843 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd allow primary affinity = true 2021-04-03T03:17:03.843 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd allow pg remap = true 2021-04-03T03:17:03.843 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:17:03.843 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:17:03.843 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on no sortbitwise = false 2021-04-03T03:17:03.844 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:17:03.844 INFO:teuthology.orchestra.run.gibba026.stdout: mon warn on too few osds = false 2021-04-03T03:17:03.844 INFO:teuthology.orchestra.run.gibba026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:17:03.844 INFO:teuthology.orchestra.run.gibba026.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:17:03.845 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.845 INFO:teuthology.orchestra.run.gibba026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:17:03.845 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.845 INFO:teuthology.orchestra.run.gibba026.stdout: osd default data pool replay window = 5 2021-04-03T03:17:03.845 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.846 INFO:teuthology.orchestra.run.gibba026.stdout: mon allow pool delete = true 2021-04-03T03:17:03.846 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.846 INFO:teuthology.orchestra.run.gibba026.stdout: mon cluster log file level = debug 2021-04-03T03:17:03.846 INFO:teuthology.orchestra.run.gibba026.stdout: debug asserts on shutdown = true 2021-04-03T03:17:03.847 INFO:teuthology.orchestra.run.gibba026.stdout: mon health detail to clog = false 2021-04-03T03:17:03.847 INFO:teuthology.orchestra.run.gibba026.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:17:03.847 INFO:teuthology.orchestra.run.gibba026.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:17:03.847 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.847 INFO:teuthology.orchestra.run.gibba026.stdout:[osd] 2021-04-03T03:17:03.848 INFO:teuthology.orchestra.run.gibba026.stdout: osd journal size = 100 2021-04-03T03:17:03.848 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.848 INFO:teuthology.orchestra.run.gibba026.stdout: osd scrub load threshold = 5.0 2021-04-03T03:17:03.848 INFO:teuthology.orchestra.run.gibba026.stdout: osd scrub max interval = 600 2021-04-03T03:17:03.849 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.849 INFO:teuthology.orchestra.run.gibba026.stdout: osd recover clone overlap = true 2021-04-03T03:17:03.849 INFO:teuthology.orchestra.run.gibba026.stdout: osd recovery max chunk = 1048576 2021-04-03T03:17:03.849 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.849 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug shutdown = true 2021-04-03T03:17:03.850 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug op order = true 2021-04-03T03:17:03.850 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug verify stray on activate = true 2021-04-03T03:17:03.850 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.850 INFO:teuthology.orchestra.run.gibba026.stdout: osd open classes on start = true 2021-04-03T03:17:03.851 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug pg log writeout = true 2021-04-03T03:17:03.851 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.851 INFO:teuthology.orchestra.run.gibba026.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:17:03.851 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.851 INFO:teuthology.orchestra.run.gibba026.stdout: osd map max advance = 10 2021-04-03T03:17:03.852 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.852 INFO:teuthology.orchestra.run.gibba026.stdout: journal zero on create = true 2021-04-03T03:17:03.852 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.852 INFO:teuthology.orchestra.run.gibba026.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:03.853 INFO:teuthology.orchestra.run.gibba026.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:03.853 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.853 INFO:teuthology.orchestra.run.gibba026.stdout: bdev debug aio = true 2021-04-03T03:17:03.853 INFO:teuthology.orchestra.run.gibba026.stdout: osd debug misdirected ops = true 2021-04-03T03:17:03.853 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore block size = 96636764160 2021-04-03T03:17:03.854 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:03.854 INFO:teuthology.orchestra.run.gibba026.stdout: bluestore fsck on mount = True 2021-04-03T03:17:03.854 INFO:teuthology.orchestra.run.gibba026.stdout: debug bluefs = 20 2021-04-03T03:17:03.854 INFO:teuthology.orchestra.run.gibba026.stdout: debug bluestore = 20 2021-04-03T03:17:03.854 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:17:03.855 INFO:teuthology.orchestra.run.gibba026.stdout: debug osd = 20 2021-04-03T03:17:03.855 INFO:teuthology.orchestra.run.gibba026.stdout: debug rocksdb = 10 2021-04-03T03:17:03.855 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:03.855 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:03.856 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:03.856 INFO:teuthology.orchestra.run.gibba026.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:03.856 INFO:teuthology.orchestra.run.gibba026.stdout: osd objectstore = bluestore 2021-04-03T03:17:03.856 INFO:teuthology.orchestra.run.gibba026.stdout: osd op complaint time = 180 2021-04-03T03:17:03.856 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.857 INFO:teuthology.orchestra.run.gibba026.stdout:[mgr] 2021-04-03T03:17:03.857 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:17:03.857 INFO:teuthology.orchestra.run.gibba026.stdout: debug mgr = 20 2021-04-03T03:17:03.857 INFO:teuthology.orchestra.run.gibba026.stdout: debug mon = 20 2021-04-03T03:17:03.858 INFO:teuthology.orchestra.run.gibba026.stdout: debug auth = 20 2021-04-03T03:17:03.858 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.858 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.858 INFO:teuthology.orchestra.run.gibba026.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:03.858 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.859 INFO:teuthology.orchestra.run.gibba026.stdout:[mon] 2021-04-03T03:17:03.859 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:17:03.859 INFO:teuthology.orchestra.run.gibba026.stdout: debug mon = 20 2021-04-03T03:17:03.859 INFO:teuthology.orchestra.run.gibba026.stdout: debug paxos = 20 2021-04-03T03:17:03.859 INFO:teuthology.orchestra.run.gibba026.stdout: debug auth = 20 2021-04-03T03:17:03.860 INFO:teuthology.orchestra.run.gibba026.stdout: mon data avail warn = 5 2021-04-03T03:17:03.860 INFO:teuthology.orchestra.run.gibba026.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:03.860 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.860 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:03.861 INFO:teuthology.orchestra.run.gibba026.stdout: mon osd prime pg temp = true 2021-04-03T03:17:03.861 INFO:teuthology.orchestra.run.gibba026.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.861 INFO:teuthology.orchestra.run.gibba026.stdout: mon op complaint time = 120 2021-04-03T03:17:03.861 INFO:teuthology.orchestra.run.gibba026.stdout: 2021-04-03T03:17:03.861 INFO:teuthology.orchestra.run.gibba026.stdout:[client] 2021-04-03T03:17:03.862 INFO:teuthology.orchestra.run.gibba026.stdout: rgw cache enabled = true 2021-04-03T03:17:03.862 INFO:teuthology.orchestra.run.gibba026.stdout: rgw enable ops log = true 2021-04-03T03:17:03.862 INFO:teuthology.orchestra.run.gibba026.stdout: rgw enable usage log = true 2021-04-03T03:17:03.862 INFO:teuthology.orchestra.run.gibba026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:03.862 INFO:teuthology.orchestra.run.gibba026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:03.863 INFO:teuthology.orchestra.run.gibba026.stdout: client mount timeout = 600 2021-04-03T03:17:03.863 INFO:teuthology.orchestra.run.gibba026.stdout: debug client = 20 2021-04-03T03:17:03.863 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:17:03.863 INFO:teuthology.orchestra.run.gibba026.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.864 INFO:teuthology.orchestra.run.gibba026.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.864 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.a] 2021-04-03T03:17:03.864 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.b] 2021-04-03T03:17:03.864 INFO:teuthology.orchestra.run.gibba026.stdout:[mon.c] 2021-04-03T03:17:03.865 INFO:teuthology.orchestra.run.gibba026.stdout:[mds.a] 2021-04-03T03:17:03.865 INFO:teuthology.orchestra.run.gibba026.stdout:[mds.b] 2021-04-03T03:17:03.865 INFO:teuthology.orchestra.run.gibba026.stdout:[mds] 2021-04-03T03:17:03.865 INFO:teuthology.orchestra.run.gibba026.stdout: debug mds = 20 2021-04-03T03:17:03.865 INFO:teuthology.orchestra.run.gibba026.stdout: debug ms = 1 2021-04-03T03:17:03.866 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal frag = True 2021-04-03T03:17:03.866 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:03.866 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal merge size = 5 2021-04-03T03:17:03.866 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal split bits = 3 2021-04-03T03:17:03.867 INFO:teuthology.orchestra.run.gibba026.stdout: mds bal split size = 100 2021-04-03T03:17:03.867 INFO:teuthology.orchestra.run.gibba026.stdout: mds debug frag = True 2021-04-03T03:17:03.867 INFO:teuthology.orchestra.run.gibba026.stdout: mds debug scatterstat = True 2021-04-03T03:17:03.867 INFO:teuthology.orchestra.run.gibba026.stdout: mds op complaint time = 180 2021-04-03T03:17:03.867 INFO:teuthology.orchestra.run.gibba026.stdout: mds verify scatter = True 2021-04-03T03:17:03.868 INFO:teuthology.orchestra.run.gibba026.stdout: osd op complaint time = 180 2021-04-03T03:17:03.868 INFO:teuthology.orchestra.run.gibba026.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.868 INFO:teuthology.orchestra.run.gibba026.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.868 INFO:teuthology.orchestra.run.gibba011.stdout:[global] 2021-04-03T03:17:03.868 INFO:teuthology.orchestra.run.gibba011.stdout: chdir = "" 2021-04-03T03:17:03.869 INFO:teuthology.orchestra.run.gibba011.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:17:03.869 INFO:teuthology.orchestra.run.gibba011.stdout: auth supported = cephx 2021-04-03T03:17:03.869 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.869 INFO:teuthology.orchestra.run.gibba011.stdout: filestore xattr use omap = true 2021-04-03T03:17:03.870 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.870 INFO:teuthology.orchestra.run.gibba011.stdout: mon clock drift allowed = 1.000 2021-04-03T03:17:03.870 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.870 INFO:teuthology.orchestra.run.gibba011.stdout: osd crush chooseleaf type = 0 2021-04-03T03:17:03.870 INFO:teuthology.orchestra.run.gibba011.stdout: auth debug = true 2021-04-03T03:17:03.871 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.871 INFO:teuthology.orchestra.run.gibba011.stdout: ms die on old message = true 2021-04-03T03:17:03.871 INFO:teuthology.orchestra.run.gibba011.stdout: ms die on bug = true 2021-04-03T03:17:03.871 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.872 INFO:teuthology.orchestra.run.gibba011.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:17:03.872 INFO:teuthology.orchestra.run.gibba011.stdout: mon pg warn max object skew = 0 2021-04-03T03:17:03.872 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.872 INFO:teuthology.orchestra.run.gibba011.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:17:03.872 INFO:teuthology.orchestra.run.gibba011.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:17:03.873 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.873 INFO:teuthology.orchestra.run.gibba011.stdout: osd pool default size = 2 2021-04-03T03:17:03.873 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.873 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd allow primary affinity = true 2021-04-03T03:17:03.873 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd allow pg remap = true 2021-04-03T03:17:03.874 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:17:03.874 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:17:03.874 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on no sortbitwise = false 2021-04-03T03:17:03.874 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:17:03.875 INFO:teuthology.orchestra.run.gibba011.stdout: mon warn on too few osds = false 2021-04-03T03:17:03.875 INFO:teuthology.orchestra.run.gibba011.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:17:03.875 INFO:teuthology.orchestra.run.gibba011.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:17:03.875 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.875 INFO:teuthology.orchestra.run.gibba011.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:17:03.876 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.876 INFO:teuthology.orchestra.run.gibba011.stdout: osd default data pool replay window = 5 2021-04-03T03:17:03.876 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.876 INFO:teuthology.orchestra.run.gibba011.stdout: mon allow pool delete = true 2021-04-03T03:17:03.876 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: mon cluster log file level = debug 2021-04-03T03:17:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: debug asserts on shutdown = true 2021-04-03T03:17:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: mon health detail to clog = false 2021-04-03T03:17:03.877 INFO:teuthology.orchestra.run.gibba011.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:17:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:17:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.878 INFO:teuthology.orchestra.run.gibba011.stdout:[osd] 2021-04-03T03:17:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: osd journal size = 100 2021-04-03T03:17:03.878 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.879 INFO:teuthology.orchestra.run.gibba011.stdout: osd scrub load threshold = 5.0 2021-04-03T03:17:03.879 INFO:teuthology.orchestra.run.gibba011.stdout: osd scrub max interval = 600 2021-04-03T03:17:03.879 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.879 INFO:teuthology.orchestra.run.gibba011.stdout: osd recover clone overlap = true 2021-04-03T03:17:03.880 INFO:teuthology.orchestra.run.gibba011.stdout: osd recovery max chunk = 1048576 2021-04-03T03:17:03.880 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.880 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug shutdown = true 2021-04-03T03:17:03.880 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug op order = true 2021-04-03T03:17:03.880 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug verify stray on activate = true 2021-04-03T03:17:03.881 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.881 INFO:teuthology.orchestra.run.gibba011.stdout: osd open classes on start = true 2021-04-03T03:17:03.881 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug pg log writeout = true 2021-04-03T03:17:03.881 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.881 INFO:teuthology.orchestra.run.gibba011.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:17:03.882 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.882 INFO:teuthology.orchestra.run.gibba011.stdout: osd map max advance = 10 2021-04-03T03:17:03.882 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.882 INFO:teuthology.orchestra.run.gibba011.stdout: journal zero on create = true 2021-04-03T03:17:03.882 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.883 INFO:teuthology.orchestra.run.gibba011.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:03.883 INFO:teuthology.orchestra.run.gibba011.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:03.883 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.883 INFO:teuthology.orchestra.run.gibba011.stdout: bdev debug aio = true 2021-04-03T03:17:03.884 INFO:teuthology.orchestra.run.gibba011.stdout: osd debug misdirected ops = true 2021-04-03T03:17:03.884 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore block size = 96636764160 2021-04-03T03:17:03.884 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:03.884 INFO:teuthology.orchestra.run.gibba011.stdout: bluestore fsck on mount = True 2021-04-03T03:17:03.885 INFO:teuthology.orchestra.run.gibba011.stdout: debug bluefs = 20 2021-04-03T03:17:03.885 INFO:teuthology.orchestra.run.gibba011.stdout: debug bluestore = 20 2021-04-03T03:17:03.885 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:03.885 INFO:teuthology.orchestra.run.gibba011.stdout: debug osd = 20 2021-04-03T03:17:03.885 INFO:teuthology.orchestra.run.gibba011.stdout: debug rocksdb = 10 2021-04-03T03:17:03.886 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:03.886 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:03.886 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:03.886 INFO:teuthology.orchestra.run.gibba011.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:03.886 INFO:teuthology.orchestra.run.gibba011.stdout: osd objectstore = bluestore 2021-04-03T03:17:03.887 INFO:teuthology.orchestra.run.gibba011.stdout: osd op complaint time = 180 2021-04-03T03:17:03.887 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.887 INFO:teuthology.orchestra.run.gibba011.stdout:[mgr] 2021-04-03T03:17:03.887 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:03.888 INFO:teuthology.orchestra.run.gibba011.stdout: debug mgr = 20 2021-04-03T03:17:03.888 INFO:teuthology.orchestra.run.gibba011.stdout: debug mon = 20 2021-04-03T03:17:03.888 INFO:teuthology.orchestra.run.gibba011.stdout: debug auth = 20 2021-04-03T03:17:03.888 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.888 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.889 INFO:teuthology.orchestra.run.gibba011.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:03.889 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.889 INFO:teuthology.orchestra.run.gibba011.stdout:[mon] 2021-04-03T03:17:03.889 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:03.889 INFO:teuthology.orchestra.run.gibba011.stdout: debug mon = 20 2021-04-03T03:17:03.890 INFO:teuthology.orchestra.run.gibba011.stdout: debug paxos = 20 2021-04-03T03:17:03.890 INFO:teuthology.orchestra.run.gibba011.stdout: debug auth = 20 2021-04-03T03:17:03.890 INFO:teuthology.orchestra.run.gibba011.stdout: mon data avail warn = 5 2021-04-03T03:17:03.890 INFO:teuthology.orchestra.run.gibba011.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:03.891 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.891 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:03.891 INFO:teuthology.orchestra.run.gibba011.stdout: mon osd prime pg temp = true 2021-04-03T03:17:03.891 INFO:teuthology.orchestra.run.gibba011.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.891 INFO:teuthology.orchestra.run.gibba011.stdout: mon op complaint time = 120 2021-04-03T03:17:03.892 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:03.892 INFO:teuthology.orchestra.run.gibba011.stdout:[client] 2021-04-03T03:17:03.892 INFO:teuthology.orchestra.run.gibba011.stdout: rgw cache enabled = true 2021-04-03T03:17:03.892 INFO:teuthology.orchestra.run.gibba011.stdout: rgw enable ops log = true 2021-04-03T03:17:03.892 INFO:teuthology.orchestra.run.gibba011.stdout: rgw enable usage log = true 2021-04-03T03:17:03.893 INFO:teuthology.orchestra.run.gibba011.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:03.893 INFO:teuthology.orchestra.run.gibba011.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:03.893 INFO:teuthology.orchestra.run.gibba011.stdout: client mount timeout = 600 2021-04-03T03:17:03.893 INFO:teuthology.orchestra.run.gibba011.stdout: debug client = 20 2021-04-03T03:17:03.894 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:03.894 INFO:teuthology.orchestra.run.gibba011.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.894 INFO:teuthology.orchestra.run.gibba011.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.894 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.a] 2021-04-03T03:17:03.894 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.b] 2021-04-03T03:17:03.895 INFO:teuthology.orchestra.run.gibba011.stdout:[mon.c] 2021-04-03T03:17:03.895 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.a] 2021-04-03T03:17:03.895 INFO:teuthology.orchestra.run.gibba011.stdout:[mds.b] 2021-04-03T03:17:03.895 INFO:teuthology.orchestra.run.gibba011.stdout:[mds] 2021-04-03T03:17:03.895 INFO:teuthology.orchestra.run.gibba011.stdout: debug mds = 20 2021-04-03T03:17:03.896 INFO:teuthology.orchestra.run.gibba011.stdout: debug ms = 1 2021-04-03T03:17:03.896 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal frag = True 2021-04-03T03:17:03.896 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:03.896 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal merge size = 5 2021-04-03T03:17:03.897 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal split bits = 3 2021-04-03T03:17:03.897 INFO:teuthology.orchestra.run.gibba011.stdout: mds bal split size = 100 2021-04-03T03:17:03.897 INFO:teuthology.orchestra.run.gibba011.stdout: mds debug frag = True 2021-04-03T03:17:03.897 INFO:teuthology.orchestra.run.gibba011.stdout: mds debug scatterstat = True 2021-04-03T03:17:03.897 INFO:teuthology.orchestra.run.gibba011.stdout: mds op complaint time = 180 2021-04-03T03:17:03.898 INFO:teuthology.orchestra.run.gibba011.stdout: mds verify scatter = True 2021-04-03T03:17:03.898 INFO:teuthology.orchestra.run.gibba011.stdout: osd op complaint time = 180 2021-04-03T03:17:03.898 INFO:teuthology.orchestra.run.gibba011.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.898 INFO:teuthology.orchestra.run.gibba011.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.901 INFO:teuthology.orchestra.run.gibba031.stdout:[global] 2021-04-03T03:17:03.902 INFO:teuthology.orchestra.run.gibba031.stdout: chdir = "" 2021-04-03T03:17:03.902 INFO:teuthology.orchestra.run.gibba031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-04-03T03:17:03.902 INFO:teuthology.orchestra.run.gibba031.stdout: auth supported = cephx 2021-04-03T03:17:03.902 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.903 INFO:teuthology.orchestra.run.gibba031.stdout: filestore xattr use omap = true 2021-04-03T03:17:03.903 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.903 INFO:teuthology.orchestra.run.gibba031.stdout: mon clock drift allowed = 1.000 2021-04-03T03:17:03.903 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.903 INFO:teuthology.orchestra.run.gibba031.stdout: osd crush chooseleaf type = 0 2021-04-03T03:17:03.904 INFO:teuthology.orchestra.run.gibba031.stdout: auth debug = true 2021-04-03T03:17:03.904 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.904 INFO:teuthology.orchestra.run.gibba031.stdout: ms die on old message = true 2021-04-03T03:17:03.904 INFO:teuthology.orchestra.run.gibba031.stdout: ms die on bug = true 2021-04-03T03:17:03.905 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.905 INFO:teuthology.orchestra.run.gibba031.stdout: mon max pg per osd = 10000 # >= luminous 2021-04-03T03:17:03.905 INFO:teuthology.orchestra.run.gibba031.stdout: mon pg warn max object skew = 0 2021-04-03T03:17:03.905 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.905 INFO:teuthology.orchestra.run.gibba031.stdout: # disable pg_autoscaler by default for new pools 2021-04-03T03:17:03.906 INFO:teuthology.orchestra.run.gibba031.stdout: osd_pool_default_pg_autoscale_mode = off 2021-04-03T03:17:03.906 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.906 INFO:teuthology.orchestra.run.gibba031.stdout: osd pool default size = 2 2021-04-03T03:17:03.906 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.906 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd allow primary affinity = true 2021-04-03T03:17:03.907 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd allow pg remap = true 2021-04-03T03:17:03.907 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on legacy crush tunables = false 2021-04-03T03:17:03.907 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on crush straw calc version zero = false 2021-04-03T03:17:03.907 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on no sortbitwise = false 2021-04-03T03:17:03.908 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on osd down out interval zero = false 2021-04-03T03:17:03.908 INFO:teuthology.orchestra.run.gibba031.stdout: mon warn on too few osds = false 2021-04-03T03:17:03.908 INFO:teuthology.orchestra.run.gibba031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-04-03T03:17:03.908 INFO:teuthology.orchestra.run.gibba031.stdout: mon_warn_on_pool_no_redundancy = false 2021-04-03T03:17:03.909 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.909 INFO:teuthology.orchestra.run.gibba031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-04-03T03:17:03.909 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.909 INFO:teuthology.orchestra.run.gibba031.stdout: osd default data pool replay window = 5 2021-04-03T03:17:03.909 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.910 INFO:teuthology.orchestra.run.gibba031.stdout: mon allow pool delete = true 2021-04-03T03:17:03.910 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.910 INFO:teuthology.orchestra.run.gibba031.stdout: mon cluster log file level = debug 2021-04-03T03:17:03.910 INFO:teuthology.orchestra.run.gibba031.stdout: debug asserts on shutdown = true 2021-04-03T03:17:03.910 INFO:teuthology.orchestra.run.gibba031.stdout: mon health detail to clog = false 2021-04-03T03:17:03.911 INFO:teuthology.orchestra.run.gibba031.stdout: mon host = "172.21.2.111,172.21.2.115,[v2:172.21.2.115:3301,v1:172.21.2.115:6790]" 2021-04-03T03:17:03.911 INFO:teuthology.orchestra.run.gibba031.stdout: fsid = 6fa64cfd-c000-43a2-869e-5811f8b4928c 2021-04-03T03:17:03.911 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.911 INFO:teuthology.orchestra.run.gibba031.stdout:[osd] 2021-04-03T03:17:03.912 INFO:teuthology.orchestra.run.gibba031.stdout: osd journal size = 100 2021-04-03T03:17:03.912 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.912 INFO:teuthology.orchestra.run.gibba031.stdout: osd scrub load threshold = 5.0 2021-04-03T03:17:03.912 INFO:teuthology.orchestra.run.gibba031.stdout: osd scrub max interval = 600 2021-04-03T03:17:03.912 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.913 INFO:teuthology.orchestra.run.gibba031.stdout: osd recover clone overlap = true 2021-04-03T03:17:03.913 INFO:teuthology.orchestra.run.gibba031.stdout: osd recovery max chunk = 1048576 2021-04-03T03:17:03.913 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.913 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug shutdown = true 2021-04-03T03:17:03.914 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug op order = true 2021-04-03T03:17:03.914 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug verify stray on activate = true 2021-04-03T03:17:03.914 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.914 INFO:teuthology.orchestra.run.gibba031.stdout: osd open classes on start = true 2021-04-03T03:17:03.914 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug pg log writeout = true 2021-04-03T03:17:03.915 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.915 INFO:teuthology.orchestra.run.gibba031.stdout: osd deep scrub update digest min age = 30 2021-04-03T03:17:03.915 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.915 INFO:teuthology.orchestra.run.gibba031.stdout: osd map max advance = 10 2021-04-03T03:17:03.916 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.916 INFO:teuthology.orchestra.run.gibba031.stdout: journal zero on create = true 2021-04-03T03:17:03.916 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.916 INFO:teuthology.orchestra.run.gibba031.stdout: filestore ondisk finisher threads = 3 2021-04-03T03:17:03.916 INFO:teuthology.orchestra.run.gibba031.stdout: filestore apply finisher threads = 3 2021-04-03T03:17:03.917 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.917 INFO:teuthology.orchestra.run.gibba031.stdout: bdev debug aio = true 2021-04-03T03:17:03.917 INFO:teuthology.orchestra.run.gibba031.stdout: osd debug misdirected ops = true 2021-04-03T03:17:03.917 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore block size = 96636764160 2021-04-03T03:17:03.918 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore compression mode = aggressive 2021-04-03T03:17:03.918 INFO:teuthology.orchestra.run.gibba031.stdout: bluestore fsck on mount = True 2021-04-03T03:17:03.918 INFO:teuthology.orchestra.run.gibba031.stdout: debug bluefs = 20 2021-04-03T03:17:03.918 INFO:teuthology.orchestra.run.gibba031.stdout: debug bluestore = 20 2021-04-03T03:17:03.918 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:03.919 INFO:teuthology.orchestra.run.gibba031.stdout: debug osd = 20 2021-04-03T03:17:03.919 INFO:teuthology.orchestra.run.gibba031.stdout: debug rocksdb = 10 2021-04-03T03:17:03.919 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd backfillfull_ratio = 0.85 2021-04-03T03:17:03.919 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd full ratio = 0.9 2021-04-03T03:17:03.920 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd nearfull ratio = 0.8 2021-04-03T03:17:03.920 INFO:teuthology.orchestra.run.gibba031.stdout: osd failsafe full ratio = 0.95 2021-04-03T03:17:03.920 INFO:teuthology.orchestra.run.gibba031.stdout: osd objectstore = bluestore 2021-04-03T03:17:03.920 INFO:teuthology.orchestra.run.gibba031.stdout: osd op complaint time = 180 2021-04-03T03:17:03.920 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.921 INFO:teuthology.orchestra.run.gibba031.stdout:[mgr] 2021-04-03T03:17:03.921 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:03.921 INFO:teuthology.orchestra.run.gibba031.stdout: debug mgr = 20 2021-04-03T03:17:03.921 INFO:teuthology.orchestra.run.gibba031.stdout: debug mon = 20 2021-04-03T03:17:03.921 INFO:teuthology.orchestra.run.gibba031.stdout: debug auth = 20 2021-04-03T03:17:03.922 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.922 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.922 INFO:teuthology.orchestra.run.gibba031.stdout: mgr/telemetry/nag = false 2021-04-03T03:17:03.922 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.922 INFO:teuthology.orchestra.run.gibba031.stdout:[mon] 2021-04-03T03:17:03.923 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:03.923 INFO:teuthology.orchestra.run.gibba031.stdout: debug mon = 20 2021-04-03T03:17:03.923 INFO:teuthology.orchestra.run.gibba031.stdout: debug paxos = 20 2021-04-03T03:17:03.923 INFO:teuthology.orchestra.run.gibba031.stdout: debug auth = 20 2021-04-03T03:17:03.924 INFO:teuthology.orchestra.run.gibba031.stdout: mon data avail warn = 5 2021-04-03T03:17:03.924 INFO:teuthology.orchestra.run.gibba031.stdout: mon mgr mkfs grace = 240 2021-04-03T03:17:03.924 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min pgs per osd = 4 2021-04-03T03:17:03.924 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd reporter subtree level = osd 2021-04-03T03:17:03.924 INFO:teuthology.orchestra.run.gibba031.stdout: mon osd prime pg temp = true 2021-04-03T03:17:03.925 INFO:teuthology.orchestra.run.gibba031.stdout: mon reweight min bytes per osd = 10 2021-04-03T03:17:03.925 INFO:teuthology.orchestra.run.gibba031.stdout: mon op complaint time = 120 2021-04-03T03:17:03.925 INFO:teuthology.orchestra.run.gibba031.stdout: 2021-04-03T03:17:03.925 INFO:teuthology.orchestra.run.gibba031.stdout:[client] 2021-04-03T03:17:03.926 INFO:teuthology.orchestra.run.gibba031.stdout: rgw cache enabled = true 2021-04-03T03:17:03.926 INFO:teuthology.orchestra.run.gibba031.stdout: rgw enable ops log = true 2021-04-03T03:17:03.926 INFO:teuthology.orchestra.run.gibba031.stdout: rgw enable usage log = true 2021-04-03T03:17:03.926 INFO:teuthology.orchestra.run.gibba031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-04-03T03:17:03.926 INFO:teuthology.orchestra.run.gibba031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-04-03T03:17:03.927 INFO:teuthology.orchestra.run.gibba031.stdout: client mount timeout = 600 2021-04-03T03:17:03.927 INFO:teuthology.orchestra.run.gibba031.stdout: debug client = 20 2021-04-03T03:17:03.927 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:03.927 INFO:teuthology.orchestra.run.gibba031.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.927 INFO:teuthology.orchestra.run.gibba031.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.928 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.a] 2021-04-03T03:17:03.928 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.b] 2021-04-03T03:17:03.928 INFO:teuthology.orchestra.run.gibba031.stdout:[mon.c] 2021-04-03T03:17:03.928 INFO:teuthology.orchestra.run.gibba031.stdout:[mds.a] 2021-04-03T03:17:03.929 INFO:teuthology.orchestra.run.gibba031.stdout:[mds.b] 2021-04-03T03:17:03.929 INFO:teuthology.orchestra.run.gibba031.stdout:[mds] 2021-04-03T03:17:03.929 INFO:teuthology.orchestra.run.gibba031.stdout: debug mds = 20 2021-04-03T03:17:03.929 INFO:teuthology.orchestra.run.gibba031.stdout: debug ms = 1 2021-04-03T03:17:03.929 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal frag = True 2021-04-03T03:17:03.930 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal fragment size max = 10000 2021-04-03T03:17:03.930 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal merge size = 5 2021-04-03T03:17:03.930 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal split bits = 3 2021-04-03T03:17:03.930 INFO:teuthology.orchestra.run.gibba031.stdout: mds bal split size = 100 2021-04-03T03:17:03.930 INFO:teuthology.orchestra.run.gibba031.stdout: mds debug frag = True 2021-04-03T03:17:03.931 INFO:teuthology.orchestra.run.gibba031.stdout: mds debug scatterstat = True 2021-04-03T03:17:03.931 INFO:teuthology.orchestra.run.gibba031.stdout: mds op complaint time = 180 2021-04-03T03:17:03.931 INFO:teuthology.orchestra.run.gibba031.stdout: mds verify scatter = True 2021-04-03T03:17:03.931 INFO:teuthology.orchestra.run.gibba031.stdout: osd op complaint time = 180 2021-04-03T03:17:03.932 INFO:teuthology.orchestra.run.gibba031.stdout: rados mon op timeout = 15m 2021-04-03T03:17:03.932 INFO:teuthology.orchestra.run.gibba031.stdout: rados osd op timeout = 15m 2021-04-03T03:17:03.932 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:17:04.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 Processor -- start 2021-04-03T03:17:04.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- start start 2021-04-03T03:17:04.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f79000603f0 0x7f79001447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79000607d0 con 0x7f79000b17b0 2021-04-03T03:17:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790014b090 con 0x7f79000add50 2021-04-03T03:17:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790014b330 con 0x7f79000603f0 2021-04-03T03:17:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78fffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f79000b17b0 0x7f7900147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58630/0 (socket says 172.21.2.111:58630) 2021-04-03T03:17:04.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78fffff700 1 -- 172.21.2.111:0/153318003 learned_addr learned my addr 172.21.2.111:0/153318003 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 --2- 172.21.2.111:0/153318003 >> v2:172.21.2.111:3300/0 conn(0x7f79000603f0 0x7f79001447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 -- 172.21.2.111:0/153318003 >> v1:172.21.2.115:6789/0 conn(0x7f79000add50 legacy=0x7f7900147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 -- 172.21.2.111:0/153318003 >> v1:172.21.2.111:6789/0 conn(0x7f79000b17b0 legacy=0x7f7900147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 -- 172.21.2.111:0/153318003 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790014b5d0 con 0x7f79000603f0 2021-04-03T03:17:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 --2- 172.21.2.111:0/153318003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f79001447e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f79000b5600 tx=0x7f78e8006dd0).ready entity=mon.0 client_cookie=9b1d868bf264f31f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78fe7fc700 1 -- 172.21.2.111:0/153318003 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e80097b0 con 0x7f79000603f0 2021-04-03T03:17:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78fe7fc700 1 -- 172.21.2.111:0/153318003 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8009910 con 0x7f79000603f0 2021-04-03T03:17:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78fe7fc700 1 -- 172.21.2.111:0/153318003 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e8009c70 con 0x7f79000603f0 2021-04-03T03:17:04.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- 172.21.2.111:0/153318003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 msgr2=0x7f79001447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 --2- 172.21.2.111:0/153318003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f79001447e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f79000b5600 tx=0x7f78e8006dd0).stop 2021-04-03T03:17:04.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- 172.21.2.111:0/153318003 shutdown_connections 2021-04-03T03:17:04.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 --2- 172.21.2.111:0/153318003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f79001447e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:04.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- 172.21.2.111:0/153318003 >> 172.21.2.111:0/153318003 conn(0x7f79000b61c0 msgr2=0x7f790005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:04.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- 172.21.2.111:0/153318003 shutdown_connections 2021-04-03T03:17:04.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- 172.21.2.111:0/153318003 wait complete. 2021-04-03T03:17:04.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 Processor -- start 2021-04-03T03:17:04.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- start start 2021-04-03T03:17:04.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f790015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79000add50 0x7f79000aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79000b17b0 0x7f79000671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7900066650 con 0x7f79000603f0 2021-04-03T03:17:04.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790015a2f0 con 0x7f79000add50 2021-04-03T03:17:04.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f7904a4b700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790015a590 con 0x7f79000b17b0 2021-04-03T03:17:04.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f790015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:04.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f790015acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57842/0 (socket says 172.21.2.111:57842) 2021-04-03T03:17:04.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.005+0000 7f78ff7fe700 1 -- 172.21.2.111:0/595040791 learned_addr learned my addr 172.21.2.111:0/595040791 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:04.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78ff7fe700 1 -- 172.21.2.111:0/595040791 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79000b17b0 msgr2=0x7f79000671f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78ff7fe700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79000b17b0 0x7f79000671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78ff7fe700 1 -- 172.21.2.111:0/595040791 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79000add50 msgr2=0x7f79000aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78ff7fe700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79000add50 0x7f79000aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78ff7fe700 1 -- 172.21.2.111:0/595040791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790015a830 con 0x7f79000603f0 2021-04-03T03:17:04.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78ff7fe700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f790015acd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f79000b5600 tx=0x7f78e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:04.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78fcff9700 1 -- 172.21.2.111:0/595040791 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e8011db0 con 0x7f79000603f0 2021-04-03T03:17:04.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78fcff9700 1 -- 172.21.2.111:0/595040791 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e801e460 con 0x7f79000603f0 2021-04-03T03:17:04.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78fcff9700 1 -- 172.21.2.111:0/595040791 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e801e7c0 con 0x7f79000603f0 2021-04-03T03:17:04.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79000a81f0 con 0x7f79000603f0 2021-04-03T03:17:04.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79001445b0 con 0x7f79000603f0 2021-04-03T03:17:04.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7900057660 con 0x7f79000603f0 2021-04-03T03:17:04.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78fcff9700 1 -- 172.21.2.111:0/595040791 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f78e801a040 con 0x7f79000603f0 2021-04-03T03:17:04.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78fcff9700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78e4056fe0 0x7f78e40594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78fcff9700 1 -- 172.21.2.111:0/595040791 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f78e8071440 con 0x7f79000603f0 2021-04-03T03:17:04.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78feffd700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78e4056fe0 0x7f78e40594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:04.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.009+0000 7f78feffd700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78e4056fe0 0x7f78e40594a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f78f0001f60 tx=0x7f78f00077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:04.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.013+0000 7f78fcff9700 1 -- 172.21.2.111:0/595040791 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f78e8046720 con 0x7f79000603f0 2021-04-03T03:17:04.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.105+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f790015a9c0 con 0x7f79000603f0 2021-04-03T03:17:04.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f78fcff9700 1 -- 172.21.2.111:0/595040791 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f78e8049d40 con 0x7f79000603f0 2021-04-03T03:17:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78e4056fe0 msgr2=0x7f78e40594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78e4056fe0 0x7f78e40594a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f78f0001f60 tx=0x7f78f00077d0).stop 2021-04-03T03:17:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 msgr2=0x7f790015acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f790015acd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f79000b5600 tx=0x7f78e8015000).stop 2021-04-03T03:17:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 shutdown_connections 2021-04-03T03:17:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f79000b17b0 0x7f79000671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f79000603f0 0x7f790015acd0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:04.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f78e4056fe0 0x7f78e40594a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:04.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 --2- 172.21.2.111:0/595040791 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f79000add50 0x7f79000aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 >> 172.21.2.111:0/595040791 conn(0x7f79000b61c0 msgr2=0x7f7900145990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:04.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 shutdown_connections 2021-04-03T03:17:04.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.109+0000 7f7904a4b700 1 -- 172.21.2.111:0/595040791 wait complete. 2021-04-03T03:17:04.118 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2021-04-03T03:17:04.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 Processor -- start 2021-04-03T03:17:04.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- start start 2021-04-03T03:17:04.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe9b00603f0 0x7fe9b00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b014b700 0x7fe9b00b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe9b00b12b0 0x7fe9b00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b00b0f60 con 0x7fe9b00603f0 2021-04-03T03:17:04.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b00607d0 con 0x7fe9b00b12b0 2021-04-03T03:17:04.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00af400 con 0x7fe9b014b700 2021-04-03T03:17:04.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe9b00603f0 0x7fe9b00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:04.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe9b00603f0 0x7fe9b00b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57854/0 (socket says 172.21.2.111:57854) 2021-04-03T03:17:04.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 -- 172.21.2.111:0/2443991800 learned_addr learned my addr 172.21.2.111:0/2443991800 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:04.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 -- 172.21.2.111:0/2443991800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b014b700 msgr2=0x7fe9b00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:04.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 --2- 172.21.2.111:0/2443991800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b014b700 0x7fe9b00b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 -- 172.21.2.111:0/2443991800 >> v2:172.21.2.115:3300/0 conn(0x7fe9b00b12b0 msgr2=0x7fe9b00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:04.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 --2- 172.21.2.111:0/2443991800 >> v2:172.21.2.115:3300/0 conn(0x7fe9b00b12b0 0x7fe9b00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 -- 172.21.2.111:0/2443991800 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00af6a0 con 0x7fe9b00603f0 2021-04-03T03:17:04.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ae59c700 1 --2- 172.21.2.111:0/2443991800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b00603f0 0x7fe9b00b0090 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fe998002f30 tx=0x7fe998006e90).ready entity=mon.0 client_cookie=e236c623eb3d2aef server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:04.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ad59a700 1 -- 172.21.2.111:0/2443991800 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe998009920 con 0x7fe9b00603f0 2021-04-03T03:17:04.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ad59a700 1 -- 172.21.2.111:0/2443991800 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998009a80 con 0x7fe9b00603f0 2021-04-03T03:17:04.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9ad59a700 1 -- 172.21.2.111:0/2443991800 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe998009de0 con 0x7fe9b00603f0 2021-04-03T03:17:04.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- 172.21.2.111:0/2443991800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b00603f0 msgr2=0x7fe9b00b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2443991800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b00603f0 0x7fe9b00b0090 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fe998002f30 tx=0x7fe998006e90).stop 2021-04-03T03:17:04.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- 172.21.2.111:0/2443991800 shutdown_connections 2021-04-03T03:17:04.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2443991800 >> v2:172.21.2.115:3300/0 conn(0x7fe9b00b12b0 0x7fe9b00b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2443991800 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b00603f0 0x7fe9b00b0090 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:04.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2443991800 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b014b700 0x7fe9b00b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- 172.21.2.111:0/2443991800 >> 172.21.2.111:0/2443991800 conn(0x7fe9b00b61c0 msgr2=0x7fe9b005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:04.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- 172.21.2.111:0/2443991800 shutdown_connections 2021-04-03T03:17:04.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- 172.21.2.111:0/2443991800 wait complete. 2021-04-03T03:17:04.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 Processor -- start 2021-04-03T03:17:04.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- start start 2021-04-03T03:17:04.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b00603f0 0x7fe9b0064e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9b00b12b0 0x7fe9b0062c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b014b700 0x7fe9b0067590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00abef0 con 0x7fe9b014b700 2021-04-03T03:17:04.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b0064440 con 0x7fe9b00b12b0 2021-04-03T03:17:04.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9af59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00646e0 con 0x7fe9b00603f0 2021-04-03T03:17:04.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9aed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b014b700 0x7fe9b0067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:04.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9aed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b014b700 0x7fe9b0067590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57860/0 (socket says 172.21.2.111:57860) 2021-04-03T03:17:04.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.173+0000 7fe9aed9d700 1 -- 172.21.2.111:0/2082648034 learned_addr learned my addr 172.21.2.111:0/2082648034 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:04.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9aed9d700 1 -- 172.21.2.111:0/2082648034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b00603f0 msgr2=0x7fe9b0064e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9aed9d700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b00603f0 0x7fe9b0064e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9aed9d700 1 -- 172.21.2.111:0/2082648034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9b00b12b0 msgr2=0x7fe9b0062c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9aed9d700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9b00b12b0 0x7fe9b0062c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9aed9d700 1 -- 172.21.2.111:0/2082648034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00a9430 con 0x7fe9b014b700 2021-04-03T03:17:04.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9aed9d700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b014b700 0x7fe9b0067590 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fe9a80032b0 tx=0x7fe9a8009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:04.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9a77fe700 1 -- 172.21.2.111:0/2082648034 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9a800ca60 con 0x7fe9b014b700 2021-04-03T03:17:04.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9a77fe700 1 -- 172.21.2.111:0/2082648034 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a800cbc0 con 0x7fe9b014b700 2021-04-03T03:17:04.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b00a9f80 con 0x7fe9b014b700 2021-04-03T03:17:04.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9a77fe700 1 -- 172.21.2.111:0/2082648034 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9a8015420 con 0x7fe9b014b700 2021-04-03T03:17:04.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b00ab2c0 con 0x7fe9b014b700 2021-04-03T03:17:04.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b0057660 con 0x7fe9b014b700 2021-04-03T03:17:04.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9a77fe700 1 -- 172.21.2.111:0/2082648034 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe9a801c070 con 0x7fe9b014b700 2021-04-03T03:17:04.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9a77fe700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe994056ea0 0x7fe994059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:04.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9a77fe700 1 -- 172.21.2.111:0/2082648034 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe9a806fd40 con 0x7fe9b014b700 2021-04-03T03:17:04.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9ae59c700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe994056ea0 0x7fe994059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:04.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.177+0000 7fe9ae59c700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe994056ea0 0x7fe994059360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fe998002f30 tx=0x7fe998015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:04.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.181+0000 7fe9a77fe700 1 -- 172.21.2.111:0/2082648034 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe9a8045130 con 0x7fe9b014b700 2021-04-03T03:17:04.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.305+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7fe9b00686d0 con 0x7fe9b014b700 2021-04-03T03:17:04.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9a77fe700 1 -- 172.21.2.111:0/2082648034 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v373) v1 ==== 128+0+0 (secure 0 0 0) 0x7fe9a8048750 con 0x7fe9b014b700 2021-04-03T03:17:04.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe994056ea0 msgr2=0x7fe994059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe994056ea0 0x7fe994059360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fe998002f30 tx=0x7fe998015000).stop 2021-04-03T03:17:04.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b014b700 msgr2=0x7fe9b0067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:04.952 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b014b700 0x7fe9b0067590 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fe9a80032b0 tx=0x7fe9a8009220).stop 2021-04-03T03:17:04.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 shutdown_connections 2021-04-03T03:17:04.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9b014b700 0x7fe9b0067590 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:04.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9b00603f0 0x7fe9b0064e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe994056ea0 0x7fe994059360 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:04.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 --2- 172.21.2.111:0/2082648034 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9b00b12b0 0x7fe9b0062c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:04.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 >> 172.21.2.111:0/2082648034 conn(0x7fe9b00b61c0 msgr2=0x7fe9b005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:04.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 shutdown_connections 2021-04-03T03:17:04.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:04.949+0000 7fe9af59e700 1 -- 172.21.2.111:0/2082648034 wait complete. 2021-04-03T03:17:04.959 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2021-04-03T03:17:04.959 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:17:04.960 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2021-04-03T03:17:05.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 Processor -- start 2021-04-03T03:17:05.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- start start 2021-04-03T03:17:05.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fee1c0603f0 0x7fee1c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:05.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fee1c14b700 0x7fee1c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:05.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1c0607d0 con 0x7fee1c0b12b0 2021-04-03T03:17:05.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1c0af400 con 0x7fee1c0603f0 2021-04-03T03:17:05.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1c0af6a0 con 0x7fee1c14b700 2021-04-03T03:17:05.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fee1c0603f0 0x7fee1c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:05.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fee1c0603f0 0x7fee1c0b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57864/0 (socket says 172.21.2.111:57864) 2021-04-03T03:17:05.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 -- 172.21.2.111:0/1821686928 learned_addr learned my addr 172.21.2.111:0/1821686928 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:05.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 -- 172.21.2.111:0/1821686928 >> v2:172.21.2.115:3300/0 conn(0x7fee1c14b700 msgr2=0x7fee1c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:05.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 --2- 172.21.2.111:0/1821686928 >> v2:172.21.2.115:3300/0 conn(0x7fee1c14b700 0x7fee1c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:05.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 -- 172.21.2.111:0/1821686928 >> v1:172.21.2.111:6789/0 conn(0x7fee1c0b12b0 legacy=0x7fee1c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:05.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 -- 172.21.2.111:0/1821686928 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c0afb40 con 0x7fee1c0603f0 2021-04-03T03:17:05.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 --2- 172.21.2.111:0/1821686928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0b0090 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fee0c002f30 tx=0x7fee0c006e90).ready entity=mon.0 client_cookie=5526d6f5fd3b630d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:05.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee1b7fe700 1 -- 172.21.2.111:0/1821686928 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0c0099c0 con 0x7fee1c0603f0 2021-04-03T03:17:05.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee1b7fe700 1 -- 172.21.2.111:0/1821686928 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c009b20 con 0x7fee1c0603f0 2021-04-03T03:17:05.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee1b7fe700 1 -- 172.21.2.111:0/1821686928 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0c009e80 con 0x7fee1c0603f0 2021-04-03T03:17:05.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- 172.21.2.111:0/1821686928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 msgr2=0x7fee1c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:05.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- 172.21.2.111:0/1821686928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0b0090 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fee0c002f30 tx=0x7fee0c006e90).stop 2021-04-03T03:17:05.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- 172.21.2.111:0/1821686928 shutdown_connections 2021-04-03T03:17:05.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- 172.21.2.111:0/1821686928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0b0090 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:05.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- 172.21.2.111:0/1821686928 >> v2:172.21.2.115:3300/0 conn(0x7fee1c14b700 0x7fee1c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:05.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- 172.21.2.111:0/1821686928 >> 172.21.2.111:0/1821686928 conn(0x7fee1c0b61c0 msgr2=0x7fee1c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:05.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- 172.21.2.111:0/1821686928 shutdown_connections 2021-04-03T03:17:05.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- 172.21.2.111:0/1821686928 wait complete. 2021-04-03T03:17:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 Processor -- start 2021-04-03T03:17:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- start start 2021-04-03T03:17:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee1c0b12b0 0x7fee1c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:05.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee1c14b700 0x7fee1c0ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0a9f20 con 0x7fee1c0603f0 2021-04-03T03:17:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0aa140 con 0x7fee1c0b12b0 2021-04-03T03:17:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee21996700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c15aa30 con 0x7fee1c14b700 2021-04-03T03:17:05.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0a6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57874/0 (socket says 172.21.2.111:57874) 2021-04-03T03:17:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.017+0000 7fee20994700 1 -- 172.21.2.111:0/1318269510 learned_addr learned my addr 172.21.2.111:0/1318269510 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee20994700 1 -- 172.21.2.111:0/1318269510 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee1c14b700 msgr2=0x7fee1c0ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee20994700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee1c14b700 0x7fee1c0ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:05.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee20994700 1 -- 172.21.2.111:0/1318269510 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee1c0b12b0 msgr2=0x7fee1c15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee20994700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee1c0b12b0 0x7fee1c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee20994700 1 -- 172.21.2.111:0/1318269510 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c15a720 con 0x7fee1c0603f0 2021-04-03T03:17:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee20994700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0a6f20 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7fee0c002f30 tx=0x7fee0c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee19ffb700 1 -- 172.21.2.111:0/1318269510 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0c009e20 con 0x7fee1c0603f0 2021-04-03T03:17:05.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee19ffb700 1 -- 172.21.2.111:0/1318269510 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c01e460 con 0x7fee1c0603f0 2021-04-03T03:17:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee19ffb700 1 -- 172.21.2.111:0/1318269510 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0c01e7c0 con 0x7fee1c0603f0 2021-04-03T03:17:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c0a8ba0 con 0x7fee1c0603f0 2021-04-03T03:17:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c0a9880 con 0x7fee1c0603f0 2021-04-03T03:17:05.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c0aa2b0 con 0x7fee1c0603f0 2021-04-03T03:17:05.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee19ffb700 1 -- 172.21.2.111:0/1318269510 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fee0c025030 con 0x7fee1c0603f0 2021-04-03T03:17:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee19ffb700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee00056ef0 0x7fee000593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee19ffb700 1 -- 172.21.2.111:0/1318269510 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fee0c071610 con 0x7fee1c0603f0 2021-04-03T03:17:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee1bfff700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee00056ef0 0x7fee000593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:05.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.021+0000 7fee1bfff700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee00056ef0 0x7fee000593b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fee04002f30 tx=0x7fee040069c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:05.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.025+0000 7fee19ffb700 1 -- 172.21.2.111:0/1318269510 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fee1c0aa2b0 con 0x7fee1c0603f0 2021-04-03T03:17:05.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.153+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7fee1c0aa2b0 con 0x7fee1c0603f0 2021-04-03T03:17:05.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee19ffb700 1 -- 172.21.2.111:0/1318269510 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v374) v1 ==== 130+0+0 (secure 0 0 0) 0x7fee1c0aa2b0 con 0x7fee1c0603f0 2021-04-03T03:17:05.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee00056ef0 msgr2=0x7fee000593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:05.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee00056ef0 0x7fee000593b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fee04002f30 tx=0x7fee040069c0).stop 2021-04-03T03:17:05.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 msgr2=0x7fee1c0a6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:05.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0a6f20 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7fee0c002f30 tx=0x7fee0c015000).stop 2021-04-03T03:17:05.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 shutdown_connections 2021-04-03T03:17:05.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fee1c14b700 0x7fee1c0ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:05.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fee1c0603f0 0x7fee1c0a6f20 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:05.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fee00056ef0 0x7fee000593b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:05.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 --2- 172.21.2.111:0/1318269510 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fee1c0b12b0 0x7fee1c15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:05.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 >> 172.21.2.111:0/1318269510 conn(0x7fee1c0b61c0 msgr2=0x7fee1c14bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:05.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 shutdown_connections 2021-04-03T03:17:05.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:05.953+0000 7fee21996700 1 -- 172.21.2.111:0/1318269510 wait complete. 2021-04-03T03:17:05.964 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:17:06.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 Processor -- start 2021-04-03T03:17:06.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- start start 2021-04-03T03:17:06.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8be80b17b0 0x7f8be8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80607d0 con 0x7f8be80add50 2021-04-03T03:17:06.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be81451c0 con 0x7f8be80b17b0 2021-04-03T03:17:06.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be814b090 con 0x7f8be80603f0 2021-04-03T03:17:06.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8be80b17b0 0x7f8be8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8be80b17b0 0x7f8be8144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57878/0 (socket says 172.21.2.111:57878) 2021-04-03T03:17:06.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 -- 172.21.2.111:0/1072466517 learned_addr learned my addr 172.21.2.111:0/1072466517 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:06.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed198700 1 --2- 172.21.2.111:0/1072466517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 -- 172.21.2.111:0/1072466517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 msgr2=0x7f8be81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:06.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 --2- 172.21.2.111:0/1072466517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be81447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 -- 172.21.2.111:0/1072466517 >> v1:172.21.2.115:6789/0 conn(0x7f8be80add50 legacy=0x7f8be8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:06.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 -- 172.21.2.111:0/1072466517 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be814b330 con 0x7f8be80b17b0 2021-04-03T03:17:06.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bed999700 1 --2- 172.21.2.111:0/1072466517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8144cd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f8be80b5600 tx=0x7f8be0009270).ready entity=mon.0 client_cookie=567987bfaf99dccd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bd7fff700 1 -- 172.21.2.111:0/1072466517 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be000bbc0 con 0x7f8be80b17b0 2021-04-03T03:17:06.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bd7fff700 1 -- 172.21.2.111:0/1072466517 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be000bd20 con 0x7f8be80b17b0 2021-04-03T03:17:06.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bd7fff700 1 -- 172.21.2.111:0/1072466517 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be0011400 con 0x7f8be80b17b0 2021-04-03T03:17:06.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- 172.21.2.111:0/1072466517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 msgr2=0x7f8be8144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 --2- 172.21.2.111:0/1072466517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8144cd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f8be80b5600 tx=0x7f8be0009270).stop 2021-04-03T03:17:06.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- 172.21.2.111:0/1072466517 shutdown_connections 2021-04-03T03:17:06.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 --2- 172.21.2.111:0/1072466517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8144cd0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 --2- 172.21.2.111:0/1072466517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- 172.21.2.111:0/1072466517 >> 172.21.2.111:0/1072466517 conn(0x7f8be80b61c0 msgr2=0x7f8be805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:06.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- 172.21.2.111:0/1072466517 shutdown_connections 2021-04-03T03:17:06.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- 172.21.2.111:0/1072466517 wait complete. 2021-04-03T03:17:06.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 Processor -- start 2021-04-03T03:17:06.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.021+0000 7f8bee19a700 1 -- start start 2021-04-03T03:17:06.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be814a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be80add50 0x7f8be8064b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8065050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8062a70 con 0x7f8be80b17b0 2021-04-03T03:17:06.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8062d10 con 0x7f8be80add50 2021-04-03T03:17:06.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8062fb0 con 0x7f8be80603f0 2021-04-03T03:17:06.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8065050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8065050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57888/0 (socket says 172.21.2.111:57888) 2021-04-03T03:17:06.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 -- 172.21.2.111:0/2472760525 learned_addr learned my addr 172.21.2.111:0/2472760525 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:06.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed198700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be814a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 -- 172.21.2.111:0/2472760525 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 msgr2=0x7f8be814a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be814a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 -- 172.21.2.111:0/2472760525 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be80add50 msgr2=0x7f8be8064b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be80add50 0x7f8be8064b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 -- 172.21.2.111:0/2472760525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8149a20 con 0x7f8be80b17b0 2021-04-03T03:17:06.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed999700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8065050 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f8be80b5600 tx=0x7f8be0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bd67fc700 1 -- 172.21.2.111:0/2472760525 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be001f420 con 0x7f8be80b17b0 2021-04-03T03:17:06.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bd67fc700 1 -- 172.21.2.111:0/2472760525 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be001f580 con 0x7f8be80b17b0 2021-04-03T03:17:06.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be80633f0 con 0x7f8be80b17b0 2021-04-03T03:17:06.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bd67fc700 1 -- 172.21.2.111:0/2472760525 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8be001f8e0 con 0x7f8be80b17b0 2021-04-03T03:17:06.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be80aab70 con 0x7f8be80b17b0 2021-04-03T03:17:06.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be8065860 con 0x7f8be80b17b0 2021-04-03T03:17:06.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bd67fc700 1 -- 172.21.2.111:0/2472760525 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8be0026030 con 0x7f8be80b17b0 2021-04-03T03:17:06.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bd67fc700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bc8056f20 0x7f8bc80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bd67fc700 1 -- 172.21.2.111:0/2472760525 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8be0070e70 con 0x7f8be80b17b0 2021-04-03T03:17:06.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed198700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bc8056f20 0x7f8bc80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.025+0000 7f8bed198700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bc8056f20 0x7f8bc80593e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f8be4008000 tx=0x7f8be4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.029+0000 7f8bd67fc700 1 -- 172.21.2.111:0/2472760525 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8be0046260 con 0x7f8be80b17b0 2021-04-03T03:17:06.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.121+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8be8063170 con 0x7f8be80b17b0 2021-04-03T03:17:06.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.125+0000 7f8bd67fc700 1 -- 172.21.2.111:0/2472760525 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8be00240c0 con 0x7f8be80b17b0 2021-04-03T03:17:06.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bc8056f20 msgr2=0x7f8bc80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bc8056f20 0x7f8bc80593e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f8be4008000 tx=0x7f8be4008040).stop 2021-04-03T03:17:06.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 msgr2=0x7f8be8065050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8065050 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f8be80b5600 tx=0x7f8be0016000).stop 2021-04-03T03:17:06.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 shutdown_connections 2021-04-03T03:17:06.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8be80b17b0 0x7f8be8065050 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8be80603f0 0x7f8be814a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8bc8056f20 0x7f8bc80593e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 --2- 172.21.2.111:0/2472760525 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8be80add50 0x7f8be8064b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 >> 172.21.2.111:0/2472760525 conn(0x7f8be80b61c0 msgr2=0x7f8be81448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:06.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 shutdown_connections 2021-04-03T03:17:06.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.129+0000 7f8bee19a700 1 -- 172.21.2.111:0/2472760525 wait complete. 2021-04-03T03:17:06.137 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:17:06.138 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:06.144 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:06.152 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:06.164 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:06.176 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:06.188 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:17:06.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:06.189 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:06.189 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:06.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:06.212 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:17:06.213 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:17:06.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:06.214 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:06.214 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:06.220 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:17:06.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:06.222 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:06.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:06.223 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:06.223 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:17:06.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:06.233 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:17:06.234 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:17:06.234 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:17:06.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 Processor -- start 2021-04-03T03:17:06.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 -- start start 2021-04-03T03:17:06.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500603f0 0x7fb4500b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb4500b0c90 0x7fb450147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4500607d0 con 0x7fb45014b700 2021-04-03T03:17:06.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4500af400 con 0x7fb4500b0c90 2021-04-03T03:17:06.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500af6a0 con 0x7fb4500603f0 2021-04-03T03:17:06.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb4500b0c90 0x7fb450147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fb4500b0c90 0x7fb450147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57892/0 (socket says 172.21.2.111:57892) 2021-04-03T03:17:06.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3963776163 learned_addr learned my addr 172.21.2.111:0/3963776163 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:06.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3963776163 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500603f0 msgr2=0x7fb4500b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:06.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 --2- 172.21.2.111:0/3963776163 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500603f0 0x7fb4500b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3963776163 >> v1:172.21.2.111:6789/0 conn(0x7fb45014b700 legacy=0x7fb4500b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3963776163 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4500afb40 con 0x7fb4500b0c90 2021-04-03T03:17:06.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44f7fe700 1 --2- 172.21.2.111:0/3963776163 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb4500b0c90 0x7fb450147680 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7fb4400010d0 tx=0x7fb448009270).ready entity=mon.0 client_cookie=d973957900e98f84 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44dffb700 1 -- 172.21.2.111:0/3963776163 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb44800bac0 con 0x7fb4500b0c90 2021-04-03T03:17:06.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44dffb700 1 -- 172.21.2.111:0/3963776163 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800bc20 con 0x7fb4500b0c90 2021-04-03T03:17:06.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44dffb700 1 -- 172.21.2.111:0/3963776163 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb44800bd90 con 0x7fb4500b0c90 2021-04-03T03:17:06.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 -- 172.21.2.111:0/3963776163 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb4500b0c90 msgr2=0x7fb450147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.293+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3963776163 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb4500b0c90 0x7fb450147680 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7fb4400010d0 tx=0x7fb448009270).stop 2021-04-03T03:17:06.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- 172.21.2.111:0/3963776163 shutdown_connections 2021-04-03T03:17:06.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3963776163 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb4500b0c90 0x7fb450147680 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3963776163 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500603f0 0x7fb4500b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- 172.21.2.111:0/3963776163 >> 172.21.2.111:0/3963776163 conn(0x7fb4500b61c0 msgr2=0x7fb45005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:06.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- 172.21.2.111:0/3963776163 shutdown_connections 2021-04-03T03:17:06.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- 172.21.2.111:0/3963776163 wait complete. 2021-04-03T03:17:06.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 Processor -- start 2021-04-03T03:17:06.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- start start 2021-04-03T03:17:06.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb4500603f0 0x7fb450066ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500b0c90 0x7fb4500673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb45014b700 0x7fb450062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45015a410 con 0x7fb45014b700 2021-04-03T03:17:06.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45015a690 con 0x7fb4500603f0 2021-04-03T03:17:06.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45015a930 con 0x7fb4500b0c90 2021-04-03T03:17:06.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb45014b700 0x7fb450062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb45014b700 0x7fb450062e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57898/0 (socket says 172.21.2.111:57898) 2021-04-03T03:17:06.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3190713117 learned_addr learned my addr 172.21.2.111:0/3190713117 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:06.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3190713117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500b0c90 msgr2=0x7fb4500673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500b0c90 0x7fb4500673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3190713117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb4500603f0 msgr2=0x7fb450066ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb4500603f0 0x7fb450066ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 -- 172.21.2.111:0/3190713117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4500a8be0 con 0x7fb45014b700 2021-04-03T03:17:06.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44f7fe700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb45014b700 0x7fb450062e10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7fb4400010d0 tx=0x7fb448016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb447fff700 1 -- 172.21.2.111:0/3190713117 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb44801f420 con 0x7fb45014b700 2021-04-03T03:17:06.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb447fff700 1 -- 172.21.2.111:0/3190713117 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44801f580 con 0x7fb45014b700 2021-04-03T03:17:06.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4500678d0 con 0x7fb45014b700 2021-04-03T03:17:06.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb447fff700 1 -- 172.21.2.111:0/3190713117 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb44801f8e0 con 0x7fb45014b700 2021-04-03T03:17:06.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4500a9450 con 0x7fb45014b700 2021-04-03T03:17:06.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb450057660 con 0x7fb45014b700 2021-04-03T03:17:06.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb447fff700 1 -- 172.21.2.111:0/3190713117 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb448026030 con 0x7fb45014b700 2021-04-03T03:17:06.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb447fff700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb434056f20 0x7fb4340593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb447fff700 1 -- 172.21.2.111:0/3190713117 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fb448070da0 con 0x7fb45014b700 2021-04-03T03:17:06.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.297+0000 7fb44effd700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb434056f20 0x7fb4340593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.301+0000 7fb44effd700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb434056f20 0x7fb4340593e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fb438002f30 tx=0x7fb438006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.301+0000 7fb447fff700 1 -- 172.21.2.111:0/3190713117 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb448046190 con 0x7fb45014b700 2021-04-03T03:17:06.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.393+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb4500a8d70 con 0x7fb45014b700 2021-04-03T03:17:06.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.393+0000 7fb447fff700 1 -- 172.21.2.111:0/3190713117 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v132) v1 ==== 81+0+34051 (secure 0 0 0) 0x7fb4480497b0 con 0x7fb45014b700 2021-04-03T03:17:06.398 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:06.398 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:17:06.398 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 132, 2021-04-03T03:17:06.398 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:17:06.399 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:17:06.399 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:16:54.343386+0000", 2021-04-03T03:17:06.399 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:17:06.399 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:17:06.400 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:17:06.400 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:17:06.400 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:17:06.400 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:17:06.400 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:17:06.401 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:17:06.401 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:17:06.401 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:06.401 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:17:06.402 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:17:06.402 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:17:06.402 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:17:06.402 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:17:06.403 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 26, 2021-04-03T03:17:06.403 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:17:06.403 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:17:06.403 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:17:06.404 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:17:06.404 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:17:06.404 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.404 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:17:06.405 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:17:06.405 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:17:06.405 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:06.405 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:06.406 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:06.406 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:06.406 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:06.406 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:06.407 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:06.407 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:06.407 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:17:06.407 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:17:06.408 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:17:06.408 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:17:06.408 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:17:06.408 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:06.409 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:06.409 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:06.409 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:06.409 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:06.409 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:06.410 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:06.410 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.410 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:17:06.410 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:06.411 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:06.411 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:06.411 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:06.411 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:06.412 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:06.412 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:06.412 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:06.412 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:06.413 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:06.413 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:06.413 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:06.413 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:06.413 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:06.414 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:06.414 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:06.414 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:06.414 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:06.415 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:06.415 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:06.415 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:06.415 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:06.416 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:06.416 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:06.416 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:06.416 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:06.416 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.417 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:06.417 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:06.417 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:06.417 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:06.418 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:06.418 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:06.418 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:06.418 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:06.418 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:06.419 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:06.419 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:06.419 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:17:06.419 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:17:06.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.420 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:06.420 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:17:06.420 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.420 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.421 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.421 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:17:06.421 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:17:06.421 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:17:06.422 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:06.422 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:06.422 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:06.422 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:06.423 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:06.423 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:06.423 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:06.423 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:06.424 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:06.424 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:06.424 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:06.424 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:06.424 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:06.425 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:06.425 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:06.425 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:06.425 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:06.426 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:06.426 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:06.426 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:06.426 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.426 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:17:06.427 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:06.427 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:06.427 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:06.427 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:06.427 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:06.428 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:06.428 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:06.428 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:06.428 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:06.429 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:06.429 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:06.429 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:06.429 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:06.429 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:06.430 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:06.430 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:06.430 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:06.430 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:06.430 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:06.431 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:06.431 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:06.431 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:06.431 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:06.432 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:06.432 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:06.432 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:06.432 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.433 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:06.433 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:06.433 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:06.433 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:06.433 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:06.434 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:06.434 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:06.434 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:06.434 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:06.434 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:06.435 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:06.435 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:17:06.435 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:06.435 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:17:06.435 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.436 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.436 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.436 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 25, 2021-04-03T03:17:06.436 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:17:06.437 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:16:51.364989+0000", 2021-04-03T03:17:06.437 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:06.437 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:06.437 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:06.437 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:06.438 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:06.438 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:06.438 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:06.438 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:06.438 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:06.439 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:06.439 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:06.439 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:06.439 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:06.439 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:06.440 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:06.440 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:06.440 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:06.440 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:06.441 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:06.441 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:06.441 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.441 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "131", 2021-04-03T03:17:06.441 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:06.442 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:06.442 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:06.442 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:06.442 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:06.442 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:06.443 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:06.443 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:06.443 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:06.443 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:06.443 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:06.444 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:06.444 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:06.444 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:06.444 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:06.445 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:06.445 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:06.445 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:06.445 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:06.445 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:06.446 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:06.446 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:06.446 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:06.446 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:06.446 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:06.447 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:06.447 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.447 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:06.447 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:06.448 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:06.448 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:06.448 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:06.448 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:06.449 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:06.449 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:06.449 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:06.449 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:06.449 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:06.450 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:17:06.450 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:17:06.450 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:17:06.450 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:17:06.450 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.451 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:06.451 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:17:06.451 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:17:06.451 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.452 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.452 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.452 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.452 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 26, 2021-04-03T03:17:06.453 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:17:06.453 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:16:52.312016+0000", 2021-04-03T03:17:06.453 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:06.453 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:06.453 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:06.454 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:06.454 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:06.454 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:06.454 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:06.455 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:06.455 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:06.455 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:06.455 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:06.455 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:06.456 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:06.456 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:06.456 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:06.456 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:06.457 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:06.457 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:06.457 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:06.457 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:06.458 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.458 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "132", 2021-04-03T03:17:06.458 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:06.458 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:06.458 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:06.459 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:06.459 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:06.459 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:06.459 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:06.460 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:06.460 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:06.460 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:06.460 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:06.461 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:06.461 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:06.461 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:06.461 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:06.462 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:06.462 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:06.462 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:06.462 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:06.463 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:06.463 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:06.463 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:06.463 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:06.463 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:06.464 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:06.464 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:06.464 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.464 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:06.465 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:06.465 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:06.465 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:06.465 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:06.466 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:06.466 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:06.466 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:06.466 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:06.467 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:06.467 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:06.467 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:17:06.467 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:06.468 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:17:06.468 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:17:06.468 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.468 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.469 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.469 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:06.469 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:17:06.469 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.469 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:17:06.470 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:17:06.470 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.470 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.470 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.471 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.471 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.471 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.471 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.472 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 129, 2021-04-03T03:17:06.472 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.472 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.472 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.473 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.473 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.473 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.473 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:17:06.473 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.474 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.474 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.474 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.474 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:17:06.475 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.475 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.475 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.475 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.476 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.476 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.476 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.476 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.476 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:17:06.477 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.477 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.477 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.477 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.478 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:17:06.478 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.478 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.478 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.478 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.479 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.479 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.479 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.479 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.480 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:17:06.480 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.480 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.480 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.481 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.481 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:17:06.481 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.481 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.481 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.482 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.482 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.482 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.482 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.483 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.483 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:17:06.483 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.483 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.483 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.484 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.484 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:17:06.484 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:06.484 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.485 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.485 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.485 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:17:06.485 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:17:06.485 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:17:06.486 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:17:06.486 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.486 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.486 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.487 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.487 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.487 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.487 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:17:06.487 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:17:06.488 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.488 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.488 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.488 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.489 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.489 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.489 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.489 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 130, 2021-04-03T03:17:06.489 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.490 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.490 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.490 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.490 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.491 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.491 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:17:06.491 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.492 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.492 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.492 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:17:06.492 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.492 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.493 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.493 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.493 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.493 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.494 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.494 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.494 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:17:06.494 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.494 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.495 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.495 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.495 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:17:06.495 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.496 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.496 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.496 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.496 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.497 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.497 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.497 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.497 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:17:06.497 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.498 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.498 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.498 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.498 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:17:06.499 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.499 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.499 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.499 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.500 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.500 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.500 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.500 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.501 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:17:06.501 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.501 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.501 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.501 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.502 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:17:06.502 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:06.502 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.502 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.503 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.503 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:17:06.503 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:17:06.503 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:17:06.504 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:17:06.504 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.504 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.504 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.505 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.505 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.505 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.505 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:17:06.506 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:17:06.506 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.506 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.506 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.506 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.507 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.507 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.507 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.507 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 130, 2021-04-03T03:17:06.508 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.508 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.508 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.508 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.509 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.509 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.509 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:17:06.509 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.509 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.510 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.510 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.510 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:17:06.510 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.511 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.511 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.511 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.511 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.512 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.512 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.512 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.512 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:17:06.512 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.513 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.513 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.513 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.513 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:17:06.514 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.514 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.514 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.514 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.515 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.515 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.515 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.515 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.515 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:17:06.516 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.516 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.516 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.516 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.517 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:17:06.517 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.517 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.517 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.518 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.518 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.518 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.518 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.518 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.519 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:17:06.519 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.519 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.519 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.520 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.520 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:17:06.520 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:06.520 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.520 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.521 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.521 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:17:06.521 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:17:06.521 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:17:06.522 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:17:06.522 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.522 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.522 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.522 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.523 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.523 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.523 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:17:06.523 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:17:06.524 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.524 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.524 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.524 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.525 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.525 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.525 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.525 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 129, 2021-04-03T03:17:06.526 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.526 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.526 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.526 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.526 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.527 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.527 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:17:06.527 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.527 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.528 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.528 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.528 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:17:06.528 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.528 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.529 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.529 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.529 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.529 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.529 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.530 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.530 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:17:06.530 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.530 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.531 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.531 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.531 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:17:06.531 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.531 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.532 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.532 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.532 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.532 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.533 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.533 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.533 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:17:06.533 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.533 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.534 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.534 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.534 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:17:06.534 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.534 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.535 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.535 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.535 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.535 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.536 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.536 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.536 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:17:06.536 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.537 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.537 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.537 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.537 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:17:06.537 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:06.538 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.538 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.538 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.538 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:17:06.539 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:17:06.539 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:17:06.539 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:17:06.539 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.540 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.540 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.540 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.540 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.540 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.541 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:17:06.541 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:17:06.541 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.541 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.542 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.542 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.542 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.542 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.543 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.543 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 130, 2021-04-03T03:17:06.543 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.543 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.543 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.544 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.544 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.544 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.544 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:17:06.545 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.545 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.545 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.545 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.546 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:17:06.546 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.546 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.546 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.547 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.547 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.547 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.547 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.548 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.548 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:17:06.548 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.548 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.548 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.549 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.549 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:17:06.549 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.549 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.550 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.550 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.550 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.550 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.551 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.551 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.551 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:17:06.551 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.551 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.552 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.552 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.552 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:17:06.552 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.553 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.553 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.553 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.553 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.554 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.554 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.554 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.554 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:17:06.554 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.555 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.555 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.555 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.555 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:17:06.556 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:06.556 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.556 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.556 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.557 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:17:06.557 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:17:06.557 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:17:06.557 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:17:06.557 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.558 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.558 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.558 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.558 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.559 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.559 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:17:06.559 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:17:06.559 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.560 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.560 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.560 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.560 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.560 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.561 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.561 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 122, 2021-04-03T03:17:06.561 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.561 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.562 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.562 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.562 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.562 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.562 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:17:06.563 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.563 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.563 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.563 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.564 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:17:06.564 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.564 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.564 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.564 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.565 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.565 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.565 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.565 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.566 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:17:06.566 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.566 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.566 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.566 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.567 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:17:06.567 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.567 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.567 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.568 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.568 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.568 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.568 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.569 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.569 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:17:06.569 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.569 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.570 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.570 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.570 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:17:06.570 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.570 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.571 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.571 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.571 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.571 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.572 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.572 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.572 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:17:06.573 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.573 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.573 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.573 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.573 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:17:06.574 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:06.574 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.574 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.574 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.574 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:17:06.575 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:17:06.575 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:17:06.575 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:17:06.575 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.576 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.576 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.576 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.576 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.577 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.577 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:17:06.577 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:17:06.577 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.577 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.578 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.578 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.578 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.578 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.579 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.579 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 130, 2021-04-03T03:17:06.579 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.579 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.579 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.580 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.580 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.580 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.580 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:17:06.580 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.581 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.581 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.581 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.581 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:17:06.582 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.582 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.582 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.582 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.582 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.583 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.583 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.583 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.583 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:17:06.584 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.584 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.584 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.584 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.584 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:17:06.585 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.585 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.585 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.585 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.585 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.586 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.586 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.586 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.586 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:17:06.587 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.587 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.587 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.587 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.587 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:17:06.588 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.588 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.588 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.588 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.588 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.589 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.589 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.589 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.589 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:17:06.590 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.590 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.590 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.590 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.590 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:17:06.591 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:06.591 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.591 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.591 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.591 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:17:06.592 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:17:06.592 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:17:06.592 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:17:06.592 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.593 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.593 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.593 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.593 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.593 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.594 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:17:06.594 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:17:06.594 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:06.594 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:06.594 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:06.595 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:06.595 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:06.595 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:06.595 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 130, 2021-04-03T03:17:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:06.596 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:17:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.597 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.598 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.598 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.598 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:17:06.598 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.599 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.599 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.599 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.599 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:06.599 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.600 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.600 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.600 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:17:06.600 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.600 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.601 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.601 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.601 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:17:06.601 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.601 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.602 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.602 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.602 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:06.602 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.602 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.603 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.603 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:17:06.603 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.603 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.604 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.604 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.604 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:17:06.604 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.604 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.605 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.605 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.605 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:06.605 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:06.605 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.606 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:06.606 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:17:06.606 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.606 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.606 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:17:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.607 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:17:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:17:06.608 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:17:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:17:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:06.609 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:06.610 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.610 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:06.610 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:17:06.610 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.610 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:17:06.611 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.611 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.611 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.611 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.611 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.612 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:17:06.612 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.612 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.612 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.612 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:17:06.613 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.613 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.613 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.613 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.613 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.614 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:17:06.614 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.614 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.614 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.614 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:17:06.615 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.615 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.615 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.615 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.616 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.616 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:17:06.616 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.616 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.616 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.617 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:17:06.617 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.617 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.617 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.617 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.618 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.618 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:17:06.618 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.618 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.618 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.619 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:17:06.619 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.619 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.619 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.620 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.620 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.620 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:17:06.620 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.620 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.621 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.621 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:17:06.621 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.621 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.621 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.622 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.622 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.622 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:17:06.622 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.622 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.623 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.623 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:17:06.623 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.623 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.623 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.624 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.624 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.624 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:17:06.624 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.625 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.625 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:06.625 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:17:06.625 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:06.625 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:06.626 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:06.626 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:06.626 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:06.626 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:17:06.626 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:06.627 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.627 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:06.627 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:17:06.627 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:17:06.627 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:17:06.628 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:17:06.628 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:17:06.628 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:17:06.628 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:17:06.628 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:17:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:17:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:17:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:17:06.629 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:17:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:17:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:17:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:17:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:17:06.630 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:17:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:17:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:17:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:17:06.631 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:17:06.632 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:17:06.632 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:17:06.632 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:17:06.632 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:06.632 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:06.633 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:17:06.633 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:17:06.633 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:17:06.633 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:17:06.633 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:17:06.634 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:17:06.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb434056f20 msgr2=0x7fb4340593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb434056f20 0x7fb4340593e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fb438002f30 tx=0x7fb438006a60).stop 2021-04-03T03:17:06.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb45014b700 msgr2=0x7fb450062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb45014b700 0x7fb450062e10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7fb4400010d0 tx=0x7fb448016000).stop 2021-04-03T03:17:06.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 shutdown_connections 2021-04-03T03:17:06.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb45014b700 0x7fb450062e10 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb434056f20 0x7fb4340593e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb4500603f0 0x7fb450066ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 --2- 172.21.2.111:0/3190713117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb4500b0c90 0x7fb4500673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 >> 172.21.2.111:0/3190713117 conn(0x7fb4500b61c0 msgr2=0x7fb450147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:06.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 shutdown_connections 2021-04-03T03:17:06.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.397+0000 7fb44ffff700 1 -- 172.21.2.111:0/3190713117 wait complete. 2021-04-03T03:17:06.638 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:06.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcffff700 1 Processor -- start 2021-04-03T03:17:06.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcffff700 1 -- start start 2021-04-03T03:17:06.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3fd00603f0 0x7f3fd00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3fd00b0c90 0x7f3fd0147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00af400 con 0x7f3fd014b700 2021-04-03T03:17:06.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd0060860 con 0x7f3fd00b0c90 2021-04-03T03:17:06.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00af6a0 con 0x7f3fd00603f0 2021-04-03T03:17:06.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcf7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3fd00b0c90 0x7f3fd0147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcf7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3fd00b0c90 0x7f3fd0147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57910/0 (socket says 172.21.2.111:57910) 2021-04-03T03:17:06.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/3868231159 learned_addr learned my addr 172.21.2.111:0/3868231159 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:06.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fceffd700 1 --2- 172.21.2.111:0/3868231159 >> v2:172.21.2.115:3300/0 conn(0x7f3fd00603f0 0x7f3fd00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/3868231159 >> v2:172.21.2.115:3300/0 conn(0x7f3fd00603f0 msgr2=0x7f3fd00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:06.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcf7fe700 1 --2- 172.21.2.111:0/3868231159 >> v2:172.21.2.115:3300/0 conn(0x7f3fd00603f0 0x7f3fd00b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/3868231159 >> v1:172.21.2.115:6789/0 conn(0x7f3fd014b700 legacy=0x7f3fd00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:06.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.697+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/3868231159 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00afb40 con 0x7f3fd00b0c90 2021-04-03T03:17:06.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 --2- 172.21.2.111:0/3868231159 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd00b0c90 0x7f3fd0147680 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f3fd006a3b0 tx=0x7f3fc8009270).ready entity=mon.0 client_cookie=1ad0e125a397e425 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcdffb700 1 -- 172.21.2.111:0/3868231159 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc800bbc0 con 0x7f3fd00b0c90 2021-04-03T03:17:06.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcdffb700 1 -- 172.21.2.111:0/3868231159 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc800bd20 con 0x7f3fd00b0c90 2021-04-03T03:17:06.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcdffb700 1 -- 172.21.2.111:0/3868231159 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc800be90 con 0x7f3fd00b0c90 2021-04-03T03:17:06.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/3868231159 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd00b0c90 msgr2=0x7f3fd0147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 --2- 172.21.2.111:0/3868231159 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd00b0c90 0x7f3fd0147680 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f3fd006a3b0 tx=0x7f3fc8009270).stop 2021-04-03T03:17:06.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/3868231159 shutdown_connections 2021-04-03T03:17:06.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 --2- 172.21.2.111:0/3868231159 >> v2:172.21.2.115:3300/0 conn(0x7f3fd00603f0 0x7f3fd00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 --2- 172.21.2.111:0/3868231159 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd00b0c90 0x7f3fd0147680 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/3868231159 >> 172.21.2.111:0/3868231159 conn(0x7f3fd00b61c0 msgr2=0x7f3fd005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:06.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/3868231159 shutdown_connections 2021-04-03T03:17:06.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/3868231159 wait complete. 2021-04-03T03:17:06.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 Processor -- start 2021-04-03T03:17:06.709 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- start start 2021-04-03T03:17:06.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3fd00603f0 0x7f3fd014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3fd00b0c90 0x7f3fd0159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd014b700 0x7f3fd00a7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00a7500 con 0x7f3fd014b700 2021-04-03T03:17:06.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00663c0 con 0x7f3fd00b0c90 2021-04-03T03:17:06.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00677e0 con 0x7f3fd00603f0 2021-04-03T03:17:06.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd014b700 0x7f3fd00a7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd014b700 0x7f3fd00a7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57916/0 (socket says 172.21.2.111:57916) 2021-04-03T03:17:06.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/2125934955 learned_addr learned my addr 172.21.2.111:0/2125934955 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:06.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/2125934955 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3fd00603f0 msgr2=0x7f3fd014cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3fd00603f0 0x7f3fd014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/2125934955 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3fd00b0c90 msgr2=0x7f3fd0159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3fd00b0c90 0x7f3fd0159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 -- 172.21.2.111:0/2125934955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00665e0 con 0x7f3fd014b700 2021-04-03T03:17:06.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcf7fe700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd014b700 0x7f3fd00a7930 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f3fd006a3b0 tx=0x7f3fc8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fc7fff700 1 -- 172.21.2.111:0/2125934955 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc801f420 con 0x7f3fd014b700 2021-04-03T03:17:06.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fc7fff700 1 -- 172.21.2.111:0/2125934955 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc801f580 con 0x7f3fd014b700 2021-04-03T03:17:06.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd0066880 con 0x7f3fd014b700 2021-04-03T03:17:06.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd00add00 con 0x7f3fd014b700 2021-04-03T03:17:06.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fc7fff700 1 -- 172.21.2.111:0/2125934955 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc801f8e0 con 0x7f3fd014b700 2021-04-03T03:17:06.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd00a7670 con 0x7f3fd014b700 2021-04-03T03:17:06.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fc7fff700 1 -- 172.21.2.111:0/2125934955 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3fc8026030 con 0x7f3fd014b700 2021-04-03T03:17:06.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fc7fff700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3fb4056f20 0x7f3fb40593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fc7fff700 1 -- 172.21.2.111:0/2125934955 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3fc8070f90 con 0x7f3fd014b700 2021-04-03T03:17:06.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.701+0000 7f3fceffd700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3fb4056f20 0x7f3fb40593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.705+0000 7f3fceffd700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3fb4056f20 0x7f3fb40593e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f3fb8007000 tx=0x7f3fb8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.705+0000 7f3fc7fff700 1 -- 172.21.2.111:0/2125934955 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3fc8046270 con 0x7f3fd014b700 2021-04-03T03:17:06.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd0057660 con 0x7f3fd014b700 2021-04-03T03:17:06.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fc7fff700 1 -- 172.21.2.111:0/2125934955 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f3fc8024870 con 0x7f3fd014b700 2021-04-03T03:17:06.830 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:06.830 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":58,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3599494763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3599494763},{"type":"v1","addr":"172.21.2.115:6833","nonce":3599494763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:53.339332+0000","modified":"2021-04-03T03:16:54.343270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7890":{"gid":7890,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/233883066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":233883066},{"type":"v1","addr":"172.21.2.111:6835","nonce":233883066}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-04-03T03:17:06.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3fb4056f20 msgr2=0x7f3fb40593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3fb4056f20 0x7f3fb40593e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f3fb8007000 tx=0x7f3fb8008040).stop 2021-04-03T03:17:06.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd014b700 msgr2=0x7f3fd00a7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd014b700 0x7f3fd00a7930 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f3fd006a3b0 tx=0x7f3fc8016000).stop 2021-04-03T03:17:06.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 shutdown_connections 2021-04-03T03:17:06.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3fd014b700 0x7f3fd00a7930 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3fd00603f0 0x7f3fd014cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3fb4056f20 0x7f3fb40593e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 --2- 172.21.2.111:0/2125934955 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3fd00b0c90 0x7f3fd0159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:06.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 >> 172.21.2.111:0/2125934955 conn(0x7f3fd00b61c0 msgr2=0x7f3fd0146a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:06.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 shutdown_connections 2021-04-03T03:17:06.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.829+0000 7f3fcffff700 1 -- 172.21.2.111:0/2125934955 wait complete. 2021-04-03T03:17:06.834 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 58 2021-04-03T03:17:06.838 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:17:06.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 Processor -- start 2021-04-03T03:17:06.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 -- start start 2021-04-03T03:17:06.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa8d0005640 0x7fa8d00d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa8d00d8e90 0x7fa8d00e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d003d840 con 0x7fa8d003f120 2021-04-03T03:17:06.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d003faa0 con 0x7fa8d00d8e90 2021-04-03T03:17:06.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d003dca0 con 0x7fa8d0005640 2021-04-03T03:17:06.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dc9b3700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fa8d003f120 0x7fa8d0041fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58712/0 (socket says 172.21.2.111:58712) 2021-04-03T03:17:06.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dc9b3700 1 -- 172.21.2.111:0/2103729858 learned_addr learned my addr 172.21.2.111:0/2103729858 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:06.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd1b4700 1 --2- 172.21.2.111:0/2103729858 >> v2:172.21.2.111:3300/0 conn(0x7fa8d00d8e90 0x7fa8d00e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8d7fff700 1 --2- 172.21.2.111:0/2103729858 >> v2:172.21.2.115:3300/0 conn(0x7fa8d0005640 0x7fa8d00d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd1b4700 1 -- 172.21.2.111:0/2103729858 >> v2:172.21.2.115:3300/0 conn(0x7fa8d0005640 msgr2=0x7fa8d00d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:06.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd1b4700 1 --2- 172.21.2.111:0/2103729858 >> v2:172.21.2.115:3300/0 conn(0x7fa8d0005640 0x7fa8d00d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd1b4700 1 -- 172.21.2.111:0/2103729858 >> v1:172.21.2.111:6789/0 conn(0x7fa8d003f120 legacy=0x7fa8d0041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd1b4700 1 -- 172.21.2.111:0/2103729858 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d003f800 con 0x7fa8d00d8e90 2021-04-03T03:17:06.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd1b4700 1 --2- 172.21.2.111:0/2103729858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00e2bc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7fa8d806b5d0 tx=0x7fa8d805ad70).ready entity=mon.0 client_cookie=cb2824cfc474335b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8d77fe700 1 -- 172.21.2.111:0/2103729858 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d805bda0 con 0x7fa8d00d8e90 2021-04-03T03:17:06.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8d77fe700 1 -- 172.21.2.111:0/2103729858 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d805df60 con 0x7fa8d00d8e90 2021-04-03T03:17:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8d77fe700 1 -- 172.21.2.111:0/2103729858 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d80699f0 con 0x7fa8d00d8e90 2021-04-03T03:17:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/2103729858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d00d8e90 msgr2=0x7fa8d00e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.905+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/2103729858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00e2bc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7fa8d806b5d0 tx=0x7fa8d805ad70).stop 2021-04-03T03:17:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/2103729858 shutdown_connections 2021-04-03T03:17:06.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/2103729858 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00e2bc0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/2103729858 >> v2:172.21.2.115:3300/0 conn(0x7fa8d0005640 0x7fa8d00d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/2103729858 >> 172.21.2.111:0/2103729858 conn(0x7fa8d00462b0 msgr2=0x7fa8d00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/2103729858 shutdown_connections 2021-04-03T03:17:06.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/2103729858 wait complete. 2021-04-03T03:17:06.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 Processor -- start 2021-04-03T03:17:06.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- start start 2021-04-03T03:17:06.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8d0005640 0x7fa8d00db070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d003f120 0x7fa8d00d7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00ea830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d003c5c0 con 0x7fa8d003f120 2021-04-03T03:17:06.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0039580 con 0x7fa8d00d8e90 2021-04-03T03:17:06.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0039820 con 0x7fa8d0005640 2021-04-03T03:17:06.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d003f120 0x7fa8d00d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d003f120 0x7fa8d00d7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57926/0 (socket says 172.21.2.111:57926) 2021-04-03T03:17:06.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 -- 172.21.2.111:0/1404864169 learned_addr learned my addr 172.21.2.111:0/1404864169 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:06.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dc9b3700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8d0005640 0x7fa8d00db070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd1b4700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 -- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8d0005640 msgr2=0x7fa8d00db070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8d0005640 0x7fa8d00db070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 -- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8d00d8e90 msgr2=0x7fa8d00ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:06.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 -- 172.21.2.111:0/1404864169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00e72a0 con 0x7fa8d003f120 2021-04-03T03:17:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd1b4700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:17:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d7fff700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d003f120 0x7fa8d00d7590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fa8c4003070 tx=0x7fa8c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d5ffb700 1 -- 172.21.2.111:0/1404864169 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8c4009980 con 0x7fa8d003f120 2021-04-03T03:17:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d5ffb700 1 -- 172.21.2.111:0/1404864169 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c4009ae0 con 0x7fa8d003f120 2021-04-03T03:17:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d00dc3c0 con 0x7fa8d003f120 2021-04-03T03:17:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d5ffb700 1 -- 172.21.2.111:0/1404864169 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8c4009e40 con 0x7fa8d003f120 2021-04-03T03:17:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00ead20 con 0x7fa8d003f120 2021-04-03T03:17:06.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0004d30 con 0x7fa8d003f120 2021-04-03T03:17:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d5ffb700 1 -- 172.21.2.111:0/1404864169 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa8c4013480 con 0x7fa8d003f120 2021-04-03T03:17:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d5ffb700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8c0056ea0 0x7fa8c0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8d5ffb700 1 -- 172.21.2.111:0/1404864169 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa8c4070a50 con 0x7fa8d003f120 2021-04-03T03:17:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dc9b3700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8c0056ea0 0x7fa8c0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:06.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.909+0000 7fa8dc9b3700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8c0056ea0 0x7fa8c0059360 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fa8cc00d000 tx=0x7fa8cc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:06.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:06.913+0000 7fa8d5ffb700 1 -- 172.21.2.111:0/1404864169 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa8c4045de0 con 0x7fa8d003f120 2021-04-03T03:17:07.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.037+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8d0006d20 con 0x7fa8d003f120 2021-04-03T03:17:07.952 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:17:07.949+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:17:07.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8d5ffb700 1 -- 172.21.2.111:0/1404864169 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8c4049400 con 0x7fa8d003f120 2021-04-03T03:17:07.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8c0056ea0 msgr2=0x7fa8c0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:07.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8c0056ea0 0x7fa8c0059360 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fa8cc00d000 tx=0x7fa8cc009040).stop 2021-04-03T03:17:07.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d003f120 msgr2=0x7fa8d00d7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:07.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d003f120 0x7fa8d00d7590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fa8c4003070 tx=0x7fa8c4008040).stop 2021-04-03T03:17:07.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 shutdown_connections 2021-04-03T03:17:07.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa8c0056ea0 0x7fa8c0059360 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:07.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa8d00d8e90 0x7fa8d00ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:07.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa8d0005640 0x7fa8d00db070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:07.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 --2- 172.21.2.111:0/1404864169 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa8d003f120 0x7fa8d00d7590 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:07.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 >> 172.21.2.111:0/1404864169 conn(0x7fa8d00462b0 msgr2=0x7fa8d0041cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:07.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 shutdown_connections 2021-04-03T03:17:07.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:07.961+0000 7fa8dd9b5700 1 -- 172.21.2.111:0/1404864169 wait complete. 2021-04-03T03:17:07.967 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:17:07.972 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:07.980 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:17:08.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.025+0000 7f0e9e5f0700 1 Processor -- start 2021-04-03T03:17:08.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.025+0000 7f0e9e5f0700 1 -- start start 2021-04-03T03:17:08.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e980ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0e98144910 0x7f0e98144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e980afff0 con 0x7f0e98060390 2021-04-03T03:17:08.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e98141260 con 0x7f0e98144910 2021-04-03T03:17:08.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980b0290 con 0x7f0e980acf10 2021-04-03T03:17:08.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0e98144910 0x7f0e98144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f0e98144910 0x7f0e98144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57956/0 (socket says 172.21.2.111:57956) 2021-04-03T03:17:08.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4264010337 learned_addr learned my addr 172.21.2.111:0/4264010337 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:08.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9d5ee700 1 --2- 172.21.2.111:0/4264010337 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e980ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4264010337 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 msgr2=0x7f0e980ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:08.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- 172.21.2.111:0/4264010337 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e980ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4264010337 >> v1:172.21.2.115:6789/0 conn(0x7f0e98060390 legacy=0x7f0e980a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:08.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4264010337 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980b0530 con 0x7f0e98144910 2021-04-03T03:17:08.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- 172.21.2.111:0/4264010337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e98144d40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f0e980b0fe0 tx=0x7f0e90009270).ready entity=mon.0 client_cookie=41b218691ffca3e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:08.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e8bfff700 1 -- 172.21.2.111:0/4264010337 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e9000bbc0 con 0x7f0e98144910 2021-04-03T03:17:08.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e8bfff700 1 -- 172.21.2.111:0/4264010337 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9000bd20 con 0x7f0e98144910 2021-04-03T03:17:08.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e8bfff700 1 -- 172.21.2.111:0/4264010337 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e90011400 con 0x7f0e98144910 2021-04-03T03:17:08.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4264010337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 msgr2=0x7f0e98144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4264010337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e98144d40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f0e980b0fe0 tx=0x7f0e90009270).stop 2021-04-03T03:17:08.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4264010337 shutdown_connections 2021-04-03T03:17:08.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4264010337 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e98144d40 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4264010337 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e980ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4264010337 >> 172.21.2.111:0/4264010337 conn(0x7f0e98063d70 msgr2=0x7f0e98064180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:08.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4264010337 shutdown_connections 2021-04-03T03:17:08.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4264010337 wait complete. 2021-04-03T03:17:08.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 Processor -- start 2021-04-03T03:17:08.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- start start 2021-04-03T03:17:08.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e98060390 0x7f0e98140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e98065670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e981566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9813e1e0 con 0x7f0e98144910 2021-04-03T03:17:08.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98066240 con 0x7f0e98060390 2021-04-03T03:17:08.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9e5f0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980aca20 con 0x7f0e980acf10 2021-04-03T03:17:08.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e981566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e981566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57962/0 (socket says 172.21.2.111:57962) 2021-04-03T03:17:08.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4242963821 learned_addr learned my addr 172.21.2.111:0/4242963821 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:08.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 msgr2=0x7f0e98065670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9cded700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e98065670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e98065670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9d5ee700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e98060390 0x7f0e98140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e98060390 msgr2=0x7f0e98140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e98060390 0x7f0e98140030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 -- 172.21.2.111:0/4242963821 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e98065b60 con 0x7f0e98144910 2021-04-03T03:17:08.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9cded700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e98065670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:08.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e9ddef700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e981566f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f0e980b0fe0 tx=0x7f0e90016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:08.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.029+0000 7f0e8a7fc700 1 -- 172.21.2.111:0/4242963821 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e9001f420 con 0x7f0e98144910 2021-04-03T03:17:08.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e8a7fc700 1 -- 172.21.2.111:0/4242963821 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9001f580 con 0x7f0e98144910 2021-04-03T03:17:08.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e8a7fc700 1 -- 172.21.2.111:0/4242963821 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e9001f8e0 con 0x7f0e98144910 2021-04-03T03:17:08.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e98064f30 con 0x7f0e98144910 2021-04-03T03:17:08.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e980ac6a0 con 0x7f0e98144910 2021-04-03T03:17:08.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e98065360 con 0x7f0e98144910 2021-04-03T03:17:08.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e8a7fc700 1 -- 172.21.2.111:0/4242963821 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0e90026030 con 0x7f0e98144910 2021-04-03T03:17:08.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e8a7fc700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e80056fb0 0x7f0e80059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e8a7fc700 1 -- 172.21.2.111:0/4242963821 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0e90070eb0 con 0x7f0e98144910 2021-04-03T03:17:08.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e9d5ee700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e80056fb0 0x7f0e80059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e9d5ee700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e80056fb0 0x7f0e80059470 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f0e94008000 tx=0x7f0e94008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:08.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.033+0000 7f0e8a7fc700 1 -- 172.21.2.111:0/4242963821 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0e90046170 con 0x7f0e98144910 2021-04-03T03:17:08.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.161+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e98057660 con 0x7f0e98144910 2021-04-03T03:17:08.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.161+0000 7f0e8a7fc700 1 -- 172.21.2.111:0/4242963821 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f0e90015070 con 0x7f0e98144910 2021-04-03T03:17:08.164 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:08.165 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":59,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3599494763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3599494763},{"type":"v1","addr":"172.21.2.115:6833","nonce":3599494763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[{"mdsmap":{"epoch":59,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:16:53.339332+0000","modified":"2021-04-03T03:17:07.953371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-04-03T03:17:08.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.161+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e80056fb0 msgr2=0x7f0e80059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.161+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e80056fb0 0x7f0e80059470 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f0e94008000 tx=0x7f0e94008040).stop 2021-04-03T03:17:08.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.161+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 msgr2=0x7f0e981566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.161+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e981566f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f0e980b0fe0 tx=0x7f0e90016000).stop 2021-04-03T03:17:08.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 shutdown_connections 2021-04-03T03:17:08.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0e98144910 0x7f0e981566f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0e80056fb0 0x7f0e80059470 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0e98060390 0x7f0e98140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 --2- 172.21.2.111:0/4242963821 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0e980acf10 0x7f0e98065670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 >> 172.21.2.111:0/4242963821 conn(0x7f0e98063d70 msgr2=0x7f0e980b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:08.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 shutdown_connections 2021-04-03T03:17:08.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.165+0000 7f0e9e5f0700 1 -- 172.21.2.111:0/4242963821 wait complete. 2021-04-03T03:17:08.169 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 59 2021-04-03T03:17:08.172 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:17:08.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6643fff700 1 Processor -- start 2021-04-03T03:17:08.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6643fff700 1 -- start start 2021-04-03T03:17:08.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6643fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0d90b0 0x7f663c0e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6643fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c03d840 con 0x7f663c005640 2021-04-03T03:17:08.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6643fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c03f800 con 0x7f663c03f120 2021-04-03T03:17:08.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6643fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c03faa0 con 0x7f663c0d90b0 2021-04-03T03:17:08.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66427fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f663c005640 0x7f663c0d89a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58760/0 (socket says 172.21.2.111:58760) 2021-04-03T03:17:08.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66427fc700 1 -- 172.21.2.111:0/995564776 learned_addr learned my addr 172.21.2.111:0/995564776 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:08.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932291623 0 0) 0x7f663c03d840 con 0x7f663c005640 2021-04-03T03:17:08.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6624003240 con 0x7f663c005640 2021-04-03T03:17:08.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0d90b0 0x7f663c0e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3611424585 0 0) 0x7f6624003240 con 0x7f663c005640 2021-04-03T03:17:08.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0d90b0 msgr2=0x7f663c0e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:08.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0d90b0 0x7f663c0e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 >> v1:172.21.2.115:6789/0 conn(0x7f663c03f120 legacy=0x7f663c041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:08.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c005ad0 con 0x7f663c005640 2021-04-03T03:17:08.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f66340016d0 con 0x7f663c005640 2021-04-03T03:17:08.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 >> v1:172.21.2.111:6789/0 conn(0x7f663c005640 legacy=0x7f663c0d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66240037e0 0x7f6624003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66240044e0 0x7f6624004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6624004e50 0x7f662400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c03faa0 con 0x7f6624004e50 2021-04-03T03:17:08.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66240052d0 con 0x7f66240044e0 2021-04-03T03:17:08.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662400bc20 con 0x7f66240037e0 2021-04-03T03:17:08.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0d90b0 0x7f663c0e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:08.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6624004e50 0x7f662400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 -- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66240037e0 msgr2=0x7f6624003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:08.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66240037e0 0x7f6624003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:08.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 -- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66240044e0 msgr2=0x7f6624004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:08.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66240044e0 0x7f6624004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:08.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 -- 172.21.2.111:0/995564776 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624003240 con 0x7f6624004e50 2021-04-03T03:17:08.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.285+0000 7f66437fe700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6624004e50 0x7f662400b730 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f664405dc50 tx=0x7f6644064040).ready entity=mon.0 client_cookie=3196c7777da88535 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:08.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664405bda0 con 0x7f6624004e50 2021-04-03T03:17:08.297 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664405df60 con 0x7f6624004e50 2021-04-03T03:17:08.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6641ffb700 1 -- 172.21.2.111:0/995564776 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66440699f0 con 0x7f6624004e50 2021-04-03T03:17:08.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/995564776 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6624004e50 msgr2=0x7f662400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 --2- 172.21.2.111:0/995564776 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6624004e50 0x7f662400b730 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f664405dc50 tx=0x7f6644064040).stop 2021-04-03T03:17:08.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f66437fe700 1 -- 172.21.2.111:0/995564776 reap_dead start 2021-04-03T03:17:08.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/995564776 shutdown_connections 2021-04-03T03:17:08.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/995564776 >> 172.21.2.111:0/995564776 conn(0x7f663c0462b0 msgr2=0x7f663c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:08.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/995564776 shutdown_connections 2021-04-03T03:17:08.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/995564776 wait complete. 2021-04-03T03:17:08.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 Processor -- start 2021-04-03T03:17:08.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- start start 2021-04-03T03:17:08.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f663c044910 0x7f663c03a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f663c0a01b0 0x7f663c0a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0a1200 0x7f663c0ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0d4550 con 0x7f663c044910 2021-04-03T03:17:08.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0a1d30 con 0x7f663c0a01b0 2021-04-03T03:17:08.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c044cf0 con 0x7f663c0a1200 2021-04-03T03:17:08.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f663c044910 0x7f663c03a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f663c044910 0x7f663c03a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57978/0 (socket says 172.21.2.111:57978) 2021-04-03T03:17:08.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 -- 172.21.2.111:0/3344661711 learned_addr learned my addr 172.21.2.111:0/3344661711 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:08.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f66437fe700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0a1200 0x7f663c0ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 -- 172.21.2.111:0/3344661711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0a1200 msgr2=0x7f663c0ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0a1200 0x7f663c0ddad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:08.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 -- 172.21.2.111:0/3344661711 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f663c0a01b0 msgr2=0x7f663c0a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:08.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f663c0a01b0 0x7f663c0a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:08.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 -- 172.21.2.111:0/3344661711 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c09f820 con 0x7f663c044910 2021-04-03T03:17:08.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6642ffd700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f663c044910 0x7f663c03a8e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f662c000f30 tx=0x7f662c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:08.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f663bfff700 1 -- 172.21.2.111:0/3344661711 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f662c00d570 con 0x7f663c044910 2021-04-03T03:17:08.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f663bfff700 1 -- 172.21.2.111:0/3344661711 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00d6d0 con 0x7f663c044910 2021-04-03T03:17:08.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c09fac0 con 0x7f663c044910 2021-04-03T03:17:08.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c09f4a0 con 0x7f663c044910 2021-04-03T03:17:08.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f663bfff700 1 -- 172.21.2.111:0/3344661711 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f662c00da30 con 0x7f663c044910 2021-04-03T03:17:08.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c004d30 con 0x7f663c044910 2021-04-03T03:17:08.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f663bfff700 1 -- 172.21.2.111:0/3344661711 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f662c00f070 con 0x7f663c044910 2021-04-03T03:17:08.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f663bfff700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6628056fb0 0x7f6628059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:08.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.289+0000 7f663bfff700 1 -- 172.21.2.111:0/3344661711 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f662c06fdb0 con 0x7f663c044910 2021-04-03T03:17:08.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.293+0000 7f66427fc700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6628056fb0 0x7f6628059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:08.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.293+0000 7f66427fc700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6628056fb0 0x7f6628059470 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f663c043d60 tx=0x7f663c03ec70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:08.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.293+0000 7f663bfff700 1 -- 172.21.2.111:0/3344661711 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f662c045030 con 0x7f663c044910 2021-04-03T03:17:08.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:08.421+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f663c006d20 con 0x7f663c044910 2021-04-03T03:17:09.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f663bfff700 1 -- 172.21.2.111:0/3344661711 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f662c048650 con 0x7f663c044910 2021-04-03T03:17:09.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6628056fb0 msgr2=0x7f6628059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:09.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6628056fb0 0x7f6628059470 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f663c043d60 tx=0x7f663c03ec70).stop 2021-04-03T03:17:09.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f663c044910 msgr2=0x7f663c03a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:09.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f663c044910 0x7f663c03a8e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f662c000f30 tx=0x7f662c006dd0).stop 2021-04-03T03:17:09.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 shutdown_connections 2021-04-03T03:17:09.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f663c0a1200 0x7f663c0ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:09.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f663c044910 0x7f663c03a8e0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:09.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6628056fb0 0x7f6628059470 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:09.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 --2- 172.21.2.111:0/3344661711 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f663c0a01b0 0x7f663c0a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:09.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 >> 172.21.2.111:0/3344661711 conn(0x7f663c0462b0 msgr2=0x7f663c045640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:09.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 shutdown_connections 2021-04-03T03:17:09.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.053+0000 7f6643fff700 1 -- 172.21.2.111:0/3344661711 wait complete. 2021-04-03T03:17:09.063 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:17:09.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 Processor -- start 2021-04-03T03:17:09.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- start start 2021-04-03T03:17:09.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 0x7f83fc1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:09.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83fc0b17b0 0x7f83fc144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:09.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc0607d0 con 0x7f83fc0add50 2021-04-03T03:17:09.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc1451c0 con 0x7f83fc0b17b0 2021-04-03T03:17:09.128 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc14b090 con 0x7f83fc0603f0 2021-04-03T03:17:09.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83fc0b17b0 0x7f83fc144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:09.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f83fc0b17b0 0x7f83fc144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57986/0 (socket says 172.21.2.111:57986) 2021-04-03T03:17:09.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 -- 172.21.2.111:0/3777765429 learned_addr learned my addr 172.21.2.111:0/3777765429 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:09.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 -- 172.21.2.111:0/3777765429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 msgr2=0x7f83fc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:09.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 --2- 172.21.2.111:0/3777765429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 0x7f83fc1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:09.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 -- 172.21.2.111:0/3777765429 >> v1:172.21.2.115:6789/0 conn(0x7f83fc0add50 legacy=0x7f83fc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:09.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 -- 172.21.2.111:0/3777765429 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc14b330 con 0x7f83fc0b17b0 2021-04-03T03:17:09.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fbfff700 1 --2- 172.21.2.111:0/3777765429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc144cd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f83fc0b5600 tx=0x7f83f4009270).ready entity=mon.0 client_cookie=984c47a88eba57f8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:09.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fa7fc700 1 -- 172.21.2.111:0/3777765429 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f400bbc0 con 0x7f83fc0b17b0 2021-04-03T03:17:09.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fa7fc700 1 -- 172.21.2.111:0/3777765429 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f400bd20 con 0x7f83fc0b17b0 2021-04-03T03:17:09.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f83fa7fc700 1 -- 172.21.2.111:0/3777765429 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f4011400 con 0x7f83fc0b17b0 2021-04-03T03:17:09.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- 172.21.2.111:0/3777765429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 msgr2=0x7f83fc144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:09.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 --2- 172.21.2.111:0/3777765429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc144cd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f83fc0b5600 tx=0x7f83f4009270).stop 2021-04-03T03:17:09.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- 172.21.2.111:0/3777765429 shutdown_connections 2021-04-03T03:17:09.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 --2- 172.21.2.111:0/3777765429 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc144cd0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:09.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 --2- 172.21.2.111:0/3777765429 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 0x7f83fc1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:09.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- 172.21.2.111:0/3777765429 >> 172.21.2.111:0/3777765429 conn(0x7f83fc0b61c0 msgr2=0x7f83fc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:09.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- 172.21.2.111:0/3777765429 shutdown_connections 2021-04-03T03:17:09.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.121+0000 7f84008e4700 1 -- 172.21.2.111:0/3777765429 wait complete. 2021-04-03T03:17:09.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 Processor -- start 2021-04-03T03:17:09.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 -- start start 2021-04-03T03:17:09.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 0x7f83fc14a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:09.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83fc0add50 0x7f83fc15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:09.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:09.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc15a100 con 0x7f83fc0b17b0 2021-04-03T03:17:09.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc066af0 con 0x7f83fc0add50 2021-04-03T03:17:09.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc066d90 con 0x7f83fc0603f0 2021-04-03T03:17:09.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:09.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:57996/0 (socket says 172.21.2.111:57996) 2021-04-03T03:17:09.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 -- 172.21.2.111:0/192283662 learned_addr learned my addr 172.21.2.111:0/192283662 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:09.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 -- 172.21.2.111:0/192283662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 msgr2=0x7f83fc14a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:09.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 0x7f83fc14a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:09.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 -- 172.21.2.111:0/192283662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83fc0add50 msgr2=0x7f83fc15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:09.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83fc0add50 0x7f83fc15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:09.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 -- 172.21.2.111:0/192283662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc067030 con 0x7f83fc0b17b0 2021-04-03T03:17:09.136 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fbfff700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc15a9b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f83fc0b5600 tx=0x7f83f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:09.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83f8ff9700 1 -- 172.21.2.111:0/192283662 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f401f420 con 0x7f83fc0b17b0 2021-04-03T03:17:09.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83f8ff9700 1 -- 172.21.2.111:0/192283662 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f401f580 con 0x7f83fc0b17b0 2021-04-03T03:17:09.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc066770 con 0x7f83fc0b17b0 2021-04-03T03:17:09.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83f8ff9700 1 -- 172.21.2.111:0/192283662 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f401f8e0 con 0x7f83fc0b17b0 2021-04-03T03:17:09.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc065fc0 con 0x7f83fc0b17b0 2021-04-03T03:17:09.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc067840 con 0x7f83fc0b17b0 2021-04-03T03:17:09.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83f8ff9700 1 -- 172.21.2.111:0/192283662 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f83f4026030 con 0x7f83fc0b17b0 2021-04-03T03:17:09.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83f8ff9700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f83e0056f20 0x7f83e00593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:09.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83f8ff9700 1 -- 172.21.2.111:0/192283662 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f83f4070f20 con 0x7f83fc0b17b0 2021-04-03T03:17:09.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fb7fe700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f83e0056f20 0x7f83e00593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:09.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.125+0000 7f83fb7fe700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f83e0056f20 0x7f83e00593e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f83ec002f30 tx=0x7f83ec006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:09.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.129+0000 7f83f8ff9700 1 -- 172.21.2.111:0/192283662 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f83f40462b0 con 0x7f83fc0b17b0 2021-04-03T03:17:09.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:09.225+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83fc066940 con 0x7f83fc0b17b0 2021-04-03T03:17:10.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.054+0000 7f83f8ff9700 1 -- 172.21.2.111:0/192283662 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v134) v1 ==== 185+0+0 (secure 0 0 0) 0x7f83f40240e0 con 0x7f83fc0b17b0 2021-04-03T03:17:10.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f83e0056f20 msgr2=0x7f83e00593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:10.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f83e0056f20 0x7f83e00593e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f83ec002f30 tx=0x7f83ec006a60).stop 2021-04-03T03:17:10.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 msgr2=0x7f83fc15a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:10.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc15a9b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f83fc0b5600 tx=0x7f83f4016000).stop 2021-04-03T03:17:10.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 shutdown_connections 2021-04-03T03:17:10.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f83fc0b17b0 0x7f83fc15a9b0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:10.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f83fc0603f0 0x7f83fc14a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:10.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f83e0056f20 0x7f83e00593e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:10.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 --2- 172.21.2.111:0/192283662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f83fc0add50 0x7f83fc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:10.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 >> 172.21.2.111:0/192283662 conn(0x7f83fc0b61c0 msgr2=0x7f83fc147320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:10.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 shutdown_connections 2021-04-03T03:17:10.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.058+0000 7f84008e4700 1 -- 172.21.2.111:0/192283662 wait complete. 2021-04-03T03:17:10.062 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:17:10.066 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:17:10.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 Processor -- start 2021-04-03T03:17:10.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- start start 2021-04-03T03:17:10.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f01f80603f0 0x7f01f81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:10.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f01f80b17b0 0x7f01f8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:10.129 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f80607d0 con 0x7f01f80add50 2021-04-03T03:17:10.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f81451c0 con 0x7f01f80b17b0 2021-04-03T03:17:10.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f814b090 con 0x7f01f80603f0 2021-04-03T03:17:10.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f01f80b17b0 0x7f01f8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:10.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f01f80b17b0 0x7f01f8144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58004/0 (socket says 172.21.2.111:58004) 2021-04-03T03:17:10.130 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 -- 172.21.2.111:0/2979334836 learned_addr learned my addr 172.21.2.111:0/2979334836 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:10.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 -- 172.21.2.111:0/2979334836 >> v2:172.21.2.115:3300/0 conn(0x7f01f80603f0 msgr2=0x7f01f81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:10.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fd8e1700 1 --2- 172.21.2.111:0/2979334836 >> v2:172.21.2.115:3300/0 conn(0x7f01f80603f0 0x7f01f81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:10.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 --2- 172.21.2.111:0/2979334836 >> v2:172.21.2.115:3300/0 conn(0x7f01f80603f0 0x7f01f81447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:10.131 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 -- 172.21.2.111:0/2979334836 >> v1:172.21.2.115:6789/0 conn(0x7f01f80add50 legacy=0x7f01f8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:10.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 -- 172.21.2.111:0/2979334836 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f814b330 con 0x7f01f80b17b0 2021-04-03T03:17:10.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fd8e1700 1 --2- 172.21.2.111:0/2979334836 >> v2:172.21.2.115:3300/0 conn(0x7f01f80603f0 0x7f01f81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:10.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe0e2700 1 --2- 172.21.2.111:0/2979334836 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80b17b0 0x7f01f8144cd0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f01f80ae520 tx=0x7f01f0009270).ready entity=mon.0 client_cookie=854109a3ccbdc7fd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:10.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fc8df700 1 -- 172.21.2.111:0/2979334836 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01f000bbc0 con 0x7f01f80b17b0 2021-04-03T03:17:10.132 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fc8df700 1 -- 172.21.2.111:0/2979334836 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000bd20 con 0x7f01f80b17b0 2021-04-03T03:17:10.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fc8df700 1 -- 172.21.2.111:0/2979334836 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01f000be90 con 0x7f01f80b17b0 2021-04-03T03:17:10.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- 172.21.2.111:0/2979334836 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80b17b0 msgr2=0x7f01f8144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:10.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/2979334836 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80b17b0 0x7f01f8144cd0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f01f80ae520 tx=0x7f01f0009270).stop 2021-04-03T03:17:10.133 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- 172.21.2.111:0/2979334836 shutdown_connections 2021-04-03T03:17:10.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/2979334836 >> v2:172.21.2.115:3300/0 conn(0x7f01f80603f0 0x7f01f81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:10.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/2979334836 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80b17b0 0x7f01f8144cd0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:10.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- 172.21.2.111:0/2979334836 >> 172.21.2.111:0/2979334836 conn(0x7f01f80b61c0 msgr2=0x7f01f805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:10.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- 172.21.2.111:0/2979334836 shutdown_connections 2021-04-03T03:17:10.134 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.126+0000 7f01fe8e3700 1 -- 172.21.2.111:0/2979334836 wait complete. 2021-04-03T03:17:10.137 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 Processor -- start 2021-04-03T03:17:10.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 -- start start 2021-04-03T03:17:10.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01f80603f0 0x7f01f80654a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:10.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80add50 0x7f01f80ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:10.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f01f80b17b0 0x7f01f8156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:10.138 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f8064b30 con 0x7f01f80add50 2021-04-03T03:17:10.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f8064d60 con 0x7f01f80603f0 2021-04-03T03:17:10.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f8065000 con 0x7f01f80b17b0 2021-04-03T03:17:10.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80add50 0x7f01f80ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:10.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80add50 0x7f01f80ab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58010/0 (socket says 172.21.2.111:58010) 2021-04-03T03:17:10.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 -- 172.21.2.111:0/844882828 learned_addr learned my addr 172.21.2.111:0/844882828 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:10.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe0e2700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f01f80b17b0 0x7f01f8156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:10.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd8e1700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01f80603f0 0x7f01f80654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:10.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 -- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f01f80b17b0 msgr2=0x7f01f8156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:10.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f01f80b17b0 0x7f01f8156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:10.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 -- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01f80603f0 msgr2=0x7f01f80654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:10.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01f80603f0 0x7f01f80654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:10.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 -- 172.21.2.111:0/844882828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f80aba30 con 0x7f01f80add50 2021-04-03T03:17:10.141 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd0e0700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80add50 0x7f01f80ab540 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f01ec001f60 tx=0x7f01ec007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:10.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01eaffd700 1 -- 172.21.2.111:0/844882828 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec002ea0 con 0x7f01f80add50 2021-04-03T03:17:10.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01eaffd700 1 -- 172.21.2.111:0/844882828 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00f470 con 0x7f01f80add50 2021-04-03T03:17:10.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01eaffd700 1 -- 172.21.2.111:0/844882828 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec00f7d0 con 0x7f01f80add50 2021-04-03T03:17:10.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f80aa5a0 con 0x7f01f80add50 2021-04-03T03:17:10.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f80aabb0 con 0x7f01f80add50 2021-04-03T03:17:10.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f80651c0 con 0x7f01f80add50 2021-04-03T03:17:10.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01eaffd700 1 -- 172.21.2.111:0/844882828 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f01ec019030 con 0x7f01f80add50 2021-04-03T03:17:10.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01eaffd700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f01d8056f60 0x7f01d8059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:10.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01eaffd700 1 -- 172.21.2.111:0/844882828 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6064+0+0 (secure 0 0 0) 0x7f01ec06f500 con 0x7f01f80add50 2021-04-03T03:17:10.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd8e1700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f01d8056f60 0x7f01d8059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:10.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.130+0000 7f01fd8e1700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f01d8056f60 0x7f01d8059420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f01f400b000 tx=0x7f01f4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:10.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.134+0000 7f01eaffd700 1 -- 172.21.2.111:0/844882828 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f01ec044a50 con 0x7f01f80add50 2021-04-03T03:17:10.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:10.230+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01f8057660 con 0x7f01f80add50 2021-04-03T03:17:11.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01eaffd700 1 -- 172.21.2.111:0/844882828 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v135) v1 ==== 173+0+0 (secure 0 0 0) 0x7f01ec048070 con 0x7f01f80add50 2021-04-03T03:17:11.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f01d8056f60 msgr2=0x7f01d8059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:11.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f01d8056f60 0x7f01d8059420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f01f400b000 tx=0x7f01f4008040).stop 2021-04-03T03:17:11.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80add50 msgr2=0x7f01f80ab540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:11.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80add50 0x7f01f80ab540 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f01ec001f60 tx=0x7f01ec007bd0).stop 2021-04-03T03:17:11.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 shutdown_connections 2021-04-03T03:17:11.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f01f80b17b0 0x7f01f8156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f01d8056f60 0x7f01d8059420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f01f80603f0 0x7f01f80654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 --2- 172.21.2.111:0/844882828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f01f80add50 0x7f01f80ab540 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 >> 172.21.2.111:0/844882828 conn(0x7f01f80b61c0 msgr2=0x7f01f80b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:11.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 shutdown_connections 2021-04-03T03:17:11.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.066+0000 7f01fe8e3700 1 -- 172.21.2.111:0/844882828 wait complete. 2021-04-03T03:17:11.071 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:17:11.075 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:17:11.075 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:17:11.076 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:17:11.076 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:17:11.076 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:17:11.077 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:17:11.077 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:17:11.077 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:17:11.081 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:17:11.081 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:17:11.082 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:17:11.083 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:17:11.126 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:17:11.127 DEBUG:teuthology.parallel:result is None 2021-04-03T03:17:11.128 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:17:11.128 DEBUG:teuthology.parallel:result is None 2021-04-03T03:17:11.129 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:17:11.163 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:17:11.166 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:17:11.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 Processor -- start 2021-04-03T03:17:11.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- start start 2021-04-03T03:17:11.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f9554005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:11.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f9554041fe0 0x7f9554042440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:11.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955403d840 con 0x7f95540d89a0 2021-04-03T03:17:11.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95540d9540 con 0x7f9554041fe0 2021-04-03T03:17:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955403faa0 con 0x7f955403f120 2021-04-03T03:17:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955ad9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f95540d89a0 0x7f95540d8e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58822/0 (socket says 172.21.2.111:58822) 2021-04-03T03:17:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955ad9d700 1 -- 172.21.2.111:0/334768304 learned_addr learned my addr 172.21.2.111:0/334768304 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:11.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955a59c700 1 --2- 172.21.2.111:0/334768304 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f9554005640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f9559d9b700 1 --2- 172.21.2.111:0/334768304 >> v2:172.21.2.115:3300/0 conn(0x7f9554041fe0 0x7f9554042440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955a59c700 1 -- 172.21.2.111:0/334768304 >> v2:172.21.2.115:3300/0 conn(0x7f9554041fe0 msgr2=0x7f9554042440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955a59c700 1 --2- 172.21.2.111:0/334768304 >> v2:172.21.2.115:3300/0 conn(0x7f9554041fe0 0x7f9554042440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955a59c700 1 -- 172.21.2.111:0/334768304 >> v1:172.21.2.111:6789/0 conn(0x7f95540d89a0 legacy=0x7f95540d8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955a59c700 1 -- 172.21.2.111:0/334768304 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554005b30 con 0x7f955403f120 2021-04-03T03:17:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955959a700 1 -- 172.21.2.111:0/334768304 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648723848 0 0) 0x7f955403d840 con 0x7f95540d89a0 2021-04-03T03:17:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955a59c700 1 --2- 172.21.2.111:0/334768304 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f9554005640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9544002f30 tx=0x7f9544006dd0).ready entity=mon.2 client_cookie=5d4b8a481f9a78c1 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955959a700 1 -- 172.21.2.111:0/334768304 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9544009760 con 0x7f955403f120 2021-04-03T03:17:11.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955959a700 1 -- 172.21.2.111:0/334768304 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440098c0 con 0x7f955403f120 2021-04-03T03:17:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955959a700 1 -- 172.21.2.111:0/334768304 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9544009c20 con 0x7f955403f120 2021-04-03T03:17:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- 172.21.2.111:0/334768304 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 msgr2=0x7f9554005640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 --2- 172.21.2.111:0/334768304 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f9554005640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9544002f30 tx=0x7f9544006dd0).stop 2021-04-03T03:17:11.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- 172.21.2.111:0/334768304 shutdown_connections 2021-04-03T03:17:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 --2- 172.21.2.111:0/334768304 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f9554005640 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 --2- 172.21.2.111:0/334768304 >> v2:172.21.2.115:3300/0 conn(0x7f9554041fe0 0x7f9554042440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- 172.21.2.111:0/334768304 >> 172.21.2.111:0/334768304 conn(0x7f95540462b0 msgr2=0x7f95540466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- 172.21.2.111:0/334768304 shutdown_connections 2021-04-03T03:17:11.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.186+0000 7f955b59e700 1 -- 172.21.2.111:0/334768304 wait complete. 2021-04-03T03:17:11.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 Processor -- start 2021-04-03T03:17:11.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 -- start start 2021-04-03T03:17:11.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f95540dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:11.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9554041fe0 0x7f95540e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:11.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95540d89a0 0x7f95540e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:11.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554039580 con 0x7f9554041fe0 2021-04-03T03:17:11.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540a3380 con 0x7f95540d89a0 2021-04-03T03:17:11.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955403a920 con 0x7f955403f120 2021-04-03T03:17:11.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9554041fe0 0x7f95540e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:11.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9554041fe0 0x7f95540e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58034/0 (socket says 172.21.2.111:58034) 2021-04-03T03:17:11.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 -- 172.21.2.111:0/4106006471 learned_addr learned my addr 172.21.2.111:0/4106006471 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:11.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955a59c700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f95540dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:11.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 -- 172.21.2.111:0/4106006471 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 msgr2=0x7f95540dd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:11.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f95540dd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 -- 172.21.2.111:0/4106006471 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95540d89a0 msgr2=0x7f95540e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95540d89a0 0x7f95540e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 -- 172.21.2.111:0/4106006471 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955403abc0 con 0x7f9554041fe0 2021-04-03T03:17:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f9559d9b700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9554041fe0 0x7f95540e7b60 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f954c00c000 tx=0x7f954c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:11.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f95537fe700 1 -- 172.21.2.111:0/4106006471 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c00ca10 con 0x7f9554041fe0 2021-04-03T03:17:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f95537fe700 1 -- 172.21.2.111:0/4106006471 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c00cb70 con 0x7f9554041fe0 2021-04-03T03:17:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955403ae60 con 0x7f9554041fe0 2021-04-03T03:17:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f95537fe700 1 -- 172.21.2.111:0/4106006471 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c0123f0 con 0x7f9554041fe0 2021-04-03T03:17:11.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9554044240 con 0x7f9554041fe0 2021-04-03T03:17:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9554046fc0 con 0x7f9554041fe0 2021-04-03T03:17:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f95537fe700 1 -- 172.21.2.111:0/4106006471 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f954c012550 con 0x7f9554041fe0 2021-04-03T03:17:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f95537fe700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9540056ea0 0x7f9540059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f95537fe700 1 -- 172.21.2.111:0/4106006471 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f954c0702a0 con 0x7f9554041fe0 2021-04-03T03:17:11.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955a59c700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9540056ea0 0x7f9540059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:11.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.190+0000 7f955a59c700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9540056ea0 0x7f9540059360 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f954401b000 tx=0x7f9544015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:11.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.194+0000 7f95537fe700 1 -- 172.21.2.111:0/4106006471 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f954c0457f0 con 0x7f9554041fe0 2021-04-03T03:17:11.298 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:11.298+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f9554004d30 con 0x7f9554041fe0 2021-04-03T03:17:12.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.186+0000 7f95537fe700 1 -- 172.21.2.111:0/4106006471 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v137) v1 ==== 97+0+0 (secure 0 0 0) 0x7f954c048e10 con 0x7f9554041fe0 2021-04-03T03:17:12.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9540056ea0 msgr2=0x7f9540059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9540056ea0 0x7f9540059360 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f954401b000 tx=0x7f9544015000).stop 2021-04-03T03:17:12.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9554041fe0 msgr2=0x7f95540e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9554041fe0 0x7f95540e7b60 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f954c00c000 tx=0x7f954c006dd0).stop 2021-04-03T03:17:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 shutdown_connections 2021-04-03T03:17:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9540056ea0 0x7f9540059360 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f95540d89a0 0x7f95540e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f955403f120 0x7f95540dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 --2- 172.21.2.111:0/4106006471 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9554041fe0 0x7f95540e7b60 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 >> 172.21.2.111:0/4106006471 conn(0x7f95540462b0 msgr2=0x7f955403dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 shutdown_connections 2021-04-03T03:17:12.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.190+0000 7f955b59e700 1 -- 172.21.2.111:0/4106006471 wait complete. 2021-04-03T03:17:12.194 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:17:12.204 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:17:12.204 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:17:12.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.302+0000 7fde61e4c700 1 Processor -- start 2021-04-03T03:17:12.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.302+0000 7fde61e4c700 1 -- start start 2021-04-03T03:17:12.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 0x7fde5c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c0607d0 con 0x7fde5c0603f0 2021-04-03T03:17:12.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c0af400 con 0x7fde5c0b0c90 2021-04-03T03:17:12.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0af6a0 con 0x7fde5c14b700 2021-04-03T03:17:12.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fde5c0603f0 0x7fde5c0b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58834/0 (socket says 172.21.2.111:58834) 2021-04-03T03:17:12.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 -- 172.21.2.111:0/2161021924 learned_addr learned my addr 172.21.2.111:0/2161021924 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:12.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5bfff700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 0x7fde5c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605980030 0 0) 0x7fde5c0607d0 con 0x7fde5c0603f0 2021-04-03T03:17:12.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3c003240 con 0x7fde5c0603f0 2021-04-03T03:17:12.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3159723920 0 0) 0x7fde3c003240 con 0x7fde5c0603f0 2021-04-03T03:17:12.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 msgr2=0x7fde5c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:12.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 0x7fde5c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.312 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 >> v1:172.21.2.115:6789/0 conn(0x7fde5c0b0c90 legacy=0x7fde5c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0afb40 con 0x7fde5c0603f0 2021-04-03T03:17:12.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fde4c0016d0 con 0x7fde5c0603f0 2021-04-03T03:17:12.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 >> v1:172.21.2.111:6789/0 conn(0x7fde5c0603f0 legacy=0x7fde5c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 0x7fde3c003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde3c003cc0 0x7fde3c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde3c0049d0 0x7fde3c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0af6a0 con 0x7fde3c0049d0 2021-04-03T03:17:12.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c004250 con 0x7fde3c003cc0 2021-04-03T03:17:12.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c00b730 con 0x7fde5c14b700 2021-04-03T03:17:12.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde3c0049d0 0x7fde3c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5bfff700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde3c003cc0 0x7fde3c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 0x7fde3c003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 -- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 msgr2=0x7fde3c003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:12.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 0x7fde3c003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 -- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde3c003cc0 msgr2=0x7fde3c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:12.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde3c003cc0 0x7fde3c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 -- 172.21.2.111:0/2161021924 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c003240 con 0x7fde3c0049d0 2021-04-03T03:17:12.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c14b700 0x7fde3c003760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:12.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5bfff700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde3c003cc0 0x7fde3c0044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:12.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde3c0049d0 0x7fde3c004e50 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fde5c0b5640 tx=0x7fde54007ef0).ready entity=mon.0 client_cookie=af484a7229009bf0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde54001540 con 0x7fde3c0049d0 2021-04-03T03:17:12.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5400c9b0 con 0x7fde3c0049d0 2021-04-03T03:17:12.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde5b7fe700 1 -- 172.21.2.111:0/2161021924 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde5400cd10 con 0x7fde3c0049d0 2021-04-03T03:17:12.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- 172.21.2.111:0/2161021924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde3c0049d0 msgr2=0x7fde3c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 --2- 172.21.2.111:0/2161021924 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde3c0049d0 0x7fde3c004e50 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fde5c0b5640 tx=0x7fde54007ef0).stop 2021-04-03T03:17:12.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde6164b700 1 -- 172.21.2.111:0/2161021924 reap_dead start 2021-04-03T03:17:12.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- 172.21.2.111:0/2161021924 shutdown_connections 2021-04-03T03:17:12.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- 172.21.2.111:0/2161021924 >> 172.21.2.111:0/2161021924 conn(0x7fde5c0b61c0 msgr2=0x7fde5c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- 172.21.2.111:0/2161021924 shutdown_connections 2021-04-03T03:17:12.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- 172.21.2.111:0/2161021924 wait complete. 2021-04-03T03:17:12.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 Processor -- start 2021-04-03T03:17:12.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- start start 2021-04-03T03:17:12.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde5c0aa5c0 0x7fde5c063580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c063a70 0x7fde5c063ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde5c159d20 0x7fde5c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0643c0 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0a94e0 con 0x7fde5c159d20 2021-04-03T03:17:12.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde61e4c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c156720 con 0x7fde5c063a70 2021-04-03T03:17:12.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde5c0aa5c0 0x7fde5c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde5c0aa5c0 0x7fde5c063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58054/0 (socket says 172.21.2.111:58054) 2021-04-03T03:17:12.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 -- 172.21.2.111:0/1761947579 learned_addr learned my addr 172.21.2.111:0/1761947579 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:12.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 -- 172.21.2.111:0/1761947579 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c063a70 msgr2=0x7fde5c063ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c063a70 0x7fde5c063ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 -- 172.21.2.111:0/1761947579 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde5c159d20 msgr2=0x7fde5c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde5c159d20 0x7fde5c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 -- 172.21.2.111:0/1761947579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0aa9a0 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.306+0000 7fde60e4a700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde5c0aa5c0 0x7fde5c063580 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fde5c062e70 tx=0x7fde4c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde59ffb700 1 -- 172.21.2.111:0/1761947579 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4c007170 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde59ffb700 1 -- 172.21.2.111:0/1761947579 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c0072d0 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde59ffb700 1 -- 172.21.2.111:0/1761947579 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4c00ba00 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c0a9710 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c0a9c50 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c0a9e10 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde59ffb700 1 -- 172.21.2.111:0/1761947579 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fde4c00bb60 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde59ffb700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde40056ef0 0x7fde400593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde59ffb700 1 -- 172.21.2.111:0/1761947579 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fde4c070220 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde5bfff700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde40056ef0 0x7fde400593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde5bfff700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde40056ef0 0x7fde400593b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fde44008000 tx=0x7fde44008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.310+0000 7fde59ffb700 1 -- 172.21.2.111:0/1761947579 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fde4c045830 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.438+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fde5c067ec0 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.438+0000 7fde59ffb700 1 -- 172.21.2.111:0/1761947579 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fde4c048e50 con 0x7fde5c0aa5c0 2021-04-03T03:17:12.440 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:12.440 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:17:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:17:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:17:12.441 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:17:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.442 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:17:12.443 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:17:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.444 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.445 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:17:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:17:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.446 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:17:12.447 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:17:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.448 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:17:12.449 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:17:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.450 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:17:12.451 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:17:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.452 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:17:12.453 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:17:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.454 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:17:12.455 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:17:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.456 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:17:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:17:12.457 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.458 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:17:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:17:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.459 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.460 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:17:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:17:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:12.461 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:17:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:17:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.462 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:17:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:17:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:17:12.463 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:17:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:17:12.464 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:17:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:17:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:17:12.465 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:17:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:17:12.466 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:17:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.467 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:17:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:17:12.468 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:17:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:17:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.469 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:17:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:17:12.470 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:17:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:17:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.471 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:12.472 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:12.472 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:17:12.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde40056ef0 msgr2=0x7fde400593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde40056ef0 0x7fde400593b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fde44008000 tx=0x7fde44008040).stop 2021-04-03T03:17:12.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde5c0aa5c0 msgr2=0x7fde5c063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde5c0aa5c0 0x7fde5c063580 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fde5c062e70 tx=0x7fde4c009040).stop 2021-04-03T03:17:12.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 shutdown_connections 2021-04-03T03:17:12.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fde40056ef0 0x7fde400593b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fde5c159d20 0x7fde5c15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fde5c0aa5c0 0x7fde5c063580 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 --2- 172.21.2.111:0/1761947579 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fde5c063a70 0x7fde5c063ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 >> 172.21.2.111:0/1761947579 conn(0x7fde5c0b61c0 msgr2=0x7fde5c14ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 shutdown_connections 2021-04-03T03:17:12.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.442+0000 7fde61e4c700 1 -- 172.21.2.111:0/1761947579 wait complete. 2021-04-03T03:17:12.476 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:12.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.534+0000 7fab1e2b5700 1 Processor -- start 2021-04-03T03:17:12.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.534+0000 7fab1e2b5700 1 -- start start 2021-04-03T03:17:12.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab180603f0 0x7fab180b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fab1814b700 0x7fab180b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180607d0 con 0x7fab180b12b0 2021-04-03T03:17:12.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab180af400 con 0x7fab1814b700 2021-04-03T03:17:12.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180af6a0 con 0x7fab180603f0 2021-04-03T03:17:12.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1dab4700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fab180b12b0 0x7fab181465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:43158/0 (socket says 172.21.2.111:43158) 2021-04-03T03:17:12.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1dab4700 1 -- 172.21.2.111:0/489722501 learned_addr learned my addr 172.21.2.111:0/489722501 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:12.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1cab2700 1 --2- 172.21.2.111:0/489722501 >> v2:172.21.2.115:3300/0 conn(0x7fab1814b700 0x7fab180b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1d2b3700 1 --2- 172.21.2.111:0/489722501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab180603f0 0x7fab180b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1d2b3700 1 -- 172.21.2.111:0/489722501 >> v2:172.21.2.115:3300/0 conn(0x7fab1814b700 msgr2=0x7fab180b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:12.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1d2b3700 1 --2- 172.21.2.111:0/489722501 >> v2:172.21.2.115:3300/0 conn(0x7fab1814b700 0x7fab180b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1d2b3700 1 -- 172.21.2.111:0/489722501 >> v1:172.21.2.115:6789/0 conn(0x7fab180b12b0 legacy=0x7fab181465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1d2b3700 1 -- 172.21.2.111:0/489722501 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180afb40 con 0x7fab180603f0 2021-04-03T03:17:12.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1cab2700 1 --2- 172.21.2.111:0/489722501 >> v2:172.21.2.115:3300/0 conn(0x7fab1814b700 0x7fab180b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:17:12.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1d2b3700 1 --2- 172.21.2.111:0/489722501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab180603f0 0x7fab180b0090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fab10001450 tx=0x7fab0c006dd0).ready entity=mon.2 client_cookie=3524a9711f4e8779 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab0bfff700 1 -- 172.21.2.111:0/489722501 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c0095d0 con 0x7fab180603f0 2021-04-03T03:17:12.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab0bfff700 1 -- 172.21.2.111:0/489722501 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c009730 con 0x7fab180603f0 2021-04-03T03:17:12.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab0bfff700 1 -- 172.21.2.111:0/489722501 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c009a90 con 0x7fab180603f0 2021-04-03T03:17:12.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- 172.21.2.111:0/489722501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab180603f0 msgr2=0x7fab180b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/489722501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab180603f0 0x7fab180b0090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fab10001450 tx=0x7fab0c006dd0).stop 2021-04-03T03:17:12.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- 172.21.2.111:0/489722501 shutdown_connections 2021-04-03T03:17:12.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/489722501 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab180603f0 0x7fab180b0090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/489722501 >> v2:172.21.2.115:3300/0 conn(0x7fab1814b700 0x7fab180b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- 172.21.2.111:0/489722501 >> 172.21.2.111:0/489722501 conn(0x7fab180b61c0 msgr2=0x7fab1805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- 172.21.2.111:0/489722501 shutdown_connections 2021-04-03T03:17:12.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.538+0000 7fab1e2b5700 1 -- 172.21.2.111:0/489722501 wait complete. 2021-04-03T03:17:12.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 Processor -- start 2021-04-03T03:17:12.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 -- start start 2021-04-03T03:17:12.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab180603f0 0x7fab180aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab180b12b0 0x7fab18062a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab1814b700 0x7fab18067620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ab2f0 con 0x7fab180b12b0 2021-04-03T03:17:12.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18062f50 con 0x7fab180603f0 2021-04-03T03:17:12.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180a9fc0 con 0x7fab1814b700 2021-04-03T03:17:12.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab180b12b0 0x7fab18062a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab180b12b0 0x7fab18062a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58068/0 (socket says 172.21.2.111:58068) 2021-04-03T03:17:12.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 -- 172.21.2.111:0/1236495265 learned_addr learned my addr 172.21.2.111:0/1236495265 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:12.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 -- 172.21.2.111:0/1236495265 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab1814b700 msgr2=0x7fab18067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab1814b700 0x7fab18067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 -- 172.21.2.111:0/1236495265 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab180603f0 msgr2=0x7fab180aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab180603f0 0x7fab180aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 -- 172.21.2.111:0/1236495265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180aa3c0 con 0x7fab180b12b0 2021-04-03T03:17:12.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1cab2700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab180b12b0 0x7fab18062a60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7fab1400c000 tx=0x7fab14006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab0a7fc700 1 -- 172.21.2.111:0/1236495265 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1400cbf0 con 0x7fab180b12b0 2021-04-03T03:17:12.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab0a7fc700 1 -- 172.21.2.111:0/1236495265 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1400cd50 con 0x7fab180b12b0 2021-04-03T03:17:12.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab18064dd0 con 0x7fab180b12b0 2021-04-03T03:17:12.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab18067ca0 con 0x7fab180b12b0 2021-04-03T03:17:12.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab0a7fc700 1 -- 172.21.2.111:0/1236495265 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab140125e0 con 0x7fab180b12b0 2021-04-03T03:17:12.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab18057660 con 0x7fab180b12b0 2021-04-03T03:17:12.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab0a7fc700 1 -- 172.21.2.111:0/1236495265 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fab14012740 con 0x7fab180b12b0 2021-04-03T03:17:12.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab0a7fc700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faaf8056f70 0x7faaf8059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab0a7fc700 1 -- 172.21.2.111:0/1236495265 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fab14070680 con 0x7fab180b12b0 2021-04-03T03:17:12.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1d2b3700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faaf8056f70 0x7faaf8059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.542+0000 7fab1d2b3700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faaf8056f70 0x7faaf8059430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fab10001450 tx=0x7fab0c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.546+0000 7fab0a7fc700 1 -- 172.21.2.111:0/1236495265 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fab14045c90 con 0x7fab180b12b0 2021-04-03T03:17:12.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab180b1b20 con 0x7fab180b12b0 2021-04-03T03:17:12.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab0a7fc700 1 -- 172.21.2.111:0/1236495265 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+13424 (secure 0 0 0) 0x7fab140492b0 con 0x7fab180b12b0 2021-04-03T03:17:12.640 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:12.640 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":137,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:12.175018+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:12.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faaf8056f70 msgr2=0x7faaf8059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faaf8056f70 0x7faaf8059430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fab10001450 tx=0x7fab0c015000).stop 2021-04-03T03:17:12.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab180b12b0 msgr2=0x7fab18062a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab180b12b0 0x7fab18062a60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7fab1400c000 tx=0x7fab14006dd0).stop 2021-04-03T03:17:12.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 shutdown_connections 2021-04-03T03:17:12.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fab1814b700 0x7fab18067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faaf8056f70 0x7faaf8059430 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fab180603f0 0x7fab180aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:12.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 --2- 172.21.2.111:0/1236495265 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fab180b12b0 0x7fab18062a60 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 >> 172.21.2.111:0/1236495265 conn(0x7fab180b61c0 msgr2=0x7fab18148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 shutdown_connections 2021-04-03T03:17:12.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.638+0000 7fab1e2b5700 1 -- 172.21.2.111:0/1236495265 wait complete. 2021-04-03T03:17:12.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:17:12.648 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:17:12.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af3de7700 1 Processor -- start 2021-04-03T03:17:12.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af3de7700 1 -- start start 2021-04-03T03:17:12.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af3de7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3af40603f0 0x7f3af41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af3de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40b17b0 0x7f3af4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af3de7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af40607d0 con 0x7f3af40add50 2021-04-03T03:17:12.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af3de7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af41451c0 con 0x7f3af40603f0 2021-04-03T03:17:12.708 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af3de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af414b090 con 0x7f3af40b17b0 2021-04-03T03:17:12.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af35e6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40b17b0 0x7f3af4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af25e4700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f3af40add50 0x7f3af4147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:43172/0 (socket says 172.21.2.111:43172) 2021-04-03T03:17:12.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af25e4700 1 -- 172.21.2.111:0/1824624899 learned_addr learned my addr 172.21.2.111:0/1824624899 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:12.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af2de5700 1 --2- 172.21.2.111:0/1824624899 >> v2:172.21.2.115:3300/0 conn(0x7f3af40603f0 0x7f3af41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af1de3700 1 -- 172.21.2.111:0/1824624899 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79629902 0 0) 0x7f3af40607d0 con 0x7f3af40add50 2021-04-03T03:17:12.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.706+0000 7f3af1de3700 1 -- 172.21.2.111:0/1824624899 --> v1:172.21.2.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ad4003240 con 0x7f3af40add50 2021-04-03T03:17:12.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af35e6700 1 -- 172.21.2.111:0/1824624899 >> v2:172.21.2.115:3300/0 conn(0x7f3af40603f0 msgr2=0x7f3af41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:12.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af35e6700 1 --2- 172.21.2.111:0/1824624899 >> v2:172.21.2.115:3300/0 conn(0x7f3af40603f0 0x7f3af41447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af35e6700 1 -- 172.21.2.111:0/1824624899 >> v1:172.21.2.115:6789/0 conn(0x7f3af40add50 legacy=0x7f3af4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af35e6700 1 -- 172.21.2.111:0/1824624899 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af414b330 con 0x7f3af40b17b0 2021-04-03T03:17:12.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 --2- 172.21.2.111:0/1824624899 >> v2:172.21.2.115:3300/0 conn(0x7f3af40603f0 0x7f3af41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:17:12.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af35e6700 1 --2- 172.21.2.111:0/1824624899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40b17b0 0x7f3af4144cd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3aec0032b0 tx=0x7f3aec0091d0).ready entity=mon.2 client_cookie=a61a0a8cd6efa9ae server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af1de3700 1 -- 172.21.2.111:0/1824624899 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aec00b940 con 0x7f3af40b17b0 2021-04-03T03:17:12.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af1de3700 1 -- 172.21.2.111:0/1824624899 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec00baa0 con 0x7f3af40b17b0 2021-04-03T03:17:12.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af1de3700 1 -- 172.21.2.111:0/1824624899 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aec00bc10 con 0x7f3af40b17b0 2021-04-03T03:17:12.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- 172.21.2.111:0/1824624899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40b17b0 msgr2=0x7f3af4144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 --2- 172.21.2.111:0/1824624899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40b17b0 0x7f3af4144cd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3aec0032b0 tx=0x7f3aec0091d0).stop 2021-04-03T03:17:12.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- 172.21.2.111:0/1824624899 shutdown_connections 2021-04-03T03:17:12.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 --2- 172.21.2.111:0/1824624899 >> v2:172.21.2.115:3300/0 conn(0x7f3af40603f0 0x7f3af41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 --2- 172.21.2.111:0/1824624899 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40b17b0 0x7f3af4144cd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- 172.21.2.111:0/1824624899 >> 172.21.2.111:0/1824624899 conn(0x7f3af40b61c0 msgr2=0x7f3af405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- 172.21.2.111:0/1824624899 shutdown_connections 2021-04-03T03:17:12.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- 172.21.2.111:0/1824624899 wait complete. 2021-04-03T03:17:12.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 Processor -- start 2021-04-03T03:17:12.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- start start 2021-04-03T03:17:12.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3af40603f0 0x7f3af40aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40add50 0x7f3af4145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3af40b17b0 0x7f3af40ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40a7070 con 0x7f3af40603f0 2021-04-03T03:17:12.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4148200 con 0x7f3af40b17b0 2021-04-03T03:17:12.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41484a0 con 0x7f3af40add50 2021-04-03T03:17:12.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3af40603f0 0x7f3af40aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3af40603f0 0x7f3af40aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58080/0 (socket says 172.21.2.111:58080) 2021-04-03T03:17:12.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 -- 172.21.2.111:0/2491817679 learned_addr learned my addr 172.21.2.111:0/2491817679 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:12.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af25e4700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40add50 0x7f3af4145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af35e6700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3af40b17b0 0x7f3af40ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 -- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40add50 msgr2=0x7f3af4145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40add50 0x7f3af4145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 -- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3af40b17b0 msgr2=0x7f3af40ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3af40b17b0 0x7f3af40ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 -- 172.21.2.111:0/2491817679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4147dd0 con 0x7f3af40603f0 2021-04-03T03:17:12.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af2de5700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3af40603f0 0x7f3af40aa920 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f3adc00c000 tx=0x7f3adc006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3aebfff700 1 -- 172.21.2.111:0/2491817679 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3adc00cbf0 con 0x7f3af40603f0 2021-04-03T03:17:12.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3aebfff700 1 -- 172.21.2.111:0/2491817679 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc00cd50 con 0x7f3af40603f0 2021-04-03T03:17:12.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af4145800 con 0x7f3af40603f0 2021-04-03T03:17:12.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3aebfff700 1 -- 172.21.2.111:0/2491817679 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3adc0125e0 con 0x7f3af40603f0 2021-04-03T03:17:12.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.710+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af40abd80 con 0x7f3af40603f0 2021-04-03T03:17:12.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.714+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af4148660 con 0x7f3af40603f0 2021-04-03T03:17:12.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.714+0000 7f3aebfff700 1 -- 172.21.2.111:0/2491817679 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3adc012740 con 0x7f3af40603f0 2021-04-03T03:17:12.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.714+0000 7f3aebfff700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3ad8056fb0 0x7f3ad8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.714+0000 7f3aebfff700 1 -- 172.21.2.111:0/2491817679 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f3adc0705d0 con 0x7f3af40603f0 2021-04-03T03:17:12.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.714+0000 7f3af25e4700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3ad8056fb0 0x7f3ad8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.714+0000 7f3af25e4700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3ad8056fb0 0x7f3ad8059470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3ae400d000 tx=0x7f3ae4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.714+0000 7f3aebfff700 1 -- 172.21.2.111:0/2491817679 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3adc045bf0 con 0x7f3af40603f0 2021-04-03T03:17:12.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.806+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3af4057660 con 0x7f3af40603f0 2021-04-03T03:17:12.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3aebfff700 1 -- 172.21.2.111:0/2491817679 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3adc049210 con 0x7f3af40603f0 2021-04-03T03:17:12.810 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:17:12.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3ad8056fb0 msgr2=0x7f3ad8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3ad8056fb0 0x7f3ad8059470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3ae400d000 tx=0x7f3ae4009040).stop 2021-04-03T03:17:12.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3af40603f0 msgr2=0x7f3af40aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3af40603f0 0x7f3af40aa920 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f3adc00c000 tx=0x7f3adc006dd0).stop 2021-04-03T03:17:12.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 shutdown_connections 2021-04-03T03:17:12.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3ad8056fb0 0x7f3ad8059470 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3af40b17b0 0x7f3af40ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3af40603f0 0x7f3af40aa920 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 --2- 172.21.2.111:0/2491817679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3af40add50 0x7f3af4145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 >> 172.21.2.111:0/2491817679 conn(0x7f3af40b61c0 msgr2=0x7f3af40b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 shutdown_connections 2021-04-03T03:17:12.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.810+0000 7f3af3de7700 1 -- 172.21.2.111:0/2491817679 wait complete. 2021-04-03T03:17:12.818 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:17:12.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 Processor -- start 2021-04-03T03:17:12.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- start start 2021-04-03T03:17:12.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f36b00603f0 0x7f36b01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f36b00b17b0 0x7f36b0147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b01451c0 con 0x7f36b00603f0 2021-04-03T03:17:12.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b00607d0 con 0x7f36b00b17b0 2021-04-03T03:17:12.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b014b090 con 0x7f36b00add50 2021-04-03T03:17:12.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f36b00603f0 0x7f36b01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f36b00603f0 0x7f36b01447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58088/0 (socket says 172.21.2.111:58088) 2021-04-03T03:17:12.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 -- 172.21.2.111:0/681027897 learned_addr learned my addr 172.21.2.111:0/681027897 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:12.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b59e0700 1 --2- 172.21.2.111:0/681027897 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b69e2700 1 --2- 172.21.2.111:0/681027897 >> v2:172.21.2.115:3300/0 conn(0x7f36b00b17b0 0x7f36b0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 -- 172.21.2.111:0/681027897 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 msgr2=0x7f36b0144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:12.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 --2- 172.21.2.111:0/681027897 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b0144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 -- 172.21.2.111:0/681027897 >> v2:172.21.2.115:3300/0 conn(0x7f36b00b17b0 msgr2=0x7f36b0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:12.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 --2- 172.21.2.111:0/681027897 >> v2:172.21.2.115:3300/0 conn(0x7f36b00b17b0 0x7f36b0147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 -- 172.21.2.111:0/681027897 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b014b330 con 0x7f36b00603f0 2021-04-03T03:17:12.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b61e1700 1 --2- 172.21.2.111:0/681027897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00603f0 0x7f36b01447e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f36a80032b0 tx=0x7f36a8009270).ready entity=mon.0 client_cookie=d1bd7f81190945eb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b51df700 1 -- 172.21.2.111:0/681027897 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a8009e60 con 0x7f36b00603f0 2021-04-03T03:17:12.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b51df700 1 -- 172.21.2.111:0/681027897 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a800bc00 con 0x7f36b00603f0 2021-04-03T03:17:12.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b51df700 1 -- 172.21.2.111:0/681027897 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a800bd70 con 0x7f36b00603f0 2021-04-03T03:17:12.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- 172.21.2.111:0/681027897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00603f0 msgr2=0x7f36b01447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 --2- 172.21.2.111:0/681027897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00603f0 0x7f36b01447e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f36a80032b0 tx=0x7f36a8009270).stop 2021-04-03T03:17:12.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- 172.21.2.111:0/681027897 shutdown_connections 2021-04-03T03:17:12.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 --2- 172.21.2.111:0/681027897 >> v2:172.21.2.115:3300/0 conn(0x7f36b00b17b0 0x7f36b0147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 --2- 172.21.2.111:0/681027897 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00603f0 0x7f36b01447e0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 --2- 172.21.2.111:0/681027897 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b0144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- 172.21.2.111:0/681027897 >> 172.21.2.111:0/681027897 conn(0x7f36b00b61c0 msgr2=0x7f36b005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- 172.21.2.111:0/681027897 shutdown_connections 2021-04-03T03:17:12.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- 172.21.2.111:0/681027897 wait complete. 2021-04-03T03:17:12.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 Processor -- start 2021-04-03T03:17:12.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.878+0000 7f36b71e3700 1 -- start start 2021-04-03T03:17:12.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36b00603f0 0x7f36b0147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00b17b0 0x7f36b00b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b014c5c0 con 0x7f36b00b17b0 2021-04-03T03:17:12.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b01468b0 con 0x7f36b00603f0 2021-04-03T03:17:12.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0146b50 con 0x7f36b00add50 2021-04-03T03:17:12.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00b17b0 0x7f36b00b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00b17b0 0x7f36b00b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58096/0 (socket says 172.21.2.111:58096) 2021-04-03T03:17:12.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 -- 172.21.2.111:0/2105473900 learned_addr learned my addr 172.21.2.111:0/2105473900 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:12.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b59e0700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b61e1700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36b00603f0 0x7f36b0147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 -- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 msgr2=0x7f36b014e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b014e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 -- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36b00603f0 msgr2=0x7f36b0147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36b00603f0 0x7f36b0147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 -- 172.21.2.111:0/2105473900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0146df0 con 0x7f36b00b17b0 2021-04-03T03:17:12.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b69e2700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00b17b0 0x7f36b00b3520 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f36ac00a000 tx=0x7f36ac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f369f7fe700 1 -- 172.21.2.111:0/2105473900 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36ac011780 con 0x7f36b00b17b0 2021-04-03T03:17:12.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f369f7fe700 1 -- 172.21.2.111:0/2105473900 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac0118e0 con 0x7f36b00b17b0 2021-04-03T03:17:12.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b01481b0 con 0x7f36b00b17b0 2021-04-03T03:17:12.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f369f7fe700 1 -- 172.21.2.111:0/2105473900 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36ac011c40 con 0x7f36b00b17b0 2021-04-03T03:17:12.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b014bf60 con 0x7f36b00b17b0 2021-04-03T03:17:12.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0146f80 con 0x7f36b00b17b0 2021-04-03T03:17:12.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f369f7fe700 1 -- 172.21.2.111:0/2105473900 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f36ac01c460 con 0x7f36b00b17b0 2021-04-03T03:17:12.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f369f7fe700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3690057050 0x7f3690059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:12.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f369f7fe700 1 -- 172.21.2.111:0/2105473900 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f36ac070430 con 0x7f36b00b17b0 2021-04-03T03:17:12.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b61e1700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3690057050 0x7f3690059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:12.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.882+0000 7f36b61e1700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3690057050 0x7f3690059510 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f36a8018000 tx=0x7f36a8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:12.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.886+0000 7f369f7fe700 1 -- 172.21.2.111:0/2105473900 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f36ac0459e0 con 0x7f36b00b17b0 2021-04-03T03:17:12.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36b0057660 con 0x7f36b00b17b0 2021-04-03T03:17:12.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f369f7fe700 1 -- 172.21.2.111:0/2105473900 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f36ac049000 con 0x7f36b00b17b0 2021-04-03T03:17:12.982 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:12.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3690057050 msgr2=0x7f3690059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3690057050 0x7f3690059510 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f36a8018000 tx=0x7f36a8016000).stop 2021-04-03T03:17:12.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00b17b0 msgr2=0x7f36b00b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:12.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00b17b0 0x7f36b00b3520 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f36ac00a000 tx=0x7f36ac008040).stop 2021-04-03T03:17:12.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 shutdown_connections 2021-04-03T03:17:12.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f36b00b17b0 0x7f36b00b3520 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3690057050 0x7f3690059510 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f36b00603f0 0x7f36b0147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 --2- 172.21.2.111:0/2105473900 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f36b00add50 0x7f36b014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:12.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 >> 172.21.2.111:0/2105473900 conn(0x7f36b00b61c0 msgr2=0x7f36b0143620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:12.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 shutdown_connections 2021-04-03T03:17:12.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:12.982+0000 7f36b71e3700 1 -- 172.21.2.111:0/2105473900 wait complete. 2021-04-03T03:17:12.990 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:17:13.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 Processor -- start 2021-04-03T03:17:13.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 -- start start 2021-04-03T03:17:13.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c14b700 0x7fca0c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0c0607d0 con 0x7fca0c0b0c90 2021-04-03T03:17:13.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0c0af400 con 0x7fca0c0603f0 2021-04-03T03:17:13.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c0af6a0 con 0x7fca0c14b700 2021-04-03T03:17:13.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca111a7700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fca0c0b0c90 0x7fca0c147680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58896/0 (socket says 172.21.2.111:58896) 2021-04-03T03:17:13.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca111a7700 1 -- 172.21.2.111:0/186172318 learned_addr learned my addr 172.21.2.111:0/186172318 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:13.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c14b700 0x7fca0c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:13.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141456781 0 0) 0x7fca0c0607d0 con 0x7fca0c0b0c90 2021-04-03T03:17:13.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ec003240 con 0x7fca0c0b0c90 2021-04-03T03:17:13.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3636508474 0 0) 0x7fc9ec003240 con 0x7fca0c0b0c90 2021-04-03T03:17:13.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c14b700 msgr2=0x7fca0c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:13.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c14b700 0x7fca0c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:13.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 >> v1:172.21.2.115:6789/0 conn(0x7fca0c0603f0 legacy=0x7fca0c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c0afb40 con 0x7fca0c0b0c90 2021-04-03T03:17:13.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fca04001a50 con 0x7fca0c0b0c90 2021-04-03T03:17:13.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 >> v1:172.21.2.111:6789/0 conn(0x7fca0c0b0c90 legacy=0x7fca0c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c14b700 0x7fc9ec003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc9ec003cc0 0x7fc9ec0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc9ec0049d0 0x7fc9ec004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c0af6a0 con 0x7fc9ec003cc0 2021-04-03T03:17:13.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec004250 con 0x7fc9ec0049d0 2021-04-03T03:17:13.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec00b730 con 0x7fca0c14b700 2021-04-03T03:17:13.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc9ec003cc0 0x7fc9ec0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:13.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 -- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c14b700 msgr2=0x7fc9ec003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:13.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c14b700 0x7fc9ec003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 -- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc9ec0049d0 msgr2=0x7fc9ec004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:13.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc9ec0049d0 0x7fc9ec004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 -- 172.21.2.111:0/186172318 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec003240 con 0x7fc9ec003cc0 2021-04-03T03:17:13.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0bfff700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc9ec003cc0 0x7fc9ec0044e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7fc9f400c000 tx=0x7fc9f4008040).ready entity=mon.0 client_cookie=98bcc817c0f88727 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:13.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f400eb60 con 0x7fc9ec003cc0 2021-04-03T03:17:13.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f400ecc0 con 0x7fc9ec003cc0 2021-04-03T03:17:13.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca0b7fe700 1 -- 172.21.2.111:0/186172318 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f40125e0 con 0x7fc9ec003cc0 2021-04-03T03:17:13.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 -- 172.21.2.111:0/186172318 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc9ec003cc0 msgr2=0x7fc9ec0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca119a8700 1 --2- 172.21.2.111:0/186172318 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc9ec003cc0 0x7fc9ec0044e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7fc9f400c000 tx=0x7fc9f4008040).stop 2021-04-03T03:17:13.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.046+0000 7fca111a7700 1 -- 172.21.2.111:0/186172318 reap_dead start 2021-04-03T03:17:13.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- 172.21.2.111:0/186172318 shutdown_connections 2021-04-03T03:17:13.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- 172.21.2.111:0/186172318 >> 172.21.2.111:0/186172318 conn(0x7fca0c0b61c0 msgr2=0x7fca0c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:13.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- 172.21.2.111:0/186172318 shutdown_connections 2021-04-03T03:17:13.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- 172.21.2.111:0/186172318 wait complete. 2021-04-03T03:17:13.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 Processor -- start 2021-04-03T03:17:13.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- start start 2021-04-03T03:17:13.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca0c0ab640 0x7fca0c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c0655e0 0x7fca0c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0c067af0 0x7fca0c062f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c156db0 con 0x7fca0c0ab640 2021-04-03T03:17:13.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c157020 con 0x7fca0c067af0 2021-04-03T03:17:13.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1572c0 con 0x7fca0c0655e0 2021-04-03T03:17:13.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca0c0ab640 0x7fca0c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:13.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca0c0ab640 0x7fca0c1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58114/0 (socket says 172.21.2.111:58114) 2021-04-03T03:17:13.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 -- 172.21.2.111:0/23005206 learned_addr learned my addr 172.21.2.111:0/23005206 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:13.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 -- 172.21.2.111:0/23005206 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c0655e0 msgr2=0x7fca0c067600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c0655e0 0x7fca0c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 -- 172.21.2.111:0/23005206 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0c067af0 msgr2=0x7fca0c062f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0c067af0 0x7fca0c062f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 -- 172.21.2.111:0/23005206 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c067f80 con 0x7fca0c0ab640 2021-04-03T03:17:13.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca109a6700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca0c0ab640 0x7fca0c1568c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fca0c0b5640 tx=0x7fc9fc007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:13.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca09ffb700 1 -- 172.21.2.111:0/23005206 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9fc00e4a0 con 0x7fca0c0ab640 2021-04-03T03:17:13.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c157a00 con 0x7fca0c0ab640 2021-04-03T03:17:13.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca09ffb700 1 -- 172.21.2.111:0/23005206 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc00e600 con 0x7fca0c0ab640 2021-04-03T03:17:13.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c0abbb0 con 0x7fca0c0ab640 2021-04-03T03:17:13.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca09ffb700 1 -- 172.21.2.111:0/23005206 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9fc00e960 con 0x7fca0c0ab640 2021-04-03T03:17:13.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c057660 con 0x7fca0c0ab640 2021-04-03T03:17:13.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca09ffb700 1 -- 172.21.2.111:0/23005206 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc9fc010030 con 0x7fca0c0ab640 2021-04-03T03:17:13.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca09ffb700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9f0056ea0 0x7fc9f0059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca09ffb700 1 -- 172.21.2.111:0/23005206 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fc9fc06fd80 con 0x7fca0c0ab640 2021-04-03T03:17:13.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca0bfff700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9f0056ea0 0x7fc9f0059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:13.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.050+0000 7fca0bfff700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9f0056ea0 0x7fc9f0059360 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fc9f400c000 tx=0x7fc9f4019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:13.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.054+0000 7fca09ffb700 1 -- 172.21.2.111:0/23005206 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc9fc045400 con 0x7fca0c0ab640 2021-04-03T03:17:13.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fca0c05fdf0 con 0x7fca0c0ab640 2021-04-03T03:17:13.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca09ffb700 1 -- 172.21.2.111:0/23005206 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc9fc071070 con 0x7fca0c0ab640 2021-04-03T03:17:13.152 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:13.152 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:17:13.152 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:13.153 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:17:13.153 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:17:13.153 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:13.153 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:13.153 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:17:13.154 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:17:13.154 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:13.154 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:17:13.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9f0056ea0 msgr2=0x7fc9f0059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9f0056ea0 0x7fc9f0059360 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fc9f400c000 tx=0x7fc9f4019000).stop 2021-04-03T03:17:13.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca0c0ab640 msgr2=0x7fca0c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca0c0ab640 0x7fca0c1568c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fca0c0b5640 tx=0x7fc9fc007bb0).stop 2021-04-03T03:17:13.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 shutdown_connections 2021-04-03T03:17:13.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9f0056ea0 0x7fc9f0059360 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:13.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0c067af0 0x7fca0c062f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca0c0ab640 0x7fca0c1568c0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:13.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 --2- 172.21.2.111:0/23005206 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca0c0655e0 0x7fca0c067600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 >> 172.21.2.111:0/23005206 conn(0x7fca0c0b61c0 msgr2=0x7fca0c14de00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:13.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.150+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 shutdown_connections 2021-04-03T03:17:13.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.154+0000 7fca119a8700 1 -- 172.21.2.111:0/23005206 wait complete. 2021-04-03T03:17:13.160 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:17:13.160 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:17:13.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 Processor -- start 2021-04-03T03:17:13.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- start start 2021-04-03T03:17:13.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efc0c0dbba0 0x7efc0c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c03d560 0x7efc0c041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0c0dc560 con 0x7efc0c041060 2021-04-03T03:17:13.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0c03de60 con 0x7efc0c0dbba0 2021-04-03T03:17:13.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c0dc800 con 0x7efc0c03d560 2021-04-03T03:17:13.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efc0c0dbba0 0x7efc0c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:13.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efc0c0dbba0 0x7efc0c0dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58122/0 (socket says 172.21.2.111:58122) 2021-04-03T03:17:13.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 -- 172.21.2.111:0/1565232517 learned_addr learned my addr 172.21.2.111:0/1565232517 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:13.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 -- 172.21.2.111:0/1565232517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c03d560 msgr2=0x7efc0c041650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:13.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 --2- 172.21.2.111:0/1565232517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c03d560 0x7efc0c041650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 -- 172.21.2.111:0/1565232517 >> v1:172.21.2.115:6789/0 conn(0x7efc0c041060 legacy=0x7efc0c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:13.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 -- 172.21.2.111:0/1565232517 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c03da30 con 0x7efc0c0dbba0 2021-04-03T03:17:13.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc12ffd700 1 --2- 172.21.2.111:0/1565232517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c0dbf80 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7efc0c0087d0 tx=0x7efc08006e90).ready entity=mon.0 client_cookie=5b2ded6b5bc607b3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:13.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc11ffb700 1 -- 172.21.2.111:0/1565232517 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc080098e0 con 0x7efc0c0dbba0 2021-04-03T03:17:13.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc11ffb700 1 -- 172.21.2.111:0/1565232517 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc08009a40 con 0x7efc0c0dbba0 2021-04-03T03:17:13.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc11ffb700 1 -- 172.21.2.111:0/1565232517 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc08009de0 con 0x7efc0c0dbba0 2021-04-03T03:17:13.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- 172.21.2.111:0/1565232517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 msgr2=0x7efc0c0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- 172.21.2.111:0/1565232517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c0dbf80 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7efc0c0087d0 tx=0x7efc08006e90).stop 2021-04-03T03:17:13.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- 172.21.2.111:0/1565232517 shutdown_connections 2021-04-03T03:17:13.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- 172.21.2.111:0/1565232517 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c03d560 0x7efc0c041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- 172.21.2.111:0/1565232517 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c0dbf80 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:13.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- 172.21.2.111:0/1565232517 >> 172.21.2.111:0/1565232517 conn(0x7efc0c0462b0 msgr2=0x7efc0c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:13.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- 172.21.2.111:0/1565232517 shutdown_connections 2021-04-03T03:17:13.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- 172.21.2.111:0/1565232517 wait complete. 2021-04-03T03:17:13.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 Processor -- start 2021-04-03T03:17:13.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- start start 2021-04-03T03:17:13.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc0c03d560 0x7efc0c0e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c041060 0x7efc0c0dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c09f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c0eb4a0 con 0x7efc0c0dbba0 2021-04-03T03:17:13.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c0eb740 con 0x7efc0c03d560 2021-04-03T03:17:13.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc13fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c03bed0 con 0x7efc0c041060 2021-04-03T03:17:13.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc137fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c09f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:13.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc137fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c09f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58128/0 (socket says 172.21.2.111:58128) 2021-04-03T03:17:13.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.218+0000 7efc137fe700 1 -- 172.21.2.111:0/2209096174 learned_addr learned my addr 172.21.2.111:0/2209096174 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:13.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc137fe700 1 -- 172.21.2.111:0/2209096174 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c041060 msgr2=0x7efc0c0dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc137fe700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c041060 0x7efc0c0dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc137fe700 1 -- 172.21.2.111:0/2209096174 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc0c03d560 msgr2=0x7efc0c0e73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:13.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc137fe700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc0c03d560 0x7efc0c0e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:13.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc137fe700 1 -- 172.21.2.111:0/2209096174 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c03c370 con 0x7efc0c0dbba0 2021-04-03T03:17:13.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc137fe700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c09f4a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7efc1406b5d0 tx=0x7efc1405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:13.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc03fff700 1 -- 172.21.2.111:0/2209096174 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc1405df60 con 0x7efc0c0dbba0 2021-04-03T03:17:13.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc03fff700 1 -- 172.21.2.111:0/2209096174 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc140699f0 con 0x7efc0c0dbba0 2021-04-03T03:17:13.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc03fff700 1 -- 172.21.2.111:0/2209096174 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc1405f4b0 con 0x7efc0c0dbba0 2021-04-03T03:17:13.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c03ad60 con 0x7efc0c0dbba0 2021-04-03T03:17:13.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c09f990 con 0x7efc0c0dbba0 2021-04-03T03:17:13.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0c03c090 con 0x7efc0c0dbba0 2021-04-03T03:17:13.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc03fff700 1 -- 172.21.2.111:0/2209096174 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7efc1408b020 con 0x7efc0c0dbba0 2021-04-03T03:17:13.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc03fff700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efbf00570a0 0x7efbf0059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:13.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc03fff700 1 -- 172.21.2.111:0/2209096174 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5596+0+0 (secure 0 0 0) 0x7efc140deda0 con 0x7efc0c0dbba0 2021-04-03T03:17:13.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc12ffd700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efbf00570a0 0x7efbf0059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:13.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc12ffd700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efbf00570a0 0x7efbf0059560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7efc0c0087d0 tx=0x7efc08015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:13.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.222+0000 7efc03fff700 1 -- 172.21.2.111:0/2209096174 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7efc140b43b0 con 0x7efc0c0dbba0 2021-04-03T03:17:13.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:13.314+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7efc0c0eb2d0 con 0x7efc0c0dbba0 2021-04-03T03:17:14.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc03fff700 1 -- 172.21.2.111:0/2209096174 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v138) v1 ==== 133+0+0 (secure 0 0 0) 0x7efc140b79d0 con 0x7efc0c0dbba0 2021-04-03T03:17:14.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efbf00570a0 msgr2=0x7efbf0059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:14.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efbf00570a0 0x7efbf0059560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7efc0c0087d0 tx=0x7efc08015000).stop 2021-04-03T03:17:14.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 msgr2=0x7efc0c09f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:14.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c09f4a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7efc1406b5d0 tx=0x7efc1405ad70).stop 2021-04-03T03:17:14.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 shutdown_connections 2021-04-03T03:17:14.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efc0c0dbba0 0x7efc0c09f4a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efbf00570a0 0x7efbf0059560 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efc0c03d560 0x7efc0c0e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:14.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 --2- 172.21.2.111:0/2209096174 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efc0c041060 0x7efc0c0dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:14.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 >> 172.21.2.111:0/2209096174 conn(0x7efc0c0462b0 msgr2=0x7efc0c045460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:14.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 shutdown_connections 2021-04-03T03:17:14.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.094+0000 7efc13fff700 1 -- 172.21.2.111:0/2209096174 wait complete. 2021-04-03T03:17:14.099 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:17:14.103 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:17:14.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 Processor -- start 2021-04-03T03:17:14.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 -- start start 2021-04-03T03:17:14.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f40780dbba0 0x7f40780dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:14.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4078041060 0x7f407803fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407803d560 0x7f4078041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407803de60 con 0x7f40780dbba0 2021-04-03T03:17:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780dc560 con 0x7f4078041060 2021-04-03T03:17:14.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40780dc800 con 0x7f407803d560 2021-04-03T03:17:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f40780dbba0 0x7f40780dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407effd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f40780dbba0 0x7f40780dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58136/0 (socket says 172.21.2.111:58136) 2021-04-03T03:17:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.158+0000 7f407effd700 1 -- 172.21.2.111:0/588626961 learned_addr learned my addr 172.21.2.111:0/588626961 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:14.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 --2- 172.21.2.111:0/588626961 >> v2:172.21.2.115:3300/0 conn(0x7f4078041060 0x7f407803fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407f7fe700 1 --2- 172.21.2.111:0/588626961 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407803d560 0x7f4078041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407effd700 1 -- 172.21.2.111:0/588626961 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407803d560 msgr2=0x7f4078041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407effd700 1 --2- 172.21.2.111:0/588626961 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407803d560 0x7f4078041650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407effd700 1 -- 172.21.2.111:0/588626961 >> v2:172.21.2.115:3300/0 conn(0x7f4078041060 msgr2=0x7f407803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:14.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407effd700 1 --2- 172.21.2.111:0/588626961 >> v2:172.21.2.115:3300/0 conn(0x7f4078041060 0x7f407803fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407effd700 1 -- 172.21.2.111:0/588626961 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407803da30 con 0x7f40780dbba0 2021-04-03T03:17:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407effd700 1 --2- 172.21.2.111:0/588626961 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40780dbba0 0x7f40780dbf80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f4068002f30 tx=0x7f4068006e90).ready entity=mon.0 client_cookie=edcef412a4bbcaea server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407dffb700 1 -- 172.21.2.111:0/588626961 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4068009920 con 0x7f40780dbba0 2021-04-03T03:17:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407dffb700 1 -- 172.21.2.111:0/588626961 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068009a80 con 0x7f40780dbba0 2021-04-03T03:17:14.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407dffb700 1 -- 172.21.2.111:0/588626961 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4068009de0 con 0x7f40780dbba0 2021-04-03T03:17:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/588626961 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40780dbba0 msgr2=0x7f40780dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 --2- 172.21.2.111:0/588626961 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40780dbba0 0x7f40780dbf80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f4068002f30 tx=0x7f4068006e90).stop 2021-04-03T03:17:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/588626961 shutdown_connections 2021-04-03T03:17:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 --2- 172.21.2.111:0/588626961 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f407803d560 0x7f4078041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 --2- 172.21.2.111:0/588626961 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40780dbba0 0x7f40780dbf80 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 --2- 172.21.2.111:0/588626961 >> v2:172.21.2.115:3300/0 conn(0x7f4078041060 0x7f407803fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/588626961 >> 172.21.2.111:0/588626961 conn(0x7f40780462b0 msgr2=0x7f40780466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/588626961 shutdown_connections 2021-04-03T03:17:14.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/588626961 wait complete. 2021-04-03T03:17:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 Processor -- start 2021-04-03T03:17:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- start start 2021-04-03T03:17:14.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f407803d560 0x7f40780a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4078041060 0x7f407803af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40780dbba0 0x7f40780d7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780df420 con 0x7f4078041060 2021-04-03T03:17:14.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780a20e0 con 0x7f407803d560 2021-04-03T03:17:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40780de0b0 con 0x7f40780dbba0 2021-04-03T03:17:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4078041060 0x7f407803af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4078041060 0x7f407803af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58144/0 (socket says 172.21.2.111:58144) 2021-04-03T03:17:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 -- 172.21.2.111:0/3738901427 learned_addr learned my addr 172.21.2.111:0/3738901427 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:14.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407effd700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f407803d560 0x7f40780a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407f7fe700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40780dbba0 0x7f40780d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 -- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40780dbba0 msgr2=0x7f40780d7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40780dbba0 0x7f40780d7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 -- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f407803d560 msgr2=0x7f40780a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:14.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f407803d560 0x7f40780a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 -- 172.21.2.111:0/3738901427 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780de2e0 con 0x7f4078041060 2021-04-03T03:17:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407e7fc700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4078041060 0x7f407803af20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f407803bc30 tx=0x7f4070008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f4077fff700 1 -- 172.21.2.111:0/3738901427 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4070009980 con 0x7f4078041060 2021-04-03T03:17:14.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f4077fff700 1 -- 172.21.2.111:0/3738901427 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4070009ae0 con 0x7f4078041060 2021-04-03T03:17:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40780de580 con 0x7f4078041060 2021-04-03T03:17:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f4077fff700 1 -- 172.21.2.111:0/3738901427 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4070009e40 con 0x7f4078041060 2021-04-03T03:17:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407809f5c0 con 0x7f4078041060 2021-04-03T03:17:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.162+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40780a0de0 con 0x7f4078041060 2021-04-03T03:17:14.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.166+0000 7f4077fff700 1 -- 172.21.2.111:0/3738901427 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4070013480 con 0x7f4078041060 2021-04-03T03:17:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.166+0000 7f4077fff700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4064057000 0x7f40640594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.166+0000 7f4077fff700 1 -- 172.21.2.111:0/3738901427 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5950+0+0 (secure 0 0 0) 0x7f40700707c0 con 0x7f4078041060 2021-04-03T03:17:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.166+0000 7f407effd700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4064057000 0x7f40640594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:14.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.166+0000 7f407effd700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4064057000 0x7f40640594c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f406801a000 tx=0x7f4068015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:14.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.166+0000 7f4077fff700 1 -- 172.21.2.111:0/3738901427 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f40780a0de0 con 0x7f4078041060 2021-04-03T03:17:14.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:14.262+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f40780a0de0 con 0x7f4078041060 2021-04-03T03:17:15.106 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f4077fff700 1 -- 172.21.2.111:0/3738901427 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v139) v1 ==== 125+0+0 (secure 0 0 0) 0x7f40780a0de0 con 0x7f4078041060 2021-04-03T03:17:15.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4064057000 msgr2=0x7f40640594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4064057000 0x7f40640594c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f406801a000 tx=0x7f4068015000).stop 2021-04-03T03:17:15.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4078041060 msgr2=0x7f407803af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4078041060 0x7f407803af20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f407803bc30 tx=0x7f4070008040).stop 2021-04-03T03:17:15.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 shutdown_connections 2021-04-03T03:17:15.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40780dbba0 0x7f40780d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4064057000 0x7f40640594c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f407803d560 0x7f40780a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 --2- 172.21.2.111:0/3738901427 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4078041060 0x7f407803af20 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 >> 172.21.2.111:0/3738901427 conn(0x7f40780462b0 msgr2=0x7f4078045a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 shutdown_connections 2021-04-03T03:17:15.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.106+0000 7f407ffff700 1 -- 172.21.2.111:0/3738901427 wait complete. 2021-04-03T03:17:15.111 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:17:15.115 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:17:15.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 Processor -- start 2021-04-03T03:17:15.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- start start 2021-04-03T03:17:15.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ed0048ef0 0x7f7ed00d55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7ed00d5f10 0x7f7ed0005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed0043120 con 0x7f7ed00d5a90 2021-04-03T03:17:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00492d0 con 0x7f7ed0048ef0 2021-04-03T03:17:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00446e0 con 0x7f7ed00d5f10 2021-04-03T03:17:15.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ed0048ef0 0x7f7ed00d55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7ed0048ef0 0x7f7ed00d55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58150/0 (socket says 172.21.2.111:58150) 2021-04-03T03:17:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2802250268 learned_addr learned my addr 172.21.2.111:0/2802250268 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2802250268 >> v2:172.21.2.115:3300/0 conn(0x7f7ed00d5f10 msgr2=0x7f7ed0005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:15.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- 172.21.2.111:0/2802250268 >> v2:172.21.2.115:3300/0 conn(0x7f7ed00d5f10 0x7f7ed0005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2802250268 >> v1:172.21.2.115:6789/0 conn(0x7f7ed00d5a90 legacy=0x7f7ed0005680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2802250268 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0044900 con 0x7f7ed0048ef0 2021-04-03T03:17:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- 172.21.2.111:0/2802250268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d55a0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f7ed0049520 tx=0x7f7ec8006e90).ready entity=mon.0 client_cookie=27f571782603e0f9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/2802250268 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec8009930 con 0x7f7ed0048ef0 2021-04-03T03:17:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/2802250268 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec8009a90 con 0x7f7ed0048ef0 2021-04-03T03:17:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed6ffd700 1 -- 172.21.2.111:0/2802250268 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec8009e30 con 0x7f7ed0048ef0 2021-04-03T03:17:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- 172.21.2.111:0/2802250268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 msgr2=0x7f7ed00d55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2802250268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d55a0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f7ed0049520 tx=0x7f7ec8006e90).stop 2021-04-03T03:17:15.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- 172.21.2.111:0/2802250268 shutdown_connections 2021-04-03T03:17:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2802250268 >> v2:172.21.2.115:3300/0 conn(0x7f7ed00d5f10 0x7f7ed0005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2802250268 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d55a0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- 172.21.2.111:0/2802250268 >> 172.21.2.111:0/2802250268 conn(0x7f7ed000f2d0 msgr2=0x7f7ed00484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.180 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- 172.21.2.111:0/2802250268 shutdown_connections 2021-04-03T03:17:15.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- 172.21.2.111:0/2802250268 wait complete. 2021-04-03T03:17:15.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 Processor -- start 2021-04-03T03:17:15.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- start start 2021-04-03T03:17:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d5a90 0x7f7ed00cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00d5f10 0x7f7ed00cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed003cf60 con 0x7f7ed0048ef0 2021-04-03T03:17:15.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed003d200 con 0x7f7ed00d5a90 2021-04-03T03:17:15.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed003d4a0 con 0x7f7ed00d5f10 2021-04-03T03:17:15.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58156/0 (socket says 172.21.2.111:58156) 2021-04-03T03:17:15.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2243655922 learned_addr learned my addr 172.21.2.111:0/2243655922 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2243655922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00d5f10 msgr2=0x7f7ed00cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00d5f10 0x7f7ed00cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2243655922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d5a90 msgr2=0x7f7ed00cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d5a90 0x7f7ed00cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 -- 172.21.2.111:0/2243655922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed00d49d0 con 0x7f7ed0048ef0 2021-04-03T03:17:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed7fff700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d44e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f7ed0049520 tx=0x7f7ec8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed57fa700 1 -- 172.21.2.111:0/2243655922 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec8011d30 con 0x7f7ed0048ef0 2021-04-03T03:17:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed57fa700 1 -- 172.21.2.111:0/2243655922 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec8011e90 con 0x7f7ed0048ef0 2021-04-03T03:17:15.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed00d4cd0 con 0x7f7ed0048ef0 2021-04-03T03:17:15.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7ed57fa700 1 -- 172.21.2.111:0/2243655922 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec801e650 con 0x7f7ed0048ef0 2021-04-03T03:17:15.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.174+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed00e0eb0 con 0x7f7ed0048ef0 2021-04-03T03:17:15.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.178+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed0004db0 con 0x7f7ed0048ef0 2021-04-03T03:17:15.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.178+0000 7f7ed57fa700 1 -- 172.21.2.111:0/2243655922 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7ec8025030 con 0x7f7ed0048ef0 2021-04-03T03:17:15.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.178+0000 7f7ed57fa700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7eb8056f20 0x7f7eb80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.178+0000 7f7ed57fa700 1 -- 172.21.2.111:0/2243655922 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f7ec8071760 con 0x7f7ed0048ef0 2021-04-03T03:17:15.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.178+0000 7f7ed77fe700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7eb8056f20 0x7f7eb80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.178+0000 7f7ed77fe700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7eb8056f20 0x7f7eb80593e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f7ecc002f50 tx=0x7f7ecc006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.178+0000 7f7ed57fa700 1 -- 172.21.2.111:0/2243655922 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7ec8046ab0 con 0x7f7ed0048ef0 2021-04-03T03:17:15.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.306+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f7ed003cd40 con 0x7f7ed0048ef0 2021-04-03T03:17:15.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7ed57fa700 1 -- 172.21.2.111:0/2243655922 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v62) v1 ==== 189+0+0 (secure 0 0 0) 0x7f7ec80230d0 con 0x7f7ed0048ef0 2021-04-03T03:17:15.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7eb8056f20 msgr2=0x7f7eb80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7eb8056f20 0x7f7eb80593e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f7ecc002f50 tx=0x7f7ecc006990).stop 2021-04-03T03:17:15.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 msgr2=0x7f7ed00d44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d44e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f7ed0049520 tx=0x7f7ec8015000).stop 2021-04-03T03:17:15.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 shutdown_connections 2021-04-03T03:17:15.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7ed00d5f10 0x7f7ed00cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7ed0048ef0 0x7f7ed00d44e0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7eb8056f20 0x7f7eb80593e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 --2- 172.21.2.111:0/2243655922 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7ed00d5a90 0x7f7ed00cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.310+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 >> 172.21.2.111:0/2243655922 conn(0x7f7ed000f2d0 msgr2=0x7f7ed00d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.314+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 shutdown_connections 2021-04-03T03:17:15.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.314+0000 7f7edd7de700 1 -- 172.21.2.111:0/2243655922 wait complete. 2021-04-03T03:17:15.316 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 27 and data pool 28 2021-04-03T03:17:15.321 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:15.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 Processor -- start 2021-04-03T03:17:15.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- start start 2021-04-03T03:17:15.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 0x7f8d540b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8d540b0c90 0x7f8d54147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540607d0 con 0x7f8d5414b700 2021-04-03T03:17:15.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540af400 con 0x7f8d540b0c90 2021-04-03T03:17:15.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540af6a0 con 0x7f8d540603f0 2021-04-03T03:17:15.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8d540b0c90 0x7f8d54147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8d540b0c90 0x7f8d54147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58176/0 (socket says 172.21.2.111:58176) 2021-04-03T03:17:15.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/3785310715 learned_addr learned my addr 172.21.2.111:0/3785310715 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d58eb7700 1 --2- 172.21.2.111:0/3785310715 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 0x7f8d540b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/3785310715 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 msgr2=0x7f8d540b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:15.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- 172.21.2.111:0/3785310715 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 0x7f8d540b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/3785310715 >> v1:172.21.2.111:6789/0 conn(0x7f8d5414b700 legacy=0x7f8d540b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d537fe700 1 -- 172.21.2.111:0/3785310715 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705308532 0 0) 0x7f8d540607d0 con 0x7f8d5414b700 2021-04-03T03:17:15.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/3785310715 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540afb40 con 0x7f8d540b0c90 2021-04-03T03:17:15.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- 172.21.2.111:0/3785310715 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d540b0c90 0x7f8d54147680 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f8d4c0032b0 tx=0x7f8d4c009270).ready entity=mon.0 client_cookie=c3b1fb4ea50f296a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d537fe700 1 -- 172.21.2.111:0/3785310715 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d4c00bc00 con 0x7f8d540b0c90 2021-04-03T03:17:15.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d537fe700 1 -- 172.21.2.111:0/3785310715 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c00bd60 con 0x7f8d540b0c90 2021-04-03T03:17:15.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d537fe700 1 -- 172.21.2.111:0/3785310715 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d4c011400 con 0x7f8d540b0c90 2021-04-03T03:17:15.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/3785310715 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d540b0c90 msgr2=0x7f8d54147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/3785310715 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d540b0c90 0x7f8d54147680 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f8d4c0032b0 tx=0x7f8d4c009270).stop 2021-04-03T03:17:15.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/3785310715 shutdown_connections 2021-04-03T03:17:15.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/3785310715 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d540b0c90 0x7f8d54147680 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/3785310715 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 0x7f8d540b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/3785310715 >> 172.21.2.111:0/3785310715 conn(0x7f8d540b61c0 msgr2=0x7f8d5405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.387 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/3785310715 shutdown_connections 2021-04-03T03:17:15.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/3785310715 wait complete. 2021-04-03T03:17:15.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 Processor -- start 2021-04-03T03:17:15.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- start start 2021-04-03T03:17:15.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 0x7f8d54064b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d540b0c90 0x7f8d54062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d5414b700 0x7f8d540673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540a9a70 con 0x7f8d5414b700 2021-04-03T03:17:15.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540ab5c0 con 0x7f8d540b0c90 2021-04-03T03:17:15.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540ab860 con 0x7f8d540603f0 2021-04-03T03:17:15.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d5414b700 0x7f8d540673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d5414b700 0x7f8d540673e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58180/0 (socket says 172.21.2.111:58180) 2021-04-03T03:17:15.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/2896145266 learned_addr learned my addr 172.21.2.111:0/2896145266 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/2896145266 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 msgr2=0x7f8d54064b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 0x7f8d54064b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/2896145266 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d540b0c90 msgr2=0x7f8d54062aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d540b0c90 0x7f8d54062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 -- 172.21.2.111:0/2896145266 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540abb00 con 0x7f8d5414b700 2021-04-03T03:17:15.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d596b8700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d5414b700 0x7f8d540673e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f8d4c0032b0 tx=0x7f8d4c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d51ffb700 1 -- 172.21.2.111:0/2896145266 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d4c020620 con 0x7f8d5414b700 2021-04-03T03:17:15.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d51ffb700 1 -- 172.21.2.111:0/2896145266 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c020780 con 0x7f8d5414b700 2021-04-03T03:17:15.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540abfa0 con 0x7f8d5414b700 2021-04-03T03:17:15.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d540aab40 con 0x7f8d5414b700 2021-04-03T03:17:15.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d51ffb700 1 -- 172.21.2.111:0/2896145266 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d4c020ae0 con 0x7f8d5414b700 2021-04-03T03:17:15.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.378+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540abc90 con 0x7f8d5414b700 2021-04-03T03:17:15.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.382+0000 7f8d51ffb700 1 -- 172.21.2.111:0/2896145266 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8d4c026020 con 0x7f8d5414b700 2021-04-03T03:17:15.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.382+0000 7f8d51ffb700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d38056f70 0x7f8d38059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.382+0000 7f8d51ffb700 1 -- 172.21.2.111:0/2896145266 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f8d4c071580 con 0x7f8d5414b700 2021-04-03T03:17:15.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.382+0000 7f8d58eb7700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d38056f70 0x7f8d38059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.382+0000 7f8d58eb7700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d38056f70 0x7f8d38059430 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8d4400a000 tx=0x7f8d44008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.382+0000 7f8d51ffb700 1 -- 172.21.2.111:0/2896145266 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8d4c046990 con 0x7f8d5414b700 2021-04-03T03:17:15.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.478+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d54057660 con 0x7f8d5414b700 2021-04-03T03:17:15.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.478+0000 7f8d51ffb700 1 -- 172.21.2.111:0/2896145266 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f8d4c015020 con 0x7f8d5414b700 2021-04-03T03:17:15.480 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:15.481 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":140,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:15.307226+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:13.328557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:14.274378+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:15.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d38056f70 msgr2=0x7f8d38059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d38056f70 0x7f8d38059430 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8d4400a000 tx=0x7f8d44008040).stop 2021-04-03T03:17:15.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d5414b700 msgr2=0x7f8d540673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d5414b700 0x7f8d540673e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f8d4c0032b0 tx=0x7f8d4c016000).stop 2021-04-03T03:17:15.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 shutdown_connections 2021-04-03T03:17:15.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8d5414b700 0x7f8d540673e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8d540603f0 0x7f8d54064b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8d38056f70 0x7f8d38059430 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 --2- 172.21.2.111:0/2896145266 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8d540b0c90 0x7f8d54062aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 >> 172.21.2.111:0/2896145266 conn(0x7f8d540b61c0 msgr2=0x7f8d54147fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 shutdown_connections 2021-04-03T03:17:15.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.482+0000 7f8d59eb9700 1 -- 172.21.2.111:0/2896145266 wait complete. 2021-04-03T03:17:15.489 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:15.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 Processor -- start 2021-04-03T03:17:15.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- start start 2021-04-03T03:17:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b7c0b3a90 0x7f3b7c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3b7c0ad660 0x7f3b7c0ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c0b24d0 con 0x7f3b7c0b8350 2021-04-03T03:17:15.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c142090 con 0x7f3b7c0b3a90 2021-04-03T03:17:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c0b8730 con 0x7f3b7c0ad660 2021-04-03T03:17:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b7c0b3a90 0x7f3b7c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f3b7c0b3a90 0x7f3b7c0ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58190/0 (socket says 172.21.2.111:58190) 2021-04-03T03:17:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 -- 172.21.2.111:0/3865039105 learned_addr learned my addr 172.21.2.111:0/3865039105 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7ade5700 1 --1- 172.21.2.111:0/3865039105 >> v1:172.21.2.111:6789/0 conn(0x7f3b7c0b8350 0x7f3b7c0b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:58980/0 (socket says 172.21.2.111:58980) 2021-04-03T03:17:15.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 -- 172.21.2.111:0/3865039105 >> v2:172.21.2.115:3300/0 conn(0x7f3b7c0ad660 msgr2=0x7f3b7c0ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:15.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 --2- 172.21.2.111:0/3865039105 >> v2:172.21.2.115:3300/0 conn(0x7f3b7c0ad660 0x7f3b7c0ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 -- 172.21.2.111:0/3865039105 >> v1:172.21.2.111:6789/0 conn(0x7f3b7c0b8350 legacy=0x7f3b7c0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 -- 172.21.2.111:0/3865039105 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c1413c0 con 0x7f3b7c0b3a90 2021-04-03T03:17:15.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7a5e4700 1 --2- 172.21.2.111:0/3865039105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b3a90 0x7f3b7c0ad170 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f3b64002f30 tx=0x7f3b64006e90).ready entity=mon.0 client_cookie=fb3b802e11ce6b22 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b79de3700 1 -- 172.21.2.111:0/3865039105 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b640099c0 con 0x7f3b7c0b3a90 2021-04-03T03:17:15.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b79de3700 1 -- 172.21.2.111:0/3865039105 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64009b20 con 0x7f3b7c0b3a90 2021-04-03T03:17:15.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b79de3700 1 -- 172.21.2.111:0/3865039105 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b64009e80 con 0x7f3b7c0b3a90 2021-04-03T03:17:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3865039105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b3a90 msgr2=0x7f3b7c0ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3865039105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b3a90 0x7f3b7c0ad170 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f3b64002f30 tx=0x7f3b64006e90).stop 2021-04-03T03:17:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3865039105 shutdown_connections 2021-04-03T03:17:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3865039105 >> v2:172.21.2.115:3300/0 conn(0x7f3b7c0ad660 0x7f3b7c0ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3865039105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b3a90 0x7f3b7c0ad170 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3865039105 >> 172.21.2.111:0/3865039105 conn(0x7f3b7c060490 msgr2=0x7f3b7c0b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3865039105 shutdown_connections 2021-04-03T03:17:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3865039105 wait complete. 2021-04-03T03:17:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 Processor -- start 2021-04-03T03:17:15.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- start start 2021-04-03T03:17:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7c0ad660 0x7f3b7c13b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7c0b3a90 0x7f3b7c13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b8350 0x7f3b7c0afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c144080 con 0x7f3b7c0b8350 2021-04-03T03:17:15.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0b19c0 con 0x7f3b7c0ad660 2021-04-03T03:17:15.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7bde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c14ec60 con 0x7f3b7c0b3a90 2021-04-03T03:17:15.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b8350 0x7f3b7c0afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b8350 0x7f3b7c0afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58198/0 (socket says 172.21.2.111:58198) 2021-04-03T03:17:15.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 -- 172.21.2.111:0/3629422002 learned_addr learned my addr 172.21.2.111:0/3629422002 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 -- 172.21.2.111:0/3629422002 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7c0b3a90 msgr2=0x7f3b7c13bdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:15.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7c0b3a90 0x7f3b7c13bdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 -- 172.21.2.111:0/3629422002 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7c0ad660 msgr2=0x7f3b7c13b900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:15.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7c0ad660 0x7f3b7c13b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 -- 172.21.2.111:0/3629422002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c14f100 con 0x7f3b7c0b8350 2021-04-03T03:17:15.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.546+0000 7f3b7b5e6700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b8350 0x7f3b7c0afd70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f3b740032b0 tx=0x7f3b74009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b73fff700 1 -- 172.21.2.111:0/3629422002 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7400ca60 con 0x7f3b7c0b8350 2021-04-03T03:17:15.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b73fff700 1 -- 172.21.2.111:0/3629422002 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7400cbc0 con 0x7f3b7c0b8350 2021-04-03T03:17:15.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c0b1140 con 0x7f3b7c0b8350 2021-04-03T03:17:15.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b73fff700 1 -- 172.21.2.111:0/3629422002 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b74015420 con 0x7f3b7c0b8350 2021-04-03T03:17:15.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c0b1640 con 0x7f3b7c0b8350 2021-04-03T03:17:15.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c14eaa0 con 0x7f3b7c0b8350 2021-04-03T03:17:15.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b73fff700 1 -- 172.21.2.111:0/3629422002 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3b7401c070 con 0x7f3b7c0b8350 2021-04-03T03:17:15.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b73fff700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ea0 0x7f3b60059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b73fff700 1 -- 172.21.2.111:0/3629422002 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3b7406fd60 con 0x7f3b7c0b8350 2021-04-03T03:17:15.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b7ade5700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ea0 0x7f3b60059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b7ade5700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ea0 0x7f3b60059360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f3b7c060990 tx=0x7f3b6c0077c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.550+0000 7f3b73fff700 1 -- 172.21.2.111:0/3629422002 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3b74045150 con 0x7f3b7c0b8350 2021-04-03T03:17:15.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b7c0b12d0 con 0x7f3b7c0b8350 2021-04-03T03:17:15.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b73fff700 1 -- 172.21.2.111:0/3629422002 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f3b74048770 con 0x7f3b7c0b8350 2021-04-03T03:17:15.648 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:15.648 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":140,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:15.307226+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:13.328557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:14.274378+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:15.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ea0 msgr2=0x7f3b60059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ea0 0x7f3b60059360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f3b7c060990 tx=0x7f3b6c0077c0).stop 2021-04-03T03:17:15.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b8350 msgr2=0x7f3b7c0afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b8350 0x7f3b7c0afd70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f3b740032b0 tx=0x7f3b74009220).stop 2021-04-03T03:17:15.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 shutdown_connections 2021-04-03T03:17:15.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3b7c0b8350 0x7f3b7c0afd70 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3b60056ea0 0x7f3b60059360 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3b7c0ad660 0x7f3b7c13b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 --2- 172.21.2.111:0/3629422002 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3b7c0b3a90 0x7f3b7c13bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 >> 172.21.2.111:0/3629422002 conn(0x7f3b7c060490 msgr2=0x7f3b7c145790 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 shutdown_connections 2021-04-03T03:17:15.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.646+0000 7f3b7bde7700 1 -- 172.21.2.111:0/3629422002 wait complete. 2021-04-03T03:17:15.656 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:17:15.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.710+0000 7f64ba217700 1 Processor -- start 2021-04-03T03:17:15.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.710+0000 7f64ba217700 1 -- start start 2021-04-03T03:17:15.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f64ac039dd0 0x7f64ac0d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f64ac03fc60 0x7f64ac005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac0d1c70 con 0x7f64ac03f880 2021-04-03T03:17:15.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0d2f00 con 0x7f64ac039dd0 2021-04-03T03:17:15.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0400d0 con 0x7f64ac03fc60 2021-04-03T03:17:15.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f64ac039dd0 0x7f64ac0d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f64ac039dd0 0x7f64ac0d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58204/0 (socket says 172.21.2.111:58204) 2021-04-03T03:17:15.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/3688210355 learned_addr learned my addr 172.21.2.111:0/3688210355 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/3688210355 >> v2:172.21.2.115:3300/0 conn(0x7f64ac03fc60 msgr2=0x7f64ac005680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:15.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- 172.21.2.111:0/3688210355 >> v2:172.21.2.115:3300/0 conn(0x7f64ac03fc60 0x7f64ac005680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/3688210355 >> v1:172.21.2.115:6789/0 conn(0x7f64ac03f880 legacy=0x7f64ac0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:15.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/3688210355 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0d1ea0 con 0x7f64ac039dd0 2021-04-03T03:17:15.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- 172.21.2.111:0/3688210355 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac039dd0 0x7f64ac0d2880 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f64ac00c350 tx=0x7f64a8006e90).ready entity=mon.0 client_cookie=6127b3a8e66ad41b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64a7fff700 1 -- 172.21.2.111:0/3688210355 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64a8009930 con 0x7f64ac039dd0 2021-04-03T03:17:15.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64a7fff700 1 -- 172.21.2.111:0/3688210355 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8009a90 con 0x7f64ac039dd0 2021-04-03T03:17:15.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64a7fff700 1 -- 172.21.2.111:0/3688210355 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64a8009e30 con 0x7f64ac039dd0 2021-04-03T03:17:15.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- 172.21.2.111:0/3688210355 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac039dd0 msgr2=0x7f64ac0d2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- 172.21.2.111:0/3688210355 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac039dd0 0x7f64ac0d2880 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f64ac00c350 tx=0x7f64a8006e90).stop 2021-04-03T03:17:15.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- 172.21.2.111:0/3688210355 shutdown_connections 2021-04-03T03:17:15.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- 172.21.2.111:0/3688210355 >> v2:172.21.2.115:3300/0 conn(0x7f64ac03fc60 0x7f64ac005680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- 172.21.2.111:0/3688210355 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac039dd0 0x7f64ac0d2880 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:15.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- 172.21.2.111:0/3688210355 >> 172.21.2.111:0/3688210355 conn(0x7f64ac0462b0 msgr2=0x7f64ac0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:15.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- 172.21.2.111:0/3688210355 shutdown_connections 2021-04-03T03:17:15.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- 172.21.2.111:0/3688210355 wait complete. 2021-04-03T03:17:15.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 Processor -- start 2021-04-03T03:17:15.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- start start 2021-04-03T03:17:15.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f64ac039dd0 0x7f64ac03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac03f880 0x7f64ac0d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f64ac03fc60 0x7f64ac03d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0da3b0 con 0x7f64ac03f880 2021-04-03T03:17:15.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0d8ee0 con 0x7f64ac039dd0 2021-04-03T03:17:15.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0d89a0 con 0x7f64ac03fc60 2021-04-03T03:17:15.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac03f880 0x7f64ac0d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac03f880 0x7f64ac0d9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58208/0 (socket says 172.21.2.111:58208) 2021-04-03T03:17:15.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/1267899244 learned_addr learned my addr 172.21.2.111:0/1267899244 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:15.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/1267899244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f64ac03fc60 msgr2=0x7f64ac03d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f64ac03fc60 0x7f64ac03d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/1267899244 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f64ac039dd0 msgr2=0x7f64ac03b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:15.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f64ac039dd0 0x7f64ac03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:15.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 -- 172.21.2.111:0/1267899244 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0d8c40 con 0x7f64ac03f880 2021-04-03T03:17:15.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64b8a14700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac03f880 0x7f64ac0d9ec0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f64ac00c350 tx=0x7f64a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64a67fc700 1 -- 172.21.2.111:0/1267899244 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64a8011ce0 con 0x7f64ac03f880 2021-04-03T03:17:15.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64a67fc700 1 -- 172.21.2.111:0/1267899244 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8011e40 con 0x7f64ac03f880 2021-04-03T03:17:15.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac03f160 con 0x7f64ac03f880 2021-04-03T03:17:15.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64a67fc700 1 -- 172.21.2.111:0/1267899244 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64a801e650 con 0x7f64ac03f880 2021-04-03T03:17:15.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.714+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac03d700 con 0x7f64ac03f880 2021-04-03T03:17:15.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.718+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac004db0 con 0x7f64ac03f880 2021-04-03T03:17:15.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.718+0000 7f64a67fc700 1 -- 172.21.2.111:0/1267899244 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f64a8025030 con 0x7f64ac03f880 2021-04-03T03:17:15.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.718+0000 7f64a67fc700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6494056ea0 0x7f6494059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:15.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.718+0000 7f64a67fc700 1 -- 172.21.2.111:0/1267899244 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f64a8071600 con 0x7f64ac03f880 2021-04-03T03:17:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.718+0000 7f64b9215700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6494056ea0 0x7f6494059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.718+0000 7f64b9215700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6494056ea0 0x7f6494059360 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f64b0002f50 tx=0x7f64b0006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:15.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.718+0000 7f64a67fc700 1 -- 172.21.2.111:0/1267899244 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f64a80469f0 con 0x7f64ac03f880 2021-04-03T03:17:15.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:15.846+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f64ac046030 con 0x7f64ac03f880 2021-04-03T03:17:16.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64a67fc700 1 -- 172.21.2.111:0/1267899244 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7f64a80230b0 con 0x7f64ac03f880 2021-04-03T03:17:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6494056ea0 msgr2=0x7f6494059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6494056ea0 0x7f6494059360 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f64b0002f50 tx=0x7f64b0006990).stop 2021-04-03T03:17:16.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac03f880 msgr2=0x7f64ac0d9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac03f880 0x7f64ac0d9ec0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f64ac00c350 tx=0x7f64a8015000).stop 2021-04-03T03:17:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 shutdown_connections 2021-04-03T03:17:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f64ac03fc60 0x7f64ac03d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6494056ea0 0x7f6494059360 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f64ac039dd0 0x7f64ac03b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 --2- 172.21.2.111:0/1267899244 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f64ac03f880 0x7f64ac0d9ec0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 >> 172.21.2.111:0/1267899244 conn(0x7f64ac0462b0 msgr2=0x7f64ac0cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 shutdown_connections 2021-04-03T03:17:16.325 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.322+0000 7f64ba217700 1 -- 172.21.2.111:0/1267899244 wait complete. 2021-04-03T03:17:16.330 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:16.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 Processor -- start 2021-04-03T03:17:16.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- start start 2021-04-03T03:17:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1540603f0 0x7fc1540b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1540b0c90 0x7fc154147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1540607d0 con 0x7fc15414b700 2021-04-03T03:17:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1540af400 con 0x7fc1540603f0 2021-04-03T03:17:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540af6a0 con 0x7fc1540b0c90 2021-04-03T03:17:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1540603f0 0x7fc1540b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1540603f0 0x7fc1540b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58218/0 (socket says 172.21.2.111:58218) 2021-04-03T03:17:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 -- 172.21.2.111:0/3105880397 learned_addr learned my addr 172.21.2.111:0/3105880397 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159144700 1 --2- 172.21.2.111:0/3105880397 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1540b0c90 0x7fc154147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 -- 172.21.2.111:0/3105880397 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1540b0c90 msgr2=0x7fc154147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 --2- 172.21.2.111:0/3105880397 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1540b0c90 0x7fc154147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 -- 172.21.2.111:0/3105880397 >> v1:172.21.2.115:6789/0 conn(0x7fc15414b700 legacy=0x7fc1540b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 -- 172.21.2.111:0/3105880397 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1540afb40 con 0x7fc1540603f0 2021-04-03T03:17:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc158943700 1 --2- 172.21.2.111:0/3105880397 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540b0090 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fc15406a3b0 tx=0x7fc144006e90).ready entity=mon.0 client_cookie=73dbb56e6b5bbc87 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc1537fe700 1 -- 172.21.2.111:0/3105880397 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1440098e0 con 0x7fc1540603f0 2021-04-03T03:17:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc1537fe700 1 -- 172.21.2.111:0/3105880397 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc144009a40 con 0x7fc1540603f0 2021-04-03T03:17:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc1537fe700 1 -- 172.21.2.111:0/3105880397 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc144009de0 con 0x7fc1540603f0 2021-04-03T03:17:16.392 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- 172.21.2.111:0/3105880397 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 msgr2=0x7fc1540b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 --2- 172.21.2.111:0/3105880397 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540b0090 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fc15406a3b0 tx=0x7fc144006e90).stop 2021-04-03T03:17:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- 172.21.2.111:0/3105880397 shutdown_connections 2021-04-03T03:17:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 --2- 172.21.2.111:0/3105880397 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1540b0c90 0x7fc154147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 --2- 172.21.2.111:0/3105880397 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540b0090 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- 172.21.2.111:0/3105880397 >> 172.21.2.111:0/3105880397 conn(0x7fc1540b61c0 msgr2=0x7fc15405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- 172.21.2.111:0/3105880397 shutdown_connections 2021-04-03T03:17:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 -- 172.21.2.111:0/3105880397 wait complete. 2021-04-03T03:17:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.386+0000 7fc159945700 1 Processor -- start 2021-04-03T03:17:16.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 -- start start 2021-04-03T03:17:16.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1540b0c90 0x7fc154067590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc15414b700 0x7fc154067a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154062d70 con 0x7fc1540603f0 2021-04-03T03:17:16.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154063010 con 0x7fc1540b0c90 2021-04-03T03:17:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540632b0 con 0x7fc15414b700 2021-04-03T03:17:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58226/0 (socket says 172.21.2.111:58226) 2021-04-03T03:17:16.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 -- 172.21.2.111:0/3356972361 learned_addr learned my addr 172.21.2.111:0/3356972361 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159144700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc15414b700 0x7fc154067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc153fff700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1540b0c90 0x7fc154067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 -- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc15414b700 msgr2=0x7fc154067a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc15414b700 0x7fc154067a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 -- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1540b0c90 msgr2=0x7fc154067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1540b0c90 0x7fc154067590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 -- 172.21.2.111:0/3356972361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc154067f70 con 0x7fc1540603f0 2021-04-03T03:17:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc158943700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540a7830 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7fc15406a3b0 tx=0x7fc144015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc151ffb700 1 -- 172.21.2.111:0/3356972361 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc144011ce0 con 0x7fc1540603f0 2021-04-03T03:17:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc151ffb700 1 -- 172.21.2.111:0/3356972361 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc144011e40 con 0x7fc1540603f0 2021-04-03T03:17:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc151ffb700 1 -- 172.21.2.111:0/3356972361 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc14401e650 con 0x7fc1540603f0 2021-04-03T03:17:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1540aa130 con 0x7fc1540603f0 2021-04-03T03:17:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc154062ae0 con 0x7fc1540603f0 2021-04-03T03:17:16.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1540aa350 con 0x7fc1540603f0 2021-04-03T03:17:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc151ffb700 1 -- 172.21.2.111:0/3356972361 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc144025030 con 0x7fc1540603f0 2021-04-03T03:17:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc151ffb700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc134056f70 0x7fc134059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc151ffb700 1 -- 172.21.2.111:0/3356972361 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc1440715e0 con 0x7fc1540603f0 2021-04-03T03:17:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc153fff700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc134056f70 0x7fc134059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.390+0000 7fc153fff700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc134056f70 0x7fc134059430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fc14800a000 tx=0x7fc148008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.394+0000 7fc151ffb700 1 -- 172.21.2.111:0/3356972361 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc1440469d0 con 0x7fc1540603f0 2021-04-03T03:17:16.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc154057660 con 0x7fc1540603f0 2021-04-03T03:17:16.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc151ffb700 1 -- 172.21.2.111:0/3356972361 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fc144023870 con 0x7fc1540603f0 2021-04-03T03:17:16.519 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:16.519 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":63,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3135556983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3135556983},{"type":"v1","addr":"172.21.2.115:6833","nonce":3135556983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:15.307326+0000","modified":"2021-04-03T03:17:16.311289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8250":{"gid":8250,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1485494348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1485494348},{"type":"v1","addr":"172.21.2.111:6835","nonce":1485494348}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-04-03T03:17:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc134056f70 msgr2=0x7fc134059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc134056f70 0x7fc134059430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fc14800a000 tx=0x7fc148008040).stop 2021-04-03T03:17:16.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 msgr2=0x7fc1540a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540a7830 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7fc15406a3b0 tx=0x7fc144015000).stop 2021-04-03T03:17:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 shutdown_connections 2021-04-03T03:17:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc15414b700 0x7fc154067a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1540603f0 0x7fc1540a7830 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc134056f70 0x7fc134059430 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 --2- 172.21.2.111:0/3356972361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1540b0c90 0x7fc154067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 >> 172.21.2.111:0/3356972361 conn(0x7fc1540b61c0 msgr2=0x7fc154147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 shutdown_connections 2021-04-03T03:17:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.518+0000 7fc159945700 1 -- 172.21.2.111:0/3356972361 wait complete. 2021-04-03T03:17:16.522 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 63 2021-04-03T03:17:16.526 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:16.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 Processor -- start 2021-04-03T03:17:16.583 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- start start 2021-04-03T03:17:16.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa19c0603f0 0x7fa19c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa19c0b0c90 0x7fa19c147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19c0af400 con 0x7fa19c14b700 2021-04-03T03:17:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa19c060860 con 0x7fa19c0603f0 2021-04-03T03:17:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa19c0af6a0 con 0x7fa19c0b0c90 2021-04-03T03:17:16.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa19c0603f0 0x7fa19c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa19c0603f0 0x7fa19c0b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58230/0 (socket says 172.21.2.111:58230) 2021-04-03T03:17:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 -- 172.21.2.111:0/3971657333 learned_addr learned my addr 172.21.2.111:0/3971657333 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 -- 172.21.2.111:0/3971657333 >> v2:172.21.2.115:3300/0 conn(0x7fa19c0b0c90 msgr2=0x7fa19c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.587 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 --2- 172.21.2.111:0/3971657333 >> v2:172.21.2.115:3300/0 conn(0x7fa19c0b0c90 0x7fa19c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 -- 172.21.2.111:0/3971657333 >> v1:172.21.2.111:6789/0 conn(0x7fa19c14b700 legacy=0x7fa19c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19959a700 1 -- 172.21.2.111:0/3971657333 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357855766 0 0) 0x7fa19c0af400 con 0x7fa19c14b700 2021-04-03T03:17:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 -- 172.21.2.111:0/3971657333 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c0afb40 con 0x7fa19c0603f0 2021-04-03T03:17:16.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19a59c700 1 --2- 172.21.2.111:0/3971657333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0603f0 0x7fa19c0b0090 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fa184002f30 tx=0x7fa184006e90).ready entity=mon.0 client_cookie=12bff361250751ec server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19959a700 1 -- 172.21.2.111:0/3971657333 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1840099c0 con 0x7fa19c0603f0 2021-04-03T03:17:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19959a700 1 -- 172.21.2.111:0/3971657333 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa184009b20 con 0x7fa19c0603f0 2021-04-03T03:17:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19959a700 1 -- 172.21.2.111:0/3971657333 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa184009e80 con 0x7fa19c0603f0 2021-04-03T03:17:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- 172.21.2.111:0/3971657333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0603f0 msgr2=0x7fa19c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3971657333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0603f0 0x7fa19c0b0090 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fa184002f30 tx=0x7fa184006e90).stop 2021-04-03T03:17:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- 172.21.2.111:0/3971657333 shutdown_connections 2021-04-03T03:17:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3971657333 >> v2:172.21.2.115:3300/0 conn(0x7fa19c0b0c90 0x7fa19c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3971657333 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0603f0 0x7fa19c0b0090 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- 172.21.2.111:0/3971657333 >> 172.21.2.111:0/3971657333 conn(0x7fa19c0b61c0 msgr2=0x7fa19c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- 172.21.2.111:0/3971657333 shutdown_connections 2021-04-03T03:17:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.582+0000 7fa19b59e700 1 -- 172.21.2.111:0/3971657333 wait complete. 2021-04-03T03:17:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 Processor -- start 2021-04-03T03:17:16.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 -- start start 2021-04-03T03:17:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa19c0603f0 0x7fa19c065b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0b0c90 0x7fa19c067310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19c14b700 0x7fa19c0ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c0a70b0 con 0x7fa19c0b0c90 2021-04-03T03:17:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c0a7650 con 0x7fa19c0603f0 2021-04-03T03:17:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c067800 con 0x7fa19c14b700 2021-04-03T03:17:16.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0b0c90 0x7fa19c067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0b0c90 0x7fa19c067310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58238/0 (socket says 172.21.2.111:58238) 2021-04-03T03:17:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 -- 172.21.2.111:0/3088237889 learned_addr learned my addr 172.21.2.111:0/3088237889 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 -- 172.21.2.111:0/3088237889 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19c14b700 msgr2=0x7fa19c0ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19c14b700 0x7fa19c0ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 -- 172.21.2.111:0/3088237889 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa19c0603f0 msgr2=0x7fa19c065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa19c0603f0 0x7fa19c065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 -- 172.21.2.111:0/3088237889 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c149770 con 0x7fa19c0b0c90 2021-04-03T03:17:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa199d9b700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0b0c90 0x7fa19c067310 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fa18c000e90 tx=0x7fa18c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa1937fe700 1 -- 172.21.2.111:0/3088237889 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18c00f470 con 0x7fa19c0b0c90 2021-04-03T03:17:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa1937fe700 1 -- 172.21.2.111:0/3088237889 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c00f5d0 con 0x7fa19c0b0c90 2021-04-03T03:17:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19c149a10 con 0x7fa19c0b0c90 2021-04-03T03:17:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19c156530 con 0x7fa19c0b0c90 2021-04-03T03:17:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa1937fe700 1 -- 172.21.2.111:0/3088237889 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa18c00f930 con 0x7fa19c0b0c90 2021-04-03T03:17:16.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19c149da0 con 0x7fa19c0b0c90 2021-04-03T03:17:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa1937fe700 1 -- 172.21.2.111:0/3088237889 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa18c019030 con 0x7fa19c0b0c90 2021-04-03T03:17:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa1937fe700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa180057000 0x7fa1800594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa1937fe700 1 -- 172.21.2.111:0/3088237889 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fa18c06f770 con 0x7fa19c0b0c90 2021-04-03T03:17:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19a59c700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa180057000 0x7fa1800594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.586+0000 7fa19a59c700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa180057000 0x7fa1800594c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fa184002f30 tx=0x7fa184015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.590+0000 7fa1937fe700 1 -- 172.21.2.111:0/3088237889 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa18c0449f0 con 0x7fa19c0b0c90 2021-04-03T03:17:16.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.678+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa19c149ba0 con 0x7fa19c0b0c90 2021-04-03T03:17:16.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.678+0000 7fa1937fe700 1 -- 172.21.2.111:0/3088237889 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fa18c048010 con 0x7fa19c0b0c90 2021-04-03T03:17:16.681 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:16.681 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":141,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:16.311403+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:13.328557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:14.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:16.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa180057000 msgr2=0x7fa1800594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa180057000 0x7fa1800594c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fa184002f30 tx=0x7fa184015000).stop 2021-04-03T03:17:16.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0b0c90 msgr2=0x7fa19c067310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0b0c90 0x7fa19c067310 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fa18c000e90 tx=0x7fa18c007bd0).stop 2021-04-03T03:17:16.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 shutdown_connections 2021-04-03T03:17:16.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa19c14b700 0x7fa19c0ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa180057000 0x7fa1800594c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa19c0603f0 0x7fa19c065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 --2- 172.21.2.111:0/3088237889 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa19c0b0c90 0x7fa19c067310 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 >> 172.21.2.111:0/3088237889 conn(0x7fa19c0b61c0 msgr2=0x7fa19c0acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 shutdown_connections 2021-04-03T03:17:16.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.682+0000 7fa19b59e700 1 -- 172.21.2.111:0/3088237889 wait complete. 2021-04-03T03:17:16.689 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:17:16.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 Processor -- start 2021-04-03T03:17:16.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- start start 2021-04-03T03:17:16.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0b38c0 0x7fad0c025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0b3ca0 con 0x7fad0c080c60 2021-04-03T03:17:16.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c025b90 con 0x7fad0c153440 2021-04-03T03:17:16.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c025970 con 0x7fad0c0b38c0 2021-04-03T03:17:16.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad10baa700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fad0c080c60 0x7fad0c0af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59038/0 (socket says 172.21.2.111:59038) 2021-04-03T03:17:16.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad10baa700 1 -- 172.21.2.111:0/775568676 learned_addr learned my addr 172.21.2.111:0/775568676 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:16.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669202042 0 0) 0x7fad0c0b3ca0 con 0x7fad0c080c60 2021-04-03T03:17:16.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf8003240 con 0x7fad0c080c60 2021-04-03T03:17:16.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 174585212 0 0) 0x7facf8003240 con 0x7fad0c080c60 2021-04-03T03:17:16.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0b38c0 msgr2=0x7fad0c025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0b38c0 0x7fad0c025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 >> v1:172.21.2.115:6789/0 conn(0x7fad0c153440 legacy=0x7fad0c080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c025e30 con 0x7fad0c080c60 2021-04-03T03:17:16.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fad04001a50 con 0x7fad0c080c60 2021-04-03T03:17:16.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 >> v1:172.21.2.111:6789/0 conn(0x7fad0c080c60 legacy=0x7fad0c0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0b38c0 0x7facf80037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8003d20 0x7facf80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7facf80049d0 0x7facf8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c025970 con 0x7facf8003d20 2021-04-03T03:17:16.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80042b0 con 0x7facf80049d0 2021-04-03T03:17:16.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf800b730 con 0x7fad0c0b38c0 2021-04-03T03:17:16.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0a59c700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8003d20 0x7facf80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0a59c700 1 -- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0b38c0 msgr2=0x7facf80037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0a59c700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0b38c0 0x7facf80037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0a59c700 1 -- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7facf80049d0 msgr2=0x7facf8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0a59c700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7facf80049d0 0x7facf8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0a59c700 1 -- 172.21.2.111:0/775568676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8003240 con 0x7facf8003d20 2021-04-03T03:17:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0a59c700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8003d20 0x7facf80044e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7facfc000f30 tx=0x7facfc007be0).ready entity=mon.0 client_cookie=2745e4d71e4ed6bb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facfc050800 con 0x7facf8003d20 2021-04-03T03:17:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc050960 con 0x7facf8003d20 2021-04-03T03:17:16.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad09d9b700 1 -- 172.21.2.111:0/775568676 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facfc050cc0 con 0x7facf8003d20 2021-04-03T03:17:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- 172.21.2.111:0/775568676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8003d20 msgr2=0x7facf80044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 --2- 172.21.2.111:0/775568676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7facf8003d20 0x7facf80044e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7facfc000f30 tx=0x7facfc007be0).stop 2021-04-03T03:17:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad10baa700 1 -- 172.21.2.111:0/775568676 reap_dead start 2021-04-03T03:17:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- 172.21.2.111:0/775568676 shutdown_connections 2021-04-03T03:17:16.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- 172.21.2.111:0/775568676 >> 172.21.2.111:0/775568676 conn(0x7fad0c061610 msgr2=0x7fad0c061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- 172.21.2.111:0/775568676 shutdown_connections 2021-04-03T03:17:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.750+0000 7fad0b59e700 1 -- 172.21.2.111:0/775568676 wait complete. 2021-04-03T03:17:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 Processor -- start 2021-04-03T03:17:16.762 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 -- start start 2021-04-03T03:17:16.764 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0ab430 0x7fad0c0ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad0c12db60 0x7fad0c0ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad0c0ac8f0 0x7fad0c0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c064900 con 0x7fad0c12db60 2021-04-03T03:17:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0ad290 con 0x7fad0c0ac8f0 2021-04-03T03:17:16.765 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c13fbc0 con 0x7fad0c0ab430 2021-04-03T03:17:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad0c12db60 0x7fad0c0ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad0c12db60 0x7fad0c0ac400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58260/0 (socket says 172.21.2.111:58260) 2021-04-03T03:17:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 -- 172.21.2.111:0/1197759043 learned_addr learned my addr 172.21.2.111:0/1197759043 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:16.766 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad10baa700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad0c0ac8f0 0x7fad0c0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 -- 172.21.2.111:0/1197759043 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0ab430 msgr2=0x7fad0c0ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0ab430 0x7fad0c0ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 -- 172.21.2.111:0/1197759043 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad0c0ac8f0 msgr2=0x7fad0c0acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad0c0ac8f0 0x7fad0c0acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.767 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 -- 172.21.2.111:0/1197759043 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0641f0 con 0x7fad0c12db60 2021-04-03T03:17:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0a59c700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad0c12db60 0x7fad0c0ac400 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fad0c0ad3d0 tx=0x7facfc007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad037fe700 1 -- 172.21.2.111:0/1197759043 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facfc058c10 con 0x7fad0c12db60 2021-04-03T03:17:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad037fe700 1 -- 172.21.2.111:0/1197759043 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc058d70 con 0x7fad0c12db60 2021-04-03T03:17:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c064490 con 0x7fad0c12db60 2021-04-03T03:17:16.768 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad037fe700 1 -- 172.21.2.111:0/1197759043 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facfc0635e0 con 0x7fad0c12db60 2021-04-03T03:17:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c07fc30 con 0x7fad0c12db60 2021-04-03T03:17:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c057660 con 0x7fad0c12db60 2021-04-03T03:17:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad037fe700 1 -- 172.21.2.111:0/1197759043 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7facfc061070 con 0x7fad0c12db60 2021-04-03T03:17:16.769 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad037fe700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facf0056f20 0x7facf00593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad037fe700 1 -- 172.21.2.111:0/1197759043 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7facfc0b5170 con 0x7fad0c12db60 2021-04-03T03:17:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0ad9d700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facf0056f20 0x7facf00593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.754+0000 7fad0ad9d700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facf0056f20 0x7facf00593e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7facf4002f30 tx=0x7facf40069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.770 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.758+0000 7fad037fe700 1 -- 172.21.2.111:0/1197759043 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7facfc08a560 con 0x7fad0c12db60 2021-04-03T03:17:16.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.882+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fad0c067f80 con 0x7fad0c12db60 2021-04-03T03:17:16.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad037fe700 1 -- 172.21.2.111:0/1197759043 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v41) v1 ==== 184+0+0 (secure 0 0 0) 0x7facfc05e070 con 0x7fad0c12db60 2021-04-03T03:17:16.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facf0056f20 msgr2=0x7facf00593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facf0056f20 0x7facf00593e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7facf4002f30 tx=0x7facf40069f0).stop 2021-04-03T03:17:16.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad0c12db60 msgr2=0x7fad0c0ac400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad0c12db60 0x7fad0c0ac400 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fad0c0ad3d0 tx=0x7facfc007be0).stop 2021-04-03T03:17:16.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 shutdown_connections 2021-04-03T03:17:16.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7facf0056f20 0x7facf00593e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fad0c0ac8f0 0x7fad0c0acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fad0c0ab430 0x7fad0c0ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 --2- 172.21.2.111:0/1197759043 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fad0c12db60 0x7fad0c0ac400 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 >> 172.21.2.111:0/1197759043 conn(0x7fad0c061610 msgr2=0x7fad0c0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 shutdown_connections 2021-04-03T03:17:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.886+0000 7fad0b59e700 1 -- 172.21.2.111:0/1197759043 wait complete. 2021-04-03T03:17:16.892 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:17:16.896 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:17:16.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d97de7700 1 Processor -- start 2021-04-03T03:17:16.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d97de7700 1 -- start start 2021-04-03T03:17:16.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d97de7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7d980b38c0 0x7f7d98025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d97de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98153440 0x7f7d98080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d97de7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d980b3ca0 con 0x7f7d98080a40 2021-04-03T03:17:16.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d97de7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980b0300 con 0x7f7d980b38c0 2021-04-03T03:17:16.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d97de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98025970 con 0x7f7d98153440 2021-04-03T03:17:16.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d975e6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7d980b38c0 0x7f7d98025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d9d29d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7d98080a40 0x7f7d9814feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59060/0 (socket says 172.21.2.111:59060) 2021-04-03T03:17:16.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d9d29d700 1 -- 172.21.2.111:0/1778765328 learned_addr learned my addr 172.21.2.111:0/1778765328 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:16.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d975e6700 1 -- 172.21.2.111:0/1778765328 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98153440 msgr2=0x7f7d98080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:16.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d975e6700 1 --2- 172.21.2.111:0/1778765328 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98153440 0x7f7d98080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d975e6700 1 -- 172.21.2.111:0/1778765328 >> v1:172.21.2.111:6789/0 conn(0x7f7d98080a40 legacy=0x7f7d9814feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d975e6700 1 -- 172.21.2.111:0/1778765328 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d98025b90 con 0x7f7d980b38c0 2021-04-03T03:17:16.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d965e4700 1 -- 172.21.2.111:0/1778765328 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640892204 0 0) 0x7f7d980b3ca0 con 0x7f7d98080a40 2021-04-03T03:17:16.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d975e6700 1 --2- 172.21.2.111:0/1778765328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98025480 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f7d80002f30 tx=0x7f7d80006dd0).ready entity=mon.0 client_cookie=a26477425a13f2d7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d965e4700 1 -- 172.21.2.111:0/1778765328 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d800097f0 con 0x7f7d980b38c0 2021-04-03T03:17:16.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d965e4700 1 -- 172.21.2.111:0/1778765328 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d80009950 con 0x7f7d980b38c0 2021-04-03T03:17:16.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.954+0000 7f7d965e4700 1 -- 172.21.2.111:0/1778765328 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d80009cb0 con 0x7f7d980b38c0 2021-04-03T03:17:16.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1778765328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 msgr2=0x7f7d98025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1778765328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98025480 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f7d80002f30 tx=0x7f7d80006dd0).stop 2021-04-03T03:17:16.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1778765328 shutdown_connections 2021-04-03T03:17:16.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1778765328 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98025480 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:16.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1778765328 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98153440 0x7f7d98080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1778765328 >> 172.21.2.111:0/1778765328 conn(0x7f7d98061610 msgr2=0x7f7d98061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:16.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1778765328 shutdown_connections 2021-04-03T03:17:16.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1778765328 wait complete. 2021-04-03T03:17:16.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 Processor -- start 2021-04-03T03:17:16.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- start start 2021-04-03T03:17:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98080a40 0x7f7d98153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98137d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d98153440 0x7f7d98138220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98154c00 con 0x7f7d980b38c0 2021-04-03T03:17:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98154960 con 0x7f7d98153440 2021-04-03T03:17:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98066b30 con 0x7f7d98080a40 2021-04-03T03:17:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98137d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58274/0 (socket says 172.21.2.111:58274) 2021-04-03T03:17:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 -- 172.21.2.111:0/1086579473 learned_addr learned my addr 172.21.2.111:0/1086579473 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 -- 172.21.2.111:0/1086579473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98080a40 msgr2=0x7f7d98153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98080a40 0x7f7d98153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 -- 172.21.2.111:0/1086579473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d98153440 msgr2=0x7f7d98138220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d98153440 0x7f7d98138220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 -- 172.21.2.111:0/1086579473 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d98066fd0 con 0x7f7d980b38c0 2021-04-03T03:17:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d96de5700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98137d30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f7d88002f30 tx=0x7f7d88006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d8ffff700 1 -- 172.21.2.111:0/1086579473 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8800d7c0 con 0x7f7d980b38c0 2021-04-03T03:17:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d8ffff700 1 -- 172.21.2.111:0/1086579473 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8800d920 con 0x7f7d980b38c0 2021-04-03T03:17:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d98065bb0 con 0x7f7d980b38c0 2021-04-03T03:17:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d981545e0 con 0x7f7d980b38c0 2021-04-03T03:17:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d8ffff700 1 -- 172.21.2.111:0/1086579473 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8800dc80 con 0x7f7d980b38c0 2021-04-03T03:17:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d98064f10 con 0x7f7d980b38c0 2021-04-03T03:17:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d8ffff700 1 -- 172.21.2.111:0/1086579473 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7d880103c0 con 0x7f7d980b38c0 2021-04-03T03:17:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.958+0000 7f7d8ffff700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d7c057000 0x7f7d7c0594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.962+0000 7f7d8ffff700 1 -- 172.21.2.111:0/1086579473 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7d8806fad0 con 0x7f7d980b38c0 2021-04-03T03:17:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.962+0000 7f7d975e6700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d7c057000 0x7f7d7c0594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.962+0000 7f7d975e6700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d7c057000 0x7f7d7c0594c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f7d80002f30 tx=0x7f7d80015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:16.962+0000 7f7d8ffff700 1 -- 172.21.2.111:0/1086579473 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7d88044e60 con 0x7f7d980b38c0 2021-04-03T03:17:17.091 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.090+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7d98066cf0 con 0x7f7d980b38c0 2021-04-03T03:17:17.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.094+0000 7f7d8ffff700 1 -- 172.21.2.111:0/1086579473 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v42) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7d88048480 con 0x7f7d980b38c0 2021-04-03T03:17:17.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.094+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d7c057000 msgr2=0x7f7d7c0594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.094+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d7c057000 0x7f7d7c0594c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f7d80002f30 tx=0x7f7d80015000).stop 2021-04-03T03:17:17.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.094+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 msgr2=0x7f7d98137d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.094+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98137d30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f7d88002f30 tx=0x7f7d88006e30).stop 2021-04-03T03:17:17.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 shutdown_connections 2021-04-03T03:17:17.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7d7c057000 0x7f7d7c0594c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7d98153440 0x7f7d98138220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:17.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7d98080a40 0x7f7d98153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:17.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 --2- 172.21.2.111:0/1086579473 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7d980b38c0 0x7f7d98137d30 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 >> 172.21.2.111:0/1086579473 conn(0x7f7d98061610 msgr2=0x7f7d980b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:17.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 shutdown_connections 2021-04-03T03:17:17.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.098+0000 7f7d97de7700 1 -- 172.21.2.111:0/1086579473 wait complete. 2021-04-03T03:17:17.100 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:17:17.104 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:17:17.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30d01d700 1 Processor -- start 2021-04-03T03:17:17.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30d01d700 1 -- start start 2021-04-03T03:17:17.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30d01d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd30003d350 0x7fd30003f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30d01d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 0x7fd3000f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30d01d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd300041240 con 0x7fd300046ce0 2021-04-03T03:17:17.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30d01d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3000f1850 con 0x7fd30003d350 2021-04-03T03:17:17.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30d01d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd300042810 con 0x7fd300042380 2021-04-03T03:17:17.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30c81c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd300046ce0 0x7fd300041c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59072/0 (socket says 172.21.2.111:59072) 2021-04-03T03:17:17.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30c81c700 1 -- 172.21.2.111:0/3468107567 learned_addr learned my addr 172.21.2.111:0/3468107567 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:17.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd307fff700 1 --2- 172.21.2.111:0/3468107567 >> v2:172.21.2.115:3300/0 conn(0x7fd30003d350 0x7fd30003f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd30e497700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 0x7fd3000f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792609933 0 0) 0x7fd300041240 con 0x7fd300046ce0 2021-04-03T03:17:17.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.162+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f0003240 con 0x7fd300046ce0 2021-04-03T03:17:17.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3741341846 0 0) 0x7fd2f0003240 con 0x7fd300046ce0 2021-04-03T03:17:17.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 msgr2=0x7fd3000f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 0x7fd3000f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 >> v2:172.21.2.115:3300/0 conn(0x7fd30003d350 msgr2=0x7fd30003f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 --2- 172.21.2.111:0/3468107567 >> v2:172.21.2.115:3300/0 conn(0x7fd30003d350 0x7fd30003f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000470c0 con 0x7fd300046ce0 2021-04-03T03:17:17.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fd2fc0016d0 con 0x7fd300046ce0 2021-04-03T03:17:17.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 >> v1:172.21.2.111:6789/0 conn(0x7fd300046ce0 legacy=0x7fd300041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00037b0 0x7fd2f0003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00044e0 0x7fd2f0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 0x7fd2f0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000f1850 con 0x7fd2f00044e0 2021-04-03T03:17:17.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300042810 con 0x7fd2f00037b0 2021-04-03T03:17:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0005340 con 0x7fd300042380 2021-04-03T03:17:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00044e0 0x7fd2f0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00037b0 0x7fd2f0003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30e497700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 0x7fd2f0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 -- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 msgr2=0x7fd2f0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd300042380 0x7fd2f0004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 -- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00037b0 msgr2=0x7fd2f0003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2f00037b0 0x7fd2f0003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 -- 172.21.2.111:0/3468107567 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f0003240 con 0x7fd2f00044e0 2021-04-03T03:17:17.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00044e0 0x7fd2f0004960 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fd2f800c000 tx=0x7fd2f8008040).ready entity=mon.0 client_cookie=fa7f16a1f15543e9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f800eb60 con 0x7fd2f00044e0 2021-04-03T03:17:17.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f800ecc0 con 0x7fd2f00044e0 2021-04-03T03:17:17.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3077fe700 1 -- 172.21.2.111:0/3468107567 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f80125e0 con 0x7fd2f00044e0 2021-04-03T03:17:17.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- 172.21.2.111:0/3468107567 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00044e0 msgr2=0x7fd2f0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 --2- 172.21.2.111:0/3468107567 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2f00044e0 0x7fd2f0004960 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fd2f800c000 tx=0x7fd2f8008040).stop 2021-04-03T03:17:17.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30e497700 1 -- 172.21.2.111:0/3468107567 reap_dead start 2021-04-03T03:17:17.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- 172.21.2.111:0/3468107567 shutdown_connections 2021-04-03T03:17:17.177 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- 172.21.2.111:0/3468107567 >> 172.21.2.111:0/3468107567 conn(0x7fd30000d0c0 msgr2=0x7fd3000462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:17.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- 172.21.2.111:0/3468107567 shutdown_connections 2021-04-03T03:17:17.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- 172.21.2.111:0/3468107567 wait complete. 2021-04-03T03:17:17.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 Processor -- start 2021-04-03T03:17:17.178 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- start start 2021-04-03T03:17:17.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3000055e0 0x7fd3000059c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd30003b910 0x7fd3000c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3000dfd10 0x7fd3000e0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000457d0 con 0x7fd3000055e0 2021-04-03T03:17:17.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000df920 con 0x7fd30003b910 2021-04-03T03:17:17.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30003aae0 con 0x7fd3000dfd10 2021-04-03T03:17:17.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3000055e0 0x7fd3000059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3000055e0 0x7fd3000059c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58290/0 (socket says 172.21.2.111:58290) 2021-04-03T03:17:17.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 -- 172.21.2.111:0/651228330 learned_addr learned my addr 172.21.2.111:0/651228330 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:17.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd307fff700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd30003b910 0x7fd3000c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30e497700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3000dfd10 0x7fd3000e0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 -- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3000dfd10 msgr2=0x7fd3000e0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3000dfd10 0x7fd3000e0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 -- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd30003b910 msgr2=0x7fd3000c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd30003b910 0x7fd3000c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 -- 172.21.2.111:0/651228330 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000c7eb0 con 0x7fd3000055e0 2021-04-03T03:17:17.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30c81c700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3000055e0 0x7fd3000059c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fd2fc00d000 tx=0x7fd2fc0076d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3057fa700 1 -- 172.21.2.111:0/651228330 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2fc00fae0 con 0x7fd3000055e0 2021-04-03T03:17:17.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3057fa700 1 -- 172.21.2.111:0/651228330 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00fc40 con 0x7fd3000055e0 2021-04-03T03:17:17.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3000e0660 con 0x7fd3000055e0 2021-04-03T03:17:17.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd3057fa700 1 -- 172.21.2.111:0/651228330 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2fc012350 con 0x7fd3000055e0 2021-04-03T03:17:17.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.166+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd300039580 con 0x7fd3000055e0 2021-04-03T03:17:17.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.170+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30003a920 con 0x7fd3000055e0 2021-04-03T03:17:17.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.170+0000 7fd3057fa700 1 -- 172.21.2.111:0/651228330 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd2fc01b030 con 0x7fd3000055e0 2021-04-03T03:17:17.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.170+0000 7fd3057fa700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2e8056ec0 0x7fd2e8059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.170+0000 7fd3057fa700 1 -- 172.21.2.111:0/651228330 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd2fc06fd40 con 0x7fd3000055e0 2021-04-03T03:17:17.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.170+0000 7fd307fff700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2e8056ec0 0x7fd2e8059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.170+0000 7fd307fff700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2e8056ec0 0x7fd2e8059380 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fd2f801c000 tx=0x7fd2f8019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.170+0000 7fd3057fa700 1 -- 172.21.2.111:0/651228330 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd2fc045020 con 0x7fd3000055e0 2021-04-03T03:17:17.299 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.298+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd300004d90 con 0x7fd3000055e0 2021-04-03T03:17:17.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.302+0000 7fd3057fa700 1 -- 172.21.2.111:0/651228330 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v43) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd2fc019030 con 0x7fd3000055e0 2021-04-03T03:17:17.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.302+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2e8056ec0 msgr2=0x7fd2e8059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.302+0000 7fd30d01d700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2e8056ec0 0x7fd2e8059380 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fd2f801c000 tx=0x7fd2f8019000).stop 2021-04-03T03:17:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.302+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3000055e0 msgr2=0x7fd3000059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.302+0000 7fd30d01d700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3000055e0 0x7fd3000059c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fd2fc00d000 tx=0x7fd2fc0076d0).stop 2021-04-03T03:17:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 shutdown_connections 2021-04-03T03:17:17.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3000dfd10 0x7fd3000e0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3000055e0 0x7fd3000059c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2e8056ec0 0x7fd2e8059380 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 --2- 172.21.2.111:0/651228330 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd30003b910 0x7fd3000c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 >> 172.21.2.111:0/651228330 conn(0x7fd30000d0c0 msgr2=0x7fd3000f10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 shutdown_connections 2021-04-03T03:17:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.306+0000 7fd30d01d700 1 -- 172.21.2.111:0/651228330 wait complete. 2021-04-03T03:17:17.308 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:17:17.312 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:17.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 Processor -- start 2021-04-03T03:17:17.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- start start 2021-04-03T03:17:17.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fca0403f940 0x7fca040d5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca040d24a0 0x7fca0403e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca04044a00 con 0x7fca04048f00 2021-04-03T03:17:17.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca040492e0 con 0x7fca0403f940 2021-04-03T03:17:17.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca040d5380 con 0x7fca040d24a0 2021-04-03T03:17:17.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca110e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca040d24a0 0x7fca0403e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca108e3700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fca04048f00 0x7fca040d1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:43394/0 (socket says 172.21.2.111:43394) 2021-04-03T03:17:17.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca110e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca040d24a0 0x7fca0403e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:41648/0 (socket says 172.21.2.111:41648) 2021-04-03T03:17:17.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca108e3700 1 -- 172.21.2.111:0/2180326621 learned_addr learned my addr 172.21.2.111:0/2180326621 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:17.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca0bfff700 1 --2- 172.21.2.111:0/2180326621 >> v2:172.21.2.115:3300/0 conn(0x7fca0403f940 0x7fca040d5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.373 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca110e4700 1 -- 172.21.2.111:0/2180326621 >> v2:172.21.2.115:3300/0 conn(0x7fca0403f940 msgr2=0x7fca040d5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca110e4700 1 --2- 172.21.2.111:0/2180326621 >> v2:172.21.2.115:3300/0 conn(0x7fca0403f940 0x7fca040d5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca110e4700 1 -- 172.21.2.111:0/2180326621 >> v1:172.21.2.115:6789/0 conn(0x7fca04048f00 legacy=0x7fca040d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca110e4700 1 -- 172.21.2.111:0/2180326621 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040d5620 con 0x7fca040d24a0 2021-04-03T03:17:17.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca110e4700 1 --2- 172.21.2.111:0/2180326621 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca040d24a0 0x7fca0403e910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fca0c06b5d0 tx=0x7fca0c05ad70).ready entity=mon.2 client_cookie=e70efd74f60be1f7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca0b7fe700 1 -- 172.21.2.111:0/2180326621 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca0c05bda0 con 0x7fca040d24a0 2021-04-03T03:17:17.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca0b7fe700 1 -- 172.21.2.111:0/2180326621 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c05df60 con 0x7fca040d24a0 2021-04-03T03:17:17.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- 172.21.2.111:0/2180326621 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca040d24a0 msgr2=0x7fca0403e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 --2- 172.21.2.111:0/2180326621 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca040d24a0 0x7fca0403e910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fca0c06b5d0 tx=0x7fca0c05ad70).stop 2021-04-03T03:17:17.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- 172.21.2.111:0/2180326621 shutdown_connections 2021-04-03T03:17:17.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 --2- 172.21.2.111:0/2180326621 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca040d24a0 0x7fca0403e910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 --2- 172.21.2.111:0/2180326621 >> v2:172.21.2.115:3300/0 conn(0x7fca0403f940 0x7fca040d5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- 172.21.2.111:0/2180326621 >> 172.21.2.111:0/2180326621 conn(0x7fca0400f320 msgr2=0x7fca040484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:17.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- 172.21.2.111:0/2180326621 shutdown_connections 2021-04-03T03:17:17.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- 172.21.2.111:0/2180326621 wait complete. 2021-04-03T03:17:17.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 Processor -- start 2021-04-03T03:17:17.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.370+0000 7fca118e5700 1 -- start start 2021-04-03T03:17:17.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0403f940 0x7fca04046ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca04048f00 0x7fca0403c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca040d24a0 0x7fca0403cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040da200 con 0x7fca040d24a0 2021-04-03T03:17:17.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040d5d20 con 0x7fca0403f940 2021-04-03T03:17:17.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca040d5fc0 con 0x7fca04048f00 2021-04-03T03:17:17.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca040d24a0 0x7fca0403cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca040d24a0 0x7fca0403cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58306/0 (socket says 172.21.2.111:58306) 2021-04-03T03:17:17.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 -- 172.21.2.111:0/1325068859 learned_addr learned my addr 172.21.2.111:0/1325068859 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:17.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca0bfff700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca04048f00 0x7fca0403c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca108e3700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0403f940 0x7fca04046ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 -- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca04048f00 msgr2=0x7fca0403c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca04048f00 0x7fca0403c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 -- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0403f940 msgr2=0x7fca04046ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0403f940 0x7fca04046ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 -- 172.21.2.111:0/1325068859 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040d9e10 con 0x7fca040d24a0 2021-04-03T03:17:17.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca110e4700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca040d24a0 0x7fca0403cc20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7fca0c06b5d0 tx=0x7fca0c064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca09ffb700 1 -- 172.21.2.111:0/1325068859 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca0c0441b0 con 0x7fca040d24a0 2021-04-03T03:17:17.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca09ffb700 1 -- 172.21.2.111:0/1325068859 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c05f2c0 con 0x7fca040d24a0 2021-04-03T03:17:17.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca040d1640 con 0x7fca040d24a0 2021-04-03T03:17:17.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca040475c0 con 0x7fca040d24a0 2021-04-03T03:17:17.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca09ffb700 1 -- 172.21.2.111:0/1325068859 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca0c05f620 con 0x7fca040d24a0 2021-04-03T03:17:17.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca040d0f70 con 0x7fca040d24a0 2021-04-03T03:17:17.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca09ffb700 1 -- 172.21.2.111:0/1325068859 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fca0c08c020 con 0x7fca040d24a0 2021-04-03T03:17:17.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca09ffb700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9ec056fb0 0x7fc9ec059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca09ffb700 1 -- 172.21.2.111:0/1325068859 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fca0c0e0b80 con 0x7fca040d24a0 2021-04-03T03:17:17.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca108e3700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9ec056fb0 0x7fc9ec059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.374+0000 7fca108e3700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9ec056fb0 0x7fc9ec059470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fc9fc00d000 tx=0x7fc9fc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.386 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.378+0000 7fca09ffb700 1 -- 172.21.2.111:0/1325068859 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fca0c0b5ef0 con 0x7fca040d24a0 2021-04-03T03:17:17.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca040da060 con 0x7fca040d24a0 2021-04-03T03:17:17.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca09ffb700 1 -- 172.21.2.111:0/1325068859 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fca040da060 con 0x7fca040d24a0 2021-04-03T03:17:17.503 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:17.503 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":63,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3135556983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3135556983},{"type":"v1","addr":"172.21.2.115:6833","nonce":3135556983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:15.307326+0000","modified":"2021-04-03T03:17:16.311289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8250":{"gid":8250,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1485494348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1485494348},{"type":"v1","addr":"172.21.2.111:6835","nonce":1485494348}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-04-03T03:17:17.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9ec056fb0 msgr2=0x7fc9ec059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9ec056fb0 0x7fc9ec059470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fc9fc00d000 tx=0x7fc9fc009040).stop 2021-04-03T03:17:17.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca040d24a0 msgr2=0x7fca0403cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca040d24a0 0x7fca0403cc20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7fca0c06b5d0 tx=0x7fca0c064030).stop 2021-04-03T03:17:17.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 shutdown_connections 2021-04-03T03:17:17.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fca040d24a0 0x7fca0403cc20 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc9ec056fb0 0x7fc9ec059470 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fca0403f940 0x7fca04046ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 --2- 172.21.2.111:0/1325068859 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fca04048f00 0x7fca0403c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 >> 172.21.2.111:0/1325068859 conn(0x7fca0400f320 msgr2=0x7fca040d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:17.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 shutdown_connections 2021-04-03T03:17:17.507 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.502+0000 7fca118e5700 1 -- 172.21.2.111:0/1325068859 wait complete. 2021-04-03T03:17:17.507 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 63 2021-04-03T03:17:17.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:17:15.307326+0000', 'modified': '2021-04-03T03:17:16.311289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8250}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8250': {'gid': 8250, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1485494348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1485494348}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1485494348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:17:17.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:17:17.713 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:17:17.785 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":8250,"want_state":"up:active","state":"up:active","rank_uptime":2.4705468220000002,"mdsmap_epoch":63,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":6.6155369329999996} 2021-04-03T03:17:17.785 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 8250, "mdsmap_epoch": 63, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 2.470546822, "state": "up:active", "uptime": 6.615536933, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:17:17.786 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:17.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 Processor -- start 2021-04-03T03:17:17.841 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- start start 2021-04-03T03:17:17.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f724c006e30 0x7f724c0df8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.842 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c044e10 con 0x7f724c0069d0 2021-04-03T03:17:17.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c02d8d0 con 0x7f724c02cbf0 2021-04-03T03:17:17.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724c02cfd0 con 0x7f724c006e30 2021-04-03T03:17:17.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7253fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f724c0069d0 0x7f724c0d7aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59108/0 (socket says 172.21.2.111:59108) 2021-04-03T03:17:17.843 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7253fff700 1 -- 172.21.2.111:0/2992712711 learned_addr learned my addr 172.21.2.111:0/2992712711 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:17.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939787992 0 0) 0x7f724c044e10 con 0x7f724c0069d0 2021-04-03T03:17:17.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 --2- 172.21.2.111:0/2992712711 >> v2:172.21.2.115:3300/0 conn(0x7f724c006e30 0x7f724c0df8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7238003240 con 0x7f724c0069d0 2021-04-03T03:17:17.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3869222541 0 0) 0x7f7238003240 con 0x7f724c0069d0 2021-04-03T03:17:17.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 >> v2:172.21.2.115:3300/0 conn(0x7f724c006e30 msgr2=0x7f724c0df8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 --2- 172.21.2.111:0/2992712711 >> v2:172.21.2.115:3300/0 conn(0x7f724c006e30 0x7f724c0df8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 >> v1:172.21.2.115:6789/0 conn(0x7f724c02cbf0 legacy=0x7f724c0469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c046290 con 0x7f724c0069d0 2021-04-03T03:17:17.853 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f72480016d0 con 0x7f724c0069d0 2021-04-03T03:17:17.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 >> v1:172.21.2.111:6789/0 conn(0x7f724c0069d0 legacy=0x7f724c0d7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7238003780 0x7f7238003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f72380044e0 0x7f7238004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7238004e50 0x7f723800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c02cfd0 con 0x7f7238004e50 2021-04-03T03:17:17.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238005300 con 0x7f72380044e0 2021-04-03T03:17:17.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723800bc20 con 0x7f7238003780 2021-04-03T03:17:17.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7238004e50 0x7f723800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7258e73700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7238003780 0x7f7238003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 -- 172.21.2.111:0/2992712711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7238003780 msgr2=0x7f7238003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7238003780 0x7f7238003ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 -- 172.21.2.111:0/2992712711 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f72380044e0 msgr2=0x7f7238004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:17.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f72380044e0 0x7f7238004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:17.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 -- 172.21.2.111:0/2992712711 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7238003240 con 0x7f7238004e50 2021-04-03T03:17:17.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 -- 172.21.2.111:0/2992712711 reap_dead start 2021-04-03T03:17:17.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259674700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7238004e50 0x7f723800b730 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f724000a000 tx=0x7f7240008040).ready entity=mon.0 client_cookie=cdf7f108ea06ab5d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f724000da00 con 0x7f7238004e50 2021-04-03T03:17:17.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724000db60 con 0x7f7238004e50 2021-04-03T03:17:17.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f72537fe700 1 -- 172.21.2.111:0/2992712711 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7240013480 con 0x7f7238004e50 2021-04-03T03:17:17.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- 172.21.2.111:0/2992712711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7238004e50 msgr2=0x7f723800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7238004e50 0x7f723800b730 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f724000a000 tx=0x7f7240008040).stop 2021-04-03T03:17:17.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- 172.21.2.111:0/2992712711 shutdown_connections 2021-04-03T03:17:17.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 --2- 172.21.2.111:0/2992712711 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7238004e50 0x7f723800b730 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- 172.21.2.111:0/2992712711 >> 172.21.2.111:0/2992712711 conn(0x7f724c047170 msgr2=0x7f724c047580 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:17.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- 172.21.2.111:0/2992712711 shutdown_connections 2021-04-03T03:17:17.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.842+0000 7f7259e75700 1 -- 172.21.2.111:0/2992712711 wait complete. 2021-04-03T03:17:17.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 Processor -- start 2021-04-03T03:17:17.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 -- start start 2021-04-03T03:17:17.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7238004e50 0x7f724c049280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f724c041bf0 0x7f724c0434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f724c040b50 0x7f724c0439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c042090 con 0x7f724c040b50 2021-04-03T03:17:17.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c03fc70 con 0x7f7238004e50 2021-04-03T03:17:17.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c03fe60 con 0x7f724c041bf0 2021-04-03T03:17:17.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f724c040b50 0x7f724c0439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f724c040b50 0x7f724c0439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58328/0 (socket says 172.21.2.111:58328) 2021-04-03T03:17:17.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 -- 172.21.2.111:0/4172752092 learned_addr learned my addr 172.21.2.111:0/4172752092 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:17.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7253fff700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f724c041bf0 0x7f724c0434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 -- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f724c041bf0 msgr2=0x7f724c0434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.862 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f724c041bf0 0x7f724c0434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7258e73700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7238004e50 0x7f724c049280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 -- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7238004e50 msgr2=0x7f724c049280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7238004e50 0x7f724c049280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 -- 172.21.2.111:0/4172752092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c040100 con 0x7f724c040b50 2021-04-03T03:17:17.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7258e73700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7238004e50 0x7f724c049280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:17.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259674700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f724c040b50 0x7f724c0439e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f724c043240 tx=0x7f7240008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7251ffb700 1 -- 172.21.2.111:0/4172752092 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7240021070 con 0x7f724c040b50 2021-04-03T03:17:17.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7251ffb700 1 -- 172.21.2.111:0/4172752092 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240013e90 con 0x7f724c040b50 2021-04-03T03:17:17.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724c03f980 con 0x7f724c040b50 2021-04-03T03:17:17.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c043ed0 con 0x7f724c040b50 2021-04-03T03:17:17.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7251ffb700 1 -- 172.21.2.111:0/4172752092 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f724001d600 con 0x7f724c040b50 2021-04-03T03:17:17.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724c004da0 con 0x7f724c040b50 2021-04-03T03:17:17.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7251ffb700 1 -- 172.21.2.111:0/4172752092 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f724001d760 con 0x7f724c040b50 2021-04-03T03:17:17.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7251ffb700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f723c056f10 0x7f723c0593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:17.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7251ffb700 1 -- 172.21.2.111:0/4172752092 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f72400712e0 con 0x7f724c040b50 2021-04-03T03:17:17.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7258e73700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f723c056f10 0x7f723c0593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:17.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.846+0000 7f7258e73700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f723c056f10 0x7f723c0593d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f7254064000 tx=0x7f7254064040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:17.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.850+0000 7f7251ffb700 1 -- 172.21.2.111:0/4172752092 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f72400465c0 con 0x7f724c040b50 2021-04-03T03:17:17.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f724c0083f0 con 0x7f724c040b50 2021-04-03T03:17:17.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7251ffb700 1 -- 172.21.2.111:0/4172752092 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f724002ffb0 con 0x7f724c040b50 2021-04-03T03:17:17.946 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:17.946 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":141,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:16.311403+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:13.328557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:14.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:17.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f723c056f10 msgr2=0x7f723c0593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f723c056f10 0x7f723c0593d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f7254064000 tx=0x7f7254064040).stop 2021-04-03T03:17:17.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f724c040b50 msgr2=0x7f724c0439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:17.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f724c040b50 0x7f724c0439e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f724c043240 tx=0x7f7240008040).stop 2021-04-03T03:17:17.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 shutdown_connections 2021-04-03T03:17:17.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f724c040b50 0x7f724c0439e0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f723c056f10 0x7f723c0593d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7238004e50 0x7f724c049280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 --2- 172.21.2.111:0/4172752092 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f724c041bf0 0x7f724c0434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:17.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 >> 172.21.2.111:0/4172752092 conn(0x7f724c047170 msgr2=0x7f724c049bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:17.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 shutdown_connections 2021-04-03T03:17:17.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:17.946+0000 7f7259e75700 1 -- 172.21.2.111:0/4172752092 wait complete. 2021-04-03T03:17:17.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:17:13.328557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:17:14.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:17:17.955 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:17:18.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6bde7700 1 Processor -- start 2021-04-03T03:17:18.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6bde7700 1 -- start start 2021-04-03T03:17:18.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6bde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 0x7f4c6c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6bde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4c6c0add50 0x7f4c6c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6bde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c0607d0 con 0x7f4c6c0b17b0 2021-04-03T03:17:18.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6bde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c1451c0 con 0x7f4c6c0add50 2021-04-03T03:17:18.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6bde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c14b090 con 0x7f4c6c0603f0 2021-04-03T03:17:18.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6a5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4c6c0add50 0x7f4c6c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6a5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f4c6c0add50 0x7f4c6c144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58334/0 (socket says 172.21.2.111:58334) 2021-04-03T03:17:18.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.010+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/4210578426 learned_addr learned my addr 172.21.2.111:0/4210578426 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/4210578426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 msgr2=0x7f4c6c1447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:18.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 --2- 172.21.2.111:0/4210578426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 0x7f4c6c1447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/4210578426 >> v1:172.21.2.115:6789/0 conn(0x7f4c6c0b17b0 legacy=0x7f4c6c147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:18.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/4210578426 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c14b330 con 0x7f4c6c0add50 2021-04-03T03:17:18.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 --2- 172.21.2.111:0/4210578426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c144cd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f4c6c0b5600 tx=0x7f4c54006e90).ready entity=mon.0 client_cookie=c30f8666d446a5b3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c69de3700 1 -- 172.21.2.111:0/4210578426 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c540098e0 con 0x7f4c6c0add50 2021-04-03T03:17:18.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c69de3700 1 -- 172.21.2.111:0/4210578426 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54009a40 con 0x7f4c6c0add50 2021-04-03T03:17:18.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c69de3700 1 -- 172.21.2.111:0/4210578426 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c54009de0 con 0x7f4c6c0add50 2021-04-03T03:17:18.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- 172.21.2.111:0/4210578426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 msgr2=0x7f4c6c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/4210578426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c144cd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f4c6c0b5600 tx=0x7f4c54006e90).stop 2021-04-03T03:17:18.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- 172.21.2.111:0/4210578426 shutdown_connections 2021-04-03T03:17:18.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/4210578426 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 0x7f4c6c1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/4210578426 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c144cd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- 172.21.2.111:0/4210578426 >> 172.21.2.111:0/4210578426 conn(0x7f4c6c0b61c0 msgr2=0x7f4c6c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- 172.21.2.111:0/4210578426 shutdown_connections 2021-04-03T03:17:18.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- 172.21.2.111:0/4210578426 wait complete. 2021-04-03T03:17:18.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 Processor -- start 2021-04-03T03:17:18.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- start start 2021-04-03T03:17:18.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 0x7f4c6c0ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c065210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4c6c0b17b0 0x7f4c6c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0a9640 con 0x7f4c6c0add50 2021-04-03T03:17:18.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c064b40 con 0x7f4c6c0b17b0 2021-04-03T03:17:18.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0aabc0 con 0x7f4c6c0603f0 2021-04-03T03:17:18.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c065210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c065210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58342/0 (socket says 172.21.2.111:58342) 2021-04-03T03:17:18.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/2491818249 learned_addr learned my addr 172.21.2.111:0/2491818249 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/2491818249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 msgr2=0x7f4c6c0ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 0x7f4c6c0ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/2491818249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4c6c0b17b0 msgr2=0x7f4c6c156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4c6c0b17b0 0x7f4c6c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 -- 172.21.2.111:0/2491818249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c064d70 con 0x7f4c6c0add50 2021-04-03T03:17:18.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6a5e4700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c065210 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f4c6c0b5600 tx=0x7f4c54015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c63fff700 1 -- 172.21.2.111:0/2491818249 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c54011e30 con 0x7f4c6c0add50 2021-04-03T03:17:18.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c63fff700 1 -- 172.21.2.111:0/2491818249 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5401e460 con 0x7f4c6c0add50 2021-04-03T03:17:18.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c63fff700 1 -- 172.21.2.111:0/2491818249 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c5401e7c0 con 0x7f4c6c0add50 2021-04-03T03:17:18.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c0a9ce0 con 0x7f4c6c0add50 2021-04-03T03:17:18.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.014+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c0aa7c0 con 0x7f4c6c0add50 2021-04-03T03:17:18.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.018+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c065890 con 0x7f4c6c0add50 2021-04-03T03:17:18.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.018+0000 7f4c63fff700 1 -- 172.21.2.111:0/2491818249 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4c5401a040 con 0x7f4c6c0add50 2021-04-03T03:17:18.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.018+0000 7f4c63fff700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4c50057050 0x7f4c50059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.018+0000 7f4c63fff700 1 -- 172.21.2.111:0/2491818249 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f4c540713b0 con 0x7f4c6c0add50 2021-04-03T03:17:18.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.018+0000 7f4c6ade5700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4c50057050 0x7f4c50059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.018+0000 7f4c6ade5700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4c50057050 0x7f4c50059510 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f4c5c002f70 tx=0x7f4c5c006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.018+0000 7f4c63fff700 1 -- 172.21.2.111:0/2491818249 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4c540467e0 con 0x7f4c6c0add50 2021-04-03T03:17:18.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.110+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4c6c064f00 con 0x7f4c6c0add50 2021-04-03T03:17:18.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c63fff700 1 -- 172.21.2.111:0/2491818249 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4c540230b0 con 0x7f4c6c0add50 2021-04-03T03:17:18.114 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:17:18.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4c50057050 msgr2=0x7f4c50059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4c50057050 0x7f4c50059510 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f4c5c002f70 tx=0x7f4c5c006990).stop 2021-04-03T03:17:18.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 msgr2=0x7f4c6c065210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c065210 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f4c6c0b5600 tx=0x7f4c54015000).stop 2021-04-03T03:17:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 shutdown_connections 2021-04-03T03:17:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4c50057050 0x7f4c50059510 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4c6c0b17b0 0x7f4c6c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4c6c0603f0 0x7f4c6c0ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 --2- 172.21.2.111:0/2491818249 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4c6c0add50 0x7f4c6c065210 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 >> 172.21.2.111:0/2491818249 conn(0x7f4c6c0b61c0 msgr2=0x7f4c6c144a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 shutdown_connections 2021-04-03T03:17:18.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.114+0000 7f4c6bde7700 1 -- 172.21.2.111:0/2491818249 wait complete. 2021-04-03T03:17:18.121 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:17:18.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 Processor -- start 2021-04-03T03:17:18.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 -- start start 2021-04-03T03:17:18.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 0x7fe530147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe530144ef0 0x7fe530147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5300607d0 con 0x7fe5300603f0 2021-04-03T03:17:18.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53014b090 con 0x7fe530144ef0 2021-04-03T03:17:18.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53014b330 con 0x7fe5300add50 2021-04-03T03:17:18.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe530144ef0 0x7fe530147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe530144ef0 0x7fe530147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58350/0 (socket says 172.21.2.111:58350) 2021-04-03T03:17:18.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 -- 172.21.2.111:0/2226908198 learned_addr learned my addr 172.21.2.111:0/2226908198 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 -- 172.21.2.111:0/2226908198 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 msgr2=0x7fe530147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 --2- 172.21.2.111:0/2226908198 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 0x7fe530147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 -- 172.21.2.111:0/2226908198 >> v1:172.21.2.115:6789/0 conn(0x7fe5300603f0 legacy=0x7fe5301447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 -- 172.21.2.111:0/2226908198 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53014b5d0 con 0x7fe530144ef0 2021-04-03T03:17:18.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52ed9d700 1 --2- 172.21.2.111:0/2226908198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe530144ef0 0x7fe530147c30 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7fe5300b5600 tx=0x7fe528009270).ready entity=mon.0 client_cookie=2b857206dded3af7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52d59a700 1 -- 172.21.2.111:0/2226908198 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe52800bbc0 con 0x7fe530144ef0 2021-04-03T03:17:18.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52d59a700 1 -- 172.21.2.111:0/2226908198 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52800bd20 con 0x7fe530144ef0 2021-04-03T03:17:18.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52d59a700 1 -- 172.21.2.111:0/2226908198 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe528011400 con 0x7fe530144ef0 2021-04-03T03:17:18.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 -- 172.21.2.111:0/2226908198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe530144ef0 msgr2=0x7fe530147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.178+0000 7fe52f59e700 1 --2- 172.21.2.111:0/2226908198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe530144ef0 0x7fe530147c30 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7fe5300b5600 tx=0x7fe528009270).stop 2021-04-03T03:17:18.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- 172.21.2.111:0/2226908198 shutdown_connections 2021-04-03T03:17:18.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 --2- 172.21.2.111:0/2226908198 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe530144ef0 0x7fe530147c30 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 --2- 172.21.2.111:0/2226908198 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 0x7fe530147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- 172.21.2.111:0/2226908198 >> 172.21.2.111:0/2226908198 conn(0x7fe5300b61c0 msgr2=0x7fe53005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- 172.21.2.111:0/2226908198 shutdown_connections 2021-04-03T03:17:18.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- 172.21.2.111:0/2226908198 wait complete. 2021-04-03T03:17:18.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 Processor -- start 2021-04-03T03:17:18.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- start start 2021-04-03T03:17:18.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5300603f0 0x7fe5300b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 0x7fe5300ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe530144ef0 0x7fe5300a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300abb80 con 0x7fe5300603f0 2021-04-03T03:17:18.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300abe20 con 0x7fe530144ef0 2021-04-03T03:17:18.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300b42e0 con 0x7fe5300add50 2021-04-03T03:17:18.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5300603f0 0x7fe5300b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5300603f0 0x7fe5300b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58354/0 (socket says 172.21.2.111:58354) 2021-04-03T03:17:18.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 -- 172.21.2.111:0/1964773411 learned_addr learned my addr 172.21.2.111:0/1964773411 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 -- 172.21.2.111:0/1964773411 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 msgr2=0x7fe5300ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 0x7fe5300ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 -- 172.21.2.111:0/1964773411 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe530144ef0 msgr2=0x7fe5300a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe530144ef0 0x7fe5300a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 -- 172.21.2.111:0/1964773411 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530147760 con 0x7fe5300603f0 2021-04-03T03:17:18.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52e59c700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5300603f0 0x7fe5300b4780 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fe520002f50 tx=0x7fe520006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe5277fe700 1 -- 172.21.2.111:0/1964773411 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe52000d4a0 con 0x7fe5300603f0 2021-04-03T03:17:18.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe5277fe700 1 -- 172.21.2.111:0/1964773411 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52000d600 con 0x7fe5300603f0 2021-04-03T03:17:18.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe5277fe700 1 -- 172.21.2.111:0/1964773411 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe52000d960 con 0x7fe5300603f0 2021-04-03T03:17:18.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5300ab730 con 0x7fe5300603f0 2021-04-03T03:17:18.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53015a120 con 0x7fe5300603f0 2021-04-03T03:17:18.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5300b44a0 con 0x7fe5300603f0 2021-04-03T03:17:18.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe5277fe700 1 -- 172.21.2.111:0/1964773411 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe52000f070 con 0x7fe5300603f0 2021-04-03T03:17:18.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe5277fe700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe514057000 0x7fe5140594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe5277fe700 1 -- 172.21.2.111:0/1964773411 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe52006fe60 con 0x7fe5300603f0 2021-04-03T03:17:18.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.182+0000 7fe52dd9b700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe514057000 0x7fe5140594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.186+0000 7fe52dd9b700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe514057000 0x7fe5140594c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fe518002f30 tx=0x7fe518006a70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.186+0000 7fe5277fe700 1 -- 172.21.2.111:0/1964773411 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe520045140 con 0x7fe5300603f0 2021-04-03T03:17:18.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe530057660 con 0x7fe5300603f0 2021-04-03T03:17:18.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe5277fe700 1 -- 172.21.2.111:0/1964773411 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe520048760 con 0x7fe5300603f0 2021-04-03T03:17:18.280 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:18.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe514057000 msgr2=0x7fe5140594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe514057000 0x7fe5140594c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fe518002f30 tx=0x7fe518006a70).stop 2021-04-03T03:17:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5300603f0 msgr2=0x7fe5300b4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5300603f0 0x7fe5300b4780 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7fe520002f50 tx=0x7fe520006dd0).stop 2021-04-03T03:17:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 shutdown_connections 2021-04-03T03:17:18.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe514057000 0x7fe5140594c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe530144ef0 0x7fe5300a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5300603f0 0x7fe5300b4780 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 --2- 172.21.2.111:0/1964773411 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5300add50 0x7fe5300ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 >> 172.21.2.111:0/1964773411 conn(0x7fe5300b61c0 msgr2=0x7fe53005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 shutdown_connections 2021-04-03T03:17:18.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.278+0000 7fe52f59e700 1 -- 172.21.2.111:0/1964773411 wait complete. 2021-04-03T03:17:18.288 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:17:18.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 Processor -- start 2021-04-03T03:17:18.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- start start 2021-04-03T03:17:18.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7c840603f0 0x7f7c841447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7c840add50 0x7f7c84144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 0x7f7c84147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c841451c0 con 0x7f7c840add50 2021-04-03T03:17:18.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c840607d0 con 0x7f7c840603f0 2021-04-03T03:17:18.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8414b090 con 0x7f7c840b17b0 2021-04-03T03:17:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7c840add50 0x7f7c84144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7c840add50 0x7f7c84144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58364/0 (socket says 172.21.2.111:58364) 2021-04-03T03:17:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 -- 172.21.2.111:0/947219057 learned_addr learned my addr 172.21.2.111:0/947219057 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 -- 172.21.2.111:0/947219057 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 msgr2=0x7f7c84147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 --2- 172.21.2.111:0/947219057 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 0x7f7c84147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 -- 172.21.2.111:0/947219057 >> v2:172.21.2.115:3300/0 conn(0x7f7c840603f0 msgr2=0x7f7c841447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 --2- 172.21.2.111:0/947219057 >> v2:172.21.2.115:3300/0 conn(0x7f7c840603f0 0x7f7c841447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 -- 172.21.2.111:0/947219057 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8414b330 con 0x7f7c840add50 2021-04-03T03:17:18.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c88fad700 1 --2- 172.21.2.111:0/947219057 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840add50 0x7f7c84144cd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f7c74002f30 tx=0x7f7c74006e90).ready entity=mon.0 client_cookie=89e6fad78f2321ca server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c73fff700 1 -- 172.21.2.111:0/947219057 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c74009920 con 0x7f7c840add50 2021-04-03T03:17:18.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c73fff700 1 -- 172.21.2.111:0/947219057 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c74009a80 con 0x7f7c840add50 2021-04-03T03:17:18.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c73fff700 1 -- 172.21.2.111:0/947219057 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c74009de0 con 0x7f7c840add50 2021-04-03T03:17:18.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/947219057 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840add50 msgr2=0x7f7c84144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/947219057 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840add50 0x7f7c84144cd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f7c74002f30 tx=0x7f7c74006e90).stop 2021-04-03T03:17:18.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/947219057 shutdown_connections 2021-04-03T03:17:18.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/947219057 >> v2:172.21.2.115:3300/0 conn(0x7f7c840603f0 0x7f7c841447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/947219057 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 0x7f7c84147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/947219057 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840add50 0x7f7c84144cd0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/947219057 >> 172.21.2.111:0/947219057 conn(0x7f7c840b61c0 msgr2=0x7f7c8405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/947219057 shutdown_connections 2021-04-03T03:17:18.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/947219057 wait complete. 2021-04-03T03:17:18.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.346+0000 7f7c8a7b0700 1 Processor -- start 2021-04-03T03:17:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 -- start start 2021-04-03T03:17:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840603f0 0x7f7c84062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c840add50 0x7f7c84147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 0x7f7c84143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840ab1f0 con 0x7f7c840603f0 2021-04-03T03:17:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840ab490 con 0x7f7c840add50 2021-04-03T03:17:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840ab730 con 0x7f7c840b17b0 2021-04-03T03:17:18.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840603f0 0x7f7c84062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840603f0 0x7f7c84062c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58366/0 (socket says 172.21.2.111:58366) 2021-04-03T03:17:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 -- 172.21.2.111:0/2620206250 learned_addr learned my addr 172.21.2.111:0/2620206250 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c89faf700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 0x7f7c84143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c88fad700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c840add50 0x7f7c84147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 -- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 msgr2=0x7f7c84143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 0x7f7c84143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 -- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c840add50 msgr2=0x7f7c84147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c840add50 0x7f7c84147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 -- 172.21.2.111:0/2620206250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840abbd0 con 0x7f7c840603f0 2021-04-03T03:17:18.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c897ae700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840603f0 0x7f7c84062c30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f7c7c0032b0 tx=0x7f7c7c009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c727fc700 1 -- 172.21.2.111:0/2620206250 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c7c00c960 con 0x7f7c840603f0 2021-04-03T03:17:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c727fc700 1 -- 172.21.2.111:0/2620206250 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c00cac0 con 0x7f7c840603f0 2021-04-03T03:17:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c727fc700 1 -- 172.21.2.111:0/2620206250 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c7c00ce20 con 0x7f7c840603f0 2021-04-03T03:17:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c840631a0 con 0x7f7c840603f0 2021-04-03T03:17:18.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c84143e80 con 0x7f7c840603f0 2021-04-03T03:17:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c840ab8f0 con 0x7f7c840603f0 2021-04-03T03:17:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c727fc700 1 -- 172.21.2.111:0/2620206250 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7c7c01c070 con 0x7f7c840603f0 2021-04-03T03:17:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c727fc700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c64056ec0 0x7f7c64059380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c727fc700 1 -- 172.21.2.111:0/2620206250 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7c7c06fc20 con 0x7f7c840603f0 2021-04-03T03:17:18.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c88fad700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c64056ec0 0x7f7c64059380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.350+0000 7f7c88fad700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c64056ec0 0x7f7c64059380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f7c7401c000 tx=0x7f7c74015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.354+0000 7f7c727fc700 1 -- 172.21.2.111:0/2620206250 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7c7c045010 con 0x7f7c840603f0 2021-04-03T03:17:18.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c84057660 con 0x7f7c840603f0 2021-04-03T03:17:18.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c727fc700 1 -- 172.21.2.111:0/2620206250 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7c7c048630 con 0x7f7c840603f0 2021-04-03T03:17:18.446 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:18.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c64056ec0 msgr2=0x7f7c64059380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c64056ec0 0x7f7c64059380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f7c7401c000 tx=0x7f7c74015000).stop 2021-04-03T03:17:18.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840603f0 msgr2=0x7f7c84062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840603f0 0x7f7c84062c30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f7c7c0032b0 tx=0x7f7c7c009290).stop 2021-04-03T03:17:18.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 shutdown_connections 2021-04-03T03:17:18.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c840b17b0 0x7f7c84143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c840603f0 0x7f7c84062c30 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c64056ec0 0x7f7c64059380 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 --2- 172.21.2.111:0/2620206250 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c840add50 0x7f7c84147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 >> 172.21.2.111:0/2620206250 conn(0x7f7c840b61c0 msgr2=0x7f7c841448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 shutdown_connections 2021-04-03T03:17:18.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.446+0000 7f7c8a7b0700 1 -- 172.21.2.111:0/2620206250 wait complete. 2021-04-03T03:17:18.454 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:17:18.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 Processor -- start 2021-04-03T03:17:18.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 -- start start 2021-04-03T03:17:18.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f00440603f0 0x7f00441447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f00440b17b0 0x7f0044144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00440607d0 con 0x7f00440add50 2021-04-03T03:17:18.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00441451c0 con 0x7f00440603f0 2021-04-03T03:17:18.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004414b090 con 0x7f00440b17b0 2021-04-03T03:17:18.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f00440603f0 0x7f00441447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f00440603f0 0x7f00441447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58374/0 (socket says 172.21.2.111:58374) 2021-04-03T03:17:18.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 -- 172.21.2.111:0/1770649956 learned_addr learned my addr 172.21.2.111:0/1770649956 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 -- 172.21.2.111:0/1770649956 >> v2:172.21.2.115:3300/0 conn(0x7f00440b17b0 msgr2=0x7f0044144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 --2- 172.21.2.111:0/1770649956 >> v2:172.21.2.115:3300/0 conn(0x7f00440b17b0 0x7f0044144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 -- 172.21.2.111:0/1770649956 >> v1:172.21.2.115:6789/0 conn(0x7f00440add50 legacy=0x7f0044147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 -- 172.21.2.111:0/1770649956 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004414b330 con 0x7f00440603f0 2021-04-03T03:17:18.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00437fe700 1 --2- 172.21.2.111:0/1770649956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440603f0 0x7f00441447e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f00440ae520 tx=0x7f0034006e90).ready entity=mon.0 client_cookie=e8e006fd74597ec9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00427fc700 1 -- 172.21.2.111:0/1770649956 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0034009930 con 0x7f00440603f0 2021-04-03T03:17:18.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00427fc700 1 -- 172.21.2.111:0/1770649956 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0034009a90 con 0x7f00440603f0 2021-04-03T03:17:18.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f00427fc700 1 -- 172.21.2.111:0/1770649956 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0034009e30 con 0x7f00440603f0 2021-04-03T03:17:18.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 -- 172.21.2.111:0/1770649956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440603f0 msgr2=0x7f00441447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.517 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.510+0000 7f0048df3700 1 --2- 172.21.2.111:0/1770649956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440603f0 0x7f00441447e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f00440ae520 tx=0x7f0034006e90).stop 2021-04-03T03:17:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- 172.21.2.111:0/1770649956 shutdown_connections 2021-04-03T03:17:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 --2- 172.21.2.111:0/1770649956 >> v2:172.21.2.115:3300/0 conn(0x7f00440b17b0 0x7f0044144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 --2- 172.21.2.111:0/1770649956 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440603f0 0x7f00441447e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- 172.21.2.111:0/1770649956 >> 172.21.2.111:0/1770649956 conn(0x7f00440b61c0 msgr2=0x7f004405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- 172.21.2.111:0/1770649956 shutdown_connections 2021-04-03T03:17:18.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- 172.21.2.111:0/1770649956 wait complete. 2021-04-03T03:17:18.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 Processor -- start 2021-04-03T03:17:18.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- start start 2021-04-03T03:17:18.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00440603f0 0x7f0044142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440add50 0x7f00441419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f00440b17b0 0x7f0044141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00440a8c70 con 0x7f00440add50 2021-04-03T03:17:18.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00440a72d0 con 0x7f00440b17b0 2021-04-03T03:17:18.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00440a75c0 con 0x7f00440603f0 2021-04-03T03:17:18.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440add50 0x7f00441419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440add50 0x7f00441419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58382/0 (socket says 172.21.2.111:58382) 2021-04-03T03:17:18.523 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 -- 172.21.2.111:0/1616471777 learned_addr learned my addr 172.21.2.111:0/1616471777 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 -- 172.21.2.111:0/1616471777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00440603f0 msgr2=0x7f0044142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00440603f0 0x7f0044142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 -- 172.21.2.111:0/1616471777 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f00440b17b0 msgr2=0x7f0044141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f00440b17b0 0x7f0044141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 -- 172.21.2.111:0/1616471777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00440aa800 con 0x7f00440add50 2021-04-03T03:17:18.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0042ffd700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440add50 0x7f00441419e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f002c000f30 tx=0x7f002c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0040ff9700 1 -- 172.21.2.111:0/1616471777 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f002c00e4a0 con 0x7f00440add50 2021-04-03T03:17:18.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0040ff9700 1 -- 172.21.2.111:0/1616471777 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c00e600 con 0x7f00440add50 2021-04-03T03:17:18.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00440a7030 con 0x7f00440add50 2021-04-03T03:17:18.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0040ff9700 1 -- 172.21.2.111:0/1616471777 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f002c00e960 con 0x7f00440add50 2021-04-03T03:17:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0044067cd0 con 0x7f00440add50 2021-04-03T03:17:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0040ff9700 1 -- 172.21.2.111:0/1616471777 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f002c01a060 con 0x7f00440add50 2021-04-03T03:17:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00440aa650 con 0x7f00440add50 2021-04-03T03:17:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0040ff9700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0028056ea0 0x7f0028059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f0040ff9700 1 -- 172.21.2.111:0/1616471777 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f002c06fb10 con 0x7f00440add50 2021-04-03T03:17:18.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f00437fe700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0028056ea0 0x7f0028059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.514+0000 7f00437fe700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0028056ea0 0x7f0028059360 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f00440ae520 tx=0x7f0034015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.518+0000 7f0040ff9700 1 -- 172.21.2.111:0/1616471777 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f002c044f00 con 0x7f00440add50 2021-04-03T03:17:18.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00440aa990 con 0x7f00440add50 2021-04-03T03:17:18.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0040ff9700 1 -- 172.21.2.111:0/1616471777 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+10 (secure 0 0 0) 0x7f002c018030 con 0x7f00440add50 2021-04-03T03:17:18.612 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:18.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0028056ea0 msgr2=0x7f0028059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0028056ea0 0x7f0028059360 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f00440ae520 tx=0x7f0034015000).stop 2021-04-03T03:17:18.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440add50 msgr2=0x7f00441419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440add50 0x7f00441419e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f002c000f30 tx=0x7f002c007bb0).stop 2021-04-03T03:17:18.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 shutdown_connections 2021-04-03T03:17:18.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0028056ea0 0x7f0028059360 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.614 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f00440b17b0 0x7f0044141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f00440603f0 0x7f0044142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 --2- 172.21.2.111:0/1616471777 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f00440add50 0x7f00441419e0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 >> 172.21.2.111:0/1616471777 conn(0x7f00440b61c0 msgr2=0x7f004405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.615 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 shutdown_connections 2021-04-03T03:17:18.616 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.610+0000 7f0048df3700 1 -- 172.21.2.111:0/1616471777 wait complete. 2021-04-03T03:17:18.619 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:17:18.620 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:18.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504ffff700 1 Processor -- start 2021-04-03T03:17:18.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504ffff700 1 -- start start 2021-04-03T03:17:18.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50500b12b0 0x7f50501522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504ffff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50500607d0 con 0x7f505014b700 2021-04-03T03:17:18.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504ffff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50500b0eb0 con 0x7f50500603f0 2021-04-03T03:17:18.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50500af400 con 0x7f50500b12b0 2021-04-03T03:17:18.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504e7fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f505014b700 0x7f50500b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59184/0 (socket says 172.21.2.111:59184) 2021-04-03T03:17:18.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504e7fc700 1 -- 172.21.2.111:0/3484709820 learned_addr learned my addr 172.21.2.111:0/3484709820 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:18.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498545932 0 0) 0x7f50500607d0 con 0x7f505014b700 2021-04-03T03:17:18.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5030003240 con 0x7f505014b700 2021-04-03T03:17:18.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504f7fe700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50500b12b0 0x7f50501522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1505749534 0 0) 0x7f5030003240 con 0x7f505014b700 2021-04-03T03:17:18.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50500b12b0 msgr2=0x7f50501522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50500b12b0 0x7f50501522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 >> v1:172.21.2.115:6789/0 conn(0x7f50500603f0 legacy=0x7f50500b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500af6a0 con 0x7f505014b700 2021-04-03T03:17:18.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.674+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f5040001a50 con 0x7f505014b700 2021-04-03T03:17:18.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 >> v1:172.21.2.111:6789/0 conn(0x7f505014b700 legacy=0x7f50500b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50500b12b0 0x7f50300037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5030003d20 0x7f50300044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50300049d0 0x7f5030004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50500af400 con 0x7f5030003d20 2021-04-03T03:17:18.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50300042b0 con 0x7f50300049d0 2021-04-03T03:17:18.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503000b730 con 0x7f50500b12b0 2021-04-03T03:17:18.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5030003d20 0x7f50300044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 -- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50500b12b0 msgr2=0x7f50300037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f50500b12b0 0x7f50300037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 -- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50300049d0 msgr2=0x7f5030004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f50300049d0 0x7f5030004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 -- 172.21.2.111:0/3484709820 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5030003240 con 0x7f5030003d20 2021-04-03T03:17:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5030003d20 0x7f50300044e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f50500620f0 tx=0x7f5040007a50).ready entity=mon.0 client_cookie=ab4911dfe74994e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5040005b80 con 0x7f5030003d20 2021-04-03T03:17:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5040005ce0 con 0x7f5030003d20 2021-04-03T03:17:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504dffb700 1 -- 172.21.2.111:0/3484709820 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50400072c0 con 0x7f5030003d20 2021-04-03T03:17:18.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/3484709820 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5030003d20 msgr2=0x7f50300044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 --2- 172.21.2.111:0/3484709820 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5030003d20 0x7f50300044e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f50500620f0 tx=0x7f5040007a50).stop 2021-04-03T03:17:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504f7fe700 1 -- 172.21.2.111:0/3484709820 reap_dead start 2021-04-03T03:17:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/3484709820 shutdown_connections 2021-04-03T03:17:18.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/3484709820 >> 172.21.2.111:0/3484709820 conn(0x7f50500b61c0 msgr2=0x7f505005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/3484709820 shutdown_connections 2021-04-03T03:17:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/3484709820 wait complete. 2021-04-03T03:17:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 Processor -- start 2021-04-03T03:17:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- start start 2021-04-03T03:17:18.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f505014e460 0x7f50500ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50500aaec0 0x7f5050146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5050146cf0 0x7f5050064120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505014e840 con 0x7f50500aaec0 2021-04-03T03:17:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50500aa290 con 0x7f5050146cf0 2021-04-03T03:17:18.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5050159750 con 0x7f505014e460 2021-04-03T03:17:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50500aaec0 0x7f5050146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50500aaec0 0x7f5050146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58400/0 (socket says 172.21.2.111:58400) 2021-04-03T03:17:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 -- 172.21.2.111:0/2689623766 learned_addr learned my addr 172.21.2.111:0/2689623766 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 -- 172.21.2.111:0/2689623766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f505014e460 msgr2=0x7f50500ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f505014e460 0x7f50500ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 -- 172.21.2.111:0/2689623766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5050146cf0 msgr2=0x7f5050064120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5050146cf0 0x7f5050064120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 -- 172.21.2.111:0/2689623766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5050159910 con 0x7f50500aaec0 2021-04-03T03:17:18.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504e7fc700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50500aaec0 0x7f5050146800 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f50500620f0 tx=0x7f5040013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f5047fff700 1 -- 172.21.2.111:0/2689623766 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5040007810 con 0x7f50500aaec0 2021-04-03T03:17:18.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f5047fff700 1 -- 172.21.2.111:0/2689623766 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5040013450 con 0x7f50500aaec0 2021-04-03T03:17:18.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5050147170 con 0x7f50500aaec0 2021-04-03T03:17:18.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f5047fff700 1 -- 172.21.2.111:0/2689623766 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50400137b0 con 0x7f50500aaec0 2021-04-03T03:17:18.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50500647a0 con 0x7f50500aaec0 2021-04-03T03:17:18.694 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5050063f90 con 0x7f50500aaec0 2021-04-03T03:17:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.678+0000 7f5047fff700 1 -- 172.21.2.111:0/2689623766 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5040026070 con 0x7f50500aaec0 2021-04-03T03:17:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.682+0000 7f5047fff700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5034057050 0x7f5034059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.682+0000 7f5047fff700 1 -- 172.21.2.111:0/2689623766 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5040070d10 con 0x7f50500aaec0 2021-04-03T03:17:18.695 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.682+0000 7f504effd700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5034057050 0x7f5034059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:18.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.682+0000 7f504effd700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5034057050 0x7f5034059510 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f5038001f60 tx=0x7f50380077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:18.696 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.682+0000 7f5047fff700 1 -- 172.21.2.111:0/2689623766 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5040046100 con 0x7f50500aaec0 2021-04-03T03:17:18.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.802+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5050057660 con 0x7f50500aaec0 2021-04-03T03:17:18.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f5047fff700 1 -- 172.21.2.111:0/2689623766 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f5040049720 con 0x7f50500aaec0 2021-04-03T03:17:18.806 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:18.806 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":63,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3135556983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3135556983},{"type":"v1","addr":"172.21.2.115:6833","nonce":3135556983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:15.307326+0000","modified":"2021-04-03T03:17:16.311289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8250":{"gid":8250,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1485494348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1485494348},{"type":"v1","addr":"172.21.2.111:6835","nonce":1485494348}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-04-03T03:17:18.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5034057050 msgr2=0x7f5034059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5034057050 0x7f5034059510 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f5038001f60 tx=0x7f50380077d0).stop 2021-04-03T03:17:18.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50500aaec0 msgr2=0x7f5050146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:18.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50500aaec0 0x7f5050146800 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f50500620f0 tx=0x7f5040013000).stop 2021-04-03T03:17:18.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 shutdown_connections 2021-04-03T03:17:18.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5034057050 0x7f5034059510 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5050146cf0 0x7f5050064120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f505014e460 0x7f50500ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:18.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 --2- 172.21.2.111:0/2689623766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f50500aaec0 0x7f5050146800 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:18.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 >> 172.21.2.111:0/2689623766 conn(0x7f50500b61c0 msgr2=0x7f50500b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:18.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 shutdown_connections 2021-04-03T03:17:18.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:18.806+0000 7f504ffff700 1 -- 172.21.2.111:0/2689623766 wait complete. 2021-04-03T03:17:18.810 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 63 2021-04-03T03:17:19.015 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:17:15.307326+0000', 'modified': '2021-04-03T03:17:16.311289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8250}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8250': {'gid': 8250, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1485494348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1485494348}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1485494348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:17:19.015 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:17:19.016 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:17:19.088 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":8250,"want_state":"up:active","state":"up:active","rank_uptime":3.7738426309999999,"mdsmap_epoch":63,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":7.9188322940000004} 2021-04-03T03:17:19.088 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 8250, "mdsmap_epoch": 63, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 3.773842631, "state": "up:active", "uptime": 7.918832294, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:17:19.089 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:17:19.089 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:17:19.090 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:17:19.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:19.090 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:19.101 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:17:19.154 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:19.155 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:19.202 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:19.203 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:19.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:19.204 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:17:19.256 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:17:19.256 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:19.302 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:17:19.309 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:19.309 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:19.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[16111]: starting ceph client 2021-04-03T03:17:19.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:2021-04-03T03:17:19.341+0000 7f73212e10c0 -1 init, newargv = 0x5633e9fbdf50 newargc=9 2021-04-03T03:17:19.351 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:19.351 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:19.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:19.353 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:17:19.354 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[16111]: starting fuse 2021-04-03T03:17:19.356 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:17:19.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:17:19.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:19.358 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:17:19.358 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:17:19.359 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:17:19.359 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:17:19.359 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:17:19.360 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:19.360 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:17:19.361 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:17:19.361 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:17:19.362 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:19.362 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:17:19.362 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:17:19.363 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:17:19.363 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:17:19.364 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:19.364 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:17:19.365 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:17:19.365 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:17:19.365 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:17:19.366 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:17:19.366 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:19.367 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:17:19.367 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:17:19.425 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.16111.asok 2021-04-03T03:17:19.427 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.16111.asok 2021-04-03T03:17:19.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:19.428 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.16111.asok status 2021-04-03T03:17:19.527 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:17:19.527 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:17:19.528 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:17:19.528 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:17:19.528 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:17:19.528 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:17:19.529 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:17:19.529 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "16111", 2021-04-03T03:17:19.529 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:17:19.529 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:19.529 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:17:19.530 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:17:19.530 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 5701, 2021-04-03T03:17:19.530 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:17:19.530 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:17:19.531 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:17:19.531 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 5701 2021-04-03T03:17:19.531 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:19.531 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:17:19.531 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:17:19.532 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:17:19.532 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2354170985 2021-04-03T03:17:19.532 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:17:19.532 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:19.533 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:17:19.533 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:17:19.533 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:17:19.533 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 2354170985 2021-04-03T03:17:19.533 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:19.534 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.5701 172.21.2.126:0/2354170985", 2021-04-03T03:17:19.534 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/2354170985", 2021-04-03T03:17:19.534 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:17:19.534 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 63, 2021-04-03T03:17:19.535 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 141, 2021-04-03T03:17:19.535 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:17:19.535 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:17:19.535 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:17:19.535 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:17:19.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:19.537 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:19.538 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:19.543 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:17:19.543 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:17:19.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:19.544 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:19.593 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:19.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.650+0000 7f3e78a44700 1 Processor -- start 2021-04-03T03:17:19.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.650+0000 7f3e78a44700 1 -- start start 2021-04-03T03:17:19.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.650+0000 7f3e78a44700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c046b70 0x7f3e6c0d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.650+0000 7f3e78a44700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3e6c042300 0x7f3e6c043920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c0d4ef0 con 0x7f3e6c045690 2021-04-03T03:17:19.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c046f50 con 0x7f3e6c042300 2021-04-03T03:17:19.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d4020 con 0x7f3e6c046b70 2021-04-03T03:17:19.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3e6c045690 0x7f3e6c0d33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59204/0 (socket says 172.21.2.111:59204) 2021-04-03T03:17:19.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 -- 172.21.2.111:0/1331566803 learned_addr learned my addr 172.21.2.111:0/1331566803 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:19.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376064080 0 0) 0x7f3e6c0d4ef0 con 0x7f3e6c045690 2021-04-03T03:17:19.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e58003240 con 0x7f3e6c045690 2021-04-03T03:17:19.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 --2- 172.21.2.111:0/1331566803 >> v2:172.21.2.115:3300/0 conn(0x7f3e6c042300 0x7f3e6c043920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e737fe700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c046b70 0x7f3e6c0d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2652538595 0 0) 0x7f3e58003240 con 0x7f3e6c045690 2021-04-03T03:17:19.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c046b70 msgr2=0x7f3e6c0d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:19.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c046b70 0x7f3e6c0d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 >> v2:172.21.2.115:3300/0 conn(0x7f3e6c042300 msgr2=0x7f3e6c043920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:19.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 --2- 172.21.2.111:0/1331566803 >> v2:172.21.2.115:3300/0 conn(0x7f3e6c042300 0x7f3e6c043920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0d42c0 con 0x7f3e6c045690 2021-04-03T03:17:19.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f3e740450f0 con 0x7f3e6c045690 2021-04-03T03:17:19.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 >> v1:172.21.2.111:6789/0 conn(0x7f3e6c045690 legacy=0x7f3e6c0d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e58003730 0x7f3e58003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3e58004060 0x7f3e58004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e58004aa0 0x7f3e58004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c046f50 con 0x7f3e58004aa0 2021-04-03T03:17:19.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d4020 con 0x7f3e58004060 2021-04-03T03:17:19.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5800b830 con 0x7f3e58003730 2021-04-03T03:17:19.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e58004aa0 0x7f3e58004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 -- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e58003730 msgr2=0x7f3e58003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:19.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e58003730 0x7f3e58003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 -- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3e58004060 msgr2=0x7f3e58004540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:19.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3e58004060 0x7f3e58004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 -- 172.21.2.111:0/1331566803 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0d42c0 con 0x7f3e58004aa0 2021-04-03T03:17:19.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e58004aa0 0x7f3e58004f20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f3e7405e060 tx=0x7f3e7405ad70).ready entity=mon.0 client_cookie=f16e31712d2bac7 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:19.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e7405b040 con 0x7f3e58004aa0 2021-04-03T03:17:19.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74061110 con 0x7f3e58004aa0 2021-04-03T03:17:19.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e727fc700 1 -- 172.21.2.111:0/1331566803 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e7405b1a0 con 0x7f3e58004aa0 2021-04-03T03:17:19.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- 172.21.2.111:0/1331566803 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e58004aa0 msgr2=0x7f3e58004f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 --2- 172.21.2.111:0/1331566803 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e58004aa0 0x7f3e58004f20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f3e7405e060 tx=0x7f3e7405ad70).stop 2021-04-03T03:17:19.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e73fff700 1 -- 172.21.2.111:0/1331566803 reap_dead start 2021-04-03T03:17:19.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- 172.21.2.111:0/1331566803 shutdown_connections 2021-04-03T03:17:19.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- 172.21.2.111:0/1331566803 >> 172.21.2.111:0/1331566803 conn(0x7f3e6c04a6e0 msgr2=0x7f3e6c04aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:19.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- 172.21.2.111:0/1331566803 shutdown_connections 2021-04-03T03:17:19.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- 172.21.2.111:0/1331566803 wait complete. 2021-04-03T03:17:19.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 Processor -- start 2021-04-03T03:17:19.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- start start 2021-04-03T03:17:19.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3e6c0d6700 0x7f3e6c0d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e6c0e3620 0x7f3e6c0d85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c0d8ae0 0x7f3e6c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d5e20 con 0x7f3e6c0e3620 2021-04-03T03:17:19.669 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0db1a0 con 0x7f3e6c0d6700 2021-04-03T03:17:19.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e78a44700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d6050 con 0x7f3e6c0d8ae0 2021-04-03T03:17:19.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e6c0e3620 0x7f3e6c0d85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e6c0e3620 0x7f3e6c0d85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58422/0 (socket says 172.21.2.111:58422) 2021-04-03T03:17:19.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 -- 172.21.2.111:0/3492991303 learned_addr learned my addr 172.21.2.111:0/3492991303 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:19.670 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 -- 172.21.2.111:0/3492991303 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c0d8ae0 msgr2=0x7f3e6c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c0d8ae0 0x7f3e6c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 -- 172.21.2.111:0/3492991303 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3e6c0d6700 msgr2=0x7f3e6c0d6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3e6c0d6700 0x7f3e6c0d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.671 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 -- 172.21.2.111:0/3492991303 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0e3aa0 con 0x7f3e6c0e3620 2021-04-03T03:17:19.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.654+0000 7f3e72ffd700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e6c0e3620 0x7f3e6c0d85f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f3e6800a000 tx=0x7f3e68008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:19.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e70ff9700 1 -- 172.21.2.111:0/3492991303 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e6800f750 con 0x7f3e6c0e3620 2021-04-03T03:17:19.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e70ff9700 1 -- 172.21.2.111:0/3492991303 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6800f040 con 0x7f3e6c0e3620 2021-04-03T03:17:19.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c0d53e0 con 0x7f3e6c0e3620 2021-04-03T03:17:19.672 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e70ff9700 1 -- 172.21.2.111:0/3492991303 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e6800faa0 con 0x7f3e6c0e3620 2021-04-03T03:17:19.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c0dadd0 con 0x7f3e6c0e3620 2021-04-03T03:17:19.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c0db000 con 0x7f3e6c0e3620 2021-04-03T03:17:19.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e70ff9700 1 -- 172.21.2.111:0/3492991303 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3e6800fc00 con 0x7f3e6c0e3620 2021-04-03T03:17:19.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e70ff9700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3e5c057050 0x7f3e5c059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e70ff9700 1 -- 172.21.2.111:0/3492991303 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3e68070370 con 0x7f3e6c0e3620 2021-04-03T03:17:19.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e737fe700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3e5c057050 0x7f3e5c059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e737fe700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3e5c057050 0x7f3e5c059510 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f3e6000a000 tx=0x7f3e60008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:19.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.658+0000 7f3e70ff9700 1 -- 172.21.2.111:0/3492991303 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3e68045650 con 0x7f3e6c0e3620 2021-04-03T03:17:19.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.754+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e6c0db000 con 0x7f3e6c0e3620 2021-04-03T03:17:19.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.754+0000 7f3e70ff9700 1 -- 172.21.2.111:0/3492991303 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f3e6c0db000 con 0x7f3e6c0e3620 2021-04-03T03:17:19.756 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:19.757 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":141,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:16.311403+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:13.328557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:14.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:19.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3e5c057050 msgr2=0x7f3e5c059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3e5c057050 0x7f3e5c059510 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f3e6000a000 tx=0x7f3e60008040).stop 2021-04-03T03:17:19.758 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e6c0e3620 msgr2=0x7f3e6c0d85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e6c0e3620 0x7f3e6c0d85f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f3e6800a000 tx=0x7f3e68008040).stop 2021-04-03T03:17:19.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 shutdown_connections 2021-04-03T03:17:19.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3e6c0d8ae0 0x7f3e6c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.759 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3e5c057050 0x7f3e5c059510 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3e6c0d6700 0x7f3e6c0d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 --2- 172.21.2.111:0/3492991303 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3e6c0e3620 0x7f3e6c0d85f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 >> 172.21.2.111:0/3492991303 conn(0x7f3e6c04a6e0 msgr2=0x7f3e6c0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:19.760 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 shutdown_connections 2021-04-03T03:17:19.761 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.758+0000 7f3e78a44700 1 -- 172.21.2.111:0/3492991303 wait complete. 2021-04-03T03:17:19.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:17:13.328557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:17:14.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:17:19.766 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:17:19.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 Processor -- start 2021-04-03T03:17:19.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- start start 2021-04-03T03:17:19.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2884144ef0 0x7f2884147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840607d0 con 0x7f28840add50 2021-04-03T03:17:19.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288414b090 con 0x7f28840603f0 2021-04-03T03:17:19.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288414b330 con 0x7f2884144ef0 2021-04-03T03:17:19.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f28837fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2884144ef0 0x7f2884147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f28837fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f2884144ef0 0x7f2884147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58430/0 (socket says 172.21.2.111:58430) 2021-04-03T03:17:19.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f28837fe700 1 -- 172.21.2.111:0/1705547928 learned_addr learned my addr 172.21.2.111:0/1705547928 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:19.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f28837fe700 1 -- 172.21.2.111:0/1705547928 >> v1:172.21.2.115:6789/0 conn(0x7f28840603f0 legacy=0x7f28841447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:19.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f28837fe700 1 -- 172.21.2.111:0/1705547928 >> v1:172.21.2.111:6789/0 conn(0x7f28840add50 legacy=0x7f2884147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:19.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f28837fe700 1 -- 172.21.2.111:0/1705547928 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288414b5d0 con 0x7f2884144ef0 2021-04-03T03:17:19.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f28837fe700 1 --2- 172.21.2.111:0/1705547928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2884144ef0 0x7f2884147e50 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f28840ae520 tx=0x7f287c009270).ready entity=mon.0 client_cookie=b09642c86f9ea9c8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:19.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2881ffb700 1 -- 172.21.2.111:0/1705547928 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287c00bb80 con 0x7f2884144ef0 2021-04-03T03:17:19.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2881ffb700 1 -- 172.21.2.111:0/1705547928 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c00bce0 con 0x7f2884144ef0 2021-04-03T03:17:19.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2881ffb700 1 -- 172.21.2.111:0/1705547928 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287c00be50 con 0x7f2884144ef0 2021-04-03T03:17:19.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- 172.21.2.111:0/1705547928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2884144ef0 msgr2=0x7f2884147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 --2- 172.21.2.111:0/1705547928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2884144ef0 0x7f2884147e50 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f28840ae520 tx=0x7f287c009270).stop 2021-04-03T03:17:19.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- 172.21.2.111:0/1705547928 shutdown_connections 2021-04-03T03:17:19.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 --2- 172.21.2.111:0/1705547928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f2884144ef0 0x7f2884147e50 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- 172.21.2.111:0/1705547928 >> 172.21.2.111:0/1705547928 conn(0x7f28840b61c0 msgr2=0x7f288405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:19.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- 172.21.2.111:0/1705547928 shutdown_connections 2021-04-03T03:17:19.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.822+0000 7f2883fff700 1 -- 172.21.2.111:0/1705547928 wait complete. 2021-04-03T03:17:19.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 Processor -- start 2021-04-03T03:17:19.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 -- start start 2021-04-03T03:17:19.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28840603f0 0x7f2884063f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28840add50 0x7f2884156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2884144ef0 0x7f2884156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2884063180 con 0x7f28840603f0 2021-04-03T03:17:19.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2884157910 con 0x7f2884144ef0 2021-04-03T03:17:19.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28840645c0 con 0x7f28840add50 2021-04-03T03:17:19.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28840603f0 0x7f2884063f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28840603f0 0x7f2884063f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58438/0 (socket says 172.21.2.111:58438) 2021-04-03T03:17:19.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 -- 172.21.2.111:0/261734031 learned_addr learned my addr 172.21.2.111:0/261734031 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:19.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 -- 172.21.2.111:0/261734031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28840add50 msgr2=0x7f2884156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28840add50 0x7f2884156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 -- 172.21.2.111:0/261734031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2884144ef0 msgr2=0x7f2884156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2884144ef0 0x7f2884156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 -- 172.21.2.111:0/261734031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884064860 con 0x7f28840603f0 2021-04-03T03:17:19.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2882ffd700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28840603f0 0x7f2884063f50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f2874002f70 tx=0x7f2874006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:19.836 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f287bfff700 1 -- 172.21.2.111:0/261734031 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287400d570 con 0x7f28840603f0 2021-04-03T03:17:19.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f287bfff700 1 -- 172.21.2.111:0/261734031 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287400d6d0 con 0x7f28840603f0 2021-04-03T03:17:19.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28840aa390 con 0x7f28840603f0 2021-04-03T03:17:19.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2884157610 con 0x7f28840603f0 2021-04-03T03:17:19.837 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f287bfff700 1 -- 172.21.2.111:0/261734031 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287400da30 con 0x7f28840603f0 2021-04-03T03:17:19.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28840a9a30 con 0x7f28840603f0 2021-04-03T03:17:19.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f287bfff700 1 -- 172.21.2.111:0/261734031 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f287400f070 con 0x7f28840603f0 2021-04-03T03:17:19.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f287bfff700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2868056ea0 0x7f2868059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f287bfff700 1 -- 172.21.2.111:0/261734031 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f287406fe50 con 0x7f28840603f0 2021-04-03T03:17:19.838 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f28827fc700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2868056ea0 0x7f2868059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.826+0000 7f28827fc700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2868056ea0 0x7f2868059360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f286c001f60 tx=0x7f286c0077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:19.839 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.830+0000 7f287bfff700 1 -- 172.21.2.111:0/261734031 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f2874045130 con 0x7f28840603f0 2021-04-03T03:17:19.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f28840649f0 con 0x7f28840603f0 2021-04-03T03:17:19.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f287bfff700 1 -- 172.21.2.111:0/261734031 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2874071020 con 0x7f28840603f0 2021-04-03T03:17:19.924 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:17:19.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2868056ea0 msgr2=0x7f2868059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2868056ea0 0x7f2868059360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f286c001f60 tx=0x7f286c0077f0).stop 2021-04-03T03:17:19.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28840603f0 msgr2=0x7f2884063f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28840603f0 0x7f2884063f50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f2874002f70 tx=0x7f2874006dd0).stop 2021-04-03T03:17:19.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 shutdown_connections 2021-04-03T03:17:19.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f2868056ea0 0x7f2868059360 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f2884144ef0 0x7f2884156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f28840603f0 0x7f2884063f50 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 --2- 172.21.2.111:0/261734031 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f28840add50 0x7f2884156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:19.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 >> 172.21.2.111:0/261734031 conn(0x7f28840b61c0 msgr2=0x7f288405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:19.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 shutdown_connections 2021-04-03T03:17:19.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.922+0000 7f2883fff700 1 -- 172.21.2.111:0/261734031 wait complete. 2021-04-03T03:17:19.932 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:17:19.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 Processor -- start 2021-04-03T03:17:19.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- start start 2021-04-03T03:17:19.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf380603f0 0x7faf381447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380607d0 con 0x7faf380b17b0 2021-04-03T03:17:19.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3814b090 con 0x7faf380add50 2021-04-03T03:17:19.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3814b330 con 0x7faf380603f0 2021-04-03T03:17:19.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3e4ae700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7faf380b17b0 0x7faf38147e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59236/0 (socket says 172.21.2.111:59236) 2021-04-03T03:17:19.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3e4ae700 1 -- 172.21.2.111:0/2958715117 learned_addr learned my addr 172.21.2.111:0/2958715117 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:19.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287043506 0 0) 0x7faf380607d0 con 0x7faf380b17b0 2021-04-03T03:17:19.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf20003240 con 0x7faf380b17b0 2021-04-03T03:17:19.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3dcad700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf380603f0 0x7faf381447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:19.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3762330920 0 0) 0x7faf20003240 con 0x7faf380b17b0 2021-04-03T03:17:19.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf380603f0 msgr2=0x7faf381447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:19.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf380603f0 0x7faf381447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:19.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 >> v1:172.21.2.115:6789/0 conn(0x7faf380add50 legacy=0x7faf38147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3814b5d0 con 0x7faf380b17b0 2021-04-03T03:17:19.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7faf30001a50 con 0x7faf380b17b0 2021-04-03T03:17:19.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 >> v1:172.21.2.111:6789/0 conn(0x7faf380b17b0 legacy=0x7faf38147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:19.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf380603f0 0x7faf200037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf20003d40 0x7faf200044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf200049d0 0x7faf20004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:19.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3814b330 con 0x7faf20003d40 2021-04-03T03:17:20.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200042d0 con 0x7faf200049d0 2021-04-03T03:17:20.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2000b730 con 0x7faf380603f0 2021-04-03T03:17:20.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3d4ac700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf20003d40 0x7faf200044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3e4ae700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf200049d0 0x7faf20004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3d4ac700 1 -- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf380603f0 msgr2=0x7faf200037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3d4ac700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf380603f0 0x7faf200037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3d4ac700 1 -- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf200049d0 msgr2=0x7faf20004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3d4ac700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf200049d0 0x7faf20004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3d4ac700 1 -- 172.21.2.111:0/2958715117 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20003240 con 0x7faf20003d40 2021-04-03T03:17:20.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3d4ac700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf20003d40 0x7faf200044e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7faf380b5640 tx=0x7faf34007b70).ready entity=mon.0 client_cookie=ed0a8e486bdfffbd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf3400c600 con 0x7faf20003d40 2021-04-03T03:17:20.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3400c760 con 0x7faf20003d40 2021-04-03T03:17:20.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ccab700 1 -- 172.21.2.111:0/2958715117 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf3400cac0 con 0x7faf20003d40 2021-04-03T03:17:20.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- 172.21.2.111:0/2958715117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf20003d40 msgr2=0x7faf200044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 --2- 172.21.2.111:0/2958715117 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf20003d40 0x7faf200044e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7faf380b5640 tx=0x7faf34007b70).stop 2021-04-03T03:17:20.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3e4ae700 1 -- 172.21.2.111:0/2958715117 reap_dead start 2021-04-03T03:17:20.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- 172.21.2.111:0/2958715117 shutdown_connections 2021-04-03T03:17:20.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- 172.21.2.111:0/2958715117 >> 172.21.2.111:0/2958715117 conn(0x7faf380b61c0 msgr2=0x7faf3805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- 172.21.2.111:0/2958715117 shutdown_connections 2021-04-03T03:17:20.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.990+0000 7faf3ecaf700 1 -- 172.21.2.111:0/2958715117 wait complete. 2021-04-03T03:17:20.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 Processor -- start 2021-04-03T03:17:20.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 -- start start 2021-04-03T03:17:20.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf3814a2d0 0x7faf3815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf38062a50 0x7faf38067030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf38067520 0x7faf38144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3814a6b0 con 0x7faf3814a2d0 2021-04-03T03:17:20.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38066350 con 0x7faf38067520 2021-04-03T03:17:20.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815a090 con 0x7faf38062a50 2021-04-03T03:17:20.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf3814a2d0 0x7faf3815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf3814a2d0 0x7faf3815a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58456/0 (socket says 172.21.2.111:58456) 2021-04-03T03:17:20.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 -- 172.21.2.111:0/470852662 learned_addr learned my addr 172.21.2.111:0/470852662 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:20.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 -- 172.21.2.111:0/470852662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf38062a50 msgr2=0x7faf38067030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf38062a50 0x7faf38067030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 -- 172.21.2.111:0/470852662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf38067520 msgr2=0x7faf38144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf38067520 0x7faf38144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 -- 172.21.2.111:0/470852662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3815a330 con 0x7faf3814a2d0 2021-04-03T03:17:20.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3dcad700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf3814a2d0 0x7faf3815a7d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7faf2c00c000 tx=0x7faf2c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf2affd700 1 -- 172.21.2.111:0/470852662 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf2c016070 con 0x7faf3814a2d0 2021-04-03T03:17:20.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf2affd700 1 -- 172.21.2.111:0/470852662 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c00ca10 con 0x7faf3814a2d0 2021-04-03T03:17:20.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf2affd700 1 -- 172.21.2.111:0/470852662 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf2c00cd70 con 0x7faf3814a2d0 2021-04-03T03:17:20.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf38066040 con 0x7faf3814a2d0 2021-04-03T03:17:20.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf380abf80 con 0x7faf3814a2d0 2021-04-03T03:17:20.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf38057660 con 0x7faf3814a2d0 2021-04-03T03:17:20.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf2affd700 1 -- 172.21.2.111:0/470852662 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7faf2c01b410 con 0x7faf3814a2d0 2021-04-03T03:17:20.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf2affd700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf18057000 0x7faf180594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf2affd700 1 -- 172.21.2.111:0/470852662 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7faf2c070470 con 0x7faf3814a2d0 2021-04-03T03:17:20.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3d4ac700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf18057000 0x7faf180594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.994+0000 7faf3d4ac700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf18057000 0x7faf180594c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7faf380b5640 tx=0x7faf34016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:19.998+0000 7faf2affd700 1 -- 172.21.2.111:0/470852662 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7faf2c0456f0 con 0x7faf3814a2d0 2021-04-03T03:17:20.092 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.090+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faf380b1590 con 0x7faf3814a2d0 2021-04-03T03:17:20.093 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.090+0000 7faf2affd700 1 -- 172.21.2.111:0/470852662 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7faf2c048d10 con 0x7faf3814a2d0 2021-04-03T03:17:20.093 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:20.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf18057000 msgr2=0x7faf180594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.094 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf18057000 0x7faf180594c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7faf380b5640 tx=0x7faf34016000).stop 2021-04-03T03:17:20.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf3814a2d0 msgr2=0x7faf3815a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf3814a2d0 0x7faf3815a7d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7faf2c00c000 tx=0x7faf2c006dd0).stop 2021-04-03T03:17:20.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 shutdown_connections 2021-04-03T03:17:20.095 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faf18057000 0x7faf180594c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf38067520 0x7faf38144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf3814a2d0 0x7faf3815a7d0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 --2- 172.21.2.111:0/470852662 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf38062a50 0x7faf38067030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.096 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 >> 172.21.2.111:0/470852662 conn(0x7faf380b61c0 msgr2=0x7faf380ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 shutdown_connections 2021-04-03T03:17:20.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.094+0000 7faf3ecaf700 1 -- 172.21.2.111:0/470852662 wait complete. 2021-04-03T03:17:20.101 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:17:20.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 Processor -- start 2021-04-03T03:17:20.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 -- start start 2021-04-03T03:17:20.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f15840603f0 0x7f15841447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15840b17b0 0x7f1584144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840607d0 con 0x7f15840add50 2021-04-03T03:17:20.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15841451c0 con 0x7f15840b17b0 2021-04-03T03:17:20.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158414b090 con 0x7f15840603f0 2021-04-03T03:17:20.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15840b17b0 0x7f1584144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f15840b17b0 0x7f1584144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58464/0 (socket says 172.21.2.111:58464) 2021-04-03T03:17:20.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 -- 172.21.2.111:0/2144369510 learned_addr learned my addr 172.21.2.111:0/2144369510 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:20.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 -- 172.21.2.111:0/2144369510 >> v2:172.21.2.115:3300/0 conn(0x7f15840603f0 msgr2=0x7f15841447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:20.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 --2- 172.21.2.111:0/2144369510 >> v2:172.21.2.115:3300/0 conn(0x7f15840603f0 0x7f15841447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 -- 172.21.2.111:0/2144369510 >> v1:172.21.2.115:6789/0 conn(0x7f15840add50 legacy=0x7f1584147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:20.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 -- 172.21.2.111:0/2144369510 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158414b330 con 0x7f15840b17b0 2021-04-03T03:17:20.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1588f04700 1 --2- 172.21.2.111:0/2144369510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840b17b0 0x7f1584144cd0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f15840ae520 tx=0x7f157c009270).ready entity=mon.0 client_cookie=bb9c92da5a6b104b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1582ffd700 1 -- 172.21.2.111:0/2144369510 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f157c00bbc0 con 0x7f15840b17b0 2021-04-03T03:17:20.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1582ffd700 1 -- 172.21.2.111:0/2144369510 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00bd20 con 0x7f15840b17b0 2021-04-03T03:17:20.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1582ffd700 1 -- 172.21.2.111:0/2144369510 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f157c00be90 con 0x7f15840b17b0 2021-04-03T03:17:20.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 -- 172.21.2.111:0/2144369510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840b17b0 msgr2=0x7f1584144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.158+0000 7f1589705700 1 --2- 172.21.2.111:0/2144369510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840b17b0 0x7f1584144cd0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f15840ae520 tx=0x7f157c009270).stop 2021-04-03T03:17:20.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- 172.21.2.111:0/2144369510 shutdown_connections 2021-04-03T03:17:20.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 --2- 172.21.2.111:0/2144369510 >> v2:172.21.2.115:3300/0 conn(0x7f15840603f0 0x7f15841447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 --2- 172.21.2.111:0/2144369510 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840b17b0 0x7f1584144cd0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- 172.21.2.111:0/2144369510 >> 172.21.2.111:0/2144369510 conn(0x7f15840b61c0 msgr2=0x7f158405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- 172.21.2.111:0/2144369510 shutdown_connections 2021-04-03T03:17:20.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- 172.21.2.111:0/2144369510 wait complete. 2021-04-03T03:17:20.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 Processor -- start 2021-04-03T03:17:20.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- start start 2021-04-03T03:17:20.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840603f0 0x7f15840aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15840add50 0x7f1584065270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15840b17b0 0x7f15840ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840a9da0 con 0x7f15840603f0 2021-04-03T03:17:20.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840abf70 con 0x7f15840add50 2021-04-03T03:17:20.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1584064b30 con 0x7f15840b17b0 2021-04-03T03:17:20.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840603f0 0x7f15840aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840603f0 0x7f15840aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58468/0 (socket says 172.21.2.111:58468) 2021-04-03T03:17:20.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 -- 172.21.2.111:0/2142211322 learned_addr learned my addr 172.21.2.111:0/2142211322 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:20.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 -- 172.21.2.111:0/2142211322 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15840b17b0 msgr2=0x7f15840ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15840b17b0 0x7f15840ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 -- 172.21.2.111:0/2142211322 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15840add50 msgr2=0x7f1584065270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15840add50 0x7f1584065270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 -- 172.21.2.111:0/2142211322 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584064dd0 con 0x7f15840603f0 2021-04-03T03:17:20.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1583fff700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840603f0 0x7f15840aa9d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f1574002f70 tx=0x7f1574006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15817fa700 1 -- 172.21.2.111:0/2142211322 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f157400d570 con 0x7f15840603f0 2021-04-03T03:17:20.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15817fa700 1 -- 172.21.2.111:0/2142211322 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157400d6d0 con 0x7f15840603f0 2021-04-03T03:17:20.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15817fa700 1 -- 172.21.2.111:0/2142211322 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f157400da30 con 0x7f15840603f0 2021-04-03T03:17:20.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15840aaec0 con 0x7f15840603f0 2021-04-03T03:17:20.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15840658f0 con 0x7f15840603f0 2021-04-03T03:17:20.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1584064f60 con 0x7f15840603f0 2021-04-03T03:17:20.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15817fa700 1 -- 172.21.2.111:0/2142211322 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f157400f070 con 0x7f15840603f0 2021-04-03T03:17:20.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15817fa700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1568056ef0 0x7f15680593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.175 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15817fa700 1 -- 172.21.2.111:0/2142211322 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f157406fca0 con 0x7f15840603f0 2021-04-03T03:17:20.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15837fe700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1568056ef0 0x7f15680593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.162+0000 7f15837fe700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1568056ef0 0x7f15680593b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f156c002f70 tx=0x7f156c0069b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.176 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.166+0000 7f15817fa700 1 -- 172.21.2.111:0/2142211322 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1574045090 con 0x7f15840603f0 2021-04-03T03:17:20.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.258+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1584057690 con 0x7f15840603f0 2021-04-03T03:17:20.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.258+0000 7f15817fa700 1 -- 172.21.2.111:0/2142211322 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+10 (secure 0 0 0) 0x7f15740486b0 con 0x7f15840603f0 2021-04-03T03:17:20.261 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:20.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1568056ef0 msgr2=0x7f15680593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1568056ef0 0x7f15680593b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f156c002f70 tx=0x7f156c0069b0).stop 2021-04-03T03:17:20.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840603f0 msgr2=0x7f15840aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840603f0 0x7f15840aa9d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f1574002f70 tx=0x7f1574006dd0).stop 2021-04-03T03:17:20.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 shutdown_connections 2021-04-03T03:17:20.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f15840b17b0 0x7f15840ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f15840603f0 0x7f15840aa9d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1568056ef0 0x7f15680593b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 --2- 172.21.2.111:0/2142211322 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f15840add50 0x7f1584065270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 >> 172.21.2.111:0/2142211322 conn(0x7f15840b61c0 msgr2=0x7f1584147320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 shutdown_connections 2021-04-03T03:17:20.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.262+0000 7f1589705700 1 -- 172.21.2.111:0/2142211322 wait complete. 2021-04-03T03:17:20.268 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:17:20.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 Processor -- start 2021-04-03T03:17:20.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- start start 2021-04-03T03:17:20.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340dbba0 0x7f7e340dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7e3403d560 0x7f7e34041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e340dc560 con 0x7f7e34041060 2021-04-03T03:17:20.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3403de60 con 0x7f7e3403d560 2021-04-03T03:17:20.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340dc800 con 0x7f7e340dbba0 2021-04-03T03:17:20.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3affd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7e34041060 0x7f7e3403fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59270/0 (socket says 172.21.2.111:59270) 2021-04-03T03:17:20.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3affd700 1 -- 172.21.2.111:0/1663185941 learned_addr learned my addr 172.21.2.111:0/1663185941 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:20.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617280621 0 0) 0x7f7e340dc560 con 0x7f7e34041060 2021-04-03T03:17:20.333 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e28003240 con 0x7f7e34041060 2021-04-03T03:17:20.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340dbba0 0x7f7e340dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 838116831 0 0) 0x7f7e28003240 con 0x7f7e34041060 2021-04-03T03:17:20.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3bfff700 1 --2- 172.21.2.111:0/1663185941 >> v2:172.21.2.115:3300/0 conn(0x7f7e3403d560 0x7f7e34041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.334 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340dbba0 msgr2=0x7f7e340dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340dbba0 0x7f7e340dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 >> v2:172.21.2.115:3300/0 conn(0x7f7e3403d560 msgr2=0x7f7e34041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 --2- 172.21.2.111:0/1663185941 >> v2:172.21.2.115:3300/0 conn(0x7f7e3403d560 0x7f7e34041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.335 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3403da30 con 0x7f7e34041060 2021-04-03T03:17:20.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f7e300016d0 con 0x7f7e34041060 2021-04-03T03:17:20.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 >> v1:172.21.2.111:6789/0 conn(0x7f7e34041060 legacy=0x7f7e3403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280037b0 0x7f7e28003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.336 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340dbba0 0x7f7e280044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e280049d0 0x7f7e28004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3403de60 con 0x7f7e280037b0 2021-04-03T03:17:20.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340dc800 con 0x7f7e280049d0 2021-04-03T03:17:20.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280042a0 con 0x7f7e340dbba0 2021-04-03T03:17:20.337 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280037b0 0x7f7e28003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3bfff700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e280049d0 0x7f7e28004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 -- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340dbba0 msgr2=0x7f7e280044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340dbba0 0x7f7e280044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 -- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e280049d0 msgr2=0x7f7e28004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e280049d0 0x7f7e28004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 -- 172.21.2.111:0/1663185941 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28003240 con 0x7f7e280037b0 2021-04-03T03:17:20.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280037b0 0x7f7e28003bd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f7e2c007000 tx=0x7f7e2c008040).ready entity=mon.0 client_cookie=91a767d144869802 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c00a950 con 0x7f7e280037b0 2021-04-03T03:17:20.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c00c030 con 0x7f7e280037b0 2021-04-03T03:17:20.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3a7fc700 1 -- 172.21.2.111:0/1663185941 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c00aca0 con 0x7f7e280037b0 2021-04-03T03:17:20.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- 172.21.2.111:0/1663185941 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280037b0 msgr2=0x7f7e28003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 --2- 172.21.2.111:0/1663185941 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280037b0 0x7f7e28003bd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f7e2c007000 tx=0x7f7e2c008040).stop 2021-04-03T03:17:20.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e3bfff700 1 -- 172.21.2.111:0/1663185941 reap_dead start 2021-04-03T03:17:20.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- 172.21.2.111:0/1663185941 shutdown_connections 2021-04-03T03:17:20.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- 172.21.2.111:0/1663185941 >> 172.21.2.111:0/1663185941 conn(0x7f7e340462b0 msgr2=0x7f7e340466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- 172.21.2.111:0/1663185941 shutdown_connections 2021-04-03T03:17:20.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.326+0000 7f7e408de700 1 -- 172.21.2.111:0/1663185941 wait complete. 2021-04-03T03:17:20.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 Processor -- start 2021-04-03T03:17:20.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 -- start start 2021-04-03T03:17:20.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e34044ea0 0x7f7e340a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e3403c170 0x7f7e340a2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340a2d30 0x7f7e340d4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3403b7b0 con 0x7f7e34044ea0 2021-04-03T03:17:20.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e34045280 con 0x7f7e3403c170 2021-04-03T03:17:20.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3403b270 con 0x7f7e340a2d30 2021-04-03T03:17:20.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e34044ea0 0x7f7e340a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e34044ea0 0x7f7e340a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58486/0 (socket says 172.21.2.111:58486) 2021-04-03T03:17:20.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 -- 172.21.2.111:0/517715399 learned_addr learned my addr 172.21.2.111:0/517715399 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:20.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 -- 172.21.2.111:0/517715399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340a2d30 msgr2=0x7f7e340d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.344 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340a2d30 0x7f7e340d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 -- 172.21.2.111:0/517715399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e3403c170 msgr2=0x7f7e340a2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e3403c170 0x7f7e340a2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 -- 172.21.2.111:0/517715399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3403b510 con 0x7f7e34044ea0 2021-04-03T03:17:20.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3b7fe700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e34044ea0 0x7f7e340a1520 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f7e2c007000 tx=0x7f7e2c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e38ff9700 1 -- 172.21.2.111:0/517715399 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c020070 con 0x7f7e34044ea0 2021-04-03T03:17:20.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e38ff9700 1 -- 172.21.2.111:0/517715399 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c00c040 con 0x7f7e34044ea0 2021-04-03T03:17:20.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3403aef0 con 0x7f7e34044ea0 2021-04-03T03:17:20.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e38ff9700 1 -- 172.21.2.111:0/517715399 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c01c410 con 0x7f7e34044ea0 2021-04-03T03:17:20.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3409feb0 con 0x7f7e34044ea0 2021-04-03T03:17:20.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e34004db0 con 0x7f7e34044ea0 2021-04-03T03:17:20.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e38ff9700 1 -- 172.21.2.111:0/517715399 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7e2c01c570 con 0x7f7e34044ea0 2021-04-03T03:17:20.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e38ff9700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e1c056ea0 0x7f7e1c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.347 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e38ff9700 1 -- 172.21.2.111:0/517715399 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7e2c071610 con 0x7f7e34044ea0 2021-04-03T03:17:20.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3affd700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e1c056ea0 0x7f7e1c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.330+0000 7f7e3affd700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e1c056ea0 0x7f7e1c059360 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7e300013e0 tx=0x7f7e30001280).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.334+0000 7f7e38ff9700 1 -- 172.21.2.111:0/517715399 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7e2c0469a0 con 0x7f7e34044ea0 2021-04-03T03:17:20.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.426+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e340a24c0 con 0x7f7e34044ea0 2021-04-03T03:17:20.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.426+0000 7f7e38ff9700 1 -- 172.21.2.111:0/517715399 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7e2c049fc0 con 0x7f7e34044ea0 2021-04-03T03:17:20.430 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:20.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e1c056ea0 msgr2=0x7f7e1c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e1c056ea0 0x7f7e1c059360 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7e300013e0 tx=0x7f7e30001280).stop 2021-04-03T03:17:20.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e34044ea0 msgr2=0x7f7e340a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e34044ea0 0x7f7e340a1520 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f7e2c007000 tx=0x7f7e2c008040).stop 2021-04-03T03:17:20.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 shutdown_connections 2021-04-03T03:17:20.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e340a2d30 0x7f7e340d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e34044ea0 0x7f7e340a1520 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e1c056ea0 0x7f7e1c059360 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 --2- 172.21.2.111:0/517715399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e3403c170 0x7f7e340a2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 >> 172.21.2.111:0/517715399 conn(0x7f7e340462b0 msgr2=0x7f7e34045b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 shutdown_connections 2021-04-03T03:17:20.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.430+0000 7f7e408de700 1 -- 172.21.2.111:0/517715399 wait complete. 2021-04-03T03:17:20.437 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:17:20.438 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:20.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 Processor -- start 2021-04-03T03:17:20.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- start start 2021-04-03T03:17:20.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f3c4014b700 0x7f3c400b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 0x7f3c40147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400607d0 con 0x7f3c400603f0 2021-04-03T03:17:20.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c400af400 con 0x7f3c4014b700 2021-04-03T03:17:20.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400af6a0 con 0x7f3c400b0c90 2021-04-03T03:17:20.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3f7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f3c400603f0 0x7f3c400b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59286/0 (socket says 172.21.2.111:59286) 2021-04-03T03:17:20.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3f7fe700 1 -- 172.21.2.111:0/3102827021 learned_addr learned my addr 172.21.2.111:0/3102827021 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:20.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958734230 0 0) 0x7f3c400607d0 con 0x7f3c400603f0 2021-04-03T03:17:20.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c20003240 con 0x7f3c400603f0 2021-04-03T03:17:20.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 0x7f3c40147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3effd700 1 --2- 172.21.2.111:0/3102827021 >> v2:172.21.2.115:3300/0 conn(0x7f3c4014b700 0x7f3c400b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2276079244 0 0) 0x7f3c20003240 con 0x7f3c400603f0 2021-04-03T03:17:20.499 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 msgr2=0x7f3c40147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 0x7f3c40147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 >> v2:172.21.2.115:3300/0 conn(0x7f3c4014b700 msgr2=0x7f3c400b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 --2- 172.21.2.111:0/3102827021 >> v2:172.21.2.115:3300/0 conn(0x7f3c4014b700 0x7f3c400b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400afb40 con 0x7f3c400603f0 2021-04-03T03:17:20.500 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f3c300016d0 con 0x7f3c400603f0 2021-04-03T03:17:20.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 >> v1:172.21.2.111:6789/0 conn(0x7f3c400603f0 legacy=0x7f3c400b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 0x7f3c200037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c20003d10 0x7f3c200044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.501 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c200049d0 0x7f3c20004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400af400 con 0x7f3c200049d0 2021-04-03T03:17:20.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400af6a0 con 0x7f3c20003d10 2021-04-03T03:17:20.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200042d0 con 0x7f3c400b0c90 2021-04-03T03:17:20.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c200049d0 0x7f3c20004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.502 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3effd700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c20003d10 0x7f3c200044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3f7fe700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 0x7f3c200037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 -- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 msgr2=0x7f3c200037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400b0c90 0x7f3c200037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.503 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 -- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c20003d10 msgr2=0x7f3c200044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:20.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c20003d10 0x7f3c200044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 -- 172.21.2.111:0/3102827021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c20003240 con 0x7f3c200049d0 2021-04-03T03:17:20.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c200049d0 0x7f3c20004e50 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f3c3800a000 tx=0x7f3c38008040).ready entity=mon.0 client_cookie=1570943ece67b09 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.504 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c3800dd90 con 0x7f3c200049d0 2021-04-03T03:17:20.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c38013480 con 0x7f3c200049d0 2021-04-03T03:17:20.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3e7fc700 1 -- 172.21.2.111:0/3102827021 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c380137e0 con 0x7f3c200049d0 2021-04-03T03:17:20.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- 172.21.2.111:0/3102827021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c200049d0 msgr2=0x7f3c20004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 --2- 172.21.2.111:0/3102827021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c200049d0 0x7f3c20004e50 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f3c3800a000 tx=0x7f3c38008040).stop 2021-04-03T03:17:20.505 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c3ffff700 1 -- 172.21.2.111:0/3102827021 reap_dead start 2021-04-03T03:17:20.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- 172.21.2.111:0/3102827021 shutdown_connections 2021-04-03T03:17:20.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- 172.21.2.111:0/3102827021 >> 172.21.2.111:0/3102827021 conn(0x7f3c400b61c0 msgr2=0x7f3c4005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- 172.21.2.111:0/3102827021 shutdown_connections 2021-04-03T03:17:20.506 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.494+0000 7f3c44f49700 1 -- 172.21.2.111:0/3102827021 wait complete. 2021-04-03T03:17:20.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 Processor -- start 2021-04-03T03:17:20.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 -- start start 2021-04-03T03:17:20.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400a7310 0x7f3c40159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c400aad70 0x7f3c40159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.509 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c4015a470 0x7f3c40146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400abf30 con 0x7f3c4015a470 2021-04-03T03:17:20.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400a76f0 con 0x7f3c400aad70 2021-04-03T03:17:20.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400a9cb0 con 0x7f3c400a7310 2021-04-03T03:17:20.510 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c4015a470 0x7f3c40146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c4015a470 0x7f3c40146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58508/0 (socket says 172.21.2.111:58508) 2021-04-03T03:17:20.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 -- 172.21.2.111:0/4022918811 learned_addr learned my addr 172.21.2.111:0/4022918811 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:20.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3effd700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c400aad70 0x7f3c40159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 -- 172.21.2.111:0/4022918811 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400a7310 msgr2=0x7f3c40159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.511 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400a7310 0x7f3c40159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 -- 172.21.2.111:0/4022918811 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c400aad70 msgr2=0x7f3c40159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c400aad70 0x7f3c40159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 -- 172.21.2.111:0/4022918811 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400a9f50 con 0x7f3c4015a470 2021-04-03T03:17:20.512 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3ffff700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c4015a470 0x7f3c40146800 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7f3c30001830 tx=0x7f3c38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3cff9700 1 -- 172.21.2.111:0/4022918811 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c38022070 con 0x7f3c4015a470 2021-04-03T03:17:20.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3cff9700 1 -- 172.21.2.111:0/4022918811 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3801e820 con 0x7f3c4015a470 2021-04-03T03:17:20.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c400aa1f0 con 0x7f3c4015a470 2021-04-03T03:17:20.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3cff9700 1 -- 172.21.2.111:0/4022918811 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c3801eb80 con 0x7f3c4015a470 2021-04-03T03:17:20.513 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c400628d0 con 0x7f3c4015a470 2021-04-03T03:17:20.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c40057660 con 0x7f3c4015a470 2021-04-03T03:17:20.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3cff9700 1 -- 172.21.2.111:0/4022918811 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f3c3801ece0 con 0x7f3c4015a470 2021-04-03T03:17:20.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3cff9700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3c24056ea0 0x7f3c24059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:20.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3cff9700 1 -- 172.21.2.111:0/4022918811 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f3c380720b0 con 0x7f3c4015a470 2021-04-03T03:17:20.514 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3f7fe700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3c24056ea0 0x7f3c24059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:20.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.498+0000 7f3c3f7fe700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3c24056ea0 0x7f3c24059360 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f3c30008000 tx=0x7f3c30009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:20.515 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.502+0000 7f3c3cff9700 1 -- 172.21.2.111:0/4022918811 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3c38047550 con 0x7f3c4015a470 2021-04-03T03:17:20.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c40159830 con 0x7f3c4015a470 2021-04-03T03:17:20.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c3cff9700 1 -- 172.21.2.111:0/4022918811 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f3c3804ab70 con 0x7f3c4015a470 2021-04-03T03:17:20.627 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:20.627 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":63,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3135556983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3135556983},{"type":"v1","addr":"172.21.2.115:6833","nonce":3135556983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:15.307326+0000","modified":"2021-04-03T03:17:16.311289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8250":{"gid":8250,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1485494348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1485494348},{"type":"v1","addr":"172.21.2.111:6835","nonce":1485494348}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-04-03T03:17:20.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3c24056ea0 msgr2=0x7f3c24059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3c24056ea0 0x7f3c24059360 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f3c30008000 tx=0x7f3c30009040).stop 2021-04-03T03:17:20.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c4015a470 msgr2=0x7f3c40146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:20.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c4015a470 0x7f3c40146800 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7f3c30001830 tx=0x7f3c38008040).stop 2021-04-03T03:17:20.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 shutdown_connections 2021-04-03T03:17:20.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f3c4015a470 0x7f3c40146800 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f3c400a7310 0x7f3c40159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:20.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3c24056ea0 0x7f3c24059360 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 --2- 172.21.2.111:0/4022918811 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f3c400aad70 0x7f3c40159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:20.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 >> 172.21.2.111:0/4022918811 conn(0x7f3c400b61c0 msgr2=0x7f3c400b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:20.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 shutdown_connections 2021-04-03T03:17:20.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:20.626+0000 7f3c44f49700 1 -- 172.21.2.111:0/4022918811 wait complete. 2021-04-03T03:17:20.631 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 63 2021-04-03T03:17:20.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:17:15.307326+0000', 'modified': '2021-04-03T03:17:16.311289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8250}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8250': {'gid': 8250, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/1485494348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 1485494348}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 1485494348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:17:20.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:17:20.837 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:17:20.914 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":8250,"want_state":"up:active","state":"up:active","rank_uptime":5.5998366490000002,"mdsmap_epoch":63,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":9.7448270669999992} 2021-04-03T03:17:20.914 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 8250, "mdsmap_epoch": 63, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 5.599836649, "state": "up:active", "uptime": 9.744827067, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:17:20.915 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:17:20.915 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:17:20.916 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:17:20.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:20.916 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:20.927 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:17:20.979 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:20.980 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:21.028 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:21.028 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:21.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.030 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:17:21.081 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:17:21.081 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:21.126 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:17:21.133 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:21.134 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:21.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:17:21.156+0000 7fa26d0080c0 -1 init, newargv = 0x55623a58f020 newargc=9 2021-04-03T03:17:21.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15670]: starting ceph client 2021-04-03T03:17:21.169 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:21.170 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:21.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.172 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:17:21.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15670]: starting fuse 2021-04-03T03:17:21.179 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:17:21.180 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:17:21.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.180 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:17:21.181 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:17:21.181 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:17:21.182 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:17:21.182 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:17:21.183 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:21.183 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:17:21.183 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:17:21.184 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:17:21.184 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:21.185 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:17:21.185 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:17:21.186 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:17:21.186 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:17:21.186 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:21.187 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:17:21.187 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:17:21.188 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:17:21.188 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:17:21.189 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:17:21.189 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:21.189 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:17:21.190 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:17:21.246 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15670.asok 2021-04-03T03:17:21.249 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15670.asok 2021-04-03T03:17:21.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.250 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15670.asok status 2021-04-03T03:17:21.348 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:17:21.348 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:17:21.348 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:17:21.349 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:17:21.349 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:17:21.349 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:17:21.349 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:17:21.350 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15670", 2021-04-03T03:17:21.350 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:17:21.350 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:21.350 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:17:21.351 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:17:21.351 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 5237, 2021-04-03T03:17:21.351 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:17:21.351 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:17:21.352 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:17:21.352 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 5237 2021-04-03T03:17:21.352 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:21.352 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:17:21.352 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:17:21.353 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:17:21.353 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 790564059 2021-04-03T03:17:21.353 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:17:21.353 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:21.354 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:17:21.354 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:17:21.354 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:17:21.354 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 790564059 2021-04-03T03:17:21.354 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:21.355 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.5237 172.21.2.128:0/790564059", 2021-04-03T03:17:21.355 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/790564059", 2021-04-03T03:17:21.355 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:17:21.355 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 63, 2021-04-03T03:17:21.355 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 141, 2021-04-03T03:17:21.356 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:17:21.356 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:17:21.356 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:17:21.356 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:17:21.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.358 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:21.358 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:21.364 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:17:21.364 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:21.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.365 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:21.413 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-04-03T03:17:21.413 DEBUG:teuthology.orchestra.run.gibba026:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2021-04-03T03:17:21.445 INFO:teuthology.orchestra.run.gibba026.stdout:1617419841.409211656 2021-04-03T03:17:21.446 INFO:tasks.cephfs.test_misc:rctime = 1617419841.409211656 2021-04-03T03:17:21.446 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:17:21.447 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:21.458 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:21.469 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:21.497 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:21.509 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:21.521 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:17:21.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.522 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:21.522 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:21.550 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:17:21.551 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:17:21.551 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba026.front.sepia.ceph.com... 2021-04-03T03:17:21.552 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:17:21.552 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:21.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[16111]: fuse finished with error 0 and tester_r 0 2021-04-03T03:17:21.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.638 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:21.638 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:21.643 INFO:teuthology.orchestra.run.gibba026.stdout:ext2/ext3 2021-04-03T03:17:21.643 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:17:21.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:17:21.744 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:21.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.749 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:21.749 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:21.760 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:17:21.761 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:21.762 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:17:21.762 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:17:21.763 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:21.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15670]: fuse finished with error 0 and tester_r 0 2021-04-03T03:17:21.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.839 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:21.839 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:21.889 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:17:21.890 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:17:21.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:17:21.974 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:21.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:21.984 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:21.984 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:17:21.993 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:17:21.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:21.995 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:17:21.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:17:21.996 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:17:22.044 INFO:teuthology.orchestra.run.gibba031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:17:22.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:22.046 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2021-04-03T03:17:22.107 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f1901a26700 1 Processor -- start 2021-04-03T03:17:22.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f1901a26700 1 -- start start 2021-04-03T03:17:22.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f1901a26700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f18fc0add50 0x7f18fc147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f1901a26700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc0607d0 con 0x7f18fc144ef0 2021-04-03T03:17:22.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f1901a26700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc14b090 con 0x7f18fc0603f0 2021-04-03T03:17:22.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f1901a26700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18fc14b330 con 0x7f18fc0add50 2021-04-03T03:17:22.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f18fbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f18fc0add50 0x7f18fc147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f18fbfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f18fc0add50 0x7f18fc147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58516/0 (socket says 172.21.2.111:58516) 2021-04-03T03:17:22.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f18fbfff700 1 -- 172.21.2.111:0/3201549371 learned_addr learned my addr 172.21.2.111:0/3201549371 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:22.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f1901225700 1 --1- 172.21.2.111:0/3201549371 >> v1:172.21.2.111:6789/0 conn(0x7f18fc144ef0 0x7f18fc147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59308/0 (socket says 172.21.2.111:59308) 2021-04-03T03:17:22.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f18fbfff700 1 -- 172.21.2.111:0/3201549371 >> v1:172.21.2.115:6789/0 conn(0x7f18fc0603f0 legacy=0x7f18fc1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:22.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f18fbfff700 1 -- 172.21.2.111:0/3201549371 >> v1:172.21.2.111:6789/0 conn(0x7f18fc144ef0 legacy=0x7f18fc147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f18fbfff700 1 -- 172.21.2.111:0/3201549371 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc14b5d0 con 0x7f18fc0add50 2021-04-03T03:17:22.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.106+0000 7f18fbfff700 1 --2- 172.21.2.111:0/3201549371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0add50 0x7f18fc147740 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f18f4001450 tx=0x7f18e4006e90).ready entity=mon.0 client_cookie=fa42e1ae10d51923 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18fb7fe700 1 -- 172.21.2.111:0/3201549371 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e4009710 con 0x7f18fc0add50 2021-04-03T03:17:22.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18fb7fe700 1 -- 172.21.2.111:0/3201549371 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4009870 con 0x7f18fc0add50 2021-04-03T03:17:22.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18fb7fe700 1 -- 172.21.2.111:0/3201549371 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e4009bd0 con 0x7f18fc0add50 2021-04-03T03:17:22.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3201549371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0add50 msgr2=0x7f18fc147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 --2- 172.21.2.111:0/3201549371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0add50 0x7f18fc147740 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f18f4001450 tx=0x7f18e4006e90).stop 2021-04-03T03:17:22.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3201549371 shutdown_connections 2021-04-03T03:17:22.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 --2- 172.21.2.111:0/3201549371 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0add50 0x7f18fc147740 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3201549371 >> 172.21.2.111:0/3201549371 conn(0x7f18fc0b61c0 msgr2=0x7f18fc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:22.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3201549371 shutdown_connections 2021-04-03T03:17:22.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3201549371 wait complete. 2021-04-03T03:17:22.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 Processor -- start 2021-04-03T03:17:22.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- start start 2021-04-03T03:17:22.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0603f0 0x7f18fc0aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18fc0add50 0x7f18fc0ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18fc144ef0 0x7f18fc159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc143870 con 0x7f18fc0603f0 2021-04-03T03:17:22.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0aba50 con 0x7f18fc144ef0 2021-04-03T03:17:22.118 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0abcb0 con 0x7f18fc0add50 2021-04-03T03:17:22.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0603f0 0x7f18fc0aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0603f0 0x7f18fc0aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58520/0 (socket says 172.21.2.111:58520) 2021-04-03T03:17:22.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 -- 172.21.2.111:0/3144193166 learned_addr learned my addr 172.21.2.111:0/3144193166 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:22.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 -- 172.21.2.111:0/3144193166 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18fc0add50 msgr2=0x7f18fc0ab560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:22.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18fc0add50 0x7f18fc0ab560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 -- 172.21.2.111:0/3144193166 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18fc144ef0 msgr2=0x7f18fc159f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:22.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18fc144ef0 0x7f18fc159f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.120 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 -- 172.21.2.111:0/3144193166 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0abf50 con 0x7f18fc0603f0 2021-04-03T03:17:22.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1900a24700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0603f0 0x7f18fc0aac80 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f18ec002f50 tx=0x7f18ec006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18f9ffb700 1 -- 172.21.2.111:0/3144193166 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec00d4a0 con 0x7f18fc0603f0 2021-04-03T03:17:22.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18f9ffb700 1 -- 172.21.2.111:0/3144193166 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00d600 con 0x7f18fc0603f0 2021-04-03T03:17:22.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc0a9930 con 0x7f18fc0603f0 2021-04-03T03:17:22.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc0648f0 con 0x7f18fc0603f0 2021-04-03T03:17:22.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18f9ffb700 1 -- 172.21.2.111:0/3144193166 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec00d960 con 0x7f18fc0603f0 2021-04-03T03:17:22.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc0a9fe0 con 0x7f18fc0603f0 2021-04-03T03:17:22.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18f9ffb700 1 -- 172.21.2.111:0/3144193166 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f18ec00f070 con 0x7f18fc0603f0 2021-04-03T03:17:22.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18f9ffb700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f18e0056ef0 0x7f18e00593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.122 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.110+0000 7f18f9ffb700 1 -- 172.21.2.111:0/3144193166 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f18ec06fe90 con 0x7f18fc0603f0 2021-04-03T03:17:22.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.114+0000 7f18fbfff700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f18e0056ef0 0x7f18e00593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.114+0000 7f18fbfff700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f18e0056ef0 0x7f18e00593b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f18f4001450 tx=0x7f18e4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.123 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.114+0000 7f18f9ffb700 1 -- 172.21.2.111:0/3144193166 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f18ec045170 con 0x7f18fc0603f0 2021-04-03T03:17:22.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.206+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f18fc064a80 con 0x7f18fc0603f0 2021-04-03T03:17:22.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f18f9ffb700 1 -- 172.21.2.111:0/3144193166 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f18ec048790 con 0x7f18fc0603f0 2021-04-03T03:17:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f18e0056ef0 msgr2=0x7f18e00593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f18e0056ef0 0x7f18e00593b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f18f4001450 tx=0x7f18e4015000).stop 2021-04-03T03:17:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0603f0 msgr2=0x7f18fc0aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0603f0 0x7f18fc0aac80 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f18ec002f50 tx=0x7f18ec006dd0).stop 2021-04-03T03:17:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 shutdown_connections 2021-04-03T03:17:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f18e0056ef0 0x7f18e00593b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f18fc144ef0 0x7f18fc159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f18fc0603f0 0x7f18fc0aac80 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 --2- 172.21.2.111:0/3144193166 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f18fc0add50 0x7f18fc0ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 >> 172.21.2.111:0/3144193166 conn(0x7f18fc0b61c0 msgr2=0x7f18fc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 shutdown_connections 2021-04-03T03:17:22.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.210+0000 7f1901a26700 1 -- 172.21.2.111:0/3144193166 wait complete. 2021-04-03T03:17:22.219 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2021-04-03T03:17:22.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 Processor -- start 2021-04-03T03:17:22.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 -- start start 2021-04-03T03:17:22.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11f00603f0 0x7f11f00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 0x7f11f00b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f11f00b12b0 0x7f11f00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f00b0f60 con 0x7f11f00603f0 2021-04-03T03:17:22.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f00607d0 con 0x7f11f00b12b0 2021-04-03T03:17:22.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f00af400 con 0x7f11f014b700 2021-04-03T03:17:22.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11f00603f0 0x7f11f00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f11f00603f0 0x7f11f00b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58528/0 (socket says 172.21.2.111:58528) 2021-04-03T03:17:22.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 -- 172.21.2.111:0/322883502 learned_addr learned my addr 172.21.2.111:0/322883502 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:22.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 -- 172.21.2.111:0/322883502 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 msgr2=0x7f11f00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:22.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 --2- 172.21.2.111:0/322883502 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 0x7f11f00b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 -- 172.21.2.111:0/322883502 >> v2:172.21.2.115:3300/0 conn(0x7f11f00b12b0 msgr2=0x7f11f00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:22.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 --2- 172.21.2.111:0/322883502 >> v2:172.21.2.115:3300/0 conn(0x7f11f00b12b0 0x7f11f00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 -- 172.21.2.111:0/322883502 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f00af6a0 con 0x7f11f00603f0 2021-04-03T03:17:22.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f542a700 1 --2- 172.21.2.111:0/322883502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00603f0 0x7f11f00b0090 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f11e4002f30 tx=0x7f11e4006e90).ready entity=mon.0 client_cookie=ae71f306bc5225b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11effff700 1 -- 172.21.2.111:0/322883502 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e4009920 con 0x7f11f00603f0 2021-04-03T03:17:22.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11effff700 1 -- 172.21.2.111:0/322883502 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e4009a80 con 0x7f11f00603f0 2021-04-03T03:17:22.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11effff700 1 -- 172.21.2.111:0/322883502 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e4009de0 con 0x7f11f00603f0 2021-04-03T03:17:22.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 -- 172.21.2.111:0/322883502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00603f0 msgr2=0x7f11f00b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.286 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.278+0000 7f11f642c700 1 --2- 172.21.2.111:0/322883502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00603f0 0x7f11f00b0090 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f11e4002f30 tx=0x7f11e4006e90).stop 2021-04-03T03:17:22.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- 172.21.2.111:0/322883502 shutdown_connections 2021-04-03T03:17:22.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 --2- 172.21.2.111:0/322883502 >> v2:172.21.2.115:3300/0 conn(0x7f11f00b12b0 0x7f11f00b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 --2- 172.21.2.111:0/322883502 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00603f0 0x7f11f00b0090 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 --2- 172.21.2.111:0/322883502 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 0x7f11f00b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.287 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- 172.21.2.111:0/322883502 >> 172.21.2.111:0/322883502 conn(0x7f11f00b61c0 msgr2=0x7f11f005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:22.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- 172.21.2.111:0/322883502 shutdown_connections 2021-04-03T03:17:22.288 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- 172.21.2.111:0/322883502 wait complete. 2021-04-03T03:17:22.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 Processor -- start 2021-04-03T03:17:22.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- start start 2021-04-03T03:17:22.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11f00603f0 0x7f11f00acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00b12b0 0x7f11f0156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 0x7f11f0156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01566d0 con 0x7f11f00b12b0 2021-04-03T03:17:22.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f00a8cb0 con 0x7f11f00603f0 2021-04-03T03:17:22.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f00a8ee0 con 0x7f11f014b700 2021-04-03T03:17:22.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00b12b0 0x7f11f0156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00b12b0 0x7f11f0156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58532/0 (socket says 172.21.2.111:58532) 2021-04-03T03:17:22.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 -- 172.21.2.111:0/2156325779 learned_addr learned my addr 172.21.2.111:0/2156325779 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:22.292 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 -- 172.21.2.111:0/2156325779 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 msgr2=0x7f11f0156fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:22.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 0x7f11f0156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 -- 172.21.2.111:0/2156325779 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11f00603f0 msgr2=0x7f11f00acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11f00603f0 0x7f11f00acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 -- 172.21.2.111:0/2156325779 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0065800 con 0x7f11f00b12b0 2021-04-03T03:17:22.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f4c29700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00b12b0 0x7f11f0156ac0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f11e00032b0 tx=0x7f11e00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11ee7fc700 1 -- 172.21.2.111:0/2156325779 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e000c9c0 con 0x7f11f00b12b0 2021-04-03T03:17:22.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11ee7fc700 1 -- 172.21.2.111:0/2156325779 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e000cb20 con 0x7f11f00b12b0 2021-04-03T03:17:22.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f01574a0 con 0x7f11f00b12b0 2021-04-03T03:17:22.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f0065490 con 0x7f11f00b12b0 2021-04-03T03:17:22.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11ee7fc700 1 -- 172.21.2.111:0/2156325779 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e000ce80 con 0x7f11f00b12b0 2021-04-03T03:17:22.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f0057660 con 0x7f11f00b12b0 2021-04-03T03:17:22.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11ee7fc700 1 -- 172.21.2.111:0/2156325779 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f11e0014420 con 0x7f11f00b12b0 2021-04-03T03:17:22.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.282+0000 7f11ee7fc700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11d0056ef0 0x7f11d00593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.286+0000 7f11ee7fc700 1 -- 172.21.2.111:0/2156325779 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f11e0070070 con 0x7f11f00b12b0 2021-04-03T03:17:22.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.286+0000 7f11f542a700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11d0056ef0 0x7f11d00593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.286+0000 7f11f542a700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11d0056ef0 0x7f11d00593b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f11e4002f30 tx=0x7f11e4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.286+0000 7f11ee7fc700 1 -- 172.21.2.111:0/2156325779 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f11e00452f0 con 0x7f11f00b12b0 2021-04-03T03:17:22.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.414+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f11f005fe60 con 0x7f11f00b12b0 2021-04-03T03:17:22.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.850+0000 7f11ee7fc700 1 -- 172.21.2.111:0/2156325779 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v390) v1 ==== 127+0+0 (secure 0 0 0) 0x7f11e0048910 con 0x7f11f00b12b0 2021-04-03T03:17:22.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.850+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11d0056ef0 msgr2=0x7f11d00593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.850+0000 7f11f642c700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11d0056ef0 0x7f11d00593b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f11e4002f30 tx=0x7f11e4015000).stop 2021-04-03T03:17:22.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.850+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00b12b0 msgr2=0x7f11f0156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.850+0000 7f11f642c700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00b12b0 0x7f11f0156ac0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f11e00032b0 tx=0x7f11e00091a0).stop 2021-04-03T03:17:22.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 shutdown_connections 2021-04-03T03:17:22.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f11f014b700 0x7f11f0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11d0056ef0 0x7f11d00593b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f11f00603f0 0x7f11f00acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 --2- 172.21.2.111:0/2156325779 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f11f00b12b0 0x7f11f0156ac0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 >> 172.21.2.111:0/2156325779 conn(0x7f11f00b61c0 msgr2=0x7f11f00ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:22.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 shutdown_connections 2021-04-03T03:17:22.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.854+0000 7f11f642c700 1 -- 172.21.2.111:0/2156325779 wait complete. 2021-04-03T03:17:22.860 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2021-04-03T03:17:22.861 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:17:22.861 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2021-04-03T03:17:22.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 Processor -- start 2021-04-03T03:17:22.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- start start 2021-04-03T03:17:22.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f80603f0 0x7f92f80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f92f80b0c90 0x7f92f8147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f80607d0 con 0x7f92f814b700 2021-04-03T03:17:22.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f80af400 con 0x7f92f80b0c90 2021-04-03T03:17:22.921 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80af6a0 con 0x7f92f80603f0 2021-04-03T03:17:22.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f92f80b0c90 0x7f92f8147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f92f80b0c90 0x7f92f8147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58544/0 (socket says 172.21.2.111:58544) 2021-04-03T03:17:22.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 -- 172.21.2.111:0/2284172974 learned_addr learned my addr 172.21.2.111:0/2284172974 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:22.922 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 -- 172.21.2.111:0/2284172974 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f80603f0 msgr2=0x7f92f80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:22.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f77fe700 1 --2- 172.21.2.111:0/2284172974 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f80603f0 0x7f92f80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 --2- 172.21.2.111:0/2284172974 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f80603f0 0x7f92f80b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 -- 172.21.2.111:0/2284172974 >> v1:172.21.2.111:6789/0 conn(0x7f92f814b700 legacy=0x7f92f80b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.923 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 -- 172.21.2.111:0/2284172974 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80afb40 con 0x7f92f80b0c90 2021-04-03T03:17:22.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f7fff700 1 --2- 172.21.2.111:0/2284172974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80b0c90 0x7f92f8147680 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f92f00032b0 tx=0x7f92f0009270).ready entity=mon.0 client_cookie=64f9c2b37ed5d235 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f67fc700 1 -- 172.21.2.111:0/2284172974 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92f000bc00 con 0x7f92f80b0c90 2021-04-03T03:17:22.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f67fc700 1 -- 172.21.2.111:0/2284172974 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f000bd60 con 0x7f92f80b0c90 2021-04-03T03:17:22.924 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92f67fc700 1 -- 172.21.2.111:0/2284172974 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92f0011400 con 0x7f92f80b0c90 2021-04-03T03:17:22.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- 172.21.2.111:0/2284172974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80b0c90 msgr2=0x7f92f8147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 --2- 172.21.2.111:0/2284172974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80b0c90 0x7f92f8147680 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f92f00032b0 tx=0x7f92f0009270).stop 2021-04-03T03:17:22.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- 172.21.2.111:0/2284172974 shutdown_connections 2021-04-03T03:17:22.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 --2- 172.21.2.111:0/2284172974 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80b0c90 0x7f92f8147680 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 --2- 172.21.2.111:0/2284172974 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f80603f0 0x7f92f80b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:22.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- 172.21.2.111:0/2284172974 >> 172.21.2.111:0/2284172974 conn(0x7f92f80b61c0 msgr2=0x7f92f805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:22.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- 172.21.2.111:0/2284172974 shutdown_connections 2021-04-03T03:17:22.926 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.918+0000 7f92fca4d700 1 -- 172.21.2.111:0/2284172974 wait complete. 2021-04-03T03:17:22.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 Processor -- start 2021-04-03T03:17:22.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 -- start start 2021-04-03T03:17:22.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80603f0 0x7f92f814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f92f80b0c90 0x7f92f8146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f814b700 0x7f92f8146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80a9df0 con 0x7f92f80603f0 2021-04-03T03:17:22.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8147470 con 0x7f92f80b0c90 2021-04-03T03:17:22.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80ab360 con 0x7f92f814b700 2021-04-03T03:17:22.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80603f0 0x7f92f814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80603f0 0x7f92f814ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58546/0 (socket says 172.21.2.111:58546) 2021-04-03T03:17:22.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 -- 172.21.2.111:0/3606097492 learned_addr learned my addr 172.21.2.111:0/3606097492 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:22.931 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 -- 172.21.2.111:0/3606097492 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f814b700 msgr2=0x7f92f8146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f814b700 0x7f92f8146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 -- 172.21.2.111:0/3606097492 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f92f80b0c90 msgr2=0x7f92f8146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:22.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f92f80b0c90 0x7f92f8146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:22.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 -- 172.21.2.111:0/3606097492 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80ab600 con 0x7f92f80603f0 2021-04-03T03:17:22.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f77fe700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80603f0 0x7f92f814ab50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f92e8007000 tx=0x7f92e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f4ff9700 1 -- 172.21.2.111:0/3606097492 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e800ce60 con 0x7f92f80603f0 2021-04-03T03:17:22.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f4ff9700 1 -- 172.21.2.111:0/3606097492 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e80076f0 con 0x7f92f80603f0 2021-04-03T03:17:22.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f80ab8a0 con 0x7f92f80603f0 2021-04-03T03:17:22.933 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f8156780 con 0x7f92f80603f0 2021-04-03T03:17:22.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f4ff9700 1 -- 172.21.2.111:0/3606097492 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e8007a50 con 0x7f92f80603f0 2021-04-03T03:17:22.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d40033f0 con 0x7f92f80603f0 2021-04-03T03:17:22.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f4ff9700 1 -- 172.21.2.111:0/3606097492 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f92e8007bb0 con 0x7f92f80603f0 2021-04-03T03:17:22.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f4ff9700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f92d8056ea0 0x7f92d8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:22.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f4ff9700 1 -- 172.21.2.111:0/3606097492 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f92e80703f0 con 0x7f92f80603f0 2021-04-03T03:17:22.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f6ffd700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f92d8056ea0 0x7f92d8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:22.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.922+0000 7f92f6ffd700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f92d8056ea0 0x7f92d8059360 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f92f80628d0 tx=0x7f92ec0077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:22.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:22.926+0000 7f92f4ff9700 1 -- 172.21.2.111:0/3606097492 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f92e8045670 con 0x7f92f80603f0 2021-04-03T03:17:23.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.054+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7f92d4003c20 con 0x7f92f80603f0 2021-04-03T03:17:23.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92f4ff9700 1 -- 172.21.2.111:0/3606097492 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v391) v1 ==== 139+0+0 (secure 0 0 0) 0x7f92e8048c90 con 0x7f92f80603f0 2021-04-03T03:17:23.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f92d8056ea0 msgr2=0x7f92d8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:23.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f92d8056ea0 0x7f92d8059360 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f92f80628d0 tx=0x7f92ec0077f0).stop 2021-04-03T03:17:23.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80603f0 msgr2=0x7f92f814ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:23.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80603f0 0x7f92f814ab50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f92e8007000 tx=0x7f92e8008040).stop 2021-04-03T03:17:23.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 shutdown_connections 2021-04-03T03:17:23.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f92f814b700 0x7f92f8146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:23.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f92f80603f0 0x7f92f814ab50 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:23.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f92d8056ea0 0x7f92d8059360 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:23.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 --2- 172.21.2.111:0/3606097492 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f92f80b0c90 0x7f92f8146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:23.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 >> 172.21.2.111:0/3606097492 conn(0x7f92f80b61c0 msgr2=0x7f92f8147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:23.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 shutdown_connections 2021-04-03T03:17:23.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.862+0000 7f92fca4d700 1 -- 172.21.2.111:0/3606097492 wait complete. 2021-04-03T03:17:23.870 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2021-04-03T03:17:23.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f12990b1700 1 Processor -- start 2021-04-03T03:17:23.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f12990b1700 1 -- start start 2021-04-03T03:17:23.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f12990b1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 0x7f1294147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f12990b1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12940607d0 con 0x7f12940603f0 2021-04-03T03:17:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f12990b1700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129414b090 con 0x7f1294144ef0 2021-04-03T03:17:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f12990b1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129414b330 con 0x7f12940add50 2021-04-03T03:17:23.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f1293fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f12940603f0 0x7f12941447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59346/0 (socket says 172.21.2.111:59346) 2021-04-03T03:17:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.926+0000 7f1293fff700 1 -- 172.21.2.111:0/2059702676 learned_addr learned my addr 172.21.2.111:0/2059702676 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986892915 0 0) 0x7f12940607d0 con 0x7f12940603f0 2021-04-03T03:17:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1274003240 con 0x7f12940603f0 2021-04-03T03:17:23.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 0x7f1294147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2755496518 0 0) 0x7f1274003240 con 0x7f12940603f0 2021-04-03T03:17:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 msgr2=0x7f1294147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 0x7f1294147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 >> v1:172.21.2.115:6789/0 conn(0x7f1294144ef0 legacy=0x7f1294147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:23.937 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129414b5d0 con 0x7f12940603f0 2021-04-03T03:17:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f12840016d0 con 0x7f12940603f0 2021-04-03T03:17:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 >> v1:172.21.2.111:6789/0 conn(0x7f12940603f0 legacy=0x7f12941447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1274003760 0x7f1274003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.938 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12740044e0 0x7f1274004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 0x7f1274004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129414b330 con 0x7f1274003760 2021-04-03T03:17:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1274004340 con 0x7f12740044e0 2021-04-03T03:17:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127400b730 con 0x7f12940add50 2021-04-03T03:17:23.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1274003760 0x7f1274003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12740044e0 0x7f1274004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12988b0700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 0x7f1274004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 -- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 msgr2=0x7f1274004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:23.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12940add50 0x7f1274004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 -- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12740044e0 msgr2=0x7f1274004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12740044e0 0x7f1274004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 -- 172.21.2.111:0/2059702676 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274003240 con 0x7f1274003760 2021-04-03T03:17:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12740044e0 0x7f1274004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:17:23.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1274003760 0x7f1274003b80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f12840013e0 tx=0x7f129414b760).ready entity=mon.0 client_cookie=d5b0770211ebd842 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f128400ce30 con 0x7f1274003760 2021-04-03T03:17:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12840070d0 con 0x7f1274003760 2021-04-03T03:17:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1292ffd700 1 -- 172.21.2.111:0/2059702676 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1284007430 con 0x7f1274003760 2021-04-03T03:17:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- 172.21.2.111:0/2059702676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1274003760 msgr2=0x7f1274003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:23.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 --2- 172.21.2.111:0/2059702676 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1274003760 0x7f1274003b80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f12840013e0 tx=0x7f129414b760).stop 2021-04-03T03:17:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12988b0700 1 -- 172.21.2.111:0/2059702676 reap_dead start 2021-04-03T03:17:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- 172.21.2.111:0/2059702676 shutdown_connections 2021-04-03T03:17:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- 172.21.2.111:0/2059702676 >> 172.21.2.111:0/2059702676 conn(0x7f12940b61c0 msgr2=0x7f129405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- 172.21.2.111:0/2059702676 shutdown_connections 2021-04-03T03:17:23.943 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- 172.21.2.111:0/2059702676 wait complete. 2021-04-03T03:17:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 Processor -- start 2021-04-03T03:17:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- start start 2021-04-03T03:17:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12940a9ce0 0x7f12940aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12940a6f20 0x7f12940aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.944 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f129415a110 0x7f129415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12940a9220 con 0x7f12940a6f20 2021-04-03T03:17:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12940aa980 con 0x7f12940a9ce0 2021-04-03T03:17:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1294148200 con 0x7f129415a110 2021-04-03T03:17:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12940a6f20 0x7f12940aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.945 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12940a6f20 0x7f12940aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58568/0 (socket says 172.21.2.111:58568) 2021-04-03T03:17:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 -- 172.21.2.111:0/3942167682 learned_addr learned my addr 172.21.2.111:0/3942167682 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f1293fff700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12940a9ce0 0x7f12940aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12988b0700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f129415a110 0x7f129415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 -- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f129415a110 msgr2=0x7f129415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:23.946 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f129415a110 0x7f129415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:23.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 -- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12940a9ce0 msgr2=0x7f12940aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:23.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12940a9ce0 0x7f12940aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:23.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 -- 172.21.2.111:0/3942167682 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940aa6e0 con 0x7f12940a6f20 2021-04-03T03:17:23.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12937fe700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12940a6f20 0x7f12940aba40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f127c008000 tx=0x7f127c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:23.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12917fa700 1 -- 172.21.2.111:0/3942167682 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f127c00e820 con 0x7f12940a6f20 2021-04-03T03:17:23.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12917fa700 1 -- 172.21.2.111:0/3942167682 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c00e980 con 0x7f12940a6f20 2021-04-03T03:17:23.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1294147ef0 con 0x7f12940a6f20 2021-04-03T03:17:23.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12917fa700 1 -- 172.21.2.111:0/3942167682 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f127c00ece0 con 0x7f12940a6f20 2021-04-03T03:17:23.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.930+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1294148600 con 0x7f12940a6f20 2021-04-03T03:17:23.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.934+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1294057660 con 0x7f12940a6f20 2021-04-03T03:17:23.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.934+0000 7f12917fa700 1 -- 172.21.2.111:0/3942167682 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f127c024070 con 0x7f12940a6f20 2021-04-03T03:17:23.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.934+0000 7f12917fa700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1278056ea0 0x7f1278059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:23.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.934+0000 7f12917fa700 1 -- 172.21.2.111:0/3942167682 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f127c070a30 con 0x7f12940a6f20 2021-04-03T03:17:23.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.934+0000 7f1293fff700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1278056ea0 0x7f1278059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:23.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.934+0000 7f1293fff700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1278056ea0 0x7f1278059360 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f1284019000 tx=0x7f1284013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:23.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:23.934+0000 7f12917fa700 1 -- 172.21.2.111:0/3942167682 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f127c045e20 con 0x7f12940a6f20 2021-04-03T03:17:24.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.026+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f12940ac010 con 0x7f12940a6f20 2021-04-03T03:17:24.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.030+0000 7f12917fa700 1 -- 172.21.2.111:0/3942167682 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f127c049440 con 0x7f12940a6f20 2021-04-03T03:17:24.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1278056ea0 msgr2=0x7f1278059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1278056ea0 0x7f1278059360 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f1284019000 tx=0x7f1284013000).stop 2021-04-03T03:17:24.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12940a6f20 msgr2=0x7f12940aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12940a6f20 0x7f12940aba40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f127c008000 tx=0x7f127c008040).stop 2021-04-03T03:17:24.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 shutdown_connections 2021-04-03T03:17:24.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f129415a110 0x7f129415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1278056ea0 0x7f1278059360 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12940a9ce0 0x7f12940aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 --2- 172.21.2.111:0/3942167682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12940a6f20 0x7f12940aba40 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 >> 172.21.2.111:0/3942167682 conn(0x7f12940b61c0 msgr2=0x7f129414a570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:24.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 shutdown_connections 2021-04-03T03:17:24.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.034+0000 7f12990b1700 1 -- 172.21.2.111:0/3942167682 wait complete. 2021-04-03T03:17:24.041 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:17:24.042 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:24.048 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:24.056 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:24.068 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:24.080 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:17:24.092 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:17:24.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:24.093 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:24.094 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:24.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:24.116 INFO:teuthology.orchestra.run.gibba026.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-04-03T03:17:24.117 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-04-03T03:17:24.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:24.118 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:24.118 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:24.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:24.125 INFO:teuthology.orchestra.run.gibba028.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-04-03T03:17:24.126 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:24.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:24.126 DEBUG:teuthology.orchestra.run.gibba031:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:24.127 DEBUG:teuthology.orchestra.run.gibba031:> ' -- /home/ubuntu/cephtest/mnt.2) 2021-04-03T03:17:24.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:17:24.141 INFO:teuthology.orchestra.run.gibba031.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2021-04-03T03:17:24.142 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2021-04-03T03:17:24.142 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:17:24.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 Processor -- start 2021-04-03T03:17:24.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- start start 2021-04-03T03:17:24.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f39a014b700 0x7f39a00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a00af400 con 0x7f39a00603f0 2021-04-03T03:17:24.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a00af6a0 con 0x7f39a00b0c90 2021-04-03T03:17:24.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a0060860 con 0x7f39a014b700 2021-04-03T03:17:24.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399dd9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f39a014b700 0x7f39a00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399dd9b700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f39a014b700 0x7f39a00b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58576/0 (socket says 172.21.2.111:58576) 2021-04-03T03:17:24.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399dd9b700 1 -- 172.21.2.111:0/3371178906 learned_addr learned my addr 172.21.2.111:0/3371178906 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:24.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399dd9b700 1 -- 172.21.2.111:0/3371178906 >> v1:172.21.2.115:6789/0 conn(0x7f39a00b0c90 legacy=0x7f39a0147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:24.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399dd9b700 1 -- 172.21.2.111:0/3371178906 >> v1:172.21.2.111:6789/0 conn(0x7f39a00603f0 legacy=0x7f39a00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399dd9b700 1 -- 172.21.2.111:0/3371178906 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00afb40 con 0x7f39a014b700 2021-04-03T03:17:24.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399dd9b700 1 --2- 172.21.2.111:0/3371178906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a00b07a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f39a00620f0 tx=0x7f3990006e90).ready entity=mon.0 client_cookie=279e6c9b855fa363 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399d59a700 1 -- 172.21.2.111:0/3371178906 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3990009980 con 0x7f39a014b700 2021-04-03T03:17:24.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399d59a700 1 -- 172.21.2.111:0/3371178906 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3990009ae0 con 0x7f39a014b700 2021-04-03T03:17:24.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399d59a700 1 -- 172.21.2.111:0/3371178906 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3990009e80 con 0x7f39a014b700 2021-04-03T03:17:24.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- 172.21.2.111:0/3371178906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 msgr2=0x7f39a00b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 --2- 172.21.2.111:0/3371178906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a00b07a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f39a00620f0 tx=0x7f3990006e90).stop 2021-04-03T03:17:24.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- 172.21.2.111:0/3371178906 shutdown_connections 2021-04-03T03:17:24.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 --2- 172.21.2.111:0/3371178906 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a00b07a0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- 172.21.2.111:0/3371178906 >> 172.21.2.111:0/3371178906 conn(0x7f39a00b61c0 msgr2=0x7f39a005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:24.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- 172.21.2.111:0/3371178906 shutdown_connections 2021-04-03T03:17:24.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- 172.21.2.111:0/3371178906 wait complete. 2021-04-03T03:17:24.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 Processor -- start 2021-04-03T03:17:24.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.214+0000 7f399f59e700 1 -- start start 2021-04-03T03:17:24.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f39a00603f0 0x7f39a00aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f39a00b0c90 0x7f39a0064ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a0156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00634d0 con 0x7f39a014b700 2021-04-03T03:17:24.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00ab630 con 0x7f39a00603f0 2021-04-03T03:17:24.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a0065620 con 0x7f39a00b0c90 2021-04-03T03:17:24.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a0156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a0156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58580/0 (socket says 172.21.2.111:58580) 2021-04-03T03:17:24.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 -- 172.21.2.111:0/4056975453 learned_addr learned my addr 172.21.2.111:0/4056975453 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:24.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 -- 172.21.2.111:0/4056975453 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f39a00b0c90 msgr2=0x7f39a0064ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f39a00b0c90 0x7f39a0064ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 -- 172.21.2.111:0/4056975453 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f39a00603f0 msgr2=0x7f39a00aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f39a00603f0 0x7f39a00aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 -- 172.21.2.111:0/4056975453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00ab840 con 0x7f39a014b700 2021-04-03T03:17:24.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399ed9d700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a0156690 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f39980032d0 tx=0x7f3998006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f39977fe700 1 -- 172.21.2.111:0/4056975453 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f399800c960 con 0x7f39a014b700 2021-04-03T03:17:24.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f39977fe700 1 -- 172.21.2.111:0/4056975453 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800cac0 con 0x7f39a014b700 2021-04-03T03:17:24.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a00abea0 con 0x7f39a014b700 2021-04-03T03:17:24.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f39977fe700 1 -- 172.21.2.111:0/4056975453 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f399800ce20 con 0x7f39a014b700 2021-04-03T03:17:24.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a0067f00 con 0x7f39a014b700 2021-04-03T03:17:24.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a0057660 con 0x7f39a014b700 2021-04-03T03:17:24.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f39977fe700 1 -- 172.21.2.111:0/4056975453 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f399801c070 con 0x7f39a014b700 2021-04-03T03:17:24.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f39977fe700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3984056f20 0x7f39840593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f39977fe700 1 -- 172.21.2.111:0/4056975453 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f399806fe80 con 0x7f39a014b700 2021-04-03T03:17:24.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399e59c700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3984056f20 0x7f39840593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.218+0000 7f399e59c700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3984056f20 0x7f39840593e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f39a006a3b0 tx=0x7f39880077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.222+0000 7f39977fe700 1 -- 172.21.2.111:0/4056975453 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f3998045160 con 0x7f39a014b700 2021-04-03T03:17:24.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f39a00abba0 con 0x7f39a014b700 2021-04-03T03:17:24.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f39977fe700 1 -- 172.21.2.111:0/4056975453 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f3998048780 con 0x7f39a014b700 2021-04-03T03:17:24.316 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:24.316 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:17:24.316 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 141, 2021-04-03T03:17:24.317 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:17:24.317 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:17:24.317 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:17:16.311403+0000", 2021-04-03T03:17:24.317 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:17:24.318 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:17:24.318 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:17:24.318 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:17:24.318 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:17:24.318 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:17:24.319 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:17:24.319 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:17:24.319 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:17:24.319 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:24.320 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:17:24.320 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:17:24.320 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:17:24.320 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:17:24.320 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:17:24.321 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 28, 2021-04-03T03:17:24.321 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:17:24.321 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:17:24.321 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:17:24.321 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:17:24.322 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:17:24.322 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.322 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:17:24.322 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:17:24.323 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:17:24.323 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:24.323 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:24.323 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:24.323 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:24.324 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:24.324 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:24.324 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:24.324 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:24.325 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:17:24.325 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:17:24.325 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:17:24.325 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:17:24.325 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:17:24.326 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:24.326 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:24.326 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:24.326 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:24.326 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:24.327 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:24.327 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:24.327 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.327 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:17:24.327 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:24.328 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:24.328 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:24.328 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:24.328 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:24.329 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:24.329 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:24.329 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:24.329 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:24.329 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:24.329 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:24.330 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:24.330 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:24.330 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:24.330 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:24.330 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:24.331 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:24.331 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:24.331 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:24.331 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:24.332 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:24.332 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:24.332 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:24.332 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:24.332 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:24.332 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:24.333 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.333 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:24.333 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:24.333 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:24.334 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:24.334 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:24.334 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:24.334 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:24.334 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:24.334 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:24.335 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:24.335 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:24.335 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:17:24.335 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:17:24.336 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.336 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:24.336 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:17:24.336 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.336 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.337 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.337 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:17:24.337 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:17:24.337 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:17:24.337 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:24.338 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:24.338 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:24.338 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:24.338 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:24.338 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:24.339 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:24.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:24.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:24.339 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:24.340 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:24.340 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:24.340 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:24.340 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:24.340 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:24.340 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:24.341 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:24.341 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:24.341 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:24.341 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:24.342 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.342 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:17:24.342 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:24.342 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:24.342 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:24.343 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:24.343 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:24.343 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:24.343 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:24.343 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:24.344 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:24.344 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:24.344 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:24.344 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:24.344 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:24.345 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:24.345 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:24.345 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:24.345 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:24.345 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:24.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:24.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:24.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:24.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:24.346 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:24.347 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:24.347 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:24.347 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:24.347 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.347 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:24.348 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:24.348 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:24.348 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:24.348 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:24.348 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:24.349 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:24.349 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:24.349 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:24.349 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:24.349 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:24.350 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:17:24.350 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:24.350 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:17:24.350 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.351 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.351 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.351 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 27, 2021-04-03T03:17:24.351 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:17:24.352 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:17:13.328557+0000", 2021-04-03T03:17:24.352 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:24.352 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:24.352 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:24.352 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:24.353 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:24.353 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:24.353 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:24.353 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:24.353 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:24.354 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:24.354 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:24.354 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:24.354 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:24.354 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:24.355 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:24.355 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:24.355 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:24.355 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:24.355 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:24.356 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:24.356 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.356 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "140", 2021-04-03T03:17:24.356 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:24.356 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:24.357 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:24.357 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:24.357 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:24.357 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:24.357 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:24.358 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:24.358 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:24.358 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:24.358 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:24.358 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:24.359 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:24.359 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:24.359 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:24.359 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:24.360 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:24.360 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:24.360 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:24.360 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:24.360 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:24.361 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:24.361 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:24.361 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:24.361 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:24.361 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:24.362 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.362 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:24.362 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:24.362 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:24.362 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:24.363 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:24.363 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:24.363 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:24.363 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:24.364 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:24.364 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:24.364 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:24.364 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:17:24.364 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:17:24.365 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:17:24.365 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:17:24.365 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.365 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:24.366 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:17:24.366 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:17:24.366 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.366 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.366 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.367 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.367 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 28, 2021-04-03T03:17:24.367 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:17:24.367 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:17:14.274378+0000", 2021-04-03T03:17:24.367 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:24.368 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:24.368 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:24.368 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:24.368 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:24.368 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:24.369 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:24.369 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:24.369 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:24.369 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:24.369 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:24.370 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:24.370 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:24.370 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:24.370 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:24.370 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:24.371 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:24.371 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:24.371 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:24.371 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:24.371 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.372 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "141", 2021-04-03T03:17:24.372 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:24.372 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:24.372 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:24.372 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:24.373 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:24.373 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:24.373 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:24.373 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:24.373 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:24.374 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:24.374 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:24.374 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:24.374 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:24.374 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:24.375 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:24.375 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:24.375 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:24.375 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:24.375 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:24.376 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:24.376 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:24.376 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:24.376 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:24.376 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:24.377 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:24.377 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:24.377 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.377 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:24.378 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:24.378 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:24.378 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:24.378 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:24.378 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:24.378 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:24.379 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:24.379 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:24.379 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:24.379 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:24.380 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:17:24.380 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:24.380 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:17:24.380 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:17:24.380 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.381 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.381 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.381 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:24.381 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:17:24.381 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.382 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:17:24.382 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:17:24.382 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.382 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.383 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.383 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.383 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.383 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.383 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.384 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 139, 2021-04-03T03:17:24.384 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.384 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.384 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.384 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.385 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.385 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.385 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:17:24.385 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.385 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:17:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.386 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.387 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:17:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.388 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.389 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.389 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.389 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:17:24.389 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.389 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.390 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.390 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.390 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.390 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.390 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.391 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.391 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:17:24.391 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.391 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.392 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.392 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.392 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:17:24.392 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.392 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.392 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.393 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.393 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.393 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.393 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.393 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.394 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:17:24.394 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.394 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.394 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.394 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.395 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:17:24.395 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:24.395 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.395 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.395 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.396 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:17:24.396 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:17:24.396 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:17:24.396 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:17:24.396 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.396 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.397 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.397 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.397 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.397 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.397 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:17:24.398 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:17:24.398 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.398 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.398 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.398 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.399 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.399 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.399 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.399 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 139, 2021-04-03T03:17:24.400 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.400 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.400 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.400 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.400 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.401 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.401 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:17:24.401 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.401 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.401 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.402 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.402 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:17:24.402 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.402 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.403 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.403 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.403 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.403 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.403 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.404 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.404 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:17:24.404 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.404 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.404 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.405 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.405 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:17:24.405 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.405 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.405 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.406 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.406 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.406 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.406 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.406 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.407 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:17:24.407 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.407 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.407 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.408 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.408 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:17:24.408 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.408 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.408 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.409 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.409 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.409 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.409 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.410 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.410 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:17:24.410 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.410 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.411 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.411 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.411 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:17:24.411 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:24.411 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.412 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.412 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.412 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:17:24.412 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:17:24.412 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:17:24.413 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:17:24.413 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.413 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.413 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.413 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.414 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.414 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.414 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:17:24.414 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:17:24.414 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.415 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.415 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.415 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.415 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.415 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.416 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.416 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 138, 2021-04-03T03:17:24.416 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.416 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.416 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.417 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.417 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.417 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.417 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:17:24.417 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.418 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.418 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.418 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.418 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:17:24.418 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.419 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.419 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.419 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.419 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.419 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.420 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.420 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.420 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:17:24.420 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.421 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.421 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.421 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.421 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:17:24.421 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.421 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.422 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.422 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.422 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.422 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.422 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.423 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.423 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:17:24.423 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.423 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.423 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.424 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.424 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:17:24.424 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.424 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.424 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.425 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.425 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.425 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.425 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.425 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.426 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:17:24.426 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.426 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.426 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.426 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.426 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:17:24.427 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:24.427 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.427 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.427 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.427 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:17:24.428 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:17:24.428 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:17:24.428 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:17:24.428 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.428 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.429 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.429 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.429 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.429 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.429 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:17:24.430 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:17:24.430 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.430 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.430 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.430 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.430 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.431 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.431 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.431 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 129, 2021-04-03T03:17:24.431 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.431 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.432 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.432 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.432 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.432 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.432 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:17:24.433 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.433 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.433 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.433 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.433 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:17:24.434 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.434 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.434 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.434 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.435 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.435 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.435 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.435 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.435 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:17:24.436 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.436 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.436 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.436 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.436 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:17:24.437 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.437 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.437 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.437 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.437 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.438 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.438 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.438 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.438 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:17:24.438 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.439 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.439 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.439 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.439 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:17:24.440 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.440 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.440 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.440 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.440 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.441 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.441 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.441 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.441 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:17:24.441 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.442 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.442 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.442 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.442 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:17:24.443 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:24.443 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.443 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.443 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.443 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:17:24.444 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:17:24.444 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:17:24.444 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:17:24.444 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.445 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.445 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.445 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.445 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.445 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.446 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:17:24.446 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:17:24.446 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.446 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.447 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.447 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.447 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.447 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.447 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.448 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 138, 2021-04-03T03:17:24.448 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.448 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.448 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.449 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.449 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.449 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.449 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:17:24.449 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.450 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.450 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.450 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.450 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:17:24.450 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.451 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.451 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.451 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.451 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.452 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.452 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.452 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.452 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:17:24.453 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.453 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.453 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.453 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.453 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:17:24.454 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.454 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.454 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.454 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.454 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.455 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.455 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.455 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.455 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:17:24.456 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.456 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.456 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.456 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.456 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:17:24.457 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.457 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.457 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.457 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.457 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.458 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.458 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.458 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.458 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:17:24.458 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.459 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.459 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.459 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.459 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:17:24.460 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:24.460 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.460 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.460 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.460 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:17:24.460 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:17:24.461 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:17:24.461 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:17:24.461 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.461 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.461 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.462 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.462 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.462 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.462 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:17:24.462 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:17:24.463 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.463 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.463 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.463 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.464 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.464 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.464 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.464 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 139, 2021-04-03T03:17:24.464 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.464 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.465 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.465 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.465 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.465 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.465 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:17:24.466 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.466 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.466 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.466 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.466 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:17:24.467 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.467 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.467 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.467 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.467 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.468 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.468 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.468 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.468 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:17:24.468 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.469 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.469 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.469 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.469 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:17:24.469 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.470 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.470 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.470 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.470 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.470 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.470 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.471 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.471 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:17:24.471 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.471 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.471 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.472 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.472 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:17:24.472 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.472 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.472 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.473 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.473 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.473 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.473 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.473 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.474 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:17:24.474 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.474 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.474 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.474 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.475 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:17:24.475 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:24.475 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.475 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.475 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.476 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:17:24.476 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:17:24.476 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:17:24.476 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:17:24.476 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.476 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.477 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.477 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.477 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.477 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.477 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:17:24.478 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:17:24.478 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.478 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.478 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.478 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.479 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.479 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.479 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.479 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 139, 2021-04-03T03:17:24.479 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.480 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.480 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.480 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.480 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.480 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.480 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:17:24.481 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.481 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.481 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.481 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.481 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:17:24.482 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.482 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.482 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.482 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.482 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.483 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.483 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.483 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.483 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:17:24.483 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.483 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.484 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.484 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.484 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:17:24.484 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.484 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.485 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.485 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.485 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.485 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.485 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.486 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.486 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:17:24.486 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.486 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.486 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.487 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.487 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:17:24.487 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.487 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.487 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.488 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.488 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.488 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.488 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.488 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.489 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:17:24.489 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.489 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.489 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.490 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.490 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:17:24.490 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:24.490 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.490 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.491 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.491 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:17:24.491 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:17:24.491 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:17:24.491 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:17:24.492 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.492 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.492 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.492 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.492 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.493 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.493 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:17:24.493 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:17:24.493 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:24.493 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:24.494 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:24.494 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:24.494 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:24.494 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:24.495 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:24.495 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 139, 2021-04-03T03:17:24.495 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:24.495 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:24.495 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:24.496 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.496 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.496 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.496 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:17:24.496 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.497 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.497 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.497 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.497 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:17:24.497 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.498 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.498 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.498 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.498 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:24.499 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.499 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.499 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.499 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:17:24.500 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.500 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.500 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.500 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.500 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:17:24.501 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.501 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.501 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.501 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.502 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:24.502 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.502 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.502 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.503 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:17:24.503 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.503 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.503 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.504 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.504 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:17:24.504 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.504 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.504 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.505 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.505 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:24.505 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:24.505 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.505 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:24.506 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:17:24.506 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.506 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.506 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.507 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:24.507 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:17:24.507 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:24.507 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.507 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.508 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.508 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:17:24.508 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:17:24.508 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:17:24.508 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:17:24.509 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:24.509 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:24.509 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:24.509 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:24.510 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.510 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:24.510 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:17:24.510 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.510 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:17:24.510 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.511 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.511 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.511 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.511 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.512 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:17:24.512 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.512 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.512 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.512 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:17:24.513 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.513 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.513 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.513 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.513 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.514 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:17:24.514 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.514 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.514 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.514 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:17:24.515 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.515 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.515 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.515 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.515 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.516 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:17:24.516 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.516 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.516 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.516 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:17:24.517 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.517 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.517 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.517 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.517 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.518 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:17:24.518 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.518 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.518 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.519 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:17:24.519 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.519 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.519 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.519 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.520 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.520 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:17:24.520 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.520 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.520 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.521 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:17:24.521 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.521 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.521 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.521 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.522 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.522 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:17:24.522 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.522 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.522 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.523 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:17:24.523 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.523 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.523 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.524 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.524 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.524 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:17:24.524 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.524 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.525 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:24.525 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:17:24.525 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:24.525 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:24.525 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:24.525 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:24.526 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:24.526 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:17:24.526 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:24.526 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.526 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:24.527 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:17:24.527 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:17:24.527 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:17:24.527 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:17:24.528 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:17:24.528 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:17:24.528 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:17:24.528 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:17:24.528 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:17:24.529 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:17:24.529 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:17:24.529 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:17:24.529 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:17:24.530 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:17:24.530 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:17:24.530 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:17:24.530 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:17:24.530 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.531 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:17:24.531 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:17:24.531 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:17:24.531 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:17:24.532 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:17:24.532 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:17:24.532 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:17:24.532 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:24.532 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:24.533 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:17:24.533 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:17:24.533 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:17:24.533 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:17:24.533 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:17:24.534 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:17:24.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3984056f20 msgr2=0x7f39840593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3984056f20 0x7f39840593e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f39a006a3b0 tx=0x7f39880077f0).stop 2021-04-03T03:17:24.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 msgr2=0x7f39a0156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a0156690 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f39980032d0 tx=0x7f3998006f90).stop 2021-04-03T03:17:24.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 shutdown_connections 2021-04-03T03:17:24.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f39a014b700 0x7f39a0156690 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f3984056f20 0x7f39840593e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f39a00603f0 0x7f39a00aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 --2- 172.21.2.111:0/4056975453 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f39a00b0c90 0x7f39a0064ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 >> 172.21.2.111:0/4056975453 conn(0x7f39a00b61c0 msgr2=0x7f39a014bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:24.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 shutdown_connections 2021-04-03T03:17:24.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.314+0000 7f399f59e700 1 -- 172.21.2.111:0/4056975453 wait complete. 2021-04-03T03:17:24.538 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:24.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 Processor -- start 2021-04-03T03:17:24.598 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- start start 2021-04-03T03:17:24.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7effbc0b5d60 0x7effbc0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7effbc0b44a0 0x7effbc0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc0b4bc0 con 0x7effbc0603f0 2021-04-03T03:17:24.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0607d0 con 0x7effbc0b44a0 2021-04-03T03:17:24.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0b4df0 con 0x7effbc0b5d60 2021-04-03T03:17:24.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7effbc0b44a0 0x7effbc0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7effbc0b44a0 0x7effbc0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58590/0 (socket says 172.21.2.111:58590) 2021-04-03T03:17:24.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 -- 172.21.2.111:0/1269256601 learned_addr learned my addr 172.21.2.111:0/1269256601 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:24.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 -- 172.21.2.111:0/1269256601 >> v2:172.21.2.115:3300/0 conn(0x7effbc0b5d60 msgr2=0x7effbc0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:24.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 --2- 172.21.2.111:0/1269256601 >> v2:172.21.2.115:3300/0 conn(0x7effbc0b5d60 0x7effbc0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 -- 172.21.2.111:0/1269256601 >> v1:172.21.2.115:6789/0 conn(0x7effbc0603f0 legacy=0x7effbc0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:24.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 -- 172.21.2.111:0/1269256601 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc0b5090 con 0x7effbc0b44a0 2021-04-03T03:17:24.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc08d3700 1 --2- 172.21.2.111:0/1269256601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b44a0 0x7effbc0b5530 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7effbc0b78f0 tx=0x7effb4009270).ready entity=mon.0 client_cookie=508d01067049f89a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effbaffd700 1 -- 172.21.2.111:0/1269256601 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb400bbc0 con 0x7effbc0b44a0 2021-04-03T03:17:24.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effbaffd700 1 -- 172.21.2.111:0/1269256601 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb400bd20 con 0x7effbc0b44a0 2021-04-03T03:17:24.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effbaffd700 1 -- 172.21.2.111:0/1269256601 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4011400 con 0x7effbc0b44a0 2021-04-03T03:17:24.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- 172.21.2.111:0/1269256601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b44a0 msgr2=0x7effbc0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- 172.21.2.111:0/1269256601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b44a0 0x7effbc0b5530 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7effbc0b78f0 tx=0x7effb4009270).stop 2021-04-03T03:17:24.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- 172.21.2.111:0/1269256601 shutdown_connections 2021-04-03T03:17:24.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- 172.21.2.111:0/1269256601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b44a0 0x7effbc0b5530 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- 172.21.2.111:0/1269256601 >> v2:172.21.2.115:3300/0 conn(0x7effbc0b5d60 0x7effbc0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- 172.21.2.111:0/1269256601 >> 172.21.2.111:0/1269256601 conn(0x7effbc05c7d0 msgr2=0x7effbc05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:24.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- 172.21.2.111:0/1269256601 shutdown_connections 2021-04-03T03:17:24.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- 172.21.2.111:0/1269256601 wait complete. 2021-04-03T03:17:24.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 Processor -- start 2021-04-03T03:17:24.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- start start 2021-04-03T03:17:24.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7effbc0603f0 0x7effbc146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7effbc0b44a0 0x7effbc0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.608 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b5d60 0x7effbc144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0aea20 con 0x7effbc0b5d60 2021-04-03T03:17:24.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0aecc0 con 0x7effbc0b44a0 2021-04-03T03:17:24.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.598+0000 7effc10d4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc146ac0 con 0x7effbc0603f0 2021-04-03T03:17:24.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b5d60 0x7effbc144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.609 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b5d60 0x7effbc144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58594/0 (socket says 172.21.2.111:58594) 2021-04-03T03:17:24.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 -- 172.21.2.111:0/2074638654 learned_addr learned my addr 172.21.2.111:0/2074638654 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:24.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 -- 172.21.2.111:0/2074638654 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7effbc0603f0 msgr2=0x7effbc146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7effbc0603f0 0x7effbc146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.610 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 -- 172.21.2.111:0/2074638654 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7effbc0b44a0 msgr2=0x7effbc0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7effbc0b44a0 0x7effbc0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 -- 172.21.2.111:0/2074638654 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc145bf0 con 0x7effbc0b5d60 2021-04-03T03:17:24.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc08d3700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b5d60 0x7effbc144f10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7effbc0b78f0 tx=0x7effb4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effb97fa700 1 -- 172.21.2.111:0/2074638654 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4006720 con 0x7effbc0b5d60 2021-04-03T03:17:24.611 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effb97fa700 1 -- 172.21.2.111:0/2074638654 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb4006880 con 0x7effbc0b5d60 2021-04-03T03:17:24.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effb97fa700 1 -- 172.21.2.111:0/2074638654 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb401f610 con 0x7effbc0b5d60 2021-04-03T03:17:24.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc147450 con 0x7effbc0b5d60 2021-04-03T03:17:24.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc145400 con 0x7effbc0b5d60 2021-04-03T03:17:24.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc146c80 con 0x7effbc0b5d60 2021-04-03T03:17:24.612 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effb97fa700 1 -- 172.21.2.111:0/2074638654 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7effb4026030 con 0x7effbc0b5d60 2021-04-03T03:17:24.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effb97fa700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7effa00570a0 0x7effa0059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effb97fa700 1 -- 172.21.2.111:0/2074638654 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7effb40713d0 con 0x7effbc0b5d60 2021-04-03T03:17:24.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effbbfff700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7effa00570a0 0x7effa0059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.602+0000 7effbbfff700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7effa00570a0 0x7effa0059560 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7effa4002f50 tx=0x7effa4006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.613 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.606+0000 7effb97fa700 1 -- 172.21.2.111:0/2074638654 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7effb40466b0 con 0x7effbc0b5d60 2021-04-03T03:17:24.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effbc057660 con 0x7effbc0b5d60 2021-04-03T03:17:24.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effb97fa700 1 -- 172.21.2.111:0/2074638654 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+2142 (secure 0 0 0) 0x7effb4024090 con 0x7effbc0b5d60 2021-04-03T03:17:24.730 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:24.730 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":63,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3135556983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3135556983},{"type":"v1","addr":"172.21.2.115:6833","nonce":3135556983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:15.307326+0000","modified":"2021-04-03T03:17:16.311289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8250":{"gid":8250,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/1485494348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":1485494348},{"type":"v1","addr":"172.21.2.111:6835","nonce":1485494348}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-04-03T03:17:24.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7effa00570a0 msgr2=0x7effa0059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7effa00570a0 0x7effa0059560 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7effa4002f50 tx=0x7effa4006990).stop 2021-04-03T03:17:24.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b5d60 msgr2=0x7effbc144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b5d60 0x7effbc144f10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7effbc0b78f0 tx=0x7effb4016000).stop 2021-04-03T03:17:24.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 shutdown_connections 2021-04-03T03:17:24.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7effbc0b5d60 0x7effbc144f10 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7effbc0603f0 0x7effbc146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7effa00570a0 0x7effa0059560 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 --2- 172.21.2.111:0/2074638654 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7effbc0b44a0 0x7effbc0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 >> 172.21.2.111:0/2074638654 conn(0x7effbc05c7d0 msgr2=0x7effbc0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:24.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 shutdown_connections 2021-04-03T03:17:24.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.730+0000 7effc10d4700 1 -- 172.21.2.111:0/2074638654 wait complete. 2021-04-03T03:17:24.734 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 63 2021-04-03T03:17:24.737 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:17:24.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 Processor -- start 2021-04-03T03:17:24.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- start start 2021-04-03T03:17:24.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcf540603f0 0x7fcf541447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fcf540add50 0x7fcf54144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540b17b0 0x7fcf54147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf541451c0 con 0x7fcf540603f0 2021-04-03T03:17:24.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf540607d0 con 0x7fcf540add50 2021-04-03T03:17:24.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5414b090 con 0x7fcf540b17b0 2021-04-03T03:17:24.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcf540603f0 0x7fcf541447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fcf540603f0 0x7fcf541447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58600/0 (socket says 172.21.2.111:58600) 2021-04-03T03:17:24.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 -- 172.21.2.111:0/3981053643 learned_addr learned my addr 172.21.2.111:0/3981053643 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:24.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 -- 172.21.2.111:0/3981053643 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540b17b0 msgr2=0x7fcf54147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:24.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 --2- 172.21.2.111:0/3981053643 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540b17b0 0x7fcf54147740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 -- 172.21.2.111:0/3981053643 >> v2:172.21.2.115:3300/0 conn(0x7fcf540add50 msgr2=0x7fcf54144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:24.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 --2- 172.21.2.111:0/3981053643 >> v2:172.21.2.115:3300/0 conn(0x7fcf540add50 0x7fcf54144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 -- 172.21.2.111:0/3981053643 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5414b330 con 0x7fcf540603f0 2021-04-03T03:17:24.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5259c700 1 --2- 172.21.2.111:0/3981053643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf541447e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7fcf48002f30 tx=0x7fcf48006e90).ready entity=mon.0 client_cookie=82ea192ba28f5e4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5159a700 1 -- 172.21.2.111:0/3981053643 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf48009920 con 0x7fcf540603f0 2021-04-03T03:17:24.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5159a700 1 -- 172.21.2.111:0/3981053643 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf48009a80 con 0x7fcf540603f0 2021-04-03T03:17:24.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5159a700 1 -- 172.21.2.111:0/3981053643 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf48009e20 con 0x7fcf540603f0 2021-04-03T03:17:24.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- 172.21.2.111:0/3981053643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 msgr2=0x7fcf541447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 --2- 172.21.2.111:0/3981053643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf541447e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7fcf48002f30 tx=0x7fcf48006e90).stop 2021-04-03T03:17:24.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- 172.21.2.111:0/3981053643 shutdown_connections 2021-04-03T03:17:24.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 --2- 172.21.2.111:0/3981053643 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540b17b0 0x7fcf54147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 --2- 172.21.2.111:0/3981053643 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf541447e0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:24.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 --2- 172.21.2.111:0/3981053643 >> v2:172.21.2.115:3300/0 conn(0x7fcf540add50 0x7fcf54144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- 172.21.2.111:0/3981053643 >> 172.21.2.111:0/3981053643 conn(0x7fcf540b61c0 msgr2=0x7fcf5405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:24.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- 172.21.2.111:0/3981053643 shutdown_connections 2021-04-03T03:17:24.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- 172.21.2.111:0/3981053643 wait complete. 2021-04-03T03:17:24.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 Processor -- start 2021-04-03T03:17:24.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.794+0000 7fcf5359e700 1 -- start start 2021-04-03T03:17:24.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf540b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540add50 0x7fcf54156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf540b17b0 0x7fcf54156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54147640 con 0x7fcf540603f0 2021-04-03T03:17:24.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540a7fe0 con 0x7fcf540b17b0 2021-04-03T03:17:24.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540a8280 con 0x7fcf540add50 2021-04-03T03:17:24.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf540b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf540b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58606/0 (socket says 172.21.2.111:58606) 2021-04-03T03:17:24.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 -- 172.21.2.111:0/656031633 learned_addr learned my addr 172.21.2.111:0/656031633 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:24.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 -- 172.21.2.111:0/656031633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540add50 msgr2=0x7fcf54156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540add50 0x7fcf54156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 -- 172.21.2.111:0/656031633 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf540b17b0 msgr2=0x7fcf54156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:24.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf540b17b0 0x7fcf54156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:24.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 -- 172.21.2.111:0/656031633 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54147aa0 con 0x7fcf540603f0 2021-04-03T03:17:24.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5259c700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf540b49c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7fcf48002f30 tx=0x7fcf48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf437fe700 1 -- 172.21.2.111:0/656031633 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf48011e30 con 0x7fcf540603f0 2021-04-03T03:17:24.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf437fe700 1 -- 172.21.2.111:0/656031633 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4801e460 con 0x7fcf540603f0 2021-04-03T03:17:24.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf54157240 con 0x7fcf540603f0 2021-04-03T03:17:24.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf540abe40 con 0x7fcf540603f0 2021-04-03T03:17:24.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf437fe700 1 -- 172.21.2.111:0/656031633 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf4801e7c0 con 0x7fcf540603f0 2021-04-03T03:17:24.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf540a7240 con 0x7fcf540603f0 2021-04-03T03:17:24.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf437fe700 1 -- 172.21.2.111:0/656031633 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fcf48025030 con 0x7fcf540603f0 2021-04-03T03:17:24.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf437fe700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf30056f70 0x7fcf30059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:24.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf437fe700 1 -- 172.21.2.111:0/656031633 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fcf480717b0 con 0x7fcf540603f0 2021-04-03T03:17:24.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf51d9b700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf30056f70 0x7fcf30059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:24.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.798+0000 7fcf51d9b700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf30056f70 0x7fcf30059430 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fcf4c0032b0 tx=0x7fcf4c006e10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:24.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.802+0000 7fcf437fe700 1 -- 172.21.2.111:0/656031633 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fcf48046ba0 con 0x7fcf540603f0 2021-04-03T03:17:24.927 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:24.926+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcf540a7e10 con 0x7fcf540603f0 2021-04-03T03:17:25.867 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:17:25.866+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:17:25.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.878+0000 7fcf437fe700 1 -- 172.21.2.111:0/656031633 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcf4804a1c0 con 0x7fcf540603f0 2021-04-03T03:17:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf30056f70 msgr2=0x7fcf30059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf30056f70 0x7fcf30059430 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fcf4c0032b0 tx=0x7fcf4c006e10).stop 2021-04-03T03:17:25.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 msgr2=0x7fcf540b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf540b49c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7fcf48002f30 tx=0x7fcf48015000).stop 2021-04-03T03:17:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 shutdown_connections 2021-04-03T03:17:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fcf30056f70 0x7fcf30059430 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:25.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fcf540b17b0 0x7fcf54156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fcf540603f0 0x7fcf540b49c0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 --2- 172.21.2.111:0/656031633 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fcf540add50 0x7fcf54156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 >> 172.21.2.111:0/656031633 conn(0x7fcf540b61c0 msgr2=0x7fcf5414a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 shutdown_connections 2021-04-03T03:17:25.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.882+0000 7fcf5359e700 1 -- 172.21.2.111:0/656031633 wait complete. 2021-04-03T03:17:25.886 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:17:25.890 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:25.893 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:17:25.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 Processor -- start 2021-04-03T03:17:25.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 -- start start 2021-04-03T03:17:25.948 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6c44006bc0 0x7f6c44042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c4403d360 0x7f6c440dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c440dc6e0 con 0x7f6c440dbb90 2021-04-03T03:17:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440d9560 con 0x7f6c44006bc0 2021-04-03T03:17:25.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4403dcb0 con 0x7f6c4403d360 2021-04-03T03:17:25.953 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6c44006bc0 0x7f6c44042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6c44006bc0 0x7f6c44042750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58628/0 (socket says 172.21.2.111:58628) 2021-04-03T03:17:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 -- 172.21.2.111:0/2056028569 learned_addr learned my addr 172.21.2.111:0/2056028569 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 -- 172.21.2.111:0/2056028569 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c4403d360 msgr2=0x7f6c440dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:25.954 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 --2- 172.21.2.111:0/2056028569 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c4403d360 0x7f6c440dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 -- 172.21.2.111:0/2056028569 >> v1:172.21.2.115:6789/0 conn(0x7f6c440dbb90 legacy=0x7f6c440064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 -- 172.21.2.111:0/2056028569 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440dc460 con 0x7f6c44006bc0 2021-04-03T03:17:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4bfff700 1 --2- 172.21.2.111:0/2056028569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c44006bc0 0x7f6c44042750 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7f6c44046210 tx=0x7f6c38006e90).ready entity=mon.0 client_cookie=323b153740f03f5e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:25.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4b7fe700 1 -- 172.21.2.111:0/2056028569 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c380098a0 con 0x7f6c44006bc0 2021-04-03T03:17:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4b7fe700 1 -- 172.21.2.111:0/2056028569 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c38009a00 con 0x7f6c44006bc0 2021-04-03T03:17:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c4b7fe700 1 -- 172.21.2.111:0/2056028569 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c38009da0 con 0x7f6c44006bc0 2021-04-03T03:17:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 -- 172.21.2.111:0/2056028569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c44006bc0 msgr2=0x7f6c44042750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:25.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.946+0000 7f6c51e82700 1 --2- 172.21.2.111:0/2056028569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c44006bc0 0x7f6c44042750 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7f6c44046210 tx=0x7f6c38006e90).stop 2021-04-03T03:17:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- 172.21.2.111:0/2056028569 shutdown_connections 2021-04-03T03:17:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 --2- 172.21.2.111:0/2056028569 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c4403d360 0x7f6c440dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 --2- 172.21.2.111:0/2056028569 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c44006bc0 0x7f6c44042750 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- 172.21.2.111:0/2056028569 >> 172.21.2.111:0/2056028569 conn(0x7f6c440462c0 msgr2=0x7f6c440466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:25.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- 172.21.2.111:0/2056028569 shutdown_connections 2021-04-03T03:17:25.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- 172.21.2.111:0/2056028569 wait complete. 2021-04-03T03:17:25.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 Processor -- start 2021-04-03T03:17:25.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- start start 2021-04-03T03:17:25.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c4403d360 0x7f6c440ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:25.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c440dbb90 0x7f6c4403beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:25.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c440d7b70 0x7f6c440a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:25.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4403c3a0 con 0x7f6c4403d360 2021-04-03T03:17:25.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4403afd0 con 0x7f6c440dbb90 2021-04-03T03:17:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4403ba10 con 0x7f6c440d7b70 2021-04-03T03:17:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c4403d360 0x7f6c440ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c4403d360 0x7f6c440ddc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58634/0 (socket says 172.21.2.111:58634) 2021-04-03T03:17:25.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 -- 172.21.2.111:0/3791135182 learned_addr learned my addr 172.21.2.111:0/3791135182 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 -- 172.21.2.111:0/3791135182 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c440d7b70 msgr2=0x7f6c440a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c440d7b70 0x7f6c440a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 -- 172.21.2.111:0/3791135182 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c440dbb90 msgr2=0x7f6c4403beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:25.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c440dbb90 0x7f6c4403beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 -- 172.21.2.111:0/3791135182 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4409fef0 con 0x7f6c4403d360 2021-04-03T03:17:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c50e80700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c4403d360 0x7f6c440ddc70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f6c40002eb0 tx=0x7f6c40006d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c49ffb700 1 -- 172.21.2.111:0/3791135182 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c4000d4a0 con 0x7f6c4403d360 2021-04-03T03:17:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c49ffb700 1 -- 172.21.2.111:0/3791135182 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40000e90 con 0x7f6c4403d360 2021-04-03T03:17:25.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4403aa50 con 0x7f6c4403d360 2021-04-03T03:17:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c49ffb700 1 -- 172.21.2.111:0/3791135182 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c4000d7f0 con 0x7f6c4403d360 2021-04-03T03:17:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c440a1a10 con 0x7f6c4403d360 2021-04-03T03:17:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4403b850 con 0x7f6c4403d360 2021-04-03T03:17:25.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.950+0000 7f6c49ffb700 1 -- 172.21.2.111:0/3791135182 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6c4000f030 con 0x7f6c4403d360 2021-04-03T03:17:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.954+0000 7f6c49ffb700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c34056fb0 0x7f6c34059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.954+0000 7f6c49ffb700 1 -- 172.21.2.111:0/3791135182 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6c400705d0 con 0x7f6c4403d360 2021-04-03T03:17:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.954+0000 7f6c4bfff700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c34056fb0 0x7f6c34059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:25.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.954+0000 7f6c4bfff700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c34056fb0 0x7f6c34059470 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f6c44046210 tx=0x7f6c38015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:25.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:25.954+0000 7f6c49ffb700 1 -- 172.21.2.111:0/3791135182 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6c40045850 con 0x7f6c4403d360 2021-04-03T03:17:26.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.078+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c4403b190 con 0x7f6c4403d360 2021-04-03T03:17:26.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.078+0000 7f6c49ffb700 1 -- 172.21.2.111:0/3791135182 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f6c40048e70 con 0x7f6c4403d360 2021-04-03T03:17:26.081 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:26.082 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":64,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3135556983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3135556983},{"type":"v1","addr":"172.21.2.115:6833","nonce":3135556983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:15.307326+0000","modified":"2021-04-03T03:17:25.868744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":142,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-04-03T03:17:26.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c34056fb0 msgr2=0x7f6c34059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c34056fb0 0x7f6c34059470 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f6c44046210 tx=0x7f6c38015000).stop 2021-04-03T03:17:26.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c4403d360 msgr2=0x7f6c440ddc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c4403d360 0x7f6c440ddc70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f6c40002eb0 tx=0x7f6c40006d30).stop 2021-04-03T03:17:26.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 shutdown_connections 2021-04-03T03:17:26.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6c440d7b70 0x7f6c440a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:26.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6c4403d360 0x7f6c440ddc70 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6c34056fb0 0x7f6c34059470 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 --2- 172.21.2.111:0/3791135182 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6c440dbb90 0x7f6c4403beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:26.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 >> 172.21.2.111:0/3791135182 conn(0x7f6c440462c0 msgr2=0x7f6c44041560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:26.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 shutdown_connections 2021-04-03T03:17:26.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.082+0000 7f6c51e82700 1 -- 172.21.2.111:0/3791135182 wait complete. 2021-04-03T03:17:26.086 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 64 2021-04-03T03:17:26.090 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:17:26.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 Processor -- start 2021-04-03T03:17:26.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- start start 2021-04-03T03:17:26.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400603f0 0x7f6b401447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b400add50 0x7f6b40144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f6b400b17b0 0x7f6b40147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b401451c0 con 0x7f6b400add50 2021-04-03T03:17:26.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b400607d0 con 0x7f6b400b17b0 2021-04-03T03:17:26.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4014b090 con 0x7f6b400603f0 2021-04-03T03:17:26.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b400add50 0x7f6b40144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6b400add50 0x7f6b40144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58640/0 (socket says 172.21.2.111:58640) 2021-04-03T03:17:26.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 -- 172.21.2.111:0/2227441898 learned_addr learned my addr 172.21.2.111:0/2227441898 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:26.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 -- 172.21.2.111:0/2227441898 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400603f0 msgr2=0x7f6b401447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:26.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 --2- 172.21.2.111:0/2227441898 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400603f0 0x7f6b401447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:26.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 -- 172.21.2.111:0/2227441898 >> v2:172.21.2.115:3300/0 conn(0x7f6b400b17b0 msgr2=0x7f6b40147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:26.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 --2- 172.21.2.111:0/2227441898 >> v2:172.21.2.115:3300/0 conn(0x7f6b400b17b0 0x7f6b40147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 -- 172.21.2.111:0/2227441898 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4014b330 con 0x7f6b400add50 2021-04-03T03:17:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3e5e4700 1 --2- 172.21.2.111:0/2227441898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400add50 0x7f6b40144cd0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f6b30002f30 tx=0x7f6b30006e90).ready entity=mon.0 client_cookie=f9fa2913c558ee55 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3dde3700 1 -- 172.21.2.111:0/2227441898 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b30009920 con 0x7f6b400add50 2021-04-03T03:17:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3dde3700 1 -- 172.21.2.111:0/2227441898 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30009a80 con 0x7f6b400add50 2021-04-03T03:17:26.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3dde3700 1 -- 172.21.2.111:0/2227441898 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b30009de0 con 0x7f6b400add50 2021-04-03T03:17:26.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- 172.21.2.111:0/2227441898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400add50 msgr2=0x7f6b40144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/2227441898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400add50 0x7f6b40144cd0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f6b30002f30 tx=0x7f6b30006e90).stop 2021-04-03T03:17:26.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- 172.21.2.111:0/2227441898 shutdown_connections 2021-04-03T03:17:26.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/2227441898 >> v2:172.21.2.115:3300/0 conn(0x7f6b400b17b0 0x7f6b40147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:26.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/2227441898 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400603f0 0x7f6b401447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:26.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/2227441898 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400add50 0x7f6b40144cd0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- 172.21.2.111:0/2227441898 >> 172.21.2.111:0/2227441898 conn(0x7f6b400b61c0 msgr2=0x7f6b4005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:26.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- 172.21.2.111:0/2227441898 shutdown_connections 2021-04-03T03:17:26.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- 172.21.2.111:0/2227441898 wait complete. 2021-04-03T03:17:26.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 Processor -- start 2021-04-03T03:17:26.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.146+0000 7f6b3fde7700 1 -- start start 2021-04-03T03:17:26.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400603f0 0x7f6b40062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400add50 0x7f6b40156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b400b17b0 0x7f6b401572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40157be0 con 0x7f6b400603f0 2021-04-03T03:17:26.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40156960 con 0x7f6b400b17b0 2021-04-03T03:17:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40067cd0 con 0x7f6b400add50 2021-04-03T03:17:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400603f0 0x7f6b40062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400603f0 0x7f6b40062d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58646/0 (socket says 172.21.2.111:58646) 2021-04-03T03:17:26.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 -- 172.21.2.111:0/3596877282 learned_addr learned my addr 172.21.2.111:0/3596877282 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3f5e6700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b400b17b0 0x7f6b401572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3e5e4700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400add50 0x7f6b40156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 -- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400add50 msgr2=0x7f6b40156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400add50 0x7f6b40156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 -- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b400b17b0 msgr2=0x7f6b401572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b400b17b0 0x7f6b401572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 -- 172.21.2.111:0/3596877282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b40067f70 con 0x7f6b400603f0 2021-04-03T03:17:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3ede5700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400603f0 0x7f6b40062d90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f6b28002f30 tx=0x7f6b28006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:26.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b37fff700 1 -- 172.21.2.111:0/3596877282 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2800d840 con 0x7f6b400603f0 2021-04-03T03:17:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b37fff700 1 -- 172.21.2.111:0/3596877282 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28000e90 con 0x7f6b400603f0 2021-04-03T03:17:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b400a9920 con 0x7f6b400603f0 2021-04-03T03:17:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b37fff700 1 -- 172.21.2.111:0/3596877282 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2800db90 con 0x7f6b400603f0 2021-04-03T03:17:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b400aa470 con 0x7f6b400603f0 2021-04-03T03:17:26.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b40057660 con 0x7f6b400603f0 2021-04-03T03:17:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b37fff700 1 -- 172.21.2.111:0/3596877282 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f6b28017030 con 0x7f6b400603f0 2021-04-03T03:17:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b37fff700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b24056ea0 0x7f6b24059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b37fff700 1 -- 172.21.2.111:0/3596877282 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6b2806f6a0 con 0x7f6b400603f0 2021-04-03T03:17:26.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3e5e4700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b24056ea0 0x7f6b24059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.150+0000 7f6b3e5e4700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b24056ea0 0x7f6b24059360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f6b30017000 tx=0x7f6b30015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:26.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.154+0000 7f6b37fff700 1 -- 172.21.2.111:0/3596877282 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f6b280449d0 con 0x7f6b400603f0 2021-04-03T03:17:26.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.274+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6b40148850 con 0x7f6b400603f0 2021-04-03T03:17:26.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b37fff700 1 -- 172.21.2.111:0/3596877282 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6b28047ff0 con 0x7f6b400603f0 2021-04-03T03:17:26.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b24056ea0 msgr2=0x7f6b24059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b24056ea0 0x7f6b24059360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f6b30017000 tx=0x7f6b30015000).stop 2021-04-03T03:17:26.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400603f0 msgr2=0x7f6b40062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400603f0 0x7f6b40062d90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f6b28002f30 tx=0x7f6b28006ea0).stop 2021-04-03T03:17:26.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 shutdown_connections 2021-04-03T03:17:26.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6b24056ea0 0x7f6b24059360 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6b400b17b0 0x7f6b401572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6b400603f0 0x7f6b40062d90 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 --2- 172.21.2.111:0/3596877282 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f6b400add50 0x7f6b40156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 >> 172.21.2.111:0/3596877282 conn(0x7f6b400b61c0 msgr2=0x7f6b40144980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:26.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 shutdown_connections 2021-04-03T03:17:26.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.886+0000 7f6b3fde7700 1 -- 172.21.2.111:0/3596877282 wait complete. 2021-04-03T03:17:26.897 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:17:26.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14759e700 1 Processor -- start 2021-04-03T03:17:26.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14759e700 1 -- start start 2021-04-03T03:17:26.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb1480603f0 0x7fb1481447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb148144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1480607d0 con 0x7fb1480b17b0 2021-04-03T03:17:26.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1481451c0 con 0x7fb1480603f0 2021-04-03T03:17:26.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14814b090 con 0x7fb1480add50 2021-04-03T03:17:26.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb14659c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fb1480603f0 0x7fb1481447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb145d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb148144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb146d9d700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7fb1480b17b0 0x7fb148147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:43752/0 (socket says 172.21.2.111:43752) 2021-04-03T03:17:26.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.954+0000 7fb146d9d700 1 -- 172.21.2.111:0/1918725149 learned_addr learned my addr 172.21.2.111:0/1918725149 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:26.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb145d9b700 1 -- 172.21.2.111:0/1918725149 >> v2:172.21.2.115:3300/0 conn(0x7fb1480603f0 msgr2=0x7fb1481447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:26.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb145d9b700 1 --2- 172.21.2.111:0/1918725149 >> v2:172.21.2.115:3300/0 conn(0x7fb1480603f0 0x7fb1481447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb145d9b700 1 -- 172.21.2.111:0/1918725149 >> v1:172.21.2.115:6789/0 conn(0x7fb1480b17b0 legacy=0x7fb148147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb145d9b700 1 -- 172.21.2.111:0/1918725149 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14814b330 con 0x7fb1480add50 2021-04-03T03:17:26.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb145d9b700 1 --2- 172.21.2.111:0/1918725149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb148144cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fb138002f30 tx=0x7fb138006dd0).ready entity=mon.2 client_cookie=84680b1f03c66752 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:26.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14559a700 1 -- 172.21.2.111:0/1918725149 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb138009760 con 0x7fb1480add50 2021-04-03T03:17:26.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14559a700 1 -- 172.21.2.111:0/1918725149 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1380098c0 con 0x7fb1480add50 2021-04-03T03:17:26.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14559a700 1 -- 172.21.2.111:0/1918725149 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb138009c20 con 0x7fb1480add50 2021-04-03T03:17:26.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- 172.21.2.111:0/1918725149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 msgr2=0x7fb148144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 --2- 172.21.2.111:0/1918725149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb148144cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fb138002f30 tx=0x7fb138006dd0).stop 2021-04-03T03:17:26.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- 172.21.2.111:0/1918725149 shutdown_connections 2021-04-03T03:17:26.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 --2- 172.21.2.111:0/1918725149 >> v2:172.21.2.115:3300/0 conn(0x7fb1480603f0 0x7fb1481447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 --2- 172.21.2.111:0/1918725149 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb148144cd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- 172.21.2.111:0/1918725149 >> 172.21.2.111:0/1918725149 conn(0x7fb1480b61c0 msgr2=0x7fb14805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:26.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- 172.21.2.111:0/1918725149 shutdown_connections 2021-04-03T03:17:26.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- 172.21.2.111:0/1918725149 wait complete. 2021-04-03T03:17:26.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 Processor -- start 2021-04-03T03:17:26.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- start start 2021-04-03T03:17:26.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1480603f0 0x7fb1480aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb1480ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1480b17b0 0x7fb1480abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1480649f0 con 0x7fb1480603f0 2021-04-03T03:17:26.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb148064c60 con 0x7fb1480b17b0 2021-04-03T03:17:26.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1480a9be0 con 0x7fb1480add50 2021-04-03T03:17:26.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1480603f0 0x7fb1480aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1480603f0 0x7fb1480aac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58658/0 (socket says 172.21.2.111:58658) 2021-04-03T03:17:26.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 -- 172.21.2.111:0/1132242382 learned_addr learned my addr 172.21.2.111:0/1132242382 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:26.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb145d9b700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb1480ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb146d9d700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1480b17b0 0x7fb1480abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 -- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 msgr2=0x7fb1480ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb1480ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 -- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1480b17b0 msgr2=0x7fb1480abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:26.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1480b17b0 0x7fb1480abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:26.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.958+0000 7fb14659c700 1 -- 172.21.2.111:0/1132242382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1480aa080 con 0x7fb1480603f0 2021-04-03T03:17:26.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb14659c700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1480603f0 0x7fb1480aac30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fb13000c000 tx=0x7fb130006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:26.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb13f7fe700 1 -- 172.21.2.111:0/1132242382 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13000cae0 con 0x7fb1480603f0 2021-04-03T03:17:26.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb13f7fe700 1 -- 172.21.2.111:0/1132242382 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13000cc40 con 0x7fb1480603f0 2021-04-03T03:17:26.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb13f7fe700 1 -- 172.21.2.111:0/1132242382 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1300123f0 con 0x7fb1480603f0 2021-04-03T03:17:26.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb148156a40 con 0x7fb1480603f0 2021-04-03T03:17:26.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb148156eb0 con 0x7fb1480603f0 2021-04-03T03:17:26.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb148064e20 con 0x7fb1480603f0 2021-04-03T03:17:26.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb13f7fe700 1 -- 172.21.2.111:0/1132242382 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fb130012550 con 0x7fb1480603f0 2021-04-03T03:17:26.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb13f7fe700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb12c056f20 0x7fb12c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:26.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb13f7fe700 1 -- 172.21.2.111:0/1132242382 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fb130070740 con 0x7fb1480603f0 2021-04-03T03:17:26.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb145d9b700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb12c056f20 0x7fb12c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:26.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb145d9b700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb12c056f20 0x7fb12c0593e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb138019000 tx=0x7fb138015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:26.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:26.962+0000 7fb13f7fe700 1 -- 172.21.2.111:0/1132242382 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fb130045a70 con 0x7fb1480603f0 2021-04-03T03:17:27.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.058+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb148057660 con 0x7fb1480603f0 2021-04-03T03:17:27.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.886+0000 7fb13f7fe700 1 -- 172.21.2.111:0/1132242382 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v143) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb130049090 con 0x7fb1480603f0 2021-04-03T03:17:27.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb12c056f20 msgr2=0x7fb12c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:27.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb12c056f20 0x7fb12c0593e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb138019000 tx=0x7fb138015000).stop 2021-04-03T03:17:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1480603f0 msgr2=0x7fb1480aac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1480603f0 0x7fb1480aac30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fb13000c000 tx=0x7fb130006dd0).stop 2021-04-03T03:17:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 shutdown_connections 2021-04-03T03:17:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fb12c056f20 0x7fb12c0593e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fb1480b17b0 0x7fb1480abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fb1480603f0 0x7fb1480aac30 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 --2- 172.21.2.111:0/1132242382 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fb1480add50 0x7fb1480ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 >> 172.21.2.111:0/1132242382 conn(0x7fb1480b61c0 msgr2=0x7fb148149ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:27.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 shutdown_connections 2021-04-03T03:17:27.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.890+0000 7fb14759e700 1 -- 172.21.2.111:0/1132242382 wait complete. 2021-04-03T03:17:27.893 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:17:27.897 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:17:27.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fb59e700 1 Processor -- start 2021-04-03T03:17:27.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fb59e700 1 -- start start 2021-04-03T03:17:27.956 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fb59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f48f40dbba0 0x7f48f40dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f4041060 0x7f48f403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fb59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40dc560 con 0x7f48f403d560 2021-04-03T03:17:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fb59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f403de60 con 0x7f48f40dbba0 2021-04-03T03:17:27.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40dc800 con 0x7f48f4041060 2021-04-03T03:17:27.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48f9d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f4041060 0x7f48f403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:27.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fad9d700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f48f403d560 0x7f48f4041650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:43764/0 (socket says 172.21.2.111:43764) 2021-04-03T03:17:27.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fa59c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f48f40dbba0 0x7f48f40dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:27.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fad9d700 1 -- 172.21.2.111:0/1733792622 learned_addr learned my addr 172.21.2.111:0/1733792622 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:27.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fa59c700 1 -- 172.21.2.111:0/1733792622 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f4041060 msgr2=0x7f48f403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:27.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fa59c700 1 --2- 172.21.2.111:0/1733792622 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f4041060 0x7f48f403fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fa59c700 1 -- 172.21.2.111:0/1733792622 >> v1:172.21.2.115:6789/0 conn(0x7f48f403d560 legacy=0x7f48f4041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:27.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fa59c700 1 -- 172.21.2.111:0/1733792622 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f403da30 con 0x7f48f40dbba0 2021-04-03T03:17:27.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.954+0000 7f48fa59c700 1 --2- 172.21.2.111:0/1733792622 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f40dbba0 0x7f48f40dbf80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f48f0002f30 tx=0x7f48f0006dd0).ready entity=mon.1 client_cookie=850b66ddcaf1688a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:27.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48f959a700 1 -- 172.21.2.111:0/1733792622 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48f0009760 con 0x7f48f40dbba0 2021-04-03T03:17:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48f959a700 1 -- 172.21.2.111:0/1733792622 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f00098c0 con 0x7f48f40dbba0 2021-04-03T03:17:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48f959a700 1 -- 172.21.2.111:0/1733792622 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48f0009c20 con 0x7f48f40dbba0 2021-04-03T03:17:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/1733792622 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f40dbba0 msgr2=0x7f48f40dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 --2- 172.21.2.111:0/1733792622 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f40dbba0 0x7f48f40dbf80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f48f0002f30 tx=0x7f48f0006dd0).stop 2021-04-03T03:17:27.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/1733792622 shutdown_connections 2021-04-03T03:17:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 --2- 172.21.2.111:0/1733792622 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f40dbba0 0x7f48f40dbf80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 --2- 172.21.2.111:0/1733792622 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f4041060 0x7f48f403fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/1733792622 >> 172.21.2.111:0/1733792622 conn(0x7f48f40462b0 msgr2=0x7f48f40466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:27.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/1733792622 shutdown_connections 2021-04-03T03:17:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/1733792622 wait complete. 2021-04-03T03:17:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 Processor -- start 2021-04-03T03:17:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- start start 2021-04-03T03:17:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f403d560 0x7f48f40a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:27.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f4041060 0x7f48f40ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:27.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48f40dbba0 0x7f48f40a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:27.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40decd0 con 0x7f48f40dbba0 2021-04-03T03:17:27.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f409f820 con 0x7f48f4041060 2021-04-03T03:17:27.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f409fac0 con 0x7f48f403d560 2021-04-03T03:17:27.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48f40dbba0 0x7f48f40a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:27.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48f40dbba0 0x7f48f40a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58674/0 (socket says 172.21.2.111:58674) 2021-04-03T03:17:27.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 -- 172.21.2.111:0/2024673925 learned_addr learned my addr 172.21.2.111:0/2024673925 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:27.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fa59c700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f403d560 0x7f48f40a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:27.966 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48f9d9b700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f4041060 0x7f48f40ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:27.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 -- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f403d560 msgr2=0x7f48f40a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f403d560 0x7f48f40a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 -- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f4041060 msgr2=0x7f48f40ddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f4041060 0x7f48f40ddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:27.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 -- 172.21.2.111:0/2024673925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f409f4a0 con 0x7f48f40dbba0 2021-04-03T03:17:27.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fad9d700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48f40dbba0 0x7f48f40a2450 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f48f403f960 tx=0x7f48fc05ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:27.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48eb7fe700 1 -- 172.21.2.111:0/2024673925 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48fc05df60 con 0x7f48f40dbba0 2021-04-03T03:17:27.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48eb7fe700 1 -- 172.21.2.111:0/2024673925 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc0699f0 con 0x7f48f40dbba0 2021-04-03T03:17:27.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f403be00 con 0x7f48f40dbba0 2021-04-03T03:17:27.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48eb7fe700 1 -- 172.21.2.111:0/2024673925 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48fc06b5d0 con 0x7f48f40dbba0 2021-04-03T03:17:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f40a32a0 con 0x7f48f40dbba0 2021-04-03T03:17:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.958+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f409fc80 con 0x7f48f40dbba0 2021-04-03T03:17:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.962+0000 7f48eb7fe700 1 -- 172.21.2.111:0/2024673925 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f48fc061110 con 0x7f48f40dbba0 2021-04-03T03:17:27.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.962+0000 7f48eb7fe700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d8057050 0x7f48d8059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.962+0000 7f48eb7fe700 1 -- 172.21.2.111:0/2024673925 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6064+0+0 (secure 0 0 0) 0x7f48fc0def60 con 0x7f48f40dbba0 2021-04-03T03:17:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.962+0000 7f48fa59c700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d8057050 0x7f48d8059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.962+0000 7f48fa59c700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d8057050 0x7f48d8059510 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f48f001b000 tx=0x7f48f0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:27.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:27.962+0000 7f48eb7fe700 1 -- 172.21.2.111:0/2024673925 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f48fc0b43a0 con 0x7f48f40dbba0 2021-04-03T03:17:28.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.058+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48f4004db0 con 0x7f48f40dbba0 2021-04-03T03:17:28.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48eb7fe700 1 -- 172.21.2.111:0/2024673925 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v144) v1 ==== 173+0+0 (secure 0 0 0) 0x7f48fc08a020 con 0x7f48f40dbba0 2021-04-03T03:17:28.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d8057050 msgr2=0x7f48d8059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:28.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d8057050 0x7f48d8059510 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f48f001b000 tx=0x7f48f0015000).stop 2021-04-03T03:17:28.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48f40dbba0 msgr2=0x7f48f40a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:28.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48f40dbba0 0x7f48f40a2450 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f48f403f960 tx=0x7f48fc05ad70).stop 2021-04-03T03:17:28.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 shutdown_connections 2021-04-03T03:17:28.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f48f40dbba0 0x7f48f40a2450 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:28.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f48f403d560 0x7f48f40a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:28.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f48d8057050 0x7f48d8059510 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:28.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 --2- 172.21.2.111:0/2024673925 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f48f4041060 0x7f48f40ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:28.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 >> 172.21.2.111:0/2024673925 conn(0x7f48f40462b0 msgr2=0x7f48f403f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:28.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 shutdown_connections 2021-04-03T03:17:28.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:28.898+0000 7f48fb59e700 1 -- 172.21.2.111:0/2024673925 wait complete. 2021-04-03T03:17:28.903 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:17:28.907 INFO:tasks.ceph.mds.a:Restarting daemon 2021-04-03T03:17:28.908 INFO:tasks.ceph.mds.a:Stopping old one... 2021-04-03T03:17:28.908 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T03:17:28.908 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:17:28.909 INFO:tasks.ceph.mds.b:Restarting daemon 2021-04-03T03:17:28.909 INFO:tasks.ceph.mds.b:Stopping old one... 2021-04-03T03:17:28.909 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T03:17:28.909 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T03:17:28.913 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T03:17:28.913 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-04-03T03:17:28.916 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T03:17:28.916 DEBUG:teuthology.orchestra.run.gibba015:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-04-03T03:17:28.959 INFO:tasks.ceph.mds.a:Started 2021-04-03T03:17:28.959 DEBUG:teuthology.parallel:result is None 2021-04-03T03:17:28.960 INFO:tasks.ceph.mds.b:Started 2021-04-03T03:17:28.960 DEBUG:teuthology.parallel:result is None 2021-04-03T03:17:28.961 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2021-04-03T03:17:28.998 INFO:tasks.ceph.mds.b.gibba015.stdout:starting mds.b at 2021-04-03T03:17:28.999 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:17:29.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 Processor -- start 2021-04-03T03:17:29.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- start start 2021-04-03T03:17:29.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8fcc0416d0 0x7f8fcc0406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:29.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8fcc03f5b0 0x7f8fcc0dabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:29.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc047510 con 0x7f8fcc047130 2021-04-03T03:17:29.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc03d540 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0413b0 con 0x7f8fcc0416d0 2021-04-03T03:17:29.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd96c0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f8fcc03f5b0 0x7f8fcc0dabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:29.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd8ebf700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f8fcc047130 0x7f8fcc03eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:43788/0 (socket says 172.21.2.111:43788) 2021-04-03T03:17:29.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd8ebf700 1 -- 172.21.2.111:0/2781626994 learned_addr learned my addr 172.21.2.111:0/2781626994 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:29.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd3fff700 1 --2- 172.21.2.111:0/2781626994 >> v2:172.21.2.115:3300/0 conn(0x7f8fcc0416d0 0x7f8fcc0406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:29.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd96c0700 1 -- 172.21.2.111:0/2781626994 >> v2:172.21.2.115:3300/0 conn(0x7f8fcc0416d0 msgr2=0x7f8fcc0406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:29.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd96c0700 1 --2- 172.21.2.111:0/2781626994 >> v2:172.21.2.115:3300/0 conn(0x7f8fcc0416d0 0x7f8fcc0406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:29.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd96c0700 1 -- 172.21.2.111:0/2781626994 >> v1:172.21.2.115:6789/0 conn(0x7f8fcc047130 legacy=0x7f8fcc03eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:29.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd96c0700 1 -- 172.21.2.111:0/2781626994 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc03ff80 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd96c0700 1 --2- 172.21.2.111:0/2781626994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0dabe0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f8fc8002f30 tx=0x7f8fc8006dd0).ready entity=mon.0 client_cookie=60e4f39b2ad86770 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:29.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd37fe700 1 -- 172.21.2.111:0/2781626994 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc80097b0 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd37fe700 1 -- 172.21.2.111:0/2781626994 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc8009910 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd37fe700 1 -- 172.21.2.111:0/2781626994 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc8009c70 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/2781626994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 msgr2=0x7f8fcc0dabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:29.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/2781626994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0dabe0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f8fc8002f30 tx=0x7f8fc8006dd0).stop 2021-04-03T03:17:29.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/2781626994 shutdown_connections 2021-04-03T03:17:29.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/2781626994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0dabe0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:29.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/2781626994 >> v2:172.21.2.115:3300/0 conn(0x7f8fcc0416d0 0x7f8fcc0406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:29.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/2781626994 >> 172.21.2.111:0/2781626994 conn(0x7f8fcc00d510 msgr2=0x7f8fcc046700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:29.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/2781626994 shutdown_connections 2021-04-03T03:17:29.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.022+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/2781626994 wait complete. 2021-04-03T03:17:29.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 Processor -- start 2021-04-03T03:17:29.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 -- start start 2021-04-03T03:17:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0e7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fcc0416d0 0x7f8fcc0a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fcc047130 0x7f8fcc0d8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0ac5b0 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0eb8c0 con 0x7f8fcc0416d0 2021-04-03T03:17:29.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0e7c10 con 0x7f8fcc047130 2021-04-03T03:17:29.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:29.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fcc0416d0 0x7f8fcc0a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:29.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0e7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58700/0 (socket says 172.21.2.111:58700) 2021-04-03T03:17:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 -- 172.21.2.111:0/3160491491 learned_addr learned my addr 172.21.2.111:0/3160491491 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd96c0700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fcc047130 0x7f8fcc0d8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 -- 172.21.2.111:0/3160491491 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fcc047130 msgr2=0x7f8fcc0d8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fcc047130 0x7f8fcc0d8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:29.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 -- 172.21.2.111:0/3160491491 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fcc0416d0 msgr2=0x7f8fcc0a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:29.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fcc0416d0 0x7f8fcc0a30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:29.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 -- 172.21.2.111:0/3160491491 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0d9760 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd8ebf700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0e7720 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f8fcc040dc0 tx=0x7f8fd405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:29.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd1ffb700 1 -- 172.21.2.111:0/3160491491 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd405df60 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd1ffb700 1 -- 172.21.2.111:0/3160491491 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd40699f0 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc0ebb60 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0dc0e0 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd1ffb700 1 -- 172.21.2.111:0/3160491491 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd406b5d0 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc0ac720 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd1ffb700 1 -- 172.21.2.111:0/3160491491 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8fd4061110 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd1ffb700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8fbc0570c0 0x7f8fbc059580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:29.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd1ffb700 1 -- 172.21.2.111:0/3160491491 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f8fd40def60 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd3fff700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8fbc0570c0 0x7f8fbc059580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:29.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.026+0000 7f8fd3fff700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8fbc0570c0 0x7f8fbc059580 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f8fc000b000 tx=0x7f8fc0006dd0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:29.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.030+0000 7f8fd1ffb700 1 -- 172.21.2.111:0/3160491491 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8fcc0ac720 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:29.134+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f8fcc0ac6f0 con 0x7f8fcc03f5b0 2021-04-03T03:17:29.318 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:29.321 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:29.324 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:29.327 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:29.330 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:30.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd1ffb700 1 -- 172.21.2.111:0/3160491491 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v146) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8fd40b4510 con 0x7f8fcc03f5b0 2021-04-03T03:17:30.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8fbc0570c0 msgr2=0x7f8fbc059580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8fbc0570c0 0x7f8fbc059580 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f8fc000b000 tx=0x7f8fc0006dd0).stop 2021-04-03T03:17:30.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 msgr2=0x7f8fcc0e7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0e7720 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f8fcc040dc0 tx=0x7f8fd405ad70).stop 2021-04-03T03:17:30.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 shutdown_connections 2021-04-03T03:17:30.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8fcc047130 0x7f8fcc0d8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8fcc03f5b0 0x7f8fcc0e7720 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8fbc0570c0 0x7f8fbc059580 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 --2- 172.21.2.111:0/3160491491 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8fcc0416d0 0x7f8fcc0a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 >> 172.21.2.111:0/3160491491 conn(0x7f8fcc00d510 msgr2=0x7f8fcc045760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 shutdown_connections 2021-04-03T03:17:30.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.026+0000 7f8fd9ec1700 1 -- 172.21.2.111:0/3160491491 wait complete. 2021-04-03T03:17:30.032 INFO:teuthology.orchestra.run.gibba011.stderr: removed all blacklist entries 2021-04-03T03:17:30.038 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2021-04-03T03:17:30.039 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2021-04-03T03:17:30.097 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.094+0000 7fd2e141c700 1 Processor -- start 2021-04-03T03:17:30.098 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.094+0000 7fd2e141c700 1 -- start start 2021-04-03T03:17:30.099 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d4042330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd2d4041ea0 0x7fd2d40d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d4040020 con 0x7fd2d4041690 2021-04-03T03:17:30.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d403e350 con 0x7fd2d4041ea0 2021-04-03T03:17:30.100 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4042820 con 0x7fd2d403d680 2021-04-03T03:17:30.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd2d4041ea0 0x7fd2d40d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd2d4041ea0 0x7fd2d40d3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:48414/0 (socket says 172.21.2.111:48414) 2021-04-03T03:17:30.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 -- 172.21.2.111:0/414806399 learned_addr learned my addr 172.21.2.111:0/414806399 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:30.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2dbfff700 1 --2- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d4042330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 -- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 msgr2=0x7fd2d4042330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 --2- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d4042330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 -- 172.21.2.111:0/414806399 >> v1:172.21.2.115:6789/0 conn(0x7fd2d4041690 legacy=0x7fd2d403c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 -- 172.21.2.111:0/414806399 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d403da60 con 0x7fd2d4041ea0 2021-04-03T03:17:30.102 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 --2- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40d3f90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fd2d0002f30 tx=0x7fd2d0006e90).ready entity=mon.1 client_cookie=531bf7b555d8d6d0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2daffd700 1 -- 172.21.2.111:0/414806399 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d0009850 con 0x7fd2d4041ea0 2021-04-03T03:17:30.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2daffd700 1 -- 172.21.2.111:0/414806399 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d00099b0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2daffd700 1 -- 172.21.2.111:0/414806399 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d0009d10 con 0x7fd2d4041ea0 2021-04-03T03:17:30.103 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041ea0 msgr2=0x7fd2d40d3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40d3f90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fd2d0002f30 tx=0x7fd2d0006e90).stop 2021-04-03T03:17:30.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- 172.21.2.111:0/414806399 shutdown_connections 2021-04-03T03:17:30.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d4042330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- 172.21.2.111:0/414806399 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40d3f90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.104 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- 172.21.2.111:0/414806399 >> 172.21.2.111:0/414806399 conn(0x7fd2d40462c0 msgr2=0x7fd2d40466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- 172.21.2.111:0/414806399 shutdown_connections 2021-04-03T03:17:30.105 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- 172.21.2.111:0/414806399 wait complete. 2021-04-03T03:17:30.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 Processor -- start 2021-04-03T03:17:30.108 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- start start 2021-04-03T03:17:30.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041690 0x7fd2d40e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d40e7440 con 0x7fd2d4041ea0 2021-04-03T03:17:30.109 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d403c240 con 0x7fd2d4041690 2021-04-03T03:17:30.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e141c700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d40e6ca0 con 0x7fd2d403d680 2021-04-03T03:17:30.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58714/0 (socket says 172.21.2.111:58714) 2021-04-03T03:17:30.110 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 -- 172.21.2.111:0/448616405 learned_addr learned my addr 172.21.2.111:0/448616405 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:30.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2db7fe700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041690 0x7fd2d40e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2dbfff700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 -- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 msgr2=0x7fd2d403bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.111 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d403bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 -- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041690 msgr2=0x7fd2d40e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041690 0x7fd2d40e67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 -- 172.21.2.111:0/448616405 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d40e6f00 con 0x7fd2d4041ea0 2021-04-03T03:17:30.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2e0c1b700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40cdf00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fd2d4040980 tx=0x7fd2dc05ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.098+0000 7fd2d97fa700 1 -- 172.21.2.111:0/448616405 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc05df60 con 0x7fd2d4041ea0 2021-04-03T03:17:30.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2d97fa700 1 -- 172.21.2.111:0/448616405 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc0699f0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2d97fa700 1 -- 172.21.2.111:0/448616405 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc05f4b0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d40e71a0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d4039cb0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d4004da0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2d97fa700 1 -- 172.21.2.111:0/448616405 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd2dc05f610 con 0x7fd2d4041ea0 2021-04-03T03:17:30.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2d97fa700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2bc056fb0 0x7fd2bc059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2d97fa700 1 -- 172.21.2.111:0/448616405 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fd2dc0df210 con 0x7fd2d4041ea0 2021-04-03T03:17:30.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2dbfff700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2bc056fb0 0x7fd2bc059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2dbfff700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2bc056fb0 0x7fd2bc059470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd2cc00b000 tx=0x7fd2cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.102+0000 7fd2d97fa700 1 -- 172.21.2.111:0/448616405 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd2dc0b47a0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd2d4046ff0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2d97fa700 1 -- 172.21.2.111:0/448616405 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fd2dc0b7dc0 con 0x7fd2d4041ea0 2021-04-03T03:17:30.232 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:30.232 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:17:30.232 INFO:teuthology.orchestra.run.gibba011.stdout: "auth_dump": [ 2021-04-03T03:17:30.233 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.233 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.a", 2021-04-03T03:17:30.233 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8M0wDBAACz+ZCJdGjeE/DQ8JCeGzHw==", 2021-04-03T03:17:30.233 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.233 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:30.234 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.234 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.234 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.234 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.235 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.235 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.235 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mds.b", 2021-04-03T03:17:30.235 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgoKuPERAAcQtZKG1oRsT0FpWTVGlrDA==", 2021-04-03T03:17:30.236 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.236 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:30.236 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.236 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.236 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.237 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.237 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.237 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.237 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.0", 2021-04-03T03:17:30.237 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAQ3GdgPWWOHBAAQSfMn2E6ETUDgbZCJPQaPQ==", 2021-04-03T03:17:30.238 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.238 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.238 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.238 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.239 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.239 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.239 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.239 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.1", 2021-04-03T03:17:30.239 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAT3Gdg5rwEAhAAj/Va7kUsXvjeuR8zAGoT3Q==", 2021-04-03T03:17:30.240 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.240 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.240 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.240 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.241 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.241 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.241 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.241 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.2", 2021-04-03T03:17:30.241 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAV3GdgyfrPIxAAVbOXP21+gLw7A2KDvLBqJg==", 2021-04-03T03:17:30.242 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.242 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.242 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.242 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.242 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.243 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.243 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.243 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.3", 2021-04-03T03:17:30.243 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAY3GdgwSe7CxAAgEGP99bv7tL32hZ/bgCKtQ==", 2021-04-03T03:17:30.244 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.244 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.244 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.244 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.244 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.245 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.245 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.245 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.4", 2021-04-03T03:17:30.245 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAb3Gdg+3ckMRAAThzKGGXVB/j/CZZZHGV73Q==", 2021-04-03T03:17:30.246 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.246 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.246 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.246 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.246 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.247 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.247 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.247 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.5", 2021-04-03T03:17:30.247 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAe3Gdg5XnTFRAAYaMevU+bFcMH+HeO5Ui6yQ==", 2021-04-03T03:17:30.248 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.248 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.248 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.248 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.248 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.249 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.249 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.249 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.6", 2021-04-03T03:17:30.249 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAg3GdgUmWbNxAACxJ5vRJWJZERHQSQwHdaYA==", 2021-04-03T03:17:30.249 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.250 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.250 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.250 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.250 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.251 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.251 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.251 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "osd.7", 2021-04-03T03:17:30.251 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAj3Gdgx/6tHBAA2LIjBc72dIz3929QVdPbyA==", 2021-04-03T03:17:30.251 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.252 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.252 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.252 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.253 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.253 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.253 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.0", 2021-04-03T03:17:30.253 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg8hgJFhAAqhAiWS2fS5Zica5XPZhiEg==", 2021-04-03T03:17:30.253 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:30.254 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:17:30.254 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:17:30.254 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.255 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.255 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.255 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.1", 2021-04-03T03:17:30.255 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgVneAFxAAMnGgqeHVFG2joPPpk43exQ==", 2021-04-03T03:17:30.255 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:30.256 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:17:30.256 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:17:30.256 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.256 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.257 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.257 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.2", 2021-04-03T03:17:30.257 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgBek3GRAAeiBPjrVeFxTMgsKrCebdZA==", 2021-04-03T03:17:30.257 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow", 2021-04-03T03:17:30.258 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow r", 2021-04-03T03:17:30.258 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow rw pool=cephfs_data" 2021-04-03T03:17:30.258 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.258 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.259 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.259 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "client.admin", 2021-04-03T03:17:30.259 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAO3GdgT5JiJxAAY0hYiecsFu8MVBmBxrvsUw==", 2021-04-03T03:17:30.259 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.259 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:17:30.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr": "allow *", 2021-04-03T03:17:30.260 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow *", 2021-04-03T03:17:30.260 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.260 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.260 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.261 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.261 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.x", 2021-04-03T03:17:30.261 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3GdgE//OCRAAO2EbQ0nPuu5lWrQiRgVRUg==", 2021-04-03T03:17:30.261 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:17:30.262 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:17:30.262 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.262 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.262 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.263 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.263 INFO:teuthology.orchestra.run.gibba011.stdout: "entity": "mgr.y", 2021-04-03T03:17:30.263 INFO:teuthology.orchestra.run.gibba011.stdout: "key": "AQAP3Gdg+9pHCBAAHRy4dUnZ3M18zS5dnKt/GA==", 2021-04-03T03:17:30.263 INFO:teuthology.orchestra.run.gibba011.stdout: "caps": { 2021-04-03T03:17:30.263 INFO:teuthology.orchestra.run.gibba011.stdout: "mds": "allow *", 2021-04-03T03:17:30.264 INFO:teuthology.orchestra.run.gibba011.stdout: "mon": "allow profile mgr", 2021-04-03T03:17:30.264 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": "allow *" 2021-04-03T03:17:30.264 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.264 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:30.265 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:30.265 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:17:30.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2bc056fb0 msgr2=0x7fd2bc059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2bc056fb0 0x7fd2bc059470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd2cc00b000 tx=0x7fd2cc008040).stop 2021-04-03T03:17:30.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d4041ea0 msgr2=0x7fd2d40cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40cdf00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fd2d4040980 tx=0x7fd2dc05ad70).stop 2021-04-03T03:17:30.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 shutdown_connections 2021-04-03T03:17:30.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2d4041ea0 0x7fd2d40cdf00 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2d403d680 0x7fd2d403bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd2bc056fb0 0x7fd2bc059470 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 --2- 172.21.2.111:0/448616405 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd2d4041690 0x7fd2d40e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 >> 172.21.2.111:0/448616405 conn(0x7fd2d40462c0 msgr2=0x7fd2d4006760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 shutdown_connections 2021-04-03T03:17:30.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.230+0000 7fd2e141c700 1 -- 172.21.2.111:0/448616405 wait complete. 2021-04-03T03:17:30.270 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:30.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 Processor -- start 2021-04-03T03:17:30.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- start start 2021-04-03T03:17:30.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f8a840b12b0 0x7f8a841522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a840607d0 con 0x7f8a8414b700 2021-04-03T03:17:30.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a840b0eb0 con 0x7f8a840603f0 2021-04-03T03:17:30.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a840af400 con 0x7f8a840b12b0 2021-04-03T03:17:30.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8939e700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f8a8414b700 0x7f8a840b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59514/0 (socket says 172.21.2.111:59514) 2021-04-03T03:17:30.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8939e700 1 -- 172.21.2.111:0/2303300067 learned_addr learned my addr 172.21.2.111:0/2303300067 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:30.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/2303300067 >> v2:172.21.2.115:3300/0 conn(0x7f8a840b12b0 0x7f8a841522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462401250 0 0) 0x7f8a840607d0 con 0x7f8a8414b700 2021-04-03T03:17:30.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a6c003240 con 0x7f8a8414b700 2021-04-03T03:17:30.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3827852648 0 0) 0x7f8a6c003240 con 0x7f8a8414b700 2021-04-03T03:17:30.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 >> v2:172.21.2.115:3300/0 conn(0x7f8a840b12b0 msgr2=0x7f8a841522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 --2- 172.21.2.111:0/2303300067 >> v2:172.21.2.115:3300/0 conn(0x7f8a840b12b0 0x7f8a841522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 >> v1:172.21.2.115:6789/0 conn(0x7f8a840603f0 legacy=0x7f8a840b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840af6a0 con 0x7f8a8414b700 2021-04-03T03:17:30.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f8a780016d0 con 0x7f8a8414b700 2021-04-03T03:17:30.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 >> v1:172.21.2.111:6789/0 conn(0x7f8a8414b700 legacy=0x7f8a840b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a6c0037b0 0x7f8a6c003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a6c0044e0 0x7f8a6c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a6c004e50 0x7f8a6c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a840af400 con 0x7f8a6c004e50 2021-04-03T03:17:30.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c005300 con 0x7f8a6c0044e0 2021-04-03T03:17:30.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c00bc20 con 0x7f8a6c0037b0 2021-04-03T03:17:30.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a6c004e50 0x7f8a6c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/2303300067 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a6c0037b0 msgr2=0x7f8a6c003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a6c0037b0 0x7f8a6c003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/2303300067 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a6c0044e0 msgr2=0x7f8a6c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a6c0044e0 0x7f8a6c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/2303300067 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c003240 con 0x7f8a6c004e50 2021-04-03T03:17:30.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/2303300067 reap_dead start 2021-04-03T03:17:30.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a6c004e50 0x7f8a6c00b730 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f8a7c00c000 tx=0x7f8a7c0091d0).ready entity=mon.0 client_cookie=1a4655b91632cf14 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7c00eca0 con 0x7f8a6c004e50 2021-04-03T03:17:30.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c00ee00 con 0x7f8a6c004e50 2021-04-03T03:17:30.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a88b9d700 1 -- 172.21.2.111:0/2303300067 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7c0125e0 con 0x7f8a6c004e50 2021-04-03T03:17:30.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- 172.21.2.111:0/2303300067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a6c004e50 msgr2=0x7f8a6c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a6c004e50 0x7f8a6c00b730 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f8a7c00c000 tx=0x7f8a7c0091d0).stop 2021-04-03T03:17:30.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- 172.21.2.111:0/2303300067 shutdown_connections 2021-04-03T03:17:30.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/2303300067 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a6c004e50 0x7f8a6c00b730 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- 172.21.2.111:0/2303300067 >> 172.21.2.111:0/2303300067 conn(0x7f8a840b61c0 msgr2=0x7f8a8405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.485 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- 172.21.2.111:0/2303300067 shutdown_connections 2021-04-03T03:17:30.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.366+0000 7f8a8aba1700 1 -- 172.21.2.111:0/2303300067 wait complete. 2021-04-03T03:17:30.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 Processor -- start 2021-04-03T03:17:30.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 -- start start 2021-04-03T03:17:30.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a6c004e50 0x7f8a840ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a840aacb0 0x7f8a84065460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a84065ca0 0x7f8a84156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84063790 con 0x7f8a84065ca0 2021-04-03T03:17:30.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84063a30 con 0x7f8a6c004e50 2021-04-03T03:17:30.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a840aa810 con 0x7f8a840aacb0 2021-04-03T03:17:30.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a84065ca0 0x7f8a84156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a84065ca0 0x7f8a84156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58732/0 (socket says 172.21.2.111:58732) 2021-04-03T03:17:30.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/3562465156 learned_addr learned my addr 172.21.2.111:0/3562465156 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:30.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/3562465156 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a840aacb0 msgr2=0x7f8a84065460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a840aacb0 0x7f8a84065460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/3562465156 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a6c004e50 msgr2=0x7f8a840ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a6c004e50 0x7f8a840ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 -- 172.21.2.111:0/3562465156 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840634b0 con 0x7f8a84065ca0 2021-04-03T03:17:30.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8a3a0700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a84065ca0 0x7f8a84156240 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7f8a7c00c000 tx=0x7f8a7c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a76ffd700 1 -- 172.21.2.111:0/3562465156 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7c01b630 con 0x7f8a84065ca0 2021-04-03T03:17:30.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a76ffd700 1 -- 172.21.2.111:0/3562465156 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c01b790 con 0x7f8a84065ca0 2021-04-03T03:17:30.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a84156730 con 0x7f8a84065ca0 2021-04-03T03:17:30.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a76ffd700 1 -- 172.21.2.111:0/3562465156 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7c01baf0 con 0x7f8a84065ca0 2021-04-03T03:17:30.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a84156c20 con 0x7f8a84065ca0 2021-04-03T03:17:30.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a840aa9d0 con 0x7f8a84065ca0 2021-04-03T03:17:30.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a76ffd700 1 -- 172.21.2.111:0/3562465156 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f8a7c01bc50 con 0x7f8a84065ca0 2021-04-03T03:17:30.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a76ffd700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a64056ea0 0x7f8a64059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a76ffd700 1 -- 172.21.2.111:0/3562465156 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5596+0+0 (secure 0 0 0) 0x7f8a7c071060 con 0x7f8a84065ca0 2021-04-03T03:17:30.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a89b9f700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a64056ea0 0x7f8a64059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.370+0000 7f8a89b9f700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a64056ea0 0x7f8a64059360 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f8a8406a3b0 tx=0x7f8a800077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.374+0000 7f8a76ffd700 1 -- 172.21.2.111:0/3562465156 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8a7c046670 con 0x7f8a84065ca0 2021-04-03T03:17:30.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a84057660 con 0x7f8a84065ca0 2021-04-03T03:17:30.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a76ffd700 1 -- 172.21.2.111:0/3562465156 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f8a7c016020 con 0x7f8a84065ca0 2021-04-03T03:17:30.493 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:30.493 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":146,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:30.024224+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a64056ea0 msgr2=0x7f8a64059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a64056ea0 0x7f8a64059360 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f8a8406a3b0 tx=0x7f8a800077d0).stop 2021-04-03T03:17:30.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a84065ca0 msgr2=0x7f8a84156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a84065ca0 0x7f8a84156240 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7f8a7c00c000 tx=0x7f8a7c01a000).stop 2021-04-03T03:17:30.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 shutdown_connections 2021-04-03T03:17:30.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f8a84065ca0 0x7f8a84156240 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8a64056ea0 0x7f8a64059360 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f8a6c004e50 0x7f8a840ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 --2- 172.21.2.111:0/3562465156 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f8a840aacb0 0x7f8a84065460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 >> 172.21.2.111:0/3562465156 conn(0x7f8a840b61c0 msgr2=0x7f8a84149bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 shutdown_connections 2021-04-03T03:17:30.498 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.466+0000 7f8a8aba1700 1 -- 172.21.2.111:0/3562465156 wait complete. 2021-04-03T03:17:30.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-04-03T03:17:30.500 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:17:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.558+0000 7ff3afde7700 1 Processor -- start 2021-04-03T03:17:30.561 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.558+0000 7ff3afde7700 1 -- start start 2021-04-03T03:17:30.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b00603f0 0x7ff3b01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff3b00add50 0x7ff3b0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b00607d0 con 0x7ff3b00b17b0 2021-04-03T03:17:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b01451c0 con 0x7ff3b00add50 2021-04-03T03:17:30.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b014b090 con 0x7ff3b00603f0 2021-04-03T03:17:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3af5e6700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7ff3b00b17b0 0x7ff3b0147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59532/0 (socket says 172.21.2.111:59532) 2021-04-03T03:17:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3af5e6700 1 -- 172.21.2.111:0/1078582093 learned_addr learned my addr 172.21.2.111:0/1078582093 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892613550 0 0) 0x7ff3b00607d0 con 0x7ff3b00b17b0 2021-04-03T03:17:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff390003240 con 0x7ff3b00b17b0 2021-04-03T03:17:30.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 --2- 172.21.2.111:0/1078582093 >> v2:172.21.2.115:3300/0 conn(0x7ff3b00add50 0x7ff3b0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3aede5700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b00603f0 0x7ff3b01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1946015332 0 0) 0x7ff390003240 con 0x7ff3b00b17b0 2021-04-03T03:17:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b00603f0 msgr2=0x7ff3b01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b00603f0 0x7ff3b01447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 >> v2:172.21.2.115:3300/0 conn(0x7ff3b00add50 msgr2=0x7ff3b0144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 --2- 172.21.2.111:0/1078582093 >> v2:172.21.2.115:3300/0 conn(0x7ff3b00add50 0x7ff3b0144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b014b330 con 0x7ff3b00b17b0 2021-04-03T03:17:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7ff3a8001a50 con 0x7ff3b00b17b0 2021-04-03T03:17:30.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 >> v1:172.21.2.111:6789/0 conn(0x7ff3b00b17b0 legacy=0x7ff3b0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff390003730 0x7ff390003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3900044e0 0x7ff390004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b00603f0 0x7ff390004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b014b090 con 0x7ff3900044e0 2021-04-03T03:17:30.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390005390 con 0x7ff390003730 2021-04-03T03:17:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900042f0 con 0x7ff3b00603f0 2021-04-03T03:17:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3900044e0 0x7ff390004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 -- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b00603f0 msgr2=0x7ff390004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b00603f0 0x7ff390004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 -- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff390003730 msgr2=0x7ff390003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff390003730 0x7ff390003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 -- 172.21.2.111:0/1078582093 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390003240 con 0x7ff3900044e0 2021-04-03T03:17:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3ae5e4700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3900044e0 0x7ff390004960 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7ff3a000a000 tx=0x7ff3a0008040).ready entity=mon.0 client_cookie=f26c48337c430430 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a000da00 con 0x7ff3900044e0 2021-04-03T03:17:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a000db60 con 0x7ff3900044e0 2021-04-03T03:17:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3adde3700 1 -- 172.21.2.111:0/1078582093 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a0013480 con 0x7ff3900044e0 2021-04-03T03:17:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- 172.21.2.111:0/1078582093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3900044e0 msgr2=0x7ff390004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 --2- 172.21.2.111:0/1078582093 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3900044e0 0x7ff390004960 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7ff3a000a000 tx=0x7ff3a0008040).stop 2021-04-03T03:17:30.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3af5e6700 1 -- 172.21.2.111:0/1078582093 reap_dead start 2021-04-03T03:17:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- 172.21.2.111:0/1078582093 shutdown_connections 2021-04-03T03:17:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- 172.21.2.111:0/1078582093 >> 172.21.2.111:0/1078582093 conn(0x7ff3b00b61c0 msgr2=0x7ff3b005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- 172.21.2.111:0/1078582093 shutdown_connections 2021-04-03T03:17:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- 172.21.2.111:0/1078582093 wait complete. 2021-04-03T03:17:30.573 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 Processor -- start 2021-04-03T03:17:30.574 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- start start 2021-04-03T03:17:30.576 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b014a410 0x7ff3b015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b0156d80 0x7ff3b0157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3b01576f0 0x7ff3b00b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0156990 con 0x7ff3b014a410 2021-04-03T03:17:30.577 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00a9170 con 0x7ff3b01576f0 2021-04-03T03:17:30.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3afde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b015ae20 con 0x7ff3b0156d80 2021-04-03T03:17:30.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3aede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b014a410 0x7ff3b015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3aede5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b014a410 0x7ff3b015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58750/0 (socket says 172.21.2.111:58750) 2021-04-03T03:17:30.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.562+0000 7ff3aede5700 1 -- 172.21.2.111:0/728007860 learned_addr learned my addr 172.21.2.111:0/728007860 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:30.578 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3aede5700 1 -- 172.21.2.111:0/728007860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b0156d80 msgr2=0x7ff3b0157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3aede5700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b0156d80 0x7ff3b0157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3aede5700 1 -- 172.21.2.111:0/728007860 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3b01576f0 msgr2=0x7ff3b00b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3aede5700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3b01576f0 0x7ff3b00b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.579 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3aede5700 1 -- 172.21.2.111:0/728007860 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b014a7f0 con 0x7ff3b014a410 2021-04-03T03:17:30.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3aede5700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b014a410 0x7ff3b015a930 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7ff3b00b4ab0 tx=0x7ff398008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3a7fff700 1 -- 172.21.2.111:0/728007860 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff398015070 con 0x7ff3b014a410 2021-04-03T03:17:30.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3a7fff700 1 -- 172.21.2.111:0/728007860 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff398010580 con 0x7ff3b014a410 2021-04-03T03:17:30.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3a7fff700 1 -- 172.21.2.111:0/728007860 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3980108e0 con 0x7ff3b014a410 2021-04-03T03:17:30.580 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b00a93a0 con 0x7ff3b014a410 2021-04-03T03:17:30.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b00b4650 con 0x7ff3b014a410 2021-04-03T03:17:30.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0057660 con 0x7ff3b014a410 2021-04-03T03:17:30.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3a7fff700 1 -- 172.21.2.111:0/728007860 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff398010a40 con 0x7ff3b014a410 2021-04-03T03:17:30.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3a7fff700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff394056f20 0x7ff3940593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3a7fff700 1 -- 172.21.2.111:0/728007860 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5596+0+0 (secure 0 0 0) 0x7ff39806fda0 con 0x7ff3b014a410 2021-04-03T03:17:30.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3ae5e4700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff394056f20 0x7ff3940593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.566+0000 7ff3ae5e4700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff394056f20 0x7ff3940593e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7ff3a000a000 tx=0x7ff3a0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.570+0000 7ff3a7fff700 1 -- 172.21.2.111:0/728007860 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff3980454c0 con 0x7ff3b014a410 2021-04-03T03:17:30.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff3b0067fb0 con 0x7ff3b014a410 2021-04-03T03:17:30.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3a7fff700 1 -- 172.21.2.111:0/728007860 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v146) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff398048ae0 con 0x7ff3b014a410 2021-04-03T03:17:30.664 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:17:30.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff394056f20 msgr2=0x7ff3940593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff394056f20 0x7ff3940593e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7ff3a000a000 tx=0x7ff3a0008040).stop 2021-04-03T03:17:30.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b014a410 msgr2=0x7ff3b015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b014a410 0x7ff3b015a930 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7ff3b00b4ab0 tx=0x7ff398008040).stop 2021-04-03T03:17:30.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 shutdown_connections 2021-04-03T03:17:30.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff394056f20 0x7ff3940593e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3b01576f0 0x7ff3b00b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3b014a410 0x7ff3b015a930 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 --2- 172.21.2.111:0/728007860 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3b0156d80 0x7ff3b0157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 >> 172.21.2.111:0/728007860 conn(0x7ff3b00b61c0 msgr2=0x7ff3b014d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 shutdown_connections 2021-04-03T03:17:30.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.662+0000 7ff3afde7700 1 -- 172.21.2.111:0/728007860 wait complete. 2021-04-03T03:17:30.671 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:17:30.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef66efd700 1 Processor -- start 2021-04-03T03:17:30.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef66efd700 1 -- start start 2021-04-03T03:17:30.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef66efd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600603f0 0x7fef601447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef66efd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fef600b17b0 0x7fef60144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef66efd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef600607d0 con 0x7fef600add50 2021-04-03T03:17:30.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef66efd700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef601451c0 con 0x7fef600b17b0 2021-04-03T03:17:30.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef66efd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6014b090 con 0x7fef600603f0 2021-04-03T03:17:30.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef666fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fef600b17b0 0x7fef60144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef666fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fef600b17b0 0x7fef60144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58762/0 (socket says 172.21.2.111:58762) 2021-04-03T03:17:30.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef666fc700 1 -- 172.21.2.111:0/1960642994 learned_addr learned my addr 172.21.2.111:0/1960642994 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef666fc700 1 -- 172.21.2.111:0/1960642994 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600603f0 msgr2=0x7fef601447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.726+0000 7fef666fc700 1 --2- 172.21.2.111:0/1960642994 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600603f0 0x7fef601447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef666fc700 1 -- 172.21.2.111:0/1960642994 >> v1:172.21.2.115:6789/0 conn(0x7fef600add50 legacy=0x7fef60147740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:30.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef666fc700 1 -- 172.21.2.111:0/1960642994 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6014b330 con 0x7fef600b17b0 2021-04-03T03:17:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef666fc700 1 --2- 172.21.2.111:0/1960642994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600b17b0 0x7fef60144cd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7fef600b5600 tx=0x7fef58009270).ready entity=mon.0 client_cookie=5ab129d4df9f7a32 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef64ef9700 1 -- 172.21.2.111:0/1960642994 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef5800bbc0 con 0x7fef600b17b0 2021-04-03T03:17:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef64ef9700 1 -- 172.21.2.111:0/1960642994 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5800bd20 con 0x7fef600b17b0 2021-04-03T03:17:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef64ef9700 1 -- 172.21.2.111:0/1960642994 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef58011400 con 0x7fef600b17b0 2021-04-03T03:17:30.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/1960642994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600b17b0 msgr2=0x7fef60144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 --2- 172.21.2.111:0/1960642994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600b17b0 0x7fef60144cd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7fef600b5600 tx=0x7fef58009270).stop 2021-04-03T03:17:30.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/1960642994 shutdown_connections 2021-04-03T03:17:30.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 --2- 172.21.2.111:0/1960642994 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600b17b0 0x7fef60144cd0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 --2- 172.21.2.111:0/1960642994 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600603f0 0x7fef601447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/1960642994 >> 172.21.2.111:0/1960642994 conn(0x7fef600b61c0 msgr2=0x7fef6005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/1960642994 shutdown_connections 2021-04-03T03:17:30.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/1960642994 wait complete. 2021-04-03T03:17:30.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 Processor -- start 2021-04-03T03:17:30.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- start start 2021-04-03T03:17:30.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600603f0 0x7fef60145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef600add50 0x7fef6015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.739 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600b17b0 0x7fef6015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef60146910 con 0x7fef600603f0 2021-04-03T03:17:30.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef600a6fc0 con 0x7fef600add50 2021-04-03T03:17:30.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef600ab3c0 con 0x7fef600b17b0 2021-04-03T03:17:30.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600603f0 0x7fef60145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600603f0 0x7fef60145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58764/0 (socket says 172.21.2.111:58764) 2021-04-03T03:17:30.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 -- 172.21.2.111:0/4035435433 learned_addr learned my addr 172.21.2.111:0/4035435433 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:30.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 -- 172.21.2.111:0/4035435433 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600b17b0 msgr2=0x7fef6015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600b17b0 0x7fef6015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.741 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 -- 172.21.2.111:0/4035435433 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef600add50 msgr2=0x7fef6015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef600add50 0x7fef6015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 -- 172.21.2.111:0/4035435433 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef600a7750 con 0x7fef600603f0 2021-04-03T03:17:30.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef65efb700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600603f0 0x7fef60145bd0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fef600a96a0 tx=0x7fef5c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef4effd700 1 -- 172.21.2.111:0/4035435433 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef5c00d570 con 0x7fef600603f0 2021-04-03T03:17:30.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef4effd700 1 -- 172.21.2.111:0/4035435433 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c00d6d0 con 0x7fef600603f0 2021-04-03T03:17:30.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef600ab620 con 0x7fef600603f0 2021-04-03T03:17:30.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef4effd700 1 -- 172.21.2.111:0/4035435433 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef5c00da30 con 0x7fef600603f0 2021-04-03T03:17:30.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef60148470 con 0x7fef600603f0 2021-04-03T03:17:30.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.730+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef60057660 con 0x7fef600603f0 2021-04-03T03:17:30.743 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.734+0000 7fef4effd700 1 -- 172.21.2.111:0/4035435433 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fef5c00f070 con 0x7fef600603f0 2021-04-03T03:17:30.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.734+0000 7fef4effd700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef40056f70 0x7fef40059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.734+0000 7fef4effd700 1 -- 172.21.2.111:0/4035435433 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fef5c06fac0 con 0x7fef600603f0 2021-04-03T03:17:30.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.734+0000 7fef656fa700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef40056f70 0x7fef40059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.734+0000 7fef656fa700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef40056f70 0x7fef40059430 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fef50002f70 tx=0x7fef500069b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.744 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.734+0000 7fef4effd700 1 -- 172.21.2.111:0/4035435433 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fef5c071e60 con 0x7fef600603f0 2021-04-03T03:17:30.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fef600a7100 con 0x7fef600603f0 2021-04-03T03:17:30.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef4effd700 1 -- 172.21.2.111:0/4035435433 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v146) v1 ==== 92+0+10 (secure 0 0 0) 0x7fef5c0450d0 con 0x7fef600603f0 2021-04-03T03:17:30.831 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:30.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef40056f70 msgr2=0x7fef40059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef40056f70 0x7fef40059430 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fef50002f70 tx=0x7fef500069b0).stop 2021-04-03T03:17:30.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600603f0 msgr2=0x7fef60145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600603f0 0x7fef60145bd0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fef600a96a0 tx=0x7fef5c006dd0).stop 2021-04-03T03:17:30.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 shutdown_connections 2021-04-03T03:17:30.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fef600b17b0 0x7fef6015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fef600603f0 0x7fef60145bd0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fef40056f70 0x7fef40059430 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 --2- 172.21.2.111:0/4035435433 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fef600add50 0x7fef6015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 >> 172.21.2.111:0/4035435433 conn(0x7fef600b61c0 msgr2=0x7fef600b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.834 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 shutdown_connections 2021-04-03T03:17:30.835 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.830+0000 7fef66efd700 1 -- 172.21.2.111:0/4035435433 wait complete. 2021-04-03T03:17:30.839 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2021-04-03T03:17:30.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 Processor -- start 2021-04-03T03:17:30.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- start start 2021-04-03T03:17:30.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe9780603f0 0x7fe9780b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780af400 con 0x7fe97814b700 2021-04-03T03:17:30.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780af6a0 con 0x7fe9780b0c90 2021-04-03T03:17:30.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe978060860 con 0x7fe9780603f0 2021-04-03T03:17:30.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe97814b700 0x7fe9780b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59566/0 (socket says 172.21.2.111:59566) 2021-04-03T03:17:30.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 -- 172.21.2.111:0/3370149235 learned_addr learned my addr 172.21.2.111:0/3370149235 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:30.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426621999 0 0) 0x7fe9780af400 con 0x7fe97814b700 2021-04-03T03:17:30.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe964003240 con 0x7fe97814b700 2021-04-03T03:17:30.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe977fff700 1 --2- 172.21.2.111:0/3370149235 >> v2:172.21.2.115:3300/0 conn(0x7fe9780603f0 0x7fe9780b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4165770540 0 0) 0x7fe964003240 con 0x7fe97814b700 2021-04-03T03:17:30.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 >> v2:172.21.2.115:3300/0 conn(0x7fe9780603f0 msgr2=0x7fe9780b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 --2- 172.21.2.111:0/3370149235 >> v2:172.21.2.115:3300/0 conn(0x7fe9780603f0 0x7fe9780b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 >> v1:172.21.2.115:6789/0 conn(0x7fe9780b0c90 legacy=0x7fe978147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780afb40 con 0x7fe97814b700 2021-04-03T03:17:30.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe96c0016d0 con 0x7fe97814b700 2021-04-03T03:17:30.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 >> v1:172.21.2.111:6789/0 conn(0x7fe97814b700 legacy=0x7fe9780b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9640037b0 0x7fe964003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe964004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe964004e50 0x7fe96400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978060860 con 0x7fe9640044e0 2021-04-03T03:17:30.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe964005300 con 0x7fe9640037b0 2021-04-03T03:17:30.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96400bc20 con 0x7fe964004e50 2021-04-03T03:17:30.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe964004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 -- 172.21.2.111:0/3370149235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe964004e50 msgr2=0x7fe96400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe964004e50 0x7fe96400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 -- 172.21.2.111:0/3370149235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9640037b0 msgr2=0x7fe964003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:30.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9640037b0 0x7fe964003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 -- 172.21.2.111:0/3370149235 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780afb40 con 0x7fe9640044e0 2021-04-03T03:17:30.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97c9b6700 1 -- 172.21.2.111:0/3370149235 reap_dead start 2021-04-03T03:17:30.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe9777fe700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe964004960 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fe96c000fe0 tx=0x7fe96c0077d0).ready entity=mon.0 client_cookie=f45a59e3de2174b0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe96c00ce20 con 0x7fe9640044e0 2021-04-03T03:17:30.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c0072e0 con 0x7fe9640044e0 2021-04-03T03:17:30.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe976ffd700 1 -- 172.21.2.111:0/3370149235 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe96c0057e0 con 0x7fe9640044e0 2021-04-03T03:17:30.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- 172.21.2.111:0/3370149235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 msgr2=0x7fe964004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:30.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe964004960 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fe96c000fe0 tx=0x7fe96c0077d0).stop 2021-04-03T03:17:30.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- 172.21.2.111:0/3370149235 shutdown_connections 2021-04-03T03:17:30.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/3370149235 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe964004960 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:30.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- 172.21.2.111:0/3370149235 >> 172.21.2.111:0/3370149235 conn(0x7fe9780b61c0 msgr2=0x7fe97805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:30.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- 172.21.2.111:0/3370149235 shutdown_connections 2021-04-03T03:17:30.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.894+0000 7fe97d1b7700 1 -- 172.21.2.111:0/3370149235 wait complete. 2021-04-03T03:17:30.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 Processor -- start 2021-04-03T03:17:30.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 -- start start 2021-04-03T03:17:30.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe9780a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9780aa610 0x7fe978156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9781572a0 0x7fe9780645b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780629e0 con 0x7fe9640044e0 2021-04-03T03:17:30.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978062c80 con 0x7fe9781572a0 2021-04-03T03:17:30.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe978062f20 con 0x7fe9780aa610 2021-04-03T03:17:30.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe9780a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe9780a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58784/0 (socket says 172.21.2.111:58784) 2021-04-03T03:17:30.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 -- 172.21.2.111:0/51298708 learned_addr learned my addr 172.21.2.111:0/51298708 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:30.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 -- 172.21.2.111:0/51298708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9780aa610 msgr2=0x7fe978156db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:30.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9780aa610 0x7fe978156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 -- 172.21.2.111:0/51298708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9781572a0 msgr2=0x7fe9780645b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:30.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9781572a0 0x7fe9780645b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:30.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 -- 172.21.2.111:0/51298708 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978157720 con 0x7fe9640044e0 2021-04-03T03:17:30.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe977fff700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe9780a7780 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7fe96800a000 tx=0x7fe968008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.911 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9757fa700 1 -- 172.21.2.111:0/51298708 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe968009820 con 0x7fe9640044e0 2021-04-03T03:17:30.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9757fa700 1 -- 172.21.2.111:0/51298708 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968009980 con 0x7fe9640044e0 2021-04-03T03:17:30.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9781579a0 con 0x7fe9640044e0 2021-04-03T03:17:30.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9757fa700 1 -- 172.21.2.111:0/51298708 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe968009ce0 con 0x7fe9640044e0 2021-04-03T03:17:30.912 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe978156870 con 0x7fe9640044e0 2021-04-03T03:17:30.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9780630e0 con 0x7fe9640044e0 2021-04-03T03:17:30.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9757fa700 1 -- 172.21.2.111:0/51298708 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe968013480 con 0x7fe9640044e0 2021-04-03T03:17:30.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9757fa700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe958056ea0 0x7fe958059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:30.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9757fa700 1 -- 172.21.2.111:0/51298708 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fe96806ff70 con 0x7fe9640044e0 2021-04-03T03:17:30.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9777fe700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe958056ea0 0x7fe958059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:30.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.898+0000 7fe9777fe700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe958056ea0 0x7fe958059360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fe96c000fe0 tx=0x7fe96c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:30.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.902+0000 7fe9757fa700 1 -- 172.21.2.111:0/51298708 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe968045630 con 0x7fe9640044e0 2021-04-03T03:17:30.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.994+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe978057660 con 0x7fe9640044e0 2021-04-03T03:17:30.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.994+0000 7fe9757fa700 1 -- 172.21.2.111:0/51298708 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v146) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe968048c50 con 0x7fe9640044e0 2021-04-03T03:17:30.998 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:30.998 INFO:teuthology.orchestra.run.gibba011.stdout:[ 2021-04-03T03:17:30.998 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.998 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 1, 2021-04-03T03:17:30.999 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "device_health_metrics" 2021-04-03T03:17:30.999 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:30.999 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:30.999 INFO:teuthology.orchestra.run.gibba011.stdout: "poolnum": 2, 2021-04-03T03:17:30.999 INFO:teuthology.orchestra.run.gibba011.stdout: "poolname": "rbd" 2021-04-03T03:17:31.000 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:31.000 INFO:teuthology.orchestra.run.gibba011.stdout:] 2021-04-03T03:17:31.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe958056ea0 msgr2=0x7fe958059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe958056ea0 0x7fe958059360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fe96c000fe0 tx=0x7fe96c015000).stop 2021-04-03T03:17:31.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 msgr2=0x7fe9780a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe9780a7780 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7fe96800a000 tx=0x7fe968008040).stop 2021-04-03T03:17:31.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 shutdown_connections 2021-04-03T03:17:31.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe958056ea0 0x7fe958059360 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:31.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe9781572a0 0x7fe9780645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:31.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe9640044e0 0x7fe9780a7780 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:31.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 --2- 172.21.2.111:0/51298708 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe9780aa610 0x7fe978156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:31.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 >> 172.21.2.111:0/51298708 conn(0x7fe9780b61c0 msgr2=0x7fe9780b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:31.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 shutdown_connections 2021-04-03T03:17:31.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:30.998+0000 7fe97d1b7700 1 -- 172.21.2.111:0/51298708 wait complete. 2021-04-03T03:17:31.005 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-04-03T03:17:31.006 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2021-04-03T03:17:31.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 Processor -- start 2021-04-03T03:17:31.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- start start 2021-04-03T03:17:31.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd238144ef0 0x7fd238147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:31.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2380607d0 con 0x7fd2380603f0 2021-04-03T03:17:31.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23814b090 con 0x7fd2380add50 2021-04-03T03:17:31.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23814b330 con 0x7fd238144ef0 2021-04-03T03:17:31.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd237fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd238144ef0 0x7fd238147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:31.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd237fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd238144ef0 0x7fd238147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58794/0 (socket says 172.21.2.111:58794) 2021-04-03T03:17:31.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd237fff700 1 -- 172.21.2.111:0/2836964370 learned_addr learned my addr 172.21.2.111:0/2836964370 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:31.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd2377fe700 1 --1- 172.21.2.111:0/2836964370 >> v1:172.21.2.111:6789/0 conn(0x7fd2380603f0 0x7fd2381447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59584/0 (socket says 172.21.2.111:59584) 2021-04-03T03:17:31.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd237fff700 1 -- 172.21.2.111:0/2836964370 >> v1:172.21.2.115:6789/0 conn(0x7fd2380add50 legacy=0x7fd238147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:31.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd237fff700 1 -- 172.21.2.111:0/2836964370 >> v1:172.21.2.111:6789/0 conn(0x7fd2380603f0 legacy=0x7fd2381447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd237fff700 1 -- 172.21.2.111:0/2836964370 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23814b5d0 con 0x7fd238144ef0 2021-04-03T03:17:31.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd237fff700 1 --2- 172.21.2.111:0/2836964370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd238144ef0 0x7fd238147e50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7fd2380b5600 tx=0x7fd230009270).ready entity=mon.0 client_cookie=ae4e7cd28c928c55 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:31.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd2367fc700 1 -- 172.21.2.111:0/2836964370 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd23000bbc0 con 0x7fd238144ef0 2021-04-03T03:17:31.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd2367fc700 1 -- 172.21.2.111:0/2836964370 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23000bd20 con 0x7fd238144ef0 2021-04-03T03:17:31.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd2367fc700 1 -- 172.21.2.111:0/2836964370 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd23000be90 con 0x7fd238144ef0 2021-04-03T03:17:31.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- 172.21.2.111:0/2836964370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd238144ef0 msgr2=0x7fd238147e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/2836964370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd238144ef0 0x7fd238147e50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7fd2380b5600 tx=0x7fd230009270).stop 2021-04-03T03:17:31.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- 172.21.2.111:0/2836964370 shutdown_connections 2021-04-03T03:17:31.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/2836964370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd238144ef0 0x7fd238147e50 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:31.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- 172.21.2.111:0/2836964370 >> 172.21.2.111:0/2836964370 conn(0x7fd2380b61c0 msgr2=0x7fd23805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:31.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- 172.21.2.111:0/2836964370 shutdown_connections 2021-04-03T03:17:31.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.062+0000 7fd23cbf3700 1 -- 172.21.2.111:0/2836964370 wait complete. 2021-04-03T03:17:31.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 Processor -- start 2021-04-03T03:17:31.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 -- start start 2021-04-03T03:17:31.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2380603f0 0x7fd23814a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:31.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2380add50 0x7fd2380aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:31.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd238144ef0 0x7fd238157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:31.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238062de0 con 0x7fd2380603f0 2021-04-03T03:17:31.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2380a9cf0 con 0x7fd238144ef0 2021-04-03T03:17:31.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2380a8030 con 0x7fd2380add50 2021-04-03T03:17:31.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2380603f0 0x7fd23814a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:31.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2380603f0 0x7fd23814a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58798/0 (socket says 172.21.2.111:58798) 2021-04-03T03:17:31.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 -- 172.21.2.111:0/3099577764 learned_addr learned my addr 172.21.2.111:0/3099577764 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:31.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd237fff700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd238144ef0 0x7fd238157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:31.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 -- 172.21.2.111:0/3099577764 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2380add50 msgr2=0x7fd2380aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2380add50 0x7fd2380aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:31.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 -- 172.21.2.111:0/3099577764 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd238144ef0 msgr2=0x7fd238157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd238144ef0 0x7fd238157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:31.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 -- 172.21.2.111:0/3099577764 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238157a80 con 0x7fd2380603f0 2021-04-03T03:17:31.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd2377fe700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2380603f0 0x7fd23814a900 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7fd2380ae520 tx=0x7fd228007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:31.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd234ff9700 1 -- 172.21.2.111:0/3099577764 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2280072e0 con 0x7fd2380603f0 2021-04-03T03:17:31.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd234ff9700 1 -- 172.21.2.111:0/3099577764 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228005530 con 0x7fd2380603f0 2021-04-03T03:17:31.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd234ff9700 1 -- 172.21.2.111:0/3099577764 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd228005890 con 0x7fd2380603f0 2021-04-03T03:17:31.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2380a9850 con 0x7fd2380603f0 2021-04-03T03:17:31.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2380a86e0 con 0x7fd2380603f0 2021-04-03T03:17:31.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2380a81f0 con 0x7fd2380603f0 2021-04-03T03:17:31.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd234ff9700 1 -- 172.21.2.111:0/3099577764 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd228010070 con 0x7fd2380603f0 2021-04-03T03:17:31.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd234ff9700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd21c056f20 0x7fd21c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:31.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd234ff9700 1 -- 172.21.2.111:0/3099577764 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5596+0+0 (secure 0 0 0) 0x7fd22806f570 con 0x7fd2380603f0 2021-04-03T03:17:31.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd236ffd700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd21c056f20 0x7fd21c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:31.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.066+0000 7fd236ffd700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd21c056f20 0x7fd21c0593e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fd220002f50 tx=0x7fd220006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:31.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.070+0000 7fd234ff9700 1 -- 172.21.2.111:0/3099577764 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd228044c90 con 0x7fd2380603f0 2021-04-03T03:17:31.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.162+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fd2380a9eb0 con 0x7fd2380603f0 2021-04-03T03:17:31.925 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.922+0000 7fd234ff9700 1 -- 172.21.2.111:0/3099577764 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v147) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd2280482b0 con 0x7fd2380603f0 2021-04-03T03:17:31.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd21c056f20 msgr2=0x7fd21c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd21c056f20 0x7fd21c0593e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fd220002f50 tx=0x7fd220006990).stop 2021-04-03T03:17:31.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2380603f0 msgr2=0x7fd23814a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:31.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2380603f0 0x7fd23814a900 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7fd2380ae520 tx=0x7fd228007be0).stop 2021-04-03T03:17:31.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 shutdown_connections 2021-04-03T03:17:31.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd21c056f20 0x7fd21c0593e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:31.929 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd238144ef0 0x7fd238157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:31.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd2380603f0 0x7fd23814a900 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:31.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 --2- 172.21.2.111:0/3099577764 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd2380add50 0x7fd2380aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:31.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 >> 172.21.2.111:0/3099577764 conn(0x7fd2380b61c0 msgr2=0x7fd2380b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:31.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 shutdown_connections 2021-04-03T03:17:31.930 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.926+0000 7fd23cbf3700 1 -- 172.21.2.111:0/3099577764 wait complete. 2021-04-03T03:17:31.931 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' created 2021-04-03T03:17:31.940 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2021-04-03T03:17:31.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 Processor -- start 2021-04-03T03:17:31.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- start start 2021-04-03T03:17:32.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7e28041060 0x7f7e280dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:32.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e2803d560 0x7f7e28041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:32.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e280dc560 con 0x7f7e280dbba0 2021-04-03T03:17:32.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2803de60 con 0x7f7e28041060 2021-04-03T03:17:32.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280dc800 con 0x7f7e2803d560 2021-04-03T03:17:32.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7e28041060 0x7f7e280dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:32.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7e28041060 0x7f7e280dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58808/0 (socket says 172.21.2.111:58808) 2021-04-03T03:17:32.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 -- 172.21.2.111:0/558228034 learned_addr learned my addr 172.21.2.111:0/558228034 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:32.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 -- 172.21.2.111:0/558228034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e2803d560 msgr2=0x7f7e28041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:32.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 --2- 172.21.2.111:0/558228034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e2803d560 0x7f7e28041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:32.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 -- 172.21.2.111:0/558228034 >> v1:172.21.2.111:6789/0 conn(0x7f7e280dbba0 legacy=0x7f7e2803fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:32.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 -- 172.21.2.111:0/558228034 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2803da30 con 0x7f7e28041060 2021-04-03T03:17:32.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e35006700 1 --2- 172.21.2.111:0/558228034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e28041060 0x7f7e280dbf80 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f7e2c002f30 tx=0x7f7e2c006e90).ready entity=mon.0 client_cookie=3d038efff53517e2 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:32.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e34805700 1 -- 172.21.2.111:0/558228034 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c009970 con 0x7f7e28041060 2021-04-03T03:17:32.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e34805700 1 -- 172.21.2.111:0/558228034 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c009ad0 con 0x7f7e28041060 2021-04-03T03:17:32.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e34805700 1 -- 172.21.2.111:0/558228034 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c009e30 con 0x7f7e28041060 2021-04-03T03:17:32.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- 172.21.2.111:0/558228034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e28041060 msgr2=0x7f7e280dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:32.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- 172.21.2.111:0/558228034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e28041060 0x7f7e280dbf80 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f7e2c002f30 tx=0x7f7e2c006e90).stop 2021-04-03T03:17:32.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- 172.21.2.111:0/558228034 shutdown_connections 2021-04-03T03:17:32.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- 172.21.2.111:0/558228034 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e2803d560 0x7f7e28041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- 172.21.2.111:0/558228034 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e28041060 0x7f7e280dbf80 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- 172.21.2.111:0/558228034 >> 172.21.2.111:0/558228034 conn(0x7f7e280462b0 msgr2=0x7f7e280466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- 172.21.2.111:0/558228034 shutdown_connections 2021-04-03T03:17:32.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- 172.21.2.111:0/558228034 wait complete. 2021-04-03T03:17:32.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 Processor -- start 2021-04-03T03:17:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- start start 2021-04-03T03:17:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e2803d560 0x7f7e2803a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e28041060 0x7f7e280396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:32.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280dbba0 0x7f7e280dd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280e7380 con 0x7f7e280dbba0 2021-04-03T03:17:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280e7620 con 0x7f7e2803d560 2021-04-03T03:17:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36809700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280e78c0 con 0x7f7e28041060 2021-04-03T03:17:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:31.998+0000 7f7e36008700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280dbba0 0x7f7e280dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:32.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280dbba0 0x7f7e280dd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58814/0 (socket says 172.21.2.111:58814) 2021-04-03T03:17:32.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 -- 172.21.2.111:0/2051358063 learned_addr learned my addr 172.21.2.111:0/2051358063 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:32.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 -- 172.21.2.111:0/2051358063 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e28041060 msgr2=0x7f7e280396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:32.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e28041060 0x7f7e280396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:32.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 -- 172.21.2.111:0/2051358063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e2803d560 msgr2=0x7f7e2803a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:32.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e2803d560 0x7f7e2803a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:32.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 -- 172.21.2.111:0/2051358063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2803bb00 con 0x7f7e280dbba0 2021-04-03T03:17:32.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36008700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280dbba0 0x7f7e280dd990 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f7e3006b5d0 tx=0x7f7e3005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:32.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e22ffd700 1 -- 172.21.2.111:0/2051358063 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e3005f2c0 con 0x7f7e280dbba0 2021-04-03T03:17:32.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e22ffd700 1 -- 172.21.2.111:0/2051358063 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3005f420 con 0x7f7e280dbba0 2021-04-03T03:17:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e28039b90 con 0x7f7e280dbba0 2021-04-03T03:17:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e22ffd700 1 -- 172.21.2.111:0/2051358063 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e3005f780 con 0x7f7e280dbba0 2021-04-03T03:17:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e280e7d60 con 0x7f7e280dbba0 2021-04-03T03:17:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2803bcf0 con 0x7f7e280dbba0 2021-04-03T03:17:32.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e22ffd700 1 -- 172.21.2.111:0/2051358063 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7e3008b020 con 0x7f7e280dbba0 2021-04-03T03:17:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e22ffd700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e10056ea0 0x7f7e10059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e22ffd700 1 -- 172.21.2.111:0/2051358063 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5950+0+0 (secure 0 0 0) 0x7f7e300df050 con 0x7f7e280dbba0 2021-04-03T03:17:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e35807700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e10056ea0 0x7f7e10059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.002+0000 7f7e35807700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e10056ea0 0x7f7e10059360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f7e280087d0 tx=0x7f7e240077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:32.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.006+0000 7f7e22ffd700 1 -- 172.21.2.111:0/2051358063 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7e300b45b0 con 0x7f7e280dbba0 2021-04-03T03:17:32.101 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.102+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f7e280e7a80 con 0x7f7e280dbba0 2021-04-03T03:17:32.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.933+0000 7f7e22ffd700 1 -- 172.21.2.111:0/2051358063 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v148) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7e300b7bd0 con 0x7f7e280dbba0 2021-04-03T03:17:32.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e10056ea0 msgr2=0x7f7e10059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:32.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e10056ea0 0x7f7e10059360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f7e280087d0 tx=0x7f7e240077f0).stop 2021-04-03T03:17:32.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280dbba0 msgr2=0x7f7e280dd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:32.939 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280dbba0 0x7f7e280dd990 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f7e3006b5d0 tx=0x7f7e3005ad70).stop 2021-04-03T03:17:32.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 shutdown_connections 2021-04-03T03:17:32.940 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7e280dbba0 0x7f7e280dd990 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:32.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7e10056ea0 0x7f7e10059360 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:32.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7e2803d560 0x7f7e2803a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:32.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 --2- 172.21.2.111:0/2051358063 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7e28041060 0x7f7e280396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:32.941 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 >> 172.21.2.111:0/2051358063 conn(0x7f7e280462b0 msgr2=0x7f7e28045770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:32.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 shutdown_connections 2021-04-03T03:17:32.942 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:32.937+0000 7f7e36809700 1 -- 172.21.2.111:0/2051358063 wait complete. 2021-04-03T03:17:32.943 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' created 2021-04-03T03:17:32.949 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2021-04-03T03:17:33.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.005+0000 7f4df7fff700 1 Processor -- start 2021-04-03T03:17:33.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.005+0000 7f4df7fff700 1 -- start start 2021-04-03T03:17:33.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df0047df0 0x7f4df00d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df00481d0 con 0x7f4df0042090 2021-04-03T03:17:33.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df00d0e70 con 0x7f4df00429a0 2021-04-03T03:17:33.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00d1110 con 0x7f4df0047df0 2021-04-03T03:17:33.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4df0042090 0x7f4df00d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59614/0 (socket says 172.21.2.111:59614) 2021-04-03T03:17:33.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 -- 172.21.2.111:0/3846060165 learned_addr learned my addr 172.21.2.111:0/3846060165 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:33.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062859536 0 0) 0x7f4df00481d0 con 0x7f4df0042090 2021-04-03T03:17:33.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ddc003240 con 0x7f4df0042090 2021-04-03T03:17:33.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df0047df0 0x7f4df00d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3350799667 0 0) 0x7f4ddc003240 con 0x7f4df0042090 2021-04-03T03:17:33.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df0047df0 msgr2=0x7f4df00d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df0047df0 0x7f4df00d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 >> v1:172.21.2.115:6789/0 conn(0x7f4df00429a0 legacy=0x7f4df003d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00d13b0 con 0x7f4df0042090 2021-04-03T03:17:33.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f4dec0016d0 con 0x7f4df0042090 2021-04-03T03:17:33.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 >> v1:172.21.2.111:6789/0 conn(0x7f4df0042090 legacy=0x7f4df00d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df0047df0 0x7f4ddc003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ddc003c50 0x7f4ddc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ddc0049d0 0x7f4ddc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00d1110 con 0x7f4ddc003c50 2021-04-03T03:17:33.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0041e0 con 0x7f4ddc0049d0 2021-04-03T03:17:33.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc00b730 con 0x7f4df0047df0 2021-04-03T03:17:33.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ddc003c50 0x7f4ddc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df77fe700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ddc0049d0 0x7f4ddc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 -- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df0047df0 msgr2=0x7f4ddc003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df0047df0 0x7f4ddc003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 -- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ddc0049d0 msgr2=0x7f4ddc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4ddc0049d0 0x7f4ddc004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 -- 172.21.2.111:0/3846060165 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc003240 con 0x7f4ddc003c50 2021-04-03T03:17:33.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ddc003c50 0x7f4ddc0044e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f4df000d140 tx=0x7f4df00088d0).ready entity=mon.0 client_cookie=49bb7a37bfbd0cc5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dec0070d0 con 0x7f4ddc003c50 2021-04-03T03:17:33.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec007230 con 0x7f4ddc003c50 2021-04-03T03:17:33.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df5ffb700 1 -- 172.21.2.111:0/3846060165 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dec0055f0 con 0x7f4ddc003c50 2021-04-03T03:17:33.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/3846060165 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ddc003c50 msgr2=0x7f4ddc0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 --2- 172.21.2.111:0/3846060165 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4ddc003c50 0x7f4ddc0044e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f4df000d140 tx=0x7f4df00088d0).stop 2021-04-03T03:17:33.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df77fe700 1 -- 172.21.2.111:0/3846060165 reap_dead start 2021-04-03T03:17:33.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/3846060165 shutdown_connections 2021-04-03T03:17:33.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/3846060165 >> 172.21.2.111:0/3846060165 conn(0x7f4df000e210 msgr2=0x7f4df00473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:33.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/3846060165 shutdown_connections 2021-04-03T03:17:33.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/3846060165 wait complete. 2021-04-03T03:17:33.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 Processor -- start 2021-04-03T03:17:33.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- start start 2021-04-03T03:17:33.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4df00d20d0 0x7f4df00d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4df00d4310 0x7f4df003c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df003aba0 0x7f4df003c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df003a690 con 0x7f4df00d20d0 2021-04-03T03:17:33.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00e6070 con 0x7f4df00d4310 2021-04-03T03:17:33.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df003a8f0 con 0x7f4df003aba0 2021-04-03T03:17:33.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4df00d20d0 0x7f4df00d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4df00d20d0 0x7f4df00d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58832/0 (socket says 172.21.2.111:58832) 2021-04-03T03:17:33.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 -- 172.21.2.111:0/143070744 learned_addr learned my addr 172.21.2.111:0/143070744 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:33.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df77fe700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df003aba0 0x7f4df003c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df67fc700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4df00d4310 0x7f4df003c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 -- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df003aba0 msgr2=0x7f4df003c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df003aba0 0x7f4df003c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 -- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4df00d4310 msgr2=0x7f4df003c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4df00d4310 0x7f4df003c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 -- 172.21.2.111:0/143070744 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00e5dd0 con 0x7f4df00d20d0 2021-04-03T03:17:33.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df6ffd700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4df00d20d0 0x7f4df00d24b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f4de8007000 tx=0x7f4de8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4de7fff700 1 -- 172.21.2.111:0/143070744 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4de8015070 con 0x7f4df00d20d0 2021-04-03T03:17:33.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4de7fff700 1 -- 172.21.2.111:0/143070744 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8010580 con 0x7f4df00d20d0 2021-04-03T03:17:33.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df003b810 con 0x7f4df00d20d0 2021-04-03T03:17:33.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4de7fff700 1 -- 172.21.2.111:0/143070744 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4de80108e0 con 0x7f4df00d20d0 2021-04-03T03:17:33.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df00e2cb0 con 0x7f4df00d20d0 2021-04-03T03:17:33.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.009+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df0005200 con 0x7f4df00d20d0 2021-04-03T03:17:33.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.013+0000 7f4de7fff700 1 -- 172.21.2.111:0/143070744 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4de8010a40 con 0x7f4df00d20d0 2021-04-03T03:17:33.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.013+0000 7f4de7fff700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dd4056fb0 0x7f4dd4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.013+0000 7f4df67fc700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dd4056fb0 0x7f4dd4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.013+0000 7f4df67fc700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dd4056fb0 0x7f4dd4059470 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f4dec019000 tx=0x7f4dec013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.013+0000 7f4de7fff700 1 -- 172.21.2.111:0/143070744 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f4de8048d70 con 0x7f4df00d20d0 2021-04-03T03:17:33.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.013+0000 7f4de7fff700 1 -- 172.21.2.111:0/143070744 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4de8045110 con 0x7f4df00d20d0 2021-04-03T03:17:33.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.141+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f4df00e2820 con 0x7f4df00d20d0 2021-04-03T03:17:33.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4de7fff700 1 -- 172.21.2.111:0/143070744 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v67) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4de8048730 con 0x7f4df00d20d0 2021-04-03T03:17:33.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dd4056fb0 msgr2=0x7f4dd4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dd4056fb0 0x7f4dd4059470 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f4dec019000 tx=0x7f4dec013000).stop 2021-04-03T03:17:33.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4df00d20d0 msgr2=0x7f4df00d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4df00d20d0 0x7f4df00d24b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f4de8007000 tx=0x7f4de8008040).stop 2021-04-03T03:17:33.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 shutdown_connections 2021-04-03T03:17:33.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4df003aba0 0x7f4df003c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4df00d20d0 0x7f4df00d24b0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4dd4056fb0 0x7f4dd4059470 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 --2- 172.21.2.111:0/143070744 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4df00d4310 0x7f4df003c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 >> 172.21.2.111:0/143070744 conn(0x7f4df000e210 msgr2=0x7f4df003db00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:33.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 shutdown_connections 2021-04-03T03:17:33.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.145+0000 7f4df7fff700 1 -- 172.21.2.111:0/143070744 wait complete. 2021-04-03T03:17:33.154 INFO:teuthology.orchestra.run.gibba011.stderr:new fs with metadata pool 29 and data pool 30 2021-04-03T03:17:33.156 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:33.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc1752a5700 1 Processor -- start 2021-04-03T03:17:33.213 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc1752a5700 1 -- start start 2021-04-03T03:17:33.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc1752a5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1700603f0 0x7fc1700b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc1752a5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700b0c90 0x7fc170147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc1752a5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1700607d0 con 0x7fc17014b700 2021-04-03T03:17:33.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc1752a5700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1700af400 con 0x7fc1700603f0 2021-04-03T03:17:33.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc1752a5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700af6a0 con 0x7fc1700b0c90 2021-04-03T03:17:33.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc16ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1700603f0 0x7fc1700b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc16ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1700603f0 0x7fc1700b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58848/0 (socket says 172.21.2.111:58848) 2021-04-03T03:17:33.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc16ffff700 1 -- 172.21.2.111:0/1136858685 learned_addr learned my addr 172.21.2.111:0/1136858685 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:33.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc16ffff700 1 -- 172.21.2.111:0/1136858685 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700b0c90 msgr2=0x7fc170147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc16ffff700 1 --2- 172.21.2.111:0/1136858685 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700b0c90 0x7fc170147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc16ffff700 1 -- 172.21.2.111:0/1136858685 >> v1:172.21.2.115:6789/0 conn(0x7fc17014b700 legacy=0x7fc1700b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.209+0000 7fc16ffff700 1 -- 172.21.2.111:0/1136858685 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700afb40 con 0x7fc1700603f0 2021-04-03T03:17:33.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16ffff700 1 --2- 172.21.2.111:0/1136858685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700603f0 0x7fc1700b0090 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7fc17006a3b0 tx=0x7fc160006e90).ready entity=mon.0 client_cookie=2bb8697601468e41 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16effd700 1 -- 172.21.2.111:0/1136858685 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1600098e0 con 0x7fc1700603f0 2021-04-03T03:17:33.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16effd700 1 -- 172.21.2.111:0/1136858685 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160009a40 con 0x7fc1700603f0 2021-04-03T03:17:33.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16effd700 1 -- 172.21.2.111:0/1136858685 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc160009de0 con 0x7fc1700603f0 2021-04-03T03:17:33.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/1136858685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700603f0 msgr2=0x7fc1700b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 --2- 172.21.2.111:0/1136858685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700603f0 0x7fc1700b0090 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7fc17006a3b0 tx=0x7fc160006e90).stop 2021-04-03T03:17:33.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/1136858685 shutdown_connections 2021-04-03T03:17:33.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 --2- 172.21.2.111:0/1136858685 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700b0c90 0x7fc170147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 --2- 172.21.2.111:0/1136858685 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700603f0 0x7fc1700b0090 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/1136858685 >> 172.21.2.111:0/1136858685 conn(0x7fc1700b61c0 msgr2=0x7fc17005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:33.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/1136858685 shutdown_connections 2021-04-03T03:17:33.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/1136858685 wait complete. 2021-04-03T03:17:33.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 Processor -- start 2021-04-03T03:17:33.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- start start 2021-04-03T03:17:33.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700603f0 0x7fc17014a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700b0c90 0x7fc1700628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc17014b700 0x7fc17015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170063ba0 con 0x7fc1700b0c90 2021-04-03T03:17:33.222 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170159b80 con 0x7fc17014b700 2021-04-03T03:17:33.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc170159db0 con 0x7fc1700603f0 2021-04-03T03:17:33.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700b0c90 0x7fc1700628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700b0c90 0x7fc1700628d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58858/0 (socket says 172.21.2.111:58858) 2021-04-03T03:17:33.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 -- 172.21.2.111:0/161151236 learned_addr learned my addr 172.21.2.111:0/161151236 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:33.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 -- 172.21.2.111:0/161151236 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700603f0 msgr2=0x7fc17014a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700603f0 0x7fc17014a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 -- 172.21.2.111:0/161151236 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc17014b700 msgr2=0x7fc17015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc17014b700 0x7fc17015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 -- 172.21.2.111:0/161151236 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17014b000 con 0x7fc1700b0c90 2021-04-03T03:17:33.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16f7fe700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700b0c90 0x7fc1700628d0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7fc158002ed0 tx=0x7fc158006d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16d7fa700 1 -- 172.21.2.111:0/161151236 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc15800f510 con 0x7fc1700b0c90 2021-04-03T03:17:33.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16d7fa700 1 -- 172.21.2.111:0/161151236 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15800f670 con 0x7fc1700b0c90 2021-04-03T03:17:33.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1700a6fb0 con 0x7fc1700b0c90 2021-04-03T03:17:33.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16d7fa700 1 -- 172.21.2.111:0/161151236 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc15800f7e0 con 0x7fc1700b0c90 2021-04-03T03:17:33.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc170063f90 con 0x7fc1700b0c90 2021-04-03T03:17:33.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc170057660 con 0x7fc1700b0c90 2021-04-03T03:17:33.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16d7fa700 1 -- 172.21.2.111:0/161151236 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc15800f070 con 0x7fc1700b0c90 2021-04-03T03:17:33.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16d7fa700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc154056ea0 0x7fc154059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16d7fa700 1 -- 172.21.2.111:0/161151236 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc15806fa60 con 0x7fc1700b0c90 2021-04-03T03:17:33.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.213+0000 7fc16ffff700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc154056ea0 0x7fc154059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.217+0000 7fc16ffff700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc154056ea0 0x7fc154059360 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc17006a3b0 tx=0x7fc160015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.217+0000 7fc16d7fa700 1 -- 172.21.2.111:0/161151236 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc158044e50 con 0x7fc1700b0c90 2021-04-03T03:17:33.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.309+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc170159ef0 con 0x7fc1700b0c90 2021-04-03T03:17:33.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc16d7fa700 1 -- 172.21.2.111:0/161151236 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fc158048470 con 0x7fc1700b0c90 2021-04-03T03:17:33.314 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:33.314 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":150,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:33.144933+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:31.174355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:32.113426+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:33.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc154056ea0 msgr2=0x7fc154059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc154056ea0 0x7fc154059360 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc17006a3b0 tx=0x7fc160015000).stop 2021-04-03T03:17:33.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700b0c90 msgr2=0x7fc1700628d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700b0c90 0x7fc1700628d0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7fc158002ed0 tx=0x7fc158006d30).stop 2021-04-03T03:17:33.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 shutdown_connections 2021-04-03T03:17:33.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc154056ea0 0x7fc154059360 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc17014b700 0x7fc17015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1700603f0 0x7fc17014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 --2- 172.21.2.111:0/161151236 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1700b0c90 0x7fc1700628d0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 >> 172.21.2.111:0/161151236 conn(0x7fc1700b61c0 msgr2=0x7fc170147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:33.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 shutdown_connections 2021-04-03T03:17:33.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.313+0000 7fc1752a5700 1 -- 172.21.2.111:0/161151236 wait complete. 2021-04-03T03:17:33.322 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:33.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.385+0000 7f49c9e87700 1 Processor -- start 2021-04-03T03:17:33.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.385+0000 7f49c9e87700 1 -- start start 2021-04-03T03:17:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.385+0000 7f49c9e87700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f49c414b700 0x7f49c40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.385+0000 7f49c9e87700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40af400 con 0x7f49c40603f0 2021-04-03T03:17:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.385+0000 7f49c9e87700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40af6a0 con 0x7f49c40b0c90 2021-04-03T03:17:33.393 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.385+0000 7f49c9e87700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c4060860 con 0x7f49c414b700 2021-04-03T03:17:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f49c40603f0 0x7f49c40b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59654/0 (socket says 172.21.2.111:59654) 2021-04-03T03:17:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 -- 172.21.2.111:0/2966908021 learned_addr learned my addr 172.21.2.111:0/2966908021 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593987338 0 0) 0x7f49c40af400 con 0x7f49c40603f0 2021-04-03T03:17:33.394 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a4003240 con 0x7f49c40603f0 2021-04-03T03:17:33.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c3fff700 1 --2- 172.21.2.111:0/2966908021 >> v2:172.21.2.115:3300/0 conn(0x7f49c414b700 0x7f49c40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 258073510 0 0) 0x7f49a4003240 con 0x7f49c40603f0 2021-04-03T03:17:33.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 >> v2:172.21.2.115:3300/0 conn(0x7f49c414b700 msgr2=0x7f49c40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.395 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 --2- 172.21.2.111:0/2966908021 >> v2:172.21.2.115:3300/0 conn(0x7f49c414b700 0x7f49c40b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 >> v1:172.21.2.115:6789/0 conn(0x7f49c40b0c90 legacy=0x7f49c4147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40afb40 con 0x7f49c40603f0 2021-04-03T03:17:33.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f49ac0016d0 con 0x7f49c40603f0 2021-04-03T03:17:33.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 >> v1:172.21.2.111:6789/0 conn(0x7f49c40603f0 legacy=0x7f49c40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.396 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49a4003730 0x7f49a4003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49a40044e0 0x7f49a4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49a4004e50 0x7f49a400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4060860 con 0x7f49a4003730 2021-04-03T03:17:33.397 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a4004240 con 0x7f49a4004e50 2021-04-03T03:17:33.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a400bc20 con 0x7f49a40044e0 2021-04-03T03:17:33.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49a4003730 0x7f49a4003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 -- 172.21.2.111:0/2966908021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49a40044e0 msgr2=0x7f49a4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49a40044e0 0x7f49a4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 -- 172.21.2.111:0/2966908021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49a4004e50 msgr2=0x7f49a400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49a4004e50 0x7f49a400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 -- 172.21.2.111:0/2966908021 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4003240 con 0x7f49a4003730 2021-04-03T03:17:33.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9686700 1 -- 172.21.2.111:0/2966908021 reap_dead start 2021-04-03T03:17:33.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c8e85700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49a4003730 0x7f49a4003b30 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f49c40620f0 tx=0x7f49c40af830).ready entity=mon.0 client_cookie=3ee7da17d3a016c3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49ac00ce00 con 0x7f49a4003730 2021-04-03T03:17:33.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac0072e0 con 0x7f49a4003730 2021-04-03T03:17:33.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c37fe700 1 -- 172.21.2.111:0/2966908021 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49ac0057e0 con 0x7f49a4003730 2021-04-03T03:17:33.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- 172.21.2.111:0/2966908021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49a4003730 msgr2=0x7f49a4003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49a4003730 0x7f49a4003b30 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f49c40620f0 tx=0x7f49c40af830).stop 2021-04-03T03:17:33.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- 172.21.2.111:0/2966908021 shutdown_connections 2021-04-03T03:17:33.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 --2- 172.21.2.111:0/2966908021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49a4003730 0x7f49a4003b30 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- 172.21.2.111:0/2966908021 >> 172.21.2.111:0/2966908021 conn(0x7f49c40b61c0 msgr2=0x7f49c405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:33.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- 172.21.2.111:0/2966908021 shutdown_connections 2021-04-03T03:17:33.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- 172.21.2.111:0/2966908021 wait complete. 2021-04-03T03:17:33.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 Processor -- start 2021-04-03T03:17:33.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- start start 2021-04-03T03:17:33.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49a4003730 0x7f49c4159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49c40ab690 0x7f49c40a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49c40aaf20 0x7f49c4064460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4064070 con 0x7f49c40ab690 2021-04-03T03:17:33.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40a9d60 con 0x7f49a4003730 2021-04-03T03:17:33.405 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c9e87700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40aa000 con 0x7f49c40aaf20 2021-04-03T03:17:33.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49c40ab690 0x7f49c40a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49c40ab690 0x7f49c40a74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58876/0 (socket says 172.21.2.111:58876) 2021-04-03T03:17:33.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.389+0000 7f49c3fff700 1 -- 172.21.2.111:0/136299805 learned_addr learned my addr 172.21.2.111:0/136299805 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:33.406 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c3fff700 1 -- 172.21.2.111:0/136299805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49c40aaf20 msgr2=0x7f49c4064460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c3fff700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49c40aaf20 0x7f49c4064460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c3fff700 1 -- 172.21.2.111:0/136299805 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49a4003730 msgr2=0x7f49c4159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c3fff700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49a4003730 0x7f49c4159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c3fff700 1 -- 172.21.2.111:0/136299805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40aa2a0 con 0x7f49c40ab690 2021-04-03T03:17:33.407 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c3fff700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49c40ab690 0x7f49c40a74c0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f49b400a000 tx=0x7f49b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c1ffb700 1 -- 172.21.2.111:0/136299805 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b400f7e0 con 0x7f49c40ab690 2021-04-03T03:17:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c1ffb700 1 -- 172.21.2.111:0/136299805 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400f940 con 0x7f49c40ab690 2021-04-03T03:17:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c4064950 con 0x7f49c40ab690 2021-04-03T03:17:33.408 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c4064e40 con 0x7f49c40ab690 2021-04-03T03:17:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c1ffb700 1 -- 172.21.2.111:0/136299805 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b400fca0 con 0x7f49c40ab690 2021-04-03T03:17:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c0033f0 con 0x7f49c40ab690 2021-04-03T03:17:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c1ffb700 1 -- 172.21.2.111:0/136299805 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f49b4013480 con 0x7f49c40ab690 2021-04-03T03:17:33.409 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c1ffb700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f49a8056ef0 0x7f49a80593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c1ffb700 1 -- 172.21.2.111:0/136299805 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f49b4070360 con 0x7f49c40ab690 2021-04-03T03:17:33.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c8e85700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f49a8056ef0 0x7f49a80593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.393+0000 7f49c8e85700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f49a8056ef0 0x7f49a80593b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f49c40620f0 tx=0x7f49ac015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.410 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.397+0000 7f49c1ffb700 1 -- 172.21.2.111:0/136299805 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f49b40455e0 con 0x7f49c40ab690 2021-04-03T03:17:33.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.485+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f499c0036a0 con 0x7f49c40ab690 2021-04-03T03:17:33.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c1ffb700 1 -- 172.21.2.111:0/136299805 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f49b4048c00 con 0x7f49c40ab690 2021-04-03T03:17:33.491 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:33.491 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":150,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:33.144933+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:31.174355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:32.113426+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:33.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f49a8056ef0 msgr2=0x7f49a80593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f49a8056ef0 0x7f49a80593b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f49c40620f0 tx=0x7f49ac015000).stop 2021-04-03T03:17:33.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49c40ab690 msgr2=0x7f49c40a74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49c40ab690 0x7f49c40a74c0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f49b400a000 tx=0x7f49b4008040).stop 2021-04-03T03:17:33.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 shutdown_connections 2021-04-03T03:17:33.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f49c40aaf20 0x7f49c4064460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f49a8056ef0 0x7f49a80593b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49a4003730 0x7f49c4159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 --2- 172.21.2.111:0/136299805 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49c40ab690 0x7f49c40a74c0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 >> 172.21.2.111:0/136299805 conn(0x7f49c40b61c0 msgr2=0x7f49c405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:33.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 shutdown_connections 2021-04-03T03:17:33.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.489+0000 7f49c9e87700 1 -- 172.21.2.111:0/136299805 wait complete. 2021-04-03T03:17:33.499 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2021-04-03T03:17:33.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.553+0000 7f93a8ca8700 1 Processor -- start 2021-04-03T03:17:33.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.553+0000 7f93a8ca8700 1 -- start start 2021-04-03T03:17:33.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.553+0000 7f93a8ca8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.553+0000 7f93a8ca8700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f939c041060 0x7f939c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.553+0000 7f93a8ca8700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c0dc560 con 0x7f939c03d560 2021-04-03T03:17:33.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.553+0000 7f93a8ca8700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c03de60 con 0x7f939c041060 2021-04-03T03:17:33.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.553+0000 7f93a8ca8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0dc800 con 0x7f939c0dbba0 2021-04-03T03:17:33.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a2ffd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f939c041060 0x7f939c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a2ffd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f939c041060 0x7f939c03fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:48594/0 (socket says 172.21.2.111:48594) 2021-04-03T03:17:33.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a2ffd700 1 -- 172.21.2.111:0/3801325108 learned_addr learned my addr 172.21.2.111:0/3801325108 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:33.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 -- 172.21.2.111:0/3801325108 >> v2:172.21.2.115:3300/0 conn(0x7f939c041060 msgr2=0x7f939c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:33.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 --2- 172.21.2.111:0/3801325108 >> v2:172.21.2.115:3300/0 conn(0x7f939c041060 0x7f939c03fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 -- 172.21.2.111:0/3801325108 >> v1:172.21.2.115:6789/0 conn(0x7f939c03d560 legacy=0x7f939c041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 -- 172.21.2.111:0/3801325108 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c03da30 con 0x7f939c0dbba0 2021-04-03T03:17:33.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 --2- 172.21.2.111:0/3801325108 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c0dbf80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f9394002f30 tx=0x7f9394006dd0).ready entity=mon.2 client_cookie=7fbffbbc2b520546 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.563 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a27fc700 1 -- 172.21.2.111:0/3801325108 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9394009760 con 0x7f939c0dbba0 2021-04-03T03:17:33.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a27fc700 1 -- 172.21.2.111:0/3801325108 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93940098c0 con 0x7f939c0dbba0 2021-04-03T03:17:33.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a27fc700 1 -- 172.21.2.111:0/3801325108 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9394009c20 con 0x7f939c0dbba0 2021-04-03T03:17:33.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- 172.21.2.111:0/3801325108 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 msgr2=0x7f939c0dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/3801325108 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c0dbf80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f9394002f30 tx=0x7f9394006dd0).stop 2021-04-03T03:17:33.564 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- 172.21.2.111:0/3801325108 shutdown_connections 2021-04-03T03:17:33.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/3801325108 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c0dbf80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/3801325108 >> v2:172.21.2.115:3300/0 conn(0x7f939c041060 0x7f939c03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- 172.21.2.111:0/3801325108 >> 172.21.2.111:0/3801325108 conn(0x7f939c0462b0 msgr2=0x7f939c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:33.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- 172.21.2.111:0/3801325108 shutdown_connections 2021-04-03T03:17:33.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- 172.21.2.111:0/3801325108 wait complete. 2021-04-03T03:17:33.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 Processor -- start 2021-04-03T03:17:33.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- start start 2021-04-03T03:17:33.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f939c03d560 0x7f939c0e7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f939c041060 0x7f939c0e7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c09f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0a0b60 con 0x7f939c03d560 2021-04-03T03:17:33.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0e7540 con 0x7f939c041060 2021-04-03T03:17:33.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a8ca8700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c03a750 con 0x7f939c0dbba0 2021-04-03T03:17:33.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f939c03d560 0x7f939c0e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f939c03d560 0x7f939c0e7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58890/0 (socket says 172.21.2.111:58890) 2021-04-03T03:17:33.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 -- 172.21.2.111:0/1999619991 learned_addr learned my addr 172.21.2.111:0/1999619991 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:33.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a37fe700 1 -- 172.21.2.111:0/1999619991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 msgr2=0x7f939c09f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.557+0000 7f93a3fff700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c09f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a37fe700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c09f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:33.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a37fe700 1 -- 172.21.2.111:0/1999619991 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f939c041060 msgr2=0x7f939c0e7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:33.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a37fe700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f939c041060 0x7f939c0e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:33.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a37fe700 1 -- 172.21.2.111:0/1999619991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c03a9f0 con 0x7f939c03d560 2021-04-03T03:17:33.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a37fe700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f939c03d560 0x7f939c0e7050 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f9394002f30 tx=0x7f9394015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a0ff9700 1 -- 172.21.2.111:0/1999619991 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939401d450 con 0x7f939c03d560 2021-04-03T03:17:33.569 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a0ff9700 1 -- 172.21.2.111:0/1999619991 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939401d5b0 con 0x7f939c03d560 2021-04-03T03:17:33.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a0ff9700 1 -- 172.21.2.111:0/1999619991 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939401d910 con 0x7f939c03d560 2021-04-03T03:17:33.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c03a440 con 0x7f939c03d560 2021-04-03T03:17:33.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c0e7fe0 con 0x7f939c03d560 2021-04-03T03:17:33.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c004db0 con 0x7f939c03d560 2021-04-03T03:17:33.570 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a0ff9700 1 -- 172.21.2.111:0/1999619991 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f939400f070 con 0x7f939c03d560 2021-04-03T03:17:33.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a0ff9700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9384056f20 0x7f93840593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:33.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a0ff9700 1 -- 172.21.2.111:0/1999619991 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9394072bc0 con 0x7f939c03d560 2021-04-03T03:17:33.571 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a2ffd700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9384056f20 0x7f93840593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:33.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a2ffd700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9384056f20 0x7f93840593e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f939800c000 tx=0x7f9398006e90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:33.572 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.561+0000 7f93a0ff9700 1 -- 172.21.2.111:0/1999619991 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f939404b8d0 con 0x7f939c03d560 2021-04-03T03:17:33.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:33.685+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f939c0083c0 con 0x7f939c03d560 2021-04-03T03:17:34.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.153+0000 7f93a0ff9700 1 -- 172.21.2.111:0/1999619991 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7f939404b4c0 con 0x7f939c03d560 2021-04-03T03:17:34.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9384056f20 msgr2=0x7f93840593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9384056f20 0x7f93840593e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f939800c000 tx=0x7f9398006e90).stop 2021-04-03T03:17:34.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f939c03d560 msgr2=0x7f939c0e7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f939c03d560 0x7f939c0e7050 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f9394002f30 tx=0x7f9394015000).stop 2021-04-03T03:17:34.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 shutdown_connections 2021-04-03T03:17:34.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f939c0dbba0 0x7f939c09f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f939c03d560 0x7f939c0e7050 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9384056f20 0x7f93840593e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 --2- 172.21.2.111:0/1999619991 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f939c041060 0x7f939c0e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 >> 172.21.2.111:0/1999619991 conn(0x7f939c0462b0 msgr2=0x7f939c006150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 shutdown_connections 2021-04-03T03:17:34.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.157+0000 7f93a8ca8700 1 -- 172.21.2.111:0/1999619991 wait complete. 2021-04-03T03:17:34.167 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:34.223 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 Processor -- start 2021-04-03T03:17:34.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- start start 2021-04-03T03:17:34.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe6c40603f0 0x7fe6c40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 0x7fe6c40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c40607d0 con 0x7fe6c40b12b0 2021-04-03T03:17:34.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40af400 con 0x7fe6c40603f0 2021-04-03T03:17:34.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40af6a0 con 0x7fe6c414b700 2021-04-03T03:17:34.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe6c40603f0 0x7fe6c40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe6c40603f0 0x7fe6c40b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58896/0 (socket says 172.21.2.111:58896) 2021-04-03T03:17:34.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 -- 172.21.2.111:0/2177979682 learned_addr learned my addr 172.21.2.111:0/2177979682 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:34.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 -- 172.21.2.111:0/2177979682 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 msgr2=0x7fe6c40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:34.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 --2- 172.21.2.111:0/2177979682 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 0x7fe6c40b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 -- 172.21.2.111:0/2177979682 >> v1:172.21.2.111:6789/0 conn(0x7fe6c40b12b0 legacy=0x7fe6c41465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 -- 172.21.2.111:0/2177979682 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40afb40 con 0x7fe6c40603f0 2021-04-03T03:17:34.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c2ffd700 1 --2- 172.21.2.111:0/2177979682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40603f0 0x7fe6c40b0090 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7fe6ac002f30 tx=0x7fe6ac006e90).ready entity=mon.0 client_cookie=a0821a70f925e7d9 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c1ffb700 1 -- 172.21.2.111:0/2177979682 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6ac009970 con 0x7fe6c40603f0 2021-04-03T03:17:34.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c1ffb700 1 -- 172.21.2.111:0/2177979682 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac009ad0 con 0x7fe6c40603f0 2021-04-03T03:17:34.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c1ffb700 1 -- 172.21.2.111:0/2177979682 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6ac009e30 con 0x7fe6c40603f0 2021-04-03T03:17:34.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- 172.21.2.111:0/2177979682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40603f0 msgr2=0x7fe6c40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/2177979682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40603f0 0x7fe6c40b0090 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7fe6ac002f30 tx=0x7fe6ac006e90).stop 2021-04-03T03:17:34.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- 172.21.2.111:0/2177979682 shutdown_connections 2021-04-03T03:17:34.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/2177979682 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40603f0 0x7fe6c40b0090 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/2177979682 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 0x7fe6c40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- 172.21.2.111:0/2177979682 >> 172.21.2.111:0/2177979682 conn(0x7fe6c40b61c0 msgr2=0x7fe6c405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- 172.21.2.111:0/2177979682 shutdown_connections 2021-04-03T03:17:34.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.221+0000 7fe6c3fff700 1 -- 172.21.2.111:0/2177979682 wait complete. 2021-04-03T03:17:34.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 Processor -- start 2021-04-03T03:17:34.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 -- start start 2021-04-03T03:17:34.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6c40603f0 0x7fe6c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40b12b0 0x7fe6c4064b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 0x7fe6c40aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40abaa0 con 0x7fe6c40b12b0 2021-04-03T03:17:34.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40a9d80 con 0x7fe6c40603f0 2021-04-03T03:17:34.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40ab370 con 0x7fe6c414b700 2021-04-03T03:17:34.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40b12b0 0x7fe6c4064b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40b12b0 0x7fe6c4064b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58906/0 (socket says 172.21.2.111:58906) 2021-04-03T03:17:34.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 -- 172.21.2.111:0/3132180516 learned_addr learned my addr 172.21.2.111:0/3132180516 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:34.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c2ffd700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6c40603f0 0x7fe6c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 -- 172.21.2.111:0/3132180516 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 msgr2=0x7fe6c40aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 0x7fe6c40aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 -- 172.21.2.111:0/3132180516 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6c40603f0 msgr2=0x7fe6c4062d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6c40603f0 0x7fe6c4062d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 -- 172.21.2.111:0/3132180516 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40ab560 con 0x7fe6c40b12b0 2021-04-03T03:17:34.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c27fc700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40b12b0 0x7fe6c4064b70 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7fe6b4002f30 tx=0x7fe6b4006e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6bbfff700 1 -- 172.21.2.111:0/3132180516 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6b400d570 con 0x7fe6c40b12b0 2021-04-03T03:17:34.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6bbfff700 1 -- 172.21.2.111:0/3132180516 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4000e90 con 0x7fe6c40b12b0 2021-04-03T03:17:34.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c40ab800 con 0x7fe6c40b12b0 2021-04-03T03:17:34.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6bbfff700 1 -- 172.21.2.111:0/3132180516 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6b400d8c0 con 0x7fe6c40b12b0 2021-04-03T03:17:34.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c4159b00 con 0x7fe6c40b12b0 2021-04-03T03:17:34.237 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c4057660 con 0x7fe6c40b12b0 2021-04-03T03:17:34.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6bbfff700 1 -- 172.21.2.111:0/3132180516 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe6b400f070 con 0x7fe6c40b12b0 2021-04-03T03:17:34.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6bbfff700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6a8056ea0 0x7fe6a8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6bbfff700 1 -- 172.21.2.111:0/3132180516 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe6b406f920 con 0x7fe6c40b12b0 2021-04-03T03:17:34.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c2ffd700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6a8056ea0 0x7fe6a8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.238 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.225+0000 7fe6c2ffd700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6a8056ea0 0x7fe6a8059360 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fe6ac017000 tx=0x7fe6ac015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.229+0000 7fe6bbfff700 1 -- 172.21.2.111:0/3132180516 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe6b4044d10 con 0x7fe6c40b12b0 2021-04-03T03:17:34.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.349+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6c40686d0 con 0x7fe6c40b12b0 2021-04-03T03:17:34.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.349+0000 7fe6bbfff700 1 -- 172.21.2.111:0/3132180516 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fe6b406de60 con 0x7fe6c40b12b0 2021-04-03T03:17:34.353 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:34.354 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":68,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3455926982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3455926982},{"type":"v1","addr":"172.21.2.115:6833","nonce":3455926982}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:33.145029+0000","modified":"2021-04-03T03:17:34.147803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/295340141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":295340141},{"type":"v1","addr":"172.21.2.111:6835","nonce":295340141}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-04-03T03:17:34.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6a8056ea0 msgr2=0x7fe6a8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6a8056ea0 0x7fe6a8059360 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fe6ac017000 tx=0x7fe6ac015000).stop 2021-04-03T03:17:34.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40b12b0 msgr2=0x7fe6c4064b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40b12b0 0x7fe6c4064b70 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7fe6b4002f30 tx=0x7fe6b4006e00).stop 2021-04-03T03:17:34.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 shutdown_connections 2021-04-03T03:17:34.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe6c414b700 0x7fe6c40aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe6a8056ea0 0x7fe6a8059360 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe6c40603f0 0x7fe6c4062d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 --2- 172.21.2.111:0/3132180516 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe6c40b12b0 0x7fe6c4064b70 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 >> 172.21.2.111:0/3132180516 conn(0x7fe6c40b61c0 msgr2=0x7fe6c40b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 shutdown_connections 2021-04-03T03:17:34.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.353+0000 7fe6c3fff700 1 -- 172.21.2.111:0/3132180516 wait complete. 2021-04-03T03:17:34.357 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 68 2021-04-03T03:17:34.361 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:34.415 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69fa778700 1 Processor -- start 2021-04-03T03:17:34.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69fa778700 1 -- start start 2021-04-03T03:17:34.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69fa778700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40603f0 0x7f69f40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.416 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69fa778700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f69f414b700 0x7f69f40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69fa778700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f40607d0 con 0x7f69f40b12b0 2021-04-03T03:17:34.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69fa778700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f40af400 con 0x7f69f414b700 2021-04-03T03:17:34.417 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69fa778700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40af6a0 con 0x7f69f40603f0 2021-04-03T03:17:34.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69f8f75700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f69f414b700 0x7f69f40b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69f9776700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40603f0 0x7f69f40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.418 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69f9f77700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f69f40b12b0 0x7f69f41465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:44010/0 (socket says 172.21.2.111:44010) 2021-04-03T03:17:34.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.413+0000 7f69f9f77700 1 -- 172.21.2.111:0/503519240 learned_addr learned my addr 172.21.2.111:0/503519240 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:34.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9776700 1 -- 172.21.2.111:0/503519240 >> v2:172.21.2.115:3300/0 conn(0x7f69f414b700 msgr2=0x7f69f40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:34.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9776700 1 --2- 172.21.2.111:0/503519240 >> v2:172.21.2.115:3300/0 conn(0x7f69f414b700 0x7f69f40b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9776700 1 -- 172.21.2.111:0/503519240 >> v1:172.21.2.115:6789/0 conn(0x7f69f40b12b0 legacy=0x7f69f41465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.419 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9776700 1 -- 172.21.2.111:0/503519240 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f40afb40 con 0x7f69f40603f0 2021-04-03T03:17:34.420 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9776700 1 --2- 172.21.2.111:0/503519240 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40603f0 0x7f69f40b0090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f69f0002f30 tx=0x7f69f0006dd0).ready entity=mon.2 client_cookie=c62663836c086a41 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69e7fff700 1 -- 172.21.2.111:0/503519240 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f0009760 con 0x7f69f40603f0 2021-04-03T03:17:34.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69e7fff700 1 -- 172.21.2.111:0/503519240 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f00098c0 con 0x7f69f40603f0 2021-04-03T03:17:34.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69e7fff700 1 -- 172.21.2.111:0/503519240 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f0009c20 con 0x7f69f40603f0 2021-04-03T03:17:34.423 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- 172.21.2.111:0/503519240 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40603f0 msgr2=0x7f69f40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 --2- 172.21.2.111:0/503519240 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40603f0 0x7f69f40b0090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f69f0002f30 tx=0x7f69f0006dd0).stop 2021-04-03T03:17:34.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- 172.21.2.111:0/503519240 shutdown_connections 2021-04-03T03:17:34.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 --2- 172.21.2.111:0/503519240 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40603f0 0x7f69f40b0090 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 --2- 172.21.2.111:0/503519240 >> v2:172.21.2.115:3300/0 conn(0x7f69f414b700 0x7f69f40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- 172.21.2.111:0/503519240 >> 172.21.2.111:0/503519240 conn(0x7f69f40b61c0 msgr2=0x7f69f405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- 172.21.2.111:0/503519240 shutdown_connections 2021-04-03T03:17:34.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- 172.21.2.111:0/503519240 wait complete. 2021-04-03T03:17:34.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 Processor -- start 2021-04-03T03:17:34.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- start start 2021-04-03T03:17:34.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69f40603f0 0x7f69f4062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40b12b0 0x7f69f4146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69f414b700 0x7f69f4146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40ab6f0 con 0x7f69f414b700 2021-04-03T03:17:34.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40aa050 con 0x7f69f40603f0 2021-04-03T03:17:34.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40aa2f0 con 0x7f69f40b12b0 2021-04-03T03:17:34.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69f414b700 0x7f69f4146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69f414b700 0x7f69f4146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58920/0 (socket says 172.21.2.111:58920) 2021-04-03T03:17:34.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 -- 172.21.2.111:0/2132717019 learned_addr learned my addr 172.21.2.111:0/2132717019 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:34.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9776700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69f40603f0 0x7f69f4062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f8f75700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40b12b0 0x7f69f4146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 -- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40b12b0 msgr2=0x7f69f4146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40b12b0 0x7f69f4146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 -- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69f40603f0 msgr2=0x7f69f4062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69f40603f0 0x7f69f4062c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 -- 172.21.2.111:0/2132717019 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4156ee0 con 0x7f69f414b700 2021-04-03T03:17:34.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f8f75700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40b12b0 0x7f69f4146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:34.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69f9f77700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69f414b700 0x7f69f4146e20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f69f40628d0 tx=0x7f69ec007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69e67fc700 1 -- 172.21.2.111:0/2132717019 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69ec00ce20 con 0x7f69f414b700 2021-04-03T03:17:34.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69e67fc700 1 -- 172.21.2.111:0/2132717019 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec005b80 con 0x7f69f414b700 2021-04-03T03:17:34.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f40a9d40 con 0x7f69f414b700 2021-04-03T03:17:34.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.417+0000 7f69e67fc700 1 -- 172.21.2.111:0/2132717019 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69ec007110 con 0x7f69f414b700 2021-04-03T03:17:34.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f4147310 con 0x7f69f414b700 2021-04-03T03:17:34.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f4157070 con 0x7f69f414b700 2021-04-03T03:17:34.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69e67fc700 1 -- 172.21.2.111:0/2132717019 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f69ec010030 con 0x7f69f414b700 2021-04-03T03:17:34.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69e67fc700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69d4056fb0 0x7f69d4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69e67fc700 1 -- 172.21.2.111:0/2132717019 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f69ec070260 con 0x7f69f414b700 2021-04-03T03:17:34.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69f9776700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69d4056fb0 0x7f69d4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69f9776700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69d4056fb0 0x7f69d4059470 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f69f001b000 tx=0x7f69f0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.421+0000 7f69e67fc700 1 -- 172.21.2.111:0/2132717019 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f69ec045540 con 0x7f69f414b700 2021-04-03T03:17:34.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69f4057660 con 0x7f69f414b700 2021-04-03T03:17:34.516 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69e67fc700 1 -- 172.21.2.111:0/2132717019 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f69ec048b60 con 0x7f69f414b700 2021-04-03T03:17:34.517 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:34.517 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":151,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:34.148833+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:31.174355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:32.113426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:34.518 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69d4056fb0 msgr2=0x7f69d4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69d4056fb0 0x7f69d4059470 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f69f001b000 tx=0x7f69f0015000).stop 2021-04-03T03:17:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69f414b700 msgr2=0x7f69f4146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69f414b700 0x7f69f4146e20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f69f40628d0 tx=0x7f69ec007ee0).stop 2021-04-03T03:17:34.519 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 shutdown_connections 2021-04-03T03:17:34.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f69f414b700 0x7f69f4146e20 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f69d4056fb0 0x7f69d4059470 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f69f40603f0 0x7f69f4062c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.520 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.513+0000 7f69fa778700 1 --2- 172.21.2.111:0/2132717019 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f69f40b12b0 0x7f69f4146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.517+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 >> 172.21.2.111:0/2132717019 conn(0x7f69f40b61c0 msgr2=0x7f69f40acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.517+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 shutdown_connections 2021-04-03T03:17:34.521 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.517+0000 7f69fa778700 1 -- 172.21.2.111:0/2132717019 wait complete. 2021-04-03T03:17:34.525 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:17:34.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4948cfb700 1 Processor -- start 2021-04-03T03:17:34.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4948cfb700 1 -- start start 2021-04-03T03:17:34.588 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4948cfb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944153440 0x7f4944080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4948cfb700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440b3ca0 con 0x7f49440b38c0 2021-04-03T03:17:34.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4948cfb700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4944025b90 con 0x7f4944025480 2021-04-03T03:17:34.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4948cfb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944025e30 con 0x7f4944153440 2021-04-03T03:17:34.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4943fff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f49440b38c0 0x7f4944080550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59716/0 (socket says 172.21.2.111:59716) 2021-04-03T03:17:34.589 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4943fff700 1 -- 172.21.2.111:0/1897214367 learned_addr learned my addr 172.21.2.111:0/1897214367 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:34.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280056740 0 0) 0x7f49440b3ca0 con 0x7f49440b38c0 2021-04-03T03:17:34.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.581+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4930003240 con 0x7f49440b38c0 2021-04-03T03:17:34.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944153440 0x7f4944080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.590 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3693275840 0 0) 0x7f4930003240 con 0x7f49440b38c0 2021-04-03T03:17:34.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944153440 msgr2=0x7f4944080c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:34.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944153440 0x7f4944080c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 >> v1:172.21.2.115:6789/0 conn(0x7f4944025480 legacy=0x7f49440af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:34.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440260d0 con 0x7f49440b38c0 2021-04-03T03:17:34.591 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f492c0016d0 con 0x7f49440b38c0 2021-04-03T03:17:34.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 >> v1:172.21.2.111:6789/0 conn(0x7f49440b38c0 legacy=0x7f4944080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944153440 0x7f49300037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4930003d20 0x7f49300044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.592 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49300049d0 0x7f4930004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944025e30 con 0x7f4930003d20 2021-04-03T03:17:34.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300042b0 con 0x7f49300049d0 2021-04-03T03:17:34.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493004f740 con 0x7f4944153440 2021-04-03T03:17:34.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4930003d20 0x7f49300044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.593 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 -- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944153440 msgr2=0x7f49300037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:34.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944153440 0x7f49300037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 -- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49300049d0 msgr2=0x7f4930004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:34.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f49300049d0 0x7f4930004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 -- 172.21.2.111:0/1897214367 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930003240 con 0x7f4930003d20 2021-04-03T03:17:34.594 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4930003d20 0x7f49300044e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f493400a000 tx=0x7f4934008040).ready entity=mon.0 client_cookie=59129b93c4ff16bf server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493400da00 con 0x7f4930003d20 2021-04-03T03:17:34.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400db60 con 0x7f4930003d20 2021-04-03T03:17:34.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4942ffd700 1 -- 172.21.2.111:0/1897214367 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4934013480 con 0x7f4930003d20 2021-04-03T03:17:34.595 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- 172.21.2.111:0/1897214367 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4930003d20 msgr2=0x7f49300044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 --2- 172.21.2.111:0/1897214367 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4930003d20 0x7f49300044e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f493400a000 tx=0x7f4934008040).stop 2021-04-03T03:17:34.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f494a175700 1 -- 172.21.2.111:0/1897214367 reap_dead start 2021-04-03T03:17:34.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- 172.21.2.111:0/1897214367 shutdown_connections 2021-04-03T03:17:34.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- 172.21.2.111:0/1897214367 >> 172.21.2.111:0/1897214367 conn(0x7f4944061610 msgr2=0x7f4944061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.596 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- 172.21.2.111:0/1897214367 shutdown_connections 2021-04-03T03:17:34.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- 172.21.2.111:0/1897214367 wait complete. 2021-04-03T03:17:34.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 Processor -- start 2021-04-03T03:17:34.597 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- start start 2021-04-03T03:17:34.599 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944150a00 0x7f49440ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49440a9da0 0x7f4944063f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4944064430 0x7f4944065a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944150de0 con 0x7f49440a9da0 2021-04-03T03:17:34.600 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944151c90 con 0x7f4944064430 2021-04-03T03:17:34.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944151f30 con 0x7f4944150a00 2021-04-03T03:17:34.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49440a9da0 0x7f4944063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49440a9da0 0x7f4944063f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58936/0 (socket says 172.21.2.111:58936) 2021-04-03T03:17:34.601 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 -- 172.21.2.111:0/3256549610 learned_addr learned my addr 172.21.2.111:0/3256549610 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:34.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f494a175700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4944064430 0x7f4944065a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4943fff700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944150a00 0x7f49440ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 -- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944150a00 msgr2=0x7f49440ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944150a00 0x7f49440ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.602 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 -- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4944064430 msgr2=0x7f4944065a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4944064430 0x7f4944065a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 -- 172.21.2.111:0/3256549610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440ad620 con 0x7f49440a9da0 2021-04-03T03:17:34.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f494a175700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4944064430 0x7f4944065a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:17:34.603 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f49437fe700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49440a9da0 0x7f4944063f40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f493400a000 tx=0x7f4934008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4940ff9700 1 -- 172.21.2.111:0/3256549610 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4934021070 con 0x7f49440a9da0 2021-04-03T03:17:34.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4940ff9700 1 -- 172.21.2.111:0/3256549610 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493401d630 con 0x7f49440a9da0 2021-04-03T03:17:34.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49441523d0 con 0x7f49440a9da0 2021-04-03T03:17:34.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4940ff9700 1 -- 172.21.2.111:0/3256549610 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493401d990 con 0x7f49440a9da0 2021-04-03T03:17:34.604 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.585+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49440660e0 con 0x7f49440a9da0 2021-04-03T03:17:34.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.589+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4944057660 con 0x7f49440a9da0 2021-04-03T03:17:34.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.589+0000 7f4940ff9700 1 -- 172.21.2.111:0/3256549610 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f493401daf0 con 0x7f49440a9da0 2021-04-03T03:17:34.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.589+0000 7f4940ff9700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4928056e90 0x7f4928059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.589+0000 7f4940ff9700 1 -- 172.21.2.111:0/3256549610 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f49340715c0 con 0x7f49440a9da0 2021-04-03T03:17:34.605 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.589+0000 7f4943fff700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4928056e90 0x7f4928059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.589+0000 7f4943fff700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4928056e90 0x7f4928059350 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f492c00b000 tx=0x7f4944025d20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.606 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.589+0000 7f4940ff9700 1 -- 172.21.2.111:0/3256549610 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f49340468a0 con 0x7f49440a9da0 2021-04-03T03:17:34.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.717+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f49441520a0 con 0x7f49440a9da0 2021-04-03T03:17:34.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4940ff9700 1 -- 172.21.2.111:0/3256549610 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v44) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4934049ec0 con 0x7f49440a9da0 2021-04-03T03:17:34.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4928056e90 msgr2=0x7f4928059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4928056e90 0x7f4928059350 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f492c00b000 tx=0x7f4944025d20).stop 2021-04-03T03:17:34.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49440a9da0 msgr2=0x7f4944063f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49440a9da0 0x7f4944063f40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f493400a000 tx=0x7f4934008040).stop 2021-04-03T03:17:34.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 shutdown_connections 2021-04-03T03:17:34.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4928056e90 0x7f4928059350 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4944064430 0x7f4944065a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4944150a00 0x7f49440ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 --2- 172.21.2.111:0/3256549610 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f49440a9da0 0x7f4944063f40 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.721+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 >> 172.21.2.111:0/3256549610 conn(0x7f4944061610 msgr2=0x7f49440b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.725+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 shutdown_connections 2021-04-03T03:17:34.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.725+0000 7f4948cfb700 1 -- 172.21.2.111:0/3256549610 wait complete. 2021-04-03T03:17:34.729 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.0 2021-04-03T03:17:34.733 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:17:34.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6335762700 1 Processor -- start 2021-04-03T03:17:34.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6335762700 1 -- start start 2021-04-03T03:17:34.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6335762700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 0x7f632803f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6335762700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6328042380 0x7f63280f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6335762700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6328041240 con 0x7f6328046ce0 2021-04-03T03:17:34.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6335762700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63280f1850 con 0x7f6328042380 2021-04-03T03:17:34.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6335762700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6328042810 con 0x7f632803d350 2021-04-03T03:17:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6336bdc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f6328042380 0x7f63280f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6334f61700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f6328046ce0 0x7f6328041c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59738/0 (socket says 172.21.2.111:59738) 2021-04-03T03:17:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.789+0000 7f6334f61700 1 -- 172.21.2.111:0/3738533128 learned_addr learned my addr 172.21.2.111:0/3738533128 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:34.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 -- 172.21.2.111:0/3738533128 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 msgr2=0x7f632803f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 --2- 172.21.2.111:0/3738533128 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 0x7f632803f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 -- 172.21.2.111:0/3738533128 >> v1:172.21.2.111:6789/0 conn(0x7f6328046ce0 legacy=0x7f6328041c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 -- 172.21.2.111:0/3738533128 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280470c0 con 0x7f6328042380 2021-04-03T03:17:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 --2- 172.21.2.111:0/3738533128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328042380 0x7f63280f51b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f633006b5d0 tx=0x7f633005ad70).ready entity=mon.0 client_cookie=40345e6fdb77a32f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632f7fe700 1 -- 172.21.2.111:0/3738533128 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f633005bda0 con 0x7f6328042380 2021-04-03T03:17:34.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632f7fe700 1 -- 172.21.2.111:0/3738533128 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633005df60 con 0x7f6328042380 2021-04-03T03:17:34.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632f7fe700 1 -- 172.21.2.111:0/3738533128 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63300699f0 con 0x7f6328042380 2021-04-03T03:17:34.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3738533128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328042380 msgr2=0x7f63280f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 --2- 172.21.2.111:0/3738533128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328042380 0x7f63280f51b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f633006b5d0 tx=0x7f633005ad70).stop 2021-04-03T03:17:34.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3738533128 shutdown_connections 2021-04-03T03:17:34.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 --2- 172.21.2.111:0/3738533128 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328042380 0x7f63280f51b0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 --2- 172.21.2.111:0/3738533128 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 0x7f632803f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3738533128 >> 172.21.2.111:0/3738533128 conn(0x7f632800d0c0 msgr2=0x7f63280462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3738533128 shutdown_connections 2021-04-03T03:17:34.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3738533128 wait complete. 2021-04-03T03:17:34.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 Processor -- start 2021-04-03T03:17:34.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- start start 2021-04-03T03:17:34.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 0x7f6328045870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6328042380 0x7f63280d3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328046ce0 0x7f63280d3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632803c5b0 con 0x7f6328046ce0 2021-04-03T03:17:34.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632803bdd0 con 0x7f6328042380 2021-04-03T03:17:34.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632803c070 con 0x7f632803d350 2021-04-03T03:17:34.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328046ce0 0x7f63280d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328046ce0 0x7f63280d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58954/0 (socket says 172.21.2.111:58954) 2021-04-03T03:17:34.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 -- 172.21.2.111:0/3643902032 learned_addr learned my addr 172.21.2.111:0/3643902032 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:34.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632ffff700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6328042380 0x7f63280d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 -- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 msgr2=0x7f6328045870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 0x7f6328045870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 -- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6328042380 msgr2=0x7f63280d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6328042380 0x7f63280d3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 -- 172.21.2.111:0/3643902032 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632803c310 con 0x7f6328046ce0 2021-04-03T03:17:34.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632ffff700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6328042380 0x7f63280d3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:34.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6336bdc700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328046ce0 0x7f63280d3800 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f633006b5d0 tx=0x7f6330064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632d7fa700 1 -- 172.21.2.111:0/3643902032 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63300699f0 con 0x7f6328046ce0 2021-04-03T03:17:34.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632d7fa700 1 -- 172.21.2.111:0/3643902032 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63300441b0 con 0x7f6328046ce0 2021-04-03T03:17:34.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f632d7fa700 1 -- 172.21.2.111:0/3643902032 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f633005f4b0 con 0x7f6328046ce0 2021-04-03T03:17:34.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63280b7370 con 0x7f6328046ce0 2021-04-03T03:17:34.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63280d3e80 con 0x7f6328046ce0 2021-04-03T03:17:34.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.793+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63280b7560 con 0x7f6328046ce0 2021-04-03T03:17:34.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.797+0000 7f632d7fa700 1 -- 172.21.2.111:0/3643902032 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f633007b020 con 0x7f6328046ce0 2021-04-03T03:17:34.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.797+0000 7f632d7fa700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6310057040 0x7f6310059500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:34.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.797+0000 7f632d7fa700 1 -- 172.21.2.111:0/3643902032 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f63300cf6f0 con 0x7f6328046ce0 2021-04-03T03:17:34.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.797+0000 7f6334f61700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6310057040 0x7f6310059500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:34.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.797+0000 7f6334f61700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6310057040 0x7f6310059500 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f63280f5090 tx=0x7f6320007840).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:34.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.797+0000 7f632d7fa700 1 -- 172.21.2.111:0/3643902032 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f63300a4ae0 con 0x7f6328046ce0 2021-04-03T03:17:34.928 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.925+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6328004d90 con 0x7f6328046ce0 2021-04-03T03:17:34.932 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.929+0000 7f632d7fa700 1 -- 172.21.2.111:0/3643902032 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v45) v1 ==== 184+0+0 (secure 0 0 0) 0x7f63300a8100 con 0x7f6328046ce0 2021-04-03T03:17:34.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.929+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6310057040 msgr2=0x7f6310059500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.929+0000 7f6335762700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6310057040 0x7f6310059500 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f63280f5090 tx=0x7f6320007840).stop 2021-04-03T03:17:34.934 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.929+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328046ce0 msgr2=0x7f63280d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.929+0000 7f6335762700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328046ce0 0x7f63280d3800 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f633006b5d0 tx=0x7f6330064030).stop 2021-04-03T03:17:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 shutdown_connections 2021-04-03T03:17:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f6328046ce0 0x7f63280d3800 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.935 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f632803d350 0x7f6328045870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6310057040 0x7f6310059500 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 --2- 172.21.2.111:0/3643902032 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f6328042380 0x7f63280d3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 >> 172.21.2.111:0/3643902032 conn(0x7f632800d0c0 msgr2=0x7f63280f1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 shutdown_connections 2021-04-03T03:17:34.936 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.933+0000 7f6335762700 1 -- 172.21.2.111:0/3643902032 wait complete. 2021-04-03T03:17:34.937 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.1 2021-04-03T03:17:34.941 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-04-03T03:17:35.002 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 Processor -- start 2021-04-03T03:17:35.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 -- start start 2021-04-03T03:17:35.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f5003d350 0x7f9f5003f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9f50042380 0x7f9f500f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.003 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f50041240 con 0x7f9f50046ce0 2021-04-03T03:17:35.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500f1850 con 0x7f9f50042380 2021-04-03T03:17:35.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50042810 con 0x7f9f5003d350 2021-04-03T03:17:35.004 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9f50042380 0x7f9f500f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f9f50042380 0x7f9f500f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58960/0 (socket says 172.21.2.111:58960) 2021-04-03T03:17:35.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/3748933679 learned_addr learned my addr 172.21.2.111:0/3748933679 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:35.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/3748933679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f5003d350 msgr2=0x7f9f5003f650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:35.005 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 --2- 172.21.2.111:0/3748933679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f5003d350 0x7f9f5003f650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/3748933679 >> v1:172.21.2.115:6789/0 conn(0x7f9f50046ce0 legacy=0x7f9f50041c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:35.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/3748933679 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500470c0 con 0x7f9f50042380 2021-04-03T03:17:35.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f5c9cd700 1 --2- 172.21.2.111:0/3748933679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50042380 0x7f9f500f51b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7f9f500f5090 tx=0x7f9f5805ad70).ready entity=mon.0 client_cookie=773b0f97b00f0c7c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f55b83700 1 -- 172.21.2.111:0/3748933679 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f5805bda0 con 0x7f9f50042380 2021-04-03T03:17:35.006 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f55b83700 1 -- 172.21.2.111:0/3748933679 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5805df60 con 0x7f9f50042380 2021-04-03T03:17:35.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f55b83700 1 -- 172.21.2.111:0/3748933679 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f580699f0 con 0x7f9f50042380 2021-04-03T03:17:35.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 -- 172.21.2.111:0/3748933679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50042380 msgr2=0x7f9f500f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:34.997+0000 7f9f57386700 1 --2- 172.21.2.111:0/3748933679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50042380 0x7f9f500f51b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7f9f500f5090 tx=0x7f9f5805ad70).stop 2021-04-03T03:17:35.007 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- 172.21.2.111:0/3748933679 shutdown_connections 2021-04-03T03:17:35.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 --2- 172.21.2.111:0/3748933679 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50042380 0x7f9f500f51b0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 --2- 172.21.2.111:0/3748933679 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f5003d350 0x7f9f5003f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- 172.21.2.111:0/3748933679 >> 172.21.2.111:0/3748933679 conn(0x7f9f5000d0c0 msgr2=0x7f9f500462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- 172.21.2.111:0/3748933679 shutdown_connections 2021-04-03T03:17:35.009 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- 172.21.2.111:0/3748933679 wait complete. 2021-04-03T03:17:35.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 Processor -- start 2021-04-03T03:17:35.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- start start 2021-04-03T03:17:35.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9f5003d350 0x7f9f500f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f50042380 0x7f9f500ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50046ce0 0x7f9f50039ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500a87d0 con 0x7f9f50046ce0 2021-04-03T03:17:35.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500dfc10 con 0x7f9f5003d350 2021-04-03T03:17:35.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500acb40 con 0x7f9f50042380 2021-04-03T03:17:35.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50046ce0 0x7f9f50039ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50046ce0 0x7f9f50039ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58968/0 (socket says 172.21.2.111:58968) 2021-04-03T03:17:35.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/2307527971 learned_addr learned my addr 172.21.2.111:0/2307527971 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:35.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/2307527971 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f50042380 msgr2=0x7f9f500ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f50042380 0x7f9f500ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.014 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/2307527971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9f5003d350 msgr2=0x7f9f500f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9f5003d350 0x7f9f500f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 -- 172.21.2.111:0/2307527971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500dfe40 con 0x7f9f50046ce0 2021-04-03T03:17:35.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f5c9cd700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50046ce0 0x7f9f50039ca0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7f9f5003bb30 tx=0x7f9f58064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f4f7fe700 1 -- 172.21.2.111:0/2307527971 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f580699f0 con 0x7f9f50046ce0 2021-04-03T03:17:35.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f4f7fe700 1 -- 172.21.2.111:0/2307527971 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f580441b0 con 0x7f9f50046ce0 2021-04-03T03:17:35.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5003a190 con 0x7f9f50046ce0 2021-04-03T03:17:35.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f500ab500 con 0x7f9f50046ce0 2021-04-03T03:17:35.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f4f7fe700 1 -- 172.21.2.111:0/2307527971 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f5805f4b0 con 0x7f9f50046ce0 2021-04-03T03:17:35.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f500ac990 con 0x7f9f50046ce0 2021-04-03T03:17:35.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f4f7fe700 1 -- 172.21.2.111:0/2307527971 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f9f5807b020 con 0x7f9f50046ce0 2021-04-03T03:17:35.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.001+0000 7f9f4f7fe700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9f3c056e90 0x7f9f3c059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.005+0000 7f9f4f7fe700 1 -- 172.21.2.111:0/2307527971 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f9f580cf890 con 0x7f9f50046ce0 2021-04-03T03:17:35.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.005+0000 7f9f56b85700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9f3c056e90 0x7f9f3c059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.005+0000 7f9f56b85700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9f3c056e90 0x7f9f3c059350 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f9f48000e90 tx=0x7f9f48006900).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.005+0000 7f9f4f7fe700 1 -- 172.21.2.111:0/2307527971 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f9f580a4b70 con 0x7f9f50046ce0 2021-04-03T03:17:35.135 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.133+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9f500dffd0 con 0x7f9f50046ce0 2021-04-03T03:17:35.140 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.137+0000 7f9f4f7fe700 1 -- 172.21.2.111:0/2307527971 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v46) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9f580a8190 con 0x7f9f50046ce0 2021-04-03T03:17:35.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.137+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9f3c056e90 msgr2=0x7f9f3c059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.137+0000 7f9f57386700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9f3c056e90 0x7f9f3c059350 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f9f48000e90 tx=0x7f9f48006900).stop 2021-04-03T03:17:35.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.137+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50046ce0 msgr2=0x7f9f50039ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.142 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.137+0000 7f9f57386700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50046ce0 0x7f9f50039ca0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7f9f5003bb30 tx=0x7f9f58064030).stop 2021-04-03T03:17:35.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 shutdown_connections 2021-04-03T03:17:35.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f9f50046ce0 0x7f9f50039ca0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9f3c056e90 0x7f9f3c059350 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.143 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f9f5003d350 0x7f9f500f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 --2- 172.21.2.111:0/2307527971 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f9f50042380 0x7f9f500ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 >> 172.21.2.111:0/2307527971 conn(0x7f9f5000d0c0 msgr2=0x7f9f50044a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 shutdown_connections 2021-04-03T03:17:35.144 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.141+0000 7f9f57386700 1 -- 172.21.2.111:0/2307527971 wait complete. 2021-04-03T03:17:35.145 INFO:teuthology.orchestra.run.gibba011.stderr:updated caps for client.2 2021-04-03T03:17:35.149 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:35.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.201+0000 7ff3ad9de700 1 Processor -- start 2021-04-03T03:17:35.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.201+0000 7ff3ad9de700 1 -- start start 2021-04-03T03:17:35.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00dbb90 0x7ff3a0042750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff3a0006bc0 0x7ff3a00dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a00dc6e0 con 0x7ff3a003d360 2021-04-03T03:17:35.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a00d9560 con 0x7ff3a0006bc0 2021-04-03T03:17:35.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a003dcb0 con 0x7ff3a00dbb90 2021-04-03T03:17:35.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff3a0006bc0 0x7ff3a00dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff3a0006bc0 0x7ff3a00dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58976/0 (socket says 172.21.2.111:58976) 2021-04-03T03:17:35.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 -- 172.21.2.111:0/2782660582 learned_addr learned my addr 172.21.2.111:0/2782660582 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:35.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 -- 172.21.2.111:0/2782660582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00dbb90 msgr2=0x7ff3a0042750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:35.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 --2- 172.21.2.111:0/2782660582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00dbb90 0x7ff3a0042750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 -- 172.21.2.111:0/2782660582 >> v1:172.21.2.115:6789/0 conn(0x7ff3a003d360 legacy=0x7ff3a00064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:35.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 -- 172.21.2.111:0/2782660582 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00dc460 con 0x7ff3a0006bc0 2021-04-03T03:17:35.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad1dd700 1 --2- 172.21.2.111:0/2782660582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a0006bc0 0x7ff3a00dbf70 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7ff3a0047210 tx=0x7ff3a805ad70).ready entity=mon.0 client_cookie=11f75bcf457c0c90 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3a77fe700 1 -- 172.21.2.111:0/2782660582 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a805bda0 con 0x7ff3a0006bc0 2021-04-03T03:17:35.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3a77fe700 1 -- 172.21.2.111:0/2782660582 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a805df60 con 0x7ff3a0006bc0 2021-04-03T03:17:35.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3a77fe700 1 -- 172.21.2.111:0/2782660582 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a80699f0 con 0x7ff3a0006bc0 2021-04-03T03:17:35.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- 172.21.2.111:0/2782660582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a0006bc0 msgr2=0x7ff3a00dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/2782660582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a0006bc0 0x7ff3a00dbf70 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7ff3a0047210 tx=0x7ff3a805ad70).stop 2021-04-03T03:17:35.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- 172.21.2.111:0/2782660582 shutdown_connections 2021-04-03T03:17:35.211 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/2782660582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a0006bc0 0x7ff3a00dbf70 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/2782660582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00dbb90 0x7ff3a0042750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- 172.21.2.111:0/2782660582 >> 172.21.2.111:0/2782660582 conn(0x7ff3a00462c0 msgr2=0x7ff3a00466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- 172.21.2.111:0/2782660582 shutdown_connections 2021-04-03T03:17:35.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- 172.21.2.111:0/2782660582 wait complete. 2021-04-03T03:17:35.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 Processor -- start 2021-04-03T03:17:35.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- start start 2021-04-03T03:17:35.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a003d360 0x7ff3a003b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3a00dbb90 0x7ff3a00d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00a3210 0x7ff3a009f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00a05c0 con 0x7ff3a003d360 2021-04-03T03:17:35.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00a0860 con 0x7ff3a00dbb90 2021-04-03T03:17:35.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00a2d70 con 0x7ff3a00a3210 2021-04-03T03:17:35.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a003d360 0x7ff3a003b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a003d360 0x7ff3a003b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58978/0 (socket says 172.21.2.111:58978) 2021-04-03T03:17:35.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 -- 172.21.2.111:0/3717885946 learned_addr learned my addr 172.21.2.111:0/3717885946 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:35.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 -- 172.21.2.111:0/3717885946 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00a3210 msgr2=0x7ff3a009f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00a3210 0x7ff3a009f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 -- 172.21.2.111:0/3717885946 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3a00dbb90 msgr2=0x7ff3a00d5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3a00dbb90 0x7ff3a00d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 -- 172.21.2.111:0/3717885946 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00a0b00 con 0x7ff3a003d360 2021-04-03T03:17:35.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ac9dc700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a003d360 0x7ff3a003b5a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7ff398002f30 tx=0x7ff398006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3a5ffb700 1 -- 172.21.2.111:0/3717885946 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff39800d7c0 con 0x7ff3a003d360 2021-04-03T03:17:35.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3a5ffb700 1 -- 172.21.2.111:0/3717885946 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39800d920 con 0x7ff3a003d360 2021-04-03T03:17:35.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3a5ffb700 1 -- 172.21.2.111:0/3717885946 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff39800dc80 con 0x7ff3a003d360 2021-04-03T03:17:35.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a00a18a0 con 0x7ff3a003d360 2021-04-03T03:17:35.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.205+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a00eaa00 con 0x7ff3a003d360 2021-04-03T03:17:35.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.209+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a0004da0 con 0x7ff3a003d360 2021-04-03T03:17:35.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.209+0000 7ff3a5ffb700 1 -- 172.21.2.111:0/3717885946 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff398018030 con 0x7ff3a003d360 2021-04-03T03:17:35.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.209+0000 7ff3a5ffb700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff388056ef0 0x7ff3880593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.209+0000 7ff3a5ffb700 1 -- 172.21.2.111:0/3717885946 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7ff39806fc50 con 0x7ff3a003d360 2021-04-03T03:17:35.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.209+0000 7ff3a7fff700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff388056ef0 0x7ff3880593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.209+0000 7ff3a7fff700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff388056ef0 0x7ff3880593b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7ff39c000f30 tx=0x7ff39c0077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.209+0000 7ff3a5ffb700 1 -- 172.21.2.111:0/3717885946 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff398044fe0 con 0x7ff3a003d360 2021-04-03T03:17:35.338 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3a00db650 con 0x7ff3a003d360 2021-04-03T03:17:35.339 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3a5ffb700 1 -- 172.21.2.111:0/3717885946 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2139 (secure 0 0 0) 0x7ff398048600 con 0x7ff3a003d360 2021-04-03T03:17:35.339 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:35.339 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":68,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3455926982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3455926982},{"type":"v1","addr":"172.21.2.115:6833","nonce":3455926982}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:33.145029+0000","modified":"2021-04-03T03:17:34.147803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/295340141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":295340141},{"type":"v1","addr":"172.21.2.111:6835","nonce":295340141}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-04-03T03:17:35.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff388056ef0 msgr2=0x7ff3880593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.340 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff388056ef0 0x7ff3880593b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7ff39c000f30 tx=0x7ff39c0077d0).stop 2021-04-03T03:17:35.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a003d360 msgr2=0x7ff3a003b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a003d360 0x7ff3a003b5a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7ff398002f30 tx=0x7ff398006e30).stop 2021-04-03T03:17:35.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 shutdown_connections 2021-04-03T03:17:35.341 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff3a00a3210 0x7ff3a009f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff3a003d360 0x7ff3a003b5a0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff388056ef0 0x7ff3880593b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 --2- 172.21.2.111:0/3717885946 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff3a00dbb90 0x7ff3a00d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 >> 172.21.2.111:0/3717885946 conn(0x7ff3a00462c0 msgr2=0x7ff3a00dadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.342 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 shutdown_connections 2021-04-03T03:17:35.343 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.337+0000 7ff3ad9de700 1 -- 172.21.2.111:0/3717885946 wait complete. 2021-04-03T03:17:35.343 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 68 2021-04-03T03:17:35.548 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:17:33.145029+0000', 'modified': '2021-04-03T03:17:34.147803+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8526': {'gid': 8526, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/295340141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 295340141}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 295340141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:17:35.548 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:17:35.549 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:17:35.620 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":8526,"want_state":"up:active","state":"up:active","rank_uptime":2.4682378960000002,"mdsmap_epoch":68,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":6.6147126380000003} 2021-04-03T03:17:35.621 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 8526, "mdsmap_epoch": 68, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 2.468237896, "state": "up:active", "uptime": 6.614712638, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:17:35.621 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:35.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.673+0000 7f045759e700 1 Processor -- start 2021-04-03T03:17:35.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.673+0000 7f045759e700 1 -- start start 2021-04-03T03:17:35.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.673+0000 7f045759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04580603f0 0x7f04580b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.673+0000 7f045759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f045814b700 0x7f04580b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.673+0000 7f045759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04580607d0 con 0x7f04580b12b0 2021-04-03T03:17:35.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.673+0000 7f045759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04580af400 con 0x7f045814b700 2021-04-03T03:17:35.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.673+0000 7f045759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04580af6a0 con 0x7f04580603f0 2021-04-03T03:17:35.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045659c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04580603f0 0x7f04580b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.679 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0456d9d700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f04580b12b0 0x7f04581465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:44082/0 (socket says 172.21.2.111:44082) 2021-04-03T03:17:35.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0456d9d700 1 -- 172.21.2.111:0/485947160 learned_addr learned my addr 172.21.2.111:0/485947160 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:35.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0455d9b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f045814b700 0x7f04580b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0455d9b700 1 -- 172.21.2.111:0/485947160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04580603f0 msgr2=0x7f04580b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:35.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0455d9b700 1 --2- 172.21.2.111:0/485947160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04580603f0 0x7f04580b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.680 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0455d9b700 1 -- 172.21.2.111:0/485947160 >> v1:172.21.2.115:6789/0 conn(0x7f04580b12b0 legacy=0x7f04581465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0455d9b700 1 -- 172.21.2.111:0/485947160 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04580afb40 con 0x7f045814b700 2021-04-03T03:17:35.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f0455d9b700 1 --2- 172.21.2.111:0/485947160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f045814b700 0x7f04580b0580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f0448002f30 tx=0x7f0448006dd0).ready entity=mon.1 client_cookie=894cb8d372c40004 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045559a700 1 -- 172.21.2.111:0/485947160 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0448009760 con 0x7f045814b700 2021-04-03T03:17:35.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045559a700 1 -- 172.21.2.111:0/485947160 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04480098c0 con 0x7f045814b700 2021-04-03T03:17:35.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045559a700 1 -- 172.21.2.111:0/485947160 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0448009c20 con 0x7f045814b700 2021-04-03T03:17:35.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 -- 172.21.2.111:0/485947160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f045814b700 msgr2=0x7f04580b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 --2- 172.21.2.111:0/485947160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f045814b700 0x7f04580b0580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f0448002f30 tx=0x7f0448006dd0).stop 2021-04-03T03:17:35.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 -- 172.21.2.111:0/485947160 shutdown_connections 2021-04-03T03:17:35.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 --2- 172.21.2.111:0/485947160 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f04580603f0 0x7f04580b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 --2- 172.21.2.111:0/485947160 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f045814b700 0x7f04580b0580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 -- 172.21.2.111:0/485947160 >> 172.21.2.111:0/485947160 conn(0x7f04580b61c0 msgr2=0x7f045805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 -- 172.21.2.111:0/485947160 shutdown_connections 2021-04-03T03:17:35.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 -- 172.21.2.111:0/485947160 wait complete. 2021-04-03T03:17:35.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 Processor -- start 2021-04-03T03:17:35.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.677+0000 7f045759e700 1 -- start start 2021-04-03T03:17:35.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04580603f0 0x7f0458157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f04580b12b0 0x7f04580aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f045814b700 0x7f0458146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04580aa190 con 0x7f04580603f0 2021-04-03T03:17:35.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04580aa430 con 0x7f04580b12b0 2021-04-03T03:17:35.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04580a9cc0 con 0x7f045814b700 2021-04-03T03:17:35.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04580603f0 0x7f0458157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04580603f0 0x7f0458157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58988/0 (socket says 172.21.2.111:58988) 2021-04-03T03:17:35.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 -- 172.21.2.111:0/1290881347 learned_addr learned my addr 172.21.2.111:0/1290881347 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:35.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 -- 172.21.2.111:0/1290881347 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f045814b700 msgr2=0x7f0458146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f045814b700 0x7f0458146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 -- 172.21.2.111:0/1290881347 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f04580b12b0 msgr2=0x7f04580aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f0455d9b700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f04580b12b0 0x7f04580aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f04580b12b0 0x7f04580aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 -- 172.21.2.111:0/1290881347 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04580ab600 con 0x7f04580603f0 2021-04-03T03:17:35.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045659c700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04580603f0 0x7f0458157260 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f044000c000 tx=0x7f0440006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f044f7fe700 1 -- 172.21.2.111:0/1290881347 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044000cae0 con 0x7f04580603f0 2021-04-03T03:17:35.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f044f7fe700 1 -- 172.21.2.111:0/1290881347 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044000cc40 con 0x7f04580603f0 2021-04-03T03:17:35.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f044f7fe700 1 -- 172.21.2.111:0/1290881347 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04400123f0 con 0x7f04580603f0 2021-04-03T03:17:35.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0458062da0 con 0x7f04580603f0 2021-04-03T03:17:35.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0458146f80 con 0x7f04580603f0 2021-04-03T03:17:35.691 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04580ab7f0 con 0x7f04580603f0 2021-04-03T03:17:35.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f044f7fe700 1 -- 172.21.2.111:0/1290881347 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0440012550 con 0x7f04580603f0 2021-04-03T03:17:35.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f044f7fe700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f043c056f70 0x7f043c059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f044f7fe700 1 -- 172.21.2.111:0/1290881347 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f04400706e0 con 0x7f04580603f0 2021-04-03T03:17:35.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f0455d9b700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f043c056f70 0x7f043c059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.692 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.681+0000 7f0455d9b700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f043c056f70 0x7f043c059430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f0448016000 tx=0x7f0448015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.693 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.685+0000 7f044f7fe700 1 -- 172.21.2.111:0/1290881347 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0440045a70 con 0x7f04580603f0 2021-04-03T03:17:35.778 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04580aa5a0 con 0x7f04580603f0 2021-04-03T03:17:35.779 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f044f7fe700 1 -- 172.21.2.111:0/1290881347 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f0440049090 con 0x7f04580603f0 2021-04-03T03:17:35.779 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:35.779 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":151,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:34.148833+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:31.174355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:32.113426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:35.780 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f043c056f70 msgr2=0x7f043c059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f043c056f70 0x7f043c059430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f0448016000 tx=0x7f0448015000).stop 2021-04-03T03:17:35.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04580603f0 msgr2=0x7f0458157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04580603f0 0x7f0458157260 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f044000c000 tx=0x7f0440006dd0).stop 2021-04-03T03:17:35.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 shutdown_connections 2021-04-03T03:17:35.781 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f045814b700 0x7f0458146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f04580603f0 0x7f0458157260 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f043c056f70 0x7f043c059430 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 --2- 172.21.2.111:0/1290881347 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f04580b12b0 0x7f04580aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.782 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 >> 172.21.2.111:0/1290881347 conn(0x7f04580b61c0 msgr2=0x7f04580acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 shutdown_connections 2021-04-03T03:17:35.783 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.777+0000 7f045759e700 1 -- 172.21.2.111:0/1290881347 wait complete. 2021-04-03T03:17:35.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:17:31.174355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:17:32.113426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:17:35.788 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:17:35.844 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1cdf41700 1 Processor -- start 2021-04-03T03:17:35.845 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1cdf41700 1 -- start start 2021-04-03T03:17:35.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1cdf41700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1c0041060 0x7fc1c00dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1cdf41700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc1c003d560 0x7fc1c0041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1cdf41700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c00dc560 con 0x7fc1c00dbba0 2021-04-03T03:17:35.847 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1cdf41700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c003de60 con 0x7fc1c0041060 2021-04-03T03:17:35.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1cdf41700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c00dc800 con 0x7fc1c003d560 2021-04-03T03:17:35.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1c0041060 0x7fc1c00dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc1c0041060 0x7fc1c00dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:58998/0 (socket says 172.21.2.111:58998) 2021-04-03T03:17:35.848 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 -- 172.21.2.111:0/3042569754 learned_addr learned my addr 172.21.2.111:0/3042569754 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:35.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 -- 172.21.2.111:0/3042569754 >> v2:172.21.2.115:3300/0 conn(0x7fc1c003d560 msgr2=0x7fc1c0041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:35.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 --2- 172.21.2.111:0/3042569754 >> v2:172.21.2.115:3300/0 conn(0x7fc1c003d560 0x7fc1c0041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 -- 172.21.2.111:0/3042569754 >> v1:172.21.2.115:6789/0 conn(0x7fc1c00dbba0 legacy=0x7fc1c003fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:35.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 -- 172.21.2.111:0/3042569754 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c003da30 con 0x7fc1c0041060 2021-04-03T03:17:35.849 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c7fff700 1 --2- 172.21.2.111:0/3042569754 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c0041060 0x7fc1c00dbf80 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7fc1c00087d0 tx=0x7fc1b8006e90).ready entity=mon.0 client_cookie=72266a72a796a3e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.841+0000 7fc1c77fe700 1 -- 172.21.2.111:0/3042569754 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1b8009930 con 0x7fc1c0041060 2021-04-03T03:17:35.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c77fe700 1 -- 172.21.2.111:0/3042569754 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b8009a90 con 0x7fc1c0041060 2021-04-03T03:17:35.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c77fe700 1 -- 172.21.2.111:0/3042569754 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1b8009e30 con 0x7fc1c0041060 2021-04-03T03:17:35.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/3042569754 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c0041060 msgr2=0x7fc1c00dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.850 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/3042569754 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c0041060 0x7fc1c00dbf80 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7fc1c00087d0 tx=0x7fc1b8006e90).stop 2021-04-03T03:17:35.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/3042569754 shutdown_connections 2021-04-03T03:17:35.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/3042569754 >> v2:172.21.2.115:3300/0 conn(0x7fc1c003d560 0x7fc1c0041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/3042569754 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c0041060 0x7fc1c00dbf80 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.851 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/3042569754 >> 172.21.2.111:0/3042569754 conn(0x7fc1c00462b0 msgr2=0x7fc1c00466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/3042569754 shutdown_connections 2021-04-03T03:17:35.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/3042569754 wait complete. 2021-04-03T03:17:35.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 Processor -- start 2021-04-03T03:17:35.852 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- start start 2021-04-03T03:17:35.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c003d560 0x7fc1c003b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1c0041060 0x7fc1c00d6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.854 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1c00dbba0 0x7fc1c00d6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c00a0e40 con 0x7fc1c003d560 2021-04-03T03:17:35.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c0039900 con 0x7fc1c0041060 2021-04-03T03:17:35.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c0039ba0 con 0x7fc1c00dbba0 2021-04-03T03:17:35.855 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c003d560 0x7fc1c003b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c003d560 0x7fc1c003b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59002/0 (socket says 172.21.2.111:59002) 2021-04-03T03:17:35.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 -- 172.21.2.111:0/1489405486 learned_addr learned my addr 172.21.2.111:0/1489405486 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:35.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 -- 172.21.2.111:0/1489405486 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1c00dbba0 msgr2=0x7fc1c00d6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1c00dbba0 0x7fc1c00d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.856 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 -- 172.21.2.111:0/1489405486 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1c0041060 msgr2=0x7fc1c00d6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1c0041060 0x7fc1c00d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 -- 172.21.2.111:0/1489405486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c003b7a0 con 0x7fc1c003d560 2021-04-03T03:17:35.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1ccf3f700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c003d560 0x7fc1c003b2b0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7fc1bc002ed0 tx=0x7fc1bc006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c5ffb700 1 -- 172.21.2.111:0/1489405486 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1bc00f5e0 con 0x7fc1c003d560 2021-04-03T03:17:35.857 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c5ffb700 1 -- 172.21.2.111:0/1489405486 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc00f740 con 0x7fc1c003d560 2021-04-03T03:17:35.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c5ffb700 1 -- 172.21.2.111:0/1489405486 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1bc00f8b0 con 0x7fc1c003d560 2021-04-03T03:17:35.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c00d6d40 con 0x7fc1c003d560 2021-04-03T03:17:35.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c003bcf0 con 0x7fc1c003d560 2021-04-03T03:17:35.858 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c0039f60 con 0x7fc1c003d560 2021-04-03T03:17:35.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c5ffb700 1 -- 172.21.2.111:0/1489405486 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc1bc00f070 con 0x7fc1c003d560 2021-04-03T03:17:35.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c5ffb700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1a8056fb0 0x7fc1a8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:35.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c5ffb700 1 -- 172.21.2.111:0/1489405486 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc1bc06fab0 con 0x7fc1c003d560 2021-04-03T03:17:35.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.845+0000 7fc1c7fff700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1a8056fb0 0x7fc1a8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:35.859 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.849+0000 7fc1c7fff700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1a8056fb0 0x7fc1a8059470 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fc1c00087d0 tx=0x7fc1b8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:35.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.849+0000 7fc1c5ffb700 1 -- 172.21.2.111:0/1489405486 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc1bc044ea0 con 0x7fc1c003d560 2021-04-03T03:17:35.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc1c00a0ca0 con 0x7fc1c003d560 2021-04-03T03:17:35.947 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1c5ffb700 1 -- 172.21.2.111:0/1489405486 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc1c00a0ca0 con 0x7fc1c003d560 2021-04-03T03:17:35.948 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:17:35.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1a8056fb0 msgr2=0x7fc1a8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1a8056fb0 0x7fc1a8059470 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fc1c00087d0 tx=0x7fc1b8015000).stop 2021-04-03T03:17:35.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c003d560 msgr2=0x7fc1c003b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:35.949 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c003d560 0x7fc1c003b2b0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7fc1bc002ed0 tx=0x7fc1bc006d50).stop 2021-04-03T03:17:35.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 shutdown_connections 2021-04-03T03:17:35.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc1c00dbba0 0x7fc1c00d6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc1c003d560 0x7fc1c003b2b0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.950 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc1a8056fb0 0x7fc1a8059470 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:35.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 --2- 172.21.2.111:0/1489405486 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc1c0041060 0x7fc1c00d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:35.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 >> 172.21.2.111:0/1489405486 conn(0x7fc1c00462b0 msgr2=0x7fc1c0005920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:35.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 shutdown_connections 2021-04-03T03:17:35.951 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:35.945+0000 7fc1cdf41700 1 -- 172.21.2.111:0/1489405486 wait complete. 2021-04-03T03:17:35.956 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:17:36.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aabde7700 1 Processor -- start 2021-04-03T03:17:36.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aabde7700 1 -- start start 2021-04-03T03:17:36.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aabde7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7aa403d560 0x7f7aa40dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aabde7700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa40dc560 con 0x7f7aa40dbba0 2021-04-03T03:17:36.015 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aabde7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa40dc800 con 0x7f7aa4041060 2021-04-03T03:17:36.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aabde7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa403de60 con 0x7f7aa403d560 2021-04-03T03:17:36.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aab5e6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7aa403d560 0x7f7aa40dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aab5e6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f7aa403d560 0x7f7aa40dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59010/0 (socket says 172.21.2.111:59010) 2021-04-03T03:17:36.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aab5e6700 1 -- 172.21.2.111:0/3037905622 learned_addr learned my addr 172.21.2.111:0/3037905622 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:36.016 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aab5e6700 1 -- 172.21.2.111:0/3037905622 >> v1:172.21.2.115:6789/0 conn(0x7f7aa4041060 legacy=0x7f7aa4041650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T03:17:36.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aab5e6700 1 -- 172.21.2.111:0/3037905622 >> v1:172.21.2.111:6789/0 conn(0x7f7aa40dbba0 legacy=0x7f7aa403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aab5e6700 1 -- 172.21.2.111:0/3037905622 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa403da30 con 0x7f7aa403d560 2021-04-03T03:17:36.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.009+0000 7f7aab5e6700 1 --2- 172.21.2.111:0/3037905622 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa403d560 0x7f7aa40dbf80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f7aa4040a80 tx=0x7f7aac05ad70).ready entity=mon.0 client_cookie=4c28cfed6633f7df server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.017 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa9de3700 1 -- 172.21.2.111:0/3037905622 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aac05bda0 con 0x7f7aa403d560 2021-04-03T03:17:36.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa9de3700 1 -- 172.21.2.111:0/3037905622 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac05df60 con 0x7f7aa403d560 2021-04-03T03:17:36.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa9de3700 1 -- 172.21.2.111:0/3037905622 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aac0699f0 con 0x7f7aa403d560 2021-04-03T03:17:36.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/3037905622 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa403d560 msgr2=0x7f7aa40dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 --2- 172.21.2.111:0/3037905622 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa403d560 0x7f7aa40dbf80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f7aa4040a80 tx=0x7f7aac05ad70).stop 2021-04-03T03:17:36.018 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/3037905622 shutdown_connections 2021-04-03T03:17:36.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 --2- 172.21.2.111:0/3037905622 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa403d560 0x7f7aa40dbf80 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/3037905622 >> 172.21.2.111:0/3037905622 conn(0x7f7aa40462b0 msgr2=0x7f7aa40466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/3037905622 shutdown_connections 2021-04-03T03:17:36.019 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/3037905622 wait complete. 2021-04-03T03:17:36.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 Processor -- start 2021-04-03T03:17:36.020 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- start start 2021-04-03T03:17:36.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7aa403d560 0x7f7aa403ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7aa4041060 0x7f7aa40dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa40dbba0 0x7f7aa40dced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40dd3c0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa403bf50 con 0x7f7aa4041060 2021-04-03T03:17:36.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa403b5d0 con 0x7f7aa403d560 2021-04-03T03:17:36.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa40dbba0 0x7f7aa40dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa40dbba0 0x7f7aa40dced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59014/0 (socket says 172.21.2.111:59014) 2021-04-03T03:17:36.023 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 -- 172.21.2.111:0/2667494579 learned_addr learned my addr 172.21.2.111:0/2667494579 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 -- 172.21.2.111:0/2667494579 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7aa403d560 msgr2=0x7f7aa403ba60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7aa403d560 0x7f7aa403ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 -- 172.21.2.111:0/2667494579 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7aa4041060 msgr2=0x7f7aa40dc9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T03:17:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7aa4041060 0x7f7aa40dc9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.024 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 -- 172.21.2.111:0/2667494579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4039b40 con 0x7f7aa40dbba0 2021-04-03T03:17:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aab5e6700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa40dbba0 0x7f7aa40dced0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f7aa4040a80 tx=0x7f7aac064030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa3fff700 1 -- 172.21.2.111:0/2667494579 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aac0699f0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa3fff700 1 -- 172.21.2.111:0/2667494579 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac0441b0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.025 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa4039de0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa40d6360 con 0x7f7aa40dbba0 2021-04-03T03:17:36.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa3fff700 1 -- 172.21.2.111:0/2667494579 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aac05f4b0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa4004db0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa3fff700 1 -- 172.21.2.111:0/2667494579 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7aac08c020 con 0x7f7aa40dbba0 2021-04-03T03:17:36.026 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa3fff700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7a90057000 0x7f7a900594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.013+0000 7f7aa3fff700 1 -- 172.21.2.111:0/2667494579 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f7aac0e0dc0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.017+0000 7f7aaade5700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7a90057000 0x7f7a900594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.017+0000 7f7aaade5700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7a90057000 0x7f7a900594c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f7aa40087d0 tx=0x7f7a940077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.027 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.017+0000 7f7aa3fff700 1 -- 172.21.2.111:0/2667494579 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7aac0b60a0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.112 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.109+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7aa4004db0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.113 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.109+0000 7f7aa3fff700 1 -- 172.21.2.111:0/2667494579 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7aac0b96c0 con 0x7f7aa40dbba0 2021-04-03T03:17:36.113 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:36.114 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7a90057000 msgr2=0x7f7a900594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7a90057000 0x7f7a900594c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f7aa40087d0 tx=0x7f7a940077f0).stop 2021-04-03T03:17:36.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa40dbba0 msgr2=0x7f7aa40dced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa40dbba0 0x7f7aa40dced0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f7aa4040a80 tx=0x7f7aac064030).stop 2021-04-03T03:17:36.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 shutdown_connections 2021-04-03T03:17:36.115 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7aa40dbba0 0x7f7aa40dced0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7aa403d560 0x7f7aa403ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7a90057000 0x7f7a900594c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 --2- 172.21.2.111:0/2667494579 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7aa4041060 0x7f7aa40dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.116 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 >> 172.21.2.111:0/2667494579 conn(0x7f7aa40462b0 msgr2=0x7f7aa40061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 shutdown_connections 2021-04-03T03:17:36.117 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.113+0000 7f7aabde7700 1 -- 172.21.2.111:0/2667494579 wait complete. 2021-04-03T03:17:36.121 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:17:36.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 Processor -- start 2021-04-03T03:17:36.179 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- start start 2021-04-03T03:17:36.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdd500603f0 0x7fdd501447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.181 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500b17b0 0x7fdd50144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd500607d0 con 0x7fdd500add50 2021-04-03T03:17:36.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd501451c0 con 0x7fdd500603f0 2021-04-03T03:17:36.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5014b090 con 0x7fdd500b17b0 2021-04-03T03:17:36.182 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd558be700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fdd500603f0 0x7fdd501447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd550bd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fdd500add50 0x7fdd50147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59814/0 (socket says 172.21.2.111:59814) 2021-04-03T03:17:36.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd550bd700 1 -- 172.21.2.111:0/1124776370 learned_addr learned my addr 172.21.2.111:0/1124776370 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:36.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd560bf700 1 --2- 172.21.2.111:0/1124776370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500b17b0 0x7fdd50144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd558be700 1 -- 172.21.2.111:0/1124776370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500b17b0 msgr2=0x7fdd50144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:36.183 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd548bc700 1 -- 172.21.2.111:0/1124776370 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247322632 0 0) 0x7fdd500607d0 con 0x7fdd500add50 2021-04-03T03:17:36.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd558be700 1 --2- 172.21.2.111:0/1124776370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500b17b0 0x7fdd50144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd558be700 1 -- 172.21.2.111:0/1124776370 >> v1:172.21.2.111:6789/0 conn(0x7fdd500add50 legacy=0x7fdd50147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd558be700 1 -- 172.21.2.111:0/1124776370 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5014b330 con 0x7fdd500603f0 2021-04-03T03:17:36.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd560bf700 1 --2- 172.21.2.111:0/1124776370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500b17b0 0x7fdd50144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:36.184 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd558be700 1 --2- 172.21.2.111:0/1124776370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500603f0 0x7fdd501447e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fdd4c002f30 tx=0x7fdd4c006dd0).ready entity=mon.0 client_cookie=aab0e2c2db2e11a5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd548bc700 1 -- 172.21.2.111:0/1124776370 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd4c0097f0 con 0x7fdd500603f0 2021-04-03T03:17:36.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd548bc700 1 -- 172.21.2.111:0/1124776370 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c009950 con 0x7fdd500603f0 2021-04-03T03:17:36.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd548bc700 1 -- 172.21.2.111:0/1124776370 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd4c009cb0 con 0x7fdd500603f0 2021-04-03T03:17:36.185 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- 172.21.2.111:0/1124776370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500603f0 msgr2=0x7fdd501447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 --2- 172.21.2.111:0/1124776370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500603f0 0x7fdd501447e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fdd4c002f30 tx=0x7fdd4c006dd0).stop 2021-04-03T03:17:36.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- 172.21.2.111:0/1124776370 shutdown_connections 2021-04-03T03:17:36.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 --2- 172.21.2.111:0/1124776370 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500b17b0 0x7fdd50144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 --2- 172.21.2.111:0/1124776370 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500603f0 0x7fdd501447e0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- 172.21.2.111:0/1124776370 >> 172.21.2.111:0/1124776370 conn(0x7fdd500b61c0 msgr2=0x7fdd5005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- 172.21.2.111:0/1124776370 shutdown_connections 2021-04-03T03:17:36.187 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.177+0000 7fdd568c0700 1 -- 172.21.2.111:0/1124776370 wait complete. 2021-04-03T03:17:36.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 Processor -- start 2021-04-03T03:17:36.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 -- start start 2021-04-03T03:17:36.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500603f0 0x7fdd50148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd500add50 0x7fdd50145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500b17b0 0x7fdd50159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500b4320 con 0x7fdd500b17b0 2021-04-03T03:17:36.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50062fb0 con 0x7fdd500add50 2021-04-03T03:17:36.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50146720 con 0x7fdd500603f0 2021-04-03T03:17:36.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500b17b0 0x7fdd50159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500b17b0 0x7fdd50159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59032/0 (socket says 172.21.2.111:59032) 2021-04-03T03:17:36.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 -- 172.21.2.111:0/4021668650 learned_addr learned my addr 172.21.2.111:0/4021668650 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:36.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd550bd700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd500add50 0x7fdd50145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd558be700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500603f0 0x7fdd50148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 -- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500603f0 msgr2=0x7fdd50148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500603f0 0x7fdd50148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 -- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd500add50 msgr2=0x7fdd50145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd500add50 0x7fdd50145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 -- 172.21.2.111:0/4021668650 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50062b10 con 0x7fdd500b17b0 2021-04-03T03:17:36.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd558be700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500603f0 0x7fdd50148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T03:17:36.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd560bf700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500b17b0 0x7fdd50159e70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fdd48007000 tx=0x7fdd48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd3effd700 1 -- 172.21.2.111:0/4021668650 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd48007800 con 0x7fdd500b17b0 2021-04-03T03:17:36.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd3effd700 1 -- 172.21.2.111:0/4021668650 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48007960 con 0x7fdd500b17b0 2021-04-03T03:17:36.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd501469c0 con 0x7fdd500b17b0 2021-04-03T03:17:36.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd3effd700 1 -- 172.21.2.111:0/4021668650 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd48007cc0 con 0x7fdd500b17b0 2021-04-03T03:17:36.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd50145990 con 0x7fdd500b17b0 2021-04-03T03:17:36.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd50067980 con 0x7fdd500b17b0 2021-04-03T03:17:36.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd3effd700 1 -- 172.21.2.111:0/4021668650 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fdd4801b460 con 0x7fdd500b17b0 2021-04-03T03:17:36.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd3effd700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd30056fb0 0x7fdd30059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd3effd700 1 -- 172.21.2.111:0/4021668650 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fdd48070660 con 0x7fdd500b17b0 2021-04-03T03:17:36.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd558be700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd30056fb0 0x7fdd30059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.181+0000 7fdd558be700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd30056fb0 0x7fdd30059470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fdd4c017000 tx=0x7fdd4c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.185+0000 7fdd3effd700 1 -- 172.21.2.111:0/4021668650 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fdd480459f0 con 0x7fdd500b17b0 2021-04-03T03:17:36.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.277+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd50057660 con 0x7fdd500b17b0 2021-04-03T03:17:36.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.277+0000 7fdd3effd700 1 -- 172.21.2.111:0/4021668650 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdd48049010 con 0x7fdd500b17b0 2021-04-03T03:17:36.282 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:36.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd30056fb0 msgr2=0x7fdd30059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd30056fb0 0x7fdd30059470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fdd4c017000 tx=0x7fdd4c015000).stop 2021-04-03T03:17:36.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500b17b0 msgr2=0x7fdd50159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500b17b0 0x7fdd50159e70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fdd48007000 tx=0x7fdd48008040).stop 2021-04-03T03:17:36.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 shutdown_connections 2021-04-03T03:17:36.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fdd500b17b0 0x7fdd50159e70 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.284 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fdd500603f0 0x7fdd50148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fdd30056fb0 0x7fdd30059470 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 --2- 172.21.2.111:0/4021668650 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fdd500add50 0x7fdd50145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 >> 172.21.2.111:0/4021668650 conn(0x7fdd500b61c0 msgr2=0x7fdd500b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 shutdown_connections 2021-04-03T03:17:36.285 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.281+0000 7fdd568c0700 1 -- 172.21.2.111:0/4021668650 wait complete. 2021-04-03T03:17:36.290 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:17:36.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 Processor -- start 2021-04-03T03:17:36.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 -- start start 2021-04-03T03:17:36.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0d340423a0 0x7f0d3403c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340d38b0 0x7f0d340cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d34043400 con 0x7f0d34047580 2021-04-03T03:17:36.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d34047960 con 0x7f0d340423a0 2021-04-03T03:17:36.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34042ec0 con 0x7f0d340d38b0 2021-04-03T03:17:36.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0d34047580 0x7f0d3403bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59828/0 (socket says 172.21.2.111:59828) 2021-04-03T03:17:36.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 -- 172.21.2.111:0/2336209450 learned_addr learned my addr 172.21.2.111:0/2336209450 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:36.356 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220424878 0 0) 0x7f0d34043400 con 0x7f0d34047580 2021-04-03T03:17:36.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d28003240 con 0x7f0d34047580 2021-04-03T03:17:36.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3879817813 0 0) 0x7f0d28003240 con 0x7f0d34047580 2021-04-03T03:17:36.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340d38b0 msgr2=0x7f0d340cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:36.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340d38b0 0x7f0d340cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.357 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 >> v2:172.21.2.115:3300/0 conn(0x7f0d340423a0 msgr2=0x7f0d3403c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:36.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 --2- 172.21.2.111:0/2336209450 >> v2:172.21.2.115:3300/0 conn(0x7f0d340423a0 0x7f0d3403c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d34043160 con 0x7f0d34047580 2021-04-03T03:17:36.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f0d2c0016d0 con 0x7f0d34047580 2021-04-03T03:17:36.358 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 >> v1:172.21.2.111:6789/0 conn(0x7f0d34047580 legacy=0x7f0d3403bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d280037b0 0x7f0d28003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d280044e0 0x7f0d28004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340d38b0 0x7f0d28004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34047960 con 0x7f0d280037b0 2021-04-03T03:17:36.359 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34042ec0 con 0x7f0d280044e0 2021-04-03T03:17:36.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d28005340 con 0x7f0d340d38b0 2021-04-03T03:17:36.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d280037b0 0x7f0d28003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 -- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340d38b0 msgr2=0x7f0d28004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:36.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340d38b0 0x7f0d28004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.360 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 -- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d280044e0 msgr2=0x7f0d28004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:36.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d280044e0 0x7f0d28004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 -- 172.21.2.111:0/2336209450 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28003240 con 0x7f0d280037b0 2021-04-03T03:17:36.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3b7fe700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d280037b0 0x7f0d28003bd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f0d340d0310 tx=0x7f0d340432f0).ready entity=mon.0 client_cookie=13235f4321f927a6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.361 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2c0070d0 con 0x7f0d280037b0 2021-04-03T03:17:36.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c007230 con 0x7f0d280037b0 2021-04-03T03:17:36.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3a7fc700 1 -- 172.21.2.111:0/2336209450 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2c0055f0 con 0x7f0d280037b0 2021-04-03T03:17:36.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 -- 172.21.2.111:0/2336209450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d280037b0 msgr2=0x7f0d28003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d4082e700 1 --2- 172.21.2.111:0/2336209450 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d280037b0 0x7f0d28003bd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f0d340d0310 tx=0x7f0d340432f0).stop 2021-04-03T03:17:36.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.349+0000 7f0d3bfff700 1 -- 172.21.2.111:0/2336209450 reap_dead start 2021-04-03T03:17:36.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- 172.21.2.111:0/2336209450 shutdown_connections 2021-04-03T03:17:36.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- 172.21.2.111:0/2336209450 >> 172.21.2.111:0/2336209450 conn(0x7f0d3400d9a0 msgr2=0x7f0d34046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- 172.21.2.111:0/2336209450 shutdown_connections 2021-04-03T03:17:36.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- 172.21.2.111:0/2336209450 wait complete. 2021-04-03T03:17:36.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 Processor -- start 2021-04-03T03:17:36.364 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- start start 2021-04-03T03:17:36.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340ccf20 0x7f0d340cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d340d1cd0 0x7f0d340d3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d340de460 0x7f0d340de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340d2990 con 0x7f0d340de460 2021-04-03T03:17:36.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3403c9b0 con 0x7f0d340d1cd0 2021-04-03T03:17:36.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340cd300 con 0x7f0d340ccf20 2021-04-03T03:17:36.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d340de460 0x7f0d340de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d340de460 0x7f0d340de8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59052/0 (socket says 172.21.2.111:59052) 2021-04-03T03:17:36.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 -- 172.21.2.111:0/1441220787 learned_addr learned my addr 172.21.2.111:0/1441220787 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:36.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 -- 172.21.2.111:0/1441220787 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340ccf20 msgr2=0x7f0d340cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340ccf20 0x7f0d340cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 -- 172.21.2.111:0/1441220787 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d340d1cd0 msgr2=0x7f0d340d3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d340d1cd0 0x7f0d340d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 -- 172.21.2.111:0/1441220787 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340d2250 con 0x7f0d340de460 2021-04-03T03:17:36.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3bfff700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d340de460 0x7f0d340de8e0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f0d2c0018b0 tx=0x7f0d3c05ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d38ff9700 1 -- 172.21.2.111:0/1441220787 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d3c0699f0 con 0x7f0d340de460 2021-04-03T03:17:36.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d38ff9700 1 -- 172.21.2.111:0/1441220787 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c05b040 con 0x7f0d340de460 2021-04-03T03:17:36.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d340d2bc0 con 0x7f0d340de460 2021-04-03T03:17:36.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d38ff9700 1 -- 172.21.2.111:0/1441220787 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d3c06a8a0 con 0x7f0d340de460 2021-04-03T03:17:36.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d340b8ed0 con 0x7f0d340de460 2021-04-03T03:17:36.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d340d2d50 con 0x7f0d340de460 2021-04-03T03:17:36.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d38ff9700 1 -- 172.21.2.111:0/1441220787 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0d3c061070 con 0x7f0d340de460 2021-04-03T03:17:36.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d38ff9700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d1c056f20 0x7f0d1c0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d38ff9700 1 -- 172.21.2.111:0/1441220787 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f0d3c0de780 con 0x7f0d340de460 2021-04-03T03:17:36.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3b7fe700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d1c056f20 0x7f0d1c0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.353+0000 7f0d3b7fe700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d1c056f20 0x7f0d1c0593e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f0d340d0310 tx=0x7f0d2c013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.357+0000 7f0d38ff9700 1 -- 172.21.2.111:0/1441220787 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0d3c0b3b70 con 0x7f0d340de460 2021-04-03T03:17:36.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d34004db0 con 0x7f0d340de460 2021-04-03T03:17:36.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d38ff9700 1 -- 172.21.2.111:0/1441220787 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0d3c0b7190 con 0x7f0d340de460 2021-04-03T03:17:36.455 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:36.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d1c056f20 msgr2=0x7f0d1c0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d1c056f20 0x7f0d1c0593e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f0d340d0310 tx=0x7f0d2c013000).stop 2021-04-03T03:17:36.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d340de460 msgr2=0x7f0d340de8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d340de460 0x7f0d340de8e0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f0d2c0018b0 tx=0x7f0d3c05ad70).stop 2021-04-03T03:17:36.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 shutdown_connections 2021-04-03T03:17:36.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0d340de460 0x7f0d340de8e0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0d340ccf20 0x7f0d340cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0d1c056f20 0x7f0d1c0593e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 --2- 172.21.2.111:0/1441220787 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0d340d1cd0 0x7f0d340d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 >> 172.21.2.111:0/1441220787 conn(0x7f0d3400d9a0 msgr2=0x7f0d34046b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 shutdown_connections 2021-04-03T03:17:36.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.453+0000 7f0d4082e700 1 -- 172.21.2.111:0/1441220787 wait complete. 2021-04-03T03:17:36.463 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:17:36.463 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:36.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.517+0000 7f5af32c5700 1 Processor -- start 2021-04-03T03:17:36.522 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- start start 2021-04-03T03:17:36.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5aec0603f0 0x7f5aec0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0af400 con 0x7f5aec14b700 2021-04-03T03:17:36.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0af6a0 con 0x7f5aec0b0c90 2021-04-03T03:17:36.524 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec060860 con 0x7f5aec0603f0 2021-04-03T03:17:36.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af22c3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f5aec0603f0 0x7f5aec0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af1ac2700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f5aec14b700 0x7f5aec0b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59850/0 (socket says 172.21.2.111:59850) 2021-04-03T03:17:36.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af1ac2700 1 -- 172.21.2.111:0/1896050378 learned_addr learned my addr 172.21.2.111:0/1896050378 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:36.525 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af12c1700 1 -- 172.21.2.111:0/1896050378 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910160967 0 0) 0x7f5aec0af400 con 0x7f5aec14b700 2021-04-03T03:17:36.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af12c1700 1 -- 172.21.2.111:0/1896050378 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad4003240 con 0x7f5aec14b700 2021-04-03T03:17:36.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af22c3700 1 -- 172.21.2.111:0/1896050378 >> v1:172.21.2.115:6789/0 conn(0x7f5aec0b0c90 legacy=0x7f5aec147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:36.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af22c3700 1 -- 172.21.2.111:0/1896050378 >> v1:172.21.2.111:6789/0 conn(0x7f5aec14b700 legacy=0x7f5aec0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af22c3700 1 -- 172.21.2.111:0/1896050378 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0afb40 con 0x7f5aec0603f0 2021-04-03T03:17:36.526 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af22c3700 1 --2- 172.21.2.111:0/1896050378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec0603f0 0x7f5aec0b0090 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f5aec0620f0 tx=0x7f5ae8006dd0).ready entity=mon.0 client_cookie=ce9173a2243ab734 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af12c1700 1 -- 172.21.2.111:0/1896050378 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ae8009770 con 0x7f5aec0603f0 2021-04-03T03:17:36.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af12c1700 1 -- 172.21.2.111:0/1896050378 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae80098d0 con 0x7f5aec0603f0 2021-04-03T03:17:36.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af12c1700 1 -- 172.21.2.111:0/1896050378 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ae8009c70 con 0x7f5aec0603f0 2021-04-03T03:17:36.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- 172.21.2.111:0/1896050378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec0603f0 msgr2=0x7f5aec0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.527 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 --2- 172.21.2.111:0/1896050378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec0603f0 0x7f5aec0b0090 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f5aec0620f0 tx=0x7f5ae8006dd0).stop 2021-04-03T03:17:36.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- 172.21.2.111:0/1896050378 shutdown_connections 2021-04-03T03:17:36.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 --2- 172.21.2.111:0/1896050378 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec0603f0 0x7f5aec0b0090 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.528 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- 172.21.2.111:0/1896050378 >> 172.21.2.111:0/1896050378 conn(0x7f5aec0b61c0 msgr2=0x7f5aec05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- 172.21.2.111:0/1896050378 shutdown_connections 2021-04-03T03:17:36.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- 172.21.2.111:0/1896050378 wait complete. 2021-04-03T03:17:36.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 Processor -- start 2021-04-03T03:17:36.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- start start 2021-04-03T03:17:36.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aec0603f0 0x7f5aec156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aec0b0c90 0x7f5aec062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec14b700 0x7f5aec159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec065430 con 0x7f5aec14b700 2021-04-03T03:17:36.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec065690 con 0x7f5aec0b0c90 2021-04-03T03:17:36.532 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af32c5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec159aa0 con 0x7f5aec0603f0 2021-04-03T03:17:36.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af2ac4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec14b700 0x7f5aec159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af2ac4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec14b700 0x7f5aec159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59066/0 (socket says 172.21.2.111:59066) 2021-04-03T03:17:36.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af2ac4700 1 -- 172.21.2.111:0/929054490 learned_addr learned my addr 172.21.2.111:0/929054490 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:36.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af2ac4700 1 -- 172.21.2.111:0/929054490 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aec0603f0 msgr2=0x7f5aec156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.533 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af2ac4700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aec0603f0 0x7f5aec156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af2ac4700 1 -- 172.21.2.111:0/929054490 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aec0b0c90 msgr2=0x7f5aec062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.521+0000 7f5af2ac4700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aec0b0c90 0x7f5aec062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5af2ac4700 1 -- 172.21.2.111:0/929054490 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0659b0 con 0x7f5aec14b700 2021-04-03T03:17:36.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5af2ac4700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec14b700 0x7f5aec159f40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f5ae40032d0 tx=0x7f5ae4006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.534 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5adf7fe700 1 -- 172.21.2.111:0/929054490 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ae400c960 con 0x7f5aec14b700 2021-04-03T03:17:36.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5adf7fe700 1 -- 172.21.2.111:0/929054490 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae400cac0 con 0x7f5aec14b700 2021-04-03T03:17:36.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec067530 con 0x7f5aec14b700 2021-04-03T03:17:36.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec065f50 con 0x7f5aec14b700 2021-04-03T03:17:36.535 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5adf7fe700 1 -- 172.21.2.111:0/929054490 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ae400ce20 con 0x7f5aec14b700 2021-04-03T03:17:36.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec067810 con 0x7f5aec14b700 2021-04-03T03:17:36.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5adf7fe700 1 -- 172.21.2.111:0/929054490 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f5ae401c070 con 0x7f5aec14b700 2021-04-03T03:17:36.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5adf7fe700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5acc056ef0 0x7f5acc0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:36.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5adf7fe700 1 -- 172.21.2.111:0/929054490 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f5ae406fe70 con 0x7f5aec14b700 2021-04-03T03:17:36.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5af22c3700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5acc056ef0 0x7f5acc0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:36.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.525+0000 7f5af22c3700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5acc056ef0 0x7f5acc0593b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f5aec0620f0 tx=0x7f5ae8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:36.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.529+0000 7f5adf7fe700 1 -- 172.21.2.111:0/929054490 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f5ae4045150 con 0x7f5aec14b700 2021-04-03T03:17:36.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aec057660 con 0x7f5aec14b700 2021-04-03T03:17:36.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5adf7fe700 1 -- 172.21.2.111:0/929054490 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f5ae4048770 con 0x7f5aec14b700 2021-04-03T03:17:36.652 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:36.652 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":68,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3455926982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3455926982},{"type":"v1","addr":"172.21.2.115:6833","nonce":3455926982}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:33.145029+0000","modified":"2021-04-03T03:17:34.147803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/295340141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":295340141},{"type":"v1","addr":"172.21.2.111:6835","nonce":295340141}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-04-03T03:17:36.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5acc056ef0 msgr2=0x7f5acc0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5acc056ef0 0x7f5acc0593b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f5aec0620f0 tx=0x7f5ae8015000).stop 2021-04-03T03:17:36.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec14b700 msgr2=0x7f5aec159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:36.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec14b700 0x7f5aec159f40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f5ae40032d0 tx=0x7f5ae4006f90).stop 2021-04-03T03:17:36.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 shutdown_connections 2021-04-03T03:17:36.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f5aec14b700 0x7f5aec159f40 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f5aec0603f0 0x7f5aec156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5acc056ef0 0x7f5acc0593b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:36.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 --2- 172.21.2.111:0/929054490 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5aec0b0c90 0x7f5aec062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:36.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 >> 172.21.2.111:0/929054490 conn(0x7f5aec0b61c0 msgr2=0x7f5aec147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:36.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 shutdown_connections 2021-04-03T03:17:36.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:36.649+0000 7f5af32c5700 1 -- 172.21.2.111:0/929054490 wait complete. 2021-04-03T03:17:36.656 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 68 2021-04-03T03:17:36.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:17:33.145029+0000', 'modified': '2021-04-03T03:17:34.147803+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8526': {'gid': 8526, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/295340141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 295340141}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 295340141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:17:36.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:17:36.862 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:17:36.932 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":8526,"want_state":"up:active","state":"up:active","rank_uptime":3.7804075410000002,"mdsmap_epoch":68,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":7.9268810280000004} 2021-04-03T03:17:36.933 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 8526, "mdsmap_epoch": 68, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 3.780407541, "state": "up:active", "uptime": 7.926881028, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:17:36.933 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:17:36.933 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-04-03T03:17:36.934 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba026.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-04-03T03:17:36.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:36.935 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:36.945 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:17:36.998 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:36.998 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:37.046 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:37.047 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:37.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:37.048 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:17:37.099 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:17:37.100 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:37.146 DEBUG:teuthology.orchestra.run.gibba026:> sudo modprobe fuse 2021-04-03T03:17:37.153 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:37.153 DEBUG:teuthology.orchestra.run.gibba026:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:37.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[16195]: starting ceph client2021-04-03T03:17:37.181+0000 7fcbee2280c0 -1 init, newargv = 0x55ee38183aa0 newargc=9 2021-04-03T03:17:37.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr: 2021-04-03T03:17:37.191 INFO:teuthology.orchestra.run.gibba026.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:37.192 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:37.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:37.193 DEBUG:teuthology.orchestra.run.gibba026:> ls /sys/fs/fuse/connections 2021-04-03T03:17:37.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba026.stderr:ceph-fuse[16195]: starting fuse 2021-04-03T03:17:37.202 INFO:teuthology.orchestra.run.gibba026.stdout:49 2021-04-03T03:17:37.203 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:17:37.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:37.203 DEBUG:teuthology.orchestra.run.gibba026:> sudo python3 -c ' 2021-04-03T03:17:37.204 DEBUG:teuthology.orchestra.run.gibba026:> import glob 2021-04-03T03:17:37.204 DEBUG:teuthology.orchestra.run.gibba026:> import re 2021-04-03T03:17:37.205 DEBUG:teuthology.orchestra.run.gibba026:> import os 2021-04-03T03:17:37.205 DEBUG:teuthology.orchestra.run.gibba026:> import subprocess 2021-04-03T03:17:37.205 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:37.206 DEBUG:teuthology.orchestra.run.gibba026:> def find_socket(client_name): 2021-04-03T03:17:37.206 DEBUG:teuthology.orchestra.run.gibba026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-04-03T03:17:37.207 DEBUG:teuthology.orchestra.run.gibba026:> files = glob.glob(asok_path) 2021-04-03T03:17:37.207 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:37.207 DEBUG:teuthology.orchestra.run.gibba026:> # Given a non-glob path, it better be there 2021-04-03T03:17:37.208 DEBUG:teuthology.orchestra.run.gibba026:> if "*" not in asok_path: 2021-04-03T03:17:37.208 DEBUG:teuthology.orchestra.run.gibba026:> assert(len(files) == 1) 2021-04-03T03:17:37.209 DEBUG:teuthology.orchestra.run.gibba026:> return files[0] 2021-04-03T03:17:37.209 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:37.209 DEBUG:teuthology.orchestra.run.gibba026:> for f in files: 2021-04-03T03:17:37.210 DEBUG:teuthology.orchestra.run.gibba026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:17:37.210 DEBUG:teuthology.orchestra.run.gibba026:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:17:37.210 DEBUG:teuthology.orchestra.run.gibba026:> return f 2021-04-03T03:17:37.211 DEBUG:teuthology.orchestra.run.gibba026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:17:37.211 DEBUG:teuthology.orchestra.run.gibba026:> 2021-04-03T03:17:37.212 DEBUG:teuthology.orchestra.run.gibba026:> print(find_socket("client.0")) 2021-04-03T03:17:37.212 DEBUG:teuthology.orchestra.run.gibba026:> ' 2021-04-03T03:17:37.293 INFO:teuthology.orchestra.run.gibba026.stdout:/var/run/ceph/ceph-client.0.16195.asok 2021-04-03T03:17:37.297 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.16195.asok 2021-04-03T03:17:37.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:37.298 DEBUG:teuthology.orchestra.run.gibba026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.16195.asok status 2021-04-03T03:17:37.355 INFO:teuthology.orchestra.run.gibba026.stdout:{ 2021-04-03T03:17:37.355 INFO:teuthology.orchestra.run.gibba026.stdout: "metadata": { 2021-04-03T03:17:37.356 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:17:37.356 INFO:teuthology.orchestra.run.gibba026.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:17:37.356 INFO:teuthology.orchestra.run.gibba026.stdout: "entity_id": "0", 2021-04-03T03:17:37.356 INFO:teuthology.orchestra.run.gibba026.stdout: "hostname": "gibba026", 2021-04-03T03:17:37.356 INFO:teuthology.orchestra.run.gibba026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-04-03T03:17:37.357 INFO:teuthology.orchestra.run.gibba026.stdout: "pid": "16195", 2021-04-03T03:17:37.357 INFO:teuthology.orchestra.run.gibba026.stdout: "root": "/" 2021-04-03T03:17:37.357 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:37.357 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_count": 0, 2021-04-03T03:17:37.357 INFO:teuthology.orchestra.run.gibba026.stdout: "dentry_pinned_count": 0, 2021-04-03T03:17:37.358 INFO:teuthology.orchestra.run.gibba026.stdout: "id": 5297, 2021-04-03T03:17:37.358 INFO:teuthology.orchestra.run.gibba026.stdout: "inst": { 2021-04-03T03:17:37.358 INFO:teuthology.orchestra.run.gibba026.stdout: "name": { 2021-04-03T03:17:37.358 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "client", 2021-04-03T03:17:37.359 INFO:teuthology.orchestra.run.gibba026.stdout: "num": 5297 2021-04-03T03:17:37.359 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:37.359 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:17:37.359 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:17:37.359 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:17:37.360 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3153674520 2021-04-03T03:17:37.360 INFO:teuthology.orchestra.run.gibba026.stdout: } 2021-04-03T03:17:37.360 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:37.360 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": { 2021-04-03T03:17:37.360 INFO:teuthology.orchestra.run.gibba026.stdout: "type": "v1", 2021-04-03T03:17:37.361 INFO:teuthology.orchestra.run.gibba026.stdout: "addr": "172.21.2.126:0", 2021-04-03T03:17:37.361 INFO:teuthology.orchestra.run.gibba026.stdout: "nonce": 3153674520 2021-04-03T03:17:37.361 INFO:teuthology.orchestra.run.gibba026.stdout: }, 2021-04-03T03:17:37.361 INFO:teuthology.orchestra.run.gibba026.stdout: "inst_str": "client.5297 172.21.2.126:0/3153674520", 2021-04-03T03:17:37.362 INFO:teuthology.orchestra.run.gibba026.stdout: "addr_str": "172.21.2.126:0/3153674520", 2021-04-03T03:17:37.362 INFO:teuthology.orchestra.run.gibba026.stdout: "inode_count": 1, 2021-04-03T03:17:37.362 INFO:teuthology.orchestra.run.gibba026.stdout: "mds_epoch": 68, 2021-04-03T03:17:37.362 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch": 151, 2021-04-03T03:17:37.362 INFO:teuthology.orchestra.run.gibba026.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:17:37.363 INFO:teuthology.orchestra.run.gibba026.stdout: "blacklisted": false, 2021-04-03T03:17:37.363 INFO:teuthology.orchestra.run.gibba026.stdout: "fs_name": "cephfs" 2021-04-03T03:17:37.363 INFO:teuthology.orchestra.run.gibba026.stdout:} 2021-04-03T03:17:37.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:37.365 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:37.365 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:37.369 INFO:teuthology.orchestra.run.gibba026.stdout:fuseblk 2021-04-03T03:17:37.370 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-04-03T03:17:37.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:37.371 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:17:37.420 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T03:17:37.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5839b0e700 1 Processor -- start 2021-04-03T03:17:37.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5839b0e700 1 -- start start 2021-04-03T03:17:37.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5839b0e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 0x7f58340b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5839b0e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f58340b0c90 0x7f5834147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5839b0e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340607d0 con 0x7f58340603f0 2021-04-03T03:17:37.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5839b0e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340af400 con 0x7f58340b0c90 2021-04-03T03:17:37.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5839b0e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340af6a0 con 0x7f583414b700 2021-04-03T03:17:37.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5838b0c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f58340603f0 0x7f58340b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59862/0 (socket says 172.21.2.111:59862) 2021-04-03T03:17:37.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f583930d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f58340b0c90 0x7f5834147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.477+0000 7f5838b0c700 1 -- 172.21.2.111:0/1471825582 learned_addr learned my addr 172.21.2.111:0/1471825582 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:37.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f58337fe700 1 -- 172.21.2.111:0/1471825582 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223430752 0 0) 0x7f58340607d0 con 0x7f58340603f0 2021-04-03T03:17:37.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5833fff700 1 --2- 172.21.2.111:0/1471825582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 0x7f58340b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f58337fe700 1 -- 172.21.2.111:0/1471825582 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5814003240 con 0x7f58340603f0 2021-04-03T03:17:37.486 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f583930d700 1 -- 172.21.2.111:0/1471825582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 msgr2=0x7f58340b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:37.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f583930d700 1 --2- 172.21.2.111:0/1471825582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 0x7f58340b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f583930d700 1 -- 172.21.2.111:0/1471825582 >> v1:172.21.2.111:6789/0 conn(0x7f58340603f0 legacy=0x7f58340b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f583930d700 1 -- 172.21.2.111:0/1471825582 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340afb40 con 0x7f58340b0c90 2021-04-03T03:17:37.487 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f583930d700 1 --2- 172.21.2.111:0/1471825582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340b0c90 0x7f5834147680 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f58240032b0 tx=0x7f58240091d0).ready entity=mon.0 client_cookie=10fbb8ebb86449a5 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f58337fe700 1 -- 172.21.2.111:0/1471825582 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f582400b9a0 con 0x7f58340b0c90 2021-04-03T03:17:37.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f58337fe700 1 -- 172.21.2.111:0/1471825582 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582400bb00 con 0x7f58340b0c90 2021-04-03T03:17:37.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f58337fe700 1 -- 172.21.2.111:0/1471825582 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f582400bc70 con 0x7f58340b0c90 2021-04-03T03:17:37.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- 172.21.2.111:0/1471825582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340b0c90 msgr2=0x7f5834147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.488 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1471825582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340b0c90 0x7f5834147680 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f58240032b0 tx=0x7f58240091d0).stop 2021-04-03T03:17:37.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- 172.21.2.111:0/1471825582 shutdown_connections 2021-04-03T03:17:37.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1471825582 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340b0c90 0x7f5834147680 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1471825582 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 0x7f58340b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- 172.21.2.111:0/1471825582 >> 172.21.2.111:0/1471825582 conn(0x7f58340b61c0 msgr2=0x7f583405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:37.489 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- 172.21.2.111:0/1471825582 shutdown_connections 2021-04-03T03:17:37.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- 172.21.2.111:0/1471825582 wait complete. 2021-04-03T03:17:37.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 Processor -- start 2021-04-03T03:17:37.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- start start 2021-04-03T03:17:37.490 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340603f0 0x7f5834157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f58340b0c90 0x7f58340669e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 0x7f5834066ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583415a3d0 con 0x7f58340603f0 2021-04-03T03:17:37.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583415a670 con 0x7f58340b0c90 2021-04-03T03:17:37.491 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5834157fc0 con 0x7f583414b700 2021-04-03T03:17:37.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340603f0 0x7f5834157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340603f0 0x7f5834157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59076/0 (socket says 172.21.2.111:59076) 2021-04-03T03:17:37.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 -- 172.21.2.111:0/1172416713 learned_addr learned my addr 172.21.2.111:0/1172416713 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:37.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 -- 172.21.2.111:0/1172416713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 msgr2=0x7f5834066ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.492 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 0x7f5834066ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 -- 172.21.2.111:0/1172416713 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f58340b0c90 msgr2=0x7f58340669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f58340b0c90 0x7f58340669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 -- 172.21.2.111:0/1172416713 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834065580 con 0x7f58340603f0 2021-04-03T03:17:37.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5838b0c700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340603f0 0x7f5834157ad0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f581c0010d0 tx=0x7f581c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.493 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5831ffb700 1 -- 172.21.2.111:0/1172416713 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f581c0072e0 con 0x7f58340603f0 2021-04-03T03:17:37.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5831ffb700 1 -- 172.21.2.111:0/1172416713 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c007440 con 0x7f58340603f0 2021-04-03T03:17:37.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5834158490 con 0x7f58340603f0 2021-04-03T03:17:37.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5831ffb700 1 -- 172.21.2.111:0/1172416713 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f581c0057e0 con 0x7f58340603f0 2021-04-03T03:17:37.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.481+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58340673c0 con 0x7f58340603f0 2021-04-03T03:17:37.494 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.485+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583415a240 con 0x7f58340603f0 2021-04-03T03:17:37.495 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.485+0000 7f5831ffb700 1 -- 172.21.2.111:0/1172416713 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f581c010070 con 0x7f58340603f0 2021-04-03T03:17:37.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.485+0000 7f5831ffb700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056ef0 0x7f58180593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.485+0000 7f5831ffb700 1 -- 172.21.2.111:0/1172416713 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f581c06fa10 con 0x7f58340603f0 2021-04-03T03:17:37.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.485+0000 7f5833fff700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056ef0 0x7f58180593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.496 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.485+0000 7f5833fff700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056ef0 0x7f58180593b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f582c007000 tx=0x7f582c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.497 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.485+0000 7f5831ffb700 1 -- 172.21.2.111:0/1172416713 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f581c044cf0 con 0x7f58340603f0 2021-04-03T03:17:37.581 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.577+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5834158180 con 0x7f58340603f0 2021-04-03T03:17:37.582 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5831ffb700 1 -- 172.21.2.111:0/1172416713 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f581c048310 con 0x7f58340603f0 2021-04-03T03:17:37.582 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:37.583 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":151,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:34.148833+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-04-03T03:17:31.174355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-04-03T03:17:32.113426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T03:17:37.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056ef0 msgr2=0x7f58180593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056ef0 0x7f58180593b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f582c007000 tx=0x7f582c008040).stop 2021-04-03T03:17:37.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340603f0 msgr2=0x7f5834157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340603f0 0x7f5834157ad0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f581c0010d0 tx=0x7f581c007a80).stop 2021-04-03T03:17:37.584 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 shutdown_connections 2021-04-03T03:17:37.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583414b700 0x7f5834066ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58340603f0 0x7f5834157ad0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056ef0 0x7f58180593b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.585 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 --2- 172.21.2.111:0/1172416713 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f58340b0c90 0x7f58340669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 >> 172.21.2.111:0/1172416713 conn(0x7f58340b61c0 msgr2=0x7f58340b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:37.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 shutdown_connections 2021-04-03T03:17:37.586 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.581+0000 7f5839b0e700 1 -- 172.21.2.111:0/1172416713 wait complete. 2021-04-03T03:17:37.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-04-03T03:08:29.718060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-04-03T03:08:38.084175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2021-04-03T03:17:31.174355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2021-04-03T03:17:32.113426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-04-03T03:17:37.591 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2021-04-03T03:17:37.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.649+0000 7fe53b59e700 1 Processor -- start 2021-04-03T03:17:37.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.649+0000 7fe53b59e700 1 -- start start 2021-04-03T03:17:37.654 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.649+0000 7fe53b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5340dbba0 0x7fe5340dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.649+0000 7fe53b59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe53403d560 0x7fe534041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.649+0000 7fe53b59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5340dc560 con 0x7fe534041060 2021-04-03T03:17:37.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.649+0000 7fe53b59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53403de60 con 0x7fe53403d560 2021-04-03T03:17:37.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.649+0000 7fe53b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340dc800 con 0x7fe5340dbba0 2021-04-03T03:17:37.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe53403d560 0x7fe534041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe53403d560 0x7fe534041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59086/0 (socket says 172.21.2.111:59086) 2021-04-03T03:17:37.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/839654105 learned_addr learned my addr 172.21.2.111:0/839654105 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:37.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe539d9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe534041060 0x7fe53403fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59878/0 (socket says 172.21.2.111:59878) 2021-04-03T03:17:37.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/839654105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5340dbba0 msgr2=0x7fe5340dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:37.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- 172.21.2.111:0/839654105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5340dbba0 0x7fe5340dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/839654105 >> v1:172.21.2.111:6789/0 conn(0x7fe534041060 legacy=0x7fe53403fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/839654105 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53403da30 con 0x7fe53403d560 2021-04-03T03:17:37.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- 172.21.2.111:0/839654105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe53403d560 0x7fe534041650 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7fe5240010d0 tx=0x7fe53c05ad70).ready entity=mon.0 client_cookie=29a6c3846041ffd3 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53959a700 1 -- 172.21.2.111:0/839654105 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c05bda0 con 0x7fe53403d560 2021-04-03T03:17:37.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53959a700 1 -- 172.21.2.111:0/839654105 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c05df60 con 0x7fe53403d560 2021-04-03T03:17:37.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53959a700 1 -- 172.21.2.111:0/839654105 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c0699f0 con 0x7fe53403d560 2021-04-03T03:17:37.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/839654105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe53403d560 msgr2=0x7fe534041650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 --2- 172.21.2.111:0/839654105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe53403d560 0x7fe534041650 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7fe5240010d0 tx=0x7fe53c05ad70).stop 2021-04-03T03:17:37.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/839654105 shutdown_connections 2021-04-03T03:17:37.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 --2- 172.21.2.111:0/839654105 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe53403d560 0x7fe534041650 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 --2- 172.21.2.111:0/839654105 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe5340dbba0 0x7fe5340dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/839654105 >> 172.21.2.111:0/839654105 conn(0x7fe5340462b0 msgr2=0x7fe5340466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:37.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/839654105 shutdown_connections 2021-04-03T03:17:37.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/839654105 wait complete. 2021-04-03T03:17:37.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 Processor -- start 2021-04-03T03:17:37.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- start start 2021-04-03T03:17:37.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe53403d560 0x7fe5340a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe534041060 0x7fe5340ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5340dbba0 0x7fe5340a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53409f820 con 0x7fe5340dbba0 2021-04-03T03:17:37.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340d7cc0 con 0x7fe53403d560 2021-04-03T03:17:37.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340e7a70 con 0x7fe534041060 2021-04-03T03:17:37.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5340dbba0 0x7fe5340a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5340dbba0 0x7fe5340a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59092/0 (socket says 172.21.2.111:59092) 2021-04-03T03:17:37.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/895443981 learned_addr learned my addr 172.21.2.111:0/895443981 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:37.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/895443981 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe534041060 msgr2=0x7fe5340ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe534041060 0x7fe5340ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/895443981 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe53403d560 msgr2=0x7fe5340a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.665 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe53403d560 0x7fe5340a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 -- 172.21.2.111:0/895443981 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340e7d10 con 0x7fe5340dbba0 2021-04-03T03:17:37.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53ad9d700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5340dbba0 0x7fe5340a09c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7fe5240010d0 tx=0x7fe53c065000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe5337fe700 1 -- 172.21.2.111:0/895443981 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c0441b0 con 0x7fe5340dbba0 2021-04-03T03:17:37.666 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe5337fe700 1 -- 172.21.2.111:0/895443981 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c05f2c0 con 0x7fe5340dbba0 2021-04-03T03:17:37.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5340ac5b0 con 0x7fe5340dbba0 2021-04-03T03:17:37.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe5337fe700 1 -- 172.21.2.111:0/895443981 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c05f620 con 0x7fe5340dbba0 2021-04-03T03:17:37.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5340a0eb0 con 0x7fe5340dbba0 2021-04-03T03:17:37.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.653+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe534004db0 con 0x7fe5340dbba0 2021-04-03T03:17:37.667 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.657+0000 7fe5337fe700 1 -- 172.21.2.111:0/895443981 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe53c06adb0 con 0x7fe5340dbba0 2021-04-03T03:17:37.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.657+0000 7fe5337fe700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe520056ea0 0x7fe520059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.657+0000 7fe5337fe700 1 -- 172.21.2.111:0/895443981 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fe53c0e15e0 con 0x7fe5340dbba0 2021-04-03T03:17:37.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.657+0000 7fe53a59c700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe520056ea0 0x7fe520059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.657+0000 7fe53a59c700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe520056ea0 0x7fe520059360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fe52c002f30 tx=0x7fe52c0069f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.668 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.657+0000 7fe5337fe700 1 -- 172.21.2.111:0/895443981 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe53c0b6970 con 0x7fe5340dbba0 2021-04-03T03:17:37.752 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe5340083c0 con 0x7fe5340dbba0 2021-04-03T03:17:37.753 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe5337fe700 1 -- 172.21.2.111:0/895443981 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe53c0b9f90 con 0x7fe5340dbba0 2021-04-03T03:17:37.753 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 1 2021-04-03T03:17:37.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe520056ea0 msgr2=0x7fe520059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.754 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe520056ea0 0x7fe520059360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fe52c002f30 tx=0x7fe52c0069f0).stop 2021-04-03T03:17:37.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5340dbba0 msgr2=0x7fe5340a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5340dbba0 0x7fe5340a09c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7fe5240010d0 tx=0x7fe53c065000).stop 2021-04-03T03:17:37.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 shutdown_connections 2021-04-03T03:17:37.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe5340dbba0 0x7fe5340a09c0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.755 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe520056ea0 0x7fe520059360 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe53403d560 0x7fe5340a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 --2- 172.21.2.111:0/895443981 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe534041060 0x7fe5340ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.749+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 >> 172.21.2.111:0/895443981 conn(0x7fe5340462b0 msgr2=0x7fe534041490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:37.756 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.753+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 shutdown_connections 2021-04-03T03:17:37.757 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.753+0000 7fe53b59e700 1 -- 172.21.2.111:0/895443981 wait complete. 2021-04-03T03:17:37.761 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2021-04-03T03:17:37.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 Processor -- start 2021-04-03T03:17:37.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 -- start start 2021-04-03T03:17:37.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12180603f0 0x7f12181447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f12180add50 0x7f1218144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180b17b0 0x7f1218147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12181451c0 con 0x7f12180603f0 2021-04-03T03:17:37.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12180607d0 con 0x7f12180add50 2021-04-03T03:17:37.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.813+0000 7f121efe6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121814b090 con 0x7f12180b17b0 2021-04-03T03:17:37.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12180603f0 0x7f12181447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f12180603f0 0x7f12181447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59096/0 (socket says 172.21.2.111:59096) 2021-04-03T03:17:37.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 -- 172.21.2.111:0/3422114121 learned_addr learned my addr 172.21.2.111:0/3422114121 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:37.821 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 -- 172.21.2.111:0/3422114121 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180b17b0 msgr2=0x7f1218147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:37.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 --2- 172.21.2.111:0/3422114121 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180b17b0 0x7f1218147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 -- 172.21.2.111:0/3422114121 >> v2:172.21.2.115:3300/0 conn(0x7f12180add50 msgr2=0x7f1218144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:37.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 --2- 172.21.2.111:0/3422114121 >> v2:172.21.2.115:3300/0 conn(0x7f12180add50 0x7f1218144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 -- 172.21.2.111:0/3422114121 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121814b330 con 0x7f12180603f0 2021-04-03T03:17:37.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121dfe4700 1 --2- 172.21.2.111:0/3422114121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180603f0 0x7f12181447e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7f1214002f30 tx=0x7f1214006e90).ready entity=mon.0 client_cookie=69f232970fe79991 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121cfe2700 1 -- 172.21.2.111:0/3422114121 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1214009920 con 0x7f12180603f0 2021-04-03T03:17:37.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121cfe2700 1 -- 172.21.2.111:0/3422114121 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1214009a80 con 0x7f12180603f0 2021-04-03T03:17:37.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121cfe2700 1 -- 172.21.2.111:0/3422114121 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1214009de0 con 0x7f12180603f0 2021-04-03T03:17:37.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- 172.21.2.111:0/3422114121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180603f0 msgr2=0x7f12181447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 --2- 172.21.2.111:0/3422114121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180603f0 0x7f12181447e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7f1214002f30 tx=0x7f1214006e90).stop 2021-04-03T03:17:37.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- 172.21.2.111:0/3422114121 shutdown_connections 2021-04-03T03:17:37.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 --2- 172.21.2.111:0/3422114121 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180b17b0 0x7f1218147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 --2- 172.21.2.111:0/3422114121 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180603f0 0x7f12181447e0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 --2- 172.21.2.111:0/3422114121 >> v2:172.21.2.115:3300/0 conn(0x7f12180add50 0x7f1218144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- 172.21.2.111:0/3422114121 >> 172.21.2.111:0/3422114121 conn(0x7f12180b61c0 msgr2=0x7f121805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:37.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- 172.21.2.111:0/3422114121 shutdown_connections 2021-04-03T03:17:37.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- 172.21.2.111:0/3422114121 wait complete. 2021-04-03T03:17:37.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 Processor -- start 2021-04-03T03:17:37.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- start start 2021-04-03T03:17:37.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12180603f0 0x7f1218142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180add50 0x7f12181568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180b17b0 0x7f121815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12180a8920 con 0x7f12180b17b0 2021-04-03T03:17:37.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218067fa0 con 0x7f12180603f0 2021-04-03T03:17:37.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12180a9c50 con 0x7f12180add50 2021-04-03T03:17:37.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180b17b0 0x7f121815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180b17b0 0x7f121815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59104/0 (socket says 172.21.2.111:59104) 2021-04-03T03:17:37.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 -- 172.21.2.111:0/1400799483 learned_addr learned my addr 172.21.2.111:0/1400799483 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:37.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 -- 172.21.2.111:0/1400799483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180add50 msgr2=0x7f12181568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180add50 0x7f12181568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 -- 172.21.2.111:0/1400799483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12180603f0 msgr2=0x7f1218142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12180603f0 0x7f1218142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 -- 172.21.2.111:0/1400799483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180a9eb0 con 0x7f12180b17b0 2021-04-03T03:17:37.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121e7e5700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180b17b0 0x7f121815a630 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f12100032b0 tx=0x7f12100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f120affd700 1 -- 172.21.2.111:0/1400799483 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f121000c900 con 0x7f12180b17b0 2021-04-03T03:17:37.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f120affd700 1 -- 172.21.2.111:0/1400799483 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121000ca60 con 0x7f12180b17b0 2021-04-03T03:17:37.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1218159ef0 con 0x7f12180b17b0 2021-04-03T03:17:37.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1218147900 con 0x7f12180b17b0 2021-04-03T03:17:37.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.817+0000 7f120affd700 1 -- 172.21.2.111:0/1400799483 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f121000cdc0 con 0x7f12180b17b0 2021-04-03T03:17:37.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.821+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121815a320 con 0x7f12180b17b0 2021-04-03T03:17:37.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.821+0000 7f120affd700 1 -- 172.21.2.111:0/1400799483 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1210015470 con 0x7f12180b17b0 2021-04-03T03:17:37.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.821+0000 7f120affd700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f80570a0 0x7f11f8059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.821+0000 7f120affd700 1 -- 172.21.2.111:0/1400799483 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1210070230 con 0x7f12180b17b0 2021-04-03T03:17:37.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.821+0000 7f121dfe4700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f80570a0 0x7f11f8059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.821+0000 7f121dfe4700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f80570a0 0x7f11f8059560 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f1214002f30 tx=0x7f1214015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.833 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.821+0000 7f120affd700 1 -- 172.21.2.111:0/1400799483 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f12100454b0 con 0x7f12180b17b0 2021-04-03T03:17:37.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1218057660 con 0x7f12180b17b0 2021-04-03T03:17:37.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f120affd700 1 -- 172.21.2.111:0/1400799483 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1210048ad0 con 0x7f12180b17b0 2021-04-03T03:17:37.917 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:37.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f80570a0 msgr2=0x7f11f8059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f80570a0 0x7f11f8059560 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f1214002f30 tx=0x7f1214015000).stop 2021-04-03T03:17:37.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180b17b0 msgr2=0x7f121815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180b17b0 0x7f121815a630 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f12100032b0 tx=0x7f12100090f0).stop 2021-04-03T03:17:37.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 shutdown_connections 2021-04-03T03:17:37.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f12180b17b0 0x7f121815a630 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f11f80570a0 0x7f11f8059560 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f12180603f0 0x7f1218142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 --2- 172.21.2.111:0/1400799483 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f12180add50 0x7f12181568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:37.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 >> 172.21.2.111:0/1400799483 conn(0x7f12180b61c0 msgr2=0x7f12180b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:37.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 shutdown_connections 2021-04-03T03:17:37.920 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.913+0000 7f121efe6700 1 -- 172.21.2.111:0/1400799483 wait complete. 2021-04-03T03:17:37.924 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2021-04-03T03:17:37.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 Processor -- start 2021-04-03T03:17:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- start start 2021-04-03T03:17:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f484c05fce0 0x7f484c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c143ac0 0x7f484c140110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c0600c0 con 0x7f484c0b1b70 2021-04-03T03:17:37.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c0acb80 con 0x7f484c05fce0 2021-04-03T03:17:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0ace20 con 0x7f484c143ac0 2021-04-03T03:17:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f484c05fce0 0x7f484c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f484c05fce0 0x7f484c0b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59110/0 (socket says 172.21.2.111:59110) 2021-04-03T03:17:37.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 -- 172.21.2.111:0/2621144107 learned_addr learned my addr 172.21.2.111:0/2621144107 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:37.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484bfff700 1 --2- 172.21.2.111:0/2621144107 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c143ac0 0x7f484c140110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 -- 172.21.2.111:0/2621144107 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c143ac0 msgr2=0x7f484c140110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:37.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 --2- 172.21.2.111:0/2621144107 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c143ac0 0x7f484c140110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 -- 172.21.2.111:0/2621144107 >> v1:172.21.2.115:6789/0 conn(0x7f484c0b1b70 legacy=0x7f484c13fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 -- 172.21.2.111:0/2621144107 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0afc60 con 0x7f484c05fce0 2021-04-03T03:17:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484b7fe700 1 --2- 172.21.2.111:0/2621144107 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c05fce0 0x7f484c0b0130 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f483c002f30 tx=0x7f483c006f20).ready entity=mon.0 client_cookie=987f59eb3fd5e88b server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484a7fc700 1 -- 172.21.2.111:0/2621144107 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c00cc40 con 0x7f484c05fce0 2021-04-03T03:17:37.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484a7fc700 1 -- 172.21.2.111:0/2621144107 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c00cda0 con 0x7f484c05fce0 2021-04-03T03:17:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f484a7fc700 1 -- 172.21.2.111:0/2621144107 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c010600 con 0x7f484c05fce0 2021-04-03T03:17:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- 172.21.2.111:0/2621144107 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c05fce0 msgr2=0x7f484c0b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- 172.21.2.111:0/2621144107 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c05fce0 0x7f484c0b0130 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f483c002f30 tx=0x7f483c006f20).stop 2021-04-03T03:17:37.991 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- 172.21.2.111:0/2621144107 shutdown_connections 2021-04-03T03:17:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- 172.21.2.111:0/2621144107 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c143ac0 0x7f484c140110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- 172.21.2.111:0/2621144107 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c05fce0 0x7f484c0b0130 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- 172.21.2.111:0/2621144107 >> 172.21.2.111:0/2621144107 conn(0x7f484c0b5fb0 msgr2=0x7f484c0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- 172.21.2.111:0/2621144107 shutdown_connections 2021-04-03T03:17:37.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- 172.21.2.111:0/2621144107 wait complete. 2021-04-03T03:17:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 Processor -- start 2021-04-03T03:17:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- start start 2021-04-03T03:17:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c05fce0 0x7f484c13c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f484c0b1b70 0x7f484c066750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c143ac0 0x7f484c14e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0639b0 con 0x7f484c143ac0 2021-04-03T03:17:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c066010 con 0x7f484c0b1b70 2021-04-03T03:17:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.981+0000 7f4850c30700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c066c40 con 0x7f484c05fce0 2021-04-03T03:17:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c143ac0 0x7f484c14e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c143ac0 0x7f484c14e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59118/0 (socket says 172.21.2.111:59118) 2021-04-03T03:17:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 -- 172.21.2.111:0/3044927616 learned_addr learned my addr 172.21.2.111:0/3044927616 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484affd700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f484c0b1b70 0x7f484c066750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484b7fe700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c05fce0 0x7f484c13c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 -- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c05fce0 msgr2=0x7f484c13c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c05fce0 0x7f484c13c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 -- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f484c0b1b70 msgr2=0x7f484c066750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f484c0b1b70 0x7f484c066750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 -- 172.21.2.111:0/3044927616 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c063630 con 0x7f484c143ac0 2021-04-03T03:17:37.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484b7fe700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c05fce0 0x7f484c13c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-04-03T03:17:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484bfff700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c143ac0 0x7f484c14e080 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f4844006de0 tx=0x7f48440091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4848ff9700 1 -- 172.21.2.111:0/3044927616 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484400c9a0 con 0x7f484c143ac0 2021-04-03T03:17:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4848ff9700 1 -- 172.21.2.111:0/3044927616 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484400cb00 con 0x7f484c143ac0 2021-04-03T03:17:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c0647f0 con 0x7f484c143ac0 2021-04-03T03:17:37.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4848ff9700 1 -- 172.21.2.111:0/3044927616 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484400ce60 con 0x7f484c143ac0 2021-04-03T03:17:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c0670a0 con 0x7f484c143ac0 2021-04-03T03:17:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c057660 con 0x7f484c143ac0 2021-04-03T03:17:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4848ff9700 1 -- 172.21.2.111:0/3044927616 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4844022070 con 0x7f484c143ac0 2021-04-03T03:17:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4848ff9700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4830056f20 0x7f48300593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:37.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f4848ff9700 1 -- 172.21.2.111:0/3044927616 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f48440707f0 con 0x7f484c143ac0 2021-04-03T03:17:37.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484b7fe700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4830056f20 0x7f48300593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:37.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.985+0000 7f484b7fe700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4830056f20 0x7f48300593e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f483c01b000 tx=0x7f483c017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:37.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:37.989+0000 7f4848ff9700 1 -- 172.21.2.111:0/3044927616 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4844045b80 con 0x7f484c143ac0 2021-04-03T03:17:38.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f484c063820 con 0x7f484c143ac0 2021-04-03T03:17:38.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4848ff9700 1 -- 172.21.2.111:0/3044927616 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7f48440160a0 con 0x7f484c143ac0 2021-04-03T03:17:38.084 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:38.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4830056f20 msgr2=0x7f48300593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4830056f20 0x7f48300593e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f483c01b000 tx=0x7f483c017000).stop 2021-04-03T03:17:38.085 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c143ac0 msgr2=0x7f484c14e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c143ac0 0x7f484c14e080 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7f4844006de0 tx=0x7f48440091d0).stop 2021-04-03T03:17:38.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 shutdown_connections 2021-04-03T03:17:38.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f484c143ac0 0x7f484c14e080 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.086 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f484c05fce0 0x7f484c13c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4830056f20 0x7f48300593e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 --2- 172.21.2.111:0/3044927616 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f484c0b1b70 0x7f484c066750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 >> 172.21.2.111:0/3044927616 conn(0x7f484c0b5fb0 msgr2=0x7f484c05c730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:38.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 shutdown_connections 2021-04-03T03:17:38.087 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.081+0000 7f4850c30700 1 -- 172.21.2.111:0/3044927616 wait complete. 2021-04-03T03:17:38.091 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2021-04-03T03:17:38.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.145+0000 7fd9954a5700 1 Processor -- start 2021-04-03T03:17:38.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.145+0000 7fd9954a5700 1 -- start start 2021-04-03T03:17:38.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd990067d20 0x7fd9900aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9900678f0 con 0x7fd9900af8c0 2021-04-03T03:17:38.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9900b5f00 con 0x7fd9900b5b20 2021-04-03T03:17:38.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9900b2d70 con 0x7fd990067d20 2021-04-03T03:17:38.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd9900af8c0 0x7fd99013a740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59916/0 (socket says 172.21.2.111:59916) 2021-04-03T03:17:38.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 -- 172.21.2.111:0/3038316910 learned_addr learned my addr 172.21.2.111:0/3038316910 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:38.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391641023 0 0) 0x7fd9900678f0 con 0x7fd9900af8c0 2021-04-03T03:17:38.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd970003240 con 0x7fd9900af8c0 2021-04-03T03:17:38.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 --2- 172.21.2.111:0/3038316910 >> v2:172.21.2.115:3300/0 conn(0x7fd990067d20 0x7fd9900aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:38.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1041852998 0 0) 0x7fd970003240 con 0x7fd9900af8c0 2021-04-03T03:17:38.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 >> v2:172.21.2.115:3300/0 conn(0x7fd990067d20 msgr2=0x7fd9900aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:38.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 --2- 172.21.2.111:0/3038316910 >> v2:172.21.2.115:3300/0 conn(0x7fd990067d20 0x7fd9900aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 >> v1:172.21.2.115:6789/0 conn(0x7fd9900b5b20 legacy=0x7fd9901447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:38.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9900b3010 con 0x7fd9900af8c0 2021-04-03T03:17:38.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fd988001a50 con 0x7fd9900af8c0 2021-04-03T03:17:38.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 >> v1:172.21.2.111:6789/0 conn(0x7fd9900af8c0 legacy=0x7fd99013a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd970003800 0x7fd970003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9700044e0 0x7fd970004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd970004e50 0x7fd97000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900b2d70 con 0x7fd9700044e0 2021-04-03T03:17:38.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd970005300 con 0x7fd970003800 2021-04-03T03:17:38.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97000bc20 con 0x7fd970004e50 2021-04-03T03:17:38.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9700044e0 0x7fd970004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:38.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 -- 172.21.2.111:0/3038316910 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd970004e50 msgr2=0x7fd97000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:38.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd970004e50 0x7fd97000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 -- 172.21.2.111:0/3038316910 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd970003800 msgr2=0x7fd970003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:38.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd970003800 0x7fd970003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 -- 172.21.2.111:0/3038316910 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd970003240 con 0x7fd9700044e0 2021-04-03T03:17:38.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 -- 172.21.2.111:0/3038316910 reap_dead start 2021-04-03T03:17:38.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98e7fc700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9700044e0 0x7fd970004960 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7fd98000a000 tx=0x7fd980008040).ready entity=mon.0 client_cookie=4991def9694572af server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:38.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd98000da60 con 0x7fd9700044e0 2021-04-03T03:17:38.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98000dbc0 con 0x7fd9700044e0 2021-04-03T03:17:38.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98dffb700 1 -- 172.21.2.111:0/3038316910 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd980013480 con 0x7fd9700044e0 2021-04-03T03:17:38.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- 172.21.2.111:0/3038316910 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9700044e0 msgr2=0x7fd970004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9700044e0 0x7fd970004960 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7fd98000a000 tx=0x7fd980008040).stop 2021-04-03T03:17:38.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- 172.21.2.111:0/3038316910 shutdown_connections 2021-04-03T03:17:38.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 --2- 172.21.2.111:0/3038316910 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd9700044e0 0x7fd970004960 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- 172.21.2.111:0/3038316910 >> 172.21.2.111:0/3038316910 conn(0x7fd990062d10 msgr2=0x7fd990063120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:38.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- 172.21.2.111:0/3038316910 shutdown_connections 2021-04-03T03:17:38.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- 172.21.2.111:0/3038316910 wait complete. 2021-04-03T03:17:38.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 Processor -- start 2021-04-03T03:17:38.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- start start 2021-04-03T03:17:38.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9700044e0 0x7fd99013df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd99013a750 0x7fd99013c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd99013c6f0 0x7fd990141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99013e450 con 0x7fd99013c6f0 2021-04-03T03:17:38.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99013cb70 con 0x7fd99013a750 2021-04-03T03:17:38.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd9954a5700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99013daf0 con 0x7fd9700044e0 2021-04-03T03:17:38.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd99013c6f0 0x7fd990141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd99013c6f0 0x7fd990141650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59136/0 (socket says 172.21.2.111:59136) 2021-04-03T03:17:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 -- 172.21.2.111:0/2528738965 learned_addr learned my addr 172.21.2.111:0/2528738965 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 -- 172.21.2.111:0/2528738965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9700044e0 msgr2=0x7fd99013df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9700044e0 0x7fd99013df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 -- 172.21.2.111:0/2528738965 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd99013a750 msgr2=0x7fd99013c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd99013a750 0x7fd99013c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 -- 172.21.2.111:0/2528738965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99013ffe0 con 0x7fd99013c6f0 2021-04-03T03:17:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.149+0000 7fd98f7fe700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd99013c6f0 0x7fd990141650 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7fd988001450 tx=0x7fd9900b31a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:38.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd987fff700 1 -- 172.21.2.111:0/2528738965 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd988004a90 con 0x7fd99013c6f0 2021-04-03T03:17:38.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd987fff700 1 -- 172.21.2.111:0/2528738965 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd988004bf0 con 0x7fd99013c6f0 2021-04-03T03:17:38.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99013b160 con 0x7fd99013c6f0 2021-04-03T03:17:38.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd990141b40 con 0x7fd99013c6f0 2021-04-03T03:17:38.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd987fff700 1 -- 172.21.2.111:0/2528738965 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd988007030 con 0x7fd99013c6f0 2021-04-03T03:17:38.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99013b590 con 0x7fd99013c6f0 2021-04-03T03:17:38.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd987fff700 1 -- 172.21.2.111:0/2528738965 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd988010070 con 0x7fd99013c6f0 2021-04-03T03:17:38.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd987fff700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd974056ea0 0x7fd974059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd987fff700 1 -- 172.21.2.111:0/2528738965 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fd98806fd10 con 0x7fd99013c6f0 2021-04-03T03:17:38.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd98effd700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd974056ea0 0x7fd974059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:38.169 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd98effd700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd974056ea0 0x7fd974059360 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fd978001f60 tx=0x7fd9780077d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:38.170 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.153+0000 7fd987fff700 1 -- 172.21.2.111:0/2528738965 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd988044ff0 con 0x7fd99013c6f0 2021-04-03T03:17:38.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd990057660 con 0x7fd99013c6f0 2021-04-03T03:17:38.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd987fff700 1 -- 172.21.2.111:0/2528738965 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd988048610 con 0x7fd99013c6f0 2021-04-03T03:17:38.252 INFO:teuthology.orchestra.run.gibba011.stdout:pg_num: 8 2021-04-03T03:17:38.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd974056ea0 msgr2=0x7fd974059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd974056ea0 0x7fd974059360 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fd978001f60 tx=0x7fd9780077d0).stop 2021-04-03T03:17:38.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd99013c6f0 msgr2=0x7fd990141650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd99013c6f0 0x7fd990141650 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7fd988001450 tx=0x7fd9900b31a0).stop 2021-04-03T03:17:38.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 shutdown_connections 2021-04-03T03:17:38.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd99013c6f0 0x7fd990141650 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9700044e0 0x7fd99013df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd974056ea0 0x7fd974059360 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 --2- 172.21.2.111:0/2528738965 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd99013a750 0x7fd99013c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 >> 172.21.2.111:0/2528738965 conn(0x7fd990062d10 msgr2=0x7fd9900ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:38.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 shutdown_connections 2021-04-03T03:17:38.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.249+0000 7fd9954a5700 1 -- 172.21.2.111:0/2528738965 wait complete. 2021-04-03T03:17:38.260 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-04-03T03:17:38.260 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:38.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 Processor -- start 2021-04-03T03:17:38.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- start start 2021-04-03T03:17:38.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fc6b40acb30 0x7fc6b41473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc6b4142ef0 0x7fc6b40ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b40607d0 con 0x7fc6b40603f0 2021-04-03T03:17:38.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b41433a0 con 0x7fc6b4142ef0 2021-04-03T03:17:38.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b40b03a0 con 0x7fc6b40acb30 2021-04-03T03:17:38.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc6b4142ef0 0x7fc6b40ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:38.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc6b4142ef0 0x7fc6b40ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59144/0 (socket says 172.21.2.111:59144) 2021-04-03T03:17:38.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 -- 172.21.2.111:0/2774118541 learned_addr learned my addr 172.21.2.111:0/2774118541 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:38.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 -- 172.21.2.111:0/2774118541 >> v2:172.21.2.115:3300/0 conn(0x7fc6b40acb30 msgr2=0x7fc6b41473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:38.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 --2- 172.21.2.111:0/2774118541 >> v2:172.21.2.115:3300/0 conn(0x7fc6b40acb30 0x7fc6b41473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 -- 172.21.2.111:0/2774118541 >> v1:172.21.2.115:6789/0 conn(0x7fc6b40603f0 legacy=0x7fc6b41427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:38.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 -- 172.21.2.111:0/2774118541 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b40af800 con 0x7fc6b4142ef0 2021-04-03T03:17:38.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b2d9d700 1 --2- 172.21.2.111:0/2774118541 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b4142ef0 0x7fc6b40ae350 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fc6b40ae230 tx=0x7fc6ac009270).ready entity=mon.0 client_cookie=7de8c771035a430f server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:38.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b159a700 1 -- 172.21.2.111:0/2774118541 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ac00bbc0 con 0x7fc6b4142ef0 2021-04-03T03:17:38.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b159a700 1 -- 172.21.2.111:0/2774118541 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac00bd20 con 0x7fc6b4142ef0 2021-04-03T03:17:38.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b159a700 1 -- 172.21.2.111:0/2774118541 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ac00be90 con 0x7fc6b4142ef0 2021-04-03T03:17:38.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- 172.21.2.111:0/2774118541 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b4142ef0 msgr2=0x7fc6b40ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 --2- 172.21.2.111:0/2774118541 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b4142ef0 0x7fc6b40ae350 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fc6b40ae230 tx=0x7fc6ac009270).stop 2021-04-03T03:17:38.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- 172.21.2.111:0/2774118541 shutdown_connections 2021-04-03T03:17:38.322 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 --2- 172.21.2.111:0/2774118541 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b4142ef0 0x7fc6b40ae350 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 --2- 172.21.2.111:0/2774118541 >> v2:172.21.2.115:3300/0 conn(0x7fc6b40acb30 0x7fc6b41473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- 172.21.2.111:0/2774118541 >> 172.21.2.111:0/2774118541 conn(0x7fc6b40b61c0 msgr2=0x7fc6b405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:38.323 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- 172.21.2.111:0/2774118541 shutdown_connections 2021-04-03T03:17:38.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.313+0000 7fc6b359e700 1 -- 172.21.2.111:0/2774118541 wait complete. 2021-04-03T03:17:38.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 Processor -- start 2021-04-03T03:17:38.324 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 -- start start 2021-04-03T03:17:38.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b40603f0 0x7fc6b41469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.326 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6b40acb30 0x7fc6b4146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6b4142ef0 0x7fc6b40abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4146200 con 0x7fc6b40603f0 2021-04-03T03:17:38.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40a0420 con 0x7fc6b4142ef0 2021-04-03T03:17:38.327 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40a0150 con 0x7fc6b40acb30 2021-04-03T03:17:38.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b40603f0 0x7fc6b41469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:38.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b40603f0 0x7fc6b41469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59146/0 (socket says 172.21.2.111:59146) 2021-04-03T03:17:38.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 -- 172.21.2.111:0/1035104051 learned_addr learned my addr 172.21.2.111:0/1035104051 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:38.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 -- 172.21.2.111:0/1035104051 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6b40acb30 msgr2=0x7fc6b4146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.328 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6b40acb30 0x7fc6b4146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 -- 172.21.2.111:0/1035104051 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6b4142ef0 msgr2=0x7fc6b40abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6b4142ef0 0x7fc6b40abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 -- 172.21.2.111:0/1035104051 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4144e50 con 0x7fc6b40603f0 2021-04-03T03:17:38.329 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b259c700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b40603f0 0x7fc6b41469d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fc69c002f50 tx=0x7fc69c006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:38.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6ab7fe700 1 -- 172.21.2.111:0/1035104051 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc69c00d4a0 con 0x7fc6b40603f0 2021-04-03T03:17:38.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6ab7fe700 1 -- 172.21.2.111:0/1035104051 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c00d600 con 0x7fc6b40603f0 2021-04-03T03:17:38.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b4145160 con 0x7fc6b40603f0 2021-04-03T03:17:38.330 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b40abff0 con 0x7fc6b40603f0 2021-04-03T03:17:38.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6ab7fe700 1 -- 172.21.2.111:0/1035104051 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc69c00d960 con 0x7fc6b40603f0 2021-04-03T03:17:38.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b4057660 con 0x7fc6b40603f0 2021-04-03T03:17:38.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6ab7fe700 1 -- 172.21.2.111:0/1035104051 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc69c00f070 con 0x7fc6b40603f0 2021-04-03T03:17:38.331 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6ab7fe700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc698056ef0 0x7fc6980593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:38.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6ab7fe700 1 -- 172.21.2.111:0/1035104051 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7fc69c06fe50 con 0x7fc6b40603f0 2021-04-03T03:17:38.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b1d9b700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc698056ef0 0x7fc6980593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:38.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.317+0000 7fc6b1d9b700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc698056ef0 0x7fc6980593b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fc6a4002f30 tx=0x7fc6a4006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:38.332 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.321+0000 7fc6ab7fe700 1 -- 172.21.2.111:0/1035104051 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fc69c045130 con 0x7fc6b40603f0 2021-04-03T03:17:38.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.441+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6b40686d0 con 0x7fc6b40603f0 2021-04-03T03:17:38.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6ab7fe700 1 -- 172.21.2.111:0/1035104051 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fc69c048750 con 0x7fc6b40603f0 2021-04-03T03:17:38.447 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:38.447 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":68,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3455926982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3455926982},{"type":"v1","addr":"172.21.2.115:6833","nonce":3455926982}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:33.145029+0000","modified":"2021-04-03T03:17:34.147803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/295340141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":295340141},{"type":"v1","addr":"172.21.2.111:6835","nonce":295340141}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-04-03T03:17:38.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc698056ef0 msgr2=0x7fc6980593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc698056ef0 0x7fc6980593b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fc6a4002f30 tx=0x7fc6a4006a60).stop 2021-04-03T03:17:38.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b40603f0 msgr2=0x7fc6b41469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:38.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b40603f0 0x7fc6b41469d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fc69c002f50 tx=0x7fc69c006dd0).stop 2021-04-03T03:17:38.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 shutdown_connections 2021-04-03T03:17:38.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc698056ef0 0x7fc6980593b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc6b4142ef0 0x7fc6b40abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc6b40603f0 0x7fc6b41469d0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:38.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 --2- 172.21.2.111:0/1035104051 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc6b40acb30 0x7fc6b4146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:38.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 >> 172.21.2.111:0/1035104051 conn(0x7fc6b40b61c0 msgr2=0x7fc6b40b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:38.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 shutdown_connections 2021-04-03T03:17:38.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:38.445+0000 7fc6b359e700 1 -- 172.21.2.111:0/1035104051 wait complete. 2021-04-03T03:17:38.450 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 68 2021-04-03T03:17:38.655 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-04-03T03:17:33.145029+0000', 'modified': '2021-04-03T03:17:34.147803+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8526': {'gid': 8526, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.111:6835/295340141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.111:6834', 'nonce': 295340141}, {'type': 'v1', 'addr': '172.21.2.111:6835', 'nonce': 295340141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-04-03T03:17:38.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-04-03T03:17:38.656 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-04-03T03:17:38.733 INFO:teuthology.orchestra.run.gibba011.stdout:{"cluster_fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","whoami":0,"id":8526,"want_state":"up:active","state":"up:active","rank_uptime":5.5804743549999998,"mdsmap_epoch":68,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":9.7269645120000003} 2021-04-03T03:17:38.733 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", "id": 8526, "mdsmap_epoch": 68, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 5.580474355, "state": "up:active", "uptime": 9.726964512, "want_state": "up:active", "whoami": 0 } 2021-04-03T03:17:38.733 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-04-03T03:17:38.734 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-04-03T03:17:38.734 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba028.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-04-03T03:17:38.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:38.735 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:38.745 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:17:38.799 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:38.799 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:38.848 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:38.848 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:38.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:38.849 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:17:38.901 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-04-03T03:17:38.901 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:38.947 DEBUG:teuthology.orchestra.run.gibba028:> sudo modprobe fuse 2021-04-03T03:17:38.960 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-04-03T03:17:38.960 DEBUG:teuthology.orchestra.run.gibba028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-04-03T03:17:38.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15752]: starting ceph client 2021-04-03T03:17:38.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:2021-04-03T03:17:38.981+0000 7f8aff1eb0c0 -1 init, newargv = 0x56311e6cc1e0 newargc=9 2021-04-03T03:17:38.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15752]: starting fuse 2021-04-03T03:17:38.994 INFO:teuthology.orchestra.run.gibba028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-04-03T03:17:38.995 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-04-03T03:17:38.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:38.996 DEBUG:teuthology.orchestra.run.gibba028:> ls /sys/fs/fuse/connections 2021-04-03T03:17:39.041 INFO:teuthology.orchestra.run.gibba028.stdout:49 2021-04-03T03:17:39.041 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2021-04-03T03:17:39.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:39.042 DEBUG:teuthology.orchestra.run.gibba028:> sudo python3 -c ' 2021-04-03T03:17:39.042 DEBUG:teuthology.orchestra.run.gibba028:> import glob 2021-04-03T03:17:39.043 DEBUG:teuthology.orchestra.run.gibba028:> import re 2021-04-03T03:17:39.043 DEBUG:teuthology.orchestra.run.gibba028:> import os 2021-04-03T03:17:39.043 DEBUG:teuthology.orchestra.run.gibba028:> import subprocess 2021-04-03T03:17:39.044 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:39.044 DEBUG:teuthology.orchestra.run.gibba028:> def find_socket(client_name): 2021-04-03T03:17:39.045 DEBUG:teuthology.orchestra.run.gibba028:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-04-03T03:17:39.045 DEBUG:teuthology.orchestra.run.gibba028:> files = glob.glob(asok_path) 2021-04-03T03:17:39.045 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:39.046 DEBUG:teuthology.orchestra.run.gibba028:> # Given a non-glob path, it better be there 2021-04-03T03:17:39.046 DEBUG:teuthology.orchestra.run.gibba028:> if "*" not in asok_path: 2021-04-03T03:17:39.046 DEBUG:teuthology.orchestra.run.gibba028:> assert(len(files) == 1) 2021-04-03T03:17:39.047 DEBUG:teuthology.orchestra.run.gibba028:> return files[0] 2021-04-03T03:17:39.047 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:39.048 DEBUG:teuthology.orchestra.run.gibba028:> for f in files: 2021-04-03T03:17:39.048 DEBUG:teuthology.orchestra.run.gibba028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-04-03T03:17:39.048 DEBUG:teuthology.orchestra.run.gibba028:> if os.path.exists("/proc/{0}".format(pid)): 2021-04-03T03:17:39.049 DEBUG:teuthology.orchestra.run.gibba028:> return f 2021-04-03T03:17:39.049 DEBUG:teuthology.orchestra.run.gibba028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-04-03T03:17:39.049 DEBUG:teuthology.orchestra.run.gibba028:> 2021-04-03T03:17:39.050 DEBUG:teuthology.orchestra.run.gibba028:> print(find_socket("client.1")) 2021-04-03T03:17:39.050 DEBUG:teuthology.orchestra.run.gibba028:> ' 2021-04-03T03:17:39.160 INFO:teuthology.orchestra.run.gibba028.stdout:/var/run/ceph/ceph-client.1.15752.asok 2021-04-03T03:17:39.164 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15752.asok 2021-04-03T03:17:39.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:39.164 DEBUG:teuthology.orchestra.run.gibba028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15752.asok status 2021-04-03T03:17:39.219 INFO:teuthology.orchestra.run.gibba028.stdout:{ 2021-04-03T03:17:39.220 INFO:teuthology.orchestra.run.gibba028.stdout: "metadata": { 2021-04-03T03:17:39.220 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_sha1": "5feb67b5bda898b8c739180363606432d4f39831", 2021-04-03T03:17:39.220 INFO:teuthology.orchestra.run.gibba028.stdout: "ceph_version": "ceph version 15.2.10-137-g5feb67b5 (5feb67b5bda898b8c739180363606432d4f39831) octopus (stable)", 2021-04-03T03:17:39.220 INFO:teuthology.orchestra.run.gibba028.stdout: "entity_id": "1", 2021-04-03T03:17:39.220 INFO:teuthology.orchestra.run.gibba028.stdout: "hostname": "gibba028", 2021-04-03T03:17:39.221 INFO:teuthology.orchestra.run.gibba028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-04-03T03:17:39.221 INFO:teuthology.orchestra.run.gibba028.stdout: "pid": "15752", 2021-04-03T03:17:39.221 INFO:teuthology.orchestra.run.gibba028.stdout: "root": "/" 2021-04-03T03:17:39.221 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:39.221 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_count": 0, 2021-04-03T03:17:39.222 INFO:teuthology.orchestra.run.gibba028.stdout: "dentry_pinned_count": 0, 2021-04-03T03:17:39.222 INFO:teuthology.orchestra.run.gibba028.stdout: "id": 5309, 2021-04-03T03:17:39.222 INFO:teuthology.orchestra.run.gibba028.stdout: "inst": { 2021-04-03T03:17:39.222 INFO:teuthology.orchestra.run.gibba028.stdout: "name": { 2021-04-03T03:17:39.223 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "client", 2021-04-03T03:17:39.223 INFO:teuthology.orchestra.run.gibba028.stdout: "num": 5309 2021-04-03T03:17:39.223 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:39.223 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:17:39.223 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:17:39.224 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:17:39.224 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 304504963 2021-04-03T03:17:39.224 INFO:teuthology.orchestra.run.gibba028.stdout: } 2021-04-03T03:17:39.224 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:39.224 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": { 2021-04-03T03:17:39.225 INFO:teuthology.orchestra.run.gibba028.stdout: "type": "v1", 2021-04-03T03:17:39.225 INFO:teuthology.orchestra.run.gibba028.stdout: "addr": "172.21.2.128:0", 2021-04-03T03:17:39.225 INFO:teuthology.orchestra.run.gibba028.stdout: "nonce": 304504963 2021-04-03T03:17:39.225 INFO:teuthology.orchestra.run.gibba028.stdout: }, 2021-04-03T03:17:39.225 INFO:teuthology.orchestra.run.gibba028.stdout: "inst_str": "client.5309 172.21.2.128:0/304504963", 2021-04-03T03:17:39.226 INFO:teuthology.orchestra.run.gibba028.stdout: "addr_str": "172.21.2.128:0/304504963", 2021-04-03T03:17:39.226 INFO:teuthology.orchestra.run.gibba028.stdout: "inode_count": 1, 2021-04-03T03:17:39.226 INFO:teuthology.orchestra.run.gibba028.stdout: "mds_epoch": 68, 2021-04-03T03:17:39.226 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch": 151, 2021-04-03T03:17:39.227 INFO:teuthology.orchestra.run.gibba028.stdout: "osd_epoch_barrier": 0, 2021-04-03T03:17:39.227 INFO:teuthology.orchestra.run.gibba028.stdout: "blacklisted": false, 2021-04-03T03:17:39.227 INFO:teuthology.orchestra.run.gibba028.stdout: "fs_name": "cephfs" 2021-04-03T03:17:39.227 INFO:teuthology.orchestra.run.gibba028.stdout:} 2021-04-03T03:17:39.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:39.229 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:39.229 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:39.234 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:17:39.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:39.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:39.235 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:39.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:39.284 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:39.284 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:39.330 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:17:39.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:39.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:39.331 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:39.331 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:39.379 INFO:teuthology.orchestra.run.gibba028.stdout:fuseblk 2021-04-03T03:17:39.380 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-04-03T03:17:39.380 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba028.front.sepia.ceph.com... 2021-04-03T03:17:39.381 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-04-03T03:17:39.381 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:39.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba028.stderr:ceph-fuse[15752]: fuse finished with error 0 and tester_r 0 2021-04-03T03:17:39.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:17:39.465 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:17:39.465 DEBUG:teuthology.orchestra.run.gibba028:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:39.513 INFO:teuthology.orchestra.run.gibba028.stdout:ext2/ext3 2021-04-03T03:17:39.514 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-04-03T03:17:39.514 INFO:teuthology.orchestra.run:waiting for 900 2021-04-03T03:17:39.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-04-03T03:17:39.588 DEBUG:teuthology.orchestra.run.gibba028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-04-03T03:17:39.592 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-04-03T03:17:39.592 DEBUG:teuthology.orchestra.run.gibba026:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2021-04-03T03:17:39.611 INFO:teuthology.orchestra.run.gibba026.stdout: File: "." 2021-04-03T03:17:39.611 INFO:teuthology.orchestra.run.gibba026.stdout: ID: 0 Namelen: 255 Type: fuseblk 2021-04-03T03:17:39.611 INFO:teuthology.orchestra.run.gibba026.stdout:Block size: 4194304 Fundamental block size: 4194304 2021-04-03T03:17:39.612 INFO:teuthology.orchestra.run.gibba026.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2021-04-03T03:17:39.612 INFO:teuthology.orchestra.run.gibba026.stdout:Inodes: Total: 1 Free: 0 2021-04-03T03:17:39.613 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2021-04-03T03:17:39.673 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.669+0000 7f468c9cc700 1 Processor -- start 2021-04-03T03:17:39.674 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.669+0000 7f468c9cc700 1 -- start start 2021-04-03T03:17:39.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f46880603f0 0x7f46880b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:39.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f468814b700 0x7f46880b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:39.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f46880b12b0 0x7f46880b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:39.675 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880b0f60 con 0x7f46880603f0 2021-04-03T03:17:39.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880607d0 con 0x7f46880b12b0 2021-04-03T03:17:39.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880af400 con 0x7f468814b700 2021-04-03T03:17:39.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f46880603f0 0x7f46880b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:39.676 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f46880603f0 0x7f46880b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59154/0 (socket says 172.21.2.111:59154) 2021-04-03T03:17:39.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 -- 172.21.2.111:0/2500716404 learned_addr learned my addr 172.21.2.111:0/2500716404 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:39.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 -- 172.21.2.111:0/2500716404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f468814b700 msgr2=0x7f46880b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:39.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 --2- 172.21.2.111:0/2500716404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f468814b700 0x7f46880b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.677 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 -- 172.21.2.111:0/2500716404 >> v2:172.21.2.115:3300/0 conn(0x7f46880b12b0 msgr2=0x7f46880b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:39.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 --2- 172.21.2.111:0/2500716404 >> v2:172.21.2.115:3300/0 conn(0x7f46880b12b0 0x7f46880b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 -- 172.21.2.111:0/2500716404 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880af6a0 con 0x7f46880603f0 2021-04-03T03:17:39.678 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46877fe700 1 --2- 172.21.2.111:0/2500716404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46880603f0 0x7f46880b0090 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f4678002f30 tx=0x7f4678006e90).ready entity=mon.0 client_cookie=1eaea998bf46b25a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:39.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46867fc700 1 -- 172.21.2.111:0/2500716404 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4678009920 con 0x7f46880603f0 2021-04-03T03:17:39.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46867fc700 1 -- 172.21.2.111:0/2500716404 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678009a80 con 0x7f46880603f0 2021-04-03T03:17:39.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f46867fc700 1 -- 172.21.2.111:0/2500716404 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4678009de0 con 0x7f46880603f0 2021-04-03T03:17:39.681 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- 172.21.2.111:0/2500716404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46880603f0 msgr2=0x7f46880b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:39.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- 172.21.2.111:0/2500716404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46880603f0 0x7f46880b0090 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f4678002f30 tx=0x7f4678006e90).stop 2021-04-03T03:17:39.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- 172.21.2.111:0/2500716404 shutdown_connections 2021-04-03T03:17:39.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- 172.21.2.111:0/2500716404 >> v2:172.21.2.115:3300/0 conn(0x7f46880b12b0 0x7f46880b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.682 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- 172.21.2.111:0/2500716404 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f46880603f0 0x7f46880b0090 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:39.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- 172.21.2.111:0/2500716404 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f468814b700 0x7f46880b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- 172.21.2.111:0/2500716404 >> 172.21.2.111:0/2500716404 conn(0x7f46880b61c0 msgr2=0x7f468805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:39.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- 172.21.2.111:0/2500716404 shutdown_connections 2021-04-03T03:17:39.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- 172.21.2.111:0/2500716404 wait complete. 2021-04-03T03:17:39.683 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 Processor -- start 2021-04-03T03:17:39.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- start start 2021-04-03T03:17:39.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46880603f0 0x7f46880aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:39.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46880b12b0 0x7f4688062c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:39.684 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f468814b700 0x7f46880ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:39.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4688157340 con 0x7f468814b700 2021-04-03T03:17:39.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880ab120 con 0x7f46880603f0 2021-04-03T03:17:39.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880abd50 con 0x7f46880b12b0 2021-04-03T03:17:39.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f468814b700 0x7f46880ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:39.685 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f468814b700 0x7f46880ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59162/0 (socket says 172.21.2.111:59162) 2021-04-03T03:17:39.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 -- 172.21.2.111:0/1779716446 learned_addr learned my addr 172.21.2.111:0/1779716446 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:39.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 -- 172.21.2.111:0/1779716446 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46880b12b0 msgr2=0x7f4688062c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:39.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46880b12b0 0x7f4688062c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.686 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 -- 172.21.2.111:0/1779716446 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46880603f0 msgr2=0x7f46880aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:39.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46880603f0 0x7f46880aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 -- 172.21.2.111:0/1779716446 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880a9f40 con 0x7f468814b700 2021-04-03T03:17:39.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4687fff700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f468814b700 0x7f46880ab860 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f46800032b0 tx=0x7f4680009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:39.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4684ff9700 1 -- 172.21.2.111:0/1779716446 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468000ca60 con 0x7f468814b700 2021-04-03T03:17:39.687 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4684ff9700 1 -- 172.21.2.111:0/1779716446 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000cbc0 con 0x7f468814b700 2021-04-03T03:17:39.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4688156e40 con 0x7f468814b700 2021-04-03T03:17:39.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f4684ff9700 1 -- 172.21.2.111:0/1779716446 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4680015420 con 0x7f468814b700 2021-04-03T03:17:39.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.673+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46880a8040 con 0x7f468814b700 2021-04-03T03:17:39.688 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.677+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4688157150 con 0x7f468814b700 2021-04-03T03:17:39.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.677+0000 7f4684ff9700 1 -- 172.21.2.111:0/1779716446 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f468001c070 con 0x7f468814b700 2021-04-03T03:17:39.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.677+0000 7f4684ff9700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f466c056ef0 0x7f466c0593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:39.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.677+0000 7f4684ff9700 1 -- 172.21.2.111:0/1779716446 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f468006fee0 con 0x7f468814b700 2021-04-03T03:17:39.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.677+0000 7f46877fe700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f466c056ef0 0x7f466c0593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:39.689 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.677+0000 7f46877fe700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f466c056ef0 0x7f466c0593b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f4678002f30 tx=0x7f4678015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:39.690 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.677+0000 7f4684ff9700 1 -- 172.21.2.111:0/1779716446 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f46800451c0 con 0x7f468814b700 2021-04-03T03:17:39.772 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.769+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4688057660 con 0x7f468814b700 2021-04-03T03:17:39.773 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.769+0000 7f4684ff9700 1 -- 172.21.2.111:0/1779716446 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v151) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f46800487e0 con 0x7f468814b700 2021-04-03T03:17:39.774 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:39.774 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T03:17:39.774 INFO:teuthology.orchestra.run.gibba011.stdout: "epoch": 151, 2021-04-03T03:17:39.774 INFO:teuthology.orchestra.run.gibba011.stdout: "fsid": "6fa64cfd-c000-43a2-869e-5811f8b4928c", 2021-04-03T03:17:39.775 INFO:teuthology.orchestra.run.gibba011.stdout: "created": "2021-04-03T03:08:26.873888+0000", 2021-04-03T03:17:39.775 INFO:teuthology.orchestra.run.gibba011.stdout: "modified": "2021-04-03T03:17:34.148833+0000", 2021-04-03T03:17:39.775 INFO:teuthology.orchestra.run.gibba011.stdout: "last_up_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:17:39.775 INFO:teuthology.orchestra.run.gibba011.stdout: "last_in_change": "2021-04-03T03:08:34.085903+0000", 2021-04-03T03:17:39.775 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-04-03T03:17:39.776 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_num": 5799936, 2021-04-03T03:17:39.776 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_set": [ 2021-04-03T03:17:39.776 INFO:teuthology.orchestra.run.gibba011.stdout: "pglog_hardlimit", 2021-04-03T03:17:39.776 INFO:teuthology.orchestra.run.gibba011.stdout: "purged_snapdirs", 2021-04-03T03:17:39.776 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_deletes", 2021-04-03T03:17:39.777 INFO:teuthology.orchestra.run.gibba011.stdout: "sortbitwise" 2021-04-03T03:17:39.777 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:39.777 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_version": 5, 2021-04-03T03:17:39.777 INFO:teuthology.orchestra.run.gibba011.stdout: "full_ratio": 0.94999998807907104, 2021-04-03T03:17:39.777 INFO:teuthology.orchestra.run.gibba011.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-04-03T03:17:39.778 INFO:teuthology.orchestra.run.gibba011.stdout: "nearfull_ratio": 0.85000002384185791, 2021-04-03T03:17:39.778 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_snapshot": "", 2021-04-03T03:17:39.778 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_max": 30, 2021-04-03T03:17:39.778 INFO:teuthology.orchestra.run.gibba011.stdout: "max_osd": 8, 2021-04-03T03:17:39.779 INFO:teuthology.orchestra.run.gibba011.stdout: "require_min_compat_client": "jewel", 2021-04-03T03:17:39.779 INFO:teuthology.orchestra.run.gibba011.stdout: "min_compat_client": "jewel", 2021-04-03T03:17:39.779 INFO:teuthology.orchestra.run.gibba011.stdout: "require_osd_release": "octopus", 2021-04-03T03:17:39.779 INFO:teuthology.orchestra.run.gibba011.stdout: "pools": [ 2021-04-03T03:17:39.779 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.780 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 1, 2021-04-03T03:17:39.780 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "device_health_metrics", 2021-04-03T03:17:39.780 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:29.718060+0000", 2021-04-03T03:17:39.780 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:39.780 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:39.781 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:39.781 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:39.781 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:39.781 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:39.781 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:39.782 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:39.782 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 1, 2021-04-03T03:17:39.782 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 1, 2021-04-03T03:17:39.782 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 1, 2021-04-03T03:17:39.782 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 1, 2021-04-03T03:17:39.783 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 1, 2021-04-03T03:17:39.783 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:39.783 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:39.783 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:39.783 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:39.784 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:39.784 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:39.784 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:39.784 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.785 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "15", 2021-04-03T03:17:39.785 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:39.785 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:39.785 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:39.785 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:39.786 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:39.786 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:39.786 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:39.786 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:39.786 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:39.786 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:39.787 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:39.787 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:39.787 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:39.787 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:39.787 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:39.788 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:39.788 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:39.788 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:39.788 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:39.788 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:39.789 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:39.789 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:39.789 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:39.789 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:39.790 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:39.790 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:39.790 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.790 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:39.790 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:39.790 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:39.791 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:39.791 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:39.791 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:39.791 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:39.791 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:39.792 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:39.792 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:39.792 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:39.792 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:17:39.792 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 1 2021-04-03T03:17:39.793 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.793 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:39.793 INFO:teuthology.orchestra.run.gibba011.stdout: "mgr_devicehealth": {} 2021-04-03T03:17:39.793 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.793 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.794 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.794 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 2, 2021-04-03T03:17:39.794 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "rbd", 2021-04-03T03:17:39.794 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:08:38.084175+0000", 2021-04-03T03:17:39.795 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:39.795 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:39.795 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:39.795 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:39.795 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:39.796 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:39.796 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:39.796 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:39.796 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:39.797 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:39.797 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:39.797 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:39.797 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:39.797 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:39.798 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:39.798 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:39.798 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:39.798 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:39.798 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:39.799 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:39.799 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.799 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "18", 2021-04-03T03:17:39.799 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:39.799 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:39.800 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:39.800 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:39.800 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:39.800 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:39.801 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:39.801 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:39.801 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:39.801 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:39.801 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:39.802 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:39.802 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:39.802 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:39.802 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:39.803 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:39.803 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:39.803 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:39.803 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:39.804 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:39.804 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:39.804 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:39.804 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:39.805 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:39.805 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:39.805 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:39.805 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.805 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:39.806 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:39.806 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:39.806 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:39.806 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:39.806 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:39.807 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:39.807 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:39.807 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:39.807 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:39.807 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:39.808 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:17:39.808 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:39.808 INFO:teuthology.orchestra.run.gibba011.stdout: "rbd": {} 2021-04-03T03:17:39.808 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.808 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.809 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.809 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 29, 2021-04-03T03:17:39.809 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_metadata", 2021-04-03T03:17:39.809 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:17:31.174355+0000", 2021-04-03T03:17:39.809 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:39.810 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:39.810 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:39.810 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:39.810 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:39.810 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:39.811 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:39.811 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:39.811 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:39.811 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:39.811 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:39.812 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:39.812 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:39.812 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:39.812 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:39.812 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:39.813 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:39.813 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:39.813 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:39.813 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:39.813 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.814 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "150", 2021-04-03T03:17:39.814 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:39.814 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:39.814 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:39.815 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:39.815 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:39.815 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:39.815 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:39.815 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:39.816 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:39.816 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:39.816 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:39.816 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:39.816 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:39.817 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:39.817 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:39.817 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:39.817 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:39.817 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:39.818 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:39.818 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:39.818 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:39.818 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:39.818 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:39.819 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:39.819 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:39.819 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:39.819 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.819 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:39.820 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:39.820 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:39.820 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:39.820 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:39.820 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:39.821 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:39.821 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:39.821 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:39.821 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:39.821 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:39.822 INFO:teuthology.orchestra.run.gibba011.stdout: "options": { 2021-04-03T03:17:39.822 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_bias": 4, 2021-04-03T03:17:39.822 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_min": 16, 2021-04-03T03:17:39.822 INFO:teuthology.orchestra.run.gibba011.stdout: "recovery_priority": 5 2021-04-03T03:17:39.822 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.823 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:39.823 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:17:39.823 INFO:teuthology.orchestra.run.gibba011.stdout: "metadata": "cephfs" 2021-04-03T03:17:39.823 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.823 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.824 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.824 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.824 INFO:teuthology.orchestra.run.gibba011.stdout: "pool": 30, 2021-04-03T03:17:39.824 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_name": "cephfs_data", 2021-04-03T03:17:39.824 INFO:teuthology.orchestra.run.gibba011.stdout: "create_time": "2021-04-03T03:17:32.113426+0000", 2021-04-03T03:17:39.825 INFO:teuthology.orchestra.run.gibba011.stdout: "flags": 1, 2021-04-03T03:17:39.825 INFO:teuthology.orchestra.run.gibba011.stdout: "flags_names": "hashpspool", 2021-04-03T03:17:39.825 INFO:teuthology.orchestra.run.gibba011.stdout: "type": 1, 2021-04-03T03:17:39.825 INFO:teuthology.orchestra.run.gibba011.stdout: "size": 2, 2021-04-03T03:17:39.825 INFO:teuthology.orchestra.run.gibba011.stdout: "min_size": 1, 2021-04-03T03:17:39.826 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_rule": 0, 2021-04-03T03:17:39.826 INFO:teuthology.orchestra.run.gibba011.stdout: "object_hash": 2, 2021-04-03T03:17:39.826 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_autoscale_mode": "off", 2021-04-03T03:17:39.826 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num": 8, 2021-04-03T03:17:39.827 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num": 8, 2021-04-03T03:17:39.827 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_placement_num_target": 8, 2021-04-03T03:17:39.827 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_target": 8, 2021-04-03T03:17:39.827 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_num_pending": 8, 2021-04-03T03:17:39.827 INFO:teuthology.orchestra.run.gibba011.stdout: "last_pg_merge_meta": { 2021-04-03T03:17:39.828 INFO:teuthology.orchestra.run.gibba011.stdout: "source_pgid": "0.0", 2021-04-03T03:17:39.828 INFO:teuthology.orchestra.run.gibba011.stdout: "ready_epoch": 0, 2021-04-03T03:17:39.828 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_started": 0, 2021-04-03T03:17:39.828 INFO:teuthology.orchestra.run.gibba011.stdout: "last_epoch_clean": 0, 2021-04-03T03:17:39.828 INFO:teuthology.orchestra.run.gibba011.stdout: "source_version": "0'0", 2021-04-03T03:17:39.829 INFO:teuthology.orchestra.run.gibba011.stdout: "target_version": "0'0" 2021-04-03T03:17:39.829 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.829 INFO:teuthology.orchestra.run.gibba011.stdout: "last_change": "151", 2021-04-03T03:17:39.829 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend": "0", 2021-04-03T03:17:39.830 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_prenautilus": "0", 2021-04-03T03:17:39.830 INFO:teuthology.orchestra.run.gibba011.stdout: "last_force_op_resend_preluminous": "0", 2021-04-03T03:17:39.830 INFO:teuthology.orchestra.run.gibba011.stdout: "auid": 0, 2021-04-03T03:17:39.830 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_mode": "selfmanaged", 2021-04-03T03:17:39.830 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_seq": 0, 2021-04-03T03:17:39.831 INFO:teuthology.orchestra.run.gibba011.stdout: "snap_epoch": 0, 2021-04-03T03:17:39.831 INFO:teuthology.orchestra.run.gibba011.stdout: "pool_snaps": [], 2021-04-03T03:17:39.831 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps": "[]", 2021-04-03T03:17:39.831 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_bytes": 0, 2021-04-03T03:17:39.831 INFO:teuthology.orchestra.run.gibba011.stdout: "quota_max_objects": 0, 2021-04-03T03:17:39.832 INFO:teuthology.orchestra.run.gibba011.stdout: "tiers": [], 2021-04-03T03:17:39.832 INFO:teuthology.orchestra.run.gibba011.stdout: "tier_of": -1, 2021-04-03T03:17:39.832 INFO:teuthology.orchestra.run.gibba011.stdout: "read_tier": -1, 2021-04-03T03:17:39.832 INFO:teuthology.orchestra.run.gibba011.stdout: "write_tier": -1, 2021-04-03T03:17:39.832 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_mode": "none", 2021-04-03T03:17:39.833 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_bytes": 0, 2021-04-03T03:17:39.833 INFO:teuthology.orchestra.run.gibba011.stdout: "target_max_objects": 0, 2021-04-03T03:17:39.833 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-04-03T03:17:39.833 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-04-03T03:17:39.833 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_target_full_ratio_micro": 800000, 2021-04-03T03:17:39.834 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_flush_age": 0, 2021-04-03T03:17:39.834 INFO:teuthology.orchestra.run.gibba011.stdout: "cache_min_evict_age": 0, 2021-04-03T03:17:39.834 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profile": "", 2021-04-03T03:17:39.834 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_params": { 2021-04-03T03:17:39.834 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "none" 2021-04-03T03:17:39.835 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.835 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_period": 0, 2021-04-03T03:17:39.835 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_count": 0, 2021-04-03T03:17:39.835 INFO:teuthology.orchestra.run.gibba011.stdout: "use_gmt_hitset": true, 2021-04-03T03:17:39.835 INFO:teuthology.orchestra.run.gibba011.stdout: "min_read_recency_for_promote": 0, 2021-04-03T03:17:39.836 INFO:teuthology.orchestra.run.gibba011.stdout: "min_write_recency_for_promote": 0, 2021-04-03T03:17:39.836 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_grade_decay_rate": 0, 2021-04-03T03:17:39.836 INFO:teuthology.orchestra.run.gibba011.stdout: "hit_set_search_last_n": 0, 2021-04-03T03:17:39.836 INFO:teuthology.orchestra.run.gibba011.stdout: "grade_table": [], 2021-04-03T03:17:39.837 INFO:teuthology.orchestra.run.gibba011.stdout: "stripe_width": 0, 2021-04-03T03:17:39.837 INFO:teuthology.orchestra.run.gibba011.stdout: "expected_num_objects": 0, 2021-04-03T03:17:39.837 INFO:teuthology.orchestra.run.gibba011.stdout: "fast_read": false, 2021-04-03T03:17:39.837 INFO:teuthology.orchestra.run.gibba011.stdout: "options": {}, 2021-04-03T03:17:39.837 INFO:teuthology.orchestra.run.gibba011.stdout: "application_metadata": { 2021-04-03T03:17:39.837 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs": { 2021-04-03T03:17:39.838 INFO:teuthology.orchestra.run.gibba011.stdout: "data": "cephfs" 2021-04-03T03:17:39.838 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.838 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.838 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.838 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:39.839 INFO:teuthology.orchestra.run.gibba011.stdout: "osds": [ 2021-04-03T03:17:39.839 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.839 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:17:39.839 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b944db97-2a22-405b-847f-9d436ceab0ad", 2021-04-03T03:17:39.840 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.840 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.840 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.840 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.840 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.841 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.841 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.841 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 148, 2021-04-03T03:17:39.841 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.841 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.842 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.842 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.842 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.842 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.842 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6812", 2021-04-03T03:17:39.843 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.843 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.843 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.843 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.843 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6814", 2021-04-03T03:17:39.844 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.844 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.844 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.844 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.845 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.845 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.845 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.845 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.845 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6816", 2021-04-03T03:17:39.846 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.846 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.846 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.846 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.846 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6818", 2021-04-03T03:17:39.847 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.847 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.847 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.847 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.847 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.848 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.848 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.848 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.848 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6824", 2021-04-03T03:17:39.848 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.849 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.849 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.849 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.849 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6825", 2021-04-03T03:17:39.849 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.850 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.850 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.850 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.850 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.850 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.851 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.851 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.851 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6820", 2021-04-03T03:17:39.851 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.851 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.852 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.852 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.852 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6822", 2021-04-03T03:17:39.852 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15909 2021-04-03T03:17:39.852 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.853 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.853 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.853 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6814/15909", 2021-04-03T03:17:39.853 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6818/15909", 2021-04-03T03:17:39.853 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6825/15909", 2021-04-03T03:17:39.854 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6822/15909", 2021-04-03T03:17:39.854 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.854 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.854 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.854 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.855 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.855 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.855 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:17:39.855 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "a1b55cbd-c469-405c-92a4-a45166c2194a", 2021-04-03T03:17:39.855 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.856 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.856 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.856 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.856 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.857 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.857 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.857 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 147, 2021-04-03T03:17:39.857 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.857 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.858 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.858 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.858 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.858 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.858 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6802", 2021-04-03T03:17:39.859 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.859 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.859 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.859 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.859 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6803", 2021-04-03T03:17:39.860 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.860 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.860 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.860 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.860 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.861 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.861 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.861 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.861 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6804", 2021-04-03T03:17:39.862 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.862 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.862 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.862 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.862 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6805", 2021-04-03T03:17:39.863 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.863 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.863 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.863 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.863 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.864 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.864 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.864 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.864 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6808", 2021-04-03T03:17:39.864 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.865 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.865 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.865 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.865 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6809", 2021-04-03T03:17:39.866 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.866 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.866 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.866 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.866 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.867 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.867 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.867 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.867 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6806", 2021-04-03T03:17:39.867 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.868 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.868 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.868 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.868 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6807", 2021-04-03T03:17:39.869 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15910 2021-04-03T03:17:39.869 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.869 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.869 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.869 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6803/15910", 2021-04-03T03:17:39.870 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6805/15910", 2021-04-03T03:17:39.870 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6809/15910", 2021-04-03T03:17:39.870 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6807/15910", 2021-04-03T03:17:39.870 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.870 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.871 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.871 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.871 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.871 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.871 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:17:39.872 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "0954b9bc-6c00-42c3-b666-fe5b20532ca1", 2021-04-03T03:17:39.872 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.872 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.872 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.873 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.873 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.873 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.873 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.873 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 148, 2021-04-03T03:17:39.874 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.874 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.874 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.874 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.874 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.875 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.875 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6810", 2021-04-03T03:17:39.875 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.875 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.876 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.876 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.876 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6811", 2021-04-03T03:17:39.876 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.876 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.877 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.877 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.877 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.877 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.878 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.878 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.878 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6813", 2021-04-03T03:17:39.878 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.878 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.879 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.879 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.879 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6815", 2021-04-03T03:17:39.879 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.879 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.880 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.880 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.880 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.880 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.880 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.881 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.881 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6821", 2021-04-03T03:17:39.881 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.881 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.881 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.882 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.882 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6823", 2021-04-03T03:17:39.882 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.882 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.882 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.883 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.883 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.883 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.883 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.883 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.884 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6817", 2021-04-03T03:17:39.884 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.884 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.885 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.885 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.885 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6819", 2021-04-03T03:17:39.885 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15911 2021-04-03T03:17:39.885 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.886 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.886 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.886 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6811/15911", 2021-04-03T03:17:39.886 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6815/15911", 2021-04-03T03:17:39.887 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6823/15911", 2021-04-03T03:17:39.887 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6819/15911", 2021-04-03T03:17:39.887 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.887 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.887 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.888 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.888 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.888 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.888 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:17:39.888 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "69ab5309-0edd-4359-a5b8-56d9ceac0649", 2021-04-03T03:17:39.889 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.889 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.889 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.889 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.890 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.890 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.890 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.890 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 149, 2021-04-03T03:17:39.890 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.891 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.891 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.891 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.891 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.891 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.892 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6826", 2021-04-03T03:17:39.892 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.892 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.892 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.893 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.893 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6827", 2021-04-03T03:17:39.893 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.893 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.893 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.894 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.894 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.894 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.894 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6828", 2021-04-03T03:17:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.895 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6829", 2021-04-03T03:17:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.896 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.897 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.898 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6832", 2021-04-03T03:17:39.898 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.898 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.898 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.898 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.899 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6833", 2021-04-03T03:17:39.899 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.899 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.899 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.900 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.900 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.900 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.900 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.900 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.901 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6830", 2021-04-03T03:17:39.901 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.901 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.901 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.901 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.902 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.111:6831", 2021-04-03T03:17:39.902 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 15913 2021-04-03T03:17:39.902 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.902 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.903 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.903 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.111:6827/15913", 2021-04-03T03:17:39.903 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.111:6829/15913", 2021-04-03T03:17:39.903 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.111:6833/15913", 2021-04-03T03:17:39.903 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.111:6831/15913", 2021-04-03T03:17:39.904 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.904 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.904 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.904 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.904 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.905 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.905 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:17:39.905 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "107b94d0-940a-4a28-b981-4eb105bea5fc", 2021-04-03T03:17:39.905 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.905 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.906 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.907 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 147, 2021-04-03T03:17:39.907 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.907 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.907 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.908 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.908 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.908 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.908 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6800", 2021-04-03T03:17:39.908 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.909 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.909 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.909 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.909 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6801", 2021-04-03T03:17:39.909 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.910 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.910 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.910 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.910 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.910 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.911 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.911 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.911 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6802", 2021-04-03T03:17:39.911 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.912 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.912 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.912 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.912 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6803", 2021-04-03T03:17:39.912 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.913 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.913 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.913 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.913 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.913 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.914 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.914 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.914 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6806", 2021-04-03T03:17:39.914 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.914 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.915 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.915 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.915 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6807", 2021-04-03T03:17:39.915 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.916 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.916 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.916 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.916 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.916 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.917 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.917 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.917 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6804", 2021-04-03T03:17:39.917 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.917 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.918 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.918 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.918 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6805", 2021-04-03T03:17:39.918 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16012 2021-04-03T03:17:39.918 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.919 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.919 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.919 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6801/16012", 2021-04-03T03:17:39.919 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6803/16012", 2021-04-03T03:17:39.919 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6807/16012", 2021-04-03T03:17:39.920 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6805/16012", 2021-04-03T03:17:39.920 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.920 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.920 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.920 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.921 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.921 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.921 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:17:39.921 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "b020dc25-a06c-4de5-8896-b6af08322240", 2021-04-03T03:17:39.922 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.922 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.922 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.922 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.922 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.923 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.923 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.923 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 148, 2021-04-03T03:17:39.923 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.923 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.924 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.924 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.924 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.924 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.925 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6816", 2021-04-03T03:17:39.925 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.925 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.925 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.925 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.926 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6817", 2021-04-03T03:17:39.926 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.926 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.926 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.926 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.927 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.927 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.927 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.927 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.927 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6818", 2021-04-03T03:17:39.928 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.928 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.928 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.928 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.928 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6819", 2021-04-03T03:17:39.929 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.929 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.929 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.929 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.929 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.930 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.930 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.930 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.930 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6822", 2021-04-03T03:17:39.930 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.931 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.931 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.931 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.931 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6823", 2021-04-03T03:17:39.932 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.932 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.932 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.932 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.932 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.933 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.933 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.933 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.933 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6820", 2021-04-03T03:17:39.933 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.934 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.934 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.934 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.934 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6821", 2021-04-03T03:17:39.935 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16014 2021-04-03T03:17:39.935 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.935 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.935 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.935 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6817/16014", 2021-04-03T03:17:39.936 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6819/16014", 2021-04-03T03:17:39.936 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6823/16014", 2021-04-03T03:17:39.936 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6821/16014", 2021-04-03T03:17:39.936 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.936 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.937 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.937 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.937 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.937 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.937 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:17:39.938 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "cc3194fc-248c-46e3-a1a6-c149abc74a9a", 2021-04-03T03:17:39.938 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.938 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.938 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.938 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.939 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.939 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.939 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.939 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 148, 2021-04-03T03:17:39.939 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.940 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.940 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.940 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.940 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.940 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.941 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6808", 2021-04-03T03:17:39.941 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.941 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.941 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.941 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.942 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6809", 2021-04-03T03:17:39.942 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.942 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.942 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.943 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.943 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.943 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.943 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.943 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.944 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6810", 2021-04-03T03:17:39.944 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.944 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.944 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.944 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.945 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6811", 2021-04-03T03:17:39.945 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.945 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.945 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.946 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.946 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.946 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.946 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.946 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.947 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6814", 2021-04-03T03:17:39.947 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.947 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.947 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.947 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.948 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6815", 2021-04-03T03:17:39.948 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.948 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.948 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.948 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.949 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.949 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.949 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.949 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.949 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6812", 2021-04-03T03:17:39.950 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.950 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.950 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.950 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.951 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6813", 2021-04-03T03:17:39.951 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16015 2021-04-03T03:17:39.951 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.951 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.951 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.952 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6809/16015", 2021-04-03T03:17:39.952 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6811/16015", 2021-04-03T03:17:39.952 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6815/16015", 2021-04-03T03:17:39.952 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6813/16015", 2021-04-03T03:17:39.952 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.953 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.953 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.953 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.953 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.953 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.954 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:17:39.954 INFO:teuthology.orchestra.run.gibba011.stdout: "uuid": "f091dcf7-acf3-4d21-b0ad-c2ab84a82035", 2021-04-03T03:17:39.954 INFO:teuthology.orchestra.run.gibba011.stdout: "up": 1, 2021-04-03T03:17:39.954 INFO:teuthology.orchestra.run.gibba011.stdout: "in": 1, 2021-04-03T03:17:39.954 INFO:teuthology.orchestra.run.gibba011.stdout: "weight": 1, 2021-04-03T03:17:39.955 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_affinity": 1, 2021-04-03T03:17:39.955 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_begin": 0, 2021-04-03T03:17:39.955 INFO:teuthology.orchestra.run.gibba011.stdout: "last_clean_end": 0, 2021-04-03T03:17:39.955 INFO:teuthology.orchestra.run.gibba011.stdout: "up_from": 13, 2021-04-03T03:17:39.956 INFO:teuthology.orchestra.run.gibba011.stdout: "up_thru": 148, 2021-04-03T03:17:39.956 INFO:teuthology.orchestra.run.gibba011.stdout: "down_at": 0, 2021-04-03T03:17:39.956 INFO:teuthology.orchestra.run.gibba011.stdout: "lost_at": 0, 2021-04-03T03:17:39.956 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addrs": { 2021-04-03T03:17:39.956 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.957 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.957 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.957 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6824", 2021-04-03T03:17:39.957 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.957 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.958 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.958 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.958 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6825", 2021-04-03T03:17:39.958 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.958 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.959 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.959 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.959 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addrs": { 2021-04-03T03:17:39.959 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.959 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.960 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.960 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6826", 2021-04-03T03:17:39.960 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.960 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.960 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.961 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.961 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6827", 2021-04-03T03:17:39.961 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.961 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.962 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.962 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.962 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addrs": { 2021-04-03T03:17:39.962 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.962 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.963 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.963 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6830", 2021-04-03T03:17:39.963 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.963 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.963 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.964 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.964 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6831", 2021-04-03T03:17:39.964 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.964 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.964 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.965 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.965 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addrs": { 2021-04-03T03:17:39.965 INFO:teuthology.orchestra.run.gibba011.stdout: "addrvec": [ 2021-04-03T03:17:39.965 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.965 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v2", 2021-04-03T03:17:39.966 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6828", 2021-04-03T03:17:39.966 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.966 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.966 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.967 INFO:teuthology.orchestra.run.gibba011.stdout: "type": "v1", 2021-04-03T03:17:39.967 INFO:teuthology.orchestra.run.gibba011.stdout: "addr": "172.21.2.115:6829", 2021-04-03T03:17:39.967 INFO:teuthology.orchestra.run.gibba011.stdout: "nonce": 16058 2021-04-03T03:17:39.967 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.967 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.968 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.968 INFO:teuthology.orchestra.run.gibba011.stdout: "public_addr": "172.21.2.115:6825/16058", 2021-04-03T03:17:39.968 INFO:teuthology.orchestra.run.gibba011.stdout: "cluster_addr": "172.21.2.115:6827/16058", 2021-04-03T03:17:39.968 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_back_addr": "172.21.2.115:6831/16058", 2021-04-03T03:17:39.968 INFO:teuthology.orchestra.run.gibba011.stdout: "heartbeat_front_addr": "172.21.2.115:6829/16058", 2021-04-03T03:17:39.969 INFO:teuthology.orchestra.run.gibba011.stdout: "state": [ 2021-04-03T03:17:39.969 INFO:teuthology.orchestra.run.gibba011.stdout: "exists", 2021-04-03T03:17:39.969 INFO:teuthology.orchestra.run.gibba011.stdout: "up" 2021-04-03T03:17:39.969 INFO:teuthology.orchestra.run.gibba011.stdout: ] 2021-04-03T03:17:39.969 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.970 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:39.970 INFO:teuthology.orchestra.run.gibba011.stdout: "osd_xinfo": [ 2021-04-03T03:17:39.970 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.970 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 0, 2021-04-03T03:17:39.970 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.971 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.971 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.971 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.971 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.972 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.417965+0000", 2021-04-03T03:17:39.972 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.972 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.972 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.972 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 1, 2021-04-03T03:17:39.973 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.973 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.973 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.973 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.973 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.974 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.629207+0000", 2021-04-03T03:17:39.974 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.974 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.974 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.974 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 2, 2021-04-03T03:17:39.975 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.975 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.975 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.975 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.976 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.976 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.680350+0000", 2021-04-03T03:17:39.976 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.976 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.976 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.977 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 3, 2021-04-03T03:17:39.977 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.977 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.977 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.977 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.978 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.978 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.507066+0000", 2021-04-03T03:17:39.978 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.978 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.978 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.979 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 4, 2021-04-03T03:17:39.979 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.979 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.979 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.980 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.980 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.980 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.665996+0000", 2021-04-03T03:17:39.980 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.980 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.981 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.981 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 5, 2021-04-03T03:17:39.981 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.981 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.981 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.982 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.982 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.982 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.763971+0000", 2021-04-03T03:17:39.982 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.982 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.983 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.983 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 6, 2021-04-03T03:17:39.983 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.983 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.984 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.984 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.984 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.984 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.675661+0000", 2021-04-03T03:17:39.984 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.985 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.985 INFO:teuthology.orchestra.run.gibba011.stdout: { 2021-04-03T03:17:39.985 INFO:teuthology.orchestra.run.gibba011.stdout: "osd": 7, 2021-04-03T03:17:39.985 INFO:teuthology.orchestra.run.gibba011.stdout: "down_stamp": "0.000000", 2021-04-03T03:17:39.985 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_probability": 0, 2021-04-03T03:17:39.986 INFO:teuthology.orchestra.run.gibba011.stdout: "laggy_interval": 0, 2021-04-03T03:17:39.986 INFO:teuthology.orchestra.run.gibba011.stdout: "features": 4540138292840890367, 2021-04-03T03:17:39.986 INFO:teuthology.orchestra.run.gibba011.stdout: "old_weight": 0, 2021-04-03T03:17:39.986 INFO:teuthology.orchestra.run.gibba011.stdout: "last_purged_snaps_scrub": "2021-04-03T03:08:32.498476+0000", 2021-04-03T03:17:39.986 INFO:teuthology.orchestra.run.gibba011.stdout: "dead_epoch": 0 2021-04-03T03:17:39.987 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.987 INFO:teuthology.orchestra.run.gibba011.stdout: ], 2021-04-03T03:17:39.987 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap": [], 2021-04-03T03:17:39.987 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_upmap_items": [], 2021-04-03T03:17:39.987 INFO:teuthology.orchestra.run.gibba011.stdout: "pg_temp": [], 2021-04-03T03:17:39.988 INFO:teuthology.orchestra.run.gibba011.stdout: "primary_temp": [], 2021-04-03T03:17:39.988 INFO:teuthology.orchestra.run.gibba011.stdout: "blacklist": {}, 2021-04-03T03:17:39.988 INFO:teuthology.orchestra.run.gibba011.stdout: "erasure_code_profiles": { 2021-04-03T03:17:39.988 INFO:teuthology.orchestra.run.gibba011.stdout: "cephfs_data": { 2021-04-03T03:17:39.988 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-device-class": "", 2021-04-03T03:17:39.989 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:17:39.989 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-root": "default", 2021-04-03T03:17:39.989 INFO:teuthology.orchestra.run.gibba011.stdout: "jerasure-per-chunk-alignment": "false", 2021-04-03T03:17:39.989 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:17:39.989 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "2", 2021-04-03T03:17:39.990 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:17:39.990 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:17:39.990 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van", 2021-04-03T03:17:39.990 INFO:teuthology.orchestra.run.gibba011.stdout: "w": "8" 2021-04-03T03:17:39.990 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.991 INFO:teuthology.orchestra.run.gibba011.stdout: "default": { 2021-04-03T03:17:39.991 INFO:teuthology.orchestra.run.gibba011.stdout: "crush-failure-domain": "osd", 2021-04-03T03:17:39.991 INFO:teuthology.orchestra.run.gibba011.stdout: "k": "2", 2021-04-03T03:17:39.991 INFO:teuthology.orchestra.run.gibba011.stdout: "m": "1", 2021-04-03T03:17:39.992 INFO:teuthology.orchestra.run.gibba011.stdout: "plugin": "jerasure", 2021-04-03T03:17:39.992 INFO:teuthology.orchestra.run.gibba011.stdout: "ruleset-failure-domain": "osd", 2021-04-03T03:17:39.992 INFO:teuthology.orchestra.run.gibba011.stdout: "technique": "reed_sol_van" 2021-04-03T03:17:39.992 INFO:teuthology.orchestra.run.gibba011.stdout: } 2021-04-03T03:17:39.992 INFO:teuthology.orchestra.run.gibba011.stdout: }, 2021-04-03T03:17:39.993 INFO:teuthology.orchestra.run.gibba011.stdout: "removed_snaps_queue": [], 2021-04-03T03:17:39.993 INFO:teuthology.orchestra.run.gibba011.stdout: "new_removed_snaps": [], 2021-04-03T03:17:39.993 INFO:teuthology.orchestra.run.gibba011.stdout: "new_purged_snaps": [], 2021-04-03T03:17:39.993 INFO:teuthology.orchestra.run.gibba011.stdout: "crush_node_flags": {}, 2021-04-03T03:17:39.993 INFO:teuthology.orchestra.run.gibba011.stdout: "device_class_flags": {} 2021-04-03T03:17:39.994 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T03:17:39.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f466c056ef0 msgr2=0x7f466c0593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:39.995 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f466c056ef0 0x7f466c0593b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f4678002f30 tx=0x7f4678015000).stop 2021-04-03T03:17:39.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f468814b700 msgr2=0x7f46880ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:39.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f468814b700 0x7f46880ab860 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f46800032b0 tx=0x7f4680009220).stop 2021-04-03T03:17:39.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 shutdown_connections 2021-04-03T03:17:39.996 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f468814b700 0x7f46880ab860 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:39.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f466c056ef0 0x7f466c0593b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:39.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f46880603f0 0x7f46880aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 --2- 172.21.2.111:0/1779716446 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f46880b12b0 0x7f4688062c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:39.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 >> 172.21.2.111:0/1779716446 conn(0x7f46880b61c0 msgr2=0x7f4688147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:39.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 shutdown_connections 2021-04-03T03:17:39.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:39.773+0000 7f468c9cc700 1 -- 172.21.2.111:0/1779716446 wait complete. 2021-04-03T03:17:39.998 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:40.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.053+0000 7f1073de7700 1 Processor -- start 2021-04-03T03:17:40.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.053+0000 7f1073de7700 1 -- start start 2021-04-03T03:17:40.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.053+0000 7f1073de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.053+0000 7f1073de7700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f106c0d7660 0x7f106c0d7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.053+0000 7f1073de7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c0db880 con 0x7f106c04b130 2021-04-03T03:17:40.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.053+0000 7f1073de7700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c04b510 con 0x7f106c0d7660 2021-04-03T03:17:40.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.053+0000 7f1073de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c009890 con 0x7f106c0456f0 2021-04-03T03:17:40.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10735e6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f106c0d7660 0x7f106c0d7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:40.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10725e4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:40.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1072de5700 1 --1- >> v1:172.21.2.115:6789/0 conn(0x7f106c04b130 0x7f106c0dbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.115:6789/0 says I am v1:172.21.2.111:44264/0 (socket says 172.21.2.111:44264) 2021-04-03T03:17:40.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1072de5700 1 -- 172.21.2.111:0/3166652889 learned_addr learned my addr 172.21.2.111:0/3166652889 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:40.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10725e4700 1 -- 172.21.2.111:0/3166652889 >> v2:172.21.2.115:3300/0 conn(0x7f106c0d7660 msgr2=0x7f106c0d7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:40.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10725e4700 1 --2- 172.21.2.111:0/3166652889 >> v2:172.21.2.115:3300/0 conn(0x7f106c0d7660 0x7f106c0d7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10725e4700 1 -- 172.21.2.111:0/3166652889 >> v1:172.21.2.115:6789/0 conn(0x7f106c04b130 legacy=0x7f106c0dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10725e4700 1 -- 172.21.2.111:0/3166652889 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c009b30 con 0x7f106c0456f0 2021-04-03T03:17:40.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1071de3700 1 -- 172.21.2.111:0/3166652889 <== mon.1 v1:172.21.2.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819274619 0 0) 0x7f106c0db880 con 0x7f106c04b130 2021-04-03T03:17:40.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10735e6700 1 --2- 172.21.2.111:0/3166652889 >> v2:172.21.2.115:3300/0 conn(0x7f106c0d7660 0x7f106c0d7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-04-03T03:17:40.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f10725e4700 1 --2- 172.21.2.111:0/3166652889 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0dc3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f1068002f30 tx=0x7f1068006dd0).ready entity=mon.2 client_cookie=9c2b6ec65e40049d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:40.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1071de3700 1 -- 172.21.2.111:0/3166652889 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1068009760 con 0x7f106c0456f0 2021-04-03T03:17:40.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1071de3700 1 -- 172.21.2.111:0/3166652889 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10680098c0 con 0x7f106c0456f0 2021-04-03T03:17:40.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1071de3700 1 -- 172.21.2.111:0/3166652889 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1068009c20 con 0x7f106c0456f0 2021-04-03T03:17:40.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 -- 172.21.2.111:0/3166652889 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 msgr2=0x7f106c0dc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 --2- 172.21.2.111:0/3166652889 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0dc3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f1068002f30 tx=0x7f1068006dd0).stop 2021-04-03T03:17:40.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 -- 172.21.2.111:0/3166652889 shutdown_connections 2021-04-03T03:17:40.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 --2- 172.21.2.111:0/3166652889 >> v2:172.21.2.115:3300/0 conn(0x7f106c0d7660 0x7f106c0d7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 --2- 172.21.2.111:0/3166652889 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0dc3c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 -- 172.21.2.111:0/3166652889 >> 172.21.2.111:0/3166652889 conn(0x7f106c011530 msgr2=0x7f106c04a700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:40.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 -- 172.21.2.111:0/3166652889 shutdown_connections 2021-04-03T03:17:40.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.057+0000 7f1073de7700 1 -- 172.21.2.111:0/3166652889 wait complete. 2021-04-03T03:17:40.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 Processor -- start 2021-04-03T03:17:40.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 -- start start 2021-04-03T03:17:40.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f106c04b130 0x7f106c0eb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106c0d7660 0x7f106c0e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0ea780 con 0x7f106c04b130 2021-04-03T03:17:40.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0ea9e0 con 0x7f106c0d7660 2021-04-03T03:17:40.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0eac80 con 0x7f106c0456f0 2021-04-03T03:17:40.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f106c04b130 0x7f106c0eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:40.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f106c04b130 0x7f106c0eb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59178/0 (socket says 172.21.2.111:59178) 2021-04-03T03:17:40.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 -- 172.21.2.111:0/2016585636 learned_addr learned my addr 172.21.2.111:0/2016585636 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:40.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 -- 172.21.2.111:0/2016585636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 msgr2=0x7f106c0ddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 -- 172.21.2.111:0/2016585636 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106c0d7660 msgr2=0x7f106c0e71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106c0d7660 0x7f106c0e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 -- 172.21.2.111:0/2016585636 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c0eb610 con 0x7f106c04b130 2021-04-03T03:17:40.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f10725e4700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f106c04b130 0x7f106c0eb120 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f1068002f30 tx=0x7f1068015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:40.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1063fff700 1 -- 172.21.2.111:0/2016585636 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f106801d450 con 0x7f106c04b130 2021-04-03T03:17:40.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1063fff700 1 -- 172.21.2.111:0/2016585636 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106801d5b0 con 0x7f106c04b130 2021-04-03T03:17:40.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1063fff700 1 -- 172.21.2.111:0/2016585636 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f106801d910 con 0x7f106c04b130 2021-04-03T03:17:40.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c0dab90 con 0x7f106c04b130 2021-04-03T03:17:40.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c040780 con 0x7f106c04b130 2021-04-03T03:17:40.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c0db120 con 0x7f106c04b130 2021-04-03T03:17:40.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1063fff700 1 -- 172.21.2.111:0/2016585636 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f106800f070 con 0x7f106c04b130 2021-04-03T03:17:40.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1063fff700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050057050 0x7f1050059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1063fff700 1 -- 172.21.2.111:0/2016585636 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7f1068071df0 con 0x7f106c04b130 2021-04-03T03:17:40.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1072de5700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050057050 0x7f1050059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:40.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.061+0000 7f1072de5700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050057050 0x7f1050059510 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f1064000e90 tx=0x7f10640076a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:40.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.065+0000 7f1063fff700 1 -- 172.21.2.111:0/2016585636 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f10680471e0 con 0x7f106c04b130 2021-04-03T03:17:40.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f106c004da0 con 0x7f106c04b130 2021-04-03T03:17:40.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1063fff700 1 -- 172.21.2.111:0/2016585636 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f106804a800 con 0x7f106c04b130 2021-04-03T03:17:40.191 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:40.191 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":68,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3455926982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3455926982},{"type":"v1","addr":"172.21.2.115:6833","nonce":3455926982}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:33.145029+0000","modified":"2021-04-03T03:17:34.147803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.2.111:6835/295340141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6834","nonce":295340141},{"type":"v1","addr":"172.21.2.111:6835","nonce":295340141}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-04-03T03:17:40.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050057050 msgr2=0x7f1050059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050057050 0x7f1050059510 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f1064000e90 tx=0x7f10640076a0).stop 2021-04-03T03:17:40.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f106c04b130 msgr2=0x7f106c0eb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f106c04b130 0x7f106c0eb120 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f1068002f30 tx=0x7f1068015000).stop 2021-04-03T03:17:40.193 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 shutdown_connections 2021-04-03T03:17:40.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1050057050 0x7f1050059510 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f106c0d7660 0x7f106c0e71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f106c0456f0 0x7f106c0ddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 --2- 172.21.2.111:0/2016585636 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f106c04b130 0x7f106c0eb120 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.194 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 >> 172.21.2.111:0/2016585636 conn(0x7f106c011530 msgr2=0x7f106c049c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:40.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 shutdown_connections 2021-04-03T03:17:40.195 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.189+0000 7f1073de7700 1 -- 172.21.2.111:0/2016585636 wait complete. 2021-04-03T03:17:40.195 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 68 2021-04-03T03:17:40.199 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2021-04-03T03:17:40.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 Processor -- start 2021-04-03T03:17:40.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 -- start start 2021-04-03T03:17:40.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efdac0603f0 0x7efdac1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 0x7efdac144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdac0607d0 con 0x7efdac0b17b0 2021-04-03T03:17:40.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdac1451c0 con 0x7efdac0603f0 2021-04-03T03:17:40.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac14b090 con 0x7efdac0add50 2021-04-03T03:17:40.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efdac0603f0 0x7efdac1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:40.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7efdac0603f0 0x7efdac1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59182/0 (socket says 172.21.2.111:59182) 2021-04-03T03:17:40.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 -- 172.21.2.111:0/1697077227 learned_addr learned my addr 172.21.2.111:0/1697077227 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:40.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 -- 172.21.2.111:0/1697077227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 msgr2=0x7efdac144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:40.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 --2- 172.21.2.111:0/1697077227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 0x7efdac144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 -- 172.21.2.111:0/1697077227 >> v1:172.21.2.115:6789/0 conn(0x7efdac0b17b0 legacy=0x7efdac147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:40.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 -- 172.21.2.111:0/1697077227 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac14b330 con 0x7efdac0603f0 2021-04-03T03:17:40.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdaa59c700 1 --2- 172.21.2.111:0/1697077227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac1447e0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7efdac0b5600 tx=0x7efd94006e90).ready entity=mon.0 client_cookie=71ba646b4145c67a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:40.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efda959a700 1 -- 172.21.2.111:0/1697077227 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd940098e0 con 0x7efdac0603f0 2021-04-03T03:17:40.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efda959a700 1 -- 172.21.2.111:0/1697077227 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94009a40 con 0x7efdac0603f0 2021-04-03T03:17:40.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efda959a700 1 -- 172.21.2.111:0/1697077227 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd94009de0 con 0x7efdac0603f0 2021-04-03T03:17:40.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 -- 172.21.2.111:0/1697077227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 msgr2=0x7efdac1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 --2- 172.21.2.111:0/1697077227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac1447e0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7efdac0b5600 tx=0x7efd94006e90).stop 2021-04-03T03:17:40.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 -- 172.21.2.111:0/1697077227 shutdown_connections 2021-04-03T03:17:40.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 --2- 172.21.2.111:0/1697077227 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac1447e0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 --2- 172.21.2.111:0/1697077227 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 0x7efdac144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.253+0000 7efdab59e700 1 -- 172.21.2.111:0/1697077227 >> 172.21.2.111:0/1697077227 conn(0x7efdac0b61c0 msgr2=0x7efdac05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:40.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- 172.21.2.111:0/1697077227 shutdown_connections 2021-04-03T03:17:40.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- 172.21.2.111:0/1697077227 wait complete. 2021-04-03T03:17:40.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 Processor -- start 2021-04-03T03:17:40.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- start start 2021-04-03T03:17:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac066220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 0x7efdac15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efdac0b17b0 0x7efdac0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac148580 con 0x7efdac0603f0 2021-04-03T03:17:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac15a370 con 0x7efdac0b17b0 2021-04-03T03:17:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac15a610 con 0x7efdac0add50 2021-04-03T03:17:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:40.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac066220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59190/0 (socket says 172.21.2.111:59190) 2021-04-03T03:17:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 -- 172.21.2.111:0/2928215527 learned_addr learned my addr 172.21.2.111:0/2928215527 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 -- 172.21.2.111:0/2928215527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 msgr2=0x7efdac15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 0x7efdac15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 -- 172.21.2.111:0/2928215527 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efdac0b17b0 msgr2=0x7efdac0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efdac0b17b0 0x7efdac0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 -- 172.21.2.111:0/2928215527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac15a8b0 con 0x7efdac0603f0 2021-04-03T03:17:40.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdaa59c700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac066220 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7efdac0b5600 tx=0x7efd94015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:40.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda37fe700 1 -- 172.21.2.111:0/2928215527 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd94011e30 con 0x7efdac0603f0 2021-04-03T03:17:40.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda37fe700 1 -- 172.21.2.111:0/2928215527 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9401e460 con 0x7efdac0603f0 2021-04-03T03:17:40.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac15b240 con 0x7efdac0603f0 2021-04-03T03:17:40.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda37fe700 1 -- 172.21.2.111:0/2928215527 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd9401e7c0 con 0x7efdac0603f0 2021-04-03T03:17:40.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac147c50 con 0x7efdac0603f0 2021-04-03T03:17:40.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac057660 con 0x7efdac0603f0 2021-04-03T03:17:40.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda37fe700 1 -- 172.21.2.111:0/2928215527 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7efd94025030 con 0x7efdac0603f0 2021-04-03T03:17:40.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda37fe700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd90056ea0 0x7efd90059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:40.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda37fe700 1 -- 172.21.2.111:0/2928215527 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6416+0+0 (secure 0 0 0) 0x7efd94071580 con 0x7efdac0603f0 2021-04-03T03:17:40.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda9d9b700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd90056ea0 0x7efd90059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:40.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.257+0000 7efda9d9b700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd90056ea0 0x7efd90059360 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7efd9c002f30 tx=0x7efd9c006a90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:40.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.261+0000 7efda37fe700 1 -- 172.21.2.111:0/2928215527 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7efd94046970 con 0x7efdac0603f0 2021-04-03T03:17:40.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.385+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efdac0af5a0 con 0x7efdac0603f0 2021-04-03T03:17:40.975 INFO:tasks.ceph.mon.a.gibba011.stderr:2021-04-03T03:17:40.973+0000 7f88ccbfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-04-03T03:17:40.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.981+0000 7efda37fe700 1 -- 172.21.2.111:0/2928215527 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7efd94023090 con 0x7efdac0603f0 2021-04-03T03:17:40.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd90056ea0 msgr2=0x7efd90059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd90056ea0 0x7efd90059360 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7efd9c002f30 tx=0x7efd9c006a90).stop 2021-04-03T03:17:40.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 msgr2=0x7efdac066220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:40.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac066220 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7efdac0b5600 tx=0x7efd94015000).stop 2021-04-03T03:17:40.988 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 shutdown_connections 2021-04-03T03:17:40.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7efd90056ea0 0x7efd90059360 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7efdac0b17b0 0x7efdac0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7efdac0603f0 0x7efdac066220 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:40.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 --2- 172.21.2.111:0/2928215527 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7efdac0add50 0x7efdac15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:40.989 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 >> 172.21.2.111:0/2928215527 conn(0x7efdac0b61c0 msgr2=0x7efdac143810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:40.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 shutdown_connections 2021-04-03T03:17:40.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:40.985+0000 7efdab59e700 1 -- 172.21.2.111:0/2928215527 wait complete. 2021-04-03T03:17:40.990 INFO:teuthology.orchestra.run.gibba011.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-04-03T03:17:40.994 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2021-04-03T03:17:40.998 INFO:tasks.ceph.mds.a.gibba011.stdout:starting mds.a at 2021-04-03T03:17:41.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7b267700 1 Processor -- start 2021-04-03T03:17:41.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7b267700 1 -- start start 2021-04-03T03:17:41.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7b267700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740b12b0 0x7f7c741522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7b267700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740607d0 con 0x7f7c740603f0 2021-04-03T03:17:41.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7b267700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740b0eb0 con 0x7f7c7414b700 2021-04-03T03:17:41.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7b267700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740af400 con 0x7f7c740b12b0 2021-04-03T03:17:41.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7a265700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7c740603f0 0x7f7c740b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:59998/0 (socket says 172.21.2.111:59998) 2021-04-03T03:17:41.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7a265700 1 -- 172.21.2.111:0/1587715155 learned_addr learned my addr 172.21.2.111:0/1587715155 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:41.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981524939 0 0) 0x7f7c740607d0 con 0x7f7c740603f0 2021-04-03T03:17:41.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c5c003240 con 0x7f7c740603f0 2021-04-03T03:17:41.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7aa66700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740b12b0 0x7f7c741522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3873755452 0 0) 0x7f7c5c003240 con 0x7f7c740603f0 2021-04-03T03:17:41.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740b12b0 msgr2=0x7f7c741522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:41.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740b12b0 0x7f7c741522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 >> v1:172.21.2.115:6789/0 conn(0x7f7c7414b700 legacy=0x7f7c740b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740af6a0 con 0x7f7c740603f0 2021-04-03T03:17:41.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f7c680016d0 con 0x7f7c740603f0 2021-04-03T03:17:41.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 >> v1:172.21.2.111:6789/0 conn(0x7f7c740603f0 legacy=0x7f7c740b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c5c003760 0x7f7c5c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c5c0044e0 0x7f7c5c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c5c004e50 0x7f7c5c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740af400 con 0x7f7c5c004e50 2021-04-03T03:17:41.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c0041f0 con 0x7f7c5c003760 2021-04-03T03:17:41.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c00bc20 con 0x7f7c5c0044e0 2021-04-03T03:17:41.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7aa66700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c5c004e50 0x7f7c5c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7aa66700 1 -- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c5c0044e0 msgr2=0x7f7c5c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:41.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7aa66700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c5c0044e0 0x7f7c5c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:41.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7aa66700 1 -- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c5c003760 msgr2=0x7f7c5c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:41.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.049+0000 7f7c7aa66700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c5c003760 0x7f7c5c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:41.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7aa66700 1 -- 172.21.2.111:0/1587715155 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740af6a0 con 0x7f7c5c004e50 2021-04-03T03:17:41.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7aa66700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c5c004e50 0x7f7c5c00b730 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f7c6c00c000 tx=0x7f7c6c0091d0).ready entity=mon.0 client_cookie=c0f105631b708e04 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:41.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c6c00ec30 con 0x7f7c5c004e50 2021-04-03T03:17:41.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c00ed90 con 0x7f7c5c004e50 2021-04-03T03:17:41.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c79263700 1 -- 172.21.2.111:0/1587715155 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c6c0125e0 con 0x7f7c5c004e50 2021-04-03T03:17:41.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/1587715155 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c5c004e50 msgr2=0x7f7c5c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 --2- 172.21.2.111:0/1587715155 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c5c004e50 0x7f7c5c00b730 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f7c6c00c000 tx=0x7f7c6c0091d0).stop 2021-04-03T03:17:41.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7aa66700 1 -- 172.21.2.111:0/1587715155 reap_dead start 2021-04-03T03:17:41.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/1587715155 shutdown_connections 2021-04-03T03:17:41.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/1587715155 >> 172.21.2.111:0/1587715155 conn(0x7f7c740b61c0 msgr2=0x7f7c7405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:41.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/1587715155 shutdown_connections 2021-04-03T03:17:41.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/1587715155 wait complete. 2021-04-03T03:17:41.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 Processor -- start 2021-04-03T03:17:41.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- start start 2021-04-03T03:17:41.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c74149ed0 0x7f7c740ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740a8330 0x7f7c740a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c74156d40 0x7f7c741455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7414a2b0 con 0x7f7c74149ed0 2021-04-03T03:17:41.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74156600 con 0x7f7c74156d40 2021-04-03T03:17:41.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741568a0 con 0x7f7c740a8330 2021-04-03T03:17:41.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c74149ed0 0x7f7c740ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c74149ed0 0x7f7c740ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59218/0 (socket says 172.21.2.111:59218) 2021-04-03T03:17:41.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 -- 172.21.2.111:0/4109494046 learned_addr learned my addr 172.21.2.111:0/4109494046 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:41.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c79a64700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740a8330 0x7f7c740a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 -- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740a8330 msgr2=0x7f7c740a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7aa66700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c74156d40 0x7f7c741455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740a8330 0x7f7c740a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 -- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c74156d40 msgr2=0x7f7c741455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c74156d40 0x7f7c741455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 -- 172.21.2.111:0/4109494046 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c741562f0 con 0x7f7c74149ed0 2021-04-03T03:17:41.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7a265700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c74149ed0 0x7f7c740ab9c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 rx=0x7f7c680013e0 tx=0x7f7c68001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:41.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c677fe700 1 -- 172.21.2.111:0/4109494046 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c68007170 con 0x7f7c74149ed0 2021-04-03T03:17:41.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c677fe700 1 -- 172.21.2.111:0/4109494046 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c680072d0 con 0x7f7c74149ed0 2021-04-03T03:17:41.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c740a98b0 con 0x7f7c74149ed0 2021-04-03T03:17:41.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74157380 con 0x7f7c74149ed0 2021-04-03T03:17:41.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c677fe700 1 -- 172.21.2.111:0/4109494046 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c6800a9e0 con 0x7f7c74149ed0 2021-04-03T03:17:41.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.053+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74156a60 con 0x7f7c74149ed0 2021-04-03T03:17:41.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.057+0000 7f7c677fe700 1 -- 172.21.2.111:0/4109494046 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7c6801b060 con 0x7f7c74149ed0 2021-04-03T03:17:41.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.057+0000 7f7c677fe700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c54056ea0 0x7f7c54059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.057+0000 7f7c677fe700 1 -- 172.21.2.111:0/4109494046 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f7c6806ff30 con 0x7f7c74149ed0 2021-04-03T03:17:41.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.057+0000 7f7c79a64700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c54056ea0 0x7f7c54059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.057+0000 7f7c79a64700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c54056ea0 0x7f7c54059360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f7c70008000 tx=0x7f7c70009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:41.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.057+0000 7f7c677fe700 1 -- 172.21.2.111:0/4109494046 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f7c680451b0 con 0x7f7c74149ed0 2021-04-03T03:17:41.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.181+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c74057660 con 0x7f7c74149ed0 2021-04-03T03:17:41.186 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c677fe700 1 -- 172.21.2.111:0/4109494046 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f7c68019030 con 0x7f7c74149ed0 2021-04-03T03:17:41.187 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T03:17:41.187 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":69,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.115:6833/3455926982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6832","nonce":3455926982},{"type":"v1","addr":"172.21.2.115:6833","nonce":3455926982}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-04-03T03:17:33.145029+0000","modified":"2021-04-03T03:17:40.974555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":152,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-04-03T03:17:41.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c54056ea0 msgr2=0x7f7c54059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c54056ea0 0x7f7c54059360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f7c70008000 tx=0x7f7c70009040).stop 2021-04-03T03:17:41.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c74149ed0 msgr2=0x7f7c740ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c74149ed0 0x7f7c740ab9c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 rx=0x7f7c680013e0 tx=0x7f7c68001b50).stop 2021-04-03T03:17:41.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 shutdown_connections 2021-04-03T03:17:41.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7c54056ea0 0x7f7c54059360 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7c74156d40 0x7f7c741455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f7c74149ed0 0x7f7c740ab9c0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 --2- 172.21.2.111:0/4109494046 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7c740a8330 0x7f7c740a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 >> 172.21.2.111:0/4109494046 conn(0x7f7c740b61c0 msgr2=0x7f7c74147680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:41.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 shutdown_connections 2021-04-03T03:17:41.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.185+0000 7f7c7b267700 1 -- 172.21.2.111:0/4109494046 wait complete. 2021-04-03T03:17:41.191 INFO:teuthology.orchestra.run.gibba011.stderr:dumped fsmap epoch 69 2021-04-03T03:17:41.195 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2021-04-03T03:17:41.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3d4f5d700 1 Processor -- start 2021-04-03T03:17:41.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3d4f5d700 1 -- start start 2021-04-03T03:17:41.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3d4f5d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00603f0 0x7fd3d01447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3d4f5d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd3d00add50 0x7fd3d0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3d4f5d700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d00607d0 con 0x7fd3d00b17b0 2021-04-03T03:17:41.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3d4f5d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d01451c0 con 0x7fd3d00add50 2021-04-03T03:17:41.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3d4f5d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d014b090 con 0x7fd3d00603f0 2021-04-03T03:17:41.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3cf7fe700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00603f0 0x7fd3d01447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3ceffd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd3d00add50 0x7fd3d0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3ceffd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd3d00add50 0x7fd3d0144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3300/0 says I am v2:172.21.2.111:48936/0 (socket says 172.21.2.111:48936) 2021-04-03T03:17:41.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.301+0000 7fd3ceffd700 1 -- 172.21.2.111:0/3354546509 learned_addr learned my addr 172.21.2.111:0/3354546509 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:41.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ceffd700 1 -- 172.21.2.111:0/3354546509 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00603f0 msgr2=0x7fd3d01447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:41.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ceffd700 1 --2- 172.21.2.111:0/3354546509 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00603f0 0x7fd3d01447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ceffd700 1 -- 172.21.2.111:0/3354546509 >> v1:172.21.2.115:6789/0 conn(0x7fd3d00b17b0 legacy=0x7fd3d0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ceffd700 1 -- 172.21.2.111:0/3354546509 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d014b330 con 0x7fd3d00add50 2021-04-03T03:17:41.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ceffd700 1 --2- 172.21.2.111:0/3354546509 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00add50 0x7fd3d0144cd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fd3b8002f30 tx=0x7fd3b8006e90).ready entity=mon.1 client_cookie=30d804cca8e6be03 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:41.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ce7fc700 1 -- 172.21.2.111:0/3354546509 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b8009850 con 0x7fd3d00add50 2021-04-03T03:17:41.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ce7fc700 1 -- 172.21.2.111:0/3354546509 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b80099b0 con 0x7fd3d00add50 2021-04-03T03:17:41.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ce7fc700 1 -- 172.21.2.111:0/3354546509 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b8009d10 con 0x7fd3d00add50 2021-04-03T03:17:41.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/3354546509 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00add50 msgr2=0x7fd3d0144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/3354546509 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00add50 0x7fd3d0144cd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fd3b8002f30 tx=0x7fd3b8006e90).stop 2021-04-03T03:17:41.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/3354546509 shutdown_connections 2021-04-03T03:17:41.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/3354546509 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00603f0 0x7fd3d01447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/3354546509 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00add50 0x7fd3d0144cd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/3354546509 >> 172.21.2.111:0/3354546509 conn(0x7fd3d00b61c0 msgr2=0x7fd3d005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/3354546509 shutdown_connections 2021-04-03T03:17:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/3354546509 wait complete. 2021-04-03T03:17:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 Processor -- start 2021-04-03T03:17:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- start start 2021-04-03T03:17:41.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00603f0 0x7fd3d00a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00add50 0x7fd3d00a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3d00b17b0 0x7fd3d0064980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d01437d0 con 0x7fd3d00b17b0 2021-04-03T03:17:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00a9750 con 0x7fd3d00603f0 2021-04-03T03:17:41.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3d4f5d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00ab970 con 0x7fd3d00add50 2021-04-03T03:17:41.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3d00b17b0 0x7fd3d0064980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3d00b17b0 0x7fd3d0064980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59234/0 (socket says 172.21.2.111:59234) 2021-04-03T03:17:41.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 -- 172.21.2.111:0/814020626 learned_addr learned my addr 172.21.2.111:0/814020626 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:41.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cf7fe700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00603f0 0x7fd3d00a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3ceffd700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00add50 0x7fd3d00a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 -- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00add50 msgr2=0x7fd3d00a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00add50 0x7fd3d00a8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 -- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00603f0 msgr2=0x7fd3d00a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00603f0 0x7fd3d00a76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 -- 172.21.2.111:0/814020626 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d00aa0e0 con 0x7fd3d00b17b0 2021-04-03T03:17:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.305+0000 7fd3cffff700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3d00b17b0 0x7fd3d0064980 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fd3d00abab0 tx=0x7fd3c8007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3ccff9700 1 -- 172.21.2.111:0/814020626 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3c800a520 con 0x7fd3d00b17b0 2021-04-03T03:17:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3ccff9700 1 -- 172.21.2.111:0/814020626 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c800a680 con 0x7fd3d00b17b0 2021-04-03T03:17:41.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d00aa380 con 0x7fd3d00b17b0 2021-04-03T03:17:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d00abb90 con 0x7fd3d00b17b0 2021-04-03T03:17:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3ccff9700 1 -- 172.21.2.111:0/814020626 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3c800a7f0 con 0x7fd3d00b17b0 2021-04-03T03:17:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d00ab450 con 0x7fd3d00b17b0 2021-04-03T03:17:41.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3ccff9700 1 -- 172.21.2.111:0/814020626 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd3c8010070 con 0x7fd3d00b17b0 2021-04-03T03:17:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3ccff9700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3b4056f70 0x7fd3b4059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3ccff9700 1 -- 172.21.2.111:0/814020626 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fd3c806fe70 con 0x7fd3d00b17b0 2021-04-03T03:17:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3cf7fe700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3b4056f70 0x7fd3b4059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:41.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.309+0000 7fd3cf7fe700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3b4056f70 0x7fd3b4059430 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fd3c000b000 tx=0x7fd3c0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:41.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.313+0000 7fd3ccff9700 1 -- 172.21.2.111:0/814020626 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd3c8045090 con 0x7fd3d00b17b0 2021-04-03T03:17:41.438 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.437+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd3d00aa510 con 0x7fd3d00b17b0 2021-04-03T03:17:41.990 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3ccff9700 1 -- 172.21.2.111:0/814020626 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd3c80486b0 con 0x7fd3d00b17b0 2021-04-03T03:17:41.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3b4056f70 msgr2=0x7fd3b4059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3b4056f70 0x7fd3b4059430 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fd3c000b000 tx=0x7fd3c0008040).stop 2021-04-03T03:17:41.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3d00b17b0 msgr2=0x7fd3d0064980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:41.992 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3d00b17b0 0x7fd3d0064980 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fd3d00abab0 tx=0x7fd3c8007f30).stop 2021-04-03T03:17:41.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 shutdown_connections 2021-04-03T03:17:41.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd3d00b17b0 0x7fd3d0064980 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd3b4056f70 0x7fd3b4059430 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd3d00603f0 0x7fd3d00a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.993 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 --2- 172.21.2.111:0/814020626 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd3d00add50 0x7fd3d00a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:41.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 >> 172.21.2.111:0/814020626 conn(0x7fd3d00b61c0 msgr2=0x7fd3d00b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:41.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 shutdown_connections 2021-04-03T03:17:41.994 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:41.989+0000 7fd3d4f5d700 1 -- 172.21.2.111:0/814020626 wait complete. 2021-04-03T03:17:41.999 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-04-03T03:17:42.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef27c6700 1 Processor -- start 2021-04-03T03:17:42.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef27c6700 1 -- start start 2021-04-03T03:17:42.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef27c6700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0ee40423a0 0x7f0ee403c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef27c6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 0x7f0ee4005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef27c6700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee4043400 con 0x7f0ee4047580 2021-04-03T03:17:42.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef27c6700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee4047960 con 0x7f0ee40423a0 2021-04-03T03:17:42.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef27c6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4042ec0 con 0x7f0ee40d38b0 2021-04-03T03:17:42.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef17c4700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0ee4047580 0x7f0ee403bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60034/0 (socket says 172.21.2.111:60034) 2021-04-03T03:17:42.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef17c4700 1 -- 172.21.2.111:0/2602356441 learned_addr learned my addr 172.21.2.111:0/2602356441 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T03:17:42.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289248626 0 0) 0x7f0ee4043400 con 0x7f0ee4047580 2021-04-03T03:17:42.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed4003240 con 0x7f0ee4047580 2021-04-03T03:17:42.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef1fc5700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 0x7f0ee4005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:42.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0ef0fc3700 1 --2- 172.21.2.111:0/2602356441 >> v2:172.21.2.115:3300/0 conn(0x7f0ee40423a0 0x7f0ee403c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:42.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2865597752 0 0) 0x7f0ed4003240 con 0x7f0ee4047580 2021-04-03T03:17:42.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 msgr2=0x7f0ee4005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:42.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 0x7f0ee4005ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:42.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 >> v2:172.21.2.115:3300/0 conn(0x7f0ee40423a0 msgr2=0x7f0ee403c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:42.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 --2- 172.21.2.111:0/2602356441 >> v2:172.21.2.115:3300/0 conn(0x7f0ee40423a0 0x7f0ee403c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:42.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4043160 con 0x7f0ee4047580 2021-04-03T03:17:42.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f0ee80016d0 con 0x7f0ee4047580 2021-04-03T03:17:42.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.053+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 >> v1:172.21.2.111:6789/0 conn(0x7f0ee4047580 legacy=0x7f0ee403bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:42.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 0x7f0ed4003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ed4003c40 0x7f0ed40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ed40049d0 0x7f0ed4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4047960 con 0x7f0ed4003c40 2021-04-03T03:17:42.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4042ec0 con 0x7f0ed40049d0 2021-04-03T03:17:42.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4004200 con 0x7f0ee40d38b0 2021-04-03T03:17:42.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef0fc3700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ed4003c40 0x7f0ed40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:42.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef17c4700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 0x7f0ed4003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:42.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef0fc3700 1 -- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 msgr2=0x7f0ed4003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:42.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef0fc3700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40d38b0 0x7f0ed4003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:42.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef0fc3700 1 -- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ed40049d0 msgr2=0x7f0ed4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:42.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef0fc3700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ed40049d0 0x7f0ed4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:42.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef0fc3700 1 -- 172.21.2.111:0/2602356441 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed4003240 con 0x7f0ed4003c40 2021-04-03T03:17:42.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef0fc3700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ed4003c40 0x7f0ed40044e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7f0edc007000 tx=0x7f0edc008040).ready entity=mon.0 client_cookie=2ab5e90f6ed8fc17 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:42.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0edc00a8f0 con 0x7f0ed4003c40 2021-04-03T03:17:42.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc00c030 con 0x7f0ed4003c40 2021-04-03T03:17:42.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0edbfff700 1 -- 172.21.2.111:0/2602356441 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0edc00ac40 con 0x7f0ed4003c40 2021-04-03T03:17:42.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/2602356441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ed4003c40 msgr2=0x7f0ed40044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:42.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 --2- 172.21.2.111:0/2602356441 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ed4003c40 0x7f0ed40044e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7f0edc007000 tx=0x7f0edc008040).stop 2021-04-03T03:17:42.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 -- 172.21.2.111:0/2602356441 reap_dead start 2021-04-03T03:17:42.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/2602356441 shutdown_connections 2021-04-03T03:17:42.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/2602356441 >> 172.21.2.111:0/2602356441 conn(0x7f0ee400d9a0 msgr2=0x7f0ee4046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:42.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/2602356441 shutdown_connections 2021-04-03T03:17:42.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/2602356441 wait complete. 2021-04-03T03:17:42.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 Processor -- start 2021-04-03T03:17:42.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- start start 2021-04-03T03:17:42.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ee40ca650 0x7f0ee40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40cb1e0 0x7f0ee40ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ee40cf3b0 0x7f0ee40d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4040160 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee40caa30 con 0x7f0ee40ca650 2021-04-03T03:17:42.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4040390 con 0x7f0ee40cb1e0 2021-04-03T03:17:42.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ee40cf3b0 0x7f0ee40d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:42.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ee40cf3b0 0x7f0ee40d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59258/0 (socket says 172.21.2.111:59258) 2021-04-03T03:17:42.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 -- 172.21.2.111:0/1672648659 learned_addr learned my addr 172.21.2.111:0/1672648659 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:42.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 -- 172.21.2.111:0/1672648659 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40cb1e0 msgr2=0x7f0ee40ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:42.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40cb1e0 0x7f0ee40ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:42.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 -- 172.21.2.111:0/1672648659 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ee40ca650 msgr2=0x7f0ee40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:42.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ee40ca650 0x7f0ee40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:42.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 -- 172.21.2.111:0/1672648659 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4040630 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef1fc5700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ee40cf3b0 0x7f0ee40d17b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f0eec05dc50 tx=0x7f0eec064040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:42.076 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0eda7fc700 1 -- 172.21.2.111:0/1672648659 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eec05bda0 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0eda7fc700 1 -- 172.21.2.111:0/1672648659 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec05df60 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee40c5610 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0eda7fc700 1 -- 172.21.2.111:0/1672648659 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eec05f4b0 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee40cda70 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.077 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.057+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee40407c0 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.061+0000 7f0eda7fc700 1 -- 172.21.2.111:0/1672648659 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f0eec05f610 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.061+0000 7f0eda7fc700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0ecc056f20 0x7f0ecc0593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:42.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.061+0000 7f0eda7fc700 1 -- 172.21.2.111:0/1672648659 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0eec0df1d0 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.061+0000 7f0ef17c4700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0ecc056f20 0x7f0ecc0593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:42.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.061+0000 7f0ef17c4700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0ecc056f20 0x7f0ecc0593e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f0ee800b000 tx=0x7f0ee40432f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:42.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.061+0000 7f0eda7fc700 1 -- 172.21.2.111:0/1672648659 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f0eec0b4560 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.157+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ee4004db0 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.997 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0eda7fc700 1 -- 172.21.2.111:0/1672648659 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 185+0+0 (secure 0 0 0) 0x7f0eec0b7b80 con 0x7f0ee40cf3b0 2021-04-03T03:17:42.998 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0ecc056f20 msgr2=0x7f0ecc0593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:42.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0ecc056f20 0x7f0ecc0593e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f0ee800b000 tx=0x7f0ee40432f0).stop 2021-04-03T03:17:42.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ee40cf3b0 msgr2=0x7f0ee40d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:42.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ee40cf3b0 0x7f0ee40d17b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f0eec05dc50 tx=0x7f0eec064040).stop 2021-04-03T03:17:42.999 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 shutdown_connections 2021-04-03T03:17:43.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0ee40cf3b0 0x7f0ee40d17b0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0ecc056f20 0x7f0ecc0593e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0ee40ca650 0x7f0ee40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:43.000 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.993+0000 7f0ef27c6700 1 --2- 172.21.2.111:0/1672648659 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f0ee40cb1e0 0x7f0ee40ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:43.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.997+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 >> 172.21.2.111:0/1672648659 conn(0x7f0ee400d9a0 msgr2=0x7f0ee40c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:43.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.997+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 shutdown_connections 2021-04-03T03:17:43.001 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:42.997+0000 7f0ef27c6700 1 -- 172.21.2.111:0/1672648659 wait complete. 2021-04-03T03:17:43.001 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_metadata' removed 2021-04-03T03:17:43.005 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-04-03T03:17:43.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 Processor -- start 2021-04-03T03:17:43.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- start start 2021-04-03T03:17:43.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:43.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1cc041060 0x7fe1cc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:43.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fe1cc03d560 0x7fe1cc041650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:43.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc03de60 con 0x7fe1cc041060 2021-04-03T03:17:43.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0dc560 con 0x7fe1cc03d560 2021-04-03T03:17:43.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0dc800 con 0x7fe1cc0dbba0 2021-04-03T03:17:43.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1cc041060 0x7fe1cc03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:43.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fe1cc041060 0x7fe1cc03fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59266/0 (socket says 172.21.2.111:59266) 2021-04-03T03:17:43.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 -- 172.21.2.111:0/1739761037 learned_addr learned my addr 172.21.2.111:0/1739761037 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:43.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1da3c2700 1 --2- 172.21.2.111:0/1739761037 >> v2:172.21.2.115:3300/0 conn(0x7fe1cc03d560 0x7fe1cc041650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:43.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d9bc1700 1 --2- 172.21.2.111:0/1739761037 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:43.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 -- 172.21.2.111:0/1739761037 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 msgr2=0x7fe1cc0dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:43.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 --2- 172.21.2.111:0/1739761037 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 -- 172.21.2.111:0/1739761037 >> v2:172.21.2.115:3300/0 conn(0x7fe1cc03d560 msgr2=0x7fe1cc041650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T03:17:43.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 --2- 172.21.2.111:0/1739761037 >> v2:172.21.2.115:3300/0 conn(0x7fe1cc03d560 0x7fe1cc041650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 -- 172.21.2.111:0/1739761037 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc03da30 con 0x7fe1cc041060 2021-04-03T03:17:43.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d93c0700 1 --2- 172.21.2.111:0/1739761037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc041060 0x7fe1cc03fa80 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fe1c4002f30 tx=0x7fe1c4006e90).ready entity=mon.0 client_cookie=890e1b0955ebc3ba server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:43.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d8bbf700 1 -- 172.21.2.111:0/1739761037 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1c4009920 con 0x7fe1cc041060 2021-04-03T03:17:43.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d8bbf700 1 -- 172.21.2.111:0/1739761037 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c4009a80 con 0x7fe1cc041060 2021-04-03T03:17:43.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1d8bbf700 1 -- 172.21.2.111:0/1739761037 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1c4009de0 con 0x7fe1cc041060 2021-04-03T03:17:43.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- 172.21.2.111:0/1739761037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc041060 msgr2=0x7fe1cc03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:43.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/1739761037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc041060 0x7fe1cc03fa80 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fe1c4002f30 tx=0x7fe1c4006e90).stop 2021-04-03T03:17:43.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- 172.21.2.111:0/1739761037 shutdown_connections 2021-04-03T03:17:43.074 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/1739761037 >> v2:172.21.2.115:3300/0 conn(0x7fe1cc03d560 0x7fe1cc041650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/1739761037 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/1739761037 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc041060 0x7fe1cc03fa80 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- 172.21.2.111:0/1739761037 >> 172.21.2.111:0/1739761037 conn(0x7fe1cc0462b0 msgr2=0x7fe1cc0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:43.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- 172.21.2.111:0/1739761037 shutdown_connections 2021-04-03T03:17:43.075 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 -- 172.21.2.111:0/1739761037 wait complete. 2021-04-03T03:17:43.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.065+0000 7fe1dabc3700 1 Processor -- start 2021-04-03T03:17:43.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 -- start start 2021-04-03T03:17:43.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc03d560 0x7fe1cc045770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:43.078 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1cc041060 0x7fe1cc0d4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:43.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0d4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:43.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0396a0 con 0x7fe1cc03d560 2021-04-03T03:17:43.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc039b80 con 0x7fe1cc041060 2021-04-03T03:17:43.079 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc039e20 con 0x7fe1cc0dbba0 2021-04-03T03:17:43.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc03d560 0x7fe1cc045770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:43.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc03d560 0x7fe1cc045770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59270/0 (socket says 172.21.2.111:59270) 2021-04-03T03:17:43.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 -- 172.21.2.111:0/3372988768 learned_addr learned my addr 172.21.2.111:0/3372988768 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T03:17:43.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1da3c2700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:43.080 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 -- 172.21.2.111:0/3372988768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 msgr2=0x7fe1cc0d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:43.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0d4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:43.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 -- 172.21.2.111:0/3372988768 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1cc041060 msgr2=0x7fe1cc0d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:43.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1cc041060 0x7fe1cc0d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:43.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 -- 172.21.2.111:0/3372988768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0398c0 con 0x7fe1cc03d560 2021-04-03T03:17:43.081 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d9bc1700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc03d560 0x7fe1cc045770 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7fe1d0007000 tx=0x7fe1d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:43.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1c2ffd700 1 -- 172.21.2.111:0/3372988768 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1d000e5e0 con 0x7fe1cc03d560 2021-04-03T03:17:43.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1c2ffd700 1 -- 172.21.2.111:0/3372988768 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d000c030 con 0x7fe1cc03d560 2021-04-03T03:17:43.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1c2ffd700 1 -- 172.21.2.111:0/3372988768 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1d000e930 con 0x7fe1cc03d560 2021-04-03T03:17:43.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0e7130 con 0x7fe1cc03d560 2021-04-03T03:17:43.082 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc045df0 con 0x7fe1cc03d560 2021-04-03T03:17:43.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc0dce80 con 0x7fe1cc03d560 2021-04-03T03:17:43.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1c2ffd700 1 -- 172.21.2.111:0/3372988768 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe1d0019030 con 0x7fe1cc03d560 2021-04-03T03:17:43.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1c2ffd700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe1b4056fb0 0x7fe1b4059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T03:17:43.083 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1c2ffd700 1 -- 172.21.2.111:0/3372988768 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6064+0+0 (secure 0 0 0) 0x7fe1d006f860 con 0x7fe1cc03d560 2021-04-03T03:17:43.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d93c0700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe1b4056fb0 0x7fe1b4059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T03:17:43.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.069+0000 7fe1d93c0700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe1b4056fb0 0x7fe1b4059470 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fe1c4002f30 tx=0x7fe1c4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T03:17:43.084 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.073+0000 7fe1c2ffd700 1 -- 172.21.2.111:0/3372988768 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe1d0042c50 con 0x7fe1cc03d560 2021-04-03T03:17:43.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:43.165+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe1cc004db0 con 0x7fe1cc03d560 2021-04-03T03:17:44.008 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1c2ffd700 1 -- 172.21.2.111:0/3372988768 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe1d0044e60 con 0x7fe1cc03d560 2021-04-03T03:17:44.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe1b4056fb0 msgr2=0x7fe1b4059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:44.010 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe1b4056fb0 0x7fe1b4059470 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fe1c4002f30 tx=0x7fe1c4015000).stop 2021-04-03T03:17:44.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc03d560 msgr2=0x7fe1cc045770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T03:17:44.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc03d560 0x7fe1cc045770 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7fe1d0007000 tx=0x7fe1d0008040).stop 2021-04-03T03:17:44.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 shutdown_connections 2021-04-03T03:17:44.011 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe1cc0dbba0 0x7fe1cc0d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:44.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe1cc03d560 0x7fe1cc045770 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:44.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe1b4056fb0 0x7fe1b4059470 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T03:17:44.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 --2- 172.21.2.111:0/3372988768 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe1cc041060 0x7fe1cc0d4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T03:17:44.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 >> 172.21.2.111:0/3372988768 conn(0x7fe1cc0462b0 msgr2=0x7fe1cc03efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T03:17:44.012 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 shutdown_connections 2021-04-03T03:17:44.013 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T03:17:44.005+0000 7fe1dabc3700 1 -- 172.21.2.111:0/3372988768 wait complete. 2021-04-03T03:17:44.013 INFO:teuthology.orchestra.run.gibba011.stderr:pool 'cephfs_data' removed 2021-04-03T03:17:44.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2021-04-03T03:17:44.018 DEBUG:teuthology.orchestra.run.gibba026:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2021-04-03T03:17:59.379 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:59.382 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:59.385 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:59.388 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:17:59.391 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:14.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:18:14.123 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:18:14.124 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:18:29.424 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:29.427 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:29.430 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:29.434 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:29.437 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:44.024 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-04-03T03:18:44.032 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-04-03T03:18:59.465 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:59.469 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:59.472 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:59.475 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:18:59.478 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:29.508 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:29.511 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:29.514 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:29.518 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:29.521 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:59.565 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:59.569 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:59.572 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:59.576 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:19:59.579 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:29.610 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:29.614 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:29.617 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:29.620 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:29.623 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:59.654 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:59.657 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:59.661 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:59.663 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:20:59.667 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:29.695 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:29.699 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:29.702 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:29.705 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:29.708 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:59.737 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:59.741 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:59.744 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:59.747 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:21:59.751 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:29.780 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:29.784 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:29.787 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:29.790 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:29.793 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:59.842 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:59.847 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:59.850 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:59.854 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:22:59.857 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:29.888 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:29.892 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:29.895 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:29.898 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:29.901 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:59.931 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:59.938 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:59.942 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:59.945 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:23:59.948 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:24:29.981 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:24:29.984 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:24:29.987 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:24:29.991 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:24:29.993 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:00.024 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:00.028 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:00.031 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:00.033 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:00.035 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:30.092 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:30.096 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:30.098 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:30.101 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:25:30.103 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:00.149 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:00.152 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:00.154 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:00.157 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:00.159 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:30.214 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:30.217 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:30.221 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:30.224 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:26:30.227 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:00.254 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:00.257 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:00.259 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:00.261 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:00.265 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:30.294 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:30.298 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:30.301 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:30.304 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:27:30.307 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:00.357 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:00.361 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:00.364 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:00.368 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:00.370 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:30.403 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:30.406 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:30.408 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:30.412 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:28:30.415 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:00.446 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:00.450 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:00.453 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:00.456 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:00.458 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:30.488 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:30.492 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:30.495 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:30.498 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:29:30.501 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:00.547 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:00.551 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:00.554 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:00.557 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:00.560 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:30.591 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:30.594 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:30.598 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:30.601 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:30:30.605 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:00.632 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:00.636 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:00.639 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:00.642 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:00.646 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:30.683 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:30.687 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:30.690 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:30.693 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:31:30.697 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:00.741 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:00.856 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:00.859 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:00.862 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:00.866 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:30.893 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:30.897 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:30.900 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:30.903 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:32:30.906 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:00.961 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:00.964 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:00.968 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:00.971 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:00.974 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:14.127 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-04-03T03:33:14.133 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-04-03T03:33:14.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:33:14.136 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-04-03T03:33:14.137 DEBUG:teuthology.orchestra.run.gibba011:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:33:14.149 DEBUG:teuthology.orchestra.run.gibba015:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:33:14.162 DEBUG:teuthology.orchestra.run.gibba026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:33:14.174 DEBUG:teuthology.orchestra.run.gibba028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:33:14.186 DEBUG:teuthology.orchestra.run.gibba031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-04-03T03:33:14.199 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-04-03T03:33:14.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:33:14.200 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:33:14.200 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:33:31.020 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:31.024 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:31.027 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:31.030 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:33:31.033 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:01.068 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:01.071 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:01.075 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:01.078 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:01.082 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:31.114 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:31.118 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:31.122 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:31.126 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:34:31.130 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:01.165 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:01.169 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:01.172 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:01.175 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:01.178 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:31.249 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:31.253 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:31.256 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:31.259 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:35:31.262 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:01.309 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:01.313 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:01.316 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:01.320 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:01.323 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:31.352 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:31.355 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:31.358 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:31.361 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:36:31.364 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:01.394 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:01.398 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:01.401 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:01.404 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:01.408 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:31.439 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:31.442 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:31.445 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:31.448 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:37:31.451 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:01.520 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:01.524 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:01.526 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:01.529 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:01.533 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:31.577 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:31.580 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:31.583 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:31.586 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:38:31.589 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:01.622 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:01.626 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:01.629 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:01.632 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:01.635 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:31.666 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:31.669 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:31.672 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:31.675 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:39:31.678 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:01.705 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:01.709 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:01.712 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:01.715 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:01.719 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:31.751 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:31.755 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:31.758 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:31.761 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:40:31.765 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:01.814 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:01.818 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:01.822 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:01.825 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:01.828 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:31.879 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:31.883 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:31.886 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:31.890 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:41:31.893 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:01.930 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:01.934 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:01.937 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:01.941 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:01.944 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:31.975 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:31.979 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:31.982 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:31.985 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:42:31.989 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:02.043 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:02.047 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:02.051 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:02.053 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:02.057 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:32.102 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:32.106 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:32.109 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:32.113 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:43:32.117 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:02.178 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:02.182 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:02.186 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:02.189 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:02.192 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:32.229 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:32.233 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:32.236 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:32.239 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:44:32.243 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:02.271 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:02.275 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:02.278 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:02.281 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:02.284 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:32.318 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:32.322 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:32.325 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:32.329 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:45:32.332 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:02.364 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:02.368 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:02.371 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:02.374 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:02.378 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:32.407 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:32.410 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:32.413 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:32.416 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:46:32.419 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:02.499 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:02.503 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:02.506 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:02.509 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:02.512 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:32.542 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:32.545 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:32.549 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:32.552 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:47:32.556 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:02.609 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:02.613 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:02.616 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:02.620 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:02.623 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:14.218 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-04-03T03:48:14.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T03:48:14.224 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2021-04-03T03:48:14.225 INFO:tasks.cephfs_test_runner:ERROR 2021-04-03T03:48:14.225 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.226 INFO:tasks.cephfs_test_runner:====================================================================== 2021-04-03T03:48:14.226 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:48:14.226 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-04-03T03:48:14.227 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.227 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-04-03T03:48:14.227 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-04-03T03:48:14.228 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-04-03T03:48:14.228 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-04-03T03:48:14.228 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-04-03T03:48:14.229 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.229 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-04-03T03:48:14.229 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.229 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.230 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2021-04-03T03:48:14.230 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2021-04-03T03:48:14.230 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2021-04-03T03:48:14.230 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2021-04-03T03:48:14.231 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-04-03T03:48:14.231 INFO:tasks.cephfs_test_runner: proc.wait() 2021-04-03T03:48:14.231 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 148, in wait 2021-04-03T03:48:14.232 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-04-03T03:48:14.232 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-04-03T03:48:14.232 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-04-03T03:48:14.232 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-04-03T03:48:14.233 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-04-03T03:48:14.233 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-04-03T03:48:14.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-04-03T03:48:14.234 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-04-03T03:48:14.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-04-03T03:48:14.234 INFO:tasks.cephfs_test_runner: for line in f: 2021-04-03T03:48:14.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-04-03T03:48:14.235 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-04-03T03:48:14.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-04-03T03:48:14.235 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-04-03T03:48:14.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-04-03T03:48:14.236 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-04-03T03:48:14.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-04-03T03:48:14.237 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-04-03T03:48:14.237 INFO:tasks.cephfs_test_runner:socket.timeout 2021-04-03T03:48:14.237 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.238 INFO:tasks.cephfs_test_runner:====================================================================== 2021-04-03T03:48:14.238 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:48:14.238 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-04-03T03:48:14.239 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.239 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-04-03T03:48:14.239 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-04-03T03:48:14.239 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-04-03T03:48:14.240 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-04-03T03:48:14.240 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-04-03T03:48:14.240 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.240 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-04-03T03:48:14.241 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.241 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.241 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2021-04-03T03:48:14.241 INFO:tasks.cephfs_test_runner: m.teardown() 2021-04-03T03:48:14.242 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2021-04-03T03:48:14.242 INFO:tasks.cephfs_test_runner: self.umount() 2021-04-03T03:48:14.242 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2021-04-03T03:48:14.243 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2021-04-03T03:48:14.243 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-04-03T03:48:14.243 INFO:tasks.cephfs_test_runner: proc.wait() 2021-04-03T03:48:14.244 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 148, in wait 2021-04-03T03:48:14.244 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-04-03T03:48:14.244 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-04-03T03:48:14.244 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-04-03T03:48:14.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-04-03T03:48:14.245 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-04-03T03:48:14.245 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-04-03T03:48:14.246 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-04-03T03:48:14.246 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-04-03T03:48:14.246 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-04-03T03:48:14.246 INFO:tasks.cephfs_test_runner: for line in f: 2021-04-03T03:48:14.247 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-04-03T03:48:14.247 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-04-03T03:48:14.247 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-04-03T03:48:14.248 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-04-03T03:48:14.248 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-04-03T03:48:14.248 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-04-03T03:48:14.249 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-04-03T03:48:14.249 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-04-03T03:48:14.249 INFO:tasks.cephfs_test_runner:socket.timeout 2021-04-03T03:48:14.249 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.250 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-04-03T03:48:14.250 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2351.227s 2021-04-03T03:48:14.250 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.251 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2021-04-03T03:48:14.251 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.251 INFO:tasks.cephfs_test_runner:====================================================================== 2021-04-03T03:48:14.252 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:48:14.252 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-04-03T03:48:14.252 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.253 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-04-03T03:48:14.253 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-04-03T03:48:14.253 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-04-03T03:48:14.254 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-04-03T03:48:14.254 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-04-03T03:48:14.254 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.254 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-04-03T03:48:14.255 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.255 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.255 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2021-04-03T03:48:14.256 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2021-04-03T03:48:14.256 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2021-04-03T03:48:14.256 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2021-04-03T03:48:14.256 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-04-03T03:48:14.257 INFO:tasks.cephfs_test_runner: proc.wait() 2021-04-03T03:48:14.257 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 148, in wait 2021-04-03T03:48:14.257 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-04-03T03:48:14.257 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-04-03T03:48:14.258 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-04-03T03:48:14.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-04-03T03:48:14.258 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-04-03T03:48:14.258 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-04-03T03:48:14.259 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-04-03T03:48:14.259 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-04-03T03:48:14.259 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-04-03T03:48:14.260 INFO:tasks.cephfs_test_runner: for line in f: 2021-04-03T03:48:14.260 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-04-03T03:48:14.260 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-04-03T03:48:14.260 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-04-03T03:48:14.261 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-04-03T03:48:14.261 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-04-03T03:48:14.261 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-04-03T03:48:14.261 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-04-03T03:48:14.262 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-04-03T03:48:14.262 INFO:tasks.cephfs_test_runner:socket.timeout 2021-04-03T03:48:14.262 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.262 INFO:tasks.cephfs_test_runner:====================================================================== 2021-04-03T03:48:14.263 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:48:14.263 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-04-03T03:48:14.263 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.264 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-04-03T03:48:14.264 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-04-03T03:48:14.264 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-04-03T03:48:14.264 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-04-03T03:48:14.265 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-04-03T03:48:14.265 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.265 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-04-03T03:48:14.265 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.266 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-04-03T03:48:14.266 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2021-04-03T03:48:14.266 INFO:tasks.cephfs_test_runner: m.teardown() 2021-04-03T03:48:14.266 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2021-04-03T03:48:14.267 INFO:tasks.cephfs_test_runner: self.umount() 2021-04-03T03:48:14.267 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2021-04-03T03:48:14.267 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2021-04-03T03:48:14.268 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-04-03T03:48:14.268 INFO:tasks.cephfs_test_runner: proc.wait() 2021-04-03T03:48:14.268 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 148, in wait 2021-04-03T03:48:14.268 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-04-03T03:48:14.269 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-04-03T03:48:14.269 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-04-03T03:48:14.269 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-04-03T03:48:14.269 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-04-03T03:48:14.270 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-04-03T03:48:14.270 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-04-03T03:48:14.270 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-04-03T03:48:14.270 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-04-03T03:48:14.271 INFO:tasks.cephfs_test_runner: for line in f: 2021-04-03T03:48:14.271 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-04-03T03:48:14.271 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-04-03T03:48:14.271 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-04-03T03:48:14.272 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-04-03T03:48:14.272 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-04-03T03:48:14.272 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-04-03T03:48:14.272 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-04-03T03:48:14.273 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-04-03T03:48:14.273 INFO:tasks.cephfs_test_runner:socket.timeout 2021-04-03T03:48:14.273 INFO:tasks.cephfs_test_runner: 2021-04-03T03:48:14.274 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:48:14.303 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-04-03T03:48:14.503 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7be90cc4d40b4adba142990aab5adfb2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-04-03T03:48:14.505 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2021-04-03T03:48:14.519 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2021-04-03T03:48:14.714 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2021-04-03T03:48:14.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-04-03T03:48:14.715 DEBUG:teuthology.orchestra.run.gibba026:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-04-03T03:48:14.715 DEBUG:teuthology.orchestra.run.gibba026:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-04-03T03:48:32.668 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:32.672 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:32.674 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:32.678 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:48:32.681 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:02.728 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:02.731 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:02.735 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:02.738 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:02.741 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:32.770 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:32.774 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:32.778 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:32.781 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:49:32.784 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:02.821 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:02.826 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:02.828 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:02.831 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:02.837 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:32.874 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:32.877 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:32.879 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:32.882 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:50:32.886 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:02.945 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:02.949 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:02.952 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:02.955 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:02.959 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:33.022 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:33.026 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:33.029 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:33.032 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:51:33.036 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:03.074 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:03.081 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:03.084 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:03.087 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:03.091 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:33.126 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:33.130 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:33.133 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:33.136 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:52:33.140 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:03.193 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:03.197 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:03.200 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:03.203 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:03.206 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:33.310 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:33.313 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:33.317 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:33.321 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:53:33.325 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:03.354 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:03.357 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:03.360 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:03.363 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:03.366 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:33.397 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:33.400 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:33.403 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:33.406 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:54:33.409 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:03.439 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:03.443 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:03.446 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:03.450 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:03.453 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:33.489 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:33.492 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:33.495 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:33.499 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:55:33.502 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:03.534 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:03.538 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:03.541 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:03.545 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:03.548 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:33.596 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:33.600 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:33.603 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:33.606 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:56:33.609 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:03.645 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:03.649 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:03.652 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:03.656 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:03.659 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:33.703 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:33.707 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:33.710 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:33.713 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:57:33.717 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:03.768 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:03.772 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:03.776 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:03.779 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:03.782 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:33.813 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:33.817 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:33.820 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:33.824 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:58:33.827 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:03.875 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:03.879 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:03.882 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:03.885 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:03.888 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:33.932 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:33.935 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:33.939 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:33.942 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T03:59:33.945 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:03.998 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:04.002 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:04.005 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:04.008 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:04.011 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:34.045 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:34.049 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:34.053 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:34.056 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:00:34.060 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:04.093 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:04.097 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:04.100 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:04.103 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:04.107 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:34.137 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:34.141 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:34.145 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:34.148 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:01:34.152 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:04.187 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:04.191 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:04.195 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:04.198 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:04.201 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:34.234 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:34.238 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:34.241 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:34.244 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:02:34.250 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:04.298 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:04.302 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:04.305 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:04.308 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:04.311 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:14.745 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-04-03T04:03:14.761 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-04-03T04:03:14.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T04:03:14.772 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-04-03T04:03:14.781 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2021-04-03T04:03:14.790 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2021-04-03T04:03:14.791 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T04:03:14.860 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 Processor -- start 2021-04-03T04:03:14.861 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- start start 2021-04-03T04:03:14.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40473f0 0x7f66b40d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:14.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f66b403dea0 0x7f66b40420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:14.863 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f66b40d0e20 0x7f66b40d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:14.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40d06e0 con 0x7f66b40d0e20 2021-04-03T04:03:14.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40d0980 con 0x7f66b403dea0 2021-04-03T04:03:14.864 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40d0410 con 0x7f66b40473f0 2021-04-03T04:03:14.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f66b40d0e20 0x7f66b40d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:14.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f66b40d0e20 0x7f66b40d1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59330/0 (socket says 172.21.2.111:59330) 2021-04-03T04:03:14.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 -- 172.21.2.111:0/2353047869 learned_addr learned my addr 172.21.2.111:0/2353047869 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:14.865 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 -- 172.21.2.111:0/2353047869 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40473f0 msgr2=0x7f66b40d4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:14.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 --2- 172.21.2.111:0/2353047869 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40473f0 0x7f66b40d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 -- 172.21.2.111:0/2353047869 >> v2:172.21.2.115:3300/0 conn(0x7f66b403dea0 msgr2=0x7f66b40420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:14.866 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 --2- 172.21.2.111:0/2353047869 >> v2:172.21.2.115:3300/0 conn(0x7f66b403dea0 0x7f66b40420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 -- 172.21.2.111:0/2353047869 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40425b0 con 0x7f66b40d0e20 2021-04-03T04:03:14.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bad9d700 1 --2- 172.21.2.111:0/2353047869 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40d0e20 0x7f66b40d1290 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f66bc06b5d0 tx=0x7f66bc05ad70).ready entity=mon.0 client_cookie=f60cc486a845b28c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:14.867 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66b959a700 1 -- 172.21.2.111:0/2353047869 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66bc05bda0 con 0x7f66b40d0e20 2021-04-03T04:03:14.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66b959a700 1 -- 172.21.2.111:0/2353047869 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc05df60 con 0x7f66b40d0e20 2021-04-03T04:03:14.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66b959a700 1 -- 172.21.2.111:0/2353047869 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66bc0699f0 con 0x7f66b40d0e20 2021-04-03T04:03:14.868 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- 172.21.2.111:0/2353047869 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40d0e20 msgr2=0x7f66b40d1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:14.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 --2- 172.21.2.111:0/2353047869 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40d0e20 0x7f66b40d1290 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f66bc06b5d0 tx=0x7f66bc05ad70).stop 2021-04-03T04:03:14.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- 172.21.2.111:0/2353047869 shutdown_connections 2021-04-03T04:03:14.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 --2- 172.21.2.111:0/2353047869 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40d0e20 0x7f66b40d1290 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:14.869 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 --2- 172.21.2.111:0/2353047869 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40473f0 0x7f66b40d4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 --2- 172.21.2.111:0/2353047869 >> v2:172.21.2.115:3300/0 conn(0x7f66b403dea0 0x7f66b40420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- 172.21.2.111:0/2353047869 >> 172.21.2.111:0/2353047869 conn(0x7f66b400d9e0 msgr2=0x7f66b4046b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:14.870 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- 172.21.2.111:0/2353047869 shutdown_connections 2021-04-03T04:03:14.871 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- 172.21.2.111:0/2353047869 wait complete. 2021-04-03T04:03:14.873 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 Processor -- start 2021-04-03T04:03:14.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.855+0000 7f66bb59e700 1 -- start start 2021-04-03T04:03:14.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66b403dea0 0x7f66b403b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:14.874 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40473f0 0x7f66b40ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:14.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40d0e20 0x7f66b40cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:14.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40cc0c0 con 0x7f66b40473f0 2021-04-03T04:03:14.875 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40cf290 con 0x7f66b403dea0 2021-04-03T04:03:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40ddd90 con 0x7f66b40d0e20 2021-04-03T04:03:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40473f0 0x7f66b40ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40473f0 0x7f66b40ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59336/0 (socket says 172.21.2.111:59336) 2021-04-03T04:03:14.876 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 -- 172.21.2.111:0/3998617790 learned_addr learned my addr 172.21.2.111:0/3998617790 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 -- 172.21.2.111:0/3998617790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40d0e20 msgr2=0x7f66b40cf730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T04:03:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40d0e20 0x7f66b40cf730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.877 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 -- 172.21.2.111:0/3998617790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66b403dea0 msgr2=0x7f66b403b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66b403dea0 0x7f66b403b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 -- 172.21.2.111:0/3998617790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40ce660 con 0x7f66b40473f0 2021-04-03T04:03:14.878 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66b9d9b700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40473f0 0x7f66b40ce170 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f66b0002f30 tx=0x7f66b0006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:14.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ab7fe700 1 -- 172.21.2.111:0/3998617790 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b000d840 con 0x7f66b40473f0 2021-04-03T04:03:14.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ab7fe700 1 -- 172.21.2.111:0/3998617790 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b0000e90 con 0x7f66b40473f0 2021-04-03T04:03:14.879 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b40cb4c0 con 0x7f66b40473f0 2021-04-03T04:03:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b40cbc00 con 0x7f66b40473f0 2021-04-03T04:03:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ab7fe700 1 -- 172.21.2.111:0/3998617790 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b000db90 con 0x7f66b40473f0 2021-04-03T04:03:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b40cff40 con 0x7f66b40473f0 2021-04-03T04:03:14.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ab7fe700 1 -- 172.21.2.111:0/3998617790 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f66b0017030 con 0x7f66b40473f0 2021-04-03T04:03:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ab7fe700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6698056ea0 0x7f6698059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ab7fe700 1 -- 172.21.2.111:0/3998617790 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f66b006f390 con 0x7f66b40473f0 2021-04-03T04:03:14.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ba59c700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6698056ea0 0x7f6698059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:14.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.859+0000 7f66ba59c700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6698056ea0 0x7f6698059360 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f66ac002f30 tx=0x7f66ac0069e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:14.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.863+0000 7f66ab7fe700 1 -- 172.21.2.111:0/3998617790 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f66b00449f0 con 0x7f66b40473f0 2021-04-03T04:03:14.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66b4004da0 con 0x7f66b40473f0 2021-04-03T04:03:14.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66ab7fe700 1 -- 172.21.2.111:0/3998617790 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13551 (secure 0 0 0) 0x7f66b0048010 con 0x7f66b40473f0 2021-04-03T04:03:14.958 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T04:03:14.959 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":154,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:43.996443+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.111:6835/295340141":"2021-04-04T03:17:40.389472+0000","172.21.2.111:6834/295340141":"2021-04-04T03:17:40.389472+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T04:03:14.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6698056ea0 msgr2=0x7f6698059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:14.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6698056ea0 0x7f6698059360 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f66ac002f30 tx=0x7f66ac0069e0).stop 2021-04-03T04:03:14.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40473f0 msgr2=0x7f66b40ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:14.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40473f0 0x7f66b40ce170 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f66b0002f30 tx=0x7f66b0006ea0).stop 2021-04-03T04:03:14.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 shutdown_connections 2021-04-03T04:03:14.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f66b40d0e20 0x7f66b40cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f6698056ea0 0x7f6698059360 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:14.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f66b403dea0 0x7f66b403b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:14.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 --2- 172.21.2.111:0/3998617790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f66b40473f0 0x7f66b40ce170 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:14.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 >> 172.21.2.111:0/3998617790 conn(0x7f66b400d9e0 msgr2=0x7f66b403c950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:14.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 shutdown_connections 2021-04-03T04:03:14.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:14.951+0000 7f66bb59e700 1 -- 172.21.2.111:0/3998617790 wait complete. 2021-04-03T04:03:14.966 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2021-04-03T04:03:15.021 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583f013700 1 Processor -- start 2021-04-03T04:03:15.022 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583f013700 1 -- start start 2021-04-03T04:03:15.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583f013700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f58380b3740 0x7f58380b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583f013700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58380aa120 con 0x7f58380600e0 2021-04-03T04:03:15.028 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583f013700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58380b1f00 con 0x7f58380b0710 2021-04-03T04:03:15.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583f013700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58380b0af0 con 0x7f58380b3740 2021-04-03T04:03:15.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583e812700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f58380600e0 0x7f5838143ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60140/0 (socket says 172.21.2.111:60140) 2021-04-03T04:03:15.029 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583e812700 1 -- 172.21.2.111:0/1337929410 learned_addr learned my addr 172.21.2.111:0/1337929410 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:15.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f58380b3740 0x7f58380b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790375454 0 0) 0x7f58380aa120 con 0x7f58380600e0 2021-04-03T04:03:15.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5820003240 con 0x7f58380600e0 2021-04-03T04:03:15.030 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2199465309 0 0) 0x7f5820003240 con 0x7f58380600e0 2021-04-03T04:03:15.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f58380b3740 msgr2=0x7f58380b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:15.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f58380b3740 0x7f58380b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.031 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 >> v1:172.21.2.115:6789/0 conn(0x7f58380b0710 legacy=0x7f58380b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380b12d0 con 0x7f58380600e0 2021-04-03T04:03:15.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f5830001a50 con 0x7f58380600e0 2021-04-03T04:03:15.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 >> v1:172.21.2.111:6789/0 conn(0x7f58380600e0 legacy=0x7f5838143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.032 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5820003760 0x7f5820003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58200044e0 0x7f5820004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f58380b3740 0x7f5820004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.033 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380b0af0 con 0x7f58200044e0 2021-04-03T04:03:15.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5820004340 con 0x7f5820003760 2021-04-03T04:03:15.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582000b730 con 0x7f58380b3740 2021-04-03T04:03:15.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58200044e0 0x7f5820004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.034 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 -- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f58380b3740 msgr2=0x7f5820004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:15.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f58380b3740 0x7f5820004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:15.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 -- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5820003760 msgr2=0x7f5820003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:15.035 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5820003760 0x7f5820003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:15.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 -- 172.21.2.111:0/1337929410 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820003240 con 0x7f58200044e0 2021-04-03T04:03:15.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.015+0000 7f583d810700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58200044e0 0x7f5820004960 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f583400c000 tx=0x7f5834008040).ready entity=mon.0 client_cookie=75c0963196887fd6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:15.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f583400ebc0 con 0x7f58200044e0 2021-04-03T04:03:15.036 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583400ed20 con 0x7f58200044e0 2021-04-03T04:03:15.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583d00f700 1 -- 172.21.2.111:0/1337929410 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58340125e0 con 0x7f58200044e0 2021-04-03T04:03:15.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/1337929410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58200044e0 msgr2=0x7f5820004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.037 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 --2- 172.21.2.111:0/1337929410 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58200044e0 0x7f5820004960 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f583400c000 tx=0x7f5834008040).stop 2021-04-03T04:03:15.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e812700 1 -- 172.21.2.111:0/1337929410 reap_dead start 2021-04-03T04:03:15.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/1337929410 shutdown_connections 2021-04-03T04:03:15.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/1337929410 >> 172.21.2.111:0/1337929410 conn(0x7f5838063d70 msgr2=0x7f5838064180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:15.038 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/1337929410 shutdown_connections 2021-04-03T04:03:15.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/1337929410 wait complete. 2021-04-03T04:03:15.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 Processor -- start 2021-04-03T04:03:15.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- start start 2021-04-03T04:03:15.039 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58381481f0 0x7f5838144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5838144aa0 0x7f583814a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583814a550 0x7f58381557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.040 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838148da0 con 0x7f58381481f0 2021-04-03T04:03:15.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838153130 con 0x7f5838144aa0 2021-04-03T04:03:15.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5838152c30 con 0x7f583814a550 2021-04-03T04:03:15.041 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58381481f0 0x7f5838144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58381481f0 0x7f5838144fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59356/0 (socket says 172.21.2.111:59356) 2021-04-03T04:03:15.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 -- 172.21.2.111:0/4196950361 learned_addr learned my addr 172.21.2.111:0/4196950361 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:15.042 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 -- 172.21.2.111:0/4196950361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583814a550 msgr2=0x7f58381557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583814a550 0x7f58381557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:15.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 -- 172.21.2.111:0/4196950361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5838144aa0 msgr2=0x7f583814a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5838144aa0 0x7f583814a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 -- 172.21.2.111:0/4196950361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838152e90 con 0x7f58381481f0 2021-04-03T04:03:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583e011700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58381481f0 0x7f5838144fb0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f58380b18f0 tx=0x7f582c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f582b7fe700 1 -- 172.21.2.111:0/4196950361 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f582c00e4a0 con 0x7f58381481f0 2021-04-03T04:03:15.044 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f582b7fe700 1 -- 172.21.2.111:0/4196950361 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c00e600 con 0x7f58381481f0 2021-04-03T04:03:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838148900 con 0x7f58381481f0 2021-04-03T04:03:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f582b7fe700 1 -- 172.21.2.111:0/4196950361 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f582c00e960 con 0x7f58381481f0 2021-04-03T04:03:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58381485d0 con 0x7f58381481f0 2021-04-03T04:03:15.045 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5838148a90 con 0x7f58381481f0 2021-04-03T04:03:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f582b7fe700 1 -- 172.21.2.111:0/4196950361 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f582c019030 con 0x7f58381481f0 2021-04-03T04:03:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f582b7fe700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056f20 0x7f58180593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.046 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f582b7fe700 1 -- 172.21.2.111:0/4196950361 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f582c06f3f0 con 0x7f58381481f0 2021-04-03T04:03:15.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583d810700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056f20 0x7f58180593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.047 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.019+0000 7f583d810700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056f20 0x7f58180593e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f583400c000 tx=0x7f5834019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:15.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.023+0000 7f582b7fe700 1 -- 172.21.2.111:0/4196950361 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f582c044ab0 con 0x7f58381481f0 2021-04-03T04:03:15.119 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.111+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58380b1f00 con 0x7f5818056f20 2021-04-03T04:03:15.121 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f582b7fe700 1 -- 172.21.2.111:0/4196950361 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66066 (secure 0 0 0) 0x7f58380b1f00 con 0x7f5818056f20 2021-04-03T04:03:15.122 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T04:03:15.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056f20 msgr2=0x7f58180593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056f20 0x7f58180593e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f583400c000 tx=0x7f5834019000).stop 2021-04-03T04:03:15.124 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58381481f0 msgr2=0x7f5838144fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58381481f0 0x7f5838144fb0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f58380b18f0 tx=0x7f582c007c00).stop 2021-04-03T04:03:15.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 shutdown_connections 2021-04-03T04:03:15.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f583814a550 0x7f58381557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:15.125 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f58381481f0 0x7f5838144fb0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f5818056f20 0x7f58180593e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 --2- 172.21.2.111:0/4196950361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f5838144aa0 0x7f583814a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:15.126 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 >> 172.21.2.111:0/4196950361 conn(0x7f5838063d70 msgr2=0x7f583805c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:15.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 shutdown_connections 2021-04-03T04:03:15.127 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.115+0000 7f583f013700 1 -- 172.21.2.111:0/4196950361 wait complete. 2021-04-03T04:03:15.127 INFO:teuthology.orchestra.run.gibba011.stderr:dumped all 2021-04-03T04:03:15.129 INFO:teuthology.orchestra.run.gibba011.stdout:{"pg_ready":true,"pg_map":{"version":1788,"stamp":"2021-04-03T04:03:13.135690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404032,"kb_used_data":15360,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570688,"statfs":{"total":773094113280,"available":764488384512,"internally_reserved":8589934592,"allocated":15728640,"data_stored":7742936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005868"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"127","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:43.711514+0000","last_change":"2021-04-03T03:08:39.165326+0000","last_active":"2021-04-03T03:17:43.711514+0000","last_peered":"2021-04-03T03:17:43.711514+0000","last_clean":"2021-04-03T03:17:43.711514+0000","last_became_active":"2021-04-03T03:08:39.165110+0000","last_became_peered":"2021-04-03T03:08:39.165110+0000","last_unstale":"2021-04-03T03:17:43.711514+0000","last_undegraded":"2021-04-03T03:17:43.711514+0000","last_fullsized":"2021-04-03T03:17:43.711514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"128","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.997771+0000","last_change":"2021-04-03T03:08:39.164016+0000","last_active":"2021-04-03T03:17:42.997771+0000","last_peered":"2021-04-03T03:17:42.997771+0000","last_clean":"2021-04-03T03:17:42.997771+0000","last_became_active":"2021-04-03T03:08:39.163862+0000","last_became_peered":"2021-04-03T03:08:39.163862+0000","last_unstale":"2021-04-03T03:17:42.997771+0000","last_undegraded":"2021-04-03T03:17:42.997771+0000","last_fullsized":"2021-04-03T03:17:42.997771+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"122","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.998099+0000","last_change":"2021-04-03T03:08:39.484523+0000","last_active":"2021-04-03T03:17:42.998099+0000","last_peered":"2021-04-03T03:17:42.998099+0000","last_clean":"2021-04-03T03:17:42.998099+0000","last_became_active":"2021-04-03T03:08:39.483298+0000","last_became_peered":"2021-04-03T03:08:39.483298+0000","last_unstale":"2021-04-03T03:17:42.998099+0000","last_undegraded":"2021-04-03T03:17:42.998099+0000","last_fullsized":"2021-04-03T03:17:42.998099+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"128","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.997753+0000","last_change":"2021-04-03T03:08:39.481713+0000","last_active":"2021-04-03T03:17:42.997753+0000","last_peered":"2021-04-03T03:17:42.997753+0000","last_clean":"2021-04-03T03:17:42.997753+0000","last_became_active":"2021-04-03T03:08:39.481308+0000","last_became_peered":"2021-04-03T03:08:39.481308+0000","last_unstale":"2021-04-03T03:17:42.997753+0000","last_undegraded":"2021-04-03T03:17:42.997753+0000","last_fullsized":"2021-04-03T03:17:42.997753+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"122","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.998041+0000","last_change":"2021-04-03T03:08:39.163433+0000","last_active":"2021-04-03T03:17:42.998041+0000","last_peered":"2021-04-03T03:17:42.998041+0000","last_clean":"2021-04-03T03:17:42.998041+0000","last_became_active":"2021-04-03T03:08:39.163151+0000","last_became_peered":"2021-04-03T03:08:39.163151+0000","last_unstale":"2021-04-03T03:17:42.998041+0000","last_undegraded":"2021-04-03T03:17:42.998041+0000","last_fullsized":"2021-04-03T03:17:42.998041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"127","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.998043+0000","last_change":"2021-04-03T03:08:35.464441+0000","last_active":"2021-04-03T03:17:42.998043+0000","last_peered":"2021-04-03T03:17:42.998043+0000","last_clean":"2021-04-03T03:17:42.998043+0000","last_became_active":"2021-04-03T03:08:35.463653+0000","last_became_peered":"2021-04-03T03:08:35.463653+0000","last_unstale":"2021-04-03T03:17:42.998043+0000","last_undegraded":"2021-04-03T03:17:42.998043+0000","last_fullsized":"2021-04-03T03:17:42.998043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_clean_scrub_stamp":"2021-04-03T03:08:29.756831+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"142","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:43.641745+0000","last_change":"2021-04-03T03:08:39.165073+0000","last_active":"2021-04-03T03:17:43.641745+0000","last_peered":"2021-04-03T03:17:43.641745+0000","last_clean":"2021-04-03T03:17:43.641745+0000","last_became_active":"2021-04-03T03:08:39.164901+0000","last_became_peered":"2021-04-03T03:08:39.164901+0000","last_unstale":"2021-04-03T03:17:43.641745+0000","last_undegraded":"2021-04-03T03:17:43.641745+0000","last_fullsized":"2021-04-03T03:17:43.641745+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"135","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:44.003798+0000","last_change":"2021-04-03T03:08:39.162544+0000","last_active":"2021-04-03T03:17:44.003798+0000","last_peered":"2021-04-03T03:17:44.003798+0000","last_clean":"2021-04-03T03:17:44.003798+0000","last_became_active":"2021-04-03T03:08:39.162345+0000","last_became_peered":"2021-04-03T03:08:39.162345+0000","last_unstale":"2021-04-03T03:17:44.003798+0000","last_undegraded":"2021-04-03T03:17:44.003798+0000","last_fullsized":"2021-04-03T03:17:44.003798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"142","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:43.641689+0000","last_change":"2021-04-03T03:08:39.165106+0000","last_active":"2021-04-03T03:17:43.641689+0000","last_peered":"2021-04-03T03:17:43.641689+0000","last_clean":"2021-04-03T03:17:43.641689+0000","last_became_active":"2021-04-03T03:08:39.164923+0000","last_became_peered":"2021-04-03T03:08:39.164923+0000","last_unstale":"2021-04-03T03:17:43.641689+0000","last_undegraded":"2021-04-03T03:17:43.641689+0000","last_fullsized":"2021-04-03T03:17:43.641689+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575506,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2649999999999999,"15min":1.282},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":1.585,"5min":1.782,"15min":1.782},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.202,"5min":1.2350000000000001,"15min":1.2470000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6100000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.80900000000000005}]},{"osd":1,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.2,"15min":1.2430000000000001},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4219999999999999,"5min":1.679,"15min":1.679},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.222,"5min":1.2430000000000001,"15min":1.268},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4750000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.89000000000000001}]},{"osd":2,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.228,"15min":1.238},"min":{"1min":0.47399999999999998,"5min":0.39100000000000001,"15min":0.33200000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.6279999999999999},"last":0.626},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":1.613,"5min":1.647,"15min":1.647},"last":0.78600000000000003}]},{"osd":3,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.495,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.621},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.296},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.92500000000000004}]},{"osd":4,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.223,"15min":0.223},"max":{"1min":1.343,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.014},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.33100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.167,"5min":1.2969999999999999,"15min":1.3600000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.45800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":1.411,"5min":1.411,"15min":1.506},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.436,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.3600000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0649999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.97499999999999998,"15min":1.0329999999999999},"min":{"1min":0.38600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.3879999999999999,"5min":1.462,"15min":1.5189999999999999},"last":1.246},{"interface":"front","average":{"1min":1.0900000000000001,"5min":0.96399999999999997,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.26300000000000001,"15min":0.24099999999999999},"max":{"1min":1.4550000000000001,"5min":1.5,"15min":1.5489999999999999},"last":0.84199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575506,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.254,"15min":1.2230000000000001},"min":{"1min":0.65500000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.53,"5min":1.653,"15min":2.044},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.2749999999999999,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":1.4750000000000001,"5min":1.6519999999999999,"15min":1.8999999999999999},"last":1.1459999999999999}]},{"osd":1,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.83199999999999996,"5min":0.32200000000000001,"15min":0.28699999999999998},"max":{"1min":1.4750000000000001,"5min":1.6160000000000001,"15min":1.6559999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.248},"min":{"1min":0.92400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.538,"5min":1.6639999999999999,"15min":1.7410000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2749999999999999,"15min":1.252},"min":{"1min":0.90200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.4990000000000001,"5min":1.768,"15min":1.768},"last":1.131},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.26,"15min":1.244},"min":{"1min":0.91200000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.52,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.2470000000000001}]},{"osd":3,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.26,"15min":1.242},"min":{"1min":0.98299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.522,"5min":1.5820000000000001,"15min":1.637},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.25,"15min":1.218},"min":{"1min":0.92300000000000004,"5min":0.46000000000000002,"15min":0.26600000000000001},"max":{"1min":1.4650000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2589999999999999}]},{"osd":4,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.99199999999999999,"15min":0.98199999999999998},"min":{"1min":0.79400000000000004,"5min":0.26600000000000001,"15min":0.161},"max":{"1min":1.3440000000000001,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.042},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0369999999999999,"15min":1.008},"min":{"1min":0.745,"5min":0.28000000000000003,"15min":0.23400000000000001},"max":{"1min":1.4350000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.84299999999999997}]},{"osd":5,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.91700000000000004,"15min":0.90100000000000002},"min":{"1min":0.76300000000000001,"5min":0.217,"15min":0.20499999999999999},"max":{"1min":1.2549999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98799999999999999,"15min":0.98399999999999999},"min":{"1min":0.86299999999999999,"5min":0.28100000000000003,"15min":0.19600000000000001},"max":{"1min":1.4019999999999999,"5min":1.5309999999999999,"15min":1.6040000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":0.999},"min":{"1min":0.83099999999999996,"5min":0.30199999999999999,"15min":0.217},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.7749999999999999},"last":1.073},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0129999999999999,"15min":1.006},"min":{"1min":0.80600000000000005,"5min":0.29499999999999998,"15min":0.189},"max":{"1min":1.2529999999999999,"5min":1.611,"15min":1.611},"last":0.96499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575506,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.224,"15min":1.208},"min":{"1min":0.81999999999999995,"5min":0.47299999999999998,"15min":0.23000000000000001},"max":{"1min":1.4299999999999999,"5min":1.6160000000000001,"15min":1.708},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.274,"15min":1.2529999999999999},"min":{"1min":0.88700000000000001,"5min":0.497,"15min":0.26600000000000001},"max":{"1min":1.4910000000000001,"5min":1.696,"15min":1.7030000000000001},"last":1.3200000000000001}]},{"osd":1,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.349,"15min":1.3340000000000001},"min":{"1min":0.998,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":1.657,"5min":1.7430000000000001,"15min":1.8129999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.93899999999999995,"5min":0.628,"15min":0.316},"max":{"1min":1.524,"5min":1.661,"15min":1.661},"last":1.2370000000000001}]},{"osd":2,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2689999999999999,"15min":1.2350000000000001},"min":{"1min":0.85999999999999999,"5min":0.63500000000000001,"15min":0.26900000000000002},"max":{"1min":1.546,"5min":1.6499999999999999,"15min":1.736},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.274,"15min":1.2529999999999999},"min":{"1min":1.0700000000000001,"5min":0.56200000000000006,"15min":0.32400000000000001},"max":{"1min":1.4650000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.4770000000000001}]},{"osd":3,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.266,"15min":1.274},"min":{"1min":1.0569999999999999,"5min":0.56599999999999995,"15min":0.29899999999999999},"max":{"1min":1.502,"5min":1.5269999999999999,"15min":1.7390000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.2769999999999999,"15min":1.2470000000000001},"min":{"1min":0.90800000000000003,"5min":0.69299999999999995,"15min":0.30199999999999999},"max":{"1min":1.5920000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.522}]},{"osd":4,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89100000000000001,"15min":0.89200000000000002},"min":{"1min":0.754,"5min":0.25900000000000001,"15min":0.17299999999999999},"max":{"1min":1.2210000000000001,"5min":1.3300000000000001,"15min":1.389},"last":0.871},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.97599999999999998,"15min":0.96699999999999997},"min":{"1min":0.76800000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.169,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.0669999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.03,"15min":1.034},"min":{"1min":0.78100000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.448,"5min":1.665,"15min":1.6759999999999999},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0649999999999999,"15min":1.0580000000000001},"min":{"1min":0.85799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3440000000000001,"5min":1.496,"15min":1.5309999999999999},"last":1.123}]},{"osd":7,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0269999999999999,"15min":1.034},"min":{"1min":0.85499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":1.2969999999999999,"5min":1.7010000000000001,"15min":1.742},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.0680000000000001,"15min":1.0620000000000001},"min":{"1min":0.748,"5min":0.22900000000000001,"15min":0.224},"max":{"1min":1.3440000000000001,"5min":1.694,"15min":1.694},"last":1.284}]}]},{"osd":1,"up_from":13,"seq":55834575507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.64600000000000002,"5min":0.29899999999999999,"15min":0.253},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.4550000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81699999999999995,"15min":0.81200000000000006},"min":{"1min":0.72699999999999998,"5min":0.376,"15min":0.23899999999999999},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.772},"last":0.72499999999999998}]},{"osd":2,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.66500000000000004,"5min":0.254,"15min":0.22600000000000001},"max":{"1min":0.95299999999999996,"5min":1.0169999999999999,"15min":1.1140000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.32500000000000001,"15min":0.222},"max":{"1min":1.0680000000000001,"5min":1.147,"15min":1.4199999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.76500000000000001,"5min":0.34599999999999997,"15min":0.26100000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.9730000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.72499999999999998,"5min":0.379,"15min":0.223},"max":{"1min":1.0049999999999999,"5min":1.2729999999999999,"15min":1.4179999999999999},"last":0.79700000000000004}]},{"osd":4,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.355,"15min":1.3420000000000001},"min":{"1min":1.048,"5min":0.70499999999999996,"15min":0.42099999999999999},"max":{"1min":1.6579999999999999,"5min":1.704,"15min":1.7569999999999999},"last":1.45},{"interface":"front","average":{"1min":1.371,"5min":1.365,"15min":1.353},"min":{"1min":1.143,"5min":0.69399999999999995,"15min":0.41799999999999998},"max":{"1min":1.546,"5min":1.675,"15min":1.8939999999999999},"last":1.51}]},{"osd":5,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":1.179,"5min":0.63400000000000001,"15min":0.35599999999999998},"max":{"1min":1.589,"5min":1.6990000000000001,"15min":1.8220000000000001},"last":1.333},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.278,"15min":1.296},"min":{"1min":0.94999999999999996,"5min":0.68200000000000005,"15min":0.36799999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.744},"last":1.3799999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.321,"15min":1.3100000000000001},"min":{"1min":1.264,"5min":0.66900000000000004,"15min":0.47599999999999998},"max":{"1min":1.613,"5min":1.6830000000000001,"15min":1.925},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.3460000000000001,"15min":1.319},"min":{"1min":1.038,"5min":0.63100000000000001,"15min":0.38400000000000001},"max":{"1min":1.6299999999999999,"5min":1.8260000000000001,"15min":1.8540000000000001},"last":1.3069999999999999}]},{"osd":7,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":1.0860000000000001,"5min":0.60399999999999998,"15min":0.41699999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.7929999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.373,"15min":1.337},"min":{"1min":1.1899999999999999,"5min":0.65400000000000003,"15min":0.45400000000000001},"max":{"1min":1.643,"5min":1.782,"15min":1.782},"last":1.415}]}]},{"osd":0,"up_from":13,"seq":55834575506,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.754,"15min":0.75},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.16700000000000001},"max":{"1min":0.83099999999999996,"5min":1.107,"15min":1.6639999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80400000000000005,"15min":0.79200000000000004},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20399999999999999},"max":{"1min":1.056,"5min":1.131,"15min":1.71},"last":0.97099999999999997}]},{"osd":2,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85399999999999998,"15min":0.83299999999999996},"min":{"1min":0.318,"5min":0.315,"15min":0.188},"max":{"1min":1.0409999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79800000000000004,"15min":0.78000000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.218},"max":{"1min":1.052,"5min":1.052,"15min":1.248},"last":0.69299999999999995}]},{"osd":3,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81299999999999994,"15min":0.79100000000000004},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.17899999999999999},"max":{"1min":1.097,"5min":1.236,"15min":1.3580000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91600000000000004,"15min":0.89000000000000001},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.22},"max":{"1min":1.1359999999999999,"5min":1.264,"15min":1.264},"last":1.0740000000000001}]},{"osd":4,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3049999999999999,"15min":1.288},"min":{"1min":0.623,"5min":0.497,"15min":0.26200000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.298,"15min":1.29},"min":{"1min":0.62,"5min":0.45500000000000002,"15min":0.28699999999999998},"max":{"1min":1.599,"5min":1.635,"15min":1.76},"last":1.4370000000000001}]},{"osd":5,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.59199999999999997,"5min":0.44,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.613,"15min":1.839},"last":1.159},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3300000000000001,"15min":1.2929999999999999},"min":{"1min":0.60399999999999998,"5min":0.505,"15min":0.32900000000000001},"max":{"1min":1.645,"5min":1.673,"15min":1.927},"last":1.3999999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.2829999999999999},"min":{"1min":0.59899999999999998,"5min":0.47499999999999998,"15min":0.27200000000000002},"max":{"1min":1.5609999999999999,"5min":1.6419999999999999,"15min":1.77},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3380000000000001,"15min":1.3109999999999999},"min":{"1min":0.61299999999999999,"5min":0.48599999999999999,"15min":0.24299999999999999},"max":{"1min":1.611,"5min":1.663,"15min":1.7629999999999999},"last":1.2390000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.357,"15min":1.359},"min":{"1min":0.63900000000000001,"5min":0.47799999999999998,"15min":0.254},"max":{"1min":1.736,"5min":1.736,"15min":1.8},"last":1.333},{"interface":"front","average":{"1min":1.395,"5min":1.409,"15min":1.391},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.29599999999999999},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.899},"last":1.3640000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575506,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.746},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.17499999999999999},"max":{"1min":0.90800000000000003,"5min":1.1040000000000001,"15min":1.2170000000000001},"last":0.874},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78900000000000003,"15min":0.78400000000000003},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.182},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.264},"last":0.73699999999999999}]},{"osd":1,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.27600000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.121},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.19900000000000001},"max":{"1min":1.087,"5min":1.2010000000000001,"15min":1.206},"last":0.83299999999999996}]},{"osd":3,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"min":{"1min":0.29599999999999999,"5min":0.24099999999999999,"15min":0.219},"max":{"1min":1.046,"5min":1.1339999999999999,"15min":1.2290000000000001},"last":1.03},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.23999999999999999,"5min":0.216,"15min":0.191},"max":{"1min":0.93600000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3520000000000001},"min":{"1min":0.61499999999999999,"5min":0.48699999999999999,"15min":0.27000000000000002},"max":{"1min":1.54,"5min":1.79,"15min":1.79},"last":1.258},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.70599999999999996,"5min":0.52200000000000002,"15min":0.315},"max":{"1min":1.591,"5min":1.8520000000000001,"15min":1.857},"last":1.4870000000000001}]},{"osd":5,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.331},"min":{"1min":0.64200000000000002,"5min":0.47999999999999998,"15min":0.27400000000000002},"max":{"1min":1.6619999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.349},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.63300000000000001,"5min":0.51400000000000001,"15min":0.29299999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.7769999999999999},"last":1.331}]},{"osd":6,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3520000000000001,"15min":1.3879999999999999},"min":{"1min":0.65700000000000003,"5min":0.46700000000000003,"15min":0.32500000000000001},"max":{"1min":1.742,"5min":1.802,"15min":1.962},"last":1.298},{"interface":"front","average":{"1min":1.278,"5min":1.3169999999999999,"15min":1.3280000000000001},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.35399999999999998},"max":{"1min":1.6299999999999999,"5min":1.744,"15min":1.9019999999999999},"last":1.454}]},{"osd":7,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.369,"15min":1.377},"min":{"1min":0.58999999999999997,"5min":0.45100000000000001,"15min":0.28199999999999997},"max":{"1min":1.645,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":0.68400000000000005,"5min":0.49299999999999999,"15min":0.29799999999999999},"max":{"1min":1.5609999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.3939999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:11 2021","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.17399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.3160000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.85399999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.185,"15min":0.185},"max":{"1min":1.1180000000000001,"5min":1.284,"15min":1.284},"last":0.89200000000000002}]},{"osd":1,"last update":"Sat Apr 3 04:02:11 2021","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.184},"max":{"1min":0.95699999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.24299999999999999,"5min":0.20599999999999999,"15min":0.193},"max":{"1min":1.022,"5min":1.165,"15min":1.2},"last":0.85999999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:02:11 2021","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.85399999999999998},"min":{"1min":0.23300000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.96899999999999997,"5min":1.171,"15min":1.236},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.89600000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.20300000000000001},"max":{"1min":1.2,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.90500000000000003}]},{"osd":4,"last update":"Sat Apr 3 04:02:11 2021","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.421,"15min":1.431},"min":{"1min":0.70699999999999996,"5min":0.40100000000000002,"15min":0.26000000000000001},"max":{"1min":1.698,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.335,"15min":1.3320000000000001},"min":{"1min":0.72399999999999998,"5min":0.373,"15min":0.27800000000000002},"max":{"1min":1.518,"5min":1.577,"15min":1.6180000000000001},"last":1.472}]},{"osd":5,"last update":"Sat Apr 3 04:02:11 2021","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3,"15min":1.3120000000000001},"min":{"1min":0.69699999999999995,"5min":0.35099999999999998,"15min":0.24199999999999999},"max":{"1min":1.5649999999999999,"5min":1.673,"15min":1.76},"last":1.512},{"interface":"front","average":{"1min":1.343,"5min":1.3740000000000001,"15min":1.3640000000000001},"min":{"1min":0.72399999999999998,"5min":0.38500000000000001,"15min":0.28499999999999998},"max":{"1min":1.6279999999999999,"5min":1.7330000000000001,"15min":1.7370000000000001},"last":1.4179999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:02:11 2021","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.365,"15min":1.353},"min":{"1min":0.71199999999999997,"5min":0.39300000000000002,"15min":0.23300000000000001},"max":{"1min":1.7310000000000001,"5min":1.76,"15min":1.7729999999999999},"last":1.623},{"interface":"front","average":{"1min":1.29,"5min":1.3009999999999999,"15min":1.327},"min":{"1min":0.68899999999999995,"5min":0.34499999999999997,"15min":0.27100000000000002},"max":{"1min":1.474,"5min":1.663,"15min":1.798},"last":1.5}]},{"osd":7,"last update":"Sat Apr 3 04:02:11 2021","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.335,"15min":1.3420000000000001},"min":{"1min":0.67500000000000004,"5min":0.376,"15min":0.314},"max":{"1min":1.6100000000000001,"5min":1.653,"15min":1.724},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.367,"15min":1.3640000000000001},"min":{"1min":0.749,"5min":0.36699999999999999,"15min":0.30499999999999999},"max":{"1min":1.6240000000000001,"5min":1.708,"15min":1.8660000000000001},"last":1.6799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.242,"15min":1.228},"min":{"1min":0.65100000000000002,"5min":0.36699999999999999,"15min":0.314},"max":{"1min":1.641,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2330000000000001,"15min":1.1990000000000001},"min":{"1min":0.68000000000000005,"5min":0.33000000000000002,"15min":0.309},"max":{"1min":1.623,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.1899999999999999}]},{"osd":1,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.248,"15min":1.2190000000000001},"min":{"1min":0.66000000000000003,"5min":0.375,"15min":0.27400000000000002},"max":{"1min":1.554,"5min":1.772,"15min":1.772},"last":1.371},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.264,"15min":1.228},"min":{"1min":0.63400000000000001,"5min":0.36799999999999999,"15min":0.28199999999999997},"max":{"1min":1.548,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.3899999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.26,"15min":1.2470000000000001},"min":{"1min":0.73199999999999998,"5min":0.42299999999999999,"15min":0.28799999999999998},"max":{"1min":1.5840000000000001,"5min":1.673,"15min":1.75},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2969999999999999,"15min":1.248},"min":{"1min":0.67400000000000004,"5min":0.34399999999999997,"15min":0.27900000000000003},"max":{"1min":1.7030000000000001,"5min":2.024,"15min":2.024},"last":1.24}]},{"osd":3,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2569999999999999},"min":{"1min":0.70999999999999996,"5min":0.379,"15min":0.29999999999999999},"max":{"1min":1.488,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3580000000000001,"15min":1.321},"min":{"1min":0.64100000000000001,"5min":0.39900000000000002,"15min":0.29799999999999999},"max":{"1min":1.593,"5min":1.99,"15min":1.99},"last":1.4510000000000001}]},{"osd":5,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.91100000000000003,"15min":0.92100000000000004},"min":{"1min":0.32800000000000001,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":1.1559999999999999,"5min":1.5509999999999999,"15min":1.6379999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.90200000000000002},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.13300000000000001},"max":{"1min":1.202,"5min":1.448,"15min":1.448},"last":0.72799999999999998}]},{"osd":6,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94499999999999995,"15min":0.93400000000000005},"min":{"1min":0.33600000000000002,"5min":0.24299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2549999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.995,"15min":0.98499999999999999},"min":{"1min":0.34699999999999998,"5min":0.219,"15min":0.219},"max":{"1min":1.3959999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1140000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.153},"max":{"1min":1.446,"5min":1.681,"15min":1.681},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":1.0209999999999999},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.23100000000000001},"max":{"1min":1.5149999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.88400000000000001}]}]}]}} 2021-04-03T04:03:15.132 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2021-04-03T04:03:15.188 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 Processor -- start 2021-04-03T04:03:15.189 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 -- start start 2021-04-03T04:03:15.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40603f0 0x7fa5d40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa5d40b0c90 0x7fa5d4147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.190 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40607d0 con 0x7fa5d414b700 2021-04-03T04:03:15.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d40af400 con 0x7fa5d40b0c90 2021-04-03T04:03:15.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40af6a0 con 0x7fa5d40603f0 2021-04-03T04:03:15.191 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40603f0 0x7fa5d40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d2d9d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa5d40b0c90 0x7fa5d4147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40603f0 0x7fa5d40b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:42712/0 (socket says 172.21.2.111:42712) 2021-04-03T04:03:15.192 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 -- 172.21.2.111:0/4184696348 learned_addr learned my addr 172.21.2.111:0/4184696348 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:15.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 -- 172.21.2.111:0/4184696348 >> v2:172.21.2.115:3300/0 conn(0x7fa5d40b0c90 msgr2=0x7fa5d4147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:15.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 --2- 172.21.2.111:0/4184696348 >> v2:172.21.2.115:3300/0 conn(0x7fa5d40b0c90 0x7fa5d4147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.196 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 -- 172.21.2.111:0/4184696348 >> v1:172.21.2.115:6789/0 conn(0x7fa5d414b700 legacy=0x7fa5d40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 -- 172.21.2.111:0/4184696348 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40afb40 con 0x7fa5d40603f0 2021-04-03T04:03:15.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d259c700 1 --2- 172.21.2.111:0/4184696348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40603f0 0x7fa5d40b0090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa5c4002f30 tx=0x7fa5c4006e90).ready entity=mon.2 client_cookie=dc126ce50b132f4 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:15.197 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d159a700 1 -- 172.21.2.111:0/4184696348 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c4009850 con 0x7fa5d40603f0 2021-04-03T04:03:15.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d159a700 1 -- 172.21.2.111:0/4184696348 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40099b0 con 0x7fa5d40603f0 2021-04-03T04:03:15.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d159a700 1 -- 172.21.2.111:0/4184696348 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c4009d10 con 0x7fa5d40603f0 2021-04-03T04:03:15.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 -- 172.21.2.111:0/4184696348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40603f0 msgr2=0x7fa5d40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.198 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.183+0000 7fa5d359e700 1 --2- 172.21.2.111:0/4184696348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40603f0 0x7fa5d40b0090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa5c4002f30 tx=0x7fa5c4006e90).stop 2021-04-03T04:03:15.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- 172.21.2.111:0/4184696348 shutdown_connections 2021-04-03T04:03:15.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 --2- 172.21.2.111:0/4184696348 >> v2:172.21.2.115:3300/0 conn(0x7fa5d40b0c90 0x7fa5d4147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.199 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 --2- 172.21.2.111:0/4184696348 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40603f0 0x7fa5d40b0090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- 172.21.2.111:0/4184696348 >> 172.21.2.111:0/4184696348 conn(0x7fa5d40b61c0 msgr2=0x7fa5d405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:15.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- 172.21.2.111:0/4184696348 shutdown_connections 2021-04-03T04:03:15.200 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- 172.21.2.111:0/4184696348 wait complete. 2021-04-03T04:03:15.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 Processor -- start 2021-04-03T04:03:15.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- start start 2021-04-03T04:03:15.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d40603f0 0x7fa5d4064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.201 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40b0c90 0x7fa5d4062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d414b700 0x7fa5d4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40abd70 con 0x7fa5d40603f0 2021-04-03T04:03:15.202 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40641d0 con 0x7fa5d414b700 2021-04-03T04:03:15.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4064470 con 0x7fa5d40b0c90 2021-04-03T04:03:15.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d40603f0 0x7fa5d4064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.203 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d40603f0 0x7fa5d4064bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59368/0 (socket says 172.21.2.111:59368) 2021-04-03T04:03:15.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 -- 172.21.2.111:0/636723763 learned_addr learned my addr 172.21.2.111:0/636723763 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:15.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 -- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40b0c90 msgr2=0x7fa5d4062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d1d9b700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40b0c90 0x7fa5d4062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.204 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d2d9d700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d414b700 0x7fa5d4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40b0c90 0x7fa5d4062c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 -- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d414b700 msgr2=0x7fa5d4159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.205 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d414b700 0x7fa5d4159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 -- 172.21.2.111:0/636723763 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4064710 con 0x7fa5d40603f0 2021-04-03T04:03:15.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d2d9d700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d414b700 0x7fa5d4159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:15.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d259c700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d40603f0 0x7fa5d4064bb0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7fa5c4002f30 tx=0x7fa5c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:15.206 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5cb7fe700 1 -- 172.21.2.111:0/636723763 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c4011ce0 con 0x7fa5d40603f0 2021-04-03T04:03:15.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5cb7fe700 1 -- 172.21.2.111:0/636723763 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c4011e40 con 0x7fa5d40603f0 2021-04-03T04:03:15.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5cb7fe700 1 -- 172.21.2.111:0/636723763 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c401e650 con 0x7fa5d40603f0 2021-04-03T04:03:15.207 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d4157740 con 0x7fa5d40603f0 2021-04-03T04:03:15.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d40aa4d0 con 0x7fa5d40603f0 2021-04-03T04:03:15.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d40aa2e0 con 0x7fa5d40603f0 2021-04-03T04:03:15.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5cb7fe700 1 -- 172.21.2.111:0/636723763 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa5c4025030 con 0x7fa5d40603f0 2021-04-03T04:03:15.208 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5cb7fe700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b8056f20 0x7fa5b80593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:15.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5cb7fe700 1 -- 172.21.2.111:0/636723763 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fa5c4071310 con 0x7fa5d40603f0 2021-04-03T04:03:15.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.187+0000 7fa5d1d9b700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b8056f20 0x7fa5b80593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:15.209 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.191+0000 7fa5d1d9b700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b8056f20 0x7fa5b80593e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fa5bc008000 tx=0x7fa5bc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:15.210 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.191+0000 7fa5cb7fe700 1 -- 172.21.2.111:0/636723763 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa5c40469d0 con 0x7fa5d40603f0 2021-04-03T04:03:15.290 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.283+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5d40648a0 con 0x7fa5d40603f0 2021-04-03T04:03:15.291 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.283+0000 7fa5cb7fe700 1 -- 172.21.2.111:0/636723763 <== mon.0 v2:172.21.2.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13551 (secure 0 0 0) 0x7fa5c4023090 con 0x7fa5d40603f0 2021-04-03T04:03:15.291 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T04:03:15.292 INFO:teuthology.orchestra.run.gibba011.stdout:{"epoch":154,"fsid":"6fa64cfd-c000-43a2-869e-5811f8b4928c","created":"2021-04-03T03:08:26.873888+0000","modified":"2021-04-03T03:17:43.996443+0000","last_up_change":"2021-04-03T03:08:34.085903+0000","last_in_change":"2021-04-03T03:08:34.085903+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-04-03T03:08:29.718060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-04-03T03:08:38.084175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b944db97-2a22-405b-847f-9d436ceab0ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6812","nonce":15909},{"type":"v1","addr":"172.21.2.111:6814","nonce":15909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6816","nonce":15909},{"type":"v1","addr":"172.21.2.111:6818","nonce":15909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6824","nonce":15909},{"type":"v1","addr":"172.21.2.111:6825","nonce":15909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6820","nonce":15909},{"type":"v1","addr":"172.21.2.111:6822","nonce":15909}]},"public_addr":"172.21.2.111:6814/15909","cluster_addr":"172.21.2.111:6818/15909","heartbeat_back_addr":"172.21.2.111:6825/15909","heartbeat_front_addr":"172.21.2.111:6822/15909","state":["exists","up"]},{"osd":1,"uuid":"a1b55cbd-c469-405c-92a4-a45166c2194a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6802","nonce":15910},{"type":"v1","addr":"172.21.2.111:6803","nonce":15910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6804","nonce":15910},{"type":"v1","addr":"172.21.2.111:6805","nonce":15910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6808","nonce":15910},{"type":"v1","addr":"172.21.2.111:6809","nonce":15910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6806","nonce":15910},{"type":"v1","addr":"172.21.2.111:6807","nonce":15910}]},"public_addr":"172.21.2.111:6803/15910","cluster_addr":"172.21.2.111:6805/15910","heartbeat_back_addr":"172.21.2.111:6809/15910","heartbeat_front_addr":"172.21.2.111:6807/15910","state":["exists","up"]},{"osd":2,"uuid":"0954b9bc-6c00-42c3-b666-fe5b20532ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6810","nonce":15911},{"type":"v1","addr":"172.21.2.111:6811","nonce":15911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6813","nonce":15911},{"type":"v1","addr":"172.21.2.111:6815","nonce":15911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6821","nonce":15911},{"type":"v1","addr":"172.21.2.111:6823","nonce":15911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6817","nonce":15911},{"type":"v1","addr":"172.21.2.111:6819","nonce":15911}]},"public_addr":"172.21.2.111:6811/15911","cluster_addr":"172.21.2.111:6815/15911","heartbeat_back_addr":"172.21.2.111:6823/15911","heartbeat_front_addr":"172.21.2.111:6819/15911","state":["exists","up"]},{"osd":3,"uuid":"69ab5309-0edd-4359-a5b8-56d9ceac0649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6826","nonce":15913},{"type":"v1","addr":"172.21.2.111:6827","nonce":15913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6828","nonce":15913},{"type":"v1","addr":"172.21.2.111:6829","nonce":15913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6832","nonce":15913},{"type":"v1","addr":"172.21.2.111:6833","nonce":15913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.111:6830","nonce":15913},{"type":"v1","addr":"172.21.2.111:6831","nonce":15913}]},"public_addr":"172.21.2.111:6827/15913","cluster_addr":"172.21.2.111:6829/15913","heartbeat_back_addr":"172.21.2.111:6833/15913","heartbeat_front_addr":"172.21.2.111:6831/15913","state":["exists","up"]},{"osd":4,"uuid":"107b94d0-940a-4a28-b981-4eb105bea5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6800","nonce":16012},{"type":"v1","addr":"172.21.2.115:6801","nonce":16012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6802","nonce":16012},{"type":"v1","addr":"172.21.2.115:6803","nonce":16012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6806","nonce":16012},{"type":"v1","addr":"172.21.2.115:6807","nonce":16012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6804","nonce":16012},{"type":"v1","addr":"172.21.2.115:6805","nonce":16012}]},"public_addr":"172.21.2.115:6801/16012","cluster_addr":"172.21.2.115:6803/16012","heartbeat_back_addr":"172.21.2.115:6807/16012","heartbeat_front_addr":"172.21.2.115:6805/16012","state":["exists","up"]},{"osd":5,"uuid":"b020dc25-a06c-4de5-8896-b6af08322240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6816","nonce":16014},{"type":"v1","addr":"172.21.2.115:6817","nonce":16014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6818","nonce":16014},{"type":"v1","addr":"172.21.2.115:6819","nonce":16014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6822","nonce":16014},{"type":"v1","addr":"172.21.2.115:6823","nonce":16014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6820","nonce":16014},{"type":"v1","addr":"172.21.2.115:6821","nonce":16014}]},"public_addr":"172.21.2.115:6817/16014","cluster_addr":"172.21.2.115:6819/16014","heartbeat_back_addr":"172.21.2.115:6823/16014","heartbeat_front_addr":"172.21.2.115:6821/16014","state":["exists","up"]},{"osd":6,"uuid":"cc3194fc-248c-46e3-a1a6-c149abc74a9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6808","nonce":16015},{"type":"v1","addr":"172.21.2.115:6809","nonce":16015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6810","nonce":16015},{"type":"v1","addr":"172.21.2.115:6811","nonce":16015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6814","nonce":16015},{"type":"v1","addr":"172.21.2.115:6815","nonce":16015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6812","nonce":16015},{"type":"v1","addr":"172.21.2.115:6813","nonce":16015}]},"public_addr":"172.21.2.115:6809/16015","cluster_addr":"172.21.2.115:6811/16015","heartbeat_back_addr":"172.21.2.115:6815/16015","heartbeat_front_addr":"172.21.2.115:6813/16015","state":["exists","up"]},{"osd":7,"uuid":"f091dcf7-acf3-4d21-b0ad-c2ab84a82035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6824","nonce":16058},{"type":"v1","addr":"172.21.2.115:6825","nonce":16058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6826","nonce":16058},{"type":"v1","addr":"172.21.2.115:6827","nonce":16058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6830","nonce":16058},{"type":"v1","addr":"172.21.2.115:6831","nonce":16058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.115:6828","nonce":16058},{"type":"v1","addr":"172.21.2.115:6829","nonce":16058}]},"public_addr":"172.21.2.115:6825/16058","cluster_addr":"172.21.2.115:6827/16058","heartbeat_back_addr":"172.21.2.115:6831/16058","heartbeat_front_addr":"172.21.2.115:6829/16058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.417965+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.629207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.680350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.507066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.665996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.763971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.675661+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-04-03T03:08:32.498476+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.111:6835/295340141":"2021-04-04T03:17:40.389472+0000","172.21.2.111:6834/295340141":"2021-04-04T03:17:40.389472+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-04-03T04:03:15.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b8056f20 msgr2=0x7fa5b80593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b8056f20 0x7fa5b80593e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fa5bc008000 tx=0x7fa5bc009040).stop 2021-04-03T04:03:15.293 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d40603f0 msgr2=0x7fa5d4064bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:15.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d40603f0 0x7fa5d4064bb0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7fa5c4002f30 tx=0x7fa5c4015000).stop 2021-04-03T04:03:15.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 shutdown_connections 2021-04-03T04:03:15.294 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa5b8056f20 0x7fa5b80593e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa5d414b700 0x7fa5d4159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa5d40603f0 0x7fa5d4064bb0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.295 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 --2- 172.21.2.111:0/636723763 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa5d40b0c90 0x7fa5d4062c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:15.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 >> 172.21.2.111:0/636723763 conn(0x7fa5d40b61c0 msgr2=0x7fa5d4147e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:15.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 shutdown_connections 2021-04-03T04:03:15.296 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:15.287+0000 7fa5d359e700 1 -- 172.21.2.111:0/636723763 wait complete. 2021-04-03T04:03:16.300 INFO:tasks.ceph:Scrubbing osd.0 2021-04-03T04:03:16.301 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:16.362 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff443fff700 1 Processor -- start 2021-04-03T04:03:16.363 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff443fff700 1 -- start start 2021-04-03T04:03:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff443fff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff43c03f880 0x7ff43c0d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff443fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff43c039dd0 0x7ff43c03fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.365 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff443fff700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0d1c70 con 0x7ff43c0ceed0 2021-04-03T04:03:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff443fff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c040150 con 0x7ff43c039dd0 2021-04-03T04:03:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff443fff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c0d2f00 con 0x7ff43c03f880 2021-04-03T04:03:16.366 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff4427fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff43c039dd0 0x7ff43c03fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff4427fc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff43c039dd0 0x7ff43c03fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59380/0 (socket says 172.21.2.111:59380) 2021-04-03T04:03:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff4427fc700 1 -- 172.21.2.111:0/1165184029 learned_addr learned my addr 172.21.2.111:0/1165184029 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.367 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff4427fc700 1 -- 172.21.2.111:0/1165184029 >> v2:172.21.2.115:3300/0 conn(0x7ff43c03f880 msgr2=0x7ff43c0d2880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff4427fc700 1 --2- 172.21.2.111:0/1165184029 >> v2:172.21.2.115:3300/0 conn(0x7ff43c03f880 0x7ff43c0d2880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff4427fc700 1 -- 172.21.2.111:0/1165184029 >> v1:172.21.2.115:6789/0 conn(0x7ff43c0ceed0 legacy=0x7ff43c005680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.355+0000 7ff4427fc700 1 -- 172.21.2.111:0/1165184029 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0d1ea0 con 0x7ff43c039dd0 2021-04-03T04:03:16.368 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 --2- 172.21.2.111:0/1165184029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c039dd0 0x7ff43c03fc60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7ff43c00c350 tx=0x7ff434006e90).ready entity=mon.0 client_cookie=4f4ca37d23d19112 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff441ffb700 1 -- 172.21.2.111:0/1165184029 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff434009930 con 0x7ff43c039dd0 2021-04-03T04:03:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff441ffb700 1 -- 172.21.2.111:0/1165184029 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff434009a90 con 0x7ff43c039dd0 2021-04-03T04:03:16.369 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff441ffb700 1 -- 172.21.2.111:0/1165184029 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff434009e30 con 0x7ff43c039dd0 2021-04-03T04:03:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/1165184029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c039dd0 msgr2=0x7ff43c03fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 --2- 172.21.2.111:0/1165184029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c039dd0 0x7ff43c03fc60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7ff43c00c350 tx=0x7ff434006e90).stop 2021-04-03T04:03:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/1165184029 shutdown_connections 2021-04-03T04:03:16.370 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 --2- 172.21.2.111:0/1165184029 >> v2:172.21.2.115:3300/0 conn(0x7ff43c03f880 0x7ff43c0d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 --2- 172.21.2.111:0/1165184029 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c039dd0 0x7ff43c03fc60 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/1165184029 >> 172.21.2.111:0/1165184029 conn(0x7ff43c0462b0 msgr2=0x7ff43c0466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.371 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/1165184029 shutdown_connections 2021-04-03T04:03:16.372 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/1165184029 wait complete. 2021-04-03T04:03:16.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 Processor -- start 2021-04-03T04:03:16.374 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- start start 2021-04-03T04:03:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff43c039dd0 0x7ff43c03e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c03f880 0x7ff43c0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.375 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff43c0ceed0 0x7ff43c0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c03d5a0 con 0x7ff43c03f880 2021-04-03T04:03:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c03d840 con 0x7ff43c039dd0 2021-04-03T04:03:16.376 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0dedb0 con 0x7ff43c0ceed0 2021-04-03T04:03:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c03f880 0x7ff43c0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c03f880 0x7ff43c0b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59386/0 (socket says 172.21.2.111:59386) 2021-04-03T04:03:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 -- 172.21.2.111:0/4036063442 learned_addr learned my addr 172.21.2.111:0/4036063442 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.377 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 -- 172.21.2.111:0/4036063442 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff43c0ceed0 msgr2=0x7ff43c0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff43c0ceed0 0x7ff43c0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 -- 172.21.2.111:0/4036063442 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff43c039dd0 msgr2=0x7ff43c03e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.378 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff43c039dd0 0x7ff43c03e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 -- 172.21.2.111:0/4036063442 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0b4e80 con 0x7ff43c03f880 2021-04-03T04:03:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff4427fc700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c03f880 0x7ff43c0b6bd0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7ff43c00c350 tx=0x7ff434015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.379 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff434011d90 con 0x7ff43c03f880 2021-04-03T04:03:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43401e460 con 0x7ff43c03f880 2021-04-03T04:03:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c0ba5e0 con 0x7ff43c03f880 2021-04-03T04:03:16.380 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c03ea70 con 0x7ff43c03f880 2021-04-03T04:03:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff43401e7c0 con 0x7ff43c03f880 2021-04-03T04:03:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.359+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff43c004db0 con 0x7ff43c03f880 2021-04-03T04:03:16.381 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff434025030 con 0x7ff43c03f880 2021-04-03T04:03:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff43bfff700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff4280570a0 0x7ff428059560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7ff434071ca0 con 0x7ff43c03f880 2021-04-03T04:03:16.382 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff43bfff700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7ff42805dec0 0x7ff4280602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 --> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4280608c0 con 0x7ff42805dec0 2021-04-03T04:03:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff434071fb0 con 0x7ff43c03f880 2021-04-03T04:03:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff442ffd700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff4280570a0 0x7ff428059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.383 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff4437fe700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7ff42805dec0 0x7ff4280602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff442ffd700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff4280570a0 0x7ff428059560 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7ff42c002f70 tx=0x7ff42c006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff4437fe700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7ff42805dec0 0x7ff4280602e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.384 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.363+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== osd.0 v2:172.21.2.111:6812/15909 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7ff4280608c0 con 0x7ff42805dec0 2021-04-03T04:03:16.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.371+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 --> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff43c03aba0 con 0x7ff42805dec0 2021-04-03T04:03:16.385 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.371+0000 7ff43bfff700 1 -- 172.21.2.111:0/4036063442 <== osd.0 v2:172.21.2.111:6812/15909 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff43c03aba0 con 0x7ff42805dec0 2021-04-03T04:03:16.386 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:16.386 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:16.387 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:16.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7ff42805dec0 msgr2=0x7ff4280602e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.388 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6812/15909,v1:172.21.2.111:6814/15909] conn(0x7ff42805dec0 0x7ff4280602e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff4280570a0 msgr2=0x7ff428059560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff4280570a0 0x7ff428059560 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7ff42c002f70 tx=0x7ff42c006990).stop 2021-04-03T04:03:16.389 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c03f880 msgr2=0x7ff43c0b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 --2- 172.21.2.111:0/4036063442 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff43c03f880 0x7ff43c0b6bd0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7ff43c00c350 tx=0x7ff434015000).stop 2021-04-03T04:03:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff4437fe700 1 -- 172.21.2.111:0/4036063442 reap_dead start 2021-04-03T04:03:16.390 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 shutdown_connections 2021-04-03T04:03:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 >> 172.21.2.111:0/4036063442 conn(0x7ff43c0462b0 msgr2=0x7ff43c0baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 shutdown_connections 2021-04-03T04:03:16.391 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.375+0000 7ff443fff700 1 -- 172.21.2.111:0/4036063442 wait complete. 2021-04-03T04:03:16.392 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 0 2021-04-03T04:03:16.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 Processor -- start 2021-04-03T04:03:16.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 -- start start 2021-04-03T04:03:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 0x7f09280e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09280090c0 con 0x7f0928043120 2021-04-03T04:03:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09280492d0 con 0x7f0928048ef0 2021-04-03T04:03:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09280448d0 con 0x7f09280e0e50 2021-04-03T04:03:16.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092effd700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f0928043120 0x7f0928006040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60186/0 (socket says 172.21.2.111:60186) 2021-04-03T04:03:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092effd700 1 -- 172.21.2.111:0/3557085003 learned_addr learned my addr 172.21.2.111:0/3557085003 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259910639 0 0) 0x7f09280090c0 con 0x7f0928043120 2021-04-03T04:03:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0914003240 con 0x7f0928043120 2021-04-03T04:03:16.457 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092ffff700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 0x7f09280e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 129092638 0 0) 0x7f0914003240 con 0x7f0928043120 2021-04-03T04:03:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 msgr2=0x7f09280e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:16.458 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 0x7f09280e3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 >> v1:172.21.2.115:6789/0 conn(0x7f0928048ef0 legacy=0x7f0928005930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928006750 con 0x7f0928043120 2021-04-03T04:03:16.459 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f091c0016d0 con 0x7f0928043120 2021-04-03T04:03:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 >> v1:172.21.2.111:6789/0 conn(0x7f0928043120 legacy=0x7f0928006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0914003760 0x7f0914003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 0x7f09140044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.460 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09140049d0 0x7f0914004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09280448d0 con 0x7f0914003760 2021-04-03T04:03:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914004250 con 0x7f09140049d0 2021-04-03T04:03:16.461 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091400b730 con 0x7f09280e0e50 2021-04-03T04:03:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092f7fe700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0914003760 0x7f0914003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092ffff700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09140049d0 0x7f0914004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092effd700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 0x7f09140044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.462 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092f7fe700 1 -- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 msgr2=0x7f09140044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092f7fe700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 0x7f09140044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092f7fe700 1 -- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09140049d0 msgr2=0x7f0914004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:16.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092f7fe700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09140049d0 0x7f0914004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092f7fe700 1 -- 172.21.2.111:0/3557085003 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914003240 con 0x7f0914003760 2021-04-03T04:03:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092ffff700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f09140049d0 0x7f0914004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092effd700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280e0e50 0x7f09140044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:16.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092f7fe700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0914003760 0x7f0914003b80 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f0928042b50 tx=0x7f0924007b70).ready entity=mon.0 client_cookie=1394c1f2c83fbefb server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f092400c660 con 0x7f0914003760 2021-04-03T04:03:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400c7c0 con 0x7f0914003760 2021-04-03T04:03:16.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092e7fc700 1 -- 172.21.2.111:0/3557085003 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f092400cb20 con 0x7f0914003760 2021-04-03T04:03:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 -- 172.21.2.111:0/3557085003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0914003760 msgr2=0x7f0914003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 --2- 172.21.2.111:0/3557085003 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f0914003760 0x7f0914003b80 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f0928042b50 tx=0x7f0924007b70).stop 2021-04-03T04:03:16.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f092ffff700 1 -- 172.21.2.111:0/3557085003 reap_dead start 2021-04-03T04:03:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 -- 172.21.2.111:0/3557085003 shutdown_connections 2021-04-03T04:03:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.443+0000 7f0934ba0700 1 -- 172.21.2.111:0/3557085003 >> 172.21.2.111:0/3557085003 conn(0x7f092800f350 msgr2=0x7f09280484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- 172.21.2.111:0/3557085003 shutdown_connections 2021-04-03T04:03:16.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- 172.21.2.111:0/3557085003 wait complete. 2021-04-03T04:03:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 Processor -- start 2021-04-03T04:03:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- start start 2021-04-03T04:03:16.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280da130 0x7f092803c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f092803cc10 0x7f092803d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09280d7560 0x7f09280d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09280a38c0 con 0x7f09280d7560 2021-04-03T04:03:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092803e870 con 0x7f092803cc10 2021-04-03T04:03:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09280a55b0 con 0x7f09280da130 2021-04-03T04:03:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09280d7560 0x7f09280d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09280d7560 0x7f09280d79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59406/0 (socket says 172.21.2.111:59406) 2021-04-03T04:03:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 -- 172.21.2.111:0/977700325 learned_addr learned my addr 172.21.2.111:0/977700325 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 -- 172.21.2.111:0/977700325 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280da130 msgr2=0x7f092803c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280da130 0x7f092803c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 -- 172.21.2.111:0/977700325 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f092803cc10 msgr2=0x7f092803d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f092803cc10 0x7f092803d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 -- 172.21.2.111:0/977700325 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092803d580 con 0x7f09280d7560 2021-04-03T04:03:16.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092ffff700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09280d7560 0x7f09280d79e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f0930067000 tx=0x7f093005ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092cff9700 1 -- 172.21.2.111:0/977700325 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09300685b0 con 0x7f09280d7560 2021-04-03T04:03:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092cff9700 1 -- 172.21.2.111:0/977700325 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930068710 con 0x7f09280d7560 2021-04-03T04:03:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09280a5330 con 0x7f09280d7560 2021-04-03T04:03:16.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092cff9700 1 -- 172.21.2.111:0/977700325 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f093005f2c0 con 0x7f09280d7560 2021-04-03T04:03:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092803e5d0 con 0x7f09280d7560 2021-04-03T04:03:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0928004d30 con 0x7f09280d7560 2021-04-03T04:03:16.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092cff9700 1 -- 172.21.2.111:0/977700325 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f093005f420 con 0x7f09280d7560 2021-04-03T04:03:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092cff9700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0918056ea0 0x7f0918059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092cff9700 1 -- 172.21.2.111:0/977700325 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f09300df930 con 0x7f09280d7560 2021-04-03T04:03:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092f7fe700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0918056ea0 0x7f0918059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.475 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.447+0000 7f092f7fe700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0918056ea0 0x7f0918059360 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f0928042b50 tx=0x7f0924017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.451+0000 7f092cff9700 1 -- 172.21.2.111:0/977700325 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f09300b4f90 con 0x7f09280d7560 2021-04-03T04:03:16.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.543+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f092803d580 con 0x7f0918056ea0 2021-04-03T04:03:16.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.543+0000 7f092cff9700 1 -- 172.21.2.111:0/977700325 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f092803d580 con 0x7f0918056ea0 2021-04-03T04:03:16.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0918056ea0 msgr2=0x7f0918059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0918056ea0 0x7f0918059360 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f0928042b50 tx=0x7f0924017000).stop 2021-04-03T04:03:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09280d7560 msgr2=0x7f09280d79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09280d7560 0x7f09280d79e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f0930067000 tx=0x7f093005ad70).stop 2021-04-03T04:03:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 shutdown_connections 2021-04-03T04:03:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f09280d7560 0x7f09280d79e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f09280da130 0x7f092803c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f0918056ea0 0x7f0918059360 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 --2- 172.21.2.111:0/977700325 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f092803cc10 0x7f092803d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 >> 172.21.2.111:0/977700325 conn(0x7f092800f350 msgr2=0x7f09280dafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 shutdown_connections 2021-04-03T04:03:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.547+0000 7f0934ba0700 1 -- 172.21.2.111:0/977700325 wait complete. 2021-04-03T04:03:16.556 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 0 to deep-scrub 2021-04-03T04:03:16.559 INFO:tasks.ceph:Scrubbing osd.1 2021-04-03T04:03:16.560 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:16.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 Processor -- start 2021-04-03T04:03:16.618 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 -- start start 2021-04-03T04:03:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f97780603f0 0x7f97781447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97780add50 0x7f9778144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.624 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97780b17b0 0x7f9778147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97781451c0 con 0x7f97780b17b0 2021-04-03T04:03:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97780607d0 con 0x7f97780603f0 2021-04-03T04:03:16.625 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977814b090 con 0x7f97780add50 2021-04-03T04:03:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97780b17b0 0x7f9778147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f97780b17b0 0x7f9778147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59414/0 (socket says 172.21.2.111:59414) 2021-04-03T04:03:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 -- 172.21.2.111:0/3843672713 learned_addr learned my addr 172.21.2.111:0/3843672713 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.626 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 -- 172.21.2.111:0/3843672713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97780add50 msgr2=0x7f9778144cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 --2- 172.21.2.111:0/3843672713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97780add50 0x7f9778144cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 -- 172.21.2.111:0/3843672713 >> v2:172.21.2.115:3300/0 conn(0x7f97780603f0 msgr2=0x7f97781447e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:16.627 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 --2- 172.21.2.111:0/3843672713 >> v2:172.21.2.115:3300/0 conn(0x7f97780603f0 0x7f97781447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 -- 172.21.2.111:0/3843672713 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977814b330 con 0x7f97780b17b0 2021-04-03T04:03:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977e6f0700 1 --2- 172.21.2.111:0/3843672713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780b17b0 0x7f9778147740 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f97700032b0 tx=0x7f9770009270).ready entity=mon.0 client_cookie=cf9c3360f174e27c server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977ceed700 1 -- 172.21.2.111:0/3843672713 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f977000bc00 con 0x7f97780b17b0 2021-04-03T04:03:16.628 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977ceed700 1 -- 172.21.2.111:0/3843672713 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977000bd60 con 0x7f97780b17b0 2021-04-03T04:03:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977ceed700 1 -- 172.21.2.111:0/3843672713 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9770011400 con 0x7f97780b17b0 2021-04-03T04:03:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 -- 172.21.2.111:0/3843672713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780b17b0 msgr2=0x7f9778147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.629 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.611+0000 7f977eef1700 1 --2- 172.21.2.111:0/3843672713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780b17b0 0x7f9778147740 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f97700032b0 tx=0x7f9770009270).stop 2021-04-03T04:03:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- 172.21.2.111:0/3843672713 shutdown_connections 2021-04-03T04:03:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 --2- 172.21.2.111:0/3843672713 >> v2:172.21.2.115:3300/0 conn(0x7f97780603f0 0x7f97781447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 --2- 172.21.2.111:0/3843672713 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780b17b0 0x7f9778147740 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.630 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 --2- 172.21.2.111:0/3843672713 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97780add50 0x7f9778144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- 172.21.2.111:0/3843672713 >> 172.21.2.111:0/3843672713 conn(0x7f97780b61c0 msgr2=0x7f977805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- 172.21.2.111:0/3843672713 shutdown_connections 2021-04-03T04:03:16.631 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- 172.21.2.111:0/3843672713 wait complete. 2021-04-03T04:03:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 Processor -- start 2021-04-03T04:03:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- start start 2021-04-03T04:03:16.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97780603f0 0x7f97780abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780add50 0x7f9778067210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97780b17b0 0x7f97780b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778064f10 con 0x7f97780add50 2021-04-03T04:03:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778066a10 con 0x7f97780603f0 2021-04-03T04:03:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9778066cb0 con 0x7f97780b17b0 2021-04-03T04:03:16.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780add50 0x7f9778067210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780add50 0x7f9778067210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59418/0 (socket says 172.21.2.111:59418) 2021-04-03T04:03:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 -- 172.21.2.111:0/1446931332 learned_addr learned my addr 172.21.2.111:0/1446931332 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 -- 172.21.2.111:0/1446931332 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97780b17b0 msgr2=0x7f97780b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f97780b17b0 0x7f97780b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 -- 172.21.2.111:0/1446931332 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97780603f0 msgr2=0x7f97780abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f97780603f0 0x7f97780abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 -- 172.21.2.111:0/1446931332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778065c50 con 0x7f97780add50 2021-04-03T04:03:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977d6ee700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780add50 0x7f9778067210 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f9768002f70 tx=0x7f9768006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f976800d570 con 0x7f97780add50 2021-04-03T04:03:16.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976800d6d0 con 0x7f97780add50 2021-04-03T04:03:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9778065f00 con 0x7f97780add50 2021-04-03T04:03:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f976800da30 con 0x7f97780add50 2021-04-03T04:03:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9778143b90 con 0x7f97780add50 2021-04-03T04:03:16.638 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9778057660 con 0x7f97780add50 2021-04-03T04:03:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f976800f070 con 0x7f97780add50 2021-04-03T04:03:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9758056ef0 0x7f97580593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.639 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f9768070110 con 0x7f97780add50 2021-04-03T04:03:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977deef700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9758056ef0 0x7f97580593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f975805dd30 0x7f9758060150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.640 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 --> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97580607d0 con 0x7f975805dd30 2021-04-03T04:03:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9768070470 con 0x7f97780add50 2021-04-03T04:03:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977e6f0700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f975805dd30 0x7f9758060150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.641 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977deef700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9758056ef0 0x7f97580593b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f9774002f70 tx=0x7f9774006990).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.615+0000 7f977e6f0700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f975805dd30 0x7f9758060150 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.642 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.619+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== osd.1 v2:172.21.2.111:6802/15910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f97580607d0 con 0x7f975805dd30 2021-04-03T04:03:16.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 --> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9778066ff0 con 0x7f975805dd30 2021-04-03T04:03:16.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f9766ffd700 1 -- 172.21.2.111:0/1446931332 <== osd.1 v2:172.21.2.111:6802/15910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9778066ff0 con 0x7f975805dd30 2021-04-03T04:03:16.644 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:16.644 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:16.644 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:16.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f975805dd30 msgr2=0x7f9758060150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6802/15910,v1:172.21.2.111:6803/15910] conn(0x7f975805dd30 0x7f9758060150 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9758056ef0 msgr2=0x7f97580593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f9758056ef0 0x7f97580593b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f9774002f70 tx=0x7f9774006990).stop 2021-04-03T04:03:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780add50 msgr2=0x7f9778067210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 --2- 172.21.2.111:0/1446931332 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f97780add50 0x7f9778067210 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f9768002f70 tx=0x7f9768006de0).stop 2021-04-03T04:03:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977e6f0700 1 -- 172.21.2.111:0/1446931332 reap_dead start 2021-04-03T04:03:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 shutdown_connections 2021-04-03T04:03:16.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 >> 172.21.2.111:0/1446931332 conn(0x7f97780b61c0 msgr2=0x7f9778147320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 shutdown_connections 2021-04-03T04:03:16.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.627+0000 7f977eef1700 1 -- 172.21.2.111:0/1446931332 wait complete. 2021-04-03T04:03:16.649 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 1 2021-04-03T04:03:16.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 Processor -- start 2021-04-03T04:03:16.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 -- start start 2021-04-03T04:03:16.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f24e80603f0 0x7f24e81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 0x7f24e8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f24e80b17b0 0x7f24e8147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e81451c0 con 0x7f24e80b17b0 2021-04-03T04:03:16.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e80607d0 con 0x7f24e80603f0 2021-04-03T04:03:16.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e814b090 con 0x7f24e80add50 2021-04-03T04:03:16.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f24e80b17b0 0x7f24e8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f24e80b17b0 0x7f24e8147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59432/0 (socket says 172.21.2.111:59432) 2021-04-03T04:03:16.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 -- 172.21.2.111:0/204043601 learned_addr learned my addr 172.21.2.111:0/204043601 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 -- 172.21.2.111:0/204043601 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 msgr2=0x7f24e8144cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:16.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 --2- 172.21.2.111:0/204043601 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 0x7f24e8144cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 -- 172.21.2.111:0/204043601 >> v2:172.21.2.115:3300/0 conn(0x7f24e80603f0 msgr2=0x7f24e81447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:16.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 --2- 172.21.2.111:0/204043601 >> v2:172.21.2.115:3300/0 conn(0x7f24e80603f0 0x7f24e81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 -- 172.21.2.111:0/204043601 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e814b330 con 0x7f24e80b17b0 2021-04-03T04:03:16.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24eccab700 1 --2- 172.21.2.111:0/204043601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e8147740 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f24e00032b0 tx=0x7f24e0009270).ready entity=mon.0 client_cookie=6e23cc872803ed3e server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24e6ffd700 1 -- 172.21.2.111:0/204043601 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24e000bc00 con 0x7f24e80b17b0 2021-04-03T04:03:16.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24e6ffd700 1 -- 172.21.2.111:0/204043601 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000bd60 con 0x7f24e80b17b0 2021-04-03T04:03:16.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24e6ffd700 1 -- 172.21.2.111:0/204043601 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24e0011400 con 0x7f24e80b17b0 2021-04-03T04:03:16.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 -- 172.21.2.111:0/204043601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 msgr2=0x7f24e8147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.724 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.707+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/204043601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e8147740 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f24e00032b0 tx=0x7f24e0009270).stop 2021-04-03T04:03:16.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- 172.21.2.111:0/204043601 shutdown_connections 2021-04-03T04:03:16.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/204043601 >> v2:172.21.2.115:3300/0 conn(0x7f24e80603f0 0x7f24e81447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/204043601 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e8147740 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.725 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/204043601 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 0x7f24e8144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- 172.21.2.111:0/204043601 >> 172.21.2.111:0/204043601 conn(0x7f24e80b61c0 msgr2=0x7f24e805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- 172.21.2.111:0/204043601 shutdown_connections 2021-04-03T04:03:16.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- 172.21.2.111:0/204043601 wait complete. 2021-04-03T04:03:16.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 Processor -- start 2021-04-03T04:03:16.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- start start 2021-04-03T04:03:16.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f24e80603f0 0x7f24e8063300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 0x7f24e80ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e80650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e80aab60 con 0x7f24e80b17b0 2021-04-03T04:03:16.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e8064c40 con 0x7f24e80603f0 2021-04-03T04:03:16.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e80aa5e0 con 0x7f24e80add50 2021-04-03T04:03:16.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e80650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e80650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59438/0 (socket says 172.21.2.111:59438) 2021-04-03T04:03:16.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 -- 172.21.2.111:0/161043893 learned_addr learned my addr 172.21.2.111:0/161043893 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 -- 172.21.2.111:0/161043893 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 msgr2=0x7f24e80ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.730 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 0x7f24e80ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 -- 172.21.2.111:0/161043893 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f24e80603f0 msgr2=0x7f24e8063300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f24e80603f0 0x7f24e8063300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.731 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 -- 172.21.2.111:0/161043893 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e80abc00 con 0x7f24e80b17b0 2021-04-03T04:03:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24eccab700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e80650e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f24e00032b0 tx=0x7f24e0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24e57fa700 1 -- 172.21.2.111:0/161043893 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24e001f610 con 0x7f24e80b17b0 2021-04-03T04:03:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24e57fa700 1 -- 172.21.2.111:0/161043893 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000bed0 con 0x7f24e80b17b0 2021-04-03T04:03:16.732 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e80655d0 con 0x7f24e80b17b0 2021-04-03T04:03:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24e57fa700 1 -- 172.21.2.111:0/161043893 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24e001f960 con 0x7f24e80b17b0 2021-04-03T04:03:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e815a170 con 0x7f24e80b17b0 2021-04-03T04:03:16.733 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e80aacd0 con 0x7f24e80b17b0 2021-04-03T04:03:16.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24e57fa700 1 -- 172.21.2.111:0/161043893 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f24e000f070 con 0x7f24e80b17b0 2021-04-03T04:03:16.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.711+0000 7f24e57fa700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24cc056ea0 0x7f24cc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.734 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.715+0000 7f24e57fa700 1 -- 172.21.2.111:0/161043893 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f24e00707b0 con 0x7f24e80b17b0 2021-04-03T04:03:16.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.715+0000 7f24e7fff700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24cc056ea0 0x7f24cc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.735 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.715+0000 7f24e7fff700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24cc056ea0 0x7f24cc059360 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f24d0002f30 tx=0x7f24d0006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.736 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.715+0000 7f24e57fa700 1 -- 172.21.2.111:0/161043893 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f24e80aacd0 con 0x7f24e80b17b0 2021-04-03T04:03:16.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f24e805f9c0 con 0x7f24cc056ea0 2021-04-03T04:03:16.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24e57fa700 1 -- 172.21.2.111:0/161043893 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f24e805f9c0 con 0x7f24cc056ea0 2021-04-03T04:03:16.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24cc056ea0 msgr2=0x7f24cc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24cc056ea0 0x7f24cc059360 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f24d0002f30 tx=0x7f24d0006a60).stop 2021-04-03T04:03:16.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 msgr2=0x7f24e80650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e80650e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f24e00032b0 tx=0x7f24e0016000).stop 2021-04-03T04:03:16.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 shutdown_connections 2021-04-03T04:03:16.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f24e80b17b0 0x7f24e80650e0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f24cc056ea0 0x7f24cc059360 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f24e80603f0 0x7f24e8063300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 --2- 172.21.2.111:0/161043893 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f24e80add50 0x7f24e80ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 >> 172.21.2.111:0/161043893 conn(0x7f24e80b61c0 msgr2=0x7f24e8144a50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 shutdown_connections 2021-04-03T04:03:16.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.807+0000 7f24ed4ac700 1 -- 172.21.2.111:0/161043893 wait complete. 2021-04-03T04:03:16.819 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 1 to deep-scrub 2021-04-03T04:03:16.822 INFO:tasks.ceph:Scrubbing osd.2 2021-04-03T04:03:16.823 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:16.880 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf1759e700 1 Processor -- start 2021-04-03T04:03:16.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf1759e700 1 -- start start 2021-04-03T04:03:16.881 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf1759e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7faf180add50 0x7faf18147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf1759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf1759e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf180607d0 con 0x7faf180603f0 2021-04-03T04:03:16.882 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf1759e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1814b090 con 0x7faf180add50 2021-04-03T04:03:16.883 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf1759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1814b330 con 0x7faf18144ef0 2021-04-03T04:03:16.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf15d9b700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7faf180add50 0x7faf18147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.884 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.115:3301/0 says I am v2:172.21.2.111:42794/0 (socket says 172.21.2.111:42794) 2021-04-03T04:03:16.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 -- 172.21.2.111:0/2459076634 learned_addr learned my addr 172.21.2.111:0/2459076634 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 -- 172.21.2.111:0/2459076634 >> v2:172.21.2.115:3300/0 conn(0x7faf180add50 msgr2=0x7faf18147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:16.885 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 --2- 172.21.2.111:0/2459076634 >> v2:172.21.2.115:3300/0 conn(0x7faf180add50 0x7faf18147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 -- 172.21.2.111:0/2459076634 >> v1:172.21.2.115:6789/0 conn(0x7faf180603f0 legacy=0x7faf181447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 -- 172.21.2.111:0/2459076634 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1814b5d0 con 0x7faf18144ef0 2021-04-03T04:03:16.886 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.875+0000 7faf16d9d700 1 --2- 172.21.2.111:0/2459076634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18147c30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7faf100032b0 tx=0x7faf10009270).ready entity=mon.2 client_cookie=67858992c2016e47 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1559a700 1 -- 172.21.2.111:0/2459076634 <== mon.2 v2:172.21.2.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf10009e60 con 0x7faf18144ef0 2021-04-03T04:03:16.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1559a700 1 -- 172.21.2.111:0/2459076634 <== mon.2 v2:172.21.2.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1000bc00 con 0x7faf18144ef0 2021-04-03T04:03:16.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1559a700 1 -- 172.21.2.111:0/2459076634 <== mon.2 v2:172.21.2.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf1000bd70 con 0x7faf18144ef0 2021-04-03T04:03:16.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/2459076634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 msgr2=0x7faf18147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 --2- 172.21.2.111:0/2459076634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18147c30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7faf100032b0 tx=0x7faf10009270).stop 2021-04-03T04:03:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/2459076634 shutdown_connections 2021-04-03T04:03:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 --2- 172.21.2.111:0/2459076634 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18147c30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 --2- 172.21.2.111:0/2459076634 >> v2:172.21.2.115:3300/0 conn(0x7faf180add50 0x7faf18147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/2459076634 >> 172.21.2.111:0/2459076634 conn(0x7faf180b61c0 msgr2=0x7faf1805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/2459076634 shutdown_connections 2021-04-03T04:03:16.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/2459076634 wait complete. 2021-04-03T04:03:16.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 Processor -- start 2021-04-03T04:03:16.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- start start 2021-04-03T04:03:16.893 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf180603f0 0x7faf18157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf180add50 0x7faf180a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf18143760 con 0x7faf180603f0 2021-04-03T04:03:16.894 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf18156d50 con 0x7faf180add50 2021-04-03T04:03:16.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf18156ff0 con 0x7faf18144ef0 2021-04-03T04:03:16.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf180603f0 0x7faf18157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.895 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf180603f0 0x7faf18157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59450/0 (socket says 172.21.2.111:59450) 2021-04-03T04:03:16.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 -- 172.21.2.111:0/3796686 learned_addr learned my addr 172.21.2.111:0/3796686 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf16d9d700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.896 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf15d9b700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf180add50 0x7faf180a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 -- 172.21.2.111:0/3796686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 msgr2=0x7faf18156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7faf18144ef0 0x7faf18156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.897 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 -- 172.21.2.111:0/3796686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf180add50 msgr2=0x7faf180a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7faf180add50 0x7faf180a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 -- 172.21.2.111:0/3796686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18157290 con 0x7faf180603f0 2021-04-03T04:03:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1659c700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf180603f0 0x7faf18157730 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7faf180b5600 tx=0x7faf00007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.898 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf00007240 con 0x7faf180603f0 2021-04-03T04:03:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf000073a0 con 0x7faf180603f0 2021-04-03T04:03:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf18157d00 con 0x7faf180603f0 2021-04-03T04:03:16.899 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf000066e0 con 0x7faf180603f0 2021-04-03T04:03:16.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf180aaa80 con 0x7faf180603f0 2021-04-03T04:03:16.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf1814a280 con 0x7faf180603f0 2021-04-03T04:03:16.900 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.879+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7faf00010030 con 0x7faf180603f0 2021-04-03T04:03:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf0f7fe700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faefc056ea0 0x7faefc059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7faf0006f960 con 0x7faf180603f0 2021-04-03T04:03:16.901 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf15d9b700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faefc056ea0 0x7faefc059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf0f7fe700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7faefc05dc60 0x7faefc060080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 --> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faefc060700 con 0x7faefc05dc60 2021-04-03T04:03:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf0006fd40 con 0x7faf180603f0 2021-04-03T04:03:16.902 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf16d9d700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7faefc05dc60 0x7faefc060080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf15d9b700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faefc056ea0 0x7faefc059360 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7faf0800b000 tx=0x7faf08008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf16d9d700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7faefc05dc60 0x7faefc060080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.903 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.883+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== osd.2 v2:172.21.2.111:6810/15911 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7faefc060700 con 0x7faefc05dc60 2021-04-03T04:03:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 --> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faf18157420 con 0x7faefc05dc60 2021-04-03T04:03:16.904 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf0f7fe700 1 -- 172.21.2.111:0/3796686 <== osd.2 v2:172.21.2.111:6810/15911 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faf18157420 con 0x7faefc05dc60 2021-04-03T04:03:16.905 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:16.905 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:16.906 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:16.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7faefc05dc60 msgr2=0x7faefc060080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.907 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6810/15911,v1:172.21.2.111:6811/15911] conn(0x7faefc05dc60 0x7faefc060080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faefc056ea0 msgr2=0x7faefc059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7faefc056ea0 0x7faefc059360 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7faf0800b000 tx=0x7faf08008040).stop 2021-04-03T04:03:16.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf180603f0 msgr2=0x7faf18157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 --2- 172.21.2.111:0/3796686 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7faf180603f0 0x7faf18157730 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7faf180b5600 tx=0x7faf00007bf0).stop 2021-04-03T04:03:16.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf16d9d700 1 -- 172.21.2.111:0/3796686 reap_dead start 2021-04-03T04:03:16.909 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 shutdown_connections 2021-04-03T04:03:16.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 >> 172.21.2.111:0/3796686 conn(0x7faf180b61c0 msgr2=0x7faf18144140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 shutdown_connections 2021-04-03T04:03:16.910 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.891+0000 7faf1759e700 1 -- 172.21.2.111:0/3796686 wait complete. 2021-04-03T04:03:16.911 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 2 2021-04-03T04:03:16.967 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.959+0000 7fd9896aa700 1 Processor -- start 2021-04-03T04:03:16.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.959+0000 7fd9896aa700 1 -- start start 2021-04-03T04:03:16.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fd9840603f0 0x7fd984067b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd98413d010 0x7fd98413d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd984140b60 con 0x7fd9840b18b0 2021-04-03T04:03:16.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9840af3f0 con 0x7fd98413d010 2021-04-03T04:03:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd984062bf0 con 0x7fd9840603f0 2021-04-03T04:03:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fd98413d010 0x7fd98413d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9837fe700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fd9840b18b0 0x7fd9840acb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60252/0 (socket says 172.21.2.111:60252) 2021-04-03T04:03:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9837fe700 1 -- 172.21.2.111:0/4236953288 learned_addr learned my addr 172.21.2.111:0/4236953288 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 -- 172.21.2.111:0/4236953288 >> v2:172.21.2.115:3300/0 conn(0x7fd9840603f0 msgr2=0x7fd984067b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- 172.21.2.111:0/4236953288 >> v2:172.21.2.115:3300/0 conn(0x7fd9840603f0 0x7fd984067b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 -- 172.21.2.111:0/4236953288 >> v1:172.21.2.111:6789/0 conn(0x7fd9840b18b0 legacy=0x7fd9840acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 -- 172.21.2.111:0/4236953288 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984068000 con 0x7fd98413d010 2021-04-03T04:03:16.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- 172.21.2.111:0/4236953288 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd98413d480 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7fd97c0032b0 tx=0x7fd97c0091d0).ready entity=mon.0 client_cookie=416feac0af7ba6dd server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd982ffd700 1 -- 172.21.2.111:0/4236953288 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd97c00ba50 con 0x7fd98413d010 2021-04-03T04:03:16.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd982ffd700 1 -- 172.21.2.111:0/4236953288 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00bbb0 con 0x7fd98413d010 2021-04-03T04:03:16.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd982ffd700 1 -- 172.21.2.111:0/4236953288 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd97c00bd20 con 0x7fd98413d010 2021-04-03T04:03:16.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- 172.21.2.111:0/4236953288 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 msgr2=0x7fd98413d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- 172.21.2.111:0/4236953288 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd98413d480 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7fd97c0032b0 tx=0x7fd97c0091d0).stop 2021-04-03T04:03:16.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- 172.21.2.111:0/4236953288 shutdown_connections 2021-04-03T04:03:16.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- 172.21.2.111:0/4236953288 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd98413d480 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:16.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- 172.21.2.111:0/4236953288 >> v2:172.21.2.115:3300/0 conn(0x7fd9840603f0 0x7fd984067b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- 172.21.2.111:0/4236953288 >> 172.21.2.111:0/4236953288 conn(0x7fd9840b61c0 msgr2=0x7fd98405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:16.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- 172.21.2.111:0/4236953288 shutdown_connections 2021-04-03T04:03:16.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- 172.21.2.111:0/4236953288 wait complete. 2021-04-03T04:03:16.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 Processor -- start 2021-04-03T04:03:16.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- start start 2021-04-03T04:03:16.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9840603f0 0x7fd9840ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd9840b18b0 0x7fd984064170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd984064660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.980 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd984066d80 con 0x7fd98413d010 2021-04-03T04:03:16.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd984133f30 con 0x7fd9840b18b0 2021-04-03T04:03:16.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd984066fb0 con 0x7fd9840603f0 2021-04-03T04:03:16.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd984064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd984064660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59466/0 (socket says 172.21.2.111:59466) 2021-04-03T04:03:16.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 -- 172.21.2.111:0/2132208309 learned_addr learned my addr 172.21.2.111:0/2132208309 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:16.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 -- 172.21.2.111:0/2132208309 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9840603f0 msgr2=0x7fd9840ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:16.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9840603f0 0x7fd9840ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 -- 172.21.2.111:0/2132208309 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd9840b18b0 msgr2=0x7fd984064170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-04-03T04:03:16.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd9840b18b0 0x7fd984064170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:16.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 -- 172.21.2.111:0/2132208309 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984134160 con 0x7fd98413d010 2021-04-03T04:03:16.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd988ea9700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd984064660 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7fd97c0032b0 tx=0x7fd97c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9817fa700 1 -- 172.21.2.111:0/2132208309 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd97c0107a0 con 0x7fd98413d010 2021-04-03T04:03:16.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9817fa700 1 -- 172.21.2.111:0/2132208309 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c010900 con 0x7fd98413d010 2021-04-03T04:03:16.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd984064b50 con 0x7fd98413d010 2021-04-03T04:03:16.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9817fa700 1 -- 172.21.2.111:0/2132208309 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd97c010c60 con 0x7fd98413d010 2021-04-03T04:03:16.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.963+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9840ae190 con 0x7fd98413d010 2021-04-03T04:03:16.985 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.967+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd984057660 con 0x7fd98413d010 2021-04-03T04:03:16.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.967+0000 7fd9817fa700 1 -- 172.21.2.111:0/2132208309 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fd97c027030 con 0x7fd98413d010 2021-04-03T04:03:16.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.967+0000 7fd9817fa700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd968056f20 0x7fd9680593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:16.986 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.967+0000 7fd9817fa700 1 -- 172.21.2.111:0/2132208309 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fd97c071000 con 0x7fd98413d010 2021-04-03T04:03:16.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.967+0000 7fd983fff700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd968056f20 0x7fd9680593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:16.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.967+0000 7fd983fff700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd968056f20 0x7fd9680593e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fd96c002f30 tx=0x7fd96c006a60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:16.987 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:16.967+0000 7fd9817fa700 1 -- 172.21.2.111:0/2132208309 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fd97c0466c0 con 0x7fd98413d010 2021-04-03T04:03:17.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9841342f0 con 0x7fd968056f20 2021-04-03T04:03:17.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9817fa700 1 -- 172.21.2.111:0/2132208309 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd96c000e90 con 0x7fd968056f20 2021-04-03T04:03:17.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd968056f20 msgr2=0x7fd9680593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd968056f20 0x7fd9680593e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fd96c002f30 tx=0x7fd96c006a60).stop 2021-04-03T04:03:17.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 msgr2=0x7fd984064660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd984064660 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7fd97c0032b0 tx=0x7fd97c016000).stop 2021-04-03T04:03:17.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 shutdown_connections 2021-04-03T04:03:17.071 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fd98413d010 0x7fd984064660 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fd9840603f0 0x7fd9840ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fd968056f20 0x7fd9680593e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.072 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 --2- 172.21.2.111:0/2132208309 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fd9840b18b0 0x7fd984064170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 >> 172.21.2.111:0/2132208309 conn(0x7fd9840b61c0 msgr2=0x7fd9840b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 shutdown_connections 2021-04-03T04:03:17.073 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.063+0000 7fd9896aa700 1 -- 172.21.2.111:0/2132208309 wait complete. 2021-04-03T04:03:17.074 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 2 to deep-scrub 2021-04-03T04:03:17.076 INFO:tasks.ceph:Scrubbing osd.3 2021-04-03T04:03:17.077 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:17.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0609ae4700 1 Processor -- start 2021-04-03T04:03:17.139 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0609ae4700 1 -- start start 2021-04-03T04:03:17.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0609ae4700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f0604144ef0 0x7f0604147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0609ae4700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040607d0 con 0x7f06040603f0 2021-04-03T04:03:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0609ae4700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060414b090 con 0x7f06040add50 2021-04-03T04:03:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0609ae4700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060414b330 con 0x7f0604144ef0 2021-04-03T04:03:17.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0608ae2700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f06040603f0 0x7f06041447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60264/0 (socket says 172.21.2.111:60264) 2021-04-03T04:03:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f0608ae2700 1 -- 172.21.2.111:0/3742414391 learned_addr learned my addr 172.21.2.111:0/3742414391 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110426603 0 0) 0x7f06040607d0 con 0x7f06040603f0 2021-04-03T04:03:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.131+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e4003240 con 0x7f06040603f0 2021-04-03T04:03:17.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3302170570 0 0) 0x7f05e4003240 con 0x7f06040603f0 2021-04-03T04:03:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 >> v2:172.21.2.115:3300/0 conn(0x7f0604144ef0 msgr2=0x7f0604147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 --2- 172.21.2.111:0/3742414391 >> v2:172.21.2.115:3300/0 conn(0x7f0604144ef0 0x7f0604147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 >> v1:172.21.2.115:6789/0 conn(0x7f06040add50 legacy=0x7f0604147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060414b5d0 con 0x7f06040603f0 2021-04-03T04:03:17.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f05ec0016d0 con 0x7f06040603f0 2021-04-03T04:03:17.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 >> v1:172.21.2.111:6789/0 conn(0x7f06040603f0 legacy=0x7f06041447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05e4003800 0x7f05e4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f05e4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05e4004e50 0x7f05e400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060414b330 con 0x7f05e40044e0 2021-04-03T04:03:17.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4005300 con 0x7f05e4004e50 2021-04-03T04:03:17.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e400bc20 con 0x7f05e4003800 2021-04-03T04:03:17.151 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0603fff700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f05e4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0603fff700 1 -- 172.21.2.111:0/3742414391 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05e4003800 msgr2=0x7f05e4003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0603fff700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f05e4003800 0x7f05e4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.152 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0603fff700 1 -- 172.21.2.111:0/3742414391 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05e4004e50 msgr2=0x7f05e400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0603fff700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f05e4004e50 0x7f05e400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0603fff700 1 -- 172.21.2.111:0/3742414391 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4003240 con 0x7f05e40044e0 2021-04-03T04:03:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06092e3700 1 -- 172.21.2.111:0/3742414391 reap_dead start 2021-04-03T04:03:17.153 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0603fff700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f05e4004960 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f05f4000f30 tx=0x7f05f4007b50).ready entity=mon.0 client_cookie=1c9f26d036de48c8 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05f400c660 con 0x7f05e40044e0 2021-04-03T04:03:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f400c7c0 con 0x7f05e40044e0 2021-04-03T04:03:17.154 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f06037fe700 1 -- 172.21.2.111:0/3742414391 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05f400cb20 con 0x7f05e40044e0 2021-04-03T04:03:17.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/3742414391 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 msgr2=0x7f05e4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f05e4004960 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f05f4000f30 tx=0x7f05f4007b50).stop 2021-04-03T04:03:17.155 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/3742414391 shutdown_connections 2021-04-03T04:03:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 --2- 172.21.2.111:0/3742414391 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f05e4004960 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/3742414391 >> 172.21.2.111:0/3742414391 conn(0x7f06040b61c0 msgr2=0x7f060405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/3742414391 shutdown_connections 2021-04-03T04:03:17.156 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/3742414391 wait complete. 2021-04-03T04:03:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 Processor -- start 2021-04-03T04:03:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- start start 2021-04-03T04:03:17.157 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f06040b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0604156630 0x7f0604156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06041572c0 0x7f0604062a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0604146c30 con 0x7f05e40044e0 2021-04-03T04:03:17.158 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040b2b40 con 0x7f0604156630 2021-04-03T04:03:17.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0604147ff0 con 0x7f06041572c0 2021-04-03T04:03:17.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f06040b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.159 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f06040b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59484/0 (socket says 172.21.2.111:59484) 2021-04-03T04:03:17.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 -- 172.21.2.111:0/783411115 learned_addr learned my addr 172.21.2.111:0/783411115 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 -- 172.21.2.111:0/783411115 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06041572c0 msgr2=0x7f0604062a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.160 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f06041572c0 0x7f0604062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 -- 172.21.2.111:0/783411115 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0604156630 msgr2=0x7f0604156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f0604156630 0x7f0604156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.161 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 -- 172.21.2.111:0/783411115 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040b25e0 con 0x7f05e40044e0 2021-04-03T04:03:17.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0608ae2700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f06040b20f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7f06040b5600 tx=0x7f060414b220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05ec007170 con 0x7f05e40044e0 2021-04-03T04:03:17.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec0072d0 con 0x7f05e40044e0 2021-04-03T04:03:17.162 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060415a9d0 con 0x7f05e40044e0 2021-04-03T04:03:17.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05ec00a9a0 con 0x7f05e40044e0 2021-04-03T04:03:17.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06041479d0 con 0x7f05e40044e0 2021-04-03T04:03:17.163 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.135+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0604146ab0 con 0x7f05e40044e0 2021-04-03T04:03:17.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f05ec01b060 con 0x7f05e40044e0 2021-04-03T04:03:17.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0601ffb700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05e8056fb0 0x7f05e8059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.164 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f05ec049580 con 0x7f05e40044e0 2021-04-03T04:03:17.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0603fff700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05e8056fb0 0x7f05e8059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0601ffb700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f05e805dcd0 0x7f05e80600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.165 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 --> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05e80606d0 con 0x7f05e805dcd0 2021-04-03T04:03:17.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05ec0011f0 con 0x7f05e40044e0 2021-04-03T04:03:17.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f06092e3700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f05e805dcd0 0x7f05e80600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0603fff700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05e8056fb0 0x7f05e8059470 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f05f4000f30 tx=0x7f05f4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.166 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f06092e3700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f05e805dcd0 0x7f05e80600f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.167 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.139+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== osd.3 v2:172.21.2.111:6826/15913 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f05e80606d0 con 0x7f05e805dcd0 2021-04-03T04:03:17.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.147+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 --> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0604146e30 con 0x7f05e805dcd0 2021-04-03T04:03:17.168 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.147+0000 7f0601ffb700 1 -- 172.21.2.111:0/783411115 <== osd.3 v2:172.21.2.111:6826/15913 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f05fc005a00 con 0x7f05e805dcd0 2021-04-03T04:03:17.169 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:17.169 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:17.169 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:17.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.147+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f05e805dcd0 msgr2=0x7f05e80600f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.171 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.147+0000 7f0609ae4700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6826/15913,v1:172.21.2.111:6827/15913] conn(0x7f05e805dcd0 0x7f05e80600f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05e8056fb0 msgr2=0x7f05e8059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f05e8056fb0 0x7f05e8059470 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f05f4000f30 tx=0x7f05f4015000).stop 2021-04-03T04:03:17.172 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 msgr2=0x7f06040b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 --2- 172.21.2.111:0/783411115 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f05e40044e0 0x7f06040b20f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7f06040b5600 tx=0x7f060414b220).stop 2021-04-03T04:03:17.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f06092e3700 1 -- 172.21.2.111:0/783411115 reap_dead start 2021-04-03T04:03:17.173 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 shutdown_connections 2021-04-03T04:03:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 >> 172.21.2.111:0/783411115 conn(0x7f06040b61c0 msgr2=0x7f0604142320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 shutdown_connections 2021-04-03T04:03:17.174 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.151+0000 7f0609ae4700 1 -- 172.21.2.111:0/783411115 wait complete. 2021-04-03T04:03:17.175 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 3 2021-04-03T04:03:17.251 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 Processor -- start 2021-04-03T04:03:17.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- start start 2021-04-03T04:03:17.252 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f818c048ef0 0x7f818c044260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c0419f0 0x7f818c044750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.253 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0d1b50 con 0x7f818c043120 2021-04-03T04:03:17.254 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c043e00 con 0x7f818c048ef0 2021-04-03T04:03:17.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0492d0 con 0x7f818c0419f0 2021-04-03T04:03:17.255 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819259c700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f818c048ef0 0x7f818c044260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f8192d9d700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f818c043120 0x7f818c0d5300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60288/0 (socket says 172.21.2.111:60288) 2021-04-03T04:03:17.256 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f8192d9d700 1 -- 172.21.2.111:0/3387365809 learned_addr learned my addr 172.21.2.111:0/3387365809 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:17.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f8191d9b700 1 --2- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c0419f0 0x7f818c044750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.257 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819259c700 1 -- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c0419f0 msgr2=0x7f818c044750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.258 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819259c700 1 --2- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c0419f0 0x7f818c044750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819159a700 1 -- 172.21.2.111:0/3387365809 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158755430 0 0) 0x7f818c0d1b50 con 0x7f818c043120 2021-04-03T04:03:17.259 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819259c700 1 -- 172.21.2.111:0/3387365809 >> v1:172.21.2.111:6789/0 conn(0x7f818c043120 legacy=0x7f818c0d5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819259c700 1 -- 172.21.2.111:0/3387365809 --> v2:172.21.2.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c042420 con 0x7f818c048ef0 2021-04-03T04:03:17.260 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819259c700 1 --2- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c048ef0 0x7f818c044260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f819406b5d0 tx=0x7f819405ad70).ready entity=mon.1 client_cookie=6e0853f611609ea server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819159a700 1 -- 172.21.2.111:0/3387365809 <== mon.1 v2:172.21.2.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f819405bda0 con 0x7f818c048ef0 2021-04-03T04:03:17.261 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819159a700 1 -- 172.21.2.111:0/3387365809 <== mon.1 v2:172.21.2.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819405df60 con 0x7f818c048ef0 2021-04-03T04:03:17.262 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819159a700 1 -- 172.21.2.111:0/3387365809 <== mon.1 v2:172.21.2.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81940699f0 con 0x7f818c048ef0 2021-04-03T04:03:17.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c048ef0 msgr2=0x7f818c044260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.263 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 --2- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c048ef0 0x7f818c044260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f819406b5d0 tx=0x7f819405ad70).stop 2021-04-03T04:03:17.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- 172.21.2.111:0/3387365809 shutdown_connections 2021-04-03T04:03:17.264 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 --2- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c048ef0 0x7f818c044260 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 --2- 172.21.2.111:0/3387365809 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c0419f0 0x7f818c044750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.265 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- 172.21.2.111:0/3387365809 >> 172.21.2.111:0/3387365809 conn(0x7f818c00f310 msgr2=0x7f818c0484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.266 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- 172.21.2.111:0/3387365809 shutdown_connections 2021-04-03T04:03:17.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.235+0000 7f819359e700 1 -- 172.21.2.111:0/3387365809 wait complete. 2021-04-03T04:03:17.267 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 Processor -- start 2021-04-03T04:03:17.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 -- start start 2021-04-03T04:03:17.268 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c0419f0 0x7f818c03c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c043120 0x7f818c0a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.269 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f818c048ef0 0x7f818c03dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0a3170 con 0x7f818c048ef0 2021-04-03T04:03:17.270 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c03e2f0 con 0x7f818c0419f0 2021-04-03T04:03:17.271 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c03e590 con 0x7f818c043120 2021-04-03T04:03:17.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f818c048ef0 0x7f818c03dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.272 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f818c048ef0 0x7f818c03dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59504/0 (socket says 172.21.2.111:59504) 2021-04-03T04:03:17.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8191d9b700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c043120 0x7f818c0a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.273 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 -- 172.21.2.111:0/2029343138 learned_addr learned my addr 172.21.2.111:0/2029343138 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819259c700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c0419f0 0x7f818c03c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.274 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 -- 172.21.2.111:0/2029343138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c043120 msgr2=0x7f818c0a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.275 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c043120 0x7f818c0a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 -- 172.21.2.111:0/2029343138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c0419f0 msgr2=0x7f818c03c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.276 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c0419f0 0x7f818c03c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 -- 172.21.2.111:0/2029343138 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0e09e0 con 0x7f818c048ef0 2021-04-03T04:03:17.277 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f8192d9d700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f818c048ef0 0x7f818c03dc50 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f8184000e90 tx=0x7f8184007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f81837fe700 1 -- 172.21.2.111:0/2029343138 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8184007320 con 0x7f818c048ef0 2021-04-03T04:03:17.278 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f81837fe700 1 -- 172.21.2.111:0/2029343138 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8184007480 con 0x7f818c048ef0 2021-04-03T04:03:17.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c0d04e0 con 0x7f818c048ef0 2021-04-03T04:03:17.279 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c0d0980 con 0x7f818c048ef0 2021-04-03T04:03:17.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f81837fe700 1 -- 172.21.2.111:0/2029343138 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8184005220 con 0x7f818c048ef0 2021-04-03T04:03:17.280 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c03e140 con 0x7f818c048ef0 2021-04-03T04:03:17.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f81837fe700 1 -- 172.21.2.111:0/2029343138 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f81840055f0 con 0x7f818c048ef0 2021-04-03T04:03:17.281 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f81837fe700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8170056ef0 0x7f81700593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f81837fe700 1 -- 172.21.2.111:0/2029343138 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f818406f970 con 0x7f818c048ef0 2021-04-03T04:03:17.282 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819259c700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8170056ef0 0x7f81700593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.239+0000 7f819259c700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8170056ef0 0x7f81700593b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f819408f000 tx=0x7f8194064030).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.283 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.243+0000 7f81837fe700 1 -- 172.21.2.111:0/2029343138 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f8184044ec0 con 0x7f818c048ef0 2021-04-03T04:03:17.345 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f818c005180 con 0x7f8170056ef0 2021-04-03T04:03:17.346 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f81837fe700 1 -- 172.21.2.111:0/2029343138 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f818c005180 con 0x7f8170056ef0 2021-04-03T04:03:17.348 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8170056ef0 msgr2=0x7f81700593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8170056ef0 0x7f81700593b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f819408f000 tx=0x7f8194064030).stop 2021-04-03T04:03:17.349 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f818c048ef0 msgr2=0x7f818c03dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.350 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f818c048ef0 0x7f818c03dc50 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f8184000e90 tx=0x7f8184007a40).stop 2021-04-03T04:03:17.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 shutdown_connections 2021-04-03T04:03:17.351 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f818c048ef0 0x7f818c03dc50 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f8170056ef0 0x7f81700593b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.352 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f818c0419f0 0x7f818c03c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 --2- 172.21.2.111:0/2029343138 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f818c043120 0x7f818c0a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.353 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 >> 172.21.2.111:0/2029343138 conn(0x7f818c00f310 msgr2=0x7f818c048400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.354 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 shutdown_connections 2021-04-03T04:03:17.355 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.339+0000 7f819359e700 1 -- 172.21.2.111:0/2029343138 wait complete. 2021-04-03T04:03:17.355 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 3 to deep-scrub 2021-04-03T04:03:17.357 INFO:tasks.ceph:Scrubbing osd.4 2021-04-03T04:03:17.359 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:17.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 Processor -- start 2021-04-03T04:03:17.421 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- start start 2021-04-03T04:03:17.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f4b500add50 0x7f4b50147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.424 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b500607d0 con 0x7f4b50144ef0 2021-04-03T04:03:17.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5014b090 con 0x7f4b500603f0 2021-04-03T04:03:17.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5014b330 con 0x7f4b500add50 2021-04-03T04:03:17.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f4b50144ef0 0x7f4b50147c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60304/0 (socket says 172.21.2.111:60304) 2021-04-03T04:03:17.425 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 -- 172.21.2.111:0/4007546831 learned_addr learned my addr 172.21.2.111:0/4007546831 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:17.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374140317 0 0) 0x7f4b500607d0 con 0x7f4b50144ef0 2021-04-03T04:03:17.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b30003240 con 0x7f4b50144ef0 2021-04-03T04:03:17.426 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4ffff700 1 --2- 172.21.2.111:0/4007546831 >> v2:172.21.2.115:3300/0 conn(0x7f4b500add50 0x7f4b50147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4231016451 0 0) 0x7f4b30003240 con 0x7f4b50144ef0 2021-04-03T04:03:17.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 >> v2:172.21.2.115:3300/0 conn(0x7f4b500add50 msgr2=0x7f4b50147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.427 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 --2- 172.21.2.111:0/4007546831 >> v2:172.21.2.115:3300/0 conn(0x7f4b500add50 0x7f4b50147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 >> v1:172.21.2.115:6789/0 conn(0x7f4b500603f0 legacy=0x7f4b501447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5014b5d0 con 0x7f4b50144ef0 2021-04-03T04:03:17.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f4b48001a50 con 0x7f4b50144ef0 2021-04-03T04:03:17.428 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 >> v1:172.21.2.111:6789/0 conn(0x7f4b50144ef0 legacy=0x7f4b50147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b30003800 0x7f4b30003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b300044e0 0x7f4b30004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.429 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b30004e50 0x7f4b3000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5014b330 con 0x7f4b30004e50 2021-04-03T04:03:17.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b30005300 con 0x7f4b30003800 2021-04-03T04:03:17.430 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3000bc20 con 0x7f4b300044e0 2021-04-03T04:03:17.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b30004e50 0x7f4b3000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 -- 172.21.2.111:0/4007546831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b300044e0 msgr2=0x7f4b30004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.431 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b300044e0 0x7f4b30004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 -- 172.21.2.111:0/4007546831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b30003800 msgr2=0x7f4b30003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b30003800 0x7f4b30003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 -- 172.21.2.111:0/4007546831 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b30003240 con 0x7f4b30004e50 2021-04-03T04:03:17.432 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 -- 172.21.2.111:0/4007546831 reap_dead start 2021-04-03T04:03:17.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55799700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b30004e50 0x7f4b3000b730 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f4b500b5600 tx=0x7f4b5014b220).ready entity=mon.0 client_cookie=50c4ebdf9da0e68d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b48005c50 con 0x7f4b30004e50 2021-04-03T04:03:17.433 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b48005db0 con 0x7f4b30004e50 2021-04-03T04:03:17.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b4f7fe700 1 -- 172.21.2.111:0/4007546831 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b480072c0 con 0x7f4b30004e50 2021-04-03T04:03:17.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- 172.21.2.111:0/4007546831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b30004e50 msgr2=0x7f4b3000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.434 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b30004e50 0x7f4b3000b730 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f4b500b5600 tx=0x7f4b5014b220).stop 2021-04-03T04:03:17.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- 172.21.2.111:0/4007546831 shutdown_connections 2021-04-03T04:03:17.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 --2- 172.21.2.111:0/4007546831 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b30004e50 0x7f4b3000b730 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.435 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- 172.21.2.111:0/4007546831 >> 172.21.2.111:0/4007546831 conn(0x7f4b500b61c0 msgr2=0x7f4b5005f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- 172.21.2.111:0/4007546831 shutdown_connections 2021-04-03T04:03:17.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.415+0000 7f4b55f9a700 1 -- 172.21.2.111:0/4007546831 wait complete. 2021-04-03T04:03:17.436 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 Processor -- start 2021-04-03T04:03:17.437 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 -- start start 2021-04-03T04:03:17.439 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b30004e50 0x7f4b500a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b50067d40 0x7f4b500ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b501565b0 0x7f4b50156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.440 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50147ab0 con 0x7f4b501565b0 2021-04-03T04:03:17.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500aa330 con 0x7f4b30004e50 2021-04-03T04:03:17.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50147d10 con 0x7f4b50067d40 2021-04-03T04:03:17.441 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b501565b0 0x7f4b50156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b501565b0 0x7f4b50156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59520/0 (socket says 172.21.2.111:59520) 2021-04-03T04:03:17.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 -- 172.21.2.111:0/247732736 learned_addr learned my addr 172.21.2.111:0/247732736 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.442 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4ffff700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b50067d40 0x7f4b500ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b54f98700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b30004e50 0x7f4b500a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 -- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b50067d40 msgr2=0x7f4b500ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4b50067d40 0x7f4b500ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.443 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 -- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b30004e50 msgr2=0x7f4b500a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4b30004e50 0x7f4b500a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 -- 172.21.2.111:0/247732736 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b501485d0 con 0x7f4b501565b0 2021-04-03T04:03:17.444 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55799700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b501565b0 0x7f4b50156a40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 rx=0x7f4b500b5600 tx=0x7f4b48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b48013640 con 0x7f4b501565b0 2021-04-03T04:03:17.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b480137a0 con 0x7f4b501565b0 2021-04-03T04:03:17.445 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b48013b00 con 0x7f4b501565b0 2021-04-03T04:03:17.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b500aa020 con 0x7f4b501565b0 2021-04-03T04:03:17.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b50147840 con 0x7f4b501565b0 2021-04-03T04:03:17.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b50147f10 con 0x7f4b501565b0 2021-04-03T04:03:17.446 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4b4801d070 con 0x7f4b501565b0 2021-04-03T04:03:17.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b34056ea0 0x7f4b34059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f4b48071190 con 0x7f4b501565b0 2021-04-03T04:03:17.447 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f4b3405dc60 0x7f4b34060080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 --> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b34060700 con 0x7f4b3405dc60 2021-04-03T04:03:17.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b48073800 con 0x7f4b501565b0 2021-04-03T04:03:17.448 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b54f98700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b34056ea0 0x7f4b34059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b54f98700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b34056ea0 0x7f4b34059360 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f4b4000a000 tx=0x7f4b40008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.419+0000 7f4b4ffff700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f4b3405dc60 0x7f4b34060080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.423+0000 7f4b4ffff700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f4b3405dc60 0x7f4b34060080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.449 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.423+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== osd.4 v2:172.21.2.115:6800/16012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f4b34060700 con 0x7f4b3405dc60 2021-04-03T04:03:17.450 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 --> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4b50057660 con 0x7f4b3405dc60 2021-04-03T04:03:17.451 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b4dffb700 1 -- 172.21.2.111:0/247732736 <== osd.4 v2:172.21.2.115:6800/16012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4b50057660 con 0x7f4b3405dc60 2021-04-03T04:03:17.451 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:17.451 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:17.452 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:17.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f4b3405dc60 msgr2=0x7f4b34060080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.115:6800/16012,v1:172.21.2.115:6801/16012] conn(0x7f4b3405dc60 0x7f4b34060080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.453 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b34056ea0 msgr2=0x7f4b34059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4b34056ea0 0x7f4b34059360 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f4b4000a000 tx=0x7f4b40008040).stop 2021-04-03T04:03:17.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b501565b0 msgr2=0x7f4b50156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.454 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 --2- 172.21.2.111:0/247732736 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4b501565b0 0x7f4b50156a40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 rx=0x7f4b500b5600 tx=0x7f4b48015000).stop 2021-04-03T04:03:17.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55799700 1 -- 172.21.2.111:0/247732736 reap_dead start 2021-04-03T04:03:17.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 shutdown_connections 2021-04-03T04:03:17.455 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 >> 172.21.2.111:0/247732736 conn(0x7f4b500b61c0 msgr2=0x7f4b50143940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 shutdown_connections 2021-04-03T04:03:17.456 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.435+0000 7f4b55f9a700 1 -- 172.21.2.111:0/247732736 wait complete. 2021-04-03T04:03:17.469 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 4 2021-04-03T04:03:17.530 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a82f2e700 1 Processor -- start 2021-04-03T04:03:17.531 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a82f2e700 1 -- start start 2021-04-03T04:03:17.536 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a82f2e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74005640 0x7f1a740d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a82f2e700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f1a740d8e90 0x7f1a740e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a82f2e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7403d840 con 0x7f1a7403f120 2021-04-03T04:03:17.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a82f2e700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a7403faa0 con 0x7f1a740d8e90 2021-04-03T04:03:17.537 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a82f2e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7403dca0 con 0x7f1a74005640 2021-04-03T04:03:17.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a81f2c700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f1a7403f120 0x7f1a74041fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60320/0 (socket says 172.21.2.111:60320) 2021-04-03T04:03:17.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a81f2c700 1 -- 172.21.2.111:0/2307503790 learned_addr learned my addr 172.21.2.111:0/2307503790 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:17.538 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481031782 0 0) 0x7f1a7403d840 con 0x7f1a7403f120 2021-04-03T04:03:17.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a64003240 con 0x7f1a7403f120 2021-04-03T04:03:17.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2858903137 0 0) 0x7f1a64003240 con 0x7f1a7403f120 2021-04-03T04:03:17.539 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74005640 msgr2=0x7f1a740d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74005640 0x7f1a740d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 >> v2:172.21.2.115:3300/0 conn(0x7f1a740d8e90 msgr2=0x7f1a740e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.540 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 --2- 172.21.2.111:0/2307503790 >> v2:172.21.2.115:3300/0 conn(0x7f1a740d8e90 0x7f1a740e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7403f800 con 0x7f1a7403f120 2021-04-03T04:03:17.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f1a700016d0 con 0x7f1a7403f120 2021-04-03T04:03:17.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.523+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 >> v1:172.21.2.111:6789/0 conn(0x7f1a7403f120 legacy=0x7f1a74041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.541 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74005640 0x7f1a640037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a64003d10 0x7f1a640044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a640049d0 0x7f1a64004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.542 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7403faa0 con 0x7f1a64003d10 2021-04-03T04:03:17.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7403dca0 con 0x7f1a640049d0 2021-04-03T04:03:17.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a640042d0 con 0x7f1a74005640 2021-04-03T04:03:17.543 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a64003d10 0x7f1a640044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74005640 0x7f1a640037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 -- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74005640 msgr2=0x7f1a640037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a74005640 0x7f1a640037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.544 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 -- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a640049d0 msgr2=0x7f1a64004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8272d700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a640049d0 0x7f1a64004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a640049d0 0x7f1a64004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.545 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 -- 172.21.2.111:0/2307503790 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64003240 con 0x7f1a64003d10 2021-04-03T04:03:17.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8272d700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a640049d0 0x7f1a64004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:17.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a64003d10 0x7f1a640044e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f1a78002f30 tx=0x7f1a78006e10).ready entity=mon.0 client_cookie=3e9c43078f793bd6 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.546 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a78009790 con 0x7f1a64003d10 2021-04-03T04:03:17.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a78000e90 con 0x7f1a64003d10 2021-04-03T04:03:17.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a80f2a700 1 -- 172.21.2.111:0/2307503790 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a78009ae0 con 0x7f1a64003d10 2021-04-03T04:03:17.547 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2307503790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a64003d10 msgr2=0x7f1a640044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 --2- 172.21.2.111:0/2307503790 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a64003d10 0x7f1a640044e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f1a78002f30 tx=0x7f1a78006e10).stop 2021-04-03T04:03:17.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8272d700 1 -- 172.21.2.111:0/2307503790 reap_dead start 2021-04-03T04:03:17.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2307503790 shutdown_connections 2021-04-03T04:03:17.548 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2307503790 >> 172.21.2.111:0/2307503790 conn(0x7f1a740462b0 msgr2=0x7f1a740466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2307503790 shutdown_connections 2021-04-03T04:03:17.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2307503790 wait complete. 2021-04-03T04:03:17.549 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 Processor -- start 2021-04-03T04:03:17.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- start start 2021-04-03T04:03:17.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a740a0a40 0x7f1a740a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.550 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a740da3a0 0x7f1a740e7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a740e7c50 0x7f1a740e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a740e7370 con 0x7f1a740a0a40 2021-04-03T04:03:17.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a740a1a40 con 0x7f1a740da3a0 2021-04-03T04:03:17.551 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a740e85c0 con 0x7f1a740e7c50 2021-04-03T04:03:17.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a740a0a40 0x7f1a740a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a740a0a40 0x7f1a740a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59542/0 (socket says 172.21.2.111:59542) 2021-04-03T04:03:17.552 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 -- 172.21.2.111:0/2572684658 learned_addr learned my addr 172.21.2.111:0/2572684658 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8172b700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a740da3a0 0x7f1a740e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a8272d700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a740e7c50 0x7f1a740e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.553 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 -- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a740e7c50 msgr2=0x7f1a740e80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a740e7c50 0x7f1a740e80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 -- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a740da3a0 msgr2=0x7f1a740e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a740da3a0 0x7f1a740e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.554 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 -- 172.21.2.111:0/2572684658 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a740a0e20 con 0x7f1a740a0a40 2021-04-03T04:03:17.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a81f2c700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a740a0a40 0x7f1a740a0070 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f1a740da240 tx=0x7f1a70009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a6effd700 1 -- 172.21.2.111:0/2572684658 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a70007170 con 0x7f1a740a0a40 2021-04-03T04:03:17.555 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a6effd700 1 -- 172.21.2.111:0/2572684658 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a700072d0 con 0x7f1a740a0a40 2021-04-03T04:03:17.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a740dad50 con 0x7f1a740a0a40 2021-04-03T04:03:17.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a6effd700 1 -- 172.21.2.111:0/2572684658 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a7000ba00 con 0x7f1a740a0a40 2021-04-03T04:03:17.556 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a740ea940 con 0x7f1a740a0a40 2021-04-03T04:03:17.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a740daba0 con 0x7f1a740a0a40 2021-04-03T04:03:17.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.527+0000 7f1a6effd700 1 -- 172.21.2.111:0/2572684658 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f1a7001a020 con 0x7f1a740a0a40 2021-04-03T04:03:17.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.531+0000 7f1a6effd700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a5c056f60 0x7f1a5c059420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.557 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.531+0000 7f1a6effd700 1 -- 172.21.2.111:0/2572684658 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f1a7006ff00 con 0x7f1a740a0a40 2021-04-03T04:03:17.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.531+0000 7f1a8172b700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a5c056f60 0x7f1a5c059420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.558 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.531+0000 7f1a8172b700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a5c056f60 0x7f1a5c059420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f1a78016000 tx=0x7f1a78015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.559 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.531+0000 7f1a6effd700 1 -- 172.21.2.111:0/2572684658 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f1a700455c0 con 0x7f1a740a0a40 2021-04-03T04:03:17.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a740e74c0 con 0x7f1a5c056f60 2021-04-03T04:03:17.632 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a6effd700 1 -- 172.21.2.111:0/2572684658 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1a7800b5a0 con 0x7f1a5c056f60 2021-04-03T04:03:17.633 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a5c056f60 msgr2=0x7f1a5c059420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a5c056f60 0x7f1a5c059420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f1a78016000 tx=0x7f1a78015000).stop 2021-04-03T04:03:17.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a740a0a40 msgr2=0x7f1a740a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.634 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a740a0a40 0x7f1a740a0070 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f1a740da240 tx=0x7f1a70009040).stop 2021-04-03T04:03:17.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 shutdown_connections 2021-04-03T04:03:17.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f1a740e7c50 0x7f1a740e80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.635 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f1a740a0a40 0x7f1a740a0070 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f1a5c056f60 0x7f1a5c059420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 --2- 172.21.2.111:0/2572684658 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f1a740da3a0 0x7f1a740e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 >> 172.21.2.111:0/2572684658 conn(0x7f1a740462b0 msgr2=0x7f1a74042c50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.636 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 shutdown_connections 2021-04-03T04:03:17.637 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.627+0000 7f1a82f2e700 1 -- 172.21.2.111:0/2572684658 wait complete. 2021-04-03T04:03:17.637 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 4 to deep-scrub 2021-04-03T04:03:17.640 INFO:tasks.ceph:Scrubbing osd.5 2021-04-03T04:03:17.641 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:17.699 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.691+0000 7f56a24ff700 1 Processor -- start 2021-04-03T04:03:17.700 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.691+0000 7f56a24ff700 1 -- start start 2021-04-03T04:03:17.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f569c0603f0 0x7f569c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.701 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f569c0add50 0x7f569c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f569c0b17b0 0x7f569c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c1451c0 con 0x7f569c0add50 2021-04-03T04:03:17.702 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c0607d0 con 0x7f569c0b17b0 2021-04-03T04:03:17.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c14b090 con 0x7f569c0603f0 2021-04-03T04:03:17.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f569c0add50 0x7f569c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.703 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f569c0add50 0x7f569c144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59550/0 (socket says 172.21.2.111:59550) 2021-04-03T04:03:17.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 -- 172.21.2.111:0/1119457038 learned_addr learned my addr 172.21.2.111:0/1119457038 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 -- 172.21.2.111:0/1119457038 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f569c0603f0 msgr2=0x7f569c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.704 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 --2- 172.21.2.111:0/1119457038 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f569c0603f0 0x7f569c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 -- 172.21.2.111:0/1119457038 >> v2:172.21.2.115:3300/0 conn(0x7f569c0b17b0 msgr2=0x7f569c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 --2- 172.21.2.111:0/1119457038 >> v2:172.21.2.115:3300/0 conn(0x7f569c0b17b0 0x7f569c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.705 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 -- 172.21.2.111:0/1119457038 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c14b330 con 0x7f569c0add50 2021-04-03T04:03:17.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a0cfc700 1 --2- 172.21.2.111:0/1119457038 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0add50 0x7f569c144cd0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f5698002f30 tx=0x7f5698006e90).ready entity=mon.0 client_cookie=fab7b89bfc970467 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f568bfff700 1 -- 172.21.2.111:0/1119457038 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5698009920 con 0x7f569c0add50 2021-04-03T04:03:17.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f568bfff700 1 -- 172.21.2.111:0/1119457038 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5698009a80 con 0x7f569c0add50 2021-04-03T04:03:17.706 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f568bfff700 1 -- 172.21.2.111:0/1119457038 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5698009de0 con 0x7f569c0add50 2021-04-03T04:03:17.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- 172.21.2.111:0/1119457038 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0add50 msgr2=0x7f569c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.707 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- 172.21.2.111:0/1119457038 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0add50 0x7f569c144cd0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f5698002f30 tx=0x7f5698006e90).stop 2021-04-03T04:03:17.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- 172.21.2.111:0/1119457038 shutdown_connections 2021-04-03T04:03:17.710 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- 172.21.2.111:0/1119457038 >> v2:172.21.2.115:3300/0 conn(0x7f569c0b17b0 0x7f569c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- 172.21.2.111:0/1119457038 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f569c0603f0 0x7f569c1447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- 172.21.2.111:0/1119457038 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0add50 0x7f569c144cd0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.711 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- 172.21.2.111:0/1119457038 >> 172.21.2.111:0/1119457038 conn(0x7f569c0b61c0 msgr2=0x7f569c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- 172.21.2.111:0/1119457038 shutdown_connections 2021-04-03T04:03:17.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- 172.21.2.111:0/1119457038 wait complete. 2021-04-03T04:03:17.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 Processor -- start 2021-04-03T04:03:17.712 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- start start 2021-04-03T04:03:17.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0603f0 0x7f569c0ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f569c0add50 0x7f569c156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.713 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f569c0b17b0 0x7f569c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1546d0 con 0x7f569c0603f0 2021-04-03T04:03:17.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0aad00 con 0x7f569c0add50 2021-04-03T04:03:17.714 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c064ce0 con 0x7f569c0b17b0 2021-04-03T04:03:17.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0603f0 0x7f569c0ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0603f0 0x7f569c0ab780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59554/0 (socket says 172.21.2.111:59554) 2021-04-03T04:03:17.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 -- 172.21.2.111:0/2798199208 learned_addr learned my addr 172.21.2.111:0/2798199208 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.715 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 -- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f569c0b17b0 msgr2=0x7f569c065180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f569c0b17b0 0x7f569c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 -- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f569c0add50 msgr2=0x7f569c156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.716 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f569c0add50 0x7f569c156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 -- 172.21.2.111:0/2798199208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0a9d10 con 0x7f569c0603f0 2021-04-03T04:03:17.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a14fd700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0603f0 0x7f569c0ab780 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f568c002f30 tx=0x7f568c006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.717 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f568c00d840 con 0x7f569c0603f0 2021-04-03T04:03:17.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c000e90 con 0x7f569c0603f0 2021-04-03T04:03:17.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c0abc70 con 0x7f569c0603f0 2021-04-03T04:03:17.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f568c00db90 con 0x7f569c0603f0 2021-04-03T04:03:17.718 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.695+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c157310 con 0x7f569c0603f0 2021-04-03T04:03:17.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f569c057660 con 0x7f569c0603f0 2021-04-03T04:03:17.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f568c021070 con 0x7f569c0603f0 2021-04-03T04:03:17.719 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f568a7fc700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f567c056ea0 0x7f567c059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f568c0701d0 con 0x7f569c0603f0 2021-04-03T04:03:17.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f56a0cfc700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f567c056ea0 0x7f567c059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.720 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f568a7fc700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f567c05dce0 0x7f567c060100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 --> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f567c060780 con 0x7f567c05dce0 2021-04-03T04:03:17.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f568c0704e0 con 0x7f569c0603f0 2021-04-03T04:03:17.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f56a0cfc700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f567c056ea0 0x7f567c059360 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f5698002f30 tx=0x7f5698015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.721 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f56a1cfe700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f567c05dce0 0x7f567c060100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.699+0000 7f56a1cfe700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f567c05dce0 0x7f567c060100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.722 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.703+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== osd.5 v2:172.21.2.115:6816/16014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f567c060780 con 0x7f567c05dce0 2021-04-03T04:03:17.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 --> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f569c157310 con 0x7f567c05dce0 2021-04-03T04:03:17.723 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f568a7fc700 1 -- 172.21.2.111:0/2798199208 <== osd.5 v2:172.21.2.115:6816/16014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f569c157310 con 0x7f567c05dce0 2021-04-03T04:03:17.724 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:17.724 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:17.725 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:17.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f567c05dce0 msgr2=0x7f567c060100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a24ff700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.115:6816/16014,v1:172.21.2.115:6817/16014] conn(0x7f567c05dce0 0x7f567c060100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.726 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f567c056ea0 msgr2=0x7f567c059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a24ff700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f567c056ea0 0x7f567c059360 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f5698002f30 tx=0x7f5698015000).stop 2021-04-03T04:03:17.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0603f0 msgr2=0x7f569c0ab780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.727 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a24ff700 1 --2- 172.21.2.111:0/2798199208 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f569c0603f0 0x7f569c0ab780 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f568c002f30 tx=0x7f568c006ea0).stop 2021-04-03T04:03:17.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.711+0000 7f56a1cfe700 1 -- 172.21.2.111:0/2798199208 reap_dead start 2021-04-03T04:03:17.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.715+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 shutdown_connections 2021-04-03T04:03:17.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.715+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 >> 172.21.2.111:0/2798199208 conn(0x7f569c0b61c0 msgr2=0x7f569c14a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.728 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.715+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 shutdown_connections 2021-04-03T04:03:17.729 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.715+0000 7f56a24ff700 1 -- 172.21.2.111:0/2798199208 wait complete. 2021-04-03T04:03:17.730 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 5 2021-04-03T04:03:17.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe19a322700 1 Processor -- start 2021-04-03T04:03:17.786 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe19a322700 1 -- start start 2021-04-03T04:03:17.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe19a322700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c03ddf0 0x7fe18c042830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.792 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe19a322700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c0d4780 con 0x7fe18c0d42e0 2021-04-03T04:03:17.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe19a322700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c047950 con 0x7fe18c047570 2021-04-03T04:03:17.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe19a322700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0d0490 con 0x7fe18c03ddf0 2021-04-03T04:03:17.793 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe198b1f700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fe18c0d42e0 0x7fe18c042120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60360/0 (socket says 172.21.2.111:60360) 2021-04-03T04:03:17.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe198b1f700 1 -- 172.21.2.111:0/93079799 learned_addr learned my addr 172.21.2.111:0/93079799 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:17.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034570997 0 0) 0x7fe18c0d4780 con 0x7fe18c0d42e0 2021-04-03T04:03:17.794 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c003240 con 0x7fe18c0d42e0 2021-04-03T04:03:17.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4148434727 0 0) 0x7fe17c003240 con 0x7fe18c0d42e0 2021-04-03T04:03:17.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c03ddf0 msgr2=0x7fe18c042830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.795 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe187fff700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c03ddf0 0x7fe18c042830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.779+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 >> v1:172.21.2.115:6789/0 conn(0x7fe18c047570 legacy=0x7fe18c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c0d0730 con 0x7fe18c0d42e0 2021-04-03T04:03:17.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7fe1880016d0 con 0x7fe18c0d42e0 2021-04-03T04:03:17.796 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 >> v1:172.21.2.111:6789/0 conn(0x7fe18c0d42e0 legacy=0x7fe18c042120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe17c0037e0 0x7fe17c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c03ddf0 0x7fe17c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.797 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe17c0049d0 0x7fe17c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0d0490 con 0x7fe17c0037e0 2021-04-03T04:03:17.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0042b0 con 0x7fe17c0049d0 2021-04-03T04:03:17.798 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c00b730 con 0x7fe18c03ddf0 2021-04-03T04:03:17.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe17c0037e0 0x7fe17c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 -- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c03ddf0 msgr2=0x7fe17c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.799 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c03ddf0 0x7fe17c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 -- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe17c0049d0 msgr2=0x7fe17c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe17c0049d0 0x7fe17c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.800 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 -- 172.21.2.111:0/93079799 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c003240 con 0x7fe17c0037e0 2021-04-03T04:03:17.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe17c0037e0 0x7fe17c003be0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7fe190001f60 tx=0x7fe190007b70).ready entity=mon.0 client_cookie=9539e67c43b0ba1a server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe19000c870 con 0x7fe17c0037e0 2021-04-03T04:03:17.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19000c9d0 con 0x7fe17c0037e0 2021-04-03T04:03:17.801 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe187fff700 1 -- 172.21.2.111:0/93079799 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe19000cd30 con 0x7fe17c0037e0 2021-04-03T04:03:17.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/93079799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe17c0037e0 msgr2=0x7fe17c003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 --2- 172.21.2.111:0/93079799 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe17c0037e0 0x7fe17c003be0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7fe190001f60 tx=0x7fe190007b70).stop 2021-04-03T04:03:17.802 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 -- 172.21.2.111:0/93079799 reap_dead start 2021-04-03T04:03:17.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/93079799 shutdown_connections 2021-04-03T04:03:17.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/93079799 >> 172.21.2.111:0/93079799 conn(0x7fe18c00d990 msgr2=0x7fe18c046b40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.803 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/93079799 shutdown_connections 2021-04-03T04:03:17.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/93079799 wait complete. 2021-04-03T04:03:17.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 Processor -- start 2021-04-03T04:03:17.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- start start 2021-04-03T04:03:17.804 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c0469e0 0x7fe18c0e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe18c0e79e0 0x7fe18c039e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe18c03a300 0x7fe18c0ea650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.805 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c045ce0 con 0x7fe18c03a300 2021-04-03T04:03:17.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c046dc0 con 0x7fe18c0e79e0 2021-04-03T04:03:17.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c045f10 con 0x7fe18c0469e0 2021-04-03T04:03:17.806 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe18c03a300 0x7fe18c0ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe18c03a300 0x7fe18c0ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59580/0 (socket says 172.21.2.111:59580) 2021-04-03T04:03:17.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 -- 172.21.2.111:0/3726350828 learned_addr learned my addr 172.21.2.111:0/3726350828 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.807 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe198b1f700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe18c0e79e0 0x7fe18c039e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199320700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c0469e0 0x7fe18c0e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 -- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c0469e0 msgr2=0x7fe18c0e74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c0469e0 0x7fe18c0e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.808 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 -- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe18c0e79e0 msgr2=0x7fe18c039e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe18c0e79e0 0x7fe18c039e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 -- 172.21.2.111:0/3726350828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c03c9c0 con 0x7fe18c03a300 2021-04-03T04:03:17.809 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe199b21700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe18c03a300 0x7fe18c0ea650 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7fe1880018b0 tx=0x7fe19405ad70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe1867fc700 1 -- 172.21.2.111:0/3726350828 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1940699f0 con 0x7fe18c03a300 2021-04-03T04:03:17.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe1867fc700 1 -- 172.21.2.111:0/3726350828 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19405b040 con 0x7fe18c03a300 2021-04-03T04:03:17.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c0458e0 con 0x7fe18c03a300 2021-04-03T04:03:17.810 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe1867fc700 1 -- 172.21.2.111:0/3726350828 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe194061070 con 0x7fe18c03a300 2021-04-03T04:03:17.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c044d90 con 0x7fe18c03a300 2021-04-03T04:03:17.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c03cd80 con 0x7fe18c03a300 2021-04-03T04:03:17.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe1867fc700 1 -- 172.21.2.111:0/3726350828 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fe1940611d0 con 0x7fe18c03a300 2021-04-03T04:03:17.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe1867fc700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe174056ea0 0x7fe174059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.783+0000 7fe1867fc700 1 -- 172.21.2.111:0/3726350828 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fe1940dece0 con 0x7fe18c03a300 2021-04-03T04:03:17.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.787+0000 7fe199320700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe174056ea0 0x7fe174059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.787+0000 7fe199320700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe174056ea0 0x7fe174059360 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe190008000 tx=0x7fe190016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.787+0000 7fe1867fc700 1 -- 172.21.2.111:0/3726350828 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fe18c03cd80 con 0x7fe18c03a300 2021-04-03T04:03:17.887 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.879+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe18c03cd80 con 0x7fe174056ea0 2021-04-03T04:03:17.888 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.879+0000 7fe1867fc700 1 -- 172.21.2.111:0/3726350828 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe18c03cd80 con 0x7fe174056ea0 2021-04-03T04:03:17.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe174056ea0 msgr2=0x7fe174059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.889 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe174056ea0 0x7fe174059360 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe190008000 tx=0x7fe190016000).stop 2021-04-03T04:03:17.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe18c03a300 msgr2=0x7fe18c0ea650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe18c03a300 0x7fe18c0ea650 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7fe1880018b0 tx=0x7fe19405ad70).stop 2021-04-03T04:03:17.890 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 shutdown_connections 2021-04-03T04:03:17.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fe18c03a300 0x7fe18c0ea650 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fe18c0469e0 0x7fe18c0e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.891 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fe174056ea0 0x7fe174059360 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 --2- 172.21.2.111:0/3726350828 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fe18c0e79e0 0x7fe18c039e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 >> 172.21.2.111:0/3726350828 conn(0x7fe18c00d990 msgr2=0x7fe18c0c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 shutdown_connections 2021-04-03T04:03:17.892 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.883+0000 7fe19a322700 1 -- 172.21.2.111:0/3726350828 wait complete. 2021-04-03T04:03:17.893 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 5 to deep-scrub 2021-04-03T04:03:17.896 INFO:tasks.ceph:Scrubbing osd.6 2021-04-03T04:03:17.896 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:17.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.947+0000 7fc361995700 1 Processor -- start 2021-04-03T04:03:17.955 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.947+0000 7fc361995700 1 -- start start 2021-04-03T04:03:17.957 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc35c0add50 0x7fc35c147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fc35c144ef0 0x7fc35c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c0607d0 con 0x7fc35c0603f0 2021-04-03T04:03:17.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c14b090 con 0x7fc35c144ef0 2021-04-03T04:03:17.958 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c14b330 con 0x7fc35c0add50 2021-04-03T04:03:17.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc360993700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7fc35c0603f0 0x7fc35c1447e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60376/0 (socket says 172.21.2.111:60376) 2021-04-03T04:03:17.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc360993700 1 -- 172.21.2.111:0/1325622909 learned_addr learned my addr 172.21.2.111:0/1325622909 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:17.959 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361194700 1 --2- 172.21.2.111:0/1325622909 >> v2:172.21.2.111:3300/0 conn(0x7fc35c144ef0 0x7fc35c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35b7fe700 1 -- 172.21.2.111:0/1325622909 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732664229 0 0) 0x7fc35c0607d0 con 0x7fc35c0603f0 2021-04-03T04:03:17.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35b7fe700 1 -- 172.21.2.111:0/1325622909 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc33c003240 con 0x7fc35c0603f0 2021-04-03T04:03:17.960 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361194700 1 -- 172.21.2.111:0/1325622909 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc35c0add50 msgr2=0x7fc35c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:17.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361194700 1 --2- 172.21.2.111:0/1325622909 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc35c0add50 0x7fc35c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361194700 1 -- 172.21.2.111:0/1325622909 >> v1:172.21.2.111:6789/0 conn(0x7fc35c0603f0 legacy=0x7fc35c1447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.961 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361194700 1 -- 172.21.2.111:0/1325622909 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c14b5d0 con 0x7fc35c144ef0 2021-04-03T04:03:17.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361194700 1 --2- 172.21.2.111:0/1325622909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c144ef0 0x7fc35c147c30 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7fc3540032b0 tx=0x7fc3540091d0).ready entity=mon.0 client_cookie=1003d47c96d77391 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35b7fe700 1 -- 172.21.2.111:0/1325622909 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc35400b9a0 con 0x7fc35c144ef0 2021-04-03T04:03:17.962 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35b7fe700 1 -- 172.21.2.111:0/1325622909 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35400bb00 con 0x7fc35c144ef0 2021-04-03T04:03:17.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35b7fe700 1 -- 172.21.2.111:0/1325622909 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc35400bc70 con 0x7fc35c144ef0 2021-04-03T04:03:17.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- 172.21.2.111:0/1325622909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c144ef0 msgr2=0x7fc35c147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.963 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- 172.21.2.111:0/1325622909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c144ef0 0x7fc35c147c30 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7fc3540032b0 tx=0x7fc3540091d0).stop 2021-04-03T04:03:17.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- 172.21.2.111:0/1325622909 shutdown_connections 2021-04-03T04:03:17.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- 172.21.2.111:0/1325622909 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c144ef0 0x7fc35c147c30 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.964 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- 172.21.2.111:0/1325622909 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc35c0add50 0x7fc35c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- 172.21.2.111:0/1325622909 >> 172.21.2.111:0/1325622909 conn(0x7fc35c0b61c0 msgr2=0x7fc35c05f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- 172.21.2.111:0/1325622909 shutdown_connections 2021-04-03T04:03:17.965 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- 172.21.2.111:0/1325622909 wait complete. 2021-04-03T04:03:17.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 Processor -- start 2021-04-03T04:03:17.968 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- start start 2021-04-03T04:03:17.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc35c0603f0 0x7fc35c0a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c0add50 0x7fc35c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.969 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc35c144ef0 0x7fc35c0ab510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0a76a0 con 0x7fc35c0add50 2021-04-03T04:03:17.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0ab0e0 con 0x7fc35c144ef0 2021-04-03T04:03:17.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c15a950 con 0x7fc35c0603f0 2021-04-03T04:03:17.970 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c0add50 0x7fc35c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c0add50 0x7fc35c15a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59592/0 (socket says 172.21.2.111:59592) 2021-04-03T04:03:17.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 -- 172.21.2.111:0/1043753520 learned_addr learned my addr 172.21.2.111:0/1043753520 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:17.971 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 -- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc35c0603f0 msgr2=0x7fc35c0a7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fc35c0603f0 0x7fc35c0a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 -- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc35c144ef0 msgr2=0x7fc35c0ab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.972 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fc35c144ef0 0x7fc35c0ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:17.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 -- 172.21.2.111:0/1043753520 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c15abf0 con 0x7fc35c0add50 2021-04-03T04:03:17.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc35bfff700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c0add50 0x7fc35c15a0e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7fc35c0a77e0 tx=0x7fc34c006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34c00d710 con 0x7fc35c0add50 2021-04-03T04:03:17.973 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c000e90 con 0x7fc35c0add50 2021-04-03T04:03:17.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34c00da60 con 0x7fc35c0add50 2021-04-03T04:03:17.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c0a7f40 con 0x7fc35c0add50 2021-04-03T04:03:17.974 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.951+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c0a6f20 con 0x7fc35c0add50 2021-04-03T04:03:17.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc35c15a7e0 con 0x7fc35c0add50 2021-04-03T04:03:17.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fc34c018020 con 0x7fc35c0add50 2021-04-03T04:03:17.975 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc359ffb700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc340056fb0 0x7fc340059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fc34c0706e0 con 0x7fc35c0add50 2021-04-03T04:03:17.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc360993700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc340056fb0 0x7fc340059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.976 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc359ffb700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7fc34005dcd0 0x7fc3400600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:17.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 --> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3400606d0 con 0x7fc34005dcd0 2021-04-03T04:03:17.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc34c070aa0 con 0x7fc35c0add50 2021-04-03T04:03:17.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc360993700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc340056fb0 0x7fc340059470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fc3440010d0 tx=0x7fc3440076a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.977 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc361194700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7fc34005dcd0 0x7fc3400600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:17.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.955+0000 7fc361194700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7fc34005dcd0 0x7fc3400600f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:17.978 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.959+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== osd.6 v2:172.21.2.115:6808/16015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fc3400606d0 con 0x7fc34005dcd0 2021-04-03T04:03:17.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.967+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 --> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc35c0ab220 con 0x7fc34005dcd0 2021-04-03T04:03:17.979 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.967+0000 7fc359ffb700 1 -- 172.21.2.111:0/1043753520 <== osd.6 v2:172.21.2.115:6808/16015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc35c0ab220 con 0x7fc34005dcd0 2021-04-03T04:03:17.980 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:17.980 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:17.980 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:17.981 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.967+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7fc34005dcd0 msgr2=0x7fc3400600f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.967+0000 7fc361995700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.115:6808/16015,v1:172.21.2.115:6809/16015] conn(0x7fc34005dcd0 0x7fc3400600f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:17.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc340056fb0 msgr2=0x7fc340059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fc340056fb0 0x7fc340059470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fc3440010d0 tx=0x7fc3440076a0).stop 2021-04-03T04:03:17.982 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c0add50 msgr2=0x7fc35c15a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:17.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 --2- 172.21.2.111:0/1043753520 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fc35c0add50 0x7fc35c15a0e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7fc35c0a77e0 tx=0x7fc34c006e70).stop 2021-04-03T04:03:17.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361194700 1 -- 172.21.2.111:0/1043753520 reap_dead start 2021-04-03T04:03:17.983 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 shutdown_connections 2021-04-03T04:03:17.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 >> 172.21.2.111:0/1043753520 conn(0x7fc35c0b61c0 msgr2=0x7fc35c0b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:17.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 shutdown_connections 2021-04-03T04:03:17.984 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:17.971+0000 7fc361995700 1 -- 172.21.2.111:0/1043753520 wait complete. 2021-04-03T04:03:17.985 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 6 2021-04-03T04:03:18.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f78256cd700 1 Processor -- start 2021-04-03T04:03:18.043 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f78256cd700 1 -- start start 2021-04-03T04:03:18.048 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f78256cd700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7f7818005640 0x7f78180d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f78256cd700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781803d840 con 0x7f781803f120 2021-04-03T04:03:18.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f78256cd700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781803f800 con 0x7f78180d8e90 2021-04-03T04:03:18.049 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f78256cd700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781803faa0 con 0x7f7818005640 2021-04-03T04:03:18.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f781ffff700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f781803f120 0x7f7818041fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60392/0 (socket says 172.21.2.111:60392) 2021-04-03T04:03:18.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.035+0000 7f781ffff700 1 -- 172.21.2.111:0/2423340979 learned_addr learned my addr 172.21.2.111:0/2423340979 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:18.050 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760805862 0 0) 0x7f781803d840 con 0x7f781803f120 2021-04-03T04:03:18.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 --> v1:172.21.2.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7804003240 con 0x7f781803f120 2021-04-03T04:03:18.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 <== mon.0 v1:172.21.2.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1722864541 0 0) 0x7f7804003240 con 0x7f781803f120 2021-04-03T04:03:18.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 >> v2:172.21.2.115:3300/0 conn(0x7f7818005640 msgr2=0x7f78180d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:18.051 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 --2- 172.21.2.111:0/2423340979 >> v2:172.21.2.115:3300/0 conn(0x7f7818005640 0x7f78180d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 >> v1:172.21.2.115:6789/0 conn(0x7f78180d8e90 legacy=0x7f78180e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:18.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 --> v1:172.21.2.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7818005ad0 con 0x7f781803f120 2021-04-03T04:03:18.052 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 <== mon.0 v1:172.21.2.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1048754291 0 0) 0x7f78140016d0 con 0x7f781803f120 2021-04-03T04:03:18.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 >> v1:172.21.2.111:6789/0 conn(0x7f781803f120 legacy=0x7f7818041fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7804003800 0x7f7804003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.053 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78040044e0 0x7f7804004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7804004e50 0x7f780400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781803faa0 con 0x7f78040044e0 2021-04-03T04:03:18.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780400bc20 con 0x7f7804004e50 2021-04-03T04:03:18.054 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780400bd60 con 0x7f7804003800 2021-04-03T04:03:18.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78040044e0 0x7f7804004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 -- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7804003800 msgr2=0x7f7804003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:18.055 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f7824ecc700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7804004e50 0x7f780400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781ffff700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7804003800 0x7f7804003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7804003800 0x7f7804003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 -- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7804004e50 msgr2=0x7f780400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:18.056 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7804004e50 0x7f780400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 -- 172.21.2.111:0/2423340979 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7804003240 con 0x7f78040044e0 2021-04-03T04:03:18.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781ffff700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7804003800 0x7f7804003c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:18.057 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f7824ecc700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f7804004e50 0x7f780400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:18.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f7824ecc700 1 -- 172.21.2.111:0/2423340979 reap_dead start 2021-04-03T04:03:18.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78040044e0 0x7f7804004960 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f7804004450 tx=0x7f780c006e10).ready entity=mon.0 client_cookie=2c7e11dbdf956e88 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f780c0097b0 con 0x7f78040044e0 2021-04-03T04:03:18.058 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c000e90 con 0x7f78040044e0 2021-04-03T04:03:18.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781effd700 1 -- 172.21.2.111:0/2423340979 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f780c009b00 con 0x7f78040044e0 2021-04-03T04:03:18.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- 172.21.2.111:0/2423340979 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78040044e0 msgr2=0x7f7804004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.059 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78040044e0 0x7f7804004960 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f7804004450 tx=0x7f780c006e10).stop 2021-04-03T04:03:18.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- 172.21.2.111:0/2423340979 shutdown_connections 2021-04-03T04:03:18.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 --2- 172.21.2.111:0/2423340979 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78040044e0 0x7f7804004960 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- 172.21.2.111:0/2423340979 >> 172.21.2.111:0/2423340979 conn(0x7f78180462b0 msgr2=0x7f78180466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.060 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- 172.21.2.111:0/2423340979 shutdown_connections 2021-04-03T04:03:18.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- 172.21.2.111:0/2423340979 wait complete. 2021-04-03T04:03:18.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 Processor -- start 2021-04-03T04:03:18.061 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- start start 2021-04-03T04:03:18.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78040044e0 0x7f781803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f781803b340 0x7f781803bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78180a2980 0x7f78180a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.062 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78180a3320 con 0x7f781803b340 2021-04-03T04:03:18.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78180a1100 con 0x7f78180a2980 2021-04-03T04:03:18.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78180a1370 con 0x7f78040044e0 2021-04-03T04:03:18.063 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f781803b340 0x7f781803bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f781803b340 0x7f781803bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59614/0 (socket says 172.21.2.111:59614) 2021-04-03T04:03:18.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 -- 172.21.2.111:0/1914787524 learned_addr learned my addr 172.21.2.111:0/1914787524 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:18.064 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 -- 172.21.2.111:0/1914787524 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78040044e0 msgr2=0x7f781803bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78040044e0 0x7f781803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 -- 172.21.2.111:0/1914787524 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78180a2980 msgr2=0x7f78180a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78180a2980 0x7f78180a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.065 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 -- 172.21.2.111:0/1914787524 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781803a460 con 0x7f781803b340 2021-04-03T04:03:18.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781f7fe700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f781803b340 0x7f781803bfc0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f7804004450 tx=0x7f780c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781d7fa700 1 -- 172.21.2.111:0/1914787524 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f780c01f3f0 con 0x7f781803b340 2021-04-03T04:03:18.066 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781d7fa700 1 -- 172.21.2.111:0/1914787524 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c01f550 con 0x7f781803b340 2021-04-03T04:03:18.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781803a900 con 0x7f781803b340 2021-04-03T04:03:18.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f781d7fa700 1 -- 172.21.2.111:0/1914787524 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f780c01f8b0 con 0x7f781803b340 2021-04-03T04:03:18.067 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.039+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78180ea8e0 con 0x7f781803b340 2021-04-03T04:03:18.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.043+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781803a2c0 con 0x7f781803b340 2021-04-03T04:03:18.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.043+0000 7f781d7fa700 1 -- 172.21.2.111:0/1914787524 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f780c01d070 con 0x7f781803b340 2021-04-03T04:03:18.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.043+0000 7f781d7fa700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7808056f70 0x7f7808059430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.068 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.043+0000 7f781d7fa700 1 -- 172.21.2.111:0/1914787524 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f780c071040 con 0x7f781803b340 2021-04-03T04:03:18.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.043+0000 7f781ffff700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7808056f70 0x7f7808059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.069 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.043+0000 7f781ffff700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7808056f70 0x7f7808059430 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f7814008000 tx=0x7f7814009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.070 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.043+0000 7f781d7fa700 1 -- 172.21.2.111:0/1914787524 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f781803a2c0 con 0x7f781803b340 2021-04-03T04:03:18.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f78180a1570 con 0x7f7808056f70 2021-04-03T04:03:18.145 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f781d7fa700 1 -- 172.21.2.111:0/1914787524 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7814007170 con 0x7f7808056f70 2021-04-03T04:03:18.146 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7808056f70 msgr2=0x7f7808059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7808056f70 0x7f7808059430 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f7814008000 tx=0x7f7814009040).stop 2021-04-03T04:03:18.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f781803b340 msgr2=0x7f781803bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.147 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f781803b340 0x7f781803bfc0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f7804004450 tx=0x7f780c015000).stop 2021-04-03T04:03:18.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 shutdown_connections 2021-04-03T04:03:18.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7808056f70 0x7f7808059430 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78180a2980 0x7f78180a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.148 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f78040044e0 0x7f781803bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 --2- 172.21.2.111:0/1914787524 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f781803b340 0x7f781803bfc0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 >> 172.21.2.111:0/1914787524 conn(0x7f78180462b0 msgr2=0x7f7818041610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.149 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 shutdown_connections 2021-04-03T04:03:18.150 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.139+0000 7f78256cd700 1 -- 172.21.2.111:0/1914787524 wait complete. 2021-04-03T04:03:18.150 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 6 to deep-scrub 2021-04-03T04:03:18.153 INFO:tasks.ceph:Scrubbing osd.7 2021-04-03T04:03:18.154 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 config set osd_debug_deep_scrub_sleep 0 2021-04-03T04:03:18.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 Processor -- start 2021-04-03T04:03:18.212 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- start start 2021-04-03T04:03:18.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f78880dcca0 0x7f78880df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.214 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788800b0f0 con 0x7f7888008820 2021-04-03T04:03:18.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78880265f0 con 0x7f788800ad10 2021-04-03T04:03:18.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7888026a90 con 0x7f78880dcca0 2021-04-03T04:03:18.215 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788dd9b700 1 --1- >> v1:172.21.2.111:6789/0 conn(0x7f7888008820 0x7f78880d7f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.111:6789/0 says I am v1:172.21.2.111:60414/0 (socket says 172.21.2.111:60414) 2021-04-03T04:03:18.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788dd9b700 1 -- 172.21.2.111:0/1380483791 learned_addr learned my addr 172.21.2.111:0/1380483791 (peer_addr_for_me v1:172.21.2.111:0/0) 2021-04-03T04:03:18.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788ed9d700 1 --2- 172.21.2.111:0/1380483791 >> v2:172.21.2.111:3300/0 conn(0x7f78880dcca0 0x7f78880df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.216 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788ed9d700 1 -- 172.21.2.111:0/1380483791 >> v1:172.21.2.115:6789/0 conn(0x7f788800ad10 legacy=0x7f78880d7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:18.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788d59a700 1 -- 172.21.2.111:0/1380483791 <== mon.0 v1:172.21.2.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091799906 0 0) 0x7f788800b0f0 con 0x7f7888008820 2021-04-03T04:03:18.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788ed9d700 1 -- 172.21.2.111:0/1380483791 >> v1:172.21.2.111:6789/0 conn(0x7f7888008820 legacy=0x7f78880d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788ed9d700 1 -- 172.21.2.111:0/1380483791 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7888042a90 con 0x7f78880dcca0 2021-04-03T04:03:18.217 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788ed9d700 1 --2- 172.21.2.111:0/1380483791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78880dcca0 0x7f78880df090 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7f78880d7750 tx=0x7f7880006dd0).ready entity=mon.0 client_cookie=6b203a75f53cfe3d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788d59a700 1 -- 172.21.2.111:0/1380483791 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7880009770 con 0x7f78880dcca0 2021-04-03T04:03:18.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788d59a700 1 -- 172.21.2.111:0/1380483791 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78800098d0 con 0x7f78880dcca0 2021-04-03T04:03:18.218 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788d59a700 1 -- 172.21.2.111:0/1380483791 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7880009c30 con 0x7f78880dcca0 2021-04-03T04:03:18.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- 172.21.2.111:0/1380483791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78880dcca0 msgr2=0x7f78880df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 --2- 172.21.2.111:0/1380483791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78880dcca0 0x7f78880df090 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7f78880d7750 tx=0x7f7880006dd0).stop 2021-04-03T04:03:18.219 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- 172.21.2.111:0/1380483791 shutdown_connections 2021-04-03T04:03:18.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 --2- 172.21.2.111:0/1380483791 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f78880dcca0 0x7f78880df090 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- 172.21.2.111:0/1380483791 >> 172.21.2.111:0/1380483791 conn(0x7f78880473d0 msgr2=0x7f78880477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.220 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- 172.21.2.111:0/1380483791 shutdown_connections 2021-04-03T04:03:18.221 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- 172.21.2.111:0/1380483791 wait complete. 2021-04-03T04:03:18.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 Processor -- start 2021-04-03T04:03:18.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- start start 2021-04-03T04:03:18.224 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7888008820 0x7f78880cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f788800ad10 0x7f7888042ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78880dcca0 0x7f78880e3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880cfcd0 con 0x7f788800ad10 2021-04-03T04:03:18.225 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788802a360 con 0x7f78880dcca0 2021-04-03T04:03:18.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.207+0000 7f788f59e700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788802a5c0 con 0x7f7888008820 2021-04-03T04:03:18.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f788800ad10 0x7f7888042ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.226 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f788800ad10 0x7f7888042ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59628/0 (socket says 172.21.2.111:59628) 2021-04-03T04:03:18.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 -- 172.21.2.111:0/194521564 learned_addr learned my addr 172.21.2.111:0/194521564 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:18.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788e59c700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7888008820 0x7f78880cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.227 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788ed9d700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78880dcca0 0x7f78880e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 -- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7888008820 msgr2=0x7f78880cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f7888008820 0x7f78880cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.228 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 -- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78880dcca0 msgr2=0x7f78880e3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f78880dcca0 0x7f78880e3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 -- 172.21.2.111:0/194521564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880e3460 con 0x7f788800ad10 2021-04-03T04:03:18.229 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788dd9b700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f788800ad10 0x7f7888042ce0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f7884000b80 tx=0x7f78840079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78840072e0 con 0x7f788800ad10 2021-04-03T04:03:18.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7884007440 con 0x7f788800ad10 2021-04-03T04:03:18.230 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78840057e0 con 0x7f788800ad10 2021-04-03T04:03:18.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78880e3150 con 0x7f788800ad10 2021-04-03T04:03:18.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7888043e80 con 0x7f788800ad10 2021-04-03T04:03:18.231 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78880e32e0 con 0x7f788800ad10 2021-04-03T04:03:18.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f7884010070 con 0x7f788800ad10 2021-04-03T04:03:18.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f786c056fb0 0x7f786c059470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.232 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f788406fcd0 con 0x7f788800ad10 2021-04-03T04:03:18.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788e59c700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f786c056fb0 0x7f786c059470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7f786c05dcd0 0x7f786c0600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.233 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 --> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f786c0606d0 con 0x7f786c05dcd0 2021-04-03T04:03:18.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78840700b0 con 0x7f788800ad10 2021-04-03T04:03:18.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788e59c700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f786c056fb0 0x7f786c059470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f7890082000 tx=0x7f789005afb0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.234 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788ed9d700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7f786c05dcd0 0x7f786c0600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f788ed9d700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7f786c05dcd0 0x7f786c0600f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.235 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.211+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== osd.7 v2:172.21.2.115:6824/16058 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f786c0606d0 con 0x7f786c05dcd0 2021-04-03T04:03:18.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 --> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f78880e4030 con 0x7f786c05dcd0 2021-04-03T04:03:18.236 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f787f7fe700 1 -- 172.21.2.111:0/194521564 <== osd.7 v2:172.21.2.115:6824/16058 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f78880e4030 con 0x7f786c05dcd0 2021-04-03T04:03:18.237 INFO:teuthology.orchestra.run.gibba011.stdout:{ 2021-04-03T04:03:18.237 INFO:teuthology.orchestra.run.gibba011.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-04-03T04:03:18.237 INFO:teuthology.orchestra.run.gibba011.stdout:} 2021-04-03T04:03:18.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7f786c05dcd0 msgr2=0x7f786c0600f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.239 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.115:6824/16058,v1:172.21.2.115:6825/16058] conn(0x7f786c05dcd0 0x7f786c0600f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f786c056fb0 msgr2=0x7f786c059470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f786c056fb0 0x7f786c059470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f7890082000 tx=0x7f789005afb0).stop 2021-04-03T04:03:18.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f788800ad10 msgr2=0x7f7888042ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.240 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 --2- 172.21.2.111:0/194521564 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f788800ad10 0x7f7888042ce0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f7884000b80 tx=0x7f78840079e0).stop 2021-04-03T04:03:18.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788ed9d700 1 -- 172.21.2.111:0/194521564 reap_dead start 2021-04-03T04:03:18.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 shutdown_connections 2021-04-03T04:03:18.241 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 >> 172.21.2.111:0/194521564 conn(0x7f78880473d0 msgr2=0x7f78880477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 shutdown_connections 2021-04-03T04:03:18.242 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.223+0000 7f788f59e700 1 -- 172.21.2.111:0/194521564 wait complete. 2021-04-03T04:03:18.242 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 7 2021-04-03T04:03:18.300 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 Processor -- start 2021-04-03T04:03:18.301 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- start start 2021-04-03T04:03:18.302 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f72b40603f0 0x7f72b41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40add50 0x7f72b4144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b40607d0 con 0x7f72b40b17b0 2021-04-03T04:03:18.303 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b41451c0 con 0x7f72b40603f0 2021-04-03T04:03:18.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b414b090 con 0x7f72b40add50 2021-04-03T04:03:18.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f72b40603f0 0x7f72b41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.304 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f72b40603f0 0x7f72b41447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59638/0 (socket says 172.21.2.111:59638) 2021-04-03T04:03:18.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 -- 172.21.2.111:0/824368271 learned_addr learned my addr 172.21.2.111:0/824368271 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:18.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b259c700 1 --2- 172.21.2.111:0/824368271 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40add50 0x7f72b4144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.305 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 -- 172.21.2.111:0/824368271 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40add50 msgr2=0x7f72b4144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:18.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 --2- 172.21.2.111:0/824368271 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40add50 0x7f72b4144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 -- 172.21.2.111:0/824368271 >> v1:172.21.2.115:6789/0 conn(0x7f72b40b17b0 legacy=0x7f72b4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:18.306 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 -- 172.21.2.111:0/824368271 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b414b330 con 0x7f72b40603f0 2021-04-03T04:03:18.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b259c700 1 --2- 172.21.2.111:0/824368271 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40add50 0x7f72b4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:18.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 --2- 172.21.2.111:0/824368271 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b41447e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f72b40b5600 tx=0x7f72a4006e90).ready entity=mon.0 client_cookie=162d5ab49e86ee13 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b1d9b700 1 -- 172.21.2.111:0/824368271 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a40098e0 con 0x7f72b40603f0 2021-04-03T04:03:18.307 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b1d9b700 1 -- 172.21.2.111:0/824368271 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a4009a40 con 0x7f72b40603f0 2021-04-03T04:03:18.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b1d9b700 1 -- 172.21.2.111:0/824368271 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a4009de0 con 0x7f72b40603f0 2021-04-03T04:03:18.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- 172.21.2.111:0/824368271 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 msgr2=0x7f72b41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.308 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- 172.21.2.111:0/824368271 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b41447e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f72b40b5600 tx=0x7f72a4006e90).stop 2021-04-03T04:03:18.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- 172.21.2.111:0/824368271 shutdown_connections 2021-04-03T04:03:18.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- 172.21.2.111:0/824368271 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b41447e0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.309 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- 172.21.2.111:0/824368271 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40add50 0x7f72b4144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- 172.21.2.111:0/824368271 >> 172.21.2.111:0/824368271 conn(0x7f72b40b61c0 msgr2=0x7f72b405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- 172.21.2.111:0/824368271 shutdown_connections 2021-04-03T04:03:18.310 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- 172.21.2.111:0/824368271 wait complete. 2021-04-03T04:03:18.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 Processor -- start 2021-04-03T04:03:18.311 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- start start 2021-04-03T04:03:18.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b40a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.313 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f72b40add50 0x7f72b415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40b17b0 0x7f72b4145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b415aa50 con 0x7f72b40603f0 2021-04-03T04:03:18.314 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b415acf0 con 0x7f72b40add50 2021-04-03T04:03:18.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b8831700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b415af90 con 0x7f72b40b17b0 2021-04-03T04:03:18.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b40a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.315 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b40a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59642/0 (socket says 172.21.2.111:59642) 2021-04-03T04:03:18.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.295+0000 7f72b2d9d700 1 -- 172.21.2.111:0/3232192293 learned_addr learned my addr 172.21.2.111:0/3232192293 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:18.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b2d9d700 1 -- 172.21.2.111:0/3232192293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40b17b0 msgr2=0x7f72b4145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b2d9d700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40b17b0 0x7f72b4145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.316 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b2d9d700 1 -- 172.21.2.111:0/3232192293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f72b40add50 msgr2=0x7f72b415a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b2d9d700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f72b40add50 0x7f72b415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b2d9d700 1 -- 172.21.2.111:0/3232192293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40a6f20 con 0x7f72b40603f0 2021-04-03T04:03:18.317 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b2d9d700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b40a7290 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7f72b40b5600 tx=0x7f72a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72a3fff700 1 -- 172.21.2.111:0/3232192293 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a4011e30 con 0x7f72b40603f0 2021-04-03T04:03:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72a3fff700 1 -- 172.21.2.111:0/3232192293 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a401e460 con 0x7f72b40603f0 2021-04-03T04:03:18.318 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b4067e00 con 0x7f72b40603f0 2021-04-03T04:03:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72a3fff700 1 -- 172.21.2.111:0/3232192293 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a401e7c0 con 0x7f72b40603f0 2021-04-03T04:03:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b4147de0 con 0x7f72b40603f0 2021-04-03T04:03:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b40a70b0 con 0x7f72b40603f0 2021-04-03T04:03:18.319 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72a3fff700 1 -- 172.21.2.111:0/3232192293 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f72a401a040 con 0x7f72b40603f0 2021-04-03T04:03:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72a3fff700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7290056f20 0x7f72900593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72a3fff700 1 -- 172.21.2.111:0/3232192293 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f72a4071590 con 0x7f72b40603f0 2021-04-03T04:03:18.320 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b259c700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7290056f20 0x7f72900593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72b259c700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7290056f20 0x7f72900593e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f72a8007000 tx=0x7f72a8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.321 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.299+0000 7f72a3fff700 1 -- 172.21.2.111:0/3232192293 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f72a4046c50 con 0x7f72b40603f0 2021-04-03T04:03:18.398 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.391+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f72b415a8d0 con 0x7f7290056f20 2021-04-03T04:03:18.399 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.391+0000 7f72a3fff700 1 -- 172.21.2.111:0/3232192293 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f72a8012470 con 0x7f7290056f20 2021-04-03T04:03:18.400 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7290056f20 msgr2=0x7f72900593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7290056f20 0x7f72900593e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f72a8007000 tx=0x7f72a8008040).stop 2021-04-03T04:03:18.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 msgr2=0x7f72b40a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.401 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b40a7290 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7f72b40b5600 tx=0x7f72a4015000).stop 2021-04-03T04:03:18.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 shutdown_connections 2021-04-03T04:03:18.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f72b40b17b0 0x7f72b4145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.402 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f72b40603f0 0x7f72b40a7290 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f7290056f20 0x7f72900593e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 --2- 172.21.2.111:0/3232192293 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f72b40add50 0x7f72b415a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.403 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 >> 172.21.2.111:0/3232192293 conn(0x7f72b40b61c0 msgr2=0x7f72b405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 shutdown_connections 2021-04-03T04:03:18.404 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.395+0000 7f72b8831700 1 -- 172.21.2.111:0/3232192293 wait complete. 2021-04-03T04:03:18.404 INFO:teuthology.orchestra.run.gibba011.stderr:instructed osd(s) 7 to deep-scrub 2021-04-03T04:03:18.407 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2021-04-03T04:03:18.463 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.455+0000 7fa3c8b36700 1 Processor -- start 2021-04-03T04:03:18.464 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.455+0000 7fa3c8b36700 1 -- start start 2021-04-03T04:03:18.465 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7fa3c40603f0 0x7fa3c40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3c40b0c90 0x7fa3c4147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.466 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c40af400 con 0x7fa3c414b700 2021-04-03T04:03:18.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c4060860 con 0x7fa3c40b0c90 2021-04-03T04:03:18.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c40af6a0 con 0x7fa3c40603f0 2021-04-03T04:03:18.467 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3c40b0c90 0x7fa3c4147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7fa3c40b0c90 0x7fa3c4147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59652/0 (socket says 172.21.2.111:59652) 2021-04-03T04:03:18.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 -- 172.21.2.111:0/4090836928 learned_addr learned my addr 172.21.2.111:0/4090836928 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:18.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 -- 172.21.2.111:0/4090836928 >> v2:172.21.2.115:3300/0 conn(0x7fa3c40603f0 msgr2=0x7fa3c40b0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:18.468 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 --2- 172.21.2.111:0/4090836928 >> v2:172.21.2.115:3300/0 conn(0x7fa3c40603f0 0x7fa3c40b0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 -- 172.21.2.111:0/4090836928 >> v1:172.21.2.115:6789/0 conn(0x7fa3c414b700 legacy=0x7fa3c40b0580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:18.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 -- 172.21.2.111:0/4090836928 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40afb40 con 0x7fa3c40b0c90 2021-04-03T04:03:18.469 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c3fff700 1 --2- 172.21.2.111:0/4090836928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40b0c90 0x7fa3c4147680 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7fa3c406a3b0 tx=0x7fa3bc009270).ready entity=mon.0 client_cookie=ef68c9c8d38ff6d0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c27fc700 1 -- 172.21.2.111:0/4090836928 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3bc00bbc0 con 0x7fa3c40b0c90 2021-04-03T04:03:18.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c27fc700 1 -- 172.21.2.111:0/4090836928 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc00bd20 con 0x7fa3c40b0c90 2021-04-03T04:03:18.470 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c27fc700 1 -- 172.21.2.111:0/4090836928 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3bc00be90 con 0x7fa3c40b0c90 2021-04-03T04:03:18.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- 172.21.2.111:0/4090836928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40b0c90 msgr2=0x7fa3c4147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/4090836928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40b0c90 0x7fa3c4147680 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7fa3c406a3b0 tx=0x7fa3bc009270).stop 2021-04-03T04:03:18.471 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- 172.21.2.111:0/4090836928 shutdown_connections 2021-04-03T04:03:18.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/4090836928 >> v2:172.21.2.115:3300/0 conn(0x7fa3c40603f0 0x7fa3c40b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/4090836928 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40b0c90 0x7fa3c4147680 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.472 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- 172.21.2.111:0/4090836928 >> 172.21.2.111:0/4090836928 conn(0x7fa3c40b61c0 msgr2=0x7fa3c405f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- 172.21.2.111:0/4090836928 shutdown_connections 2021-04-03T04:03:18.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- 172.21.2.111:0/4090836928 wait complete. 2021-04-03T04:03:18.473 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 Processor -- start 2021-04-03T04:03:18.474 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- start start 2021-04-03T04:03:18.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40603f0 0x7fa3c414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c40b0c90 0x7fa3c415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.476 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3c414b700 0x7fa3c4065af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4159c50 con 0x7fa3c40603f0 2021-04-03T04:03:18.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4159ef0 con 0x7fa3c40b0c90 2021-04-03T04:03:18.477 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4147320 con 0x7fa3c414b700 2021-04-03T04:03:18.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40603f0 0x7fa3c414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40603f0 0x7fa3c414cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59660/0 (socket says 172.21.2.111:59660) 2021-04-03T04:03:18.478 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 -- 172.21.2.111:0/1015172911 learned_addr learned my addr 172.21.2.111:0/1015172911 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:18.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 -- 172.21.2.111:0/1015172911 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3c414b700 msgr2=0x7fa3c4065af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3c414b700 0x7fa3c4065af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.479 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 -- 172.21.2.111:0/1015172911 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c40b0c90 msgr2=0x7fa3c415a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c40b0c90 0x7fa3c415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 -- 172.21.2.111:0/1015172911 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c415a880 con 0x7fa3c40603f0 2021-04-03T04:03:18.480 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c37fe700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40603f0 0x7fa3c414cfd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7fa3ac002f70 tx=0x7fa3ac006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c0ff9700 1 -- 172.21.2.111:0/1015172911 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3ac00d570 con 0x7fa3c40603f0 2021-04-03T04:03:18.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c0ff9700 1 -- 172.21.2.111:0/1015172911 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac00d6d0 con 0x7fa3c40603f0 2021-04-03T04:03:18.481 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c40a7540 con 0x7fa3c40603f0 2021-04-03T04:03:18.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c0ff9700 1 -- 172.21.2.111:0/1015172911 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3ac00da30 con 0x7fa3c40603f0 2021-04-03T04:03:18.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.459+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c40661f0 con 0x7fa3c40603f0 2021-04-03T04:03:18.482 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.463+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c415a0b0 con 0x7fa3c40603f0 2021-04-03T04:03:18.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.463+0000 7fa3c0ff9700 1 -- 172.21.2.111:0/1015172911 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7fa3ac00f070 con 0x7fa3c40603f0 2021-04-03T04:03:18.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.463+0000 7fa3c0ff9700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3a8056ea0 0x7fa3a8059360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:18.483 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.463+0000 7fa3c0ff9700 1 -- 172.21.2.111:0/1015172911 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7fa3ac06f670 con 0x7fa3c40603f0 2021-04-03T04:03:18.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.463+0000 7fa3c2ffd700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3a8056ea0 0x7fa3a8059360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:18.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.463+0000 7fa3c2ffd700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3a8056ea0 0x7fa3a8059360 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fa3b4000e90 tx=0x7fa3b4006910).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:18.484 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.463+0000 7fa3c0ff9700 1 -- 172.21.2.111:0/1015172911 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7fa3ac044d30 con 0x7fa3c40603f0 2021-04-03T04:03:18.560 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3c40a78d0 con 0x7fa3a8056ea0 2021-04-03T04:03:18.562 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c0ff9700 1 -- 172.21.2.111:0/1015172911 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66096 (secure 0 0 0) 0x7fa3b4012450 con 0x7fa3a8056ea0 2021-04-03T04:03:18.563 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T04:03:18.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3a8056ea0 msgr2=0x7fa3a8059360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3a8056ea0 0x7fa3a8059360 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fa3b4000e90 tx=0x7fa3b4006910).stop 2021-04-03T04:03:18.565 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40603f0 msgr2=0x7fa3c414cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:18.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40603f0 0x7fa3c414cfd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7fa3ac002f70 tx=0x7fa3ac006dd0).stop 2021-04-03T04:03:18.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 shutdown_connections 2021-04-03T04:03:18.566 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7fa3c414b700 0x7fa3c4065af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7fa3c40603f0 0x7fa3c414cfd0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7fa3a8056ea0 0x7fa3a8059360 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:18.567 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 --2- 172.21.2.111:0/1015172911 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7fa3c40b0c90 0x7fa3c415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:18.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 >> 172.21.2.111:0/1015172911 conn(0x7fa3c40b61c0 msgr2=0x7fa3c40ad740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:18.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 shutdown_connections 2021-04-03T04:03:18.568 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:18.555+0000 7fa3c8b36700 1 -- 172.21.2.111:0/1015172911 wait complete. 2021-04-03T04:03:18.569 INFO:teuthology.orchestra.run.gibba011.stderr:dumped all 2021-04-03T04:03:18.570 INFO:teuthology.orchestra.run.gibba011.stdout:{"pg_ready":true,"pg_map":{"version":1790,"stamp":"2021-04-03T04:03:17.136458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404032,"kb_used_data":15360,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570688,"statfs":{"total":773094113280,"available":764488384512,"internally_reserved":8589934592,"allocated":15728640,"data_stored":7742936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004914"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"127","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:43.711514+0000","last_change":"2021-04-03T03:08:39.165326+0000","last_active":"2021-04-03T03:17:43.711514+0000","last_peered":"2021-04-03T03:17:43.711514+0000","last_clean":"2021-04-03T03:17:43.711514+0000","last_became_active":"2021-04-03T03:08:39.165110+0000","last_became_peered":"2021-04-03T03:08:39.165110+0000","last_unstale":"2021-04-03T03:17:43.711514+0000","last_undegraded":"2021-04-03T03:17:43.711514+0000","last_fullsized":"2021-04-03T03:17:43.711514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"128","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.997771+0000","last_change":"2021-04-03T03:08:39.164016+0000","last_active":"2021-04-03T03:17:42.997771+0000","last_peered":"2021-04-03T03:17:42.997771+0000","last_clean":"2021-04-03T03:17:42.997771+0000","last_became_active":"2021-04-03T03:08:39.163862+0000","last_became_peered":"2021-04-03T03:08:39.163862+0000","last_unstale":"2021-04-03T03:17:42.997771+0000","last_undegraded":"2021-04-03T03:17:42.997771+0000","last_fullsized":"2021-04-03T03:17:42.997771+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"122","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.998099+0000","last_change":"2021-04-03T03:08:39.484523+0000","last_active":"2021-04-03T03:17:42.998099+0000","last_peered":"2021-04-03T03:17:42.998099+0000","last_clean":"2021-04-03T03:17:42.998099+0000","last_became_active":"2021-04-03T03:08:39.483298+0000","last_became_peered":"2021-04-03T03:08:39.483298+0000","last_unstale":"2021-04-03T03:17:42.998099+0000","last_undegraded":"2021-04-03T03:17:42.998099+0000","last_fullsized":"2021-04-03T03:17:42.998099+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"128","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.997753+0000","last_change":"2021-04-03T03:08:39.481713+0000","last_active":"2021-04-03T03:17:42.997753+0000","last_peered":"2021-04-03T03:17:42.997753+0000","last_clean":"2021-04-03T03:17:42.997753+0000","last_became_active":"2021-04-03T03:08:39.481308+0000","last_became_peered":"2021-04-03T03:08:39.481308+0000","last_unstale":"2021-04-03T03:17:42.997753+0000","last_undegraded":"2021-04-03T03:17:42.997753+0000","last_fullsized":"2021-04-03T03:17:42.997753+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"122","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.998041+0000","last_change":"2021-04-03T03:08:39.163433+0000","last_active":"2021-04-03T03:17:42.998041+0000","last_peered":"2021-04-03T03:17:42.998041+0000","last_clean":"2021-04-03T03:17:42.998041+0000","last_became_active":"2021-04-03T03:08:39.163151+0000","last_became_peered":"2021-04-03T03:08:39.163151+0000","last_unstale":"2021-04-03T03:17:42.998041+0000","last_undegraded":"2021-04-03T03:17:42.998041+0000","last_fullsized":"2021-04-03T03:17:42.998041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"127","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:42.998043+0000","last_change":"2021-04-03T03:08:35.464441+0000","last_active":"2021-04-03T03:17:42.998043+0000","last_peered":"2021-04-03T03:17:42.998043+0000","last_clean":"2021-04-03T03:17:42.998043+0000","last_became_active":"2021-04-03T03:08:35.463653+0000","last_became_peered":"2021-04-03T03:08:35.463653+0000","last_unstale":"2021-04-03T03:17:42.998043+0000","last_undegraded":"2021-04-03T03:17:42.998043+0000","last_fullsized":"2021-04-03T03:17:42.998043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:29.756831+0000","last_clean_scrub_stamp":"2021-04-03T03:08:29.756831+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"142","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:43.641745+0000","last_change":"2021-04-03T03:08:39.165073+0000","last_active":"2021-04-03T03:17:43.641745+0000","last_peered":"2021-04-03T03:17:43.641745+0000","last_clean":"2021-04-03T03:17:43.641745+0000","last_became_active":"2021-04-03T03:08:39.164901+0000","last_became_peered":"2021-04-03T03:08:39.164901+0000","last_unstale":"2021-04-03T03:17:43.641745+0000","last_undegraded":"2021-04-03T03:17:43.641745+0000","last_fullsized":"2021-04-03T03:17:43.641745+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"135","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:44.003798+0000","last_change":"2021-04-03T03:08:39.162544+0000","last_active":"2021-04-03T03:17:44.003798+0000","last_peered":"2021-04-03T03:17:44.003798+0000","last_clean":"2021-04-03T03:17:44.003798+0000","last_became_active":"2021-04-03T03:08:39.162345+0000","last_became_peered":"2021-04-03T03:08:39.162345+0000","last_unstale":"2021-04-03T03:17:44.003798+0000","last_undegraded":"2021-04-03T03:17:44.003798+0000","last_fullsized":"2021-04-03T03:17:44.003798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"142","reported_epoch":"153","state":"active+clean","last_fresh":"2021-04-03T03:17:43.641689+0000","last_change":"2021-04-03T03:08:39.165106+0000","last_active":"2021-04-03T03:17:43.641689+0000","last_peered":"2021-04-03T03:17:43.641689+0000","last_clean":"2021-04-03T03:17:43.641689+0000","last_became_active":"2021-04-03T03:08:39.164923+0000","last_became_peered":"2021-04-03T03:08:39.164923+0000","last_unstale":"2021-04-03T03:17:43.641689+0000","last_undegraded":"2021-04-03T03:17:43.641689+0000","last_fullsized":"2021-04-03T03:17:43.641689+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T03:08:38.136677+0000","last_clean_scrub_stamp":"2021-04-03T03:08:38.136677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575507,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2649999999999999,"15min":1.282},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":1.585,"5min":1.782,"15min":1.782},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.2350000000000001,"15min":1.2470000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6100000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.5980000000000001}]},{"osd":1,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.2,"15min":1.2430000000000001},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4219999999999999,"5min":1.679,"15min":1.679},"last":1.036},{"interface":"front","average":{"1min":1.222,"5min":1.2430000000000001,"15min":1.268},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4750000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.5589999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.228,"15min":1.238},"min":{"1min":0.47399999999999998,"5min":0.39100000000000001,"15min":0.33200000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.6279999999999999},"last":1.319},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":1.613,"5min":1.647,"15min":1.647},"last":1.2809999999999999}]},{"osd":3,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.495,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.621},"last":1.175},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.296},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.96899999999999997}]},{"osd":4,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.223,"15min":0.223},"max":{"1min":1.343,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.33100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.167,"5min":1.2969999999999999,"15min":1.3600000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.45800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":1.411,"5min":1.411,"15min":1.506},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.436,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.3600000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0820000000000001}]},{"osd":6,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.97499999999999998,"15min":1.0329999999999999},"min":{"1min":0.38600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.3879999999999999,"5min":1.462,"15min":1.5189999999999999},"last":1.506},{"interface":"front","average":{"1min":1.0900000000000001,"5min":0.96399999999999997,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.26300000000000001,"15min":0.24099999999999999},"max":{"1min":1.4550000000000001,"5min":1.5,"15min":1.5489999999999999},"last":1.024}]}]},{"osd":6,"up_from":13,"seq":55834575507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.254,"15min":1.2230000000000001},"min":{"1min":0.65500000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.53,"5min":1.653,"15min":2.044},"last":1.004},{"interface":"front","average":{"1min":1.232,"5min":1.2749999999999999,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":1.4750000000000001,"5min":1.6519999999999999,"15min":1.8999999999999999},"last":0.78400000000000003}]},{"osd":1,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.83199999999999996,"5min":0.32200000000000001,"15min":0.28699999999999998},"max":{"1min":1.4750000000000001,"5min":1.6160000000000001,"15min":1.6559999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.248},"min":{"1min":0.92400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.538,"5min":1.6639999999999999,"15min":1.7410000000000001},"last":0.86499999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2749999999999999,"15min":1.252},"min":{"1min":0.90200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.4990000000000001,"5min":1.768,"15min":1.768},"last":1.085},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.26,"15min":1.244},"min":{"1min":0.91200000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.52,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.2010000000000001}]},{"osd":3,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.26,"15min":1.242},"min":{"1min":0.98299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.522,"5min":1.5820000000000001,"15min":1.637},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.25,"15min":1.218},"min":{"1min":0.92300000000000004,"5min":0.46000000000000002,"15min":0.26600000000000001},"max":{"1min":1.4650000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.91300000000000003}]},{"osd":4,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.99199999999999999,"15min":0.98199999999999998},"min":{"1min":0.79400000000000004,"5min":0.26600000000000001,"15min":0.161},"max":{"1min":1.3440000000000001,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.589},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0369999999999999,"15min":1.008},"min":{"1min":0.745,"5min":0.28000000000000003,"15min":0.23400000000000001},"max":{"1min":1.4350000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.319}]},{"osd":5,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.91700000000000004,"15min":0.90100000000000002},"min":{"1min":0.76300000000000001,"5min":0.217,"15min":0.20499999999999999},"max":{"1min":1.2549999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98799999999999999,"15min":0.98399999999999999},"min":{"1min":0.86299999999999999,"5min":0.28100000000000003,"15min":0.19600000000000001},"max":{"1min":1.4019999999999999,"5min":1.5309999999999999,"15min":1.6040000000000001},"last":1.6699999999999999}]},{"osd":7,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":0.999},"min":{"1min":0.83099999999999996,"5min":0.30199999999999999,"15min":0.217},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.7749999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0129999999999999,"15min":1.006},"min":{"1min":0.80600000000000005,"5min":0.29499999999999998,"15min":0.189},"max":{"1min":1.2529999999999999,"5min":1.611,"15min":1.611},"last":1.3500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.224,"15min":1.208},"min":{"1min":0.81999999999999995,"5min":0.47299999999999998,"15min":0.23000000000000001},"max":{"1min":1.4299999999999999,"5min":1.6160000000000001,"15min":1.708},"last":1.351},{"interface":"front","average":{"1min":1.228,"5min":1.274,"15min":1.2529999999999999},"min":{"1min":0.88700000000000001,"5min":0.497,"15min":0.26600000000000001},"max":{"1min":1.4910000000000001,"5min":1.696,"15min":1.7030000000000001},"last":1.2250000000000001}]},{"osd":1,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.349,"15min":1.3340000000000001},"min":{"1min":0.998,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":1.657,"5min":1.7430000000000001,"15min":1.8129999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.2889999999999999,"15min":1.284},"min":{"1min":0.93899999999999995,"5min":0.628,"15min":0.316},"max":{"1min":1.524,"5min":1.661,"15min":1.661},"last":1.5269999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2689999999999999,"15min":1.2350000000000001},"min":{"1min":0.85999999999999999,"5min":0.63500000000000001,"15min":0.26900000000000002},"max":{"1min":1.546,"5min":1.6499999999999999,"15min":1.736},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.274,"15min":1.2529999999999999},"min":{"1min":1.0700000000000001,"5min":0.56200000000000006,"15min":0.32400000000000001},"max":{"1min":1.4650000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.2070000000000001}]},{"osd":3,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.266,"15min":1.274},"min":{"1min":1.0569999999999999,"5min":0.56599999999999995,"15min":0.29899999999999999},"max":{"1min":1.502,"5min":1.5269999999999999,"15min":1.7390000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.2769999999999999,"15min":1.2470000000000001},"min":{"1min":0.90800000000000003,"5min":0.69299999999999995,"15min":0.30199999999999999},"max":{"1min":1.5920000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.2569999999999999}]},{"osd":4,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89100000000000001,"15min":0.89200000000000002},"min":{"1min":0.754,"5min":0.25900000000000001,"15min":0.17299999999999999},"max":{"1min":1.2210000000000001,"5min":1.3300000000000001,"15min":1.389},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.97599999999999998,"15min":0.96699999999999997},"min":{"1min":0.76800000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.169,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.98499999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.03,"15min":1.034},"min":{"1min":0.78100000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.448,"5min":1.665,"15min":1.6759999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0649999999999999,"15min":1.0580000000000001},"min":{"1min":0.85799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3440000000000001,"5min":1.496,"15min":1.5309999999999999},"last":1.0820000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:02:19 2021","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0269999999999999,"15min":1.034},"min":{"1min":0.85499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":1.2969999999999999,"5min":1.7010000000000001,"15min":1.742},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.0680000000000001,"15min":1.0620000000000001},"min":{"1min":0.748,"5min":0.22900000000000001,"15min":0.224},"max":{"1min":1.3440000000000001,"5min":1.694,"15min":1.694},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834575508,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.64600000000000002,"5min":0.29899999999999999,"15min":0.253},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.4550000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81699999999999995,"15min":0.81200000000000006},"min":{"1min":0.72699999999999998,"5min":0.376,"15min":0.23899999999999999},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.772},"last":0.67100000000000004}]},{"osd":2,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.66500000000000004,"5min":0.254,"15min":0.22600000000000001},"max":{"1min":0.95299999999999996,"5min":1.0169999999999999,"15min":1.1140000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.32500000000000001,"15min":0.222},"max":{"1min":1.0680000000000001,"5min":1.147,"15min":1.4199999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.76500000000000001,"5min":0.34599999999999997,"15min":0.26100000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.9730000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.72499999999999998,"5min":0.379,"15min":0.223},"max":{"1min":1.0049999999999999,"5min":1.2729999999999999,"15min":1.4179999999999999},"last":0.747}]},{"osd":4,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.355,"15min":1.3420000000000001},"min":{"1min":1.048,"5min":0.70499999999999996,"15min":0.42099999999999999},"max":{"1min":1.6579999999999999,"5min":1.704,"15min":1.7569999999999999},"last":1.389},{"interface":"front","average":{"1min":1.371,"5min":1.365,"15min":1.353},"min":{"1min":1.143,"5min":0.69399999999999995,"15min":0.41799999999999998},"max":{"1min":1.546,"5min":1.675,"15min":1.8939999999999999},"last":1.371}]},{"osd":5,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":1.179,"5min":0.63400000000000001,"15min":0.35599999999999998},"max":{"1min":1.589,"5min":1.6990000000000001,"15min":1.8220000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.278,"15min":1.296},"min":{"1min":0.94999999999999996,"5min":0.68200000000000005,"15min":0.36799999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.744},"last":1.583}]},{"osd":6,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.321,"15min":1.3100000000000001},"min":{"1min":1.264,"5min":0.66900000000000004,"15min":0.47599999999999998},"max":{"1min":1.613,"5min":1.6830000000000001,"15min":1.925},"last":1.494},{"interface":"front","average":{"1min":1.383,"5min":1.3460000000000001,"15min":1.319},"min":{"1min":1.038,"5min":0.63100000000000001,"15min":0.38400000000000001},"max":{"1min":1.6299999999999999,"5min":1.8260000000000001,"15min":1.8540000000000001},"last":1.274}]},{"osd":7,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":1.0860000000000001,"5min":0.60399999999999998,"15min":0.41699999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.7929999999999999},"last":1.454},{"interface":"front","average":{"1min":1.415,"5min":1.373,"15min":1.337},"min":{"1min":1.1899999999999999,"5min":0.65400000000000003,"15min":0.45400000000000001},"max":{"1min":1.643,"5min":1.782,"15min":1.782},"last":1.413}]}]},{"osd":0,"up_from":13,"seq":55834575507,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.754,"15min":0.75},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.16700000000000001},"max":{"1min":0.83099999999999996,"5min":1.107,"15min":1.6639999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80400000000000005,"15min":0.79200000000000004},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20399999999999999},"max":{"1min":1.056,"5min":1.131,"15min":1.71},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85399999999999998,"15min":0.83299999999999996},"min":{"1min":0.318,"5min":0.315,"15min":0.188},"max":{"1min":1.0409999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79800000000000004,"15min":0.78000000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.218},"max":{"1min":1.052,"5min":1.052,"15min":1.248},"last":0.76700000000000002}]},{"osd":3,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81299999999999994,"15min":0.79100000000000004},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.17899999999999999},"max":{"1min":1.097,"5min":1.236,"15min":1.3580000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91600000000000004,"15min":0.89000000000000001},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.22},"max":{"1min":1.1359999999999999,"5min":1.264,"15min":1.264},"last":0.875}]},{"osd":4,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3049999999999999,"15min":1.288},"min":{"1min":0.623,"5min":0.497,"15min":0.26200000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.298,"15min":1.29},"min":{"1min":0.62,"5min":0.45500000000000002,"15min":0.28699999999999998},"max":{"1min":1.599,"5min":1.635,"15min":1.76},"last":0.78300000000000003}]},{"osd":5,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.59199999999999997,"5min":0.44,"15min":0.31900000000000001},"max":{"1min":1.575,"5min":1.613,"15min":1.839},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3300000000000001,"15min":1.2929999999999999},"min":{"1min":0.60399999999999998,"5min":0.505,"15min":0.32900000000000001},"max":{"1min":1.645,"5min":1.673,"15min":1.927},"last":1.1180000000000001}]},{"osd":6,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.2829999999999999},"min":{"1min":0.59899999999999998,"5min":0.47499999999999998,"15min":0.27200000000000002},"max":{"1min":1.5609999999999999,"5min":1.6419999999999999,"15min":1.77},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.323,"5min":1.3380000000000001,"15min":1.3109999999999999},"min":{"1min":0.61299999999999999,"5min":0.48599999999999999,"15min":0.24299999999999999},"max":{"1min":1.611,"5min":1.663,"15min":1.7629999999999999},"last":1.0669999999999999}]},{"osd":7,"last update":"Sat Apr 3 04:02:27 2021","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.357,"15min":1.359},"min":{"1min":0.63900000000000001,"5min":0.47799999999999998,"15min":0.254},"max":{"1min":1.736,"5min":1.736,"15min":1.8},"last":1.167},{"interface":"front","average":{"1min":1.395,"5min":1.409,"15min":1.391},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.29599999999999999},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.899},"last":0.91700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.746},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.17499999999999999},"max":{"1min":0.90800000000000003,"5min":1.1040000000000001,"15min":1.2170000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78900000000000003,"15min":0.78400000000000003},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.182},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.264},"last":0.76200000000000001}]},{"osd":1,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.27600000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.121},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.19900000000000001},"max":{"1min":1.087,"5min":1.2010000000000001,"15min":1.206},"last":0.89800000000000002}]},{"osd":3,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"min":{"1min":0.29599999999999999,"5min":0.24099999999999999,"15min":0.219},"max":{"1min":1.046,"5min":1.1339999999999999,"15min":1.2290000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.23999999999999999,"5min":0.216,"15min":0.191},"max":{"1min":0.93600000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.71299999999999997}]},{"osd":4,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3520000000000001},"min":{"1min":0.61499999999999999,"5min":0.48699999999999999,"15min":0.27000000000000002},"max":{"1min":1.54,"5min":1.79,"15min":1.79},"last":1.339},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.70599999999999996,"5min":0.52200000000000002,"15min":0.315},"max":{"1min":1.591,"5min":1.8520000000000001,"15min":1.857},"last":1.472}]},{"osd":5,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.331},"min":{"1min":0.64200000000000002,"5min":0.47999999999999998,"15min":0.27400000000000002},"max":{"1min":1.6619999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.367},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.63300000000000001,"5min":0.51400000000000001,"15min":0.29299999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.7769999999999999},"last":1.506}]},{"osd":6,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3520000000000001,"15min":1.3879999999999999},"min":{"1min":0.65700000000000003,"5min":0.46700000000000003,"15min":0.32500000000000001},"max":{"1min":1.742,"5min":1.802,"15min":1.962},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3169999999999999,"15min":1.3280000000000001},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.35399999999999998},"max":{"1min":1.6299999999999999,"5min":1.744,"15min":1.9019999999999999},"last":1.4550000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.369,"15min":1.377},"min":{"1min":0.58999999999999997,"5min":0.45100000000000001,"15min":0.28199999999999997},"max":{"1min":1.645,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":0.68400000000000005,"5min":0.49299999999999999,"15min":0.29799999999999999},"max":{"1min":1.5609999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.4099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575507,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.59699999999999998,"5min":0.222,"15min":0.17399999999999999},"max":{"1min":1.016,"5min":1.089,"15min":1.3160000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84999999999999998,"15min":0.85499999999999998},"min":{"1min":0.747,"5min":0.185,"15min":0.185},"max":{"1min":1.069,"5min":1.1180000000000001,"15min":1.284},"last":0.81000000000000005}]},{"osd":1,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.19500000000000001,"15min":0.184},"max":{"1min":0.96799999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.71799999999999997,"5min":0.20599999999999999,"15min":0.193},"max":{"1min":1.1599999999999999,"5min":1.165,"15min":1.2},"last":0.88500000000000001}]},{"osd":2,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.84299999999999997,"15min":0.85899999999999999},"min":{"1min":0.69399999999999995,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.89000000000000001,"15min":0.89900000000000002},"min":{"1min":0.79900000000000004,"5min":0.215,"15min":0.20300000000000001},"max":{"1min":1.1739999999999999,"5min":1.2,"15min":1.2490000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4119999999999999,"15min":1.4350000000000001},"min":{"1min":1.246,"5min":0.40100000000000002,"15min":0.26000000000000001},"max":{"1min":1.5920000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3280000000000001,"15min":1.3360000000000001},"min":{"1min":0.73899999999999999,"5min":0.373,"15min":0.27800000000000002},"max":{"1min":1.5349999999999999,"5min":1.577,"15min":1.6180000000000001},"last":1.3799999999999999}]},{"osd":5,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2909999999999999,"15min":1.3149999999999999},"min":{"1min":0.76000000000000001,"5min":0.35099999999999998,"15min":0.24199999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.76},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.93000000000000005,"5min":0.38500000000000001,"15min":0.28499999999999998},"max":{"1min":1.603,"5min":1.7330000000000001,"15min":1.7370000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.355,"15min":1.357},"min":{"1min":1.016,"5min":0.39300000000000002,"15min":0.23300000000000001},"max":{"1min":1.623,"5min":1.76,"15min":1.7729999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.302,"15min":1.331},"min":{"1min":1.1200000000000001,"5min":0.34499999999999997,"15min":0.27100000000000002},"max":{"1min":1.647,"5min":1.663,"15min":1.798},"last":1.274}]},{"osd":7,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3220000000000001,"15min":1.3460000000000001},"min":{"1min":1.0780000000000001,"5min":0.376,"15min":0.314},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.724},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3660000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.36699999999999999,"15min":0.30499999999999999},"max":{"1min":1.6799999999999999,"5min":1.708,"15min":1.8660000000000001},"last":1.4159999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.242,"15min":1.228},"min":{"1min":0.65100000000000002,"5min":0.36699999999999999,"15min":0.314},"max":{"1min":1.641,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.151},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2330000000000001,"15min":1.1990000000000001},"min":{"1min":0.68000000000000005,"5min":0.33000000000000002,"15min":0.309},"max":{"1min":1.623,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.232}]},{"osd":1,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.248,"15min":1.2190000000000001},"min":{"1min":0.66000000000000003,"5min":0.375,"15min":0.27400000000000002},"max":{"1min":1.554,"5min":1.772,"15min":1.772},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.264,"15min":1.228},"min":{"1min":0.63400000000000001,"5min":0.36799999999999999,"15min":0.28199999999999997},"max":{"1min":1.548,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.3680000000000001}]},{"osd":2,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.26,"15min":1.2470000000000001},"min":{"1min":0.73199999999999998,"5min":0.42299999999999999,"15min":0.28799999999999998},"max":{"1min":1.5840000000000001,"5min":1.673,"15min":1.75},"last":1.264},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2969999999999999,"15min":1.248},"min":{"1min":0.67400000000000004,"5min":0.34399999999999997,"15min":0.27900000000000003},"max":{"1min":1.7030000000000001,"5min":2.024,"15min":2.024},"last":1.2789999999999999}]},{"osd":3,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2569999999999999},"min":{"1min":0.70999999999999996,"5min":0.379,"15min":0.29999999999999999},"max":{"1min":1.488,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.194},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3580000000000001,"15min":1.321},"min":{"1min":0.64100000000000001,"5min":0.39900000000000002,"15min":0.29799999999999999},"max":{"1min":1.593,"5min":1.99,"15min":1.99},"last":1.3089999999999999}]},{"osd":5,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.91100000000000003,"15min":0.92100000000000004},"min":{"1min":0.32800000000000001,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":1.1559999999999999,"5min":1.5509999999999999,"15min":1.6379999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.90200000000000002},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.13300000000000001},"max":{"1min":1.202,"5min":1.448,"15min":1.448},"last":1.1279999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94499999999999995,"15min":0.93400000000000005},"min":{"1min":0.33600000000000002,"5min":0.24299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2549999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.995,"15min":0.98499999999999999},"min":{"1min":0.34699999999999998,"5min":0.219,"15min":0.219},"max":{"1min":1.3959999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.89500000000000002}]},{"osd":7,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.153},"max":{"1min":1.446,"5min":1.681,"15min":1.681},"last":1.006},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":1.0209999999999999},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.23100000000000001},"max":{"1min":1.5149999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.0600000000000001}]}]}]}} 2021-04-03T04:03:18.575 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.576 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.576 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.577 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.578 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.579 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2021-04-03T03:08:29.756831+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=29, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.579 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.580 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.581 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2021-04-03T03:08:38.136677+0000 time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=3, tm_min=8, tm_sec=38, tm_wday=5, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=4, tm_mday=3, tm_hour=4, tm_min=3, tm_sec=15, tm_wday=5, tm_yday=93, tm_isdst=0) 2021-04-03T04:03:18.581 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2021-04-03T04:03:34.340 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:34.344 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:34.347 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:34.351 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:34.354 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:38.583 DEBUG:teuthology.orchestra.run.gibba011:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2021-04-03T04:03:38.643 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.636+0000 7ff4b229d700 1 Processor -- start 2021-04-03T04:03:38.644 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.636+0000 7ff4b229d700 1 -- start start 2021-04-03T04:03:38.645 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- >> v2:172.21.2.115:3300/0 conn(0x7ff4a40d5e10 0x7ff4a4041a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4a40431a0 0x7ff4a40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- --> v1:172.21.2.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a40492e0 con 0x7ff4a4048f00 2021-04-03T04:03:38.646 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a403fac0 con 0x7ff4a40431a0 2021-04-03T04:03:38.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- --> v2:172.21.2.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a403fd60 con 0x7ff4a40d5e10 2021-04-03T04:03:38.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4a40431a0 0x7ff4a40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.647 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7ff4a40431a0 0x7ff4a40d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59668/0 (socket says 172.21.2.111:59668) 2021-04-03T04:03:38.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 -- 172.21.2.111:0/3728631021 learned_addr learned my addr 172.21.2.111:0/3728631021 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:38.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 -- 172.21.2.111:0/3728631021 >> v2:172.21.2.115:3300/0 conn(0x7ff4a40d5e10 msgr2=0x7ff4a4041a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-04-03T04:03:38.648 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 --2- 172.21.2.111:0/3728631021 >> v2:172.21.2.115:3300/0 conn(0x7ff4a40d5e10 0x7ff4a4041a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 -- 172.21.2.111:0/3728631021 >> v1:172.21.2.111:6789/0 conn(0x7ff4a4048f00 legacy=0x7ff4a40d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 -- 172.21.2.111:0/3728631021 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40d5260 con 0x7ff4a40431a0 2021-04-03T04:03:38.649 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b1a9c700 1 --2- 172.21.2.111:0/3728631021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a40431a0 0x7ff4a40d1bf0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7ff4ac06b5d0 tx=0x7ff4ac05ad70).ready entity=mon.0 client_cookie=83baa95b98a1199d server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:38.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff49bfff700 1 -- 172.21.2.111:0/3728631021 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4ac05bda0 con 0x7ff4a40431a0 2021-04-03T04:03:38.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff49bfff700 1 -- 172.21.2.111:0/3728631021 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac05df60 con 0x7ff4a40431a0 2021-04-03T04:03:38.650 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff49bfff700 1 -- 172.21.2.111:0/3728631021 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4ac0699f0 con 0x7ff4a40431a0 2021-04-03T04:03:38.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- 172.21.2.111:0/3728631021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a40431a0 msgr2=0x7ff4a40d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3728631021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a40431a0 0x7ff4a40d1bf0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7ff4ac06b5d0 tx=0x7ff4ac05ad70).stop 2021-04-03T04:03:38.651 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- 172.21.2.111:0/3728631021 shutdown_connections 2021-04-03T04:03:38.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3728631021 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a40431a0 0x7ff4a40d1bf0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3728631021 >> v2:172.21.2.115:3300/0 conn(0x7ff4a40d5e10 0x7ff4a4041a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- 172.21.2.111:0/3728631021 >> 172.21.2.111:0/3728631021 conn(0x7ff4a400f2c0 msgr2=0x7ff4a40484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:38.652 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- 172.21.2.111:0/3728631021 shutdown_connections 2021-04-03T04:03:38.653 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- 172.21.2.111:0/3728631021 wait complete. 2021-04-03T04:03:38.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 Processor -- start 2021-04-03T04:03:38.655 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- start start 2021-04-03T04:03:38.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4a40431a0 0x7ff4a40d7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a4048f00 0x7ff4a40eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.656 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4a40d5e10 0x7ff4a40465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40d9c50 con 0x7ff4a4048f00 2021-04-03T04:03:38.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40eaf60 con 0x7ff4a40d5e10 2021-04-03T04:03:38.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a403dc00 con 0x7ff4a40431a0 2021-04-03T04:03:38.657 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a4048f00 0x7ff4a40eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a4048f00 0x7ff4a40eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59672/0 (socket says 172.21.2.111:59672) 2021-04-03T04:03:38.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 -- 172.21.2.111:0/3034496361 learned_addr learned my addr 172.21.2.111:0/3034496361 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:38.658 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 -- 172.21.2.111:0/3034496361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4a40431a0 msgr2=0x7ff4a40d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4a40431a0 0x7ff4a40d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 -- 172.21.2.111:0/3034496361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4a40d5e10 msgr2=0x7ff4a40465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.659 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4a40d5e10 0x7ff4a40465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 -- 172.21.2.111:0/3034496361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40d6460 con 0x7ff4a4048f00 2021-04-03T04:03:38.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b0a9a700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a4048f00 0x7ff4a40eaa70 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7ff49c002eb0 tx=0x7ff49c006d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:38.660 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff49a7fc700 1 -- 172.21.2.111:0/3034496361 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49c00d4a0 con 0x7ff4a4048f00 2021-04-03T04:03:38.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff49a7fc700 1 -- 172.21.2.111:0/3034496361 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c000e90 con 0x7ff4a4048f00 2021-04-03T04:03:38.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a40d6700 con 0x7ff4a4048f00 2021-04-03T04:03:38.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff49a7fc700 1 -- 172.21.2.111:0/3034496361 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49c00d7f0 con 0x7ff4a4048f00 2021-04-03T04:03:38.661 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.640+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40da080 con 0x7ff4a4048f00 2021-04-03T04:03:38.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.644+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4004da0 con 0x7ff4a4048f00 2021-04-03T04:03:38.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.644+0000 7ff49a7fc700 1 -- 172.21.2.111:0/3034496361 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7ff49c00f030 con 0x7ff4a4048f00 2021-04-03T04:03:38.662 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.644+0000 7ff49a7fc700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff48c057050 0x7ff48c059510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.644+0000 7ff49a7fc700 1 -- 172.21.2.111:0/3034496361 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7ff49c06fc30 con 0x7ff4a4048f00 2021-04-03T04:03:38.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.644+0000 7ff4b129b700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff48c057050 0x7ff48c059510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.663 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.644+0000 7ff4b129b700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff48c057050 0x7ff48c059510 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7ff4a4040ec0 tx=0x7ff4a80077f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:38.664 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.644+0000 7ff49a7fc700 1 -- 172.21.2.111:0/3034496361 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7ff49c0451e0 con 0x7ff4a4048f00 2021-04-03T04:03:38.740 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 --> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4a40d9de0 con 0x7ff48c057050 2021-04-03T04:03:38.742 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff49a7fc700 1 -- 172.21.2.111:0/3034496361 <== mgr.4104 v2:172.21.2.111:6800/15722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66190 (secure 0 0 0) 0x7ff4a8007030 con 0x7ff48c057050 2021-04-03T04:03:38.743 INFO:teuthology.orchestra.run.gibba011.stdout: 2021-04-03T04:03:38.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff48c057050 msgr2=0x7ff48c059510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff48c057050 0x7ff48c059510 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7ff4a4040ec0 tx=0x7ff4a80077f0).stop 2021-04-03T04:03:38.745 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a4048f00 msgr2=0x7ff4a40eaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a4048f00 0x7ff4a40eaa70 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7ff49c002eb0 tx=0x7ff49c006d30).stop 2021-04-03T04:03:38.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 shutdown_connections 2021-04-03T04:03:38.746 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7ff48c057050 0x7ff48c059510 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7ff4a40d5e10 0x7ff4a40465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7ff4a40431a0 0x7ff4a40d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.747 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 --2- 172.21.2.111:0/3034496361 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7ff4a4048f00 0x7ff4a40eaa70 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 >> 172.21.2.111:0/3034496361 conn(0x7ff4a400f2c0 msgr2=0x7ff4a4047830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:38.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 shutdown_connections 2021-04-03T04:03:38.748 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.736+0000 7ff4b229d700 1 -- 172.21.2.111:0/3034496361 wait complete. 2021-04-03T04:03:38.748 INFO:teuthology.orchestra.run.gibba011.stderr:dumped all 2021-04-03T04:03:38.750 INFO:teuthology.orchestra.run.gibba011.stdout:{"pg_ready":true,"pg_map":{"version":1800,"stamp":"2021-04-03T04:03:37.145533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404032,"kb_used_data":15360,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570688,"statfs":{"total":773094113280,"available":764488384512,"internally_reserved":8589934592,"allocated":15728640,"data_stored":7742936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005538"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"130","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:19.553401+0000","last_change":"2021-04-03T04:03:19.553444+0000","last_active":"2021-04-03T04:03:19.553401+0000","last_peered":"2021-04-03T04:03:19.553401+0000","last_clean":"2021-04-03T04:03:19.553401+0000","last_became_active":"2021-04-03T03:08:39.165110+0000","last_became_peered":"2021-04-03T03:08:39.165110+0000","last_unstale":"2021-04-03T04:03:19.553401+0000","last_undegraded":"2021-04-03T04:03:19.553401+0000","last_fullsized":"2021-04-03T04:03:19.553401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:19.553400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:19.553400+0000","last_clean_scrub_stamp":"2021-04-03T04:03:19.553400+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"131","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:26.938495+0000","last_change":"2021-04-03T04:03:26.938539+0000","last_active":"2021-04-03T04:03:26.938495+0000","last_peered":"2021-04-03T04:03:26.938495+0000","last_clean":"2021-04-03T04:03:26.938495+0000","last_became_active":"2021-04-03T03:08:39.163862+0000","last_became_peered":"2021-04-03T03:08:39.163862+0000","last_unstale":"2021-04-03T04:03:26.938495+0000","last_undegraded":"2021-04-03T04:03:26.938495+0000","last_fullsized":"2021-04-03T04:03:26.938495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:26.938495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:26.938495+0000","last_clean_scrub_stamp":"2021-04-03T04:03:26.938495+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"125","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:22.592610+0000","last_change":"2021-04-03T04:03:22.592679+0000","last_active":"2021-04-03T04:03:22.592610+0000","last_peered":"2021-04-03T04:03:22.592610+0000","last_clean":"2021-04-03T04:03:22.592610+0000","last_became_active":"2021-04-03T03:08:39.483298+0000","last_became_peered":"2021-04-03T03:08:39.483298+0000","last_unstale":"2021-04-03T04:03:22.592610+0000","last_undegraded":"2021-04-03T04:03:22.592610+0000","last_fullsized":"2021-04-03T04:03:22.592610+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:22.592610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:22.592610+0000","last_clean_scrub_stamp":"2021-04-03T04:03:22.592610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"131","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:25.970938+0000","last_change":"2021-04-03T04:03:25.970988+0000","last_active":"2021-04-03T04:03:25.970938+0000","last_peered":"2021-04-03T04:03:25.970938+0000","last_clean":"2021-04-03T04:03:25.970938+0000","last_became_active":"2021-04-03T03:08:39.481308+0000","last_became_peered":"2021-04-03T03:08:39.481308+0000","last_unstale":"2021-04-03T04:03:25.970938+0000","last_undegraded":"2021-04-03T04:03:25.970938+0000","last_fullsized":"2021-04-03T04:03:25.970938+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:25.970937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:25.970937+0000","last_clean_scrub_stamp":"2021-04-03T04:03:25.970937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"125","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:20.562987+0000","last_change":"2021-04-03T04:03:20.563229+0000","last_active":"2021-04-03T04:03:20.562987+0000","last_peered":"2021-04-03T04:03:20.562987+0000","last_clean":"2021-04-03T04:03:20.562987+0000","last_became_active":"2021-04-03T03:08:39.163151+0000","last_became_peered":"2021-04-03T03:08:39.163151+0000","last_unstale":"2021-04-03T04:03:20.562987+0000","last_undegraded":"2021-04-03T04:03:20.562987+0000","last_fullsized":"2021-04-03T04:03:20.562987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:20.562986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:20.562986+0000","last_clean_scrub_stamp":"2021-04-03T04:03:20.562986+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"154'4","reported_seq":"132","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:18.536883+0000","last_change":"2021-04-03T04:03:18.537102+0000","last_active":"2021-04-03T04:03:18.536883+0000","last_peered":"2021-04-03T04:03:18.536883+0000","last_clean":"2021-04-03T04:03:18.536883+0000","last_became_active":"2021-04-03T03:08:35.463653+0000","last_became_peered":"2021-04-03T03:08:35.463653+0000","last_unstale":"2021-04-03T04:03:18.536883+0000","last_undegraded":"2021-04-03T04:03:18.536883+0000","last_fullsized":"2021-04-03T04:03:18.536883+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'4","last_scrub_stamp":"2021-04-03T04:03:18.536883+0000","last_deep_scrub":"154'4","last_deep_scrub_stamp":"2021-04-03T04:03:18.536883+0000","last_clean_scrub_stamp":"2021-04-03T04:03:18.536883+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"145","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:20.226998+0000","last_change":"2021-04-03T04:03:20.227043+0000","last_active":"2021-04-03T04:03:20.226998+0000","last_peered":"2021-04-03T04:03:20.226998+0000","last_clean":"2021-04-03T04:03:20.226998+0000","last_became_active":"2021-04-03T03:08:39.164901+0000","last_became_peered":"2021-04-03T03:08:39.164901+0000","last_unstale":"2021-04-03T04:03:20.226998+0000","last_undegraded":"2021-04-03T04:03:20.226998+0000","last_fullsized":"2021-04-03T04:03:20.226998+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:20.226998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:20.226998+0000","last_clean_scrub_stamp":"2021-04-03T04:03:20.226998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"138","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:20.429554+0000","last_change":"2021-04-03T04:03:20.429594+0000","last_active":"2021-04-03T04:03:20.429554+0000","last_peered":"2021-04-03T04:03:20.429554+0000","last_clean":"2021-04-03T04:03:20.429554+0000","last_became_active":"2021-04-03T03:08:39.162345+0000","last_became_peered":"2021-04-03T03:08:39.162345+0000","last_unstale":"2021-04-03T04:03:20.429554+0000","last_undegraded":"2021-04-03T04:03:20.429554+0000","last_fullsized":"2021-04-03T04:03:20.429554+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:20.429554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:20.429554+0000","last_clean_scrub_stamp":"2021-04-03T04:03:20.429554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"145","reported_epoch":"154","state":"active+clean","last_fresh":"2021-04-03T04:03:19.207867+0000","last_change":"2021-04-03T04:03:19.207918+0000","last_active":"2021-04-03T04:03:19.207867+0000","last_peered":"2021-04-03T04:03:19.207867+0000","last_clean":"2021-04-03T04:03:19.207867+0000","last_became_active":"2021-04-03T03:08:39.164923+0000","last_became_peered":"2021-04-03T03:08:39.164923+0000","last_unstale":"2021-04-03T04:03:19.207867+0000","last_undegraded":"2021-04-03T04:03:19.207867+0000","last_fullsized":"2021-04-03T04:03:19.207867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-04-03T04:03:19.207867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-04-03T04:03:19.207867+0000","last_clean_scrub_stamp":"2021-04-03T04:03:19.207867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575511,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2649999999999999,"15min":1.282},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.31},"max":{"1min":1.585,"5min":1.782,"15min":1.782},"last":1.27},{"interface":"front","average":{"1min":1.202,"5min":1.2350000000000001,"15min":1.2470000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6100000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.091}]},{"osd":1,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.2,"15min":1.2430000000000001},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4219999999999999,"5min":1.679,"15min":1.679},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.2430000000000001,"15min":1.268},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4750000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.139}]},{"osd":2,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.228,"15min":1.238},"min":{"1min":0.47399999999999998,"5min":0.39100000000000001,"15min":0.33200000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.6279999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":1.613,"5min":1.647,"15min":1.647},"last":1.0549999999999999}]},{"osd":3,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.495,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.621},"last":1.175},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.296},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":1.5720000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.223,"15min":0.223},"max":{"1min":1.343,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.33100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.167,"5min":1.2969999999999999,"15min":1.3600000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.45800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":1.411,"5min":1.411,"15min":1.506},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.436,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.3600000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Sat Apr 3 04:03:05 2021","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.97499999999999998,"15min":1.0329999999999999},"min":{"1min":0.38600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.3879999999999999,"5min":1.462,"15min":1.5189999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":0.96399999999999997,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.26300000000000001,"15min":0.24099999999999999},"max":{"1min":1.4550000000000001,"5min":1.5,"15min":1.5489999999999999},"last":1.077}]}]},{"osd":6,"up_from":13,"seq":55834575511,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.254,"15min":1.2230000000000001},"min":{"1min":0.65500000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.53,"5min":1.653,"15min":2.044},"last":1.252},{"interface":"front","average":{"1min":1.232,"5min":1.2749999999999999,"15min":1.2290000000000001},"min":{"1min":0.76200000000000001,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":1.4750000000000001,"5min":1.6519999999999999,"15min":1.8999999999999999},"last":1.012}]},{"osd":1,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.83199999999999996,"5min":0.32200000000000001,"15min":0.28699999999999998},"max":{"1min":1.4750000000000001,"5min":1.6160000000000001,"15min":1.6559999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.248},"min":{"1min":0.92400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.538,"5min":1.6639999999999999,"15min":1.7410000000000001},"last":1.1200000000000001}]},{"osd":2,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2749999999999999,"15min":1.252},"min":{"1min":0.90200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.4990000000000001,"5min":1.768,"15min":1.768},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.26,"15min":1.244},"min":{"1min":0.91200000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.52,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.079}]},{"osd":3,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.26,"15min":1.242},"min":{"1min":0.98299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.522,"5min":1.5820000000000001,"15min":1.637},"last":1.47},{"interface":"front","average":{"1min":1.242,"5min":1.25,"15min":1.218},"min":{"1min":0.92300000000000004,"5min":0.46000000000000002,"15min":0.26600000000000001},"max":{"1min":1.4650000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2070000000000001}]},{"osd":4,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.99199999999999999,"15min":0.98199999999999998},"min":{"1min":0.79400000000000004,"5min":0.26600000000000001,"15min":0.161},"max":{"1min":1.3440000000000001,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0369999999999999,"15min":1.008},"min":{"1min":0.745,"5min":0.28000000000000003,"15min":0.23400000000000001},"max":{"1min":1.4350000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.82099999999999995}]},{"osd":5,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.91700000000000004,"15min":0.90100000000000002},"min":{"1min":0.76300000000000001,"5min":0.217,"15min":0.20499999999999999},"max":{"1min":1.2549999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98799999999999999,"15min":0.98399999999999999},"min":{"1min":0.86299999999999999,"5min":0.28100000000000003,"15min":0.19600000000000001},"max":{"1min":1.4019999999999999,"5min":1.5309999999999999,"15min":1.6040000000000001},"last":0.878}]},{"osd":7,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":0.999},"min":{"1min":0.83099999999999996,"5min":0.30199999999999999,"15min":0.217},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.7749999999999999},"last":1.294},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0129999999999999,"15min":1.006},"min":{"1min":0.80600000000000005,"5min":0.29499999999999998,"15min":0.189},"max":{"1min":1.2529999999999999,"5min":1.611,"15min":1.611},"last":0.94099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575511,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:19 2021","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2230000000000001,"15min":1.214},"min":{"1min":1.0589999999999999,"5min":0.47299999999999998,"15min":0.23000000000000001},"max":{"1min":1.431,"5min":1.6160000000000001,"15min":1.708},"last":1.232},{"interface":"front","average":{"1min":1.331,"5min":1.286,"15min":1.256},"min":{"1min":1.0920000000000001,"5min":0.497,"15min":0.26600000000000001},"max":{"1min":1.514,"5min":1.696,"15min":1.7030000000000001},"last":1.298}]},{"osd":1,"last update":"Sat Apr 3 04:03:19 2021","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3480000000000001,"15min":1.343},"min":{"1min":1.173,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":1.573,"5min":1.7430000000000001,"15min":1.8129999999999999},"last":1.335},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2929999999999999,"15min":1.288},"min":{"1min":1.1639999999999999,"5min":0.628,"15min":0.316},"max":{"1min":1.5269999999999999,"5min":1.661,"15min":1.661},"last":1.3939999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:03:19 2021","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.262,"15min":1.24},"min":{"1min":1.1000000000000001,"5min":0.63500000000000001,"15min":0.26900000000000002},"max":{"1min":1.6240000000000001,"5min":1.6499999999999999,"15min":1.736},"last":1.371},{"interface":"front","average":{"1min":1.371,"5min":1.288,"15min":1.2589999999999999},"min":{"1min":1.1599999999999999,"5min":0.56200000000000006,"15min":0.32400000000000001},"max":{"1min":1.5700000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.2689999999999999}]},{"osd":3,"last update":"Sat Apr 3 04:03:19 2021","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.2889999999999999,"15min":1.2809999999999999},"min":{"1min":1.1899999999999999,"5min":0.56599999999999995,"15min":0.29899999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.7390000000000001},"last":1.21},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2569999999999999,"15min":1.248},"min":{"1min":0.97899999999999998,"5min":0.69299999999999995,"15min":0.30199999999999999},"max":{"1min":1.6220000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.49}]},{"osd":4,"last update":"Sat Apr 3 04:03:19 2021","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.71499999999999997,"5min":0.25900000000000001,"15min":0.17299999999999999},"max":{"1min":1.1220000000000001,"5min":1.3300000000000001,"15min":1.389},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.95099999999999996,"15min":0.96699999999999997},"min":{"1min":0.79600000000000004,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0669999999999999,"5min":1.3500000000000001,"15min":1.6919999999999999},"last":0.80900000000000005}]},{"osd":6,"last update":"Sat Apr 3 04:03:19 2021","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.0349999999999999,"15min":1.0389999999999999},"min":{"1min":0.88,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.4339999999999999,"5min":1.46,"15min":1.6759999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.046,"15min":1.0580000000000001},"min":{"1min":0.89900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3169999999999999,"5min":1.4910000000000001,"15min":1.5309999999999999},"last":0.96999999999999997}]},{"osd":7,"last update":"Sat Apr 3 04:03:19 2021","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0169999999999999,"15min":1.0349999999999999},"min":{"1min":0.81999999999999995,"5min":0.251,"15min":0.251},"max":{"1min":1.2849999999999999,"5min":1.7010000000000001,"15min":1.742},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0620000000000001,"15min":1.0609999999999999},"min":{"1min":0.88400000000000001,"5min":0.22900000000000001,"15min":0.224},"max":{"1min":1.389,"5min":1.694,"15min":1.694},"last":0.88800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575512,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.64600000000000002,"5min":0.29899999999999999,"15min":0.253},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.4550000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81699999999999995,"15min":0.81200000000000006},"min":{"1min":0.72699999999999998,"5min":0.376,"15min":0.23899999999999999},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.772},"last":0.94499999999999995}]},{"osd":2,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.66500000000000004,"5min":0.254,"15min":0.22600000000000001},"max":{"1min":0.95299999999999996,"5min":1.0169999999999999,"15min":1.1140000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.32500000000000001,"15min":0.222},"max":{"1min":1.0680000000000001,"5min":1.147,"15min":1.4199999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.76500000000000001,"5min":0.34599999999999997,"15min":0.26100000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.9730000000000001},"last":1.002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.72499999999999998,"5min":0.379,"15min":0.223},"max":{"1min":1.0049999999999999,"5min":1.2729999999999999,"15min":1.4179999999999999},"last":0.83499999999999996}]},{"osd":4,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.355,"15min":1.3420000000000001},"min":{"1min":1.048,"5min":0.70499999999999996,"15min":0.42099999999999999},"max":{"1min":1.6579999999999999,"5min":1.704,"15min":1.7569999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.365,"15min":1.353},"min":{"1min":1.143,"5min":0.69399999999999995,"15min":0.41799999999999998},"max":{"1min":1.546,"5min":1.675,"15min":1.8939999999999999},"last":1.5169999999999999}]},{"osd":5,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":1.179,"5min":0.63400000000000001,"15min":0.35599999999999998},"max":{"1min":1.589,"5min":1.6990000000000001,"15min":1.8220000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.278,"15min":1.296},"min":{"1min":0.94999999999999996,"5min":0.68200000000000005,"15min":0.36799999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.744},"last":1.5629999999999999}]},{"osd":6,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.321,"15min":1.3100000000000001},"min":{"1min":1.264,"5min":0.66900000000000004,"15min":0.47599999999999998},"max":{"1min":1.613,"5min":1.6830000000000001,"15min":1.925},"last":1.603},{"interface":"front","average":{"1min":1.383,"5min":1.3460000000000001,"15min":1.319},"min":{"1min":1.038,"5min":0.63100000000000001,"15min":0.38400000000000001},"max":{"1min":1.6299999999999999,"5min":1.8260000000000001,"15min":1.8540000000000001},"last":1.3400000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:02:36 2021","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":1.0860000000000001,"5min":0.60399999999999998,"15min":0.41699999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.7929999999999999},"last":1.444},{"interface":"front","average":{"1min":1.415,"5min":1.373,"15min":1.337},"min":{"1min":1.1899999999999999,"5min":0.65400000000000003,"15min":0.45400000000000001},"max":{"1min":1.643,"5min":1.782,"15min":1.782},"last":1.47}]}]},{"osd":0,"up_from":13,"seq":55834575511,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 3 04:03:28 2021","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.24399999999999999,"5min":0.22700000000000001,"15min":0.16700000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.6639999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.79400000000000004,"15min":0.78300000000000003},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20399999999999999},"max":{"1min":0.98899999999999999,"5min":1.131,"15min":1.71},"last":1.0209999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:03:28 2021","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.188},"max":{"1min":1.137,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.78100000000000003,"15min":0.77400000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.218},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.248},"last":0.89000000000000001}]},{"osd":3,"last update":"Sat Apr 3 04:03:28 2021","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79600000000000004,"15min":0.78700000000000003},"min":{"1min":0.251,"5min":0.24399999999999999,"15min":0.17899999999999999},"max":{"1min":1.1419999999999999,"5min":1.236,"15min":1.3580000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.88500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.22},"max":{"1min":1.236,"5min":1.264,"15min":1.264},"last":1.1579999999999999}]},{"osd":4,"last update":"Sat Apr 3 04:03:28 2021","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.85899999999999999,"5min":0.497,"15min":0.26200000000000001},"max":{"1min":1.6299999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.429},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2809999999999999,"15min":1.2869999999999999},"min":{"1min":0.78300000000000003,"5min":0.45500000000000002,"15min":0.28699999999999998},"max":{"1min":1.579,"5min":1.599,"15min":1.76},"last":1.385}]},{"osd":5,"last update":"Sat Apr 3 04:03:28 2021","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2609999999999999,"15min":1.2689999999999999},"min":{"1min":0.78500000000000003,"5min":0.44,"15min":0.31900000000000001},"max":{"1min":1.488,"5min":1.583,"15min":1.839},"last":1.294},{"interface":"front","average":{"1min":1.23,"5min":1.3120000000000001,"15min":1.2849999999999999},"min":{"1min":0.73199999999999998,"5min":0.505,"15min":0.32900000000000001},"max":{"1min":1.5620000000000001,"5min":1.673,"15min":1.927},"last":1.306}]},{"osd":6,"last update":"Sat Apr 3 04:03:28 2021","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2470000000000001,"15min":1.27},"min":{"1min":0.78200000000000003,"5min":0.47499999999999998,"15min":0.27200000000000002},"max":{"1min":1.403,"5min":1.6419999999999999,"15min":1.77},"last":1.27},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.73299999999999998,"5min":0.48599999999999999,"15min":0.24299999999999999},"max":{"1min":1.581,"5min":1.663,"15min":1.7629999999999999},"last":1.4530000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:03:28 2021","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.337,"15min":1.3460000000000001},"min":{"1min":0.746,"5min":0.47799999999999998,"15min":0.254},"max":{"1min":1.5640000000000001,"5min":1.736,"15min":1.8},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.387,"15min":1.3819999999999999},"min":{"1min":0.73899999999999999,"5min":0.51300000000000001,"15min":0.29599999999999999},"max":{"1min":1.7,"5min":1.7370000000000001,"15min":1.899},"last":1.5349999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575511,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.746},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.17499999999999999},"max":{"1min":0.90800000000000003,"5min":1.1040000000000001,"15min":1.2170000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78900000000000003,"15min":0.78400000000000003},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.182},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.264},"last":0.77200000000000002}]},{"osd":1,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.27600000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.121},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.19900000000000001},"max":{"1min":1.087,"5min":1.2010000000000001,"15min":1.206},"last":0.89400000000000002}]},{"osd":3,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"min":{"1min":0.29599999999999999,"5min":0.24099999999999999,"15min":0.219},"max":{"1min":1.046,"5min":1.1339999999999999,"15min":1.2290000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.23999999999999999,"5min":0.216,"15min":0.191},"max":{"1min":0.93600000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.82799999999999996}]},{"osd":4,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3520000000000001},"min":{"1min":0.61499999999999999,"5min":0.48699999999999999,"15min":0.27000000000000002},"max":{"1min":1.54,"5min":1.79,"15min":1.79},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.355,"15min":1.3720000000000001},"min":{"1min":0.70599999999999996,"5min":0.52200000000000002,"15min":0.315},"max":{"1min":1.591,"5min":1.8520000000000001,"15min":1.857},"last":1.5529999999999999}]},{"osd":5,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.331},"min":{"1min":0.64200000000000002,"5min":0.47999999999999998,"15min":0.27400000000000002},"max":{"1min":1.6619999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.63300000000000001,"5min":0.51400000000000001,"15min":0.29299999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.7769999999999999},"last":1.117}]},{"osd":6,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3520000000000001,"15min":1.3879999999999999},"min":{"1min":0.65700000000000003,"5min":0.46700000000000003,"15min":0.32500000000000001},"max":{"1min":1.742,"5min":1.802,"15min":1.962},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3169999999999999,"15min":1.3280000000000001},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.35399999999999998},"max":{"1min":1.6299999999999999,"5min":1.744,"15min":1.9019999999999999},"last":1.623}]},{"osd":7,"last update":"Sat Apr 3 04:02:40 2021","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.369,"15min":1.377},"min":{"1min":0.58999999999999997,"5min":0.45100000000000001,"15min":0.28199999999999997},"max":{"1min":1.645,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":0.68400000000000005,"5min":0.49299999999999999,"15min":0.29799999999999999},"max":{"1min":1.5609999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.2370000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575511,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.59699999999999998,"5min":0.222,"15min":0.17399999999999999},"max":{"1min":1.016,"5min":1.089,"15min":1.3160000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84999999999999998,"15min":0.85499999999999998},"min":{"1min":0.747,"5min":0.185,"15min":0.185},"max":{"1min":1.069,"5min":1.1180000000000001,"15min":1.284},"last":0.82299999999999995}]},{"osd":1,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.19500000000000001,"15min":0.184},"max":{"1min":0.96799999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.71799999999999997,"5min":0.20599999999999999,"15min":0.193},"max":{"1min":1.1599999999999999,"5min":1.165,"15min":1.2},"last":0.77700000000000002}]},{"osd":2,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.84299999999999997,"15min":0.85899999999999999},"min":{"1min":0.69399999999999995,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.89000000000000001,"15min":0.89900000000000002},"min":{"1min":0.79900000000000004,"5min":0.215,"15min":0.20300000000000001},"max":{"1min":1.1739999999999999,"5min":1.2,"15min":1.2490000000000001},"last":0.90400000000000003}]},{"osd":4,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4119999999999999,"15min":1.4350000000000001},"min":{"1min":1.246,"5min":0.40100000000000002,"15min":0.26000000000000001},"max":{"1min":1.5920000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3280000000000001,"15min":1.3360000000000001},"min":{"1min":0.73899999999999999,"5min":0.373,"15min":0.27800000000000002},"max":{"1min":1.5349999999999999,"5min":1.577,"15min":1.6180000000000001},"last":1.367}]},{"osd":5,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2909999999999999,"15min":1.3149999999999999},"min":{"1min":0.76000000000000001,"5min":0.35099999999999998,"15min":0.24199999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.76},"last":1.395},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.93000000000000005,"5min":0.38500000000000001,"15min":0.28499999999999998},"max":{"1min":1.603,"5min":1.7330000000000001,"15min":1.7370000000000001},"last":1.331}]},{"osd":6,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.355,"15min":1.357},"min":{"1min":1.016,"5min":0.39300000000000002,"15min":0.23300000000000001},"max":{"1min":1.623,"5min":1.76,"15min":1.7729999999999999},"last":1.474},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.302,"15min":1.331},"min":{"1min":1.1200000000000001,"5min":0.34499999999999997,"15min":0.27100000000000002},"max":{"1min":1.647,"5min":1.663,"15min":1.798},"last":1.6040000000000001}]},{"osd":7,"last update":"Sat Apr 3 04:03:13 2021","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3220000000000001,"15min":1.3460000000000001},"min":{"1min":1.0780000000000001,"5min":0.376,"15min":0.314},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.724},"last":1.298},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3660000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.36699999999999999,"15min":0.30499999999999999},"max":{"1min":1.6799999999999999,"5min":1.708,"15min":1.8660000000000001},"last":1.4370000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050504,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321336,"statfs":{"total":96636764160,"available":95561048064,"internally_reserved":1073741824,"allocated":1966080,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.242,"15min":1.228},"min":{"1min":0.65100000000000002,"5min":0.36699999999999999,"15min":0.314},"max":{"1min":1.641,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2330000000000001,"15min":1.1990000000000001},"min":{"1min":0.68000000000000005,"5min":0.33000000000000002,"15min":0.309},"max":{"1min":1.623,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.238}]},{"osd":1,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.248,"15min":1.2190000000000001},"min":{"1min":0.66000000000000003,"5min":0.375,"15min":0.27400000000000002},"max":{"1min":1.554,"5min":1.772,"15min":1.772},"last":1.444},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.264,"15min":1.228},"min":{"1min":0.63400000000000001,"5min":0.36799999999999999,"15min":0.28199999999999997},"max":{"1min":1.548,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.2569999999999999}]},{"osd":2,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.26,"15min":1.2470000000000001},"min":{"1min":0.73199999999999998,"5min":0.42299999999999999,"15min":0.28799999999999998},"max":{"1min":1.5840000000000001,"5min":1.673,"15min":1.75},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2969999999999999,"15min":1.248},"min":{"1min":0.67400000000000004,"5min":0.34399999999999997,"15min":0.27900000000000003},"max":{"1min":1.7030000000000001,"5min":2.024,"15min":2.024},"last":1.4059999999999999}]},{"osd":3,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.282,"15min":1.2569999999999999},"min":{"1min":0.70999999999999996,"5min":0.379,"15min":0.29999999999999999},"max":{"1min":1.488,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.325},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3580000000000001,"15min":1.321},"min":{"1min":0.64100000000000001,"5min":0.39900000000000002,"15min":0.29799999999999999},"max":{"1min":1.593,"5min":1.99,"15min":1.99},"last":1.381}]},{"osd":5,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.91100000000000003,"15min":0.92100000000000004},"min":{"1min":0.32800000000000001,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":1.1559999999999999,"5min":1.5509999999999999,"15min":1.6379999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.90200000000000002},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.13300000000000001},"max":{"1min":1.202,"5min":1.448,"15min":1.448},"last":0.81200000000000006}]},{"osd":6,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94499999999999995,"15min":0.93400000000000005},"min":{"1min":0.33600000000000002,"5min":0.24299999999999999,"15min":0.21299999999999999},"max":{"1min":1.2549999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.995,"15min":0.98499999999999999},"min":{"1min":0.34699999999999998,"5min":0.219,"15min":0.219},"max":{"1min":1.3959999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.028}]},{"osd":7,"last update":"Sat Apr 3 04:03:01 2021","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.153},"max":{"1min":1.446,"5min":1.681,"15min":1.681},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":1.0209999999999999},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.23100000000000001},"max":{"1min":1.5149999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.94799999999999995}]}]}]}} 2021-04-03T04:03:38.753 DEBUG:teuthology.orchestra.run.gibba011:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2021-04-03T04:03:38.811 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 Processor -- start 2021-04-03T04:03:38.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- start start 2021-04-03T04:03:38.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f40500b38c0 0x7f4050025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.812 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4050080c60 0x7f40500af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- --> v1:172.21.2.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4050081090 con 0x7f4050153440 2021-04-03T04:03:38.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- --> v2:172.21.2.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4050025970 con 0x7f40500b38c0 2021-04-03T04:03:38.813 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500b3ca0 con 0x7f4050080c60 2021-04-03T04:03:38.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f40500b38c0 0x7f4050025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 --2- >> v2:172.21.2.111:3300/0 conn(0x7f40500b38c0 0x7f4050025480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59678/0 (socket says 172.21.2.111:59678) 2021-04-03T04:03:38.814 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 -- 172.21.2.111:0/2082467766 learned_addr learned my addr 172.21.2.111:0/2082467766 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:38.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 -- 172.21.2.111:0/2082467766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4050080c60 msgr2=0x7f40500af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:38.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 --2- 172.21.2.111:0/2082467766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4050080c60 0x7f40500af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 -- 172.21.2.111:0/2082467766 >> v1:172.21.2.115:6789/0 conn(0x7f4050153440 legacy=0x7f4050080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-04-03T04:03:38.815 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 -- 172.21.2.111:0/2082467766 --> v2:172.21.2.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050025b90 con 0x7f40500b38c0 2021-04-03T04:03:38.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404ffff700 1 --2- 172.21.2.111:0/2082467766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b38c0 0x7f4050025480 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f4050063a70 tx=0x7f4038006e90).ready entity=mon.0 client_cookie=526e08c6a24d7807 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:38.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404effd700 1 -- 172.21.2.111:0/2082467766 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40380098e0 con 0x7f40500b38c0 2021-04-03T04:03:38.816 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404effd700 1 -- 172.21.2.111:0/2082467766 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038009a40 con 0x7f40500b38c0 2021-04-03T04:03:38.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f404effd700 1 -- 172.21.2.111:0/2082467766 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4038009de0 con 0x7f40500b38c0 2021-04-03T04:03:38.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- 172.21.2.111:0/2082467766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b38c0 msgr2=0x7f4050025480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.817 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 --2- 172.21.2.111:0/2082467766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b38c0 0x7f4050025480 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f4050063a70 tx=0x7f4038006e90).stop 2021-04-03T04:03:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- 172.21.2.111:0/2082467766 shutdown_connections 2021-04-03T04:03:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 --2- 172.21.2.111:0/2082467766 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f4050080c60 0x7f40500af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-04-03T04:03:38.818 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 --2- 172.21.2.111:0/2082467766 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f40500b38c0 0x7f4050025480 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- 172.21.2.111:0/2082467766 >> 172.21.2.111:0/2082467766 conn(0x7f4050061610 msgr2=0x7f4050061a20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- 172.21.2.111:0/2082467766 shutdown_connections 2021-04-03T04:03:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 -- 172.21.2.111:0/2082467766 wait complete. 2021-04-03T04:03:38.819 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.804+0000 7f4054ce7700 1 Processor -- start 2021-04-03T04:03:38.820 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 -- start start 2021-04-03T04:03:38.822 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4050080c60 0x7f40500abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 --2- >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40500b38c0 0x7f405013b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 --2- >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4050153440 0x7f405013b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.823 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 -- --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050065a30 con 0x7f4050080c60 2021-04-03T04:03:38.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 -- --> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050065de0 con 0x7f4050153440 2021-04-03T04:03:38.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 -- --> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4050066080 con 0x7f40500b38c0 2021-04-03T04:03:38.824 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4050080c60 0x7f40500abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 --2- >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4050080c60 0x7f40500abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.111:3300/0 says I am v2:172.21.2.111:59686/0 (socket says 172.21.2.111:59686) 2021-04-03T04:03:38.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 -- 172.21.2.111:0/3770056000 learned_addr learned my addr 172.21.2.111:0/3770056000 (peer_addr_for_me v2:172.21.2.111:0/0) 2021-04-03T04:03:38.825 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4056161700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4050153440 0x7f405013b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404f7fe700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40500b38c0 0x7f405013b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 -- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40500b38c0 msgr2=0x7f405013b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.826 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40500b38c0 0x7f405013b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 -- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4050153440 msgr2=0x7f405013b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4050153440 0x7f405013b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.827 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 -- 172.21.2.111:0/3770056000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050066320 con 0x7f4050080c60 2021-04-03T04:03:38.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404f7fe700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40500b38c0 0x7f405013b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-04-03T04:03:38.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404ffff700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4050080c60 0x7f40500abf50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7f4050063a70 tx=0x7f4038015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:38.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4038011df0 con 0x7f4050080c60 2021-04-03T04:03:38.828 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403801e460 con 0x7f4050080c60 2021-04-03T04:03:38.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500667c0 con 0x7f4050080c60 2021-04-03T04:03:38.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403801e7c0 con 0x7f4050080c60 2021-04-03T04:03:38.829 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4050067430 con 0x7f4050080c60 2021-04-03T04:03:38.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40500664b0 con 0x7f4050080c60 2021-04-03T04:03:38.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 68316+0+0 (secure 0 0 0) 0x7f4038025030 con 0x7f4050080c60 2021-04-03T04:03:38.830 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404cff9700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4034056e90 0x7f4034059350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-04-03T04:03:38.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5682+0+0 (secure 0 0 0) 0x7f4038071290 con 0x7f4050080c60 2021-04-03T04:03:38.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404f7fe700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4034056e90 0x7f4034059350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-04-03T04:03:38.831 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.808+0000 7f404f7fe700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4034056e90 0x7f4034059350 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f4040007000 tx=0x7f4040008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-04-03T04:03:38.832 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.812+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138303 (secure 0 0 0) 0x7f4038046950 con 0x7f4050080c60 2021-04-03T04:03:38.908 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.904+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 --> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4050081090 con 0x7f4050080c60 2021-04-03T04:03:38.913 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4038046950 con 0x7f4050080c60 2021-04-03T04:03:38.914 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f404cff9700 1 -- 172.21.2.111:0/3770056000 <== mon.0 v2:172.21.2.111:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4050081090 con 0x7f4050080c60 2021-04-03T04:03:38.915 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4034056e90 msgr2=0x7f4034059350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4034056e90 0x7f4034059350 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f4040007000 tx=0x7f4040008040).stop 2021-04-03T04:03:38.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4050080c60 msgr2=0x7f40500abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-04-03T04:03:38.916 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4050080c60 0x7f40500abf50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7f4050063a70 tx=0x7f4038015000).stop 2021-04-03T04:03:38.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 shutdown_connections 2021-04-03T04:03:38.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:6800/15722,v1:172.21.2.111:6801/15722] conn(0x7f4034056e90 0x7f4034059350 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3300/0,v1:172.21.2.115:6789/0] conn(0x7f4050153440 0x7f405013b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.917 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.111:3300/0,v1:172.21.2.111:6789/0] conn(0x7f4050080c60 0x7f40500abf50 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 --2- 172.21.2.111:0/3770056000 >> [v2:172.21.2.115:3301/0,v1:172.21.2.115:6790/0] conn(0x7f40500b38c0 0x7f405013b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-04-03T04:03:38.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 >> 172.21.2.111:0/3770056000 conn(0x7f4050061610 msgr2=0x7f40500b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-04-03T04:03:38.918 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 shutdown_connections 2021-04-03T04:03:38.919 INFO:teuthology.orchestra.run.gibba011.stderr:2021-04-03T04:03:38.908+0000 7f4054ce7700 1 -- 172.21.2.111:0/3770056000 wait complete. 2021-04-03T04:03:38.922 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/ceph.py", line 1839, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-04-03T04:03:38.925 INFO:teuthology.misc:Shutting down mds daemons... 2021-04-03T04:03:38.926 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-04-03T04:03:38.927 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:38.932 INFO:tasks.ceph.mds.a:Stopped 2021-04-03T04:03:38.932 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-04-03T04:03:38.933 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:38.939 INFO:tasks.ceph.mds.b:Stopped 2021-04-03T04:03:38.939 INFO:teuthology.misc:Shutting down osd daemons... 2021-04-03T04:03:38.940 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2021-04-03T04:03:38.941 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:42.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2021-04-03T04:03:45.043 INFO:tasks.ceph.osd.0:Stopped 2021-04-03T04:03:45.044 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2021-04-03T04:03:45.045 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.123 INFO:tasks.ceph.osd.1:Stopped 2021-04-03T04:03:45.124 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2021-04-03T04:03:45.125 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.189 INFO:tasks.ceph.osd.2:Stopped 2021-04-03T04:03:45.190 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2021-04-03T04:03:45.191 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.276 INFO:tasks.ceph.osd.3:Stopped 2021-04-03T04:03:45.277 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2021-04-03T04:03:45.277 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.346 INFO:tasks.ceph.osd.4:Stopped 2021-04-03T04:03:45.347 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2021-04-03T04:03:45.348 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.443 INFO:tasks.ceph.osd.5:Stopped 2021-04-03T04:03:45.444 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2021-04-03T04:03:45.445 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.534 INFO:tasks.ceph.osd.6:Stopped 2021-04-03T04:03:45.535 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2021-04-03T04:03:45.536 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.603 INFO:tasks.ceph.osd.7:Stopped 2021-04-03T04:03:45.604 INFO:teuthology.misc:Shutting down mgr daemons... 2021-04-03T04:03:45.605 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2021-04-03T04:03:45.606 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.621 INFO:tasks.ceph.mgr.y:Stopped 2021-04-03T04:03:45.622 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2021-04-03T04:03:45.623 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.634 INFO:tasks.ceph.mgr.x:Stopped 2021-04-03T04:03:45.635 INFO:teuthology.misc:Shutting down mon daemons... 2021-04-03T04:03:45.636 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2021-04-03T04:03:45.636 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.674 INFO:tasks.ceph.mon.a:Stopped 2021-04-03T04:03:45.675 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2021-04-03T04:03:45.675 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.698 INFO:tasks.ceph.mon.b:Stopped 2021-04-03T04:03:45.699 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2021-04-03T04:03:45.699 INFO:teuthology.orchestra.run:waiting for 300 2021-04-03T04:03:45.750 INFO:tasks.ceph.mon.c:Stopped 2021-04-03T04:03:45.751 INFO:tasks.ceph:Checking cluster log for badness... 2021-04-03T04:03:45.752 DEBUG:teuthology.orchestra.run.gibba011:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2021-04-03T04:03:45.764 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T04:03:45.764 DEBUG:teuthology.orchestra.run.gibba011:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2021-04-03T04:03:46.006 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T04:03:46.007 DEBUG:teuthology.orchestra.run.gibba011:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2021-04-03T04:03:46.120 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T04:03:46.121 DEBUG:teuthology.orchestra.run.gibba011:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2021-04-03T04:03:46.230 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T04:03:46.231 DEBUG:teuthology.orchestra.run.gibba011:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2021-04-03T04:03:46.349 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T04:03:46.350 DEBUG:teuthology.orchestra.run.gibba015:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2021-04-03T04:03:46.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T04:03:46.585 DEBUG:teuthology.orchestra.run.gibba015:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2021-04-03T04:03:46.727 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T04:03:46.727 DEBUG:teuthology.orchestra.run.gibba015:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2021-04-03T04:03:46.818 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T04:03:46.819 DEBUG:teuthology.orchestra.run.gibba015:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2021-04-03T04:03:46.902 INFO:tasks.ceph:Archiving mon data... 2021-04-03T04:03:46.904 DEBUG:teuthology.misc:Transferring archived files from gibba011:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/data/mon.a.tgz 2021-04-03T04:03:46.905 DEBUG:teuthology.orchestra.run.gibba011:> mktemp 2021-04-03T04:03:46.913 INFO:teuthology.orchestra.run.gibba011.stdout:/tmp/tmp.UOL0BVwExt 2021-04-03T04:03:46.914 DEBUG:teuthology.orchestra.run.gibba011:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.UOL0BVwExt 2021-04-03T04:03:47.103 DEBUG:teuthology.orchestra.run.gibba011:> sudo chmod 0666 /tmp/tmp.UOL0BVwExt 2021-04-03T04:03:47.157 DEBUG:teuthology.orchestra.remote:gibba011:/tmp/tmp.UOL0BVwExt is 562KB 2021-04-03T04:03:47.262 DEBUG:teuthology.orchestra.run.gibba011:> rm -fr /tmp/tmp.UOL0BVwExt 2021-04-03T04:03:47.270 DEBUG:teuthology.misc:Transferring archived files from gibba015:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/data/mon.b.tgz 2021-04-03T04:03:47.271 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T04:03:47.279 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.6ih3LGxFOV 2021-04-03T04:03:47.280 DEBUG:teuthology.orchestra.run.gibba015:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6ih3LGxFOV 2021-04-03T04:03:47.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2021-04-03T04:03:47.487 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.6ih3LGxFOV 2021-04-03T04:03:47.541 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.6ih3LGxFOV is 695KB 2021-04-03T04:03:47.646 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.6ih3LGxFOV 2021-04-03T04:03:47.655 DEBUG:teuthology.misc:Transferring archived files from gibba015:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/data/mon.c.tgz 2021-04-03T04:03:47.655 DEBUG:teuthology.orchestra.run.gibba015:> mktemp 2021-04-03T04:03:47.703 INFO:teuthology.orchestra.run.gibba015.stdout:/tmp/tmp.acER81mbBF 2021-04-03T04:03:47.752 DEBUG:teuthology.orchestra.run.gibba015:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.acER81mbBF 2021-04-03T04:03:47.914 DEBUG:teuthology.orchestra.run.gibba015:> sudo chmod 0666 /tmp/tmp.acER81mbBF 2021-04-03T04:03:47.974 DEBUG:teuthology.orchestra.remote:gibba015:/tmp/tmp.acER81mbBF is 695KB 2021-04-03T04:03:48.042 DEBUG:teuthology.orchestra.run.gibba015:> rm -fr /tmp/tmp.acER81mbBF 2021-04-03T04:03:48.047 INFO:tasks.ceph:Cleaning ceph cluster... 2021-04-03T04:03:48.047 DEBUG:teuthology.orchestra.run.gibba011:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-04-03T04:03:48.051 DEBUG:teuthology.orchestra.run.gibba015:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-04-03T04:03:48.090 DEBUG:teuthology.orchestra.run.gibba026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-04-03T04:03:48.093 DEBUG:teuthology.orchestra.run.gibba028:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-04-03T04:03:48.097 DEBUG:teuthology.orchestra.run.gibba031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-04-03T04:03:48.109 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2021-04-03T04:03:48.109 DEBUG:teuthology.orchestra.run.gibba011:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-04-03T04:03:48.112 DEBUG:teuthology.orchestra.run.gibba015:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-04-03T04:03:48.120 INFO:teuthology.orchestra.run.gibba011.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-04-03T04:03:48.141 DEBUG:teuthology.orchestra.run.gibba026:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-04-03T04:03:48.146 DEBUG:teuthology.orchestra.run.gibba028:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-04-03T04:03:48.148 DEBUG:teuthology.orchestra.run.gibba031:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-04-03T04:03:48.151 INFO:teuthology.orchestra.run.gibba015.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-04-03T04:03:48.166 INFO:teuthology.orchestra.run.gibba028.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-04-03T04:03:48.168 INFO:teuthology.orchestra.run.gibba026.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-04-03T04:03:48.168 INFO:teuthology.orchestra.run.gibba031.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-04-03T04:03:48.174 INFO:tasks.ceph:Archiving crash dumps... 2021-04-03T04:03:48.176 DEBUG:teuthology.misc:Transferring archived files from gibba011:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba011/crash 2021-04-03T04:03:48.177 DEBUG:teuthology.orchestra.run.gibba011:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-04-03T04:03:48.190 DEBUG:teuthology.misc:Transferring archived files from gibba015:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba015/crash 2021-04-03T04:03:48.192 DEBUG:teuthology.orchestra.run.gibba015:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-04-03T04:03:48.224 DEBUG:teuthology.misc:Transferring archived files from gibba026:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba026/crash 2021-04-03T04:03:48.225 DEBUG:teuthology.orchestra.run.gibba026:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-04-03T04:03:48.251 DEBUG:teuthology.misc:Transferring archived files from gibba028:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba028/crash 2021-04-03T04:03:48.252 DEBUG:teuthology.orchestra.run.gibba028:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-04-03T04:03:48.265 DEBUG:teuthology.misc:Transferring archived files from gibba031:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba031/crash 2021-04-03T04:03:48.267 DEBUG:teuthology.orchestra.run.gibba031:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-04-03T04:03:48.291 INFO:tasks.ceph:Shutting down logrotate 2021-04-03T04:03:48.292 DEBUG:teuthology.orchestra.run.gibba011:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.295 DEBUG:teuthology.orchestra.run.gibba015:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.298 DEBUG:teuthology.orchestra.run.gibba026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.301 DEBUG:teuthology.orchestra.run.gibba028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.307 DEBUG:teuthology.orchestra.run.gibba031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.358 DEBUG:teuthology.orchestra.run.gibba011:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.367 DEBUG:teuthology.orchestra.run.gibba015:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.378 DEBUG:teuthology.orchestra.run.gibba026:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.394 DEBUG:teuthology.orchestra.run.gibba028:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.413 DEBUG:teuthology.orchestra.run.gibba031:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-04-03T04:03:48.423 INFO:tasks.ceph:Compressing logs... 2021-04-03T04:03:48.424 DEBUG:teuthology.orchestra.run.gibba011:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:03:48.427 DEBUG:teuthology.orchestra.run.gibba015:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:03:48.430 DEBUG:teuthology.orchestra.run.gibba026:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:03:48.438 DEBUG:teuthology.orchestra.run.gibba028:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:03:48.460 DEBUG:teuthology.orchestra.run.gibba031:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:03:59.727 INFO:tasks.ceph:Archiving logs... 2021-04-03T04:03:59.728 DEBUG:teuthology.misc:Transferring archived files from gibba011:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba011/log 2021-04-03T04:03:59.730 DEBUG:teuthology.orchestra.run.gibba011:> sudo tar cz -f - -C /var/log/ceph -- . 2021-04-03T04:04:02.449 DEBUG:teuthology.misc:Transferring archived files from gibba015:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba015/log 2021-04-03T04:04:02.450 DEBUG:teuthology.orchestra.run.gibba015:> sudo tar cz -f - -C /var/log/ceph -- . 2021-04-03T04:04:03.757 DEBUG:teuthology.misc:Transferring archived files from gibba026:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba026/log 2021-04-03T04:04:03.758 DEBUG:teuthology.orchestra.run.gibba026:> sudo tar cz -f - -C /var/log/ceph -- . 2021-04-03T04:04:04.039 DEBUG:teuthology.misc:Transferring archived files from gibba028:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba028/log 2021-04-03T04:04:04.040 DEBUG:teuthology.orchestra.run.gibba028:> sudo tar cz -f - -C /var/log/ceph -- . 2021-04-03T04:04:04.062 DEBUG:teuthology.misc:Transferring archived files from gibba031:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba031/log 2021-04-03T04:04:04.064 DEBUG:teuthology.orchestra.run.gibba031:> sudo tar cz -f - -C /var/log/ceph -- . 2021-04-03T04:04:04.077 DEBUG:teuthology.run_tasks:Unwinding manager install 2021-04-03T04:04:04.089 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_5feb67b5bda898b8c739180363606432d4f39831/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-04-03T04:04:04.092 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2021-04-03T04:04:04.093 DEBUG:teuthology.orchestra.run.gibba011:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-04-03T04:04:04.096 DEBUG:teuthology.orchestra.run.gibba015:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-04-03T04:04:04.099 DEBUG:teuthology.orchestra.run.gibba026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-04-03T04:04:04.102 DEBUG:teuthology.orchestra.run.gibba028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-04-03T04:04:04.108 DEBUG:teuthology.orchestra.run.gibba031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-04-03T04:04:04.203 DEBUG:teuthology.run_tasks:Unwinding manager clock 2021-04-03T04:04:04.213 INFO:teuthology.task.clock:Checking final clock skew... 2021-04-03T04:04:04.213 DEBUG:teuthology.orchestra.run.gibba011:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T04:04:04.233 INFO:teuthology.orchestra.run.gibba011.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T04:04:04.234 INFO:teuthology.orchestra.run.gibba011.stdout:============================================================================== 2021-04-03T04:04:04.234 INFO:teuthology.orchestra.run.gibba011.stdout:+hv01.front.sepi 216.240.36.24 3 u 110 128 377 0.156 1.319 0.200 2021-04-03T04:04:04.234 INFO:teuthology.orchestra.run.gibba011.stdout:*hv02.front.sepi 163.237.218.18 2 u 119 128 377 0.159 -1.390 0.133 2021-04-03T04:04:04.235 INFO:teuthology.orchestra.run.gibba011.stdout:-hv03.front.sepi 192.207.62.39 3 u 106 128 377 0.079 2.654 0.153 2021-04-03T04:04:04.235 INFO:teuthology.orchestra.run.gibba011.stdout:+hv04.front.sepi 96.245.170.99 3 u 50 128 377 0.133 -0.095 0.186 2021-04-03T04:04:04.235 DEBUG:teuthology.orchestra.run.gibba015:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T04:04:04.252 INFO:teuthology.orchestra.run.gibba015.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T04:04:04.252 INFO:teuthology.orchestra.run.gibba015.stdout:============================================================================== 2021-04-03T04:04:04.253 INFO:teuthology.orchestra.run.gibba015.stdout:+hv01.front.sepi 216.240.36.24 3 u 87 128 377 0.111 1.575 0.247 2021-04-03T04:04:04.253 INFO:teuthology.orchestra.run.gibba015.stdout:*hv02.front.sepi 163.237.218.18 2 u 103 128 377 0.132 -1.233 0.080 2021-04-03T04:04:04.253 INFO:teuthology.orchestra.run.gibba015.stdout:-hv03.front.sepi 192.207.62.39 3 u 111 128 377 0.142 2.937 0.171 2021-04-03T04:04:04.254 INFO:teuthology.orchestra.run.gibba015.stdout:+hv04.front.sepi 129.250.35.251 3 u 104 128 377 0.154 -0.189 0.066 2021-04-03T04:04:04.254 DEBUG:teuthology.orchestra.run.gibba026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T04:04:04.273 INFO:teuthology.orchestra.run.gibba026.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T04:04:04.273 INFO:teuthology.orchestra.run.gibba026.stdout:============================================================================== 2021-04-03T04:04:04.273 INFO:teuthology.orchestra.run.gibba026.stdout:+hv01.front.sepi 216.240.36.24 3 u 25 128 377 0.127 1.146 0.166 2021-04-03T04:04:04.274 INFO:teuthology.orchestra.run.gibba026.stdout:*hv02.front.sepi 163.237.218.18 2 u 35 128 377 0.150 -1.737 0.128 2021-04-03T04:04:04.274 INFO:teuthology.orchestra.run.gibba026.stdout:-hv03.front.sepi 192.207.62.39 3 u 21 128 377 0.160 2.511 0.139 2021-04-03T04:04:04.274 INFO:teuthology.orchestra.run.gibba026.stdout:+hv04.front.sepi 96.245.170.99 3 u 23 128 377 0.215 -0.507 0.176 2021-04-03T04:04:04.275 DEBUG:teuthology.orchestra.run.gibba028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T04:04:04.284 INFO:teuthology.orchestra.run.gibba028.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T04:04:04.285 INFO:teuthology.orchestra.run.gibba028.stdout:============================================================================== 2021-04-03T04:04:04.285 INFO:teuthology.orchestra.run.gibba028.stdout:+hv01.front.sepi 216.240.36.24 3 u 32 128 377 0.157 2.322 0.228 2021-04-03T04:04:04.286 INFO:teuthology.orchestra.run.gibba028.stdout:*hv02.front.sepi 163.237.218.18 2 u 23 128 377 0.171 -0.526 0.163 2021-04-03T04:04:04.286 INFO:teuthology.orchestra.run.gibba028.stdout:-hv03.front.sepi 192.207.62.39 3 u 21 128 377 0.083 3.500 0.287 2021-04-03T04:04:04.286 INFO:teuthology.orchestra.run.gibba028.stdout:+hv04.front.sepi 96.245.170.99 3 u 28 128 377 0.236 0.567 0.214 2021-04-03T04:04:04.287 DEBUG:teuthology.orchestra.run.gibba031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-04-03T04:04:04.306 INFO:teuthology.orchestra.run.gibba031.stdout: remote refid st t when poll reach delay offset jitter 2021-04-03T04:04:04.306 INFO:teuthology.orchestra.run.gibba031.stdout:============================================================================== 2021-04-03T04:04:04.306 INFO:teuthology.orchestra.run.gibba031.stdout:+hv01.front.sepi 216.240.36.24 3 u 10 128 377 0.158 4.397 0.557 2021-04-03T04:04:04.307 INFO:teuthology.orchestra.run.gibba031.stdout:*hv02.front.sepi 163.237.218.18 2 u 18 128 377 0.115 1.169 0.701 2021-04-03T04:04:04.307 INFO:teuthology.orchestra.run.gibba031.stdout:-hv03.front.sepi 192.207.62.39 3 u 89 128 377 0.155 5.339 0.556 2021-04-03T04:04:04.307 INFO:teuthology.orchestra.run.gibba031.stdout:+hv04.front.sepi 96.245.170.99 3 u 27 128 377 0.160 2.673 0.473 2021-04-03T04:04:04.308 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2021-04-03T04:04:04.320 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2021-04-03T04:04:04.320 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2021-04-03T04:04:04.403 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-04-03T04:04:04.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-04-03T04:04:04.596 INFO:teuthology.task.internal:Duration was 3693.893087 seconds 2021-04-03T04:04:04.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2021-04-03T04:04:04.680 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2021-04-03T04:04:04.681 DEBUG:teuthology.orchestra.run.gibba011:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-04-03T04:04:04.684 DEBUG:teuthology.orchestra.run.gibba015:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-04-03T04:04:04.688 DEBUG:teuthology.orchestra.run.gibba026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-04-03T04:04:04.691 DEBUG:teuthology.orchestra.run.gibba028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-04-03T04:04:04.694 DEBUG:teuthology.orchestra.run.gibba031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-04-03T04:04:04.720 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2021-04-03T04:04:04.721 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba011.front.sepia.ceph.com 2021-04-03T04:04:04.721 DEBUG:teuthology.orchestra.run.gibba011:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-04-03T04:04:04.756 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba015.front.sepia.ceph.com 2021-04-03T04:04:04.757 DEBUG:teuthology.orchestra.run.gibba015:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-04-03T04:04:04.767 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba026.front.sepia.ceph.com 2021-04-03T04:04:04.768 DEBUG:teuthology.orchestra.run.gibba026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-04-03T04:04:04.780 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba028.front.sepia.ceph.com 2021-04-03T04:04:04.780 DEBUG:teuthology.orchestra.run.gibba028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-04-03T04:04:04.787 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba031.front.sepia.ceph.com 2021-04-03T04:04:04.787 DEBUG:teuthology.orchestra.run.gibba031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-04-03T04:04:04.794 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2021-04-03T04:04:04.795 DEBUG:teuthology.orchestra.run.gibba011:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:04:04.801 DEBUG:teuthology.orchestra.run.gibba015:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:04:04.814 DEBUG:teuthology.orchestra.run.gibba026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:04:04.826 DEBUG:teuthology.orchestra.run.gibba028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:04:04.832 DEBUG:teuthology.orchestra.run.gibba031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-04-03T04:04:04.853 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2021-04-03T04:04:04.863 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2021-04-03T04:04:04.864 DEBUG:teuthology.orchestra.run.gibba011:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-04-03T04:04:04.872 DEBUG:teuthology.orchestra.run.gibba015:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-04-03T04:04:04.881 DEBUG:teuthology.orchestra.run.gibba026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-04-03T04:04:04.891 DEBUG:teuthology.orchestra.run.gibba028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-04-03T04:04:04.902 DEBUG:teuthology.orchestra.run.gibba031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-04-03T04:04:04.913 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2021-04-03T04:04:04.925 DEBUG:teuthology.orchestra.run.gibba011:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:04.928 DEBUG:teuthology.orchestra.run.gibba015:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:04.931 DEBUG:teuthology.orchestra.run.gibba026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:04.934 DEBUG:teuthology.orchestra.run.gibba028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:04.936 INFO:teuthology.orchestra.run.gibba015.stdout:kernel.core_pattern = core 2021-04-03T04:04:04.939 INFO:teuthology.orchestra.run.gibba011.stdout:kernel.core_pattern = core 2021-04-03T04:04:04.944 INFO:teuthology.orchestra.run.gibba026.stdout:kernel.core_pattern = core 2021-04-03T04:04:04.948 DEBUG:teuthology.orchestra.run.gibba031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:04.958 INFO:teuthology.orchestra.run.gibba028.stdout:kernel.core_pattern = core 2021-04-03T04:04:04.967 INFO:teuthology.orchestra.run.gibba031.stdout:kernel.core_pattern = core 2021-04-03T04:04:04.972 DEBUG:teuthology.orchestra.run.gibba011:> test -e /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:04.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T04:04:04.996 DEBUG:teuthology.orchestra.run.gibba015:> test -e /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:05.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T04:04:05.011 DEBUG:teuthology.orchestra.run.gibba026:> test -e /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:05.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T04:04:05.017 DEBUG:teuthology.orchestra.run.gibba028:> test -e /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:05.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T04:04:05.023 DEBUG:teuthology.orchestra.run.gibba031:> test -e /home/ubuntu/cephtest/archive/coredump 2021-04-03T04:04:05.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-04-03T04:04:05.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2021-04-03T04:04:05.036 INFO:teuthology.task.internal:Transferring archived files... 2021-04-03T04:04:05.038 DEBUG:teuthology.misc:Transferring archived files from gibba011:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba011 2021-04-03T04:04:05.038 DEBUG:teuthology.orchestra.run.gibba011:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-04-03T04:04:05.055 DEBUG:teuthology.misc:Transferring archived files from gibba015:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba015 2021-04-03T04:04:05.055 DEBUG:teuthology.orchestra.run.gibba015:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-04-03T04:04:05.071 DEBUG:teuthology.misc:Transferring archived files from gibba026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba026 2021-04-03T04:04:05.072 DEBUG:teuthology.orchestra.run.gibba026:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-04-03T04:04:05.102 DEBUG:teuthology.misc:Transferring archived files from gibba028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba028 2021-04-03T04:04:05.103 DEBUG:teuthology.orchestra.run.gibba028:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-04-03T04:04:05.118 DEBUG:teuthology.misc:Transferring archived files from gibba031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-04-02_04:10:03-fs-octopus-distro-basic-gibba/6016396/remote/gibba031 2021-04-03T04:04:05.119 DEBUG:teuthology.orchestra.run.gibba031:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-04-03T04:04:05.129 INFO:teuthology.task.internal:Removing archive directory... 2021-04-03T04:04:05.129 DEBUG:teuthology.orchestra.run.gibba011:> rm -rf -- /home/ubuntu/cephtest/archive 2021-04-03T04:04:05.132 DEBUG:teuthology.orchestra.run.gibba015:> rm -rf -- /home/ubuntu/cephtest/archive 2021-04-03T04:04:05.135 DEBUG:teuthology.orchestra.run.gibba026:> rm -rf -- /home/ubuntu/cephtest/archive 2021-04-03T04:04:05.145 DEBUG:teuthology.orchestra.run.gibba028:> rm -rf -- /home/ubuntu/cephtest/archive 2021-04-03T04:04:05.159 DEBUG:teuthology.orchestra.run.gibba031:> rm -rf -- /home/ubuntu/cephtest/archive 2021-04-03T04:04:05.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2021-04-03T04:04:05.183 INFO:teuthology.task.internal:Not uploading archives. 2021-04-03T04:04:05.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2021-04-03T04:04:05.231 INFO:teuthology.task.internal:Tidying up after the test... 2021-04-03T04:04:05.232 DEBUG:teuthology.orchestra.run.gibba011:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-04-03T04:04:05.235 DEBUG:teuthology.orchestra.run.gibba015:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-04-03T04:04:05.238 DEBUG:teuthology.orchestra.run.gibba026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-04-03T04:04:05.242 DEBUG:teuthology.orchestra.run.gibba028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-04-03T04:04:05.243 INFO:teuthology.orchestra.run.gibba011.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 3 04:04 /home/ubuntu/cephtest 2021-04-03T04:04:05.245 INFO:teuthology.orchestra.run.gibba015.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 3 04:04 /home/ubuntu/cephtest 2021-04-03T04:04:05.246 DEBUG:teuthology.orchestra.run.gibba031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-04-03T04:04:05.251 INFO:teuthology.orchestra.run.gibba031.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 3 04:04 /home/ubuntu/cephtest 2021-04-03T04:04:05.252 INFO:teuthology.orchestra.run.gibba028.stdout: 524339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 3 04:04 /home/ubuntu/cephtest 2021-04-03T14:57:24.666 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-04-03T14:57:24.707 DEBUG:teuthology.task.console_log:Killing console logger for gibba011 2021-04-03T14:57:24.708 DEBUG:teuthology.task.console_log:Killing console logger for gibba015 2021-04-03T14:57:24.708 DEBUG:teuthology.task.console_log:Killing console logger for gibba026 2021-04-03T14:57:24.709 DEBUG:teuthology.task.console_log:Killing console logger for gibba028 2021-04-03T14:57:24.710 DEBUG:teuthology.task.console_log:Killing console logger for gibba031 2021-04-03T14:57:24.711 DEBUG:teuthology.exit:Finished running handlers